hmem: benchmarking the memory providers fighting for your agent's long-term recall
Every agent framework eventually hits the same wall: the context window ends and the conversation forgets. The memory-provider ecosystem that sprang up to fix this is large, fast-moving, and mostly unmeasured — a lot of READMEs, few numbers. hmem is my attempt to bring order: a living comparison of fifteen providers compatible with Hermes Agent, from 62K-star mem0 down to single-file SQLite curiosities.
The part that separates it from a link list is the benchmark discipline. Providers only get scores when a real measured adapter exists — Hindsight and Mnemosyne are listed as “not measured” precisely because their simulated stubs were excluded. The current controlled run compares BM25 lexical baseline (0.367 correctness) against Hermes built-in memory (0.333) over 30 reviewed scenarios in 10 categories across three isolated runs: 180 result documents, zero failures, machine-readable JSON published alongside the writeup.
A nightly cron scans GitHub topics for new entrants; the scan log is public, including the nights it found nothing — negative results recorded honestly.
Features:
- Quick-reference table of 15 providers: storage model, security posture, cost, supported agents, key differentiator
- Per-provider spec pages with installation and configuration detail
- Measured-only scoring policy — no simulated adapters polluting the rankings
- Controlled benchmark harness: 3 runs × 30 scenarios × 10 categories, all raw documents retained
- Nightly automated scan for new providers with a dated public audit trail
- Practical guidance: only one external provider can be active at a time in Hermes, built-in memory always stays on
Source: github.com/rahlquist/hmem