Agent Memory
Agent memory is the umbrella term for the systems that let an AI agent retain and retrieve information across sessions, beyond what fits in a single context window — implemented through several distinct architectures that preserve different structure and discard the rest.
"Agent memory" names a category, not a single design, and the differences between implementations are large enough that a claim like "the agent has memory" says little on its own. Three architecture families account for most production systems as of 2026.
Fact-extraction systems (Mem0 is the reference example) run a two-phase pipeline: an extraction phase where an LLM pulls salient facts or entities out of a conversation window, and an update phase that retrieves similar existing memories by embedding similarity and applies an LLM-decided add, update, delete, or no-op operation (Chhikara et al., 2025). What gets stored is a compressed, discrete set of facts — not the raw conversation.
Temporal knowledge graph systems (Zep, built on the open-source Graphiti engine) represent memory as entities (nodes) and time-stamped relationship edges, with each fact carrying an explicit validity window for when it became true and when it was superseded — old facts are invalidated, not deleted, so the graph can answer both "what's true now" and "what was true then" (Rasmussen et al., 2025). Retrieval combines semantic embeddings, keyword search, and graph traversal.
Tiered archival systems (Letta, built on the design introduced as MemGPT) organize memory into layers modeled loosely on operating-system memory management: a small core memory that stays in the active context, a searchable recall memory of conversation history, and a vector-indexed archival memory for long-term storage — with the agent itself deciding what to promote or retrieve via explicit tool calls (Packer et al., 2023).
Each architecture preserves a different kind of structure and loses a different kind under compression, which is precisely why a memory benchmark needs multiple task types (not one aggregate score) to tell them apart.
Choosing an agent-memory architecture is a decision about which structure you're willing to lose, not just a performance tradeoff. Extraction-based systems tend to preserve individual facts well but can drop the connective reasoning between them; graph-based systems preserve relationships and temporal ordering but retrieve at a finer granularity that a fixed budget can crowd out; tiered systems put the retrieval decision in the agent's own hands, which shifts some of the risk to how well the agent reasons about what to look up.
Common pitfalls
- Treating "agent memory" as one thing when comparing vendors — a fact-extraction system and a temporal-graph system fail in different, specific ways, and a single benchmark score averaged across task types can mask which failure mode applies to your use case.
- Assuming a larger context window is a substitute for a memory architecture — BEAM (Tavakoli et al., ICLR 2026) tests this directly at up to 10M tokens and finds that even large-context models degrade as dialogues lengthen; scale doesn't replace a retrieval strategy.