Decision Memory
Decision memory is episodic memory of the decisions an agent or a firm has made for a client — what was chosen, why, and under which rule — as distinct from conversational memory, which retains what was said during a chat.
Most agent-memory benchmarks test conversational memory: does the system correctly recall a fact a user stated several sessions ago. That's a real and useful capability, and it's the one benchmarks like LoCoMo test directly. Decision memory is a narrower, structurally different capability — retrieving not what was said, but what was decided: which option a policy recommended, whether the client followed or overrode it and why, what the outcome was, and which rule permitted or blocked each option on the table.
The two capabilities don't imply each other. A system can retrieve a stated fact perfectly and still have no way to answer 'why did we override policy in March' if its extraction pipeline never captured the override reason as a distinct, retrievable field — decision memory failures are often invisible to a benchmark built around conversational recall, because conversational-recall benchmarks don't contain the kind of question that would expose them.
DecisionSynth Bench operationalizes decision memory as five task types — direct recall (what was decided), rationale lookup (why), precedent search (which similar episodes decided the same way), temporal ordering (what preceded a decision), and rule attribution (which cited figure governed it) — each scored against a typed, known-by-construction answer key rather than a free-text judgment call.
| Conversational memory | Decision memory | |
|---|---|---|
| Retrieval unit | A chat turn or passage | A decision episode |
| Typical question | What did the client say about X? | What did we decide about X, and why? |
| Tested by | LoCoMo, LongMemEval, BEAM | DecisionSynth Bench |
Any agent that recommends, executes, or explains a decision on a client's behalf — an advisor assistant, a compliance-adjacent tool — needs its memory evaluated on decision-memory tasks specifically, not just conversational recall. A system with strong conversational memory can still fail an audit-trail reconstruction request if its pipeline never preserved the rationale or the governing rule as retrievable fields.
Common pitfalls
- Assuming a system that scores well on a conversational-recall benchmark will also handle decision-memory questions — the two test different structure, and a pipeline tuned for one can silently drop what the other needs.
- Conflating 'the agent remembers the client mentioned an inheritance' (a conversational fact) with 'the agent remembers what the firm recommended doing with it and why' (a decision) — these are different retrieval targets even when they arise from the same conversation.