Decision Episode
A decision episode is the atomic record of a single decision made for a client: what triggered it, the options considered and which were permitted, the recommendation, whether it was followed or overridden and why, the outcome, and the specific rule that governed the choice.
A decision episode is structurally different from a chat turn or a document chunk — the two units most memory systems are built to retrieve. Where a chat turn captures what someone said, a decision episode captures what was decided: a trigger (a life event whose numbers move the household's financial trajectory), the options a firm would consider with a permitted flag on each, a policy-driven recommendation, a seeded resolution (followed or overridden, with a recorded reason if overridden), an outcome with its status, and the specific cited rule keys that governed which options were even on the table.
In DecisionSynth Bench, every field in an episode derives from the household's data, its financial trajectory, or a cited fact key — no field is written by an LLM, and a field that can't be traced to one of those sources is treated as a generator bug. A representative episode: trigger `promotion_or_raise`, `followed_policy: false`, `override_reason: "bracket_timing_judgment"`, outcome `"taxable_invest · approved"`, citing the rule key `retirement.401k.elective_deferral.under_50`. Every one of those fields is queryable, and every one is a candidate for a memory system to lose.
The distinction matters for evaluation because a memory system can excel at retrieving conversational content and still have no representation for 'why was this overridden' or 'which rule applied' — those questions don't have an analog in a chat-turn-shaped memory. Testing decision recall requires a corpus built around decision episodes specifically, which is why DecisionSynth Bench's schema, not a transcript format, is the unit its QA tasks are written against.
For teams evaluating or seeding agent memory, the decision episode is the right unit to test against if the agent's job involves recommending, executing, or explaining decisions — not just answering questions about a conversation. A memory system's ingestion pipeline needs to handle structured fields (trigger, options, resolution, cited rules), not just free text, to preserve what a decision episode actually contains.
Common pitfalls
- Treating a decision episode as just a chat log with extra metadata — the fields that matter most for evaluation (override reason, cited rule) often aren't naturally present in a transcript at all.
- Assuming the trigger is arbitrary narrative color — in a deterministically generated episode, the trigger is tied to a life event that visibly moves the household's numeric trajectory; an episode citing an event the trajectory doesn't reflect would be a generator bug, not a stylistic choice.
Examples
episode_id D3-A-01-seed-103-m63: trigger promotion_or_raise; followed_policy false; override_reason bracket_timing_judgment; outcome "taxable_invest · approved"; semantic_refs ["retirement.401k.elective_deferral.under_50"]. The matching rationale_lookup QA task asks why policy was overridden, with answer_key bracket_timing_judgment.