Temporal Ordering Task
A temporal ordering task is a memory-evaluation question that asks a system to reconstruct the sequence of multiple related decisions — which happened first, second, third — a capability that requires aggregating structure across several stored records rather than answering accurately from any single one.
DecisionSynth Bench's temporal_ordering task type asks, in effect, what decisions preceded a given household's decision at month t, within a 24-month window, with the answer key an ordered list of episode ids. That's structurally different from a direct-recall question about one episode: getting it right requires the system to hold multiple records at once and know their relative order, not just retrieve the single closest match to the question.
Retrieval architecture matters unusually visibly on this task type. On the held-out scoreboard, a temporal-knowledge-graph system posted the best temporal-ordering exact-match of any LLM-backed system — 0.886, ahead of an archival verbatim-storage system's 0.760 — because its native retrieval unit is an individual edge fact carrying its own explicit validity window, giving the system a first-class notion of 'when' that a passage- or fact-level retrieval unit has to reconstruct indirectly. The same system's fine-grained retrieval unit works against it elsewhere: its exact-identifier recall on the held-out set was far lower, because a single decision record shreds into many individual edges, and a fixed retrieval budget spent across them can crowd out a complete picture of any one record.
Temporal ordering is well-represented in DecisionSynth Bench's task mix — 182 held-out tasks, the second-largest column after direct recall and rule attribution — so unlike some rarer task types, its scores support confident comparison, not just a directional read.
Evaluating temporal reasoning specifically requires multi-episode test data with a real, checkable sequence — a corpus of independent single-decision records has no ground truth for 'what came first' at all, because the question doesn't exist without related records to order.
Common pitfalls
- Assuming a system that excels at single-record recall will also handle ordering well — the two draw on different retrieval-unit properties.
- Assuming finer retrieval granularity always helps temporal ordering — it helps specifically because edge-level facts carry their own validity window, not simply because the unit is smaller.