Superseded Figure
A superseded-figure error occurs when an AI system asserts a value from law that has since been repealed or replaced — most visibly, a scheduled statutory change ('sunset') that later legislation cancelled. Unlike a prior-year value error, the figure may never have been in effect at all; it is what the law used to say would happen.
Supersession errors are the highest-stakes stale-figure mode because they tend to be categorical. In the pre-registered pilot behind the Planning Benchmark, two of four systems asserted that the estate-tax basic exclusion reverts to roughly $5–7 million for 2026 under a scheduled sunset — legislation had repealed that sunset and set the exclusion at $15 million. On the task asked, that is not a figure slightly off; it is the difference between 'no federal estate tax is owed' and a multi-million-dollar phantom liability, delivered with statutory reasoning attached.
The mode arises when a model's training data is saturated with discussion of a scheduled change — years of planning commentary about a sunset — and comparatively light on the (newer) legislation that cancelled it. The model reproduces the majority narrative of its corpus, which is exactly the wrong prior for law that changed late.
Evaluation keys handle supersession with explicit chains: the superseded amounts are enumerated as forbidden figures with reason superseded and a note recording what replaced them, sourced from verified tables that track the supersession (published value → amending law → current value). Tasks are then placed where the superseded figure flips the categorical answer, because that is where the error costs the most.
For evaluation design, supersession chains must be recorded in the figure tables themselves — a forbidden value with reason superseded and its replacing authority — because unlike prior-year values they cannot be generated by date arithmetic. They are also the strongest adversarial material: placing a scenario near the superseded boundary converts one stale table entry into a flipped categorical answer.
Common pitfalls
- Treating supersession as a variant of prior-year staleness — the superseded value may never have been in effect, and detecting it requires a recorded chain, not date math.
- Underestimating blast radius: superseded figures tend to flip categorical outcomes (taxable vs. not), not shift a number by an indexing increment.
- Assuming recency of the model fixes it — the failure reflects corpus balance about the change, not just the cutoff date.