wealthschemaresourcesarticlesWhat a 0.98 baseline means: reading a benchmark where grep nearly wins
Article

What a 0.98 baseline means: reading a benchmark where grep nearly wins

A benchmark whose simplest possible strategy scores 0.98 sounds broken. It's actually the design decision that makes the other five scoreboard rows mean something.

WealthSchema StaffDecisionSynth BenchJul 18, 20265 min read

The first row of DecisionSynth Bench's scoreboard is a system with no machine learning in it at all: store every episode as text, retrieve the ones that share the most keywords with the question, pull the answer field directly out of whatever it finds. On the 543-episode held-out set, that system scores 0.980 overall exact-match. Every vendor system tested against the same set — Letta, Mem0, Zep/Graphiti — scores lower, several of them far lower. A reader's first reaction is reasonably "if grep nearly wins, what is this benchmark actually testing?" The answer is specific enough to state directly: it's testing how much decision-relevant structure survives your system's compression, and the near-ceiling baseline is the number the rest of the scoreboard is measured against, not a symptom of the benchmark being too easy.

Why the baseline is supposed to score this high

Production memory systems don't store everything verbatim, because verbatim storage doesn't scale — the whole reason to extract facts, summarize, or build a graph is to avoid keeping every raw record forever in a form expensive to search. That means every production system is making a bet: that whatever it keeps after compression is enough to answer the questions that matter. A benchmark can only tell you whether that bet paid off if it has a reference point for what "everything" would have scored — which is exactly what a system that keeps everything and does no compression provides.

At a corpus this size — 543 held-out episodes, 1,367 tasks — verbatim storage and keyword retrieval is a genuinely near-ceiling strategy: there's little the deterministic baseline could plausibly miss, since it never throws anything away. That's the point, not an oversight. The gap between any vendor's row and this one is a direct, uncontaminated measurement of what that vendor's compression pipeline cost, because the only variable that changed between the two rows is how much structure survived the write step.

 SystemWhat it discardsOverall EM (held-out)
Naive-lexical baselineNothing — stores every episode verbatim0.980
Letta (archival)Almost nothing — near-verbatim passages0.917–0.922
Mem0 (extraction)Most field detail — keeps discrete facts only0.136–0.465
Zep/Graphiti (graph)Record cohesion — keeps fine-grained edges0.091–0.215

The number that isn't near-ceiling, and why

The baseline's precision@5 (P@5) is 0.302 — nowhere close to 1.0, and a reader tallying scores might read that as the baseline failing somewhere. It isn't. P@5 for any task with exactly one relevant piece of evidence is mathematically bounded at 0.2 by construction: one relevant item out of five returned, regardless of how well the system retrieves. Most of DecisionSynth Bench's task types (direct recall, rationale lookup, rule attribution) have exactly one relevant episode as evidence, so a P@5 near 0.2 on those tasks isn't a retrieval failure — it's the ceiling the task shape imposes on any system, including a perfect one. The blended 0.302 sits close to that structural floor precisely because the baseline is retrieving well; a lower P@5 blended average would actually indicate a problem, and a much higher one would be a sign the task mix skewed toward multi-evidence questions, not that the system improved.

What "the benchmark measures compression loss, not retrieval IQ" actually means

A benchmark where the simplest strategy nearly maxes out the score is not measuring which system is cleverest at reasoning over the corpus — a corpus this size doesn't require cleverness to search exhaustively. It's measuring something narrower and, for this specific evaluation question, more useful: given that production systems have to compress, how much of what matters survives the compression each one chose. That framing explains why the scoreboard's other rows spread out so widely below the baseline (0.980 down to 0.091) even though none of the underlying task difficulty changed between rows — what changed is how much structure each system's pipeline kept.

The naive-lexical baseline stores every episode verbatim and greps it — a near-ceiling strategy at this corpus size, and deliberately so. Production agent-memory systems do not store verbatim; they extract, summarize, and graph. The measurement is how much decision-relevant structure survives your system's compression.

bench/SCOREBOARD.md

Reading any scoreboard row against the baseline, in practice

The baseline's job is to be the first number you look at, not the last. Concretely, that means checking three things before drawing a conclusion from any vendor row on this scoreboard:

  • The overall gap. Letta's held-out overall EM (0.917–0.922) sits 0.06–0.06 below the baseline's 0.980 — a small gap, meaning its archival, near-verbatim design lost comparatively little. Mem0's best row (0.465) and Graphiti's best row (0.215) sit 0.5+ and 0.7+ below it respectively — large gaps, meaning their compression steps discarded much more.
  • Which specific columns carry the gap. A vendor's overall gap to baseline is rarely uniform across task types — Graphiti's gap is narrow on temporal ordering and enormous on rule attribution, for instance. The aggregate number alone doesn't show you where the loss concentrated.
  • Whether the metric itself is structurally bounded. Precision@5's 0.2 ceiling on single-evidence tasks is one example; any benchmark can have others. A metric that looks "bad" relative to a naive expectation of 1.0 may simply be behaving exactly as the task design predicts.

Skipping any of these three and reading only the headline overall-EM column is how a benchmark table gets misread — either by treating the near-ceiling baseline as an indictment of the benchmark, or by treating a vendor's low overall score as a uniform failure rather than a specific, diagnosable one.

Two numbers that corroborate this isn't a fluke of one run

The same near-ceiling pattern shows up on both of DecisionSynth Bench's corpora, not just the held-out set: the public dev set's baseline scores 0.950 overall EM (P@5 0.292, R@5 0.972) — close to, though not identical to, the held-out figures (0.980, 0.302, 0.994). The two runs use different episodes over disjoint households, so exact agreement isn't expected, but the consistency of the pattern — a verbatim strategy landing near-ceiling on both independently generated corpora — is itself evidence the design does what it's meant to do, rather than being an artifact of one particular sample.

Key takeaways

  • The deterministic verbatim baseline scores 0.980 overall EM on the held-out set by design — it's the ceiling reference every other score is measured against, not evidence the benchmark is too easy.
  • The gap between a vendor's score and the baseline's score is a direct measurement of what that vendor's compression pipeline cost, because verbatim storage is the one strategy that discards nothing.
  • The baseline's precision@5 (0.302) looks low only until you account for the structural cap of 0.2 on single-evidence tasks — recall@5 (0.994) is the number that actually reflects its retrieval quality.
  • The same near-ceiling pattern holds on both the public dev set (0.950 EM) and the private held-out set (0.980 EM), over disjoint households — evidence the pattern is a property of the design, not one sample.

Frequently asked questions

Does a near-ceiling baseline mean the benchmark is too easy for real memory systems?+
No — the baseline isn't a memory system in the production sense at all; it's a reference point with no compression step, included specifically so every other row's score is interpretable as a compression-loss measurement rather than a number with no context.
Why not exclude the baseline and just compare vendor systems to each other?+
Comparing vendors only to each other tells you their relative order but not how much any of them actually lost — two vendors could both be losing enormous amounts of structure and still look meaningfully different from each other while both being far from what's achievable. The baseline anchors the scale.
Is precision@5 a meaningless metric on this benchmark, then?+
Not meaningless — informative once read correctly. A system scoring meaningfully below the 0.2 structural cap on single-evidence tasks is retrieving badly; a system at or near 0.2 on those same tasks is retrieving about as well as the task shape allows. The number needs the denominator context to interpret, which is true of precision metrics generally.
Would a bigger corpus change whether verbatim storage stays near-ceiling?+
Plausibly, at some scale verbatim storage becomes impractical to search exhaustively even with simple keyword matching, and its score would likely degrade — DecisionSynth Bench's corpus sizes (591 dev / 543 held-out episodes) are within the range where that hasn't happened yet, which is part of why the baseline functions cleanly as a reference at this scale.