Retrieval Granularity: Why Passages, Facts, and Graph Edges Answer Different Questions

Three retrieval units, three different tradeoffs

Passage-level retrieval returns a chunk of original content close to how it was stored. It's coarse but high-fidelity per item — a single passage can carry more context than a question strictly needs, but it rarely omits something relevant that was in the same chunk. Fact-level retrieval, the unit a fact-extraction pipeline produces, returns discrete statements pulled out of the source — precise when a question needs exactly one fact, but only as complete as what the extraction step chose to keep. Edge-level retrieval, the unit a temporal knowledge graph produces, returns individual (entity, relationship, entity) triples — the finest-grained of the three, and the only one with an explicit notion of time built into the unit itself, since each edge can carry its own validity window for when it became true and when it was superseded.

None of the three is a strictly better default. Each is a different bet about what a typical question will need, and a system's actual performance on any specific question type follows directly from whether that bet paid off.

Why the same source record produces a different retrieval surface per unit

Granularity isn't just about how content looks once retrieved — it changes how much of a source record a fixed budget can actually carry. A single decision record with a trigger, several considered options, an outcome, and a handful of cited rules can shred into roughly two dozen individual edge facts once a graph-building pipeline extracts every relationship inside it. At a fixed retrieval budget, a system working in edge-level units is spending that budget across fragments of what might be one or two source records, where a passage-level system spends the same budget across two or three complete records. Neither is wrong, but they're answering 'what did you retrieve' completely differently for the identical underlying content.

The counter-example: where finer granularity wins decisively

Temporal ordering — reconstructing which of several decisions happened first, second, third — is the task type where edge-level granularity's cost turns into a clear win. On DecisionSynth Bench's held-out scoreboard, a temporal-knowledge-graph system posts the best temporal-ordering exact-match of any LLM-backed architecture (0.886), ahead of an archival verbatim-storage system's 0.760, even though the same graph-based system trails badly on the benchmark's overall score. The reason is specific to the retrieval unit: an edge carries its own validity window as a first-class property, so answering 'what came before what' is close to reading the structure directly, rather than reconstructing order indirectly from timestamps embedded somewhere in a passage of text.

The same fine granularity that wins temporal ordering costs the identical system on tasks needing a complete record. Exact-identifier recall — retrieving the specific outcome of one decision — scores far lower for the same graph-based system on the held-out set, because the fixed retrieval budget spent across many small edge fragments of a decision often can't reassemble the complete record a direct-recall question needs. One retrieval unit, one architecture, two opposite outcomes depending entirely on what the question is actually asking for.

Designing for your own dominant question shape

The practical design question isn't 'which granularity is best' in the abstract — it's which granularity matches the question shape that will actually dominate your system's real traffic.

Testing granularity choices empirically, at a matched budget

Granularity decisions should be verified against real task performance, not settled by architectural preference alone — and the comparison is only fair when every system under test runs at the same retrieval budget. Comparing 5 whole passages against 5 individual graph edges isn't an apples-to-apples read of how much source material each system actually surfaced; a raw k of 5 represents a very different amount of underlying content depending on the native unit. Reporting per-task-type scores rather than a single blended number is what makes a granularity mismatch visible in the results, instead of averaging it away into a score that looks mediocre everywhere without explaining why.

Key takeaways

• Retrieval granularity — passage, fact, or edge — is a first-order design choice, not an implementation detail, because it determines what a fixed retrieval budget can actually carry from any one source record.
• A single source record can shred into dozens of individual facts under fine-grained extraction, meaning the same content produces a very different retrieval surface depending on the unit chosen.
• Temporal ordering is the clean counter-example where finer, edge-level granularity wins decisively — because an edge's validity window makes 'when' a first-class retrievable property instead of something reconstructed indirectly.
• The same fine granularity that wins ordering questions can cost a system on questions needing a complete record, since a fixed budget spent across fragments can crowd out the full picture.
• Granularity choices should be tested against your system's actual dominant question shape, at a matched retrieval budget across any systems being compared — not settled by architectural preference alone.

FAQ