The schema is not the cost
If you run an LLM behind a JSON schema, you have probably worried about what the grammar is doing to the model. I went looking for that cost in a specific place and found the opposite.
0.5 items emitted · precision 0.917
0.8 items emitted · precision 0.882
At k = 1 free generation is ahead by 20.0 points. The schema is costing recall here, which is what everyone expects it to do.
Constrained decoding masks tokens the model wants, and the reported costs are real: several percentage points of accuracy across open-weight models, and at least one function-calling result where unconstrained generation with post-hoc parsing beat constrained decoding outright.
The question
Ask a model to pull every named counterparty out of a 10-K. Does it get worse at that as the number of counterparties grows, holding the document length and the schema fixed? Call that number k.
It is not a new question in spirit. Multi-needle NIAH already reports that retrieval falls as needle count rises. What nobody had measured was the same axis for structured extraction under a grammar, with input length actually held constant, reported as a slope rather than as “it declines.”
And there was a specific reason to care. In my own extraction work I had written down constrained decoding as the strongest competing explanation for any cardinality effect I might find. I recorded it as unfalsified and moved on, which is a comfortable way to stay wrong.
Building a ruler that works
You cannot answer this by annotating documents, because k is the treatment. If it comes from whichever filings you happened to sample and a human has to recover it, it is measured with error that correlates with the outcome, because an annotator's attention flags on exactly the long enumerations where the effect should live.
So I built the documents instead. Take verbatim-validated sentences from real filings, each carrying exactly one named company. Inject k of them at random positions into filler drawn from the same corpus with every capitalised name stripped out. Pad to a constant length. Gold is the set you injected, so k is exact and free.
That is the design. Getting it to work took five attempts, and the four failures are more instructive than the design is. I set a sanity gate first: at k=1, recall must clear 0.80. One needle, from a sentence the extractor itself produced in the wild.
It came back 0.333. Then, after a fix, 0.400. Then 0.400 again.
The first cause was that my injected sentences named their own original filer in the third person, while the document was attributed to a placeholder company. The model was being asked who ACME deals with and correctly declining to say Lockheed. The second round found four more defects at once: an all-caps ticker the name-detector missed, paragraph chunking that sliced sentences in half at fixed offsets, gold items that were not company names, and a matching rule strict enough to score Komatsu against Komatsu Cummins Chile, Ltda. as a miss.
Fixed all four. Recall: 0.400. At that point I stopped patching things I could see and ran a controlled comparison instead.
Dilution alone was worth 0.25 recall. I had built the filler by selecting prose for containing no company names, which made it far more inert than any real prompt. The gate was never failing because of the bank or the chunking. It was failing because I had buried one sentence in a haystack far emptier than anything the model sees in practice.
It also exposed something structural. Count, density and length are mechanically linked: density = (count × span length) / total length. Fix any two and the third follows, so every study of item count picks two and inherits a confound in the third. Fix length and let density rise with k, as I did, and high-k documents get a density advantage that partially cancels the effect, so your measurement is a lower bound. Fix density and let length grow, as multi-needle NIAH does, and length suppresses high k, giving an upper bound. There is no third option.
The result
qwen3:14b, 32k context, 75 matched documents per arm, zero truncations.
Paired on the same documents, unconstrained leads by 0.200 at k=1 and trails by 0.128 at k=16 (p = 0.005). The crossover is the effect.
So the cardinality effect is real, and the grammar is what prevents it.
The emission counts give it away. At k=16 the constrained arm emits 9.4 items of the 16 available; the unconstrained arm emits 7.2. Precision is identical at 0.946 and 0.948, so this is not a quality tradeoff. Left to decide when to stop, the model stops early. A schema takes that decision away, because a half-filled array is not a valid parse.
That also kills a number I believed earlier in the week. On a 30B mixture-of-experts model, unconstrained generation cost 18× the tokens. I reported that as a fact about removing the grammar. It was not. That model was writing 7,000 to 12,000 tokens against an 8,192-token context window, so the arm was measuring a wall. On a model whose generation fits, unconstrained costs 1.1× at k=1 and 0.7× at k=16. It is cheaper and worse.
What I would take from this
For anyone shipping structured extraction: the schema is not the thing costing you recall at high item counts. It is the thing holding recall up. If you have been considering free-form generation with post-hoc parsing to escape grammar masking, measure the item-count axis before you switch.
For anyone evaluating extraction: a single F1 over a benchmark whose k distribution is unstated is not comparable to another benchmark's F1. And if you measure the axis, say which two of count, density and length you fixed, because that choice decides whether your number is an upper or a lower bound.
The methodological lesson is narrower and more annoying. I had recorded constrained decoding as the strongest competing explanation and left it unfalsified. Writing down what you have not ruled out is not the same as ruling it out, and the gap between those two things was, in this case, the entire finding.
Code, pre-registration, evaluation set and all 450 raw model responses: cardinality-eval. The result re-scores without a GPU.