The schema is not the cost
If you run an LLM behind a JSON schema, you have probably worried about what the grammar is doing to the model. I went looking for that cost in a specific place and found the opposite.
0.5 items emitted · precision 0.917
0.8 items emitted · precision 0.882
At k = 1 free generation is ahead by 20.0 points. The schema is costing recall here, which is what everyone expects it to do.
Constrained decoding masks tokens the model wants, and the reported costs are real: several percentage points of accuracy across open-weight models, and at least one function-calling result where unconstrained generation with post-hoc parsing beat constrained decoding outright.
The question
Ask a model to pull every named counterparty out of a 10-K. Does it get worse at that as the number of counterparties grows, holding the document length and the schema fixed? Call that number k.
It is not a new question in spirit. Multi-needle NIAH already reports that retrieval falls as needle count rises. What nobody had measured was the same axis for structured extraction under a grammar, with input length actually held constant, reported as a slope rather than as “it declines.”
And there was a specific reason to care. In my own extraction work I had written down constrained decoding as the strongest competing explanation for any cardinality effect I might find. I recorded it as unfalsified and moved on, which is a comfortable way to stay wrong.
Building a ruler that works
You cannot answer this by annotating documents, because k is the treatment. If it comes from whichever filings you happened to sample and a human has to recover it, it is measured with error that correlates with the outcome, because an annotator's attention flags on exactly the long enumerations where the effect should live.
So I built the documents instead. Take verbatim-validated sentences from real filings, each carrying exactly one named company. Inject k of them at random positions into filler drawn from the same corpus with every capitalised name stripped out. Pad to a constant length. Gold is the set you injected, so k is exact and free.
That is the design. Getting it to work took five attempts, and the failures are more instructive than the design is. I set a sanity gate first: at k=1, recall must clear 0.80. One needle, from a sentence the extractor itself produced in the wild.
It came back 0.333. Then, after a fix, 0.400. Then 0.400 again.
The first cause was that my injected sentences named their own original filer in the third person, while the document was attributed to a placeholder company. The model was being asked who ACME deals with and correctly declining to say Lockheed. The second round found four more defects at once: an all-caps ticker the name-detector missed, paragraph chunking that sliced sentences in half at fixed offsets, gold items that were not company names, and a matching rule strict enough to score Komatsu against Komatsu Cummins Chile, Ltda. as a miss.
Fixed all four. Recall: 0.400. At that point I stopped patching things I could see and ran a controlled comparison instead.
Dilution alone was worth 0.25 recall. I had built the filler by selecting prose for containing no company names, which made it far more inert than any real prompt. The gate was never failing because of the bank or the chunking. It was failing because I had buried one sentence in a haystack far emptier than anything the model sees in practice.
It also exposed something structural. Count, density and length are mechanically linked: density = (count × span length) / total length. Fix any two and the third follows, so every study of item count picks two and inherits a confound in the third. Fix length and let density rise with k, as I did, and high-k documents get a density advantage that partially cancels the effect, so your measurement is a lower bound. Fix density and let length grow, as multi-needle NIAH does, and length suppresses high k, giving an upper bound. There is no third option.
The result
qwen3:14b, 32k context, 75 matched documents per arm (25 at each of k = 1, 4, 16), zero truncations. One caveat first: the k=1 gate above was never met. The final run came in at 0.440 constrained and 0.640 unconstrained against the 0.80 I had set, and the pre-registration said to abandon the constructed design at that point. I ran the comparison anyway. So the absolute levels describe the instrument, not the model, and only the between-arm contrast on identical documents is readable.
Paired on the same documents, unconstrained leads by 0.200 at k=1 and trails by 0.128 at k=16. The interaction test, which I did not pre-specify and ran after a reviewer asked for it, puts the gap at −0.082 per doubling of k (p = 0.006).
So the cardinality effect is real under free generation, and the grammar is not the cost. Whether the grammar is the cure is weaker than I first wrote. The interaction is carried by k=1, where the constrained arm returns an empty list on 13 of 25 documents. Remove those and the slope is zero (p = 0.79): above one item, both arms lose recall at about the same rate.
The emission counts show where the arms differ. At k=16 the constrained arm emits 9.4 items of the 16 available; the unconstrained arm emits 7.2. Precision is identical at 0.946 and 0.948, so this is not a quality tradeoff. The arms differ in where the model stops. An earlier version of this post said a half-filled array is not a valid parse, so the schema forces the model on. That is false: the schema is a bare list with no minimum length, the grammar allows closing the array after any element, and the constrained arm does stop early. Why the schema shifts the stopping point is not established. The experiment that would test it, a minimum item count in the schema, has not been run.
That also kills a number I believed earlier in the week. On a 30B mixture-of-experts model, unconstrained generation cost 18× the tokens. I reported that as a fact about removing the grammar. It was not. That model was writing 7,000 to 12,000 tokens against an 8,192-token context window, so the arm was measuring a wall. On a model whose generation fits, unconstrained costs 1.1× at k=1 and 0.7× at k=16. It is cheaper and worse.
What I would take from this
For anyone shipping structured extraction: the schema is not the thing costing you recall at high item counts. On this model it may be what holds recall up, but that rests on 25 documents per level and one model family. If you have been considering free-form generation with post-hoc parsing to escape grammar masking, measure the item-count axis before you switch.
For anyone evaluating extraction: a single F1 over a benchmark whose k distribution is unstated is not comparable to another benchmark's F1. And if you measure the axis, say which two of count, density and length you fixed, because that choice decides whether your number is an upper or a lower bound.
The methodological lesson is narrower and more annoying. I had recorded constrained decoding as the strongest competing explanation and left it unfalsified. Writing down what you have not ruled out is not the same as ruling it out, and the gap between those two things was, in this case, the entire finding.
One more correction to the record. The pre-registration originally specified a different second factor, model size against think/no-think. The think factor proved inert on September 11 and was replaced by constrained-vs-unconstrained decoding before the unconstrained arm was run; the rejection rule did not change. The repository's first commit contains both the pre-registration and the results, so the ordering rests on the document's own dated amendments, not on commit history.
Code, pre-registration, evaluation set and all 450 raw model responses: cardinality-eval. The result re-scores without a GPU.