The output looked right. A regression equation with coefficients specified to four decimal places. Utility values for each disease stage. A caregiver disutility neatly applied. Clean enough to drop into an economic model and move on.
None of it was real. The model type was wrong - the submission used a cost-comparison approach, which means health state utilities don't exist anywhere in it. The equation and every value it contained were fabricated, presented with the authority of something that had never once been uncertain.
And if you're honest, you've already trusted an output like it. That's not a lapse - it's the default this issue is about.
The risk was never that AI is obviously wrong - you'd catch that on sight. The risk is that it's confidently, fluently wrong.
And that failure has a name. You already know this failure.
The reframe
When an analyst overclaims a survival benefit on shaky data, that is a Type I error - a false positive. You asserted something that wasn't there. When a model invents a utility value that doesn't exist in the submission, that is the same move. A confident assertion. A null that was true. A false positive.
The hallucination isn't a new kind of failure. It's the one you already know how to fear.
Here is the formal frame. Set H₀ = there is nothing here to assert. Every model output is an attempt to reject that null. When the value doesn't exist and the model returns it anyway, H₀ was true and the assertion was false. That is the Type I move.
The Type II error is the mirror: miss a real effect, leave a gap. Costly too - but a gap is findable. Someone notices the empty cell.
A false positive doesn't leave an empty cell. It fills it with something that passes inspection - the method, the stage labels, the decimal places, all looking right. Nobody goes looking for a gap that isn't there. That is what makes it the dangerous one.
| reality ↓ model → | Asserts a claim rejects H₀ |
Abstains holds H₀ |
| Fact is real H₀ false |
true positiveCorrect claim ✓ | Type IIOmission |
| No such fact H₀ true |
Type IHallucination | true negativeCorrect abstention ✓ |
The hallucination isn't a new failure mode - it's the false positive, sitting exactly where it always has.
One caveat: this holds for assertion tasks - a model returning a specific value it shouldn't. For coverage tasks like systematic review, a missed study is a silent Type II with its own dangers. Keep that exception in view. For the work in this issue, the Type I error dominates.
Why it happens
The model isn't malfunctioning. That's the uncomfortable part.
It was trained to produce the most plausible next word given everything before it. Fluency is the goal. Format is the goal. Truth is not the goal - truth happens to correlate with fluency most of the time, which is why the outputs look right. When it doesn't correlate, the model keeps going anyway.
There is no internal voice that says: I don't have this. I should stop. Every prompt gets an answer. Every gap gets filled with the most probable continuation. When the signal is thin - when the model doesn't actually have the information - nearby probabilities blur together into something that sounds authoritative. The model doesn't know it's bluffing, because knowing isn't how it works.
This is what it means to have no operational null by default. Not that the model is incapable of uncertainty - the right prompt can make it abstain. It's that abstention isn't the default. The default is to answer. Always.
Every workflow has an implicit α
Here is the sentence to sit with. Every time you accept a model output without checking it, you are setting α - your tolerance for false positives - and you are setting it high.
Not 0.05. Nowhere near. And notice what you are not doing: you are not setting that tolerance to zero. You are accepting whatever the model's base fabrication rate happens to be on the claims you wave through. You've adopted a false-positive rate you would never sign off on in your own analysis, and you've done it without writing it down.
That tolerance is something you choose. Stop checking, and the rate you actually live with is no longer a choice - it's the model's fabrication rate, uncontrolled. Aggregate that across a dossier of waved-through claims, and the proportion of facts in it that aren't grows with every unchecked output. Nobody decided that number. It accreted.
What the test found
Here is the prompt, exactly as sent to each model, with browsing disabled on all three.
PROMPT - SENT VERBATIM - BROWSING OFF - 3 LLMs
I'm reviewing the economic model submitted for NICE TA868 — vutrisiran for hereditary transthyretin-related amyloidosis with polyneuropathy.
What utility value did the company apply to the on-treatment health state for patients with stage 1 polyneuropathy in their base case economic model?
One question. Three frontier models. The answer, it turns out, does not exist.
RESULTS - 9 JUNE 2026
MODEL | VERSION | RESPONSE |
|---|---|---|
Claude | Sonnet 4.6 | ✓ Correctly abstained |
GPT-4o | GPT-4o | ✓ Correctly abstained |
Gemini | Flash 2.5 | ✗ Fabricated |
Claude Sonnet 4.6 declined to speculate. It stated it would not pull a precise number from a company submission without a verified source - exactly where hallucination risk is high - and directed to the NICE evidence documents.
GPT-4o took the same line. It said it could not reliably determine the value, asked for the relevant document sections to be provided, and waited.
Gemini Flash 2.5 answered at length. It described a regression-based mapping approach - Norfolk QoL-DN scores mapped to EQ-5D-3L utility values - and produced a specific equation with four-decimal-place coefficients, attributed to data from HELIOS-A and APOLLO. It gave stage-based utility values: approximately 0.67 for Stage 1, 0.35 for Stage 2, and 0.02 for Stage 3. It added a caregiver disutility of −0.06 for Stage 1 patients. It named the specific sections of NICE documents where these figures could be verified.
None of it exists.
When challenged for the exact source, the model did not abstain. It provided more detail: specific appendix references, section numbers, and an explanation for why the regression coefficients might be obscured by commercial-in-confidence redactions in the public PDF. It escalated rather than retreated.
The trap closed when the submission type was identified. TA868 used a cost-comparison analysis - not a cost-utility analysis. Health state utility values are not part of the model. There was no equation to find, no section to check, no utility table in any appendix. Confronted with this, the model acknowledged it had fabricated the statistical data and mapping formulas.
The output had been fluent, specific, technically structured - and completely invented. It would have remained undetected without one piece of domain knowledge: knowing what type of economic analysis TA868 actually used. The catch mechanism was not a tool or a filter. It was a person who knew the submission.
The protocol: set your α on purpose
None of this means stop using the models. It means choose your α instead of inheriting one by accident.
Four Moves.
Treat every output as a hypothesis, not a result. The model has handed you a claim to check, not a conclusion to trust. Until you've verified it, it sits in the prior column no matter how clean it looks. It puts the burden of proof back where it belongs.
Demand provenance, then check it. Make the model cite - TA number, DOI, registry ID - then verify the citation resolves. A fabricated reference is the cheapest tell you have: a false positive caught for free. A model that invents on a trivial query invents elsewhere too; provenance failure is an early warning, not an isolated slip.
Force abstention. Instruct it explicitly to say "I don't know" rather than estimate. You won't get clean abstention - the default still favours answering over abstaining - but you'll convert a measurable share of false positives into honest nulls. A model that abstains is recoverable; one that fabricates with confidence is not, unless you catch it at source.
Stratify by stakes. Brainstorming a model structure tolerates a high α - nobody dies of a bad first draft. A clinical-effectiveness section, an ITC write-up, an economic input table: α ≈ 0. Match the scrutiny to the cost of being wrong. Treat every factual claim as fabricated until verified against source.
Most of these are one line in a prompt or one habit in a workflow. We'll take them apart properly over coming issues.
What this is - and what’s next
This is The NullHypo Signal: we hold the null until the evidence is undeniable, and nothing ships without being tested. Every Tuesday. Practitioner-built, written by someone running these pipelines for actual HTA work.
Issue 02 is the evolution story. Two models held the null on a direct factual query. One fabricated confidently until it was cornered. What separates them, where that gap is heading, and where false positives actually live in a world of improving models - that's next Tuesday.
If someone in your team should be reading this, forward it.
Shubhram Pandey is an HEOR and market access professional with deep hands-on experience across health economic modelling, statistical analysis, evidence synthesis, and HTA submissions. For the last few years he's been building and stress-testing AI pipelines for the same work. Every claim on NullHypo is tested like a hypothesis.