Judge the Fact Before You State It

Last week a Claude Code session handed me a draft with a date in it, stated at full confidence, and I almost shipped it before I checked. The session had no idea whether the date was right — it just sounded like it did.

When an AI assistant (a large language model — a program that predicts the next word) states a fact you’ll act on — a date, a number, a claim about what someone said — the cost of it being wrong lands on you, later, often in front of other people. The model sounds just as certain when it’s right as when it’s wrong, and that fluency is exactly what makes a wrong fact dangerous.

The cheap fix I keep coming back to: before shipping a load-bearing answer (one that other decisions or messages rest on), route the draft through a separate model — a second AI whose only job is to judge the draft against the actual sources. Not the model that wrote the answer grading its own work. A different pass, pointed at the evidence, returning a verdict: pass, revise, or block.

The economics are the whole point. A human’s attention is the scarce resource; a judge model’s attention is nearly free. So substitute the cheap one for the expensive one. Instead of me carefully fact-checking every claim, a judge checks it in a minute and only escalates the ones that don’t hold up.

That substitution is the architecture.

A few things made it work in practice. The claim you submit to the judge and the claim you actually ship have to contain the same facts — audit “Thursday the 26th” and then ship “the 26th” and you never checked the Thursday. I’ve caught that mismatch after the fact before. And a failure in the judge itself is not a verdict; “the judge didn’t run” means retry, never “looks fine.” Treating a crashed judge pass as a green light is the one mistake that will quietly let a wrong fact through.

The line for what needs judging: would you repeat it externally, or base a decision on it? If yes, judge it. If it’s reasoning or opinion, label it as such and move on.