My merge gate said “3 green / ready to merge.” GitHub said BLOCKED. One of us was wrong, and it turned out to be me.
Here’s the setup in plain terms. I was building an automated check that decides whether a pull request — a proposed batch of code changes — is safe to merge. The check looked at two things: CodeRabbit, an AI reviewer that leaves comments on the code, and GitHub’s own branch protection, a structural rule that physically blocks the merge button until conditions are met. My job was to combine them into one honest yes/no.
To count CodeRabbit’s open comments, I wrote what felt like an obvious filter: count threads where isResolved == false AND isOutdated == false. A thread is “outdated” when the lines it points at have moved — say an earlier commit added ten lines above it, so the comment’s line numbers no longer line up. My reasoning was: outdated means stale means probably already handled. Skip those.
The filter returned 0. Three green checks. Ship it.
Except GitHub’s mergeStateStatus sat stubbornly at BLOCKED. Every check I could see was green, and the structural gate still refused. That disagreement is the only reason I looked closer.
When I dug in, there it was: a 🔴 CRITICAL finding — a fail-open security hole where a check that should deny access let it through. It was marked isOutdated == true. But not because anyone fixed it. It was outdated purely because an earlier commit shifted the line numbers underneath it. The finding was still completely valid against the current code. My own filter had buried it.
That’s the part that still bothers me. The bug wasn’t in CodeRabbit and it wasn’t in GitHub. It was in my verification tool. I had written outdated ⇒ already addressed into the filter as if it were a fact, and it was just an assumption — my assumption. A filter you author yourself inherits every blind spot you have. It can’t catch the thing you already failed to think of, because the same brain wrote both.
What saved me was the coarser, dumber gate. GitHub’s require_conversation_resolution counts all unresolved threads regardless of outdated state. It doesn’t try to be clever about staleness. It fails closed — when in doubt, it blocks. My hand-rolled filter tried to be smart and failed open.
The filter bug and the gate that caught it give me the detail
The wrong query — note the isOutdated exclusion that encoded my assumption:
# WRONG: "outdated" silently drops still-valid findings
reviewThreads(first: 100) {
nodes { isResolved isOutdated }
}
# count = threads where isResolved == false && isOutdated == false -> 0The fix is to stop interpreting isOutdated at all. Outdated means lines moved, not finding handled:
# RIGHT: only resolution closes a thread
# unresolved = nodes.filter(t => !t.isResolved).lengthAnd the real source of truth — let the structural gate decide, don’t override it:
gh pr view "$PR" --json mergeStateStatus -q .mergeStateStatus
# BLOCKED -> investigate, even if your own checks are green
# CLEAN -> saferequire_conversation_resolution in branch protection counts every unresolved thread, outdated or not. It’s coarser than my filter and that’s exactly why it’s more trustworthy.
So here’s the rule I now follow. When you count unresolved review threads, count isResolved == false and exclude nothing else — “outdated” is about geometry, not about whether anyone fixed the problem. And when a dumb fail-closed gate disagrees with your clever green checklist, the gate is the witness and your checklist is the suspect. Go look. Don’t override.
The deepest bugs don’t live in the code under review. They live in the tool you trusted to tell you the code was clean.