Two pull requests for a leadgen tool of mine had already been through six rounds with CodeRabbit — the AI code reviewer I wired into my repo to read every change and leave comments like a human teammate would. Each round it found something real: an off-by-one, a missing null check, a query that would have quietly scanned the whole table. I fixed each one and pushed again.
Then, after the sixth fix-push, it just… stopped talking.
CI was green — the automated test suite passed, the little check turned to SUCCESS. But my merge rule doesn’t trust a green check alone. I gate merges on a fresh review body: actual written words from CodeRabbit on the latest commit, not just a passing pipeline. A green check tells me the code runs. It doesn’t tell me the code is good. Those are different questions, and I’d been burned enough to keep them separate.
So I did the obvious thing and poked it. @coderabbitai review. It acknowledged me — thumbs up — and posted nothing. I tried @coderabbitai full review. Same ack, same silence.
Here’s what I didn’t know at first: CodeRabbit has a quiet habit I’ll call incremental-skip. After you’ve pushed several fixes touching the same files, its heuristic decides a fresh walkthrough isn’t worth the tokens and just doesn’t write one. The ack is real. The review never comes.
And that’s the trap. Because the silence had at least four possible meanings, and I couldn’t tell them apart from the outside: maybe it was backlogged and would post in ten minutes. Maybe it had genuinely nothing to add — an effective approval. Maybe it skipped. Maybe my trigger got dropped on the floor. Silence is ambiguous, and the failure mode is that ambiguity quietly resolves to “approved” just because you’re tired of waiting.
I refuse to let “no comment” mean “yes.” That’s how a review gate slowly rots into a rubber stamp.
The 60-minute carveout and where the audit trail lives give me the detail
The gate normally requires a CodeRabbit review body whose commit SHA matches HEAD. The escape hatch is deliberate and slow:
merge_gate:
require: coderabbit_review_on_head_sha
carveout:
when: no_review_body_after_minutes >= 60
requires: human_override_comment
override_pattern: '^OVERRIDE:\s+.+\breason\b'Two things I made non-negotiable. First, the override is a PR comment, never the merge commit message. Squash-merges rewrite and bury commit text; the PR conversation is the durable, queryable record an auditor actually reads six months later. Second, only a human can trip the carveout. Agents can merge on a real review body all day — routine stays automated — but standing-rule exceptions stay with a person who types out why.
For these two PRs I waited out the hour, then left the override with my reasoning: six substantive cycles already resolved, green CI, incremental-skip confirmed by two ack’d-but-empty triggers. Then I merged.
The transferable part: if you gate anything on an AI’s output, decide in advance what its silence means. Don’t discover your policy at 11pm with a tired thumb over the merge button. Write down the ambiguous cases, give yourself a slow and explicit override, and put the reasoning somewhere a future human can find it — the record, not the commit.
Absence of a verdict is not a verdict. Make your pipeline say so out loud.