On a leadgen repo, two pull requests had each been through six rounds with CodeRabbit — the AI bot that reads your code change and posts review comments before a human merges it. Sixth cycle, same files. I pushed the fix, watched for the review, and got a green check mark and nothing else. No comment. No verdict. Just silence where the opinion was supposed to be.
Here’s the thing my merge rule was supposed to do, in plain terms: don’t merge until a reviewer has actually looked at this version and said something about it. A passing check was necessary but not enough — green just means “the bot ran,” not “the bot approves.” I wanted words. A real review body tied to the fix-push commit.
What I’d missed is that CodeRabbit has an incremental-skip heuristic. After enough cycles on the same files, it decides the diff is too small to bother re-reviewing, posts a SUCCESS check, and skips writing a review. I poked it the obvious way — @coderabbitai review, then @coderabbitai full review. Both got acknowledged. Neither produced a review body. The bot was politely telling me it had nothing to add, but it told me by saying nothing, which is the one answer my gate couldn’t read.
That’s the trap. Silence from an AI reviewer is genuinely ambiguous. It could mean backlog (it’s coming, wait). It could mean effective-approve (looks fine, no notes). It could mean skip (not worth re-reading). It could mean missed (the webhook dropped). Four very different situations, one identical signal: nothing. And my first instinct — “no complaints, ship it” — quietly collapses all four into “approved.” That’s how you merge something nobody actually reviewed.
So I refused to let absence stand in for a verdict. I built a narrow carveout: if the review body doesn’t show up within 60 minutes of the fix-push, the automation stops and surfaces the situation to a human. Not “I recommend merging.” Just the facts and the options — here’s the PR, here’s that CodeRabbit went quiet after six cycles, here’s what you can do. A person makes the call, and the reasoning lands as a PR comment, not buried in a commit message where nobody will find it during the next incident.
The escalation carveout give me the detail
The gate logic, roughly:
check = coderabbit_check_status(pr, fix_push_sha)
review = coderabbit_review_body(pr, since=fix_push_sha)
if check == "SUCCESS" and review:
allow_merge()
elif minutes_since(fix_push_sha) >= 60:
# Silence is ambiguous — do NOT auto-merge.
post_pr_comment(pr, render_options_no_recommendation())
require_human_override(label="coderabbit-silence-override")
else:
wait() # backlog is still possibleTwo deliberate choices. The escalation comment frames options with no pre-recommendation — pre-recommending anchors the human toward your default and quietly defeats the point of asking. And the override requires a reasoning comment on the PR itself: a durable, greppable audit trail you can replay six months later when someone asks “why did we merge this without a review?”
The mechanism that makes this work is treating “no response” as its own explicit branch in the logic, not the gap between the other branches. Any time you gate an action on an agent’s output, ask what happens when the agent returns nothing — because eventually it will. Wire the empty case to a human with framed options and no nudge, and make the override leave a trace somewhere you’ll actually look.
A green check told me the bot ran. It never told me the bot agreed. Don’t let your automation confuse the two.