On a busy Tuesday my orchestrator agent stopped on every single pull request it was watching and asked me the same question: “Want me to merge this?”
The orchestrator is the agent that runs my software delivery across five repositories. A pull request, or PR, is a proposed code change waiting to be accepted into the project. For weeks the agent had done the hard part of that job: it read the change, waited for the tests, and asked. And every time I looked at the screen, said yes, and went back to what I was doing.
Three interruptions in, I realized I was the bottleneck. The machine had already done the review. I was rubber-stamping it.
What I tried first
My first fix was batching. I told the agent to collect its approvals and ask me once an hour.
That was worse. Now the queue built up behind me, and a green change that should have shipped in minutes waited half a day for my next glance.
So I tried the opposite: merge everything green, no questions. That was worse in a different way. Green only means the automated tests passed, and I have been burned by exactly that before, when my agent’s green was a lie until I ran the real test. A green build is a floor, not a review.
The wrong turns matter here, because the fix turned out to be neither a bigger gate nor a smaller one. It was a smaller job for me.
The two gates
I gave the orchestrator merge authority under two conditions it could check itself:
- The CI is green. CI, short for continuous integration, is the machine that builds and tests every proposed change automatically.
- The automated code review reports zero unresolved threads. I use CodeRabbit, an AI reviewer that raises questions on the code in comment threads. An unresolved thread is an open question nobody has answered yet.
Both are facts a machine can read. No judgment call, no “looks fine to me”, no waiting for me. If either gate is red, the agent fixes the problem or escalates it. If both are green, it merges. Immediately.
That is the whole policy:
merge when: CI green AND unresolved review threads == 0
then: verify what landedThe habit that made it safe
The second line is the part I almost skipped.
Merging without a human in the loop is only safe if someone still looks at what landed. So I gave each repository its own post-merge check. For the frontend repo it is visual QA: the agent opens the development site, clicks through the page it changed, and screenshots it. For the core repo it is a test data workflow run against the main branch. Both run after every merge, and both report to me.
That check earns its keep. In the first two weeks it caught one layout break and one integration mismatch that nothing before the merge had seen.
One honest caveat
The gate has a race of its own, and I found it the hard way. If an agent resolves a review thread before its own fix commit lands, the gate reads “zero unresolved threads” and merges early. My auto-merge once shipped unfixed code that way. I wrote up the whole story in my auto-merge shipped unfixed code because ‘threads resolved’ won a race against ‘fix committed’, and the short version is: after any merge, check the merged code for the fix signature. Do not trust the thread state to mean the work landed.
Review the results, not the requests
Underneath the gates is one principle. The delivery pipeline already is the review. The tests are a review. The AI reviewer is a review. The post-merge checks are a review. A human gating every merge is not a fourth review. It is a queue.
So I named the practice to keep myself honest: results review. Review what landed, not what is about to land.
My own habit changed shape. I stopped reading pull requests before they land and started reading results after they land. Ten minutes a day of screenshots and test runs beats two hours of pre-merge reading, and it catches the class of problem that no pre-merge review catches: what the change actually did in the real environment.
If you run agents that ship code, try it. Give them concrete gates they can check alone. Pair every merge with a verification that runs after it. Then move your attention from the request to the result.
Handing over merge authority felt like giving up control. It was the opposite. The pipeline did the reviewing, the results did the convincing, and I got my afternoons back.