My last post said I approve every change to Khipur and read few of them: the checks select, and I design the checks. A fair question came back: then what is the human for? At my day job the question arrives from the other side. There, people do read every pull request, and now so do the machines. AI reviewers leave their comments before any person opens the diff.
Both are the same question. When a machine reviews first, what's left of the human's review? I wrote a decision record about it for work. This is the argument without the company in it, and what it changes here.
The flywheel
Coding agents raise the number of pull requests. More pull requests raise the demand for faster review. That demand produces automated reviewers, and their comments produce the feeling that the code has been checked. The feeling makes the human's pass shallower, and a shallower pass lets more through, in a pile that keeps growing. Nobody decides any of this. Each step is reasonable.
Other fields call the last step automation complacency: when a machine is usually right, the person beside it stops doing their own work and starts agreeing with it.
Two numbers from this week
Alibaba has just open-sourced the reviewer it uses internally, with a benchmark: 200 real pull requests, and the same model underneath both tools. By its own figures, 33.9% of what it flagged was a real problem, against 7.2% for Claude Code's review, and it found 20% of the known problems against 28.9%. The benchmark is the vendor's own and nobody else has repeated it, but the shape is believable.
Read those as the reviewer on the other end. With the better tool, two comments in three are still wrong. With the more thorough one, seven known problems in ten are never mentioned. The first number teaches a person to skim the comments. The second means a pull request with no comments has told them nothing. A reviewer who takes either as a sign the code was checked has stopped reviewing.
Three fields got here first
Aviation has lived with a reliable autopilot for half a century. Its answer was never to trust the autopilot less. It was to give the people a job the autopilot doesn't do. One pilot flies and the other monitors, with different duties, and below 10,000 feet the sterile cockpit rule bans everything that isn't flying the plane. Nobody in a cockpit is allowed to think the automation has it.
Medicine ran the experiment directly. In a 2023 study in Radiology, 27 radiologists read mammograms beside a suggestion they were told came from an AI. When the suggestion was right, the most experienced readers were right 82% of the time. When it was wrong, 46%. The least experienced fell from about 80% to under 20%. Nothing about their skill had changed, only what they saw first. It's why the UK's breast screening programme has two people read every mammogram, and why it matters that the second not start from the first one's answer.
Medicine also measured noise. A review of drug-safety alerts in hospital ordering systems found that clinicians overrode between 49% and 96% of them. Past some rate of false alarms, people stop reading alerts at all, the true ones included.
Self-driving cars named the worst arrangement: the car drives and the human watches, ready to take over. People are bad at watching a system that works. When the US safety board investigated the 2018 crash in Tempe, where a test vehicle killed a pedestrian while its operator looked at a phone, it listed the company's failure to address its operators' automation complacency among the causes.
Who goes first
Put together, the three say one thing: the order matters more than the tool. From safest to worst:
- The person reviews, and asks a model questions as they go. Nothing the model says reaches the pull request.
- The person forms a view first, and then sees what the model found: double reading, with the model as the second reader.
- The model's findings are on the pull request before the person arrives. This is where most review bots sit.
- The model reviews and the person watches. This is the car that drives itself until it can't.
The test is simple: is the person waiting for the bot, or is the bot waiting for the person? If the person is waiting, the bot is driving. A tool doesn't escape this by being interactive. One that presents findings for a person to accept or skip has them reacting, not reviewing.
One thing can go first without harm: a briefing. A model that says what a change does (it adds an endpoint, it queries this table, its migration adds an index) helps a person understand faster, and hasn't told them what to think of it. Pilots get the weather before they fly. A briefing isn't a finding.
Four rules
- Measure it. Anyone running an automated reviewer should be able to say what share of its findings were wrong, and what share led to a change. A reviewer nobody has measured is a guess.
- Comments, not gates. A finding blocks a merge only once its precision has earned that, which means people aren't routinely overriding it.
- The person goes first. Findings arrive after the human has engaged with the code. No findings isn't evidence the code is right, and findings aren't a reason to skip the reading.
- Approval is a person's act. No bot, script or agent approves on someone's behalf. If you didn't press the button yourself, you didn't approve it.
What this has to do with selection
A reviewer, human or model, samples. It looks at one change, once, and catches what it happens to notice that day. Selection is different: a check runs on every change, for good, and I can measure whether it works by breaking the code and seeing if it complains. So the most useful thing a reviewer's finding can do is stop being a finding. If a model is right that a class of bug can happen here, the answer is a test that makes it impossible to merge, not a comment on the next forty pull requests. A model's rule guarding something dangerous is a sign the codebase is missing a structural one.
That's how Khipur uses them. No model comments on a pull request here. Models work on discovery: a monthly review reads the codebase for rules nothing enforces, and it may keep three findings at most, each with a file and line, a concrete way it fails and a proposal to turn it into a check. A finding without a failing case isn't acted on. None of it blocks a merge. What does block is deterministic, and its false-positive rate is one I can count.
What I see on a pull request is a briefing. The scope report says which files changed and which checks cover them, and flags the kinds of change that always need a person. It doesn't say what's wrong. For planned work the agent also lists each judgment call the plan left to it, so what I read is the decisions, not the diff.
Where I'm exposed
I should be honest about that list. Approving more than a thousand pull requests I mostly haven't read is its last arrangement, the car that drives itself. The agents are usually right, and I'm the person watching.
What I have against it is forcing. The flags make me read migrations, auth and privacy code. The judgment calls make me read choices. And the approval is always mine, a word I type or a switch I turn, never something an agent can supply. The one class that merges with no approval at all, tests and docs and the like, I've stopped pretending to review. There the checks are the reviewer, the class was widened on evidence, and an escape narrows it again.
That isn't the same as reading everything, and it isn't meant to be. But the lesson from the cockpit and the reading room is the one I'd keep even if every number here changed: a second reviewer only helps if the first one still reviews. Whoever goes second should be the machine.
Sources
- Dratsch and others, Automation Bias in Mammography, Radiology, 2023.
- van der Sijs and others, Overriding of Drug Safety Alerts in Computerized Physician Order Entry, Journal of the American Medical Informatics Association, 2006.
- 14 CFR 121.542, the sterile cockpit rule.
- National Transportation Safety Board, report HAR-19/03, on the 2018 collision in Tempe, Arizona.
- OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review, arXiv 2608.09290, for the benchmark's figures.