How much human code review does AI-generated code need? The numbers the Claude Code team published
Tools in this post
As of 17 October 2026, a team with agents writing most of its code should keep one human approval on ordinary pull requests and two on risky paths, and let low-risk changes merge on automated checks only once it tracks revert and incident rates and they stay flat. That is the policy the Claude Code team published on 15 October: with Claude writing about 90% of the code, merged pull requests "more than doubled" from June to September while the share that had to be reverted "fell by more than 40 percent". The team did not publish the counts behind either number.
How much human code review does AI-generated code need?
AI-generated code needs the least human review time that does not raise reverts or incidents. That is the Claude Code team's own target: "the minimum amount of human review time that doesn't raise reverts or incidents." I would measure it per team and per risk tier rather than set it as a rule. Its goal is that most pull requests "merge with at most one human in the loop, who can be the author".
The post sets the guardrail too: "Revert and incident rates are the guardrail: if they rise, we haven't saved human time, we've only moved it." Human time counts both sides, reviewing and fixing what bad code breaks. A team that cuts review and then spends the hours on rollbacks has saved nothing.
That turns the question from a culture argument into a budget, and the budget works only if the two rates are counted the same way every month.
What the Claude Code team measured, and what it left out
The Claude Code team published three headline numbers about its own repositories: about 90% of the code written by Claude, merged pull requests more than doubled from June to September 2026, and a revert share more than 40% lower. Anthropic sells Claude Code and the Code Review product the post promotes, and every number is internal telemetry with no method section.
What the post does not give:
- Counts. No number of merged pull requests, reverts or incidents, and no baseline revert share.
- Definitions. What counts as a revert, over what window, and what counts as an incident.
- Human time. The team says it tracks "human approvals per PR (median and 95th percentile), human comments, and how long a PR waits on a person" and publishes none of the three. So the post does not show that human review time fell.
- Tiers. How many pull requests merged with zero, one or two humans.
The two published ratios do not settle whether reverts went down. At exactly double the pull requests and a 40% lower share, there would be 1.2 times as many reverts; "more than" on both sides leaves either direction open. If the extra pull requests were mostly docs and flagged changes, the share falls even when risky changes revert as often as before.
One more figure gets mixed in. The post's own 2,500 to 30,000 changes a day, which several Hacker News commenters read as Claude Code's, are from Anthropic's monorepo, and the post says "Claude Code lives in its own repository, separate from the monorepo".
Three review policies, and what each commits a team to
A team deciding how much to review has three policies to choose from as of October 2026: a human reads every pull request, review scales with the risk of each change, or agents review and merge without people. Each commits the team to different reviewer hours, different tooling and a different answer when something breaks.
| Human reads every PR | Review scaled by risk | Agent-only review | |
|---|---|---|---|
| Human approvals | One or more on every change | None on low risk, one on medium, two on high | None |
| Reviewer hours | Grow with agent output | Grow with risky changes only | Near zero |
| What must exist first | Nothing new | A risk classifier, CODEOWNERS, revert and incident tracking, fast rollback | All of that, plus a reason to trust it on every path |
| Who owns a bad change | The reviewer and the author | The author | Unclear |
| Who runs it today | Google's pre-agent median: one reviewer on a 24-line change (Sadowski et al., 2018) | The Claude Code team; Duckbill Group, with human review only on API, auth, schema and similar paths, per the six layers post | Rare; argued for in Martin Monperrus's position paper, with no new data |
The Claude Code team's tiers, from the post: low risk covers "docs, copy, internal-only features, additive changes behind a flag" and needs no human once automated review and security checks are green. Medium is ordinary product work and sensitive areas such as caching or system prompts, and needs one person. High is "the highest-risk code, plus a small set of paths that always need two people".
The checks that run before a person sees the diff
In the Claude Code team's process, a person usually sees a pull request only after the agent has verified its own work, a Claude review has posted findings, a security review has run, and lint, type checks, tests and a build have passed. A merge queue, feature flags, a staged release and fast rollback sit behind the human approval. Each layer is expected to miss things the next one catches.
- Verified before a pull request exists. Claude gets "something it can run", tests, a build or a screenshot, and has to show the output. A subagent with fresh context reviews the diff.
- AI review on every pull request, where several agents hunt for bugs, a separate group of agents votes on each finding, and it is posted only if enough of them agree it is real. Every finding has to be fixed or closed with a reason.
- Security review that can block until each finding is resolved.
- Lint, types, tests on every platform and a build, plus a template asking how the change was tested and how to undo it.
- Merge queue, flags and a gradual release, with flags that turn off without a deploy.
The review loop has numbers of its own. After four changes to how the reviewer re-reviews, the share of internal pull requests passing after one round went from 65% to 78%, and the mean number of reviews to pass from 2.08 to 1.32. The post gives no dates or counts for that comparison.
Humans keep five decisions, and the change sets the tier
The Claude Code team keeps people on five decisions: whether to build the thing at all, risky code such as billing, the auto-updater and CI workflows, architecture that others will copy, reading the evidence the checks produced, and owning the outcome. The rules around the tiers matter as much as the tiers.
- "The change sets the tier, not the author." A classifier picks the tier, and a second one asks whether this change needs a person anyway.
- "An agent's approval never counts as a person's."
- "An approval covers one specific commit." An agent's later push means approving again.
- Only authors on the repository's trusted list get the lighter paths.
On ownership, the post's rule is "you land it, you own it": if a small or medium change breaks, the person who landed it usually fixes it, as a team norm. It works, by the post's own list, because changes are small, most mistakes can be undone within minutes, regressions surface fast and the fix "isn't blame". The person directing the agent counts as a required reviewer, so on a medium change an author on the trusted list can be the only human who looked.
The risks: a metric that can be gamed, slow debt and the substitution myth
The risks in copying this policy are three. Revert and incident rates can be gamed or can miss slow damage. Outside studies find agents add review load. And the work a human reviewer does is wider than detection. The Claude Code team names one gap itself: there are no "circuit breakers that tighten approval when those numbers move" yet.
The metric. On Hacker News, one commenter asked whether this is "Goodhart's law in action", and another that revert and incident rates miss "the long-term, cumulative impact of compounding complexity". GitClear's 2026 analysis, from a vendor selling code analytics, reports code block duplication up 81% since 2023, the kind of debt no revert shows.
The load. A Harvard working paper on Jellyfish data from 718 firms, January 2021 to March 2026, found that adopting AI agents raised the days from pull request to merge by 49% and nearly doubled the share of pull requests with changes requested (Chen and Stratton). Google's 2025 DORA report found AI adoption still linked to lower delivery stability. Both predate the Claude Code numbers.
The substitution myth. John Allspaw argues that splitting review into functions an agent can each perform misses the work across them: "When an experienced engineer reads a diff and says "I don't understand this.", their confusion is the finding." Allspaw adds that review is often "the last (or sometimes only) moment when someone can be expected to challenge whether the change is even necessary", and that "the agent reviews what is there; engineers with expertise can easily notice what's missing." The post is from 24 August, in reply to Martin Monperrus's position paper, not to Anthropic. The Claude Code post keeps "should we build this at all" with people, which answers part of it, and replaces shared knowledge with explainers Claude writes on request, which Allspaw would call transmission, not the two-way conversation Allspaw's post describes.
The product also matters. One commenter argued the policy works because a Claude Code bug rarely causes "financial or irreversible safety loss", and doubted it would hold where "a minor slipup can result in a huge real world mess". A payments or medical codebase has a different revert cost.
What would change the recommendation
The recommendation moves toward more human review if a team's revert or incident rate rises after loosening a tier, if a security or compliance auditor requires a second human on every production change, or if the code fails silently. It moves toward less if the team can show that reviewers' findings on low-risk changes catch nothing the automated layers missed, over months of data.
One test helps either way. A commenter on Peter Harrison's Epistemology of Quality suggests planting known defects and comparing miss rates across layers: if the combined miss rate sits near the best single layer's, the layers share a blind spot and adding another automated reviewer buys little.
On Stackness, 10 public profiles list GitHub and 8 list Claude Code as of 17 October 2026, six of them both; none list CodeRabbit yet (data sources). The numbers are small. The AI tools developers list is where review agents show up first.
What I would do
- If the team does not count reverts and incidents per merged pull request yet, keep a human on every change and start counting. A month of baseline comes before any tier is loosened.
- If it does, write down tiers set from the change, not the author, and loosen one tier at a time, starting with docs and flagged additive changes.
- If a change touches auth, billing, CI workflows, installers or data deletion, require two people through CODEOWNERS and branch protection, whoever wrote it.
- If an agent's push lands after an approval, dismiss the approval. Never let a bot's approval fill a human slot.
- If the code fails silently (schemas, money, permissions), keep human reading there even when the rates look fine.
- If the revert or incident rate rises after a change, tighten the tier back the same week.
The six verification layers post covers what each layer catches, and the CI bottleneck numbers cover the queue that review sits in.
New posts by email
About one email a week with what we published. No account needed.
One email to confirm first. Every email has a one-click unsubscribe.
Which of these do you use?
Nobody on Stackness runs CodeRabbit yet. Tick what you use and be the first.
