I've stopped doing deep code reviews on most of my PRs. That may sound reckless for a healthcare company, but evidence bundles replaced them and arguably provide higher confidence in agent code changes. Before I look at the code, an agent exercises the change in a real build and posts what it tested, what happened, screenshots, and what it couldn't cover. I review that evidence, then skim the code.
This is the latest piece of a story we've been telling about Clipboard Engineering. In Agents Can't Iterate Against Tests That Lie, we fixed the tests agents rely on. In Tickets to Pull Requests While You Sleep, agents autonomously open PRs. In Preflight, a bot approves the simple ones. This post explains how humans review the rest.
Reading the diff tells you what the code says. Evidence tells you what the code does. I trust the second more, and you can rerun it on every push.
What an evidence bundle looks like
Sub Gautam built our cb-validate skill that agents run on frontend PRs. The skill:
- Finds the PR's preview build link
- Logs in as a test account
- Plans scenarios from the diff and the ticket
- Takes screenshots of the flows in a real browser
- Posts the result as a PR comment
- Edits the same comment on new commits so reviewers see the latest evidence.
Here is a trimmed bundle from a support chat fix:
| Check | Action | Expected | Observed | Result |
|---|---|---|---|---|
| Send a message | Send a text on a working connection | Message shows "Delivered" | Showed "Delivered" within about 10 seconds | PASS |
| Send fails (failure path) | Block the message request, then send | "Sending...", then "Failed to send" with a Retry link, never "Delivered" | "Sending..." for 30 seconds, then "Failed to send" with Retry. The message stayed on screen | PASS |
| Retry | Restore the network, tap Retry | One copy is delivered, with no duplicates | One request, one copy, "Delivered" | PASS |
Below the table, the evidence bundle lists what it did not cover and any test data it created or changed.
The bundle's screenshots for the failure path and retry. This was the second run: the first run caught a bug where a failed send showed "Delivered" and then disappeared.
Navigating around the app doesn't count as proof. The verdict has rules:
- PASS requires a verified build, every requirement mapped to a passing scenario, at least one negative check, and evidence on every pass.
- PARTIAL names exactly what it couldn't reach so we know which tooling needs updating (e.g., new seed scenarios, simulator versions).
- FAIL means the app disagreed with what the PR claims to do.
A bundle that says PARTIAL and tells me why is more useful than a reviewer who says "LGTM" without actually validating the change in the app.
How I review a PR now
What I ask the agent to do depends on the kind of change:
- Frontend changes:
cb-validateruns automatically on the PR's preview build. - Infrastructure changes: I apply them in a lower environment first, then ask the agent to test them and post the results on the PR.
- Native changes: I've given my agents access to an Android phone over Android's debugging bridge.
I have very little native app knowledge, so using an actual phone lets me investigate behavior I wouldn't be confident checking on my own. On one ticket, a phone test reproduced an analytics bug that browser tests couldn't; the agent fixed it, then ran the app again to confirm.
When I open the PR, I check four things:
- Do the scenarios cover the intended behavior and the failure paths?
- Did the agent exercise the real workflow, or a shortcut around it?
- Does the evidence come from the latest commit?
- What gaps could the agent still cover?
Here's an example of the last one: while validating a sign-in change, a passkey feature flag was on, but the agent's browser didn't support passkeys, so the first run never reached the passkey code path. I updated the cb-validate instructions to use a virtual authenticator in that situation. I'd seen agents figure this out on their own before, but writing it into the skill made the check repeatable. A gap I find in one PR's evidence becomes an improvement every engineer gets on their next PR. A gap I find by reading code helps one PR.
Then I look at the screenshots: what changed, whether the copy makes sense, whether the UI looks right. We use common UI components, so I'm not worried about pixel perfection. Only after that do I skim the code for obvious code smells, such as a new helper that duplicates an existing one, or an implementation that meets the ticket's acceptance criteria without asking whether there's a simpler solution. Agents don't ask "should this PR even exist?" That question is still mine to decide.
I leave my comments, then let the agent babysit the PR: it addresses review feedback, including CodeRabbit's, and runs a final validation round once everything is resolved. Sometimes I click around the preview myself to check responsiveness or anything else screenshots can't capture.
What the numbers say
We piloted cb-validate from September to October. Engineers chose which PRs to run it on:
- 266 evidence bundles from 24 engineers, about 14% of frontend PRs in that window.
- 5 bundles went from FAIL to PASS: a failing run, a code change, then a passing rerun on the new code. Each run found a confirmed defect that was fixed before merge.
One engineer described a bug it caught as "not super obvious," and noted that it "walks through a lot of different cases very quickly, that would take me a lot of time to set up individually."
It has limits. About 1 in 5 frontend changes have behavior a browser can't reach, and for about 7%, none of the changed behavior is reachable. Most PARTIAL results came from missing test data, which we've since improved. For native behavior, cb-validate can now build the PR's exact commit for the iOS Simulator and Android emulator, and drive it with Maestro.
Where I think this is going
Rocky Warren predicts the imminent death of human code review. We can't stop reading all code yet, but I think it'll happen sooner than many expect.
Between August 2025 and August 2026, we quadrupled deployed PRs, and the time from PR created to deployed doubled. While our code review bot approves a quarter of PRs, our review load still tripled. Human review doesn't scale with agent output.
Rocky's evidence is an internal tool, Nudge, 67k lines of code (40k of those lines are tests), 100% AI-written and reviewed. It isn't production software, but it shows where we're headed. As he put it:
To get there, we'll continue to improve our agent-led code review and validation, and drive the cost of bad deploys to zero.
Code quality will still matter, but I expect it to increasingly mean code that agents can understand, change, and verify reliably. Our definition of quality should change depending on who's doing the work.
In a (near) future where agents build entire applications from scratch, we'll put more weight on describing what a system should do and proving it does.
Even if those predictions take longer than I expect, the validation work pays off today. Have your agent perform a check you'd normally do yourself on your next PR. When it misses something, fix the instructions instead of the PR.
