Two AIs are better than one

The model that wrote my git wrapper was confident it was finished. A second model, from a different company, read the code cold and found three ways around it in one pass. Since then, nothing security-shaped in my setup ships without a second AI reviewing it.

The setup: builder and reviewer #

My home lab runs AI agents overnight, and the guard rails around them are themselves code: a git wrapper that lets agents push without unsandboxed git, an installer that puts those helpers in place as root, a hook that blocks risky commands. If any of that is wrong, the agents can walk straight past it.

That code was planned and written by one model, Claude, working with me. The reviewer was a different model from a different vendor, OpenAI’s Codex, given:

  • the diff and the files it touched, nothing else;
  • a short brief: what the code is meant to prevent, and “assume an attacker who can write any file in the repo”;
  • no access to the builder’s reasoning, plans or chat.

That last point matters most. A reviewer that has read the builder’s explanation starts to see the code the way the builder does, and inherits its blind spots along with its confidence.

What the reviewer caught #

The first version of the git wrapper was a block-list: it knew the dangerous things and refused them. The reviewer found the gaps that design invites, three of them in a single pass.

FindingWhy the builder missed it
A push to a URL with a made-up scheme like evil:// would hand control to whatever “remote helper” git found for itThe wrapper checked hosts, not schemes; it assumed every URL was SSH or HTTPS
A clean or smudge filter named in the repo’s config ran during pullFilters were blocked for some commands, but pull checks files out too
core.worktree could point checkouts at a folder outside the repoThe path checks looked at the repo, not where git would write

The second pass, on the fixed version, found one more: git’s separate per-worktree config file, which the first round of fixes had never looked at. And one finding surprised everyone. When a repo’s config sets remote.<name>.receivepack, git uses that value even when a -c override on the command line names another. The fix was a change of approach: pass every value explicitly, or refuse, and never trust an override.

Together those findings turned the wrapper from a block-list into an allow-list, which is the design it should have had from the start.

What it got wrong, and how to tell #

A reviewer told to assume the worst will find the worst, including things that aren’t there. Across the passes, a fair share of findings didn’t hold up:

  • Already handled elsewhere. A risk flagged in the wrapper that the sandbox or the hook already blocks. Worth a note, not a fix.
  • Out of scope. “An agent could push unwanted content to an allowed branch.” True, and that’s what human review before merging is for; the wrapper was never meant to judge code.
  • Theoretical. An attack needing access the attacker wouldn’t have if they could do it.

The test I use for each finding is simple: can it be written as a failing test? If the reviewer’s claim becomes a test that actually breaks the code, it’s real, and the test stays in the suite forever. If no test can show it, it goes in the notes as a known limit or gets dropped. That rule keeps the argument about evidence rather than about which model sounds more sure of itself.

Running the loop: passes and a stopping rule #

The loop itself is short:

  1. The builder writes the change and its tests.
  2. The reviewer reads the diff cold and lists findings by severity.
  3. Each finding is either turned into a failing test and fixed, recorded as a known limit, or rejected with a reason.
  4. Repeat on the new diff.

The risk is that it never ends. Every pass finds something, because a motivated reviewer can always find something, and the findings get smaller and stranger. So the loop needs a stopping rule, decided before it starts:

  • Stop when a pass finds nothing high or critical, and every remaining finding is recorded as a limit.
  • Stop after three passes regardless, and decide by hand what’s left.
  • Restart from pass one only if a fix changed the design rather than a detail.

My setup took three passes. The third was mostly confirmations and two minor notes, which is exactly what you want the last pass to look like.

Why it works, and when it isn’t worth it #

Two reasons, I think. Different models have different training and different habits, so their blind spots overlap less than two runs of the same model would. And the reviewer has nothing to defend. The builder spent an hour convincing itself the design was right; the reviewer spent none.

It’s the same reason human teams do code review, at a fraction of the cost: a review pass took minutes and cost less than a coffee.

It isn’t worth it everywhere. I skip the second model for:

  • small, reversible changes a test suite already covers;
  • content and copy, where taste matters more than holes;
  • anything where I’m the real reviewer anyway, such as a design I’ll judge by looking at it.

Where it earns its keep is anything that guards something: security helpers, installers, migrations, permissions. If the code’s job is to say no, have someone else try to make it say yes.

The rule I follow now: never let the author grade its own work.