How to Review AI-Generated Code: The 12-Point Checklist

There are two ways to review an agent's diff. One reads it. The other interrogates it. Only the second one scales — I made that case here, and every project since has confirmed it. What that post lacks is the version for the moment the diff is in front of you, standup starts in five minutes, and nobody has time for philosophy.

This is that version. Twelve points, fixed order, one question each that you put straight to the agent. Keep the order and you're done in five minutes — or you have a reason to send the diff back.

Why a list and not instinct

Instinct works on human code because human code has human mistakes: a forgotten case, a shortcut under deadline pressure, a comment that lies. You know the patterns.

AI code has different patterns. It follows conventions, names things well, comments everything — and is still wrong, in places where a human wouldn't be. An agent doesn't invent syntax errors. It invents a dependency that doesn't exist, or handles the error case by swallowing it. In a diff that doesn't look like a mistake. It looks like diligence.

That's not just my impression. Last year's numbers point the same way:

The last row matters most here: the feeling that a diff is fine is exactly the instrument that fails on AI code. The only defence against "looks diligent" is a procedure that ignores how it looks. So: a list.

The 12 points

For each point: what you check, and the question you type into the chat. The answers arrive in seconds; your job is to verify them, not to produce them.

1. Scope: does the diff do what the ticket says — and nothing else?

Agents love drive-by refactors. A ticket that says "build the export endpoint" comes back with the endpoint, a renamed date helper, and three "tidied" imports in files nobody wanted touched. Each of those is a separate risk without a separate review.

"List every file in the diff and say in one sentence why the ticket needs that change. Anything not strictly required goes into a separate diff."

2. Silent assumptions: which inputs are treated as guaranteed?

The interesting bug doesn't live in the code; it lives in what the code presupposes. null, empty, huge, invalid, duplicate — the agent made an assumption for each and wrote down none of them.

"List every assumption this code makes about inputs and state, and mark the riskiest one. What happens on null, empty, 10,000 entries, and an invalid format?"

3. Error paths: what happens when the call fails?

The most common pattern I see in agent code: try around everything, catch with a log line, return an empty default. Nothing throws, nothing breaks, and the user gets an empty list instead of an error. The bug is invisible until someone asks where the data went. CodeRabbit found error-handling and exception-path gaps nearly twice as often in AI PRs as in human ones.

"Show me every catch block and every fallback value on error. For each one: who learns about the failure, and how?"

4. Tests: does the test prove the behaviour — or just that it runs?

A test that walks the happy path and ends in assertTrue(true) is documentation, not proof. And a test the same agent wrote in the same run checks the assumptions from point 2 against themselves.

Separate step, ideally a second agent that can only read — here's how to build one: "Write tests that prove the required behaviour, including the failure cases and boundaries from points 2 and 3. Don't mock the core logic."

5. Duplicates: does this already exist in the project?

The agent sees the slice it loaded. You see the project. The third implementation of "format an amount" isn't laziness, it's ignorance — and it diverges from the first bug fix onward. GitClear analysed 211 million changed lines: in 2024, copy/pasted code exceeded moved code for the first time, and the share of moved — that is, refactored — lines fell from 25% in 2021 to under 10%.

"Search the project for existing code or patterns for this logic. Show me where. Use them instead of building something new — or explain why not."

6. Layers and patterns: does it fit the system — or just the file?

A controller that queries the database directly because that was shortest in the slice. A second event system next to the one that already exists. A third pattern for something that already has two. Locally right, systemically wrong.

"Which layer does this diff change, and which layers does the new code talk to directly? Name the patterns the project already uses for this kind of task."

7. Dependencies: does every new package exist, is it maintained, are you allowed to use it?

Invented packages have become rarer, not extinct: a USENIX Security 2025 study of 16 code models found an average of 5.2% hallucinated package names for commercial models and 21.7% for open-source ones — over 205,000 distinct names that don't exist. Attackers now register such names on purpose ("slopsquatting"). Outdated packages are more common still. And no agent raises the licence question on its own. Every new line in composer.json, package.json or pom.xml is a contract you're signing right now.

"For every new dependency: registry link, last release date, licence, and why the standard library or an existing dependency isn't enough."

8. Secrets and validation: nothing hardcoded, everything checked at the boundary?

The API key "just for testing" in the diff. The request parameter that flows unchecked into the query. Nobody ever sees either in a demo — and both look harmless in a diff because they're short. In Veracode's tests, models failed to defend against cross-site scripting in 86% of relevant cases; for Java the overall security failure rate was 72%.

"Check this diff for hardcoded secrets, unvalidated inputs, and places where user data reaches queries, shell commands or templates. Give me line numbers and one fix each."

9. Data flow: where does the data land, who can read it, does the migration roll back?

Not a side question for a company under Swiss or EU data rules: which data leaves the system, to which region, into which service. And every migration that only goes forward is a deploy with no way back.

"What data does this code write or send, and where? Is there any call to an external service? Does every migration have a working down()?"

10. Blast radius: what else hangs off the thing being changed?

A shared helper, a config value, a schema — the change is ten lines, the callers are forty. The agent didn't look at the forty because they weren't in its context.

"List every caller of the changed functions, config values and tables. For each: does it behave the same after the diff? What does a rollback look like?"

11. Readability in six months: can I explain every non-trivial line?

Not "I roughly get it", but: could I explain it to a colleague without asking the agent? The agent that wrote it will remember nothing in six months. CodeRabbit saw readability issues more than three times as often in AI PRs — the widest gap of any category. You'll be the only person who knows this code — or not.

"Explain the three least obvious places in the diff in two sentences each. If an explanation needs more, the code is too clever — simplify it."

12. Observability: when it breaks in production, how do I find out?

The failure that comes is not the one you saw in review. Logging with context, a metric, an alert — that's the difference between "a customer reports it after three days" and "we knew after three minutes".

"Which log line or metric tells me this code is failing in production — before a customer notices? Add it if it's missing."

A walk-through on a typical diff

Take a diff of the kind that lands on my desk every week: ticket "CSV export of invoices per customer", Laravel project, the agent delivers 180 lines in one run. Looks clean. The list, at speed:

• Point 1 fires: next to the export controller, the agent renamed a date helper used in eleven places. Out — separate diff.
• Point 2 fires: the export assumes every invoice has a customer. The system has drafts without one. On those the code hits a null access — or worse, see the next point.
• Point 3 fires: the whole export sits inside a try/catch that returns an empty file on any error. The user would have received an empty CSV and gone looking for the mistake in accounting.
• Point 4 fires: one test checking the endpoint returns 200. No test with zero invoices, none with a draft, none with 10,000 rows.
• Point 7 is clean: league/csv, maintained, MIT, sensible.
• Point 10 fires: the export loads every invoice into memory. For the three largest customers that's not 200 rows but 60,000 — and the request timeout is 30 seconds.
• The remaining points are clean.

Five hits, seven minutes. Not one of them would have surfaced by reading, because at every one of those places the code looks tidy. The diff went back to the agent with the five questions and returned forty minutes later, correct — streaming instead of memory, explicit errors, six tests instead of one.

How to roll the list out

Not as a wiki page. As a skill, or as a read-only review agent that asks these twelve questions before a human ever sees the diff — then the answers arrive attached to the diff and you verify instead of collecting. How to build that in twenty minutes is in the subagents guide.

And if you're taking over a codebase where the list was never applied: your problem isn't review, it's triage.

What you can do this week

Take the next agent diff and run the twelve points once, in this order, with a stopwatch — not by feel, because as the METR study showed, the feeling was off by around 40 percentage points there. Count the hits. If it's zero, either the diff is tiny or you rushed points 2 and 3. If it's three or more, you now know where your tech debt comes from — and that it isn't the speed, it's that nobody asked. The one-page checklist sits next to my keyboard; the last 20 percent were always the real job.

Sources

• Stack Overflow: Developer Survey 2025 — AI (2025)
• CodeRabbit: State of AI vs. Human Code Generation Report (17 Dec 2025)
• Veracode: 2025 GenAI Code Security Report (30 Jul 2025)
• METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (10 Jul 2025) — 16 developers, 246 tasks; the authors explicitly caution against generalising.
• GitClear: AI Copilot Code Quality 2025
• Spracklen et al.: We Have a Package for You! Package Hallucinations by Code Generating LLMs, USENIX Security 2025