Code Review Can't Scale With Agent Output. Score Blast Radius Instead

Human ability to spot defects in a code diff falls off sharply past roughly 400 changed lines, and a single coding agent prompt routinely produces more than that in one pass. Line-by-line review was never designed for this volume, and adding reviewers doesn't fix it.

The fix isn't abandoning review. It's deciding, deterministically and before a human sees anything, how much scrutiny each pull request deserves. This gives you the scoring mechanism, the tier table, and the CI workflow that enforces both.

Why uniform review breaks first, not review itself

That 400-line figure is a widely cited data point (a SmartBear analysis of a Cisco team, surfaced again in Bryan Finster's piece on AI and code review), not something measured here. Read it carefully: it says review quality degrades with volume. It says nothing about how severe a change is. A 900-line formatting pass and a 40-line change to refund logic sit at opposite ends of the danger scale, and diff size can't tell them apart.

The design bug is uniform scrutiny. Your pipeline currently asks the same thing of a typo fix and a payments-path change: one human, reading everything, at the same depth. Once agents multiply the diff count, that policy converts into a queue.

Tiering by risk already works in production. Cloudflare runs three review tiers with published cost numbers, roughly $0.20 for a trivial change versus $1.68 for a full review. Moderne applies the same principle more narrowly, mapping the reach of dependency upgrades with no model making a judgment call at all.

There's a competing school worth naming: review the spec, not the diff. Augment Code and Latent.Space both argue attention belongs upstream, at the intent, before the agent writes anything. That's correct and insufficient. Specs are rarely precise enough to guarantee the diff faithfully implements them, and neither source publishes a downstream check for drift between the spec and what shipped. They relocate the problem. They don't give you something to put in CI on Monday.

Score blast radius, not risk

A risk score answers "should I worry about this." That still needs a human to interpret the number. Blast radius answers a different question: how far does the damage reach if this change is wrong. NOFire AI's glossary draws that distinction cleanly, and it matters because the second question has an answer a policy engine can enforce as a hard threshold. Treat it as a framing tool rather than a proven result. The gate below is the actual proof.

You can approximate blast radius cheaply. No dependency graph, no call-graph analysis. Two classes of static signal get you most of the way:

• Path signals. Does the diff touch auth/, payments/, migrations/, or infrastructure config? Those directories carry known reach.
• Shape signals. How many files, how many lines, and is the change mechanical or semantic?

Keep those inputs separate. Size and severity are correlated, not identical, and collapsing them into one number is how you end up blocking a rename and waving through a schema migration.

Deterministic signals also beat asking a model to estimate risk. No extra inference call in the review loop, no added latency, and every decision is reproducible after the fact. When someone asks why a pull request auto-merged six months ago, you point at a ruleset instead of a prompt.

The scoring script and the tier it produces

The script reads changed paths and diff stats from git, scores them, and prints a tier. Drop it at .ci/blast_radius.py.

#!/usr/bin/env python3
import re, subprocess, sys

BASE = sys.argv[1] if len(sys.argv) > 1 else "origin/main"

HIGH_REACH = [
    (r"^(auth|payments|billing)/", 5),
    (r"migrations/.*\.(sql|py)$", 5),
    (r"^(infra|terraform|\.github/workflows)/", 4),
    (r"^src/utils/format\.py$", 4),   # add files that bite you
    (r"(Dockerfile|docker-compose\.ya?ml)$", 3),
]
BORING = re.compile(r"(\.lock|\.snap|\.min\.js|_pb2\.py|\.svg)$")

def sh(*args):
    return subprocess.run(["git", *args], capture_output=True,
                          text=True, check=True).stdout.strip()

rng = f"{BASE}...HEAD"
files = [f for f in sh("diff", "--name-only", rng).splitlines() if f]
renamed = len(sh("diff", "-M", "--diff-filter=R", "--name-only",
                 rng).splitlines())
churn = sum(int(n) for n in re.findall(
    r"^(\d+)\t\d+\t", sh("diff", "--numstat", rng), re.M))

semantic = [f for f in files if not BORING.search(f)]
score = max((w for f in semantic for p, w in HIGH_REACH
             if re.search(p, f)), default=0)
mechanical = len(semantic) == 0 or renamed >= len(files) * 0.6
if not mechanical:
    score += 1 if churn > 400 else 0
    score += 1 if len(semantic) > 15 else 0

tier = "auto" if score <= 1 else "flag" if score <= 3 else "block"
print(f"score={score} tier={tier} files={len(files)} "
      f"churn={churn} mechanical={mechanical}")
sys.exit(1 if tier == "block" else 0)

Three things to notice. The path score is a max, not a sum, so touching four payments files isn't four times worse than touching one. Lockfiles and generated output get stripped before scoring. And the mechanical check suppresses size penalties on rename-heavy or fully generated diffs, which is what stops a repo-wide formatting pass from demanding a human reader.

The tier maps straight to policy:

Set the block threshold tighter than feels comfortable for the first two weeks, then measure how often block fired on something genuinely trivial before loosening it.

Wiring the gate in without creating a new bottleneck

The workflow runs the script on every pull request and becomes a required status check in branch protection.

name: blast-radius
on:
  pull_request:
    types: [opened, synchronize, reopened, labeled, unlabeled]

jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
        with:
          fetch-depth: 0

      - name: Break glass
        id: bypass
        if: contains(github.event.pull_request.labels.*.name, 'break-glass')
        run: |
          echo "::warning::Gate bypassed by ${{ github.actor }}"
          echo "bypassed=true" >> "$GITHUB_OUTPUT"

      - name: Score diff
        if: steps.bypass.outputs.bypassed != 'true'
        run: |
          git fetch origin ${{ github.base_ref }}
          python3 .ci/blast_radius.py origin/${{ github.base_ref }} \
            | tee -a "$GITHUB_STEP_SUMMARY"

The break-glass label is the whole reason this survives contact with an incident. A hard gate with no override becomes the next bottleneck the first time someone needs a hotfix at 02:00, and the bypass people reach for then is editing the workflow file, which leaves no audit trail. A label leaves one: who applied it, when, on which pull request. Restrict who can apply it, alert on every use, and review the uses weekly.

This is a scoped-down version of what Cloudflare runs. Deterministic scoring only, no model-driven sub-reviewers per tier. It makes the same point with less machinery.

What this replaces, and what it doesn't

Rootly documented teams declaring "pull request review bankruptcy," merging agent output without reading it. That isn't a different strategy from tiering. It's tiering with every diff silently assigned to the lowest risk tier, never written down and never enforced. Once you name it that way, the question stops being whether to tier and becomes which diffs you're willing to put in tier zero on purpose.

Spec review is the same shape of answer at a different layer. Reviewing intent upstream reduces how much wrong code gets generated. It doesn't verify that the shipped diff matches the intent. Run both: spec review to shape what the agent builds, a blast-radius gate to decide who reads the result.

Tier by blast radius using static signals your CI can compute in a second, not uniform scrutiny and not a model's guess at risk. Copy the script into .ci/, run it in report-only mode against last month's merged pull requests, and look at how many would have landed in block before you make the check required.