TDD in the Agent Loop: Mirror or Actual Check?

A test written by the same agent that wrote the implementation isn't a check. It's a mirror. Once the same model sits on both sides of the assertion, most of what teams call test driven development (writing the test before the code, watching it fail, then making it pass) stops producing signal.

The question is which parts still mean something. The answer is narrower than most review checklists assume, and it comes with a concrete gate you can add to CI this week.

The tautology problem

A test derived from the same logic path as the implementation it checks can never fail when that implementation is wrong. Not "rarely fails." Never. The failure mode is structural, not a matter of the agent being sloppy.

Thoughtworks engineer Birgitta Böckeler ran the ceremony properly and watched it break down mechanically. In her experiment, agents skipped the red step, faked the failure, or implemented ahead of the test so it passed on the first run.

Even when it does, the red step proves less than it looks like. A red test only means something if a human or an independent process confirms why it went red. An agent confirming its own red test tells you the agent ran the suite and saw a failure. It does not tell you the failure was for the stated reason. A wrong import path is red. A misconfigured fixture is red. Both satisfy the letter of the ceremony while proving nothing.

Call the mechanical half test-first ordering: the assertion exists before the implementation does. That's the part that survives agent authorship most easily, and the part that proves the least on its own.

Mirror versus check, in code

Here's the difference in under 20 lines. A shipping fee function with a deliberate bug: the free-shipping threshold is > when the spec says orders of exactly 50 ship free.

export function shippingFee(subtotal: number): number {
  if (subtotal > 50) return 0;      // spec says >= 50
  if (subtotal > 20) return 4.99;
  return 7.99;
}

// Mirror: reads the branches back at you. Passes with the bug intact.
test("applies the correct tier", () => {
  expect(shippingFee(60)).toBe(0);
  expect(shippingFee(30)).toBe(4.99);
  expect(shippingFee(10)).toBe(7.99);
});

// Check: written against the spec, not the code. Fails.
test("orders of exactly 50 ship free", () => {
  expect(shippingFee(50)).toBe(0);
});

The first test has full line coverage of the function. It picks one value per branch, which is exactly what you get when the test is generated from the implementation's shape rather than from a contract. Boundaries are where the bugs live, and a mirror never looks at boundaries because the implementation never mentioned them.

Worth naming what usually happens in practice: the agent writes the whole test file and the whole implementation in one pass, then runs the suite to see green. That's development with generated tests, not test-driven development. The tests are downstream artifacts of the code, and they inherit its assumptions.

The invariant that actually survives

Not test-first ordering. Not red, green, refactor discipline. The load-bearing invariant is who owns the assertion.

A test functions as a check only when its author didn't also author the code it checks. That author can be a human, a separate agent session working from a spec it did not write, or a mutation-testing process scoring the suite from outside. What matters is independence, not sequence.

This invariant predates agents entirely. Agents just expose how many teams were already faking it, with human-written tests that quietly mirrored the implementation because the same developer wrote both in the same hour.

Böckeler's mutation-score comparison between TDD and non-TDD runs found no meaningful difference, which undercuts the usual defense that test-first ordering improves regression quality by itself. Take it as a signal, not a verdict: the code-quality judging in that experiment was done by another model (Opus), a real methodology gap worth weighing against the result.

Wiring the tripwire into CI

Mutation testing is the one check that doesn't care who wrote the test. It mutates the implementation (flips > to >=, swaps + for -, drops a return) and asks whether the suite notices. A mirror test with 100 percent line coverage and a near-zero mutation score gets caught regardless of authorship. That combination is not a contradiction. It's the signature.

Full-repo mutation runs are too slow to gate a pull request on. Scope the run to changed files, which Stryker supports through its incremental mode and the --mutate file filter.

name: mutation
on: pull_request

jobs:
  stryker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci

      - name: Mutate changed source files only
        run: |
          FILES=$(git diff --name-only --diff-filter=ACM \
            origin/${{ github.base_ref }}...HEAD \
            -- 'src/**/*.ts' ':!*.test.ts')
          if [ -z "$FILES" ]; then echo "no source changes"; exit 0; fi
          npx stryker run --incremental --force --mutate $FILES

$FILES is deliberately unquoted so each changed path arrives as its own argument to --mutate. --force reruns every mutant in that scope even when an incremental report already exists, which is what you want on a pull request: the changed files get a fresh score, everything else is reused from the incremental file.

Set the failure threshold in stryker.conf.json rather than in the workflow, so it lives with the project:

{
  "packageManager": "npm",
  "testRunner": "vitest",
  "reporters": ["clear-text", "progress"],
  "thresholds": { "high": 80, "low": 65, "break": 60 }
}

break is the one that matters: below that mutation score, Stryker exits non-zero and the job fails. Start it at your current score minus a few points, then ratchet up. Gate on this, not on coverage percentage.

A per-PR checklist

Four questions for any pull request containing agent-written tests. Paste them into your review template.

1. Who authored the spec the assertion is checked against? If the answer is "the same session that wrote the code," there is no oracle, only a mirror.
2. Was the test confirmed red for the stated reason, before any implementation existed? Not "did a red step happen." A wrong import is also red.
3. Does the test survive a mutation run on the file it covers? If the mutation score is near zero at high coverage, the assertions are decorative.
4. Is the assertion phrased against observable behavior or against the implementation's internal shape? Branch-shaped assertions and boundary-free inputs are the tell.

Adding a second agent as an independent tester does not remove the tautology by itself. It moves it up one level, unless the acceptance criteria that second agent grades against are human sourced. The fix lives in who authored the oracle, not in how many agents are in the loop.

Test-first ordering was never the load-bearing part of TDD, independent authorship of the assertion was, and mutation testing is the only cheap way to verify that independence after the fact. Add the changed-files Stryker job to one repo Monday, set break just under your current score, and stop asking your agents whether they followed red, green, refactor.