When the tests start defending the bug

01The trap: a baseline wearing a lab coat

The suite has two honest kinds of instrument. A probe measures and prints a number for a human to judge. A gate blocks a change — it fails the build. A gate is only legitimate if its threshold is an invariant (determinism, no crashes, a passable gate) or a doctrine/design oracle (a secured project completes; civilians go home at night) — something true independent of what the sim happens to output today.

The trap is subtle: take a probe, remember its output, and start treating that output as the target. Now any genuine improvement that moves the number looks like a regression. The test is no longer defending correctness — it's defending the status quo, bug and all.

That is exactly what happened to one number in this project: the “~8.58 WIA band” — the average wounded-in-action per deployment that balance.ts reports.

Where 8.58 came from. It is not a casualty rate from a real Korengal deployment. It is the WIA that one balance.ts run printed the day aspect-vegetation shipped at strength 0.05 (terrain.ts:665: “WIA 6.17→8.58”). A sim output, captured and crowned. From then on it was “the historical band.”

02The evidence: it reverted real wins

A curve-fit threshold is harmless until it costs you something. This one has a paper trail — three documented cases where the band, not doctrine, set the outcome:

It treated safer as broken. Issue 026 recorded HEAD's WIA of 6.92 as “below band … watch it.” Read that twice: fewer wounded soldiers was logged as a defect to correct upward. The harness practice was pushing the game to be more lethal to its own side to match a number with no real-world meaning.
It set the stopping point for a realism win. Issue 027 made the trail network denser so patrols ride the footpaths — “patrols riding the network IS the realism.” WIA rose to 12.58. The work was then narrowed not until it was right, but until “WIA 8.42, inside the ~8.58 historical band.”
It reverted features outright. Open-ground cover behind rocks was cut for moving WIA +89% (issue 020); COP high-value-target dispersion and threat-weighting were reverted for +1.00 / +0.50 KIA (issue 022).

None of this was carelessness — every one of those calls was made carefully, with A/B runs and restraint notes. The flaw was upstream of the diligence: a missing principle about what a casualty number is allowed to be.

The deeper irony. The design's whole soul is “you can win every firefight and still lose the valley.” Yet the firefight (tactical casualties) was the thing under standing protection, while the actual win condition — the COIN strategy layer (village attitudes, projects, the elders' trust) — had a probe but no gate at all. The routine verification gradient pointed exactly away from the soul of the game.

03It was policing finer than its own noise

There is a second, quieter problem. balance.ts is deterministic per seed-prefix, so re-running bal gives a byte-identical number — which looks like precision. But each prefix is just one draw of 12 deployments. Run a different prefix and you get a different sample of the same distribution. The real question is: how much does the 12-deployment mean wobble between independent draws?

We measured it — five independent prefixes, 12 deployments × 50 game-minutes each:

≈2.5

σ of mean WIA between draws

2.67–9.42

WIA range across 4 prefixes

±1.0–1.5

deltas read as “signal” before

Four draws of the identical gate config — same code, same settings, different seeds — returned 7.08, 2.67, 5.00 and 9.42 WIA. The band was being defended at a resolution finer than the harness's own sampling noise: a WIA move of 1.0–1.5 — the size that triggered “below band,” that narrowed issue 027 — sits well inside a single standard deviation (≈2.5), and the full range spans more than 4× that delta. Issue 026 had even flagged “balance.ts run-to-run σ floor” as open debt. It had simply never been measured. Now it is, and it prints on every run.

04The fix is a rule, not a patch

You cannot fix a missing principle with a code change alone — the next session would re-derive the same trap. So the deliverable is a charter (docs/wiki/Harnesses.md), enforced by CLAUDE.md, with one governing law:

A gate's threshold may never be the sim's own past output.

And one re-ordering of priorities, drawn straight from the design's soul:

The win condition gets the gate; the firefight gets a probe. The standing check that blocks a change is now the COIN discriminator — does playing counter-insurgency well beat playing it badly? Casualties are a diagnostic you read and explain, never a target you engineer back.

05What actually changed

Change	What it does
`docs/wiki/Harnesses.md` (new)	The charter: gate-vs-probe law, the anti-overfit rule, the win-condition-first principle, the coverage-gap list, an add-a-harness checklist.
`campaign-loop.ts` → standing gate	The COIN win-condition instrument now `exit(1)`s if the strategy layer goes inert (score stops discriminating, attitude stops moving, projects stop completing). Every assertion is a design oracle, not a fitted number. Added to the standing checks.
`balance.ts` → re-anchored	Header + output now state plainly: casualties are a DIAGNOSTIC, there is no WIA band to defend. It prints the measured noise floor so no future delta is ever read finer than the noise.
`CLAUDE.md`	Standing checks updated; the harness law installed where every session reads it first.

The 84-file suite itself came out of the audit healthy — most files are clean probes that print numbers and assert nothing, and the genuine invariants (determinism, no-NaN, no-stall, the material-hash pin) are exactly the kind of thing a gate should guard. The blocker was never a file. It was a habit. The habit now has a rule against it.

Residual, named (the charter's own rule). The COIN gate's principle and exit-code are in, but its practical config is not yet calibrated — the 3-seed default is too slow to finish routinely (~40 min; one weapons-free seed is 9–15 min), and the fast single-seed run under-discriminates at the 8-day minimum horizon. A gate you have not watched pass is not a green gate — so this one is logged as installed-but-uncalibrated, with a fast, deterministic, pre-vetted config as the named follow-on. That honesty is the whole point: the same rule that retired the WIA band also forbids us from claiming this gate is green before it is.