The BFD
Gauntlet.
Code quality at agent pace — a line of stations between every commit and production. Each station measures one number. Any station can stop the line.
We need a metric of quality —
not an experiment on quality.
The shape of the problem
Cursor, Claude Code, and Codex write the bulk of the code. We review, shape, gate, ship. Same five people both years — Keith, Reggie, Eli, Bobby, Matias — shipping 3,635 commits across 25 repos in the first four months of 2026. Two years prior, the same five shipped 255.
The toolset has to do the supervising. Every repo on a different testing config is the velocity penalty we cannot afford.
This deck is the answer to that conversation.
One pattern, applied across every repo, that turns "is this any good?" from a vibe-check into a number we can read.
The testing pyramid was built for human pace.
Now code arrives at machine pace — the model breaks.
Pre-agent · craft pace
- ~2 commits per dev per week, 1 active repo.
- The same humans wrote and reviewed code. Taste caught bad patterns.
- The pyramid was a sketch on the wall — describing what a healthy test suite looked like — and humans self-enforced because volume was small enough to.
Post-agent · machine pace
- ~30 commits per dev per week, 25 active repos. Cursor, Claude Code, and Codex write the bulk of every diff.
- No human can read every line at machine pace. Taste doesn't scale.
- The pyramid still describes a healthy test suite — it says nothing about whether this commit can ship.
What the pyramid alone doesn't catch
From this morning's rollout across 10 repos — none of these are tests:
- 5 missing
waitUntil: "commit"inpage.waitForURL— would have hung E2E for 30 s each - 14 high-severity npm advisories on bfd-front-door upstream
- 565 fallow clone groups + 990 health findings on bfd-platform — structural drift
- "Looser" flagged by alex as a homophone of a slur
- 1,555 frozen work orders of CSS + structural decay
- Cookies set
httpOnly: falseon NCEE staging — only surfaced on invalid-token
The pyramid is right. It's just not enough.
Reggie's NCEE testing deck still covers the test-failure class. This deck is the gauntlet that wraps it — the static-analysis, build-artifact, supply-chain, and observability stations every commit walks past on its way to ship.
The BFD Gauntlet.
Every commit walks the gauntlet on its way to production. Each station measures one number. Any station can stop the line — and nothing on the line ever weakens without a signed reason.
spec ≤ 512,884 b
"wallace 248833 → 248861 after react 19.2.5". Public reason or it doesn't merge.
|| true. No --no-verify. No "warn-only" tier. Half-on stations are the pattern the next agent copies — either the station fails on regression, or it isn't a station.
The layer set is identical across all 11 repos.
What differs is which layers are filled.
One unfilled fleet-wide: L15 test authorship — the AI maintains tests under our standards, but nobody is actively growing the suite. Documented as the second open hire in every per-repo SKILL.
Each station ratchets — tighter is free, loosening is ceremony.
|| true. No informational tier. No --no-verify. If a step is worth running it's worth failing on. The same gates catch compliance regressions — secrets sweep, audit baseline, enforce_admins. Also our Vanta substrate.Every station is just a JSON file the build refuses to regress.
// .wallace/tenant.json
{
"totalSize": 248833,
"selectorCount": 4129,
"specificity": {
"max": [0, 4, 4, 0]
},
"rules": {
"empty": { "total": 0 },
"important": { "total": 0 }
}
}
Today's measured value, written verbatim. Zero headroom.
// scripts/wallace/check.mjs
const baseline = readJson(BASELINE);
const measured = await analyzeCss(BUILT);
for (const [k, v] of entries(baseline)) {
if (measured[k] > v) {
fail(`${k}: ${measured[k]} > ${v}`);
}
}
// Exits non-zero if any metric
// regressed. No informational tier.
Any increase fails the gate. The wheel only turns one direction.
$ git log -1 .wallace/tenant.json chore(wallace): bump totalSize 248833 → 248861 (+28 bytes) react 19.2.5 ships ~28 bytes of new createRoot scaffolding we can't drop. Verified bundle diff in PR #4129.
Public reason or it doesn't merge. Loosening is ceremony.
This pattern is the whole gauntlet. Every station — built-CSS bytes (Wallace), JS bundle bytes, npm-audit count, fallow clone groups, vitest coverage floor, gate wall-time — is the same three-step template: freeze a number, gate on it, ceremony to raise it. aatm-brain has 9 stations running today.
Vendor MCPs teach the AI how to use a library.
The BFD Gauntlet enforces our standards. Two different jobs.
Sentry · Convex · Clerk · etc.
Provide the agent with up-to-date docs, idiomatic usage, working examples for that vendor's product. Soft guidance — the agent reads them, picks them up, sometimes ignores them.
- "How do I set up Clerk middleware?"
- "What's the right way to query Convex from a server action?"
- "What Sentry tags should I attach to this error?"
Output: better-formed code that uses the vendor correctly. Mode: guidance, advisory.
Black Flag standards · gates that fail builds.
Enforce our bytes-per-build budget, our coverage floor, our npm-audit baseline, our commit-message format. Hard stops — if the threshold regresses, the build is red.
- "This PR adds 28 KB to the CSS bundle — fails Wallace."
- "This change drops coverage from 45 % to 44.7 % — fails L13."
- "This commit raises the wallace threshold without a public reason — fails commit-msg."
Output: code that complies with company-wide quality contracts. Mode: enforcement, gating.
Both, not either. They live at different layers.
Vendor MCPs help the agent write better code. The gauntlet decides whether that code ships. We use vendor MCPs in every repo. We also walk every repo through the same gauntlet.
Each gate runs at the earliest cheap phase.
Defense in depth: same gate, multiple phases.
| Gate | IDEms · on save | Pre-commit~3 s · on commit | Pre-push~70 s · on push | CI Test Gate~3 min · parallel | Crondaily · persistent PR |
|---|---|---|---|---|---|
| L1 · ESLint + format | ● | ● | ● | ● | |
| L2 · TypeScript | ● | ● | ● | ||
| L7 · e2e preflight | ● | ● | |||
| L6 · Fallow preflight (diff-scoped) | ● | ● | |||
| Branch protection / commitlint / secrets | ● | ● | ● | ||
| L12 · npm audit | ● | ● | ● | ||
| A4 · Dependabot grouped weekly | ● | ● | ● | ||
| L13 · Coverage floor | ● | ● | |||
| L8 · Wallace built CSS | if CSS | ● | ● | ||
| L5 · Fallow project graph | ● | ● | ● | ||
| L9 · Bundle byte budget | ● | ● | |||
| CI Test Gate orchestration | ● | ||||
| A1 · Wall-time budget wrapper | ● | ● | |||
| L14 · Visual regression (PNG diff) | ● | ||||
| v3.2 · PostHog telemetry gate | v3.2 | ||||
| A3 · Daily report PR | ● |
The cheapest phase wins. Catch at IDE → free. Catch at CI → minutes. Catch in production → a deploy and an apology. Most gates fire at multiple phases on purpose: the same lint runs locally and in CI so a fast local loop never bypasses the slow remote one.
The directive was thin because the standards are thick.
Reggie was already running the playbook a week earlier.
What the agent ran on
"Execute org-wide v3.1 rollout per ~/.claude/plans/put-together-a-plan- reactive-starfish.md. Authorship gate: skip repos not primarily authored by Keith. DO NOT STOP UNTIL ALL ITERATIONS COMPLETE. Every repo finished with our changes working and running on main."
No per-repo instructions. The agent read the global skill, the rollout plan, the per-repo INDEX, and the standards. It picked the iteration order, opened PRs, fixed CI failures, merged 10 PRs in ~6 hours of wall-clock. 9 deployed clean to production.
The proof of mechanism — week of 04-20
The mechanism worked before we had a name for it. Reggie hand-rolled custom ESLint plugins, surfaced them via the LSP, and watched the agent self-correct. The agent treats lints as gates and rewrites until they pass.
v3.1 is the same pattern, made repo-canonical: standards surfaced as lints get respected; rules in a doc get ignored.
Thin prompt + thick standards ≫ thick prompt + thin standards.
Every minute in the SKILL pays back across every agent that comes after it.
aatm-brain · most layers wired before;
now backed by amendments + a real catch.
What we asked the agent to do
Adopt v3.1 standard evolution onto a repo that already had 11 of 14 layers wired. Add the new amendments — wall-time budget (A1), SHA-cached pre-push (A1b), JSX prose (L11), visual regression (L14, A2), daily report (A3) — and document the rest as honest gaps in the per-repo SKILL.
What actually happened
- The new
e2e:preflightstatic check caught 5 realpage.waitForURLcalls missingwaitUntil: "commit"— bugs that would have hung E2E for 30 s each in CI. Static analysis cheaper than runtime hang every time. - Pre-push had to be force-pushed with
AATM_PREPUSH_FORCE=1because the audit baseline holds 2 high CVEs out of scope for this PR. The gap is documented in the PR description and the per-repo SKILL. Honest gap, not skipped gate. - Coverage at 45 % was frozen as the new floor — gate against regression. Growing the suite is a separate workstream.
- Theme tokens + WCAG contrast deferred — biggest standing investment for v3.2.
The L7 preflight paid for itself in one PR.
5 × 30-second hangs prevented = ~2.5 minutes per CI run × every PR forever. Static check < runtime hang, always.
| Layer | Before | After |
|---|---|---|
| L1 ESLint | ✓ | ✓ held |
| L2 tsc | ✓ | ✓ held |
| L5 Fallow | ✓ | ✓ held |
| L6 Fallow preflight | ✗ | ✓ gained |
| L7 e2e preflight | ✗ | ✓ gained · caught 5 |
| L8 Wallace CSS | ✓ | ✓ held |
| L9 Bundle budget | ✓ | ✓ held |
| L10 md prose (alex) | ✓ | ✓ held |
| L11 JSX prose | ✗ | ✓ gained |
| L12 npm audit | ✗ baseline | ✗ deferred · documented |
| L13 coverage floor | ✓ 45% | ✓ frozen |
| L14 visual regression | ✗ | ✓ gained |
| A1 budget wrapper | ✓ | ✓ held |
| A1b SHA-cached pre-push | ✓ | ✓ held |
| A3 daily report cron | ✗ | ✓ gained |
| L3/L4 tokens + contrast | ✗ | ✗ documented |
5 layers gained · 9 held at the freeze · 2 gaps explicitly deferred with a paper trail.
bfd-platform · we baselined 1,555 work orders
rather than pretend we'd cleaned them.
What we asked the agent to do
Add the new amendments to the most-mature repo in the convoy — wall-time budget, SHA-cached pre-push, visual regression, daily report. Refresh the per-repo SKILL to v3.1 wording. Land the iteration as the reference for the other 9 PRs to cite.
What actually happened
- Baselined 565 fallow clone groups + 990 health findings — accumulated structural debt the new layer surfaced. Frozen as the v3.1 baseline so it can't get worse. Burndown is a separate workstream.
- Test Gate flaked twice. CI Test Gate failed with 2 E2E suites timing out (
client-feedback page loads,smoke - global agent); succeeded on retry #3. Not deterministically broken — flaky. v3.1 made the flake visible by failing fast. v3.1 didn't fix it. v3.2 work: quarantine flake-prone tests or fix them. Don't normalize retries. - Wall-time wrapper (A1) measured the full pipeline at 67.4 s under the 90 s ceiling. 22 s headroom.
The honest tradeoff
The stricter pre-commit + gauntlet pattern pushed Bobby off the blueprint repo for two days while we tuned which warnings get treated as hard stops vs. info. Stations that catch quality regressions only matter if the team can still work — calibration is its own line item.
The honest baseline beats the fake clean slate.
Frozen rot is still rot — but it's visible rot, with a paper trail. Next PR can't make it worse. Burndown is a real workstream, not a someday-refactor.
| Layer | Before | After |
|---|---|---|
| L1 ESLint | ✓ | ✓ held |
| L2 tsc | ✓ | ✓ held |
| L3 theme tokens | ✓ | ✓ held |
| L4 WCAG contrast | ✓ | ✓ held |
| L5 Fallow | ✓ | ✓ baselined 565+990 |
| L8 Wallace CSS | ✓ | ✓ held |
| L9 Bundle budget | ✓ | ✓ held |
| L11 JSX prose | ✓ | ✓ caught "Looser" |
| L12 npm audit | ✓ 1 high | ✗ baseline · burndown |
| L13 coverage floor | ✓ 38% | ✓ frozen at floor |
| L14 visual regression | ✓ | ✓ held |
| A1 budget wrapper | ✗ | ✓ gained · 67.4 s/90 s |
| A1b SHA-cached pre-push | ✗ | ✓ gained |
| A3 daily report cron | ✗ | ✓ gained |
| CI Test Gate drill | flake-prone | flake-prone · v3.2 fix |
| Carpenter burndown | ~unknown | 1,555 · baselined |
3 layers gained · 11 held · 2 visible gaps with named workstreams.
ncee, front-door, mcp, playbook, style-guide, cli, widget, muster.
Min-viable. Honest. Documented.
What "min-viable" actually shipped
Eight repos started with most layers unfilled. We didn't pretend to fill them. The iteration PR landed three things on every repo — and named the rest as gaps in the per-repo SKILL.
- Wall-time budget wrapper (A1) — every repo now has a measured ceiling. Drift visible the moment it appears.
- Daily report cron (A3) — every repo has a persistent-PR log. The day the org-perm flips, every cron starts firing.
- Per-repo SKILL with explicit gaps — ncee documents 8 unfilled layers; bfd-front-door documents 14+ upstream Astro CVEs; bfd-cli documents node:test → vitest as deferred.
The signal isn't the layers we filled. It's the layers we named as gaps. A documented gap is a hire we can plan; a hidden gap is a fire we'll fight blind.
9 of 10 repos deployed clean. The 10th flaked twice and passed on retry.
Production deploys went green for 9. bfd-platform's E2E flake unmasked an old retry-tolerant setup we'd never properly seen — that's a v3.2 fix, not a v3.1 regression.
Day-savers we'll write down for the next agent
Daily-report cron 401'd at PR creation across all 10 repos. Cause: Allow GitHub Actions to create and approve pull requests was off org-wide. Repo-level Actions perms can't override it. ~30 min to find. Now step 0 of any cron-PR workflow.
cfut_ vs cfat_Org's ~/.config/bfd/cloudflare.env held a cfut_ wrangler-OAuth token, not a cfat_ API token. Pages-create + DNS need API tokens; OAuth fails silently mid-flow. Token-type check is now the first line of every CF-touching script.
Token system + contrast gate wired only on bfd-platform. Eight UI-bearing repos ship hand-stitched canvas — every refactor risks brand drift and a11y regression. Single biggest hire on the v3.2 board. One repo per quarter cadence.
Don't hire five juniors. Hire one technical-QA SME.
Write the standards. Let the agent maintain the code under them.
One red-team / technical-QA SME.
Senior, not junior. Goes to any repo, names the missing standard, writes the gate or files the work order. Owns Vanta + compliance alongside quality. One head, not five.
The work that used to need a junior — bounded enough for the agent now.
Maintains tests, fixes lint, writes coverage, tightens stations — under the SME's standards. "AI is going to do a 10× better job of writing and maintaining those tests than an intern will." — Reggie
Pick one number your repo can measure. Freeze it. Make the build fail on regression.
Every station in the gauntlet is one instance of that. CSS bytes, bundle bytes, audit count, coverage floor, gate wall-time. Start with the one drifting in your repo this week.
The gauntlet protects the inheritor, not the author.
"I don't code anymore, so I'm not scared… eventually I learn." — Matias, 2026-04-20. The point of every station is that the next person — agent, junior, returning teammate — can't accidentally make it worse.
The team's headline: a metric of quality, not an experiment on quality. One SME unlocked. Same standards across every repo. The toolset doing the supervising. That's how five people ship like fifty without hating ourselves.
From one station to the full BFD Gauntlet.
Same five steps that ran the rollout.
Read the standards
~/.claude/skills/code-quality-setup/SKILL.md — symlinked into Claude Code, Codex, and Cursor. Every agent reads it before generating code in the new repo.
Per-repo SKILL
Drop the per-repo SKILL template at .cursor/skills/code-quality/SKILL.md. Fill in which layers are filled, pending, or N/A. The honest gap doc is the deliverable.
Hire the layers
In priority order: L1+L2 → A1 budget wrapper → A3 daily report → L6/L7 preflights → L5 fallow → L13 coverage → L8/L9 bytes → L12 audit → L10/L11 prose → L3/L4 tokens+contrast → L14 visual regression.
Add to the index
Update ~/.claude/skills/code-quality-setup/per-repo/INDEX.md — name, cron hour, filled vs. pending layers. The index is the org-wide adoption roster.
Ship
Open the iter PR. Each layer commit gets its own message. Gaps land documented, not pretended-shipped. PR description names every filled layer + every documented gap.
A metric of quality, not an experiment on quality.
One SME, five engineers, eleven repos, one gauntlet.
10 PRs merged · 9 production deploys clean · 7 real issues caught mid-flight · gaps documented, not hidden.