gauntlet

I asked for this the night we were testing Fantopy's World Cup knockout. Play the game as a manager from the round of 32, and at the same time see every machine that each click sets off.

The two views need each other. The journey decides which machinery gets hit, in what order, and from what state. The machinery is then tested from the state the journey actually left behind, with nothing mocked. A green click on top of an unverified engine proves very little, and a verified engine reached from a fake state proves less.

The kitchen picture became the rules the run asserts. Each one points at a place where ordinary testing looks the wrong way.

Sweeping every station deeply would take forever and mostly confirm things that were fine. What I asked for was “either a surface area or a deep sweep”, and the skill ended up doing both, in that order. Every station the journey touches gets a quick real pass and a score for how much rests on it and what smells off. The ranked list decides where the depth goes.

Rare failures only show up at volume. The World Cup sweep drove 8 managers through 5 round multipliers, advancement bonuses, transfers and penalty shootouts, then re-scored 1,896 predictions from 63 users. The checkout example in the skill hunts the same thing in a different world: a double tap on a slow spinner that charges a card twice.

A busy station looks fine from outside. The status code is 200 and the log says done. So gauntlet works out the right answer on its own and compares, and every verdict says which check it used and how strong that check was. “Verified by invariants only” is a useful, honest answer. A quiet green tick is the one failure the skill treats as unforgivable.

The rig that drives a simulation is software too, and it breaks. Before anything gets called a bug, the run has to prove the problem is in the system and not in the rig. The scoring shadow on the World Cup engine is the clearest case I have. Its first result looked like a disaster.

The second correction is the one worth noticing. The shadow disagreed with the engine on a real rule, the rule was checked against the written spec, and the engine was right. A second implementation that never disagrees with the first isn't checking much.

The night of 30 June, locally. The app ran in its manual simulation mode with a snapshot taken first. A small driver pushed the real lifecycle forward round by round, from the round of 32 to the final: produce match stats, eliminate the losers, resolve the bracket, score. The engine's own numbers were never taken on trust.

The surface pass crashed at the scoring station on a missing database migration, a hand-off that normal use would have hit without anyone noticing. With it fixed, 40 of 40 manager-rounds reconciled and all 1,896 predictions re-scored cleanly. The diary caught what the numbers couldn't: a real user would just have seen scores quietly stop updating. The fix for that, a loud alert when the scorer fails, came from the diary.

The run was honest about its gap too. The match stats were synthetic, so the path that pulls real stats from the data provider was never exercised.

The following night, on the shared develop server. Everything worked locally, but something was wrong on develop, and I had noticed players from Qatar and Uzbekistan still in squads after both teams went out in the group stage. I asked for a gauntlet on the whole fantasy flow there. It sent six stations deep and came back with:

One more turned out to be the harness: two local test agents both named “Dev”. The fixes shipped together as PR #1183, one of eleven pull requests that night. When production picked them up, the first cron tick flagged 792 eliminated players there and the count dropped to zero without anyone touching the database.

Rifter's scanner identifies the exact printing of a trading card from a phone camera. On 11 August the gauntlet ran it through 34 real photos from the internet: foil, sleeves, glare, toploaders, perspective, cards clipped at the edge, distant cards and compressed images. The diary from that run has the moment I care about most:

Every gate passed, and kept passing when re-verified on 13 August: 27 of 34 accepted, zero false accepts, a 900 millisecond decision time at the 95th percentile. On 16 August, a refresh of the project map asked what those green gates actually covered. Every test photo carried a box measured by hand, and the card was cropped to it before recognition ran. The app does something cruder: it crops a fixed rectangle in the middle of the frame and hopes the card is inside. Run the same 34 photos on the full frame and the picture changes.

Live telemetry from 163 real scans told the same story from the other side: about 28 seconds per card at the 95th percentile. That clock also includes a hand reaching for the next card, so it isn't recognition speed, but it made clear the 900 milliseconds described something much narrower than a phone in someone's hand.

Every number in the run was true for what it measured. The coverage line just claimed more than the photos had tested. Locating the card in the frame moved from a speed fix to a correctness fix, and this is what the honest-coverage check below is for: say which roads were walked, and which weren't.

Every run writes one actor's journey to a user.md, labelled as agent-driven or watched by a person. It catches the dead ends and silent failures that a reconciled number misses. The rule is to keep what was seen apart from what it probably means, and the agent is never allowed to invent a feeling for a user it didn't observe.

A gauntlet changes state on purpose, at volume. So it needs a sandbox or simulation mode and a snapshot before it starts, and it never touches production, real money or real users. If a system can't be isolated and reversed, the skill refuses, or builds the missing scaffolding only when I say so. Afterwards it restores only the state it owns, and keeps the diary, the findings and the small repro cases.

bedrock walks the building as an inspector and keeps the findings ledger. gauntlet drives the building through its whole life. They meet in the same AUDIT.md.

These ideas sit under the moves. A run doesn't lecture from them; they are there for anyone who wants to know why.