Prints all three scoreboards with their synthetic-corpus warning, in about a second.
Also a build gate: a block exits non-zero unless its signal beats the watchers before it and
stays under a 10% false-alarm ceiling.
The same numbers, as a page.
$npm run replaycaptured output · three blocks⌄167 lines of output
############################################################################
## ##
## SYNTHETIC DATA. THIS IS NOT MEASURED EVIDENCE ##
## ##
## All 10 base captures below were generated by seeded code ##
## (collector SYNTHETIC-DEMO). Nothing here was captured from a live ##
## retailer, and no page was ever loaded to produce it. ##
## ##
## These numbers show only that the replay harness runs and is ##
## reproducible. They say NOTHING about how the detector performs on ##
## real pages and must not be quoted as a result. They are replaced ##
## the moment real Bright Data captures exist. ##
## ##
## See fixtures/snapshots/README.md. ##
## ##
############################################################################
BLOCK 1, PRE-REGISTERED: null-rate drift vs status polling
seed 20260818 · 10 base captures · 13 mutations · 129 trials · 2026-08-23T15:58:01.174Z
The catalogue and the two watchers as registered before any of it was measured.
This block is frozen. Nothing added after 19 Aug appears in it.
MUTATION KIND TRIALS status-polling row-drift
---------------------------------------------------------------------
selector-moved-price break 10 0/10 10/10
selector-moved-partial break 10 0/10 10/10
list-price-captured break 10 0/10 0/10
currency-vanished break 10 0/10 10/10
rows-collapsed break 10 0/10 10/10
rows-halved break 10 0/10 10/10
title-emptied break 10 0/10 10/10
stale-repeat break 9 0/9 0/9
ordinary-price-moves ordinary 10 0/10 0/10
stock-churn ordinary 10 0/10 0/10
discontinued-sku ordinary 10 0/10 0/10
no-discount-today ordinary 10 0/10 0/10
identity ordinary 10 0/10 0/10
---------------------------------------------------------------------
BREAKS CAUGHT 0/79 0% 60/79 76%
FALSE ALARMS 0/50 0% 0/50 0%
^ SYNTHETIC CORPUS (SYNTHETIC-DEMO), invented captures, not measured evidence.
break = a correct detector MUST fire (counts toward breaks caught)
ordinary = a correct detector MUST NOT fire (counts toward false alarms)
WHAT EACH MUTATION MODELS
selector-moved-price price null on every row (price selector no longer matches)
selector-moved-partial price null on one retailer only (single-site redesign)
list-price-captured price holds the struck-through MRP instead of the live price
currency-vanished currency null on every row
rows-collapsed zero rows on a still-ready snapshot
rows-halved one retailer's rows disappear entirely
title-emptied title null on every row
stale-repeat rows repeat the previous capture verbatim (a cached template served with HTTP 200)
ordinary-price-moves small +/-5% price moves on a seeded subset of rows
stock-churn inStock flips on a minority of rows
discontinued-sku one or two rows drop out (end-of-life SKUs)
no-discount-today listPrice null on most rows (nothing on sale) while price holds
identity unchanged capture (control)
==============================================================================
BLOCK 2, ADDED 19 AUG: value plausibility, on shapes observed live
seed 20260818 · 10 base captures · 9 mutations · 90 trials · 2026-08-23T15:58:01.180Z
The two defects the first live captures actually produced, plus every ordinary day
from block 1 as a false-alarm control. Separate denominators, separate claim: these
are breaks the null-rate detector is structurally blind to, not extra credit for it.
MUTATION KIND TRIALS status-polling row-drift value-plausibility
---------------------------------------------------------------------------------------------
price-scaled-100x break 10 0/10 0/10 10/10
placeholder-price-repeated break 10 0/10 0/10 10/10
clearance-at-one-retailer ordinary 10 0/10 0/10 0/10
joined-config-mismatch ordinary 10 0/10 0/10 0/10
ordinary-price-moves ordinary 10 0/10 0/10 0/10
stock-churn ordinary 10 0/10 0/10 0/10
discontinued-sku ordinary 10 0/10 0/10 0/10
no-discount-today ordinary 10 0/10 0/10 0/10
identity ordinary 10 0/10 0/10 0/10
---------------------------------------------------------------------------------------------
BREAKS CAUGHT 0/20 0% 0/20 0% 20/20 100%
FALSE ALARMS 0/70 0% 0/70 0% 0/70 0%
^ SYNTHETIC CORPUS (SYNTHETIC-DEMO), invented captures, not measured evidence.
break = a correct detector MUST fire (counts toward breaks caught)
ordinary = a correct detector MUST NOT fire (counts toward false alarms)
WHAT EACH MUTATION MODELS
price-scaled-100x one retailer's prices are 100x too large (a decimal point was eaten)
placeholder-price-repeated one retailer returns a single hard-coded price for every SKU
clearance-at-one-retailer one retailer clears stock at 50-65% off (a real sale, not a defect)
joined-config-mismatch one SKU is joined to a higher-spec variant at one retailer (~2x, both correct)
ordinary-price-moves small +/-5% price moves on a seeded subset of rows
stock-churn inStock flips on a minority of rows
discontinued-sku one or two rows drop out (end-of-life SKUs)
no-discount-today listPrice null on most rows (nothing on sale) while price holds
identity unchanged capture (control)
==============================================================================
BLOCK 3, the repeated capture: what a summary of the past cannot see
seed 20260818 · 10 base captures · 6 mutations · 59 trials · 2026-08-23T15:58:01.195Z
A cached or not-found template served with HTTP 200: tonight's timestamp, last night's
rows, verbatim. Every field populated, every value real, the job green. The three
watchers above judge (snapshot, baseline statistics) and a baseline is a summary, so
none of them is ever shown the rows that are the only evidence. Same ordinary days as
the control, because a rule this narrow is worth nothing without them.
MUTATION KIND TRIALS status-polling row-drift value-plausibility row-freshness
------------------------------------------------------------------------------------------------------
stale-repeat break 9 0/9 0/9 0/9 9/9
ordinary-price-moves ordinary 10 0/10 0/10 0/10 0/10
stock-churn ordinary 10 0/10 0/10 0/10 0/10
discontinued-sku ordinary 10 0/10 0/10 0/10 0/10
no-discount-today ordinary 10 0/10 0/10 0/10 0/10
identity ordinary 10 0/10 0/10 0/10 0/10
------------------------------------------------------------------------------------------------------
BREAKS CAUGHT 0/9 0% 0/9 0% 0/9 0% 9/9 100%
FALSE ALARMS 0/50 0% 0/50 0% 0/50 0% 0/50 0%
^ SYNTHETIC CORPUS (SYNTHETIC-DEMO), invented captures, not measured evidence.
break = a correct detector MUST fire (counts toward breaks caught)
ordinary = a correct detector MUST NOT fire (counts toward false alarms)
WHAT EACH MUTATION MODELS
stale-repeat rows repeat the previous capture verbatim (a cached template served with HTTP 200)
ordinary-price-moves small +/-5% price moves on a seeded subset of rows
stock-churn inStock flips on a minority of rows
discontinued-sku one or two rows drop out (end-of-life SKUs)
no-discount-today listPrice null on most rows (nothing on sale) while price holds
identity unchanged capture (control)
==============================================================================
BLOCK 4, ADDED 22 AUG: the expiry signal, where the schema carries a validity date
seed 20260818 · 10 base captures · 4 mutations · 40 trials · 2026-08-23T15:58:01.200Z
Staged on a synthetic job-board corpus, the one domain here whose rows carry their own
expiry (`validThrough`). Every dated posting's window closed while the page still serves
them: proof from a single capture that the bytes are old, needing no history, which is why
the freshness rule above cannot make the same call. The other four run so their zero column
is visible. Ordinary days are the job board's own: a current board, the lazy-purge lag that
makes firing on any expired row unusable, and a board that publishes no expiries at all.
MUTATION KIND TRIALS status-polling row-drift value-plausibility row-freshness row-expiry
--------------------------------------------------------------------------------------------------------------------
expired-listing-served break 10 0/10 0/10 0/10 0/10 10/10
live-listings ordinary 10 0/10 0/10 0/10 0/10 0/10
some-listings-lapsed ordinary 10 0/10 0/10 0/10 0/10 0/10
expiry-absent ordinary 10 0/10 0/10 0/10 0/10 0/10
--------------------------------------------------------------------------------------------------------------------
BREAKS CAUGHT 0/10 0% 0/10 0% 0/10 0% 0/10 0% 10/10 100%
FALSE ALARMS 0/30 0% 0/30 0% 0/30 0% 0/30 0% 0/30 0%
^ SYNTHETIC CORPUS (SYNTHETIC-JOBS), invented captures, not measured evidence.
break = a correct detector MUST fire (counts toward breaks caught)
ordinary = a correct detector MUST NOT fire (counts toward false alarms)
WHAT EACH MUTATION MODELS
expired-listing-served every dated posting's validity has already ended (a cached board served with HTTP 200)
live-listings every dated posting expires weeks ahead (a healthy, current board)
some-listings-lapsed a minority of dated postings expired while still listed (ordinary board lag)
expiry-absent no posting carries validThrough (the board does not publish expiries)
PASS (block 1): row-drift caught 60/79 breaks vs 0/79 for status-polling, false alarms 0%.
PASS (block 2): value-plausibility caught 20/20 of the live shapes that row-drift scored 0/20 on, false alarms 0% over 70 ordinary days.
PASS (block 3): row-freshness caught 9/9 repeated captures that the other three scored 0/9 on, false alarms 0% over 50 ordinary days.
PASS (block 4): row-expiry caught 10/10 lapsed boards with no history that the other four scored 0/10 on, false alarms 0% over 30 ordinary days.
Exit codes: 0 pass · 1 a detector missed its claim or cried wolf · 2 bad input.