278 staged trials · seed 20260818 · zero credits

How the detector was measured

How often the check catches a real break, and how often it cries wolf.

396 real archived Croma pages, across a migration nobody here authored

Small denominator, honestly small.

real corpus

Rolling baseline: what ships: it goes quiet part way through the broken run
Quarantine baseline: a capture it fired on never becomes evidence of a healthy day

cold start, too little history to score
ordinary month, correctly quiet
break, and it fired
break, and it stayed quiet

The same months as a table ⌄

Every month in the corpus, what the marker rule says it was, and what each baseline policy's watcher did about it
month	truth	Rolling baseline	Quarantine baseline
2016-11	ordinary	cold start, too little history to score	cold start, too little history to score
2016-12	ordinary	cold start, too little history to score	cold start, too little history to score
2017-01	ordinary	cold start, too little history to score	cold start, too little history to score
2017-02	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-03	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-04	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-05	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-07	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-08	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-10	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-11	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2017-12	ordinary	ordinary month, correctly quiet	ordinary month, correctly quiet
2022-01	break	break, and it fired	break, and it fired
2022-05	break	break, and it fired	break, and it fired
2022-08	break	break, and it fired	break, and it fired
2022-12	break	break, and it fired	break, and it fired
2023-06	break	break, and it fired	break, and it fired
2024-12	break	break, and it fired	break, and it fired
2025-01	break	break, and it stayed quiet	break, and it fired
2025-02	break	break, and it stayed quiet	break, and it fired
2025-03	break	break, and it stayed quiet	break, and it fired
2025-05	break	break, and it stayed quiet	break, and it fired

Rolling baseline

window 7 · 19 scored, 3 cold start

What ships: the last few captures, whatever they contain.

Breaks caught by row-drift6/1060%

False alarms on ordinary months0/90%

Breaks caught by status-polling0/100%

first caught January 2022

Quarantine baseline

window 7 · 19 scored, 3 cold start

Suspect captures stay out of the baseline they would poison.

Breaks caught by row-drift10/10100%

False alarms on ordinary months0/90%

first caught January 2022

The gap between those two columns is the finding

Same pages, same threshold. Only the baseline differs: under the rolling window the broken months walk into it and the detector goes quiet.

status-polling caught 0 of 10. Every one of these captures returned HTTP 200, status ready.

Every caveat this number is quoted with (7)

Reproduce it: npm run heldout · gate pass

aSmall denominators⌄
10 post-migration and 12 pre-migration captures. Every rate off this corpus is a count out of a handful, and one bucket either way moves it visibly.
bOne retailer, one event⌄
Croma only, and one migration. Nothing here says anything about Reliance Digital, Vijay Sales, or retailers in general.
cNull-rate signal only⌄
Row count is fixed at 18 by construction, because bucketing irregular Wayback captures by month would make it a measurement of the Internet Archive's crawl schedule rather than of the site. Both row-count rules are structurally unable to fire, so only the null-rate / field-presence signal is measured here.
dThe easiest break shape⌄
The migration nulls all five extracted fields on 18/18 rows: maximal magnitude, not a graded one. A selector that half-broke would be a harder test and this corpus does not contain one.
eMonths, not nights⌄
Croma took years to cross this boundary and the buckets straddling it are months apart, not consecutive nights. A 2018–2021 gap sits in the middle that neither CDX inventory covers, so the exact migration night is unidentifiable from the archive.
fWe wrote the extractor⌄
The pages and the layout change are what nobody here authored. That is the part that matters, but the five selectors are ours, read off archived 2016/2017 markup. A different 2016 author might have anchored on different nodes. The raw HTML is not committed either, so re-deriving the marker counts from bytes means refetching the pages from the Internet Archive; the committed sha256 manifest is what lets a verifier check they got the same bytes.
gTwo watchers are unscored⌄
Only the two a break of this shape can be judged by are scored: the drift watcher and the status poller. The value-plausibility watcher needs several retailers pricing one SKU and this is one retailer; the freshness watcher needs consecutive captures of the same rows and each bucket is a different sample of PDPs. Neither is scored here at all.

Measured on fixtures/heldout/WAYBACK-CROMA

Null-rate drift against status polling

Registered before any of it was measured, and frozen: nothing added after 19 August appears here.

Block 1 · pre-registered

Synthetic corpus. These numbers are not measured evidence.

All 10 base captures behind the staged tables were generated by seeded code (collector SYNTHETIC-DEMO). No web page was loaded to produce any of it.

Each watcher in this block, with breaks caught and false alarms
Watcher	Breaks caught	False alarms on ordinary days
`status-polling`the obvious baseline	0/790%	0/500%
`row-drift`this repo	60/7976%	0/500%

Each staged condition, whether a correct detector must fire on it, and how often each watcher did
Condition	Must	Trials	`status-polling`	`row-drift`
`selector-moved-price` ⌄ price null on every row (price selector no longer matches)	fire	10	0/10	10/10
`selector-moved-partial` ⌄ price null on one retailer only (single-site redesign)	fire	10	0/10	10/10
`list-price-captured` ⌄ price holds the struck-through MRP instead of the live price	fire	10	0/10	0/10
`currency-vanished` ⌄ currency null on every row	fire	10	0/10	10/10
`rows-collapsed` ⌄ zero rows on a still-ready snapshot	fire	10	0/10	10/10
`rows-halved` ⌄ one retailer's rows disappear entirely	fire	10	0/10	10/10
`title-emptied` ⌄ title null on every row	fire	10	0/10	10/10
`stale-repeat` ⌄ rows repeat the previous capture verbatim (a cached template served with HTTP 200)	fire	9	0/9	0/9
`ordinary-price-moves` ⌄ small +/-5% price moves on a seeded subset of rows	stay quiet	10	0/10	0/10
`stock-churn` ⌄ inStock flips on a minority of rows	stay quiet	10	0/10	0/10
`discontinued-sku` ⌄ one or two rows drop out (end-of-life SKUs)	stay quiet	10	0/10	0/10
`no-discount-today` ⌄ listPrice null on most rows (nothing on sale) while price holds	stay quiet	10	0/10	0/10
`identity` ⌄ unchanged capture (control)	stay quiet	10	0/10	0/10
Breaks caught			0/790%	60/7976%
False alarms			0/500%	0/500%

129 trials over 10 base captures. 8 of 13 conditions must fire; 5 must not. Trials is the denominator: a condition needing a predecessor stages on 9 bases, not 10.

Synthetic · computed by replay(), seed 20260818

Value plausibility, on the two shapes seen live

The defects the first live captures produced, with every ordinary day from block 1 as a false-alarm control.

Block 2 · added 19 August

Each watcher in this block, with breaks caught and false alarms
Watcher	Breaks caught	False alarms on ordinary days
`status-polling`the obvious baseline	0/200%	0/700%
`row-drift`for comparison	0/200%	0/700%
`value-plausibility`this repo	20/20100%	0/700%

Each staged condition, whether a correct detector must fire on it, and how often each watcher did
Condition	Must	Trials	`status-polling`	`row-drift`	`value-plausibility`
`price-scaled-100x` ⌄ one retailer's prices are 100x too large (a decimal point was eaten)	fire	10	0/10	0/10	10/10
`placeholder-price-repeated` ⌄ one retailer returns a single hard-coded price for every SKU	fire	10	0/10	0/10	10/10
`clearance-at-one-retailer` ⌄ one retailer clears stock at 50-65% off (a real sale, not a defect)	stay quiet	10	0/10	0/10	0/10
`joined-config-mismatch` ⌄ one SKU is joined to a higher-spec variant at one retailer (~2x, both correct)	stay quiet	10	0/10	0/10	0/10
`ordinary-price-moves`	stay quiet	10	0/10	0/10	0/10
`stock-churn`	stay quiet	10	0/10	0/10	0/10
`discontinued-sku`	stay quiet	10	0/10	0/10	0/10
`no-discount-today`	stay quiet	10	0/10	0/10	0/10
`identity`	stay quiet	10	0/10	0/10	0/10
Breaks caught			0/200%	0/200%	20/20100%
False alarms			0/700%	0/700%	0/700%

Separate denominators: these are breaks the null-rate detector is blind to, not extra credit for it. 90 trials over 10 base captures. 2 of 9 conditions must fire; 7 must not.

Synthetic · computed by replay(), seed 20260818

The repeated capture, which a summary of the past cannot see

A cached template served with HTTP 200 and rows byte-identical to last night. This watcher is handed the previous capture's rows; the other three never see them.

Block 3 · added 20 August

Each watcher in this block, with breaks caught and false alarms
Watcher	Breaks caught	False alarms on ordinary days
`status-polling`the obvious baseline	0/90%	0/500%
`row-drift`for comparison	0/90%	0/500%
`value-plausibility`for comparison	0/90%	0/500%
`row-freshness`this repo	9/9100%	0/500%

Each staged condition, whether a correct detector must fire on it, and how often each watcher did
Condition	Must	Trials	`status-polling`	`row-drift`	`value-plausibility`	`row-freshness`
`stale-repeat`	fire	9	0/9	0/9	0/9	9/9
`ordinary-price-moves`	stay quiet	10	0/10	0/10	0/10	0/10
`stock-churn`	stay quiet	10	0/10	0/10	0/10	0/10
`discontinued-sku`	stay quiet	10	0/10	0/10	0/10	0/10
`no-discount-today`	stay quiet	10	0/10	0/10	0/10	0/10
`identity`	stay quiet	10	0/10	0/10	0/10	0/10
Breaks caught			0/90%	0/90%	0/90%	9/9100%
False alarms			0/500%	0/500%	0/500%	0/500%

59 trials over 10 base captures. 1 of 6 conditions must fire; 5 must not. Trials is the denominator: a condition needing a predecessor stages on 9 bases, not 10.

Synthetic · computed by replay(), seed 20260818

What it does not catch

The 19 missed breaks are the 2 conditions this detector cannot see, both in the catalogue on purpose.

list-price-captured0/10 caught: Wrong but plausible values, and no nulls to count. Catching it needs a value-plausibility statistic, which is block 2.
stale-repeat0/9 caught: Rows byte-identical to last night, served with HTTP 200. Catching it needs the prior rows, not a summary of them. Block 3 does, and catches every one.

The repeat is covered now; a partial repeat is not

row-freshness catches the byte-identical repeat in block 3, at a threshold of exactly zero movement. A cache serving part of the board slips under it, and no threshold above zero has been measured.

list-price-captured was declared in writing before any collector existed. That evening a real capture returned nineteen complete rows, zero nulls, every price 100× too large, and nothing fired. A human caught it.

Read off the table above

Recent decisions

The last 6 nights and repairs on record, out of 6 nights and 1 heal in the archive.

2026-08-23
Night watch ran; every check stayed quiet, 20 rows read.
2026-08-23
Night watch ran; every check stayed quiet, 20 rows read.
2026-08-23
Night watch ran; every check stayed quiet, 19 rows read.
2026-08-20
Heal ran on the value-plausibility trigger, stopped at gate approved, outcome healed, billed 19 page loads, and the collector id held.
2026-08-18
Night watch ran; every check stayed quiet, 20 rows read.
2026-08-18
Night watch ran; every check stayed quiet, 20 rows read.

Exported 2026-08-23T15:49:15.405Z. Nights that land after this export sit in the archive until the next deploy carries them.

Committed export fixtures/ledger/events.json

The measurement is a build gate

A block fails the build if its signal stops beating the watchers before it, or exceeds a 10% false-alarm ceiling.

One command, about a second, no API key and no network

Every command, with the output it printed.

Enforced by npm run replay