driftbet — The Cost of a Label

Label-free, training-free attribution of drift in data streams: not whether a stream changed, but which cause is responsible — A model/sensor fault, B noise, C world/regime shift, or D annotator (label corruption) — and how the quality of that attribution scales as a human-label budget grows from zero. A synthetic-but-adversarial testbed, a training-free label model over a panel of weak detectors, and anytime-valid guarantees (e-processes, se-BH) — reported inseparably from the measured boundaries of each mechanism.

📄 Paper: The Cost of a Label: Label-Free Drift Attribution and Its Measured Boundaries (preprint.tex → arXiv, forthcoming)

---

The finding

At zero label budget, the attributability of a cause is governed by the correlation structure of its witnesses, not by its frequency.

A cause with even one independent-channel witness is recoverable spectrally from a minority position; a cause whose witnesses all share one channel is not — and its apparent un-attributability decomposes into an instrument coverage gap plus an aggregation collapse, both fixable, rather than a property of the cause. On the recoverable cause the structure-aware aggregator closes the zero-budget misattribution gap from 0.50 (naive) to 0.00. The left end of the cost-of-label curve is thus explained, not merely measured — a mechanistic account of why the anchor-set baseline (Li, 2026) buys labels at all.

Results, and the boundaries they draw

This project reports negatives with the same care as positives; each is an applicability map of a mechanism, not a confession. Selected results and the boundaries they draw (three of the six pre-registered bets came out refuted, one non-realizable, one held, one resolved into a robustness boundary):

result	status
witness-structure finding (cause A recoverable label-free)	holds, reproducible (ρ_A 0.693 vs 0.000)
cost-of-label gap on A (0.50 → 0.00 at b=0)	holds, the visualized finding
"cause C has the steepest slope"	refuted on the aggregate; holds on the firing subset (two denominators)
silent-cause slope flattens with visibility ν	refuted — it steepens (0.69 → 0.79)
e-process latency ≤ 2× CUSUM	refuted — 4.83× (= 1.83× magnitude × 2.64× anytime)
corrupted-anchor duel (ensemble vs anchor-set)	non-realizable — fifth wall: no anchor-load-bearing class
se-BH FDR inflation under correlation	robustness measured instead — holds to ρ=0.99; inflation needs temporal lead-lag (source-verified)

Plus five identifiability walls (four found before the frontier run), an adversarial "milker" suite (sawtooth, adaptive stealth, correlated streams, ground-truth-leak canary), and a decomposed account of the anytime guarantees' prices (detection latency, a blind zone ≥ 75 σ·ticks deep).

Reproducibility — every number traces to one command

Every headline number in the paper is produced by a saved, seeded script and reported with its denominator stated alongside it (a discipline that caught a non-reproducible number of our own at the threshold of publication and returned a corrected value along a recorded path). The full number → script → denominator map is Appendix A of the paper; the scripts live in eval/:

.venv/Scripts/python -m eval.recompute_results      # M3 finding: ρ_A, C-leg audit (with denominators)
.venv/Scripts/python -m eval.figure_cost_of_label   # Figure 1 (fig_cost_of_label.png)
.venv/Scripts/python -m eval.c_coverage_diagnosis   # the ~0.87 coverage floor
.venv/Scripts/python -m eval.stavka3_lag            # latency 4.83× decomposition
.venv/Scripts/python -m eval.milker_sawtooth        # guarantee vs deployment (1 vs 43)
.venv/Scripts/python -m eval.milker_stealth         # blind-zone depth
.venv/Scripts/python -m eval.milker_sebh            # se-BH under dependence + lead-lag
.venv/Scripts/python -m eval.mutation_gt_leak       # ground-truth-leak canary

Discipline

The verdict space, thresholds, and six falsifiable bets were frozen in PREREGISTRATION.md before the first run, and negatives are reported with the same care as positives — three of the six bets came out refuted, one non-realizable. Deviations from the frozen plan were tracked and justified rather than edited silently.

Repository map

path	what
PREPRINT.md / preprint.tex	the paper (English)
PREREGISTRATION.md	the frozen pre-registration
core/	world, detectors, label models (Dawid–Skene, Jaffe), e-process, se-BH, attribution
eval/	every reproducing script + fig_cost_of_label.png
tests/	milestone tests (M0–M3)

Environment

Bit-for-bit reproducibility (seed → identical outputs) is claimed, so versions are pinned. Reference environment: CPython 3.14.2 (Windows x64); full transitive pin in requirements.lock.

python -m venv .venv
.venv\Scripts\python -m pip install -r requirements.lock
.venv\Scripts\python -m pytest -q          # 30 passed, 3 xfailed

Any version bump goes in its own commit with regenerated figures — otherwise a "reproducible" run yields different bits from a minor NumPy update.