Skip to content

bench(worldmodel): channel ablation — ax inert, ay harmful, generator at fault - #60

Merged
aurascoper merged 1 commit into
mainfrom
bench/action-channel-ablation
Aug 2, 2026
Merged

bench(worldmodel): channel ablation — ax inert, ay harmful, generator at fault#60
aurascoper merged 1 commit into
mainfrom
bench/action-channel-ablation

Conversation

@aurascoper

Copy link
Copy Markdown
Owner

Pre-registered before running. H1 (velocity is the missing variable): zeroing ax costs nothing, ax alone ≈ random. H2 (horizon binding): both ablations degrade together.

Result

arm log(d_f/d_0) vs FULL success
MPC full +0.0886 35.0%
MPC ay_only (ax=0) +0.0971 +0.0086 36.5%
MPC ax_only (ay=0) +0.0374 −0.0512 33.5%
random +0.0402 −0.0483 34.5%
zero +0.0341 −0.0545 37.0%
ORACLE full −11.11 100.0%
ORACLE ay_only −8.72 91.5%
ORACLE ax_only −4.52 73.5%

H1 confirmed, H2 refuted — |A−FULL| = 0.0086 against |B−FULL| = 0.0512, a six-fold separation.

The pre-registered discriminator resolves. ax alone is worth 73.5% to an exact controller and 33.5% to MPC — below the 37.0% of doing nothing. The authority exists; the planner cannot use it. The deficit is representational, not reachability.

New: all of MPC's harm is in ay

ay_only reproduces full MPC; ax_only collapses to the zero policy. So MPC only ever acts through ay, and those actions are uncorrelated with need (node 17, r = +0.04) — strictly worse than not acting.

Both channels fail, differently: ax is unrepresentable (vel R² = 0.033), ay is representable but at SNR 0.35, so its choices are noise.

The generator explains both

Contributed by review, verified here independently. _render_state does carry velocity — c1 = beta * (1 + 0.10·vel) — but as a 10% modulation on beta = 10^offset · 21^(−chi):

channel total var nuisance task
c0 (pos) 4.82e-02 99.95% 0.845%
c1 (vel) 1.24e-02 100.13% 0.0106%

pos gets ~80× the share. (Review reported 1.15% / 0.05% from a different freeze-point — magnitudes differ, conclusion identical; both are recorded.)

And chi/peak_amp/offset are held per trajectory, so predicting them at t+1 is exact and free. The loss is minimised by encoding the nuisance, and VICReg's variance term rewards precisely those directions.

So node 18's vel R² = 0.033 is not a training failure. It is a working objective behaving correctly on a generator that made the task variable 0.01% of the signal.

Velocity is not destroyed: vel = 10·(c1/betaPower − 1) exactly, max error 2e-15. A ratio of two observed channels — nonlinear, hence invisible to a linear probe, trivially within an MLP's reach. Nothing prevents learning it; nothing rewards it.

Closes the representation-objective line

Nodes 6–18 were measuring the wrong thing correctly. No representation objective can encode a variable made 0.01% of the observed variance while paying a variance penalty for ignoring the other 99.99%.

Next: generator repair, under node-14 blind discipline

Two candidates, not equivalent:

  1. Raise the velocity modulation depth — cheapest, but it is a knob, and tuning it until an arm wins is unfalsifiable.
  2. Let chi/offset vary within-trajectory — removes the free-prediction subsidy without touching velocity's coefficient. Changes what the objective is rewarded for rather than what it is shown. Preferred.

Either way: specify using only oracle/random/zero, never a candidate arm; require the repaired generator to reproduce the ladder ordering (oracle > ay-oracle > ax-oracle > zero ≈ random ≈ MPC) before any arm is rerun; and add an acceptance assertion that a probe on the raw observation recovers vel above a floor, so a future generator edit cannot silently reintroduce this.

Ledger-only. Verified: smoke 2 runs, ledger and ADR gates green.

… at fault

Ledger node 19. Pre-registered before running: H1 (velocity is the missing
variable) predicts zeroing ax costs nothing and ax alone performs like random;
H2 (horizon binding) predicts both ablations degrade together.

  arm                  log(d_f/d_0)   vs FULL   success
  MPC full                  +0.0886         —     35.0%
  MPC ay_only (ax=0)        +0.0971   +0.0086     36.5%
  MPC ax_only (ay=0)        +0.0374   -0.0512     33.5%
  random                    +0.0402   -0.0483     34.5%
  zero                      +0.0341   -0.0545     37.0%
  ORACLE full               -11.11              100.0%
  ORACLE ay_only             -8.72               91.5%
  ORACLE ax_only             -4.52               73.5%

H1 CONFIRMED, H2 REFUTED: |A-FULL| = 0.0086 vs |B-FULL| = 0.0512, six-fold.

THE PRE-REGISTERED DISCRIMINATOR RESOLVES. ax alone is worth 73.5% to an exact
controller and 33.5% to MPC -- below the 37.0% of doing nothing. The authority
exists; the planner cannot use it. The deficit is representational.

NEW: all of MPC's harm is in ay, the channel it CAN partly see. ay_only
reproduces full MPC; ax_only collapses to the zero policy. MPC only ever acts
through ay, and those actions are uncorrelated with need (node 17, r = +0.04),
which is strictly worse than not acting. Both channels fail differently: ax is
unrepresentable (vel R^2 = 0.033), ay is representable at SNR 0.35 so its
choices are noise.

THE GENERATOR EXPLAINS BOTH. Finding contributed by review, verified here
independently. _render_state carries velocity as a 10% modulation on beta:
c1 = beta * (1 + 0.10*vel). Variance decomposition over 4000 samples gives task
share 0.845% for c0(pos) and 0.0106% for c1(vel) -- pos gets ~80x the share.
(Review reported 1.15% / 0.05% from a different freeze-point; magnitudes differ,
conclusion identical, and both are recorded.) chi/peak_amp/offset are held per
trajectory, so predicting them at t+1 is exact and free. The loss is minimised
by encoding the nuisance and VICReg's variance term rewards precisely those
directions. Node 18's vel R^2 = 0.033 is not a training failure -- it is a
working objective behaving correctly on a generator that made the task variable
0.01% of the signal.

Velocity is NOT destroyed: vel = 10*(c1/betaPower - 1) exactly, max error 2e-15.
A ratio of two observed channels -- nonlinear, hence invisible to a linear probe,
trivially within an MLP's reach. Nothing prevents learning it; nothing rewards it.

CLOSES the representation-objective line as specified. Nodes 6-18 were measuring
the wrong thing correctly. Next is a generator repair under node-14 blind
discipline, preferring within-trajectory variation of chi/offset over raising the
velocity coefficient, because the first changes what the objective is REWARDED
for and the second is a knob that can be tuned until an arm wins.

Ledger-only. Verified: smoke 2 runs, ledger and ADR gates green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJJ8y1zo7WSqEmGNW8DQap
@aurascoper
aurascoper merged commit a893528 into main Aug 2, 2026
2 checks passed
@aurascoper
aurascoper deleted the bench/action-channel-ablation branch August 2, 2026 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant