feat(worldmodel): action-conditioned probe — the action is not represented - #59
Merged
Conversation
…ented Ledger node 18. Node 17's test (c): measure how much of WHICH ACTION WAS TAKEN survives into the representation. The prediction was too generous. CEILING, from the true (pre, post) states: ay R^2 = 1.0000 exact -- ay = (pos' - pos - vel'*DT)/(0.5*DT) is linear ax R^2 = 0.7096 short of 1.0 only because decay depends on chi FROM THE LATENT PAIR: ax R^2 = -0.1676 ay R^2 = -0.1199 Negative R^2 is worse than predicting the mean. The representation carries NOTHING a linear probe can use -- not the ~6% node 17 implied, zero. Both controls behave: mode_bit (action[2], which _step never reads) scores -0.2600 from the latent and -0.0547 from the true states; z_pre alone, which cannot see the transition, gives -0.1183 / -0.0845. THE MECHANISM IS THE ENCODER, AND vel IS THE SHARPEST NUMBER IN THIS LEDGER: peak_amp 0.9971 chi 0.9498 offset 0.6856 pos 0.6537 vel 0.0326 The three near-perfect factors are per-trajectory CONSTANTS. The two that EVOLVE are the two worst, and vel -- the channel ax acts through -- is at zero. So ax is unrecoverable in principle: recovering it needs vel and vel', neither encoded. For pos: R^2 0.6537 against sd 0.5812 leaves residual noise 0.342, while the one-step action effect is 0.120. SNR 0.35. The action's fingerprint is a third the size of the encoder's own noise on the only variable it can move. ADOPTED into the panel with the mode_bit negative control asserted in smoke_test. Mutation-checked: leaking the probe's target makes mode_bit score 1.000 and the assertion fires. That control is the only way this metric can silently lie, so it is asserted rather than eyeballed. CORRECTS NODE 17. "Under-represents by ~3.5x" becomes "does not represent at all". Node 17 measured the PREDICTOR's response to an action it receives as an explicit input; this measures whether the ENCODED TRANSITION retains which action was taken. The second is load-bearing and it is zero -- which also means node 17's 6.1% predictor response is fitting noise, not a weak signal. This closes the nodes 6-17 arc. Control was never reachable from any representation-side change, because the encoder discards the action's effect and encodes the frozen factors nearly perfectly instead. vel at R^2 = 0.033 was visible in every run since node 3 and was never read as a control result. Next test named: check whether vel is present in _render_state at all. If the observation does not carry velocity, the task is unobservable rather than hard, which is a generator defect and would retire the control line as specified. Verified: smoke 3 consecutive runs; ledger, ADR and test suites green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DJJ8y1zo7WSqEmGNW8DQap
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Node 17's test (c): measure how much of which action was taken survives into the representation. The prediction was too generous.
Result
ayis exactly recoverable from the true states —ay = (pos' − pos − vel'·DT)/(0.5·DT)is linear, so R² = 1.000 — and not at all from the latent. Negative R² is worse than predicting the mean. The representation carries nothing a linear probe can use. Not the ~6% node 17 implied; zero.Both controls behave:
mode_bitisaction[2], which_stepnever reads, andz_prealone cannot see the transition.The mechanism, and the sharpest number in the ledger
The three near-perfect factors are per-trajectory constants. The two that evolve are the two worst — and
vel, the channelaxacts through, is at zero.So
axis unrecoverable in principle: recovering it needsvelandvel', and neither is encoded.For
posthe arithmetic is just as decisive: R² 0.6537 against sd 0.5812 leaves encoder residual 0.342, while the one-step action effect is 0.120 — SNR 0.35. The action's fingerprint is a third the size of the encoder's own noise on the only variable it can move.Adopted into the panel, with its control asserted
action_recoverynow ships in the panel, andsmoke_testassertsmode_bit <= 0.05.Mutation-checked: leaking the probe's target makes
mode_bitscore 1.000 and the assertion fires. That control is the only way this metric can silently lie, so it is asserted rather than eyeballed.Corrects node 17
"Under-represents by ~3.5×" becomes "does not represent at all". Node 17 measured the predictor's response to an action it receives as an explicit input; this measures whether the encoded transition retains which action was taken. The second is the load-bearing quantity and it is zero — which also means node 17's 6.1% predictor response is fitting noise, not a weak signal.
This closes the nodes 6–17 arc
Control was never reachable from any representation-side change, because the encoder discards the action's effect and encodes the frozen factors nearly perfectly instead. Every panel metric that improved was measuring the constants.
velat R² = 0.033 has been visible in every run since node 3 and was never read as a control result.Next test, and it may retire the line
Check whether
velis present in_render_stateat all. If the observation does not carry velocity, the task is unobservable rather than hard — a generator defect, and it would retire the control line as specified rather than motivate another encoder change.Verified: smoke 3 consecutive runs; ledger, ADR and test suites green.