Skip to content

lattice(phase2): H2/H5/H4 verdicts + close CD-H1-01/CD-A2-01 + docs-truth sync - #292

Open
mattmre wants to merge 4 commits into
mainfrom
lattice/phase2-continue-20260713
Open

lattice(phase2): H2/H5/H4 verdicts + close CD-H1-01/CD-A2-01 + docs-truth sync#292
mattmre wants to merge 4 commits into
mainfrom
lattice/phase2-continue-20260713

Conversation

@mattmre

@mattmre mattmre commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

Continues the Liquified Lattice Phase II program: runs two GPU campaigns whose apparatus was already
merged (H2 swap re-run, H5 post-bank head-to-head + H4 ablation), closes the two open Carried-Debt
rows, and syncs the roadmap/CHANGELOG to git-verified reality. Docs + evidence only — zero
production .py changes.

Branch: lattice/phase2-continue-20260713 (4 commits). Block flag: CLEAR (0 carried-debt rows).

Commits

Commit What
results(h2) Re-run SciFact + NFCorpus query-encoder-swap campaigns on the H1-fixed line; regenerate both result docs. Closes CD-H1-01 + CD-A2-01.
results(h5) Run the merged H5 head-to-head driver (#290) + H4 compound_cycles (#291). Living-bank verdict: hard negative.
docs(lattice) Phase II docs-truth: ROADMAP gains a Status column (rungs 9–12 DONE / 13 PARTIAL / 15–17 OPEN); CHANGELOG apparatus + open-steps bullets.
chore(hygiene) Untrack the two residual #275 NFCorpus campaign logs (completes the L1 tracking-consistency claim; Tier-B-caught).

Key results (every number recomputed from committed manifests by the Tier B reviewer)

H1-fix regression gate PASSED — C3a baseline now equals C0/C2/C2O/C4a per seed on both datasets
(SciFact seed-42 all 0.793459; NFCorpus seed-42 all 0.555004). The contamination was on NFCorpus
(old C3a baselines differed); SciFact was already clean. Closes CD-H1-01. The run exercised the real
all-mpnet-base-v2 swap path through QueryEncoderDrift._backend — closes CD-A2-01.

H5 living-bank verdict = LIVING BANK WINS: False on both datasets (gate: C5 living must beat BOTH
C5s frozen-static AND C5r one-shot router):

Dataset C5 living C5s static C5r one-shot C5>C5s C5>C5r WINS
SciFact 0.131135 0.131135 0.180862 False False False
NFCorpus 0.046389 0.046389 0.045817 False True False

The living/annealed lifecycle is bit-identical to a frozen static bank; a one-shot router is
competitive-or-better. Living bank NON-PROMOTED. H4 (single-seed): compound_cycles=True 0.005258
vs False 0.236297 (~45× collapse) — compounding is a rejected design.

Evidence / smoke (human-readable; machine fields below)

Result docs: the five docs/drift-recovery-*.md files listed above; summary manifests + H4 run JSONs
under experiment_runs/drift-recovery/. Smoke (floor-tier): H1 regression parity PASS;
scripts/check_block_flag.py → CLEAR; test_post_bank_head_to_head OK;
test_query_encoder_drift.TestQueryEncoderDriftRealModelSmoke (CHELATED_RUN_REAL_MODEL_TESTS=1) →
Ran 1 test, OK. The campaigns themselves are the end-to-end evidence.

Brutal Honesty

  • No production code changed (verified: git diff --name-only origin/main..HEAD has zero root .py).
  • Tracking hygiene: untracked 22 NFCorpus per-run JSONs (25 MB) + 2 residual results(a5): NFCorpus swap campaign — partial recovery generalizes to second dataset #275 logs so NFCorpus
    matches SciFact's manifest-only policy; the small summary manifests carry the doc-source rows.
  • L5 (disclosed): the real-model swap smoke is opt-in (not default CI, avoids a 17-min HF hang);
    H4 is a single-seed (42) ablation — direction unambiguous, magnitude one seed.
  • Step 13 is PARTIAL, not DONE — delivered post-bank prune/re-anneal is not the
    isomer/convergence → Evidence-DAG prune the original exit criteria named; flagged for re-scope.
  • Older [Unreleased] Findings (Phase I / SHIM-on-hold) are now stale vs lattice delivery; left in
    place and deferred to a separate hygiene pass (DEFERRED_SCOPE).

The initial combined Tier B scored 97 with one LOW hygiene defect (two residual #275 NFCorpus logs
still tracked); commit eb750958 untracks them, and a fresh Tier B pass on the amended tip re-scored
the slice to a genuine 100 (severity none, no new defects). Every NDCG and gate was independently
recomputed from the committed manifests by the reviewer.

§4 required fields

EVIDENCE: five docs/drift-recovery-*.md result docs + summary manifests + H4 run JSONs under experiment_runs/drift-recovery/ (per-number recompute done by Tier B)
SMOKE: floor-tier — H1 regression parity PASS; scripts/check_block_flag.py CLEAR; test_post_bank_head_to_head OK; test_query_encoder_drift.TestQueryEncoderDriftRealModelSmoke (CHELATED_RUN_REAL_MODEL_TESTS=1) Ran 1 test OK; the campaigns are the end-to-end evidence
BHS_SELF_DRAFT: 100
BHS_SELF_DRAFT_AGENT: Fable-5 (chair / implementer)
BHS_TIER_B: 100
BHS_TIER_B_AGENT: grok-4.5 (fresh adversarial reviewer, independent of the implementer)
BHS_TIER_B_SEVERITY: none
BHS_OFFICIAL: 100
CARRY_FORWARD: none
DEFERRED_SCOPE: refresh of the older stale [Unreleased] Findings bullets (Phase I / turn-3171 / SHIM-on-hold) — separate hygiene pass; and step-13 exit-criteria re-scope (post-bank lifecycle vs isomer/convergence→DAG prune)
LOOP_ITERATIONS: 1
OPERATOR_OVERRIDE: none

🤖 Generated with Claude Code

mattmre and others added 4 commits July 13, 2026 23:55
…2-01

Re-runs the SciFact + NFCorpus query-encoder-swap campaigns on post-#276 main
(H1 fix: baseline no longer sees the C3a build-time bounded adapter) and
regenerates both auto-generated result docs. Also exercises the real
all-mpnet-base-v2 swap backend on the cached 3090, closing the two open debt rows.

EVIDENCE: experiment_runs/drift-recovery/swap/swap-campaign-manifest-2026-06.json
  (config.swap_model == all-mpnet-base-v2, record_type == drift_recovery_swap_campaign);
  regenerated docs/drift-recovery-swap-results-2026-06.md +
  docs/drift-recovery-swap-nfcorpus-results-2026-06.md.
SMOKE (floor-tier): H1-fix regression gate PASSED - C3a baseline == C0/C2/C2O/C4a
  per seed on both datasets (SciFact seed-42 all 0.793459; NFCorpus seed-42 all
  0.555004); scripts/check_block_flag.py -> CLEAR, 0 carried debt rows;
  test_query_encoder_drift.TestQueryEncoderDriftRealModelSmoke (CHELATED_RUN_REAL_MODEL_TESTS=1)
  -> Ran 1 test, OK (real backend init logged).

CD-H1-01 CLOSED: both docs regenerated; stale-C3a supersession banner dropped
  (intended end state). Contamination signal was on NFCorpus (old C3a baselines
  differed, now match); SciFact C3a baseline was already clean pre-re-run.
  Fresh C3a finals small vs old (SciFact 3-seed ~0.157/0.161/0.164 vs 0.159;
  NFCorpus 3-seed mean 0.0524 vs 0.0535). C0/C2/C2O finals bit-identical; C4a
  finals show small re-run variance (SciFact seed-42 0.2359->0.2363; NFCorpus
  mean 0.0543->0.0529) - run noise, not the H1 baseline signal.
CD-A2-01 CLOSED: real swap path run through QueryEncoderDrift._backend in the
  production campaign driver + opt-in gated smoke passed; default-CI skip remains
  intentional (17-min HF hang).

## Brutal Honesty
- No new L1/L4/L8: no production code changed; docs + evidence artifacts only.
- HYGIENE (disclosed): untracked 22 NFCorpus per-run JSONs (25 MB) that PR #275
  force-committed; same class as the already-gitignored SciFact per-run files.
  Only the 14.5 KB summary manifests (which carry the doc-source rows) stay
  tracked. Removes 25 MB of regenerated blobs from the tracked tree going
  forward (prior blobs remain in #275 history) and makes NFCorpus consistent
  with SciFact tracking policy.
- L5 (disclosed, pre-existing): the real-model swap smoke is opt-in
  (CHELATED_RUN_REAL_MODEL_TESTS=1), not default-CI, to avoid a 17-min HF hang;
  the real path is now covered by the recorded campaign artifact + this manual
  gated run, not by default CI.
- SMOKE tier: floor (regression parity + block-flag + gated smoke), not a new
  end-to-end fixture; the campaign itself is the end-to-end evidence.

BHS_SELF_DRAFT: 100
CARRY_FORWARD: none
DEFERRED_SCOPE: H5 post-bank verdict (separate PR); H4 ablation (separate)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lation

Runs the merged H5 post-bank head-to-head driver (#290) and the H4 compound_cycles
knob (#291) on the query-encoder-swap arena. Rung-13/14 runtime evidence for the
lattice program.

EVIDENCE: docs/drift-recovery-post-bank-headtohead-results-2026-06.md (SciFact),
  docs/drift-recovery-post-bank-headtohead-nfcorpus-results-2026-06.md (NFCorpus),
  docs/drift-recovery-h4-compound-cycles-ablation-2026-07.md; summary manifests +
  H4 run JSONs under experiment_runs/drift-recovery/{post-bank-headtohead*,h4-compound}/.
SMOKE (floor-tier): python -m unittest test_post_bank_head_to_head -> OK (pre-run);
  the campaign itself is the end-to-end evidence.

VERDICT (honest negative): LIVING BANK WINS = False on both datasets. The gate is
  C5 (living/annealed bank) must beat BOTH C5s (frozen static bank) AND C5r (one-shot
  router).
  - SciFact: C5 == C5s 0.131135 (bit-identical); C5r one-shot 0.180862 beats living.
  - NFCorpus: C5 == C5s 0.046389; living edges C5r 0.045817 but ties static.
  The living/annealed lifecycle adds nothing over a frozen static bank; a one-shot
  router is competitive-or-better. Living bank NON-PROMOTED (docs/next-session.md
  Disposition). C5s is the honest post-bank baseline if any is kept.
H4 (single-seed SciFact C4a seed 42): compound_cycles True 0.005258 vs False
  0.236297 (~45x collapse). Compounding is a rejected design; idempotent one-shot
  fixed point is correct. Quantifies the paper Section 5.2 overshoot note.

## Brutal Honesty
- No new L1/L4/L8: no production code changed; runs merged drivers + records results.
- L5 (disclosed): H4 is a single-seed (42) ablation; direction is unambiguous
  (~45x) but the exact magnitude is one seed, stated in the H4 doc.
- Manifests-only tracking (consistent with the L1 hygiene decision): H5 per-run
  JSONs stay gitignored; the small summary manifests (which carry the doc-source
  rows) + the two <1 MB H4 run JSONs are the tracked evidence.
- Verdict is from the real GPU run, not asserted from the driver alone; every
  number in the docs traces to a manifest row.

BHS_SELF_DRAFT: 100
CARRY_FORWARD: none (living-bank line closed as non-promoted, not debt)
DEFERRED_SCOPE: L3 docs-truth sync (separate commit, same PR)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rungs

Main advanced 33 commits since the vision/roadmap landed, but ROADMAP_EXECUTION.md
Phase II still read as all-future work (no Status column). Syncs the roadmap +
CHANGELOG to git-verified reality.

EVIDENCE: per-step status verified from git log / git show on the merged PRs and
  targeted greps for the open steps (recorded in
  docs/waypoint-research-2026-06-09/panel/l3-docs-truth-draft.md).
SMOKE (docs-only): no code changed; ruff/tests unaffected.

Changes:
- ROADMAP_EXECUTION.md Phase II table gains a Status column + a git-verified status
  snapshot: rungs 9-12 DONE, 13 PARTIAL (post-bank prune/re-anneal only, NOT
  isomer/convergence -> Evidence-DAG prune), 14 apparatus DONE with the H5
  living-bank verdict a hard negative, 15-17 OPEN.
- CHANGELOG: apparatus-merged bullet (rungs 10-14 PRs) + still-OPEN bullet
  (15/16/17); the stale "Phase II execution program" Added line corrected from
  "all future" to the delivered/open split.

## Brutal Honesty
- No delivery claimed without a merged PR. Grep-verified the three OPEN steps are
  genuinely unimplemented: no PyG/DGL (15); adapter_router + QuantizationPromotionGate
  exist but are NOT integrated as one retrieval-fitness steering plane (16); no
  pool-shard parity via block_graph (17).
- Step 13 is honestly marked PARTIAL, not DONE — the delivered post-bank lifecycle
  is not the isomer/convergence->DAG-prune the original exit criteria named. Chair
  must re-scope or implement before step 13 reads green.
- Step 10 DONE = the rung-10 control-plane DoD, NOT a claim that historical
  SHIM-CD-01/02/06 rows are CLOSED (called out inline).
- Step 11 schedule ownership is split across two modules (#260 engine + #280
  post-bank), not the single-module monopoly the original exit text implied; marked
  DONE-for-lattice-rungs with that caveat inline.
- Older [Unreleased] Findings (Phase I / turn 3171 / "SHIM on hold") are now stale
  vs lattice delivery; left in place and flagged for a separate hygiene refresh
  rather than silently rewritten in this commit.

BHS_SELF_DRAFT: 100
CARRY_FORWARD: none
DEFERRED_SCOPE: refresh of the older stale [Unreleased] Findings bullets (separate hygiene pass)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…es L1 claim)

Tier B (combined review of this slice) caught that L1 (2b0fa04) claimed "only the
summary manifests stay tracked / NFCorpus consistent with SciFact policy" but two
#275-era logs remained tracked under experiment_runs/drift-recovery/swap-nfcorpus/:
campaign.log (~191 KB) and campaign-completion.log (~115 KB). SciFact's swap tree
tracks 0 logs, so those two were the last inconsistency. Untracking them here makes
the L1 hygiene claim actually true at PR granularity.

EVIDENCE: git ls-files experiment_runs/drift-recovery/swap/*.log -> 0 (SciFact);
  the two NFCorpus logs are covered by .gitignore (experiment_runs/) and will not
  re-appear. No other file touched.
SMOKE: scripts/check_block_flag.py -> CLEAR (unchanged).

## Brutal Honesty
- L13-light correction disclosed by Tier B, not self-caught in L1 - recorded here
  transparently rather than by silently weakening the L1 prose.
- No production code; no manifests/docs/NDCG changed; only two log blobs untracked.

BHS_SELF_DRAFT: 100
CARRY_FORWARD: none

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the project's changelog, roadmap, next-session plans, and experimental manifests to document the outcomes of Phase II drift-recovery experiments. Key updates include recording the negative verdict of the H5 living-bank head-to-head evaluation (resulting in its non-promotion), documenting the H4 compound-cycles ablation study, and updating the query-encoder-swap results following the H2 re-runs, which successfully closes carried debts CD-A2-01 and CD-H1-01. There are no review comments, and therefore no feedback is provided.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant