lattice rung 13: detector-driven Evidence-DAG disintegration loop

docs/ROADMAP_EXECUTION.md

@@ -40,7 +40,7 @@ Merge gate: **CLEAR** for this sequence (`docs/next-session.md`). SHIM debts sta
 - No new doc-only shim cycles or 10-agent backlog slices.
 - No claiming SHIM-CD **CLOSED** without full substrate DoD.
 - No Model-Scope + full shim + EGGROLL at full throttle in parallel.
-- No GNN layer or disk-pool integration until Phase II steps 12–14 are honestly closed (schema **#277 DONE**; drift apparatus **DONE** with the H5 living-bank verdict a hard negative; disintegration **#279 PARTIAL** — do not start step 15 while claiming step 13 fully green without a chair re-scope).
+- No GNN layer or disk-pool integration until Phase II steps 12–14 are honestly closed (schema **#277 DONE**; drift apparatus **DONE** with the H5 living-bank verdict a hard negative; disintegration **DONE** — detector-driven Evidence-DAG edge prune, rung-13 PR). Steps 15/16/17 are now unblocked and are the endgame program.
 
 ---
 
@@ -51,20 +51,20 @@ Merge gate: **CLEAR** for this sequence (`docs/next-session.md`). SHIM debts sta
 
 Same one-track rule: finish each step before starting the next. Phase II does not reopen parallel doc-only shim cycles.
 
-**Status snapshot (2026-07, git-verified):** rungs **9–12 DONE**; **13 PARTIAL** (post-bank
-prune/re-anneal lifecycle only — not isomer/convergence → Evidence-DAG prune); **14 apparatus DONE**
-with the **H5 living-bank question closed as a hard negative** (LIVING BANK WINS = False on SciFact +
-NFCorpus); **15–17 OPEN**. Lattice apparatus PRs: #260, #277, #279–#291. Next executable feature work
-is step **15** only after chair re-scopes or closes the step **13** remainder — or **16/17** if the
-quant plane / disk pool is prioritized over GNN.
+**Status snapshot (2026-07, git-verified):** rungs **9–13 DONE** (13 completed by the detector-driven
+Evidence-DAG edge-prune loop, rung-13 PR); **14 apparatus DONE** with the **H5 living-bank question
+closed as a hard negative** (LIVING BANK WINS = False on SciFact + NFCorpus); **15–17 OPEN**. Lattice
+apparatus PRs: #260, #277, #279–#291 + rung-13. With step 13 closed, the remaining executable feature
+work is **15 (GNN), 16 (quant-aware routing plane), 17 (disk pool slice)** — being tackled as the
+endgame program (see `docs/waypoint-research-2026-06-09/panel/lattice-endgame-plan-2026-07-14.md`).
 
 | Step | Track | Status | Exit criteria |
 |------|--------|--------|----------------|
 | 9 | **Model-Scope shadow (close Phase I #7)** | **DONE** (fixture path; Model-Scope stack + #254 persist/cap tests) | One bounded steering policy on Qwen3.5-9B fixture; `persist_records` / `load_records` round-trip; bridge `max_total_interventions` exercised in test |
 | 10 | **SHIM substrate DoD** | **DONE** (rung 10: #284 A1a / #285 A1b / #286 A1c; DoD honesty #289 — diagnostics observation-only; live routes overlap rung 16) | Promotable steering-route control plane, default-safe, with quant-survival + actionable rollback (`docs/rung10-shim-substrate-dod.md`). This is the rung-10 DoD, **not** a claim that historical SHIM-CD-01/02/06 rows are CLOSED. |
 | 11 | **Annealing controller** | **DONE** (#260 engine controller + #280 post-bank schedule; schedule ownership split across two modules) | Temperature schedule(s) drive explore ↔ stabilize; unit tests prove high-T vs low-T behavior; engine wires the controller into sedimentation temperature; post-bank path uses `annealing_schedule` in the C5 lifecycle |
 | 12 | **Evidence DAG schema** | **DONE** (#277 `evidence_dag.py` + validator + JSON schema; no GNN) | Typed graph contract over `build_attribution_pool.py` output (nodes: query/cluster/actuator; edges: retrieval/intervention links); JSON schema + validator; no GNN required yet |
-| 13 | **Disintegration loop** | **PARTIAL** — post-bank prune/re-anneal DONE (#279 H5a, wired #281–#283); **NOT** isomer/convergence → Evidence-DAG edge prune | Delivered: fitness-gated prune + re-anneal on `SteeringPostBank` with lifecycle artifacts. Original exit named `isomer_detector` / `convergence_monitor` triggers on DAG edges/pool entries — **not implemented**. Re-scope exit to the post-bank mechanism, or implement that trigger. |
+| 13 | **Disintegration loop** | **DONE** — detector-driven Evidence-DAG edge prune (rung-13 PR: `evidence_dag_disintegration.py` + `EvidenceDAG.prune_edges`/`reanneal`); post-bank prune/re-anneal also DONE (#279 H5a, wired #281–#283) | `isomer_detector` (per-query strength) + `convergence_monitor` (per-cluster summary) now score Evidence-DAG edge fitness and drive fitness-gated prune + transactional re-anneal, with a before/after fitness artifact and detector provenance. Fail-closed: missing/unmatched/immature signals → neutral fitness (never prunes); non-sedimentation isomer mode is rejected. The original exit criteria (detector-triggered DAG-edge prune with recorded before/after fitness) are met. |
 | 14 | **Concept-drift experiment** | **DONE apparatus** (#258–#266 harness, #267 track-0 hygiene; swap arena #268–#276; H3 #287/#288; H4 #291; H5 driver #290). **H5 living-bank VERDICT: FAIL / non-promoted** (SciFact + NFCorpus) | Injected drift + recovery campaigns under `docs/drift-recovery-*.md`. Living annealed post-bank (C5) does **not** beat the frozen static bank + one-shot router gate. Compounding (`compound_cycles=True`) is catastrophic on the single-seed H4 ablation. |
 | 15 | **GNN prototype** | **OPEN** (no merged PR; no PyG/DGL code — only a docstring forward-ref in `evidence_dag.py`) | Lightweight GNN over evidence DAG (PyG or DGL); only after steps 12–14 green; must beat flat-pool baseline on drift fixture or fail closed |
 | 16 | **Quant-aware shim routing** | **OPEN** (pieces exist: `adapter_router`, `QuantizationPromotionGate`, route gate #285 — **not** integrated as one retrieval-fitness steering plane) | `adapter_router.py` + `QuantizationPromotionGate` integrated as steering plane; promotion requires quant survival + retrieval fitness |

docs/rung13-disintegration-loop.md

@@ -0,0 +1,93 @@
+# Rung 13: detector-driven Evidence-DAG disintegration
+
+## Detector to fitness mapping
+
+`IsomerDetector` exposes a real per-query signal: `compute_isomer_strength` is
+`1 - Jaccard` (`isomer_detector.py:75`) and each immediate `detect_*` result retains
+the exact query key and strength (`isomer_detector.py:135`, `:164`).
+`detector_signals_from_outputs` joins that key exactly to an Evidence-DAG query
+node's `node_id`, `query_id`, or `query_text`. Query fitness is:
+
+```text
+isomer fitness = 1 - isomer strength = Jaccard similarity
+```
+
+A cluster's isomer component is the minimum fitness of its joined member queries.
+No exact join means neutral fitness `1.0`; an empty detector result with explicit
+`mode="sedimentation"` cannot trigger pruning. Missing mode fails closed with an
+exception, as does any mode override or string subclass: both the expected and
+emitted modes must be the exact built-in string `"sedimentation"`, because no
+other sign convention is validated.
+
+`ConvergenceMonitor` exposes one training-run summary (`convergence_monitor.py:128`),
+not a per-query or per-cluster result. Selective scoring therefore requires callers
+to maintain one real monitor per cluster and pass
+`{cluster_node_id: monitor.get_summary()}`. The mapping is:
+
+```text
+before min_epochs                         -> 1.0 (insufficient evidence)
+converged                                 -> 1.0 (stable early-stop plateau)
+mature and not converged                  ->
+    1 - epochs_without_improvement / patience
+```
+
+The edge score is the minimum applicable query and cluster component. A mature
+run with zero stalled epochs therefore remains healthy. A run approaching its
+patience boundary falls toward zero; once it reaches the monitor's real converged
+state, that recovered stability can re-anneal a ledgered edge.
+
+Two production-wiring limits remain explicit. The engine creates convergence
+monitors as local training variables (`antigravity_engine.py:1989`, `:2323`) and
+does not currently persist their summaries. It also retains only aggregate isomer
+history, while selective scoring needs the immediate per-query `detect_*` return.
+This rung supplies the genuine scorer/prune/re-anneal contract for those real
+outputs; it does not claim the engine automatically invokes it yet.
+
+## Prune and re-anneal contract
+
+`EvidenceDAG.prune_edges(scorer, threshold, dry_run=False)` scores every edge once,
+removes only scores strictly below the threshold, never removes nodes, validates
+the graph before and after mutation, and records detached copies of removed edges
+in a runtime-only ledger. `dry_run=True` reports identical proposed decisions without
+changing the graph or ledger. Thresholds must be exact built-in integers/floats,
+finite, and in `[0, 1]`; `dry_run` must be an exact built-in boolean. Invalid
+subclasses or objects fail before callbacks or conversion hooks. Scorer and custom
+protection callbacks receive separate detached edge views, so retaining or
+mutating a view cannot alter active, pruned, retained, or dry-run state. A callback
+that directly adds/removes/mutates nodes, replaces/removes/reorders/mutates private
+edges, or mutates the pruned ledger through another reference is rejected against
+a joint state snapshot. Nodes, active edges, and the ledger are restored together
+on returned mutation or exception.
+
+`reanneal_edges(dag, scorer, threshold, recovered_signal)` considers only ledgered
+edges and applies the same exact built-in threshold contract. It rejects an
+invalid starting DAG before scoring. Scoring itself runs inside the same complete
+transaction: callback mutation or exceptions restore nodes, active edges, and the
+ledger. It then restores scores at or above the threshold, validates each
+restoration, and keeps failed/low or individually invalid ledger edges in the
+ledger.
+
+Node and edge attributes follow the serialized schema: plain JSON objects composed
+only of string keys, null, booleans, strings, integers, finite floats, lists, and
+objects. Inputs are recursively validated and detached at ingestion; custom Python
+objects, container subclasses, tuples, non-finite floats, non-string keys, and
+cycles fail closed. Transaction snapshots use the same hook-free copier rather
+than Python `deepcopy`; type checks use identity rather than equality and error
+paths do not inspect untrusted type metadata, so objects cannot execute copy,
+metaclass-equality, or type-name hooks. Ledgered edges and returned
+prune/re-anneal records are detached from live edge and nested attribute aliases.
+
+`write_disintegration_artifact` writes the prune and recovery thresholds, detector
+provenance, edge-level fitness before/after, pruned edges, re-annealed edges, and
+protected skips as deterministic JSON using an atomic same-directory replace.
+
+## Fail-closed guard
+
+The default protection treats `EdgeType.OPERATES_ON` as the current schema's
+structural actuator-to-cluster edge. Any edge with `required=True` or
+`structural=True` is also protected. Mandatory protection is always ORed with a
+caller-supplied downstream `protected_predicate`, so the custom predicate can only
+add protection and cannot weaken it. Protected edges remain in the graph even when
+their detector score is low, and the artifact records the skip. With all detector
+signals healthy or with an explicitly sedimentation-mode empty result, every score
+is `1.0` and pruning is a no-op.