Improve control-driven E2E sessions and editor state

Author: gajop

docs/design/e2e.md

@@ -5,28 +5,33 @@ description: How the SBC E2E harness drives a real editor session and records re
 
 # Graphical end-to-end testing
 
-The E2E harness tests the editor as a user sees it. It launches an isolated
-Spring process, finds its window, drives mouse and keyboard input through X11,
-and records the observable result. It complements the headless Smoke suite:
+The E2E harness tests the editor as a user sees it. It launches Spring, finds
+its window, drives mouse and keyboard input through X11, and records the
+observable result. Grouped runs reuse one resettable process per compatible
+launch environment by default; scenarios marked `@scenario(isolated=True)` get
+their own process. It
+complements the headless Smoke suite:
 Smoke proves native integration and in-engine tests; E2E proves UI ownership,
 input routing, and rendered behaviour.
 
-Scenarios that are about a feature rather than about a widget belong in
-`scenarios/control.py`, driven through the control channel instead of X11 — no
-coordinates, no sleeps, and several times faster. See
+Scenarios that are about a feature rather than about a widget belong in the
+domain modules under `tools/e2e/scenarios/`, driven through the control channel
+instead of X11 — no coordinates, no sleeps, and several times faster. See
 [programmatic-control.md](programmatic-control.md).
 
 ## Run a scenario
 
 ```bash
-just test-e2e map-export
-just test-e2e map-export --tag ui:rust
+just test-e2e map-workflows
+just test-e2e map-workflows --tag ui
 ```
 
-The runner writes one directory under `artifacts/ui-e2e/` per case. It contains
-the scenario report, input/event trace, command trace, engine log, and the
-individual screenshots taken at meaningful checkpoints. Review images at their
-native resolution.
+The runner writes the scenario report, input/event trace, command trace, engine
+log, and individual screenshots under `artifacts/ui-e2e/`. A focused invocation
+uses one run directory and writes `suite-report.md` beside it. A grouped
+invocation puts its scenario directories and one aggregate `suite-report.md`
+under a timestamped suite directory; every run report links back to that
+summary. Review images at their native resolution.
 
 ## What a scenario may assert
 
@@ -74,7 +79,9 @@ tool is required.
 
 ## Lifecycle
 
-Every run owns a temporary Spring write directory. It is removed after logs
-and screenshots have been copied to the artifact directory. `--keep-open`
-preserves the process and write directory for diagnosis; it is not normal test
-behaviour.
+Each process owns a temporary Spring write directory. Shared cases with the
+same launch environment reuse it after an undo/reload reset; isolated cases
+get a fresh one. The directory is removed after logs and screenshots have been
+copied to the artifact directory.
+`--keep-open` preserves the process and write directory for diagnosis; it is
+not normal test behaviour.

docs/design/programmatic-control.md

@@ -9,7 +9,8 @@ A control channel lets a script drive a running SpringBoard the way a user
 drives it — open an editor, set its fields, run a command, place the camera,
 capture the result — without synthesising X11 input.
 
-Status: **design, not built.**
+Status: **implemented for editor fields, registered commands, camera state, and
+ordered capture; domain handles are the next layer.**
 
 ## Why
 
@@ -41,6 +42,11 @@ requests cross to the main thread on a queue drained once per `update()`;
 replies return the same way. One request per line, no JSON-RPC batches —
 requests on a connection apply in order and reply in order.
 
+Project reloads replace the native module and therefore close existing sockets.
+The Python client detects that replacement, waits for the new `instance_id`,
+refreshes the schema, and reconnects subsequent calls; a caller does not need
+to rebuild its editor or camera handles.
+
 ## Three surfaces, all first-class
 
 **Editors** — `ui.open(tab, editor)`, `ui.set(editor, field, value)`,
@@ -64,7 +70,13 @@ is what makes binding fail fast: a client resolves its handles at connect time,
 so an unknown editor or field raises immediately with a list of what exists,
 rather than silently doing nothing three minutes into a run.
 
-Alongside these: `camera.set/get` (position, target, fov) and `capture(path)`.
+Alongside these: `camera.set/get` (rendered position, direction, fov, and
+controller distance/height), `camera.trace_screen_ray(x, y)`,
+`camera.zoom(factor)`, `capture(path)`,
+`runtime.barrier()` (two input-idle native updates, for ordering external input), and
+`runtime.reload_native_modules()` for native-module lifecycle testing. The
+reload deliberately preserves engine world/project state; it is not a project
+reset.
 
 ## The Python client
 
@@ -89,7 +101,7 @@ with control.connect(write_dir) as sb:
     lighting.groundDiffuseColor = (0.9, 0.45, 0.2, 1.0)  # and here, listing the fields
     lighting.shadowMode = "Full"
 
-    sb.camera.set(position=(2048, 900, 2048), target=(2048, 0, 2048))
+    sb.camera.set(position=(2048, 900, 2048), direction=(0, -1, 0), height=900)
     sun(dirX=0.5)
     sb.capture(out / "lit.png")
 ```
@@ -125,18 +137,20 @@ other:
 
 A reply is sent once its effect has landed, not once the request was accepted:
 `ui.open` answers when the editor is on screen, `capture` when the image is on
-disk. Requests on a connection apply and answer in order, and `capture` is
-queued at `draw_screen_post`, so an image already contains every call before it.
-No scenario needs a sleep to make a screenshot honest.
+disk, and `runtime.barrier` after two input-idle native updates. Requests on a
+connection apply and answer in order, and `capture` is queued at
+`draw_screen_post`, so an image already contains every call before it. No
+scenario needs a sleep to make a screenshot honest.
 
 Unknown method, unknown editor, unknown field, out-of-range value, and failed
 deserialisation are all structured errors returned before anything is applied.
 Nothing in this channel may fail by doing nothing.
 
 ## Scope
 
-**Built**: `describe`, `ui.open/set/get`, `command.execute`, `camera.set/get`,
-`capture`. `SBC_CONTROL_FILE` names the discovery file and turns the channel on;
+**Built**: `describe`, `ui.open/set/get`, `command.execute`,
+`camera.set/get/trace_screen_ray/zoom`,
+`capture`, `runtime.barrier`, `runtime.reload_native_modules`. `SBC_CONTROL_FILE` names the discovery file and turns the channel on;
 the E2E harness sets it per run and exposes the connection as
 `run_state.control`.
 
@@ -153,7 +167,7 @@ widget needs a real display server, and stays with the X11 harness.
 ## Relationship to the X11 harness
 
 The two coexist. Feature scenarios — set things up, run the feature, look at the
-result — move to `tools/e2e/scenarios/control.py` and get faster and
-layout-independent: `lighting` there covers what `env.lighting_panel` covers, in
+result — move to the relevant domain module under `tools/e2e/scenarios/` and get
+faster and layout-independent: `lighting` there covers the environment lighting domain, in
 2s of scenario time against 14s. Scenarios genuinely about input routing, focus
 and hit-testing keep clicking. See [e2e.md](e2e.md).

docs/porting/README.md

@@ -38,7 +38,7 @@ one small, reviewed domain at a time into `rust-stable`.
 Native UI scenarios run against the real editor window:
 
 ```bash
-just test-e2e <target> --tag ui:rust
+just test-e2e <target> --tag ui
 ```
 
 Every changed golden must be visually inspected before it is recorded as

docs/porting/e2e-control-port.md

@@ -0,0 +1,83 @@
+# E2E control-channel port
+
+Status: in progress — Environment, Scenario Info, Map editor fields, and the
+non-gesture setup in brush/state scenarios are now control-driven.
+
+The control channel should replace X11 in every domain-editor scenario that is
+setting editor state, running an editor feature, asserting emitted commands, or
+positioning the camera for a capture. Domain-editor scenarios do not test their
+own widget interaction. X11 remains the authority for the separate UI suite:
+input routing, hit testing, hover, text editing, drag/stroke behaviour, and
+dialog/widget layout.
+
+Scenarios share a resettable Spring process with compatible launch environments
+by default. Scenarios that change the project or require a fresh filesystem
+opt into `@scenario(isolated=True)` and get their own process.
+
+## Current API
+
+Available now: typed editor `open`/`get`/`set`, typed registered-command
+execution, camera `get`/`set`/`zoom`, ordered captures, and a reset boundary that
+undoes native history before reloading native modules. This already removes
+coordinates and waits from semantic editor tests.
+
+Asset fields currently travel as schema text; a semantic, validated asset value
+is not available yet. Editor action buttons, object create/select/place
+operations, project/file-dialog operations, and synthetic in-engine input are
+also not available. Those boundaries determine the partial and blocked rows
+below.
+
+## Proposed migration order
+
+| Status | Scenarios | Plan |
+| --- | --- | --- |
+| Done | `lighting`, `sky`, `water`, `info-panel`, `map-editors` | Domain fields and emitted commands use control only. |
+| Partial | `heightmap`, `texture-paint`, `metal-paint`, `grass-paint` | Scalar/editor setup uses control; the actual stroke, asset-grid choice, action selection, or project dialog remains X11. |
+| UI / blocked | `teams-panel`, `settings-panel` | Teams needs typed add/select/edit handles before it can become a domain test. Settings owns the shading-dialog interaction contract. |
+| Second batch | `props-panel`, `collision`, `units-panel`, `feature-placement-actions`, `brush-size`, `project-workflows`, `map-workflows` | Convert the domain state/command portion once object/project helpers exist. Move the remaining input setup out to focused UI coverage. |
+| Camera-only cleanup | `pattern-preview`, `texture-paint`, `metal-paint`, `grass-paint`, `cursortip`, `deselect`, `rotation`, `selection-drag`, `object-actions`, `selection`, `clipboard-actions` | Done: deterministic framing uses `camera.zoom`; no domain scenario uses mouse-wheel camera zoom. A dedicated camera-wheel test is intentionally out of scope for now. |
+| Keep X11 | `terrain-stationary-hold`, `heightmap`, all `chonsole-*`, `module-reload`, `developer-console`, `developer-console-copy`, `gallery`, `gallery-pickers`, `gallery-tooltips`, `gallery-dialogs`, `main-panel`, `hide-interface`, `all-editors`, `panel-tabs-are-choices`, `import-action`, `dialogs`, `project-status-bar`, `notifications`, `export-warning`, `ui-sweep`, `def-grid`, `feature-grid-tooltip-after-cursortip` | These explicitly test keyboard focus, pointer routing, held strokes, hover, clicking a control, modal layout, or rendering. Replacing their core interaction would stop testing what they exist to test. |
+
+The former water UI scenario's terrain-basin stroke and `/water 4` input belong in focused
+stroke/console tests, not in the water domain test. `map_export` and
+`map_roundtrip` remain blocked until a project API is deliberately designed.
+
+## Focused UI suite
+
+The retained X11 scenarios should become a compact, cross-cutting UI suite:
+one test per widget/interaction contract, not one copy per domain editor. The
+existing gallery, picker, tooltip, dialog, Chonsole, input-state, toolbar, and
+visual-sweep scenarios are its nucleus. When a domain scenario gives up a
+click/drag/picker assertion, move that coverage here only if no existing UI
+scenario already covers the same generic contract. `field_modal_handoff` is
+the first such extraction: it covers modal-binding cleanup across editors
+without being presented as Scenario Info coverage.
+
+## API additions worth designing before the second batch
+
+1. Typed asset fields: an `AssetPath`/asset-reference value accepted by
+   `Editor.set`, validated against the live field schema.
+2. Typed editor actions: invoke a named registered editor action through the
+   same behaviour path as a button, with schema discovery and no raw strings.
+3. Object domain handles: create, select, inspect, and mutate objects without
+   pretending those operations are generic editor fields.
+4. Project domain handles: create/save/load/export by typed request. Do not
+   expose file-dialog clicks as an API.
+
+Do not add `input.*` merely to avoid X11 in the retained scenarios. Its job is
+to test the native input state machine and belongs only where pointer/keyboard
+semantics are themselves under test.
+
+## Progress
+
+| Batch | Status | Notes |
+| --- | --- | --- |
+| Environment | done | `lighting`, `sky`, and `water` now cover their editor domain state. |
+| Scenario Info | done | `info-panel` now sets and reads metadata through control. |
+| Map editor fields | done | `map-editors` covers terrain, texture, metal, grass, and rendering fields through control. |
+| Brush/state setup | partial | Scalar setup is control-driven; pointer gestures and picker/action contracts remain X11. |
+| Focused UI suite | existing, to consolidate | Own generic editor/widget interaction coverage. |
+| Team domain | blocked | Needs typed add/select/edit API; keep its current test explicitly UI until then. |
+| API additions | proposed | Design only when the second batch is reached. |
+| Second batch | blocked | Depends on typed object/project/action support. |
+| Camera cleanup | done | Domain/object setup uses the typed camera surface; the only remaining wheel event is Chonsole suggestion-list scrolling. |

docs/porting/native-e2e-review.md

@@ -20,7 +20,7 @@ None.
 | `selection` | Reviewed and recaptured for the versioned 1371px window geometry. A subsequent regression holds left during a box drag, right-clicks to cancel it, checks that the transient outline is gone, then completes a fresh box selection. |
 | `feature-placement-actions` | Its panel-local visual comparison was using full-window coordinates; corrected crop coordinates now assert the Add → Brush change. |
 | `gallery`, `gallery-dialogs`, `gallery-pickers`, `gallery-tooltips` | Reviewed the current fields, drag, dialogs, pickers, and tooltips; stale references were recaptured. |
-| `native-dev-console` | Reviewed the current console/status presentation, including the live line count; stale references were recaptured. |
+| `developer-console` | Reviewed the current console/status presentation, including the live line count; stale references were recaptured. |
 | `props-panel` | Form references were updated for full-width controls. The outside-drag assertion now unambiguously marks the releasing drag and tolerates only the documented live numeric glyph variation. |
 | `units-panel` | Reviewed panel and world output; stale references were recaptured. |
 

docs/porting/verification.md

@@ -30,7 +30,7 @@ Rules:
 Run a native target with:
 
 ```bash
-just test-e2e <target> --tag ui:rust
+just test-e2e <target> --tag ui
 ```
 
 ## Editors
@@ -46,9 +46,9 @@ just test-e2e <target> --tag ui:rust
 | 7 | Map → Metal | VERIFIED | Pattern, size, rotation, amount, and painting. E2E `map-editors`, `metal-paint`. |
 | 8 | Map → Grass | VERIFIED | Pattern, detail, size, rotation, and painting. E2E `map-editors`, `grass-paint`. |
 | 9 | Map → Settings | VERIFIED | Rendering flags, splat fields, detail texture, and New/Existing texture paths. E2E `map-editors`, `settings-panel`. |
-| 10 | Env → Lighting | VERIFIED | Shadow mode, direction, six colours, and densities. E2E `lighting-panel`. |
-| 11 | Env → Sky | VERIFIED | Atmosphere colours, fog bounds, and skybox picker. E2E `sky-panel`. |
-| 12 | Env → Water | VERIFIED | Scalars, booleans, colours, Normal/Foam/base textures, and visible water. E2E `water-panel`. |
+| 10 | Env → Lighting | VERIFIED | Shadow mode, direction, six colours, and densities through typed control. E2E `lighting`. |
+| 11 | Env → Sky | VERIFIED | Atmosphere colours and fog bounds through typed control. E2E `sky`; skybox picker remains UI coverage. |
+| 12 | Env → Water | VERIFIED | Scalars, booleans, colours, and texture fields through typed control. E2E `water`; visible water and pickers remain UI coverage. |
 | 13 | Misc → Info | VERIFIED | Name, description, version, and author commit a complete scenario record. E2E `info-panel`. |
 | 14 | Misc → Teams | VERIFIED | Roster, add/remove, edit modal, resources, colour, position, side, and row refresh. E2E `teams-panel`. |
 
@@ -73,7 +73,7 @@ just test-e2e <target> --tag ui:rust
 | 36b | Cursor tooltip | VERIFIED | Empty-ground suppression, feature/unit content, and map hover. E2E `cursortip`. |
 | 37 | Pattern brush preview | VERIFIED | Fresh E2E `pattern-preview` (2026-07-18): a Terrain/Add state without a pattern had no footprint; choosing the pattern produced the inspected textured ground projection beneath the same cursor. |
 | 38 | Ray-trace agreement | TODO | Existing selection tests do not prove click, drag, and preview resolve the same point. |
-| 39 | Developer console | VERIFIED | Fresh E2E `native-dev-console` and `native-dev-console-copy` (2026-07-18): reviewed cleared/visible/hidden/Problems screens, live status controls, multiline selection, Ctrl+C, and Ctrl+A copy ownership. |
+| 39 | Developer console | VERIFIED | Fresh E2E `developer-console` and `developer-console-copy` (2026-07-18): reviewed cleared/visible/hidden/Problems screens, live status controls, multiline selection, Ctrl+C, and Ctrl+A copy ownership. |
 | 40 | Chonsole | DONE | Dedicated scenarios cover editing, completion cycling, mouse hover/click, scroll, texture/rule completion, persistence, and native reload. Re-run and inspect them as one current verification pass. |
 | 41 | Status strip | DONE | Metrics, command journal, undo/redo/clear controls, and a status golden are implemented. Re-run with developer console and inspect layout/current metrics. |
 

justfile

@@ -144,18 +144,28 @@ dev-panel config="config/ui-rust.json": build-native
 run config="config/ui-rust.json": build-native
     PYTHONPATH="{{tool_pythonpath}}" uv run --locked sbc-smoke manual --config "{{config}}"
 
-# Run black-box UI E2E tests. Does not rebuild native code; run `just build`
-# first when testing Rust UI changes.
+# Run black-box UI E2E tests. The native plugin is rebuilt first. Multi-case
+# invocations store all scenario artifacts and the aggregate suite report under
+# one timestamped suite folder.
 [group('test')]
-test-e2e target *args:
+test-e2e target *args: build-native
     PYTHONPATH="{{tool_pythonpath}}" uv run --locked sbc-e2e run "{{target}}" {{args}}
 
+# `just test-e2e all` shares compatible launch environments by default and
+# isolates only scenarios explicitly marked `@scenario(isolated=True)`.
+
+# Summarise existing E2E artifacts without launching Spring. For a bounded run:
+# `just e2e-report --after 20260730-115416 --before 20260730-121013`.
+[group('test')]
+e2e-report *args:
+    PYTHONPATH="{{tool_pythonpath}}" uv run --locked sbc-e2e report {{args}}
+
 # List every reference image and whether it is approved or still ai-reviewed.
 [group('test')]
 goldens-status:
     @PYTHONPATH="{{tool_pythonpath}}" uv run --locked sbc-e2e goldens-status
 
-# Approve a case's reference images (the human OK): `just goldens-approve rotation-rust`.
+# Approve a case's reference images (the human OK): `just goldens-approve rotation`.
 # Optionally name individual shots. Only a human runs this.
 [group('test')]
 goldens-approve case *shots:

native/src/sbc/chonsole/ui/controller.rs

@@ -41,7 +41,7 @@ impl ChonsoleController {
         if self.view.process_suggestion_clicks(core) {
             self.view.refresh(interface, core)?;
         }
-        self.view.process_suggestion_hovers();
+        self.view.process_suggestion_hovers()?;
         self.view.update(interface)
     }
     pub fn draw_screen(&mut self, interface: &NativeInterfaceRef) -> Result<(), Error> {