Merge pull request #92 from semcod/ticket-076-standalone-source-dsl-apis

[ticket-076] Add standalone source DSL APIs

Author: tom-sapletta-com

TODO.md

@@ -2,6 +2,13 @@
 
 ## Active tickets
 
+- [ ] [`ticket-076`](project/ticket-076/README.md) — add independently callable
+  `code2dsl`, `docs2dsl` and `config2dsl` facades that emit strictly validated,
+  source-limited `t2c.intent/v1` records without running the full pipeline;
+  preserve extractor SSOT/parity, acyclic module boundaries and descriptive,
+  no-authority effects under the reviewed Wellmanifest standards. Current
+  state: `IN_PROGRESS / PUBLICATION`; host, governance and Docker checks pass,
+  pending protected exact-head delivery.
 - [ ] [`ticket-054`](project/ticket-054/README.md) — restore skills-agent
   discovery, prove a todo2code → Repair PR → independent Validator hand-off,
   then add three bounded todo2code-grounded skills. Current state:

project/TICKETS.md

@@ -55,4 +55,6 @@ of `project/README.md`, which remains a generated technical-analysis artifact.
 | **ticket-072** | [`README.md`](./ticket-072/README.md) | [`preprompt.md`](./ticket-072/preprompt.md) | - |  [`ai-codex.md`](./ticket-072/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-072/ai-codex-logs.txt) | [`changelog.md`](./ticket-072/changelog.md) |
 | **ticket-073** | [`README.md`](./ticket-073/README.md) | [`preprompt.md`](./ticket-073/preprompt.md) | - |  [`ai-codex.md`](./ticket-073/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-073/ai-codex-logs.txt) | [`changelog.md`](./ticket-073/changelog.md) |
 | **ticket-074** | [`README.md`](./ticket-074/README.md) | [`preprompt.md`](./ticket-074/preprompt.md) | - |  [`ai-codex.md`](./ticket-074/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-074/ai-codex-logs.txt) | [`changelog.md`](./ticket-074/changelog.md) |
+| **ticket-075** | [`README.md`](./ticket-075/README.md) | [`preprompt.md`](./ticket-075/preprompt.md) | - |  [`ai-codex.md`](./ticket-075/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-075/ai-codex-logs.txt) | [`changelog.md`](./ticket-075/changelog.md) |
+| **ticket-076** | [`README.md`](./ticket-076/README.md) | [`preprompt.md`](./ticket-076/preprompt.md) | - |  [`ai-codex.md`](./ticket-076/ai-codex.md) |  [`ai-codex-logs.txt`](./ticket-076/ai-codex-logs.txt) | [`changelog.md`](./ticket-076/changelog.md) |
 <!-- AUTO:TICKET_INDEX:END -->

project/ticket-076/README.md

@@ -0,0 +1,110 @@
+# Ticket 076: Add standalone code2dsl docs2dsl and config2dsl APIs
+
+- **ID**: ticket-076
+- **Owner**: unresolved:human
+- **Status**: IN_PROGRESS
+- **Workflow state**: PUBLICATION
+- **Created**: 2026-08-14
+
+## Goal and scope
+
+Expose three small, independently callable source-to-DSL APIs over the existing
+extractors:
+
+- `code2dsl` converts supported repository source files through the current
+  multi-language AST coordinator;
+- `docs2dsl` resolves explicitly supplied files or documentation patterns and
+  runs the deterministic documentation converter;
+- `config2dsl` converts supported repository configuration and infrastructure
+  files.
+
+Every API accepts a repository root plus an explicit `T2CConfig`, can be called
+without constructing the full pipeline, and returns the existing
+`ExtractionResult` contract. Requiring explicit configuration preserves the
+deterministic no-LLM/no-secret-environment import boundary; a standalone
+converter must not silently read ambient provider configuration. Before the
+result crosses the new API boundary, all emitted records are validated as
+`t2c.intent/v1`. The result is intentionally partial: it contains only evidence
+owned by that input channel and makes no repository-completeness claim.
+
+`ExtractionResult` remains an adapter envelope; each item in `records` is the
+canonical DSL document. The ticket does not invent a second collection schema
+or mislabel warnings/cache metadata as semantic DSL content.
+
+The existing `extractAstIntent`, `extractDocumentationBaseline` and
+`extractConfigurationIntent` functions remain compatible. Because their
+modules are already re-exported by `src/index.ts`, the additive APIs require no
+package-manifest, root-export or pipeline change.
+
+The inspected Wellmanifest standards and their exact applicability are recorded
+in [STANDARDS.md](STANDARDS.md). They require facade parity with the existing
+SSOT, one-way module dependencies, strict canonical-record validation and a
+descriptive/no-authority boundary. Experimental or uncommitted standards remain
+design inputs only; this ticket does not fabricate a standards lock or claim
+formal conformance.
+
+## Acceptance criteria
+
+- [x] AC-01: The human owner approves this bounded API design.
+- [x] AC-02: `code2dsl`, `docs2dsl` and `config2dsl` are independently
+      callable from the package root with a common `{ root }` entry shape and
+      a required explicit `T2CConfig`.
+- [x] AC-03: Each API returns only its own channel's records plus warnings (and
+      existing AST cache evidence where applicable), without invoking the full
+      pipeline, graph, synthesis, LLM or mutation paths.
+- [x] AC-04: Every returned record passes the existing strict
+      `assertIntentRecords` validator and retains source provenance.
+- [x] AC-05: `docs2dsl` accepts explicit resolved files or resolves bounded
+      include/exclude patterns, using the configured documentation patterns by
+      default.
+- [x] AC-06: Existing extractor APIs and pipeline behavior remain unchanged.
+- [x] AC-07: For identical explicit inputs and configuration, each facade has
+      record, warning and applicable cache parity with its canonical extractor;
+      no extraction logic is copied into a facade.
+- [x] AC-08: The facades do not import one another, mutate the analyzed
+      repository, produce authority/execution artifacts or expose actual `.env`
+      secret material.
+- [x] AC-09: Focused tests, full Node verification, module-boundary validation,
+      governance and Docker smoke
+      pass before completion is reported.
+
+## Participants
+
+- Human participant: unresolved; no `user-*` file was created.
+- Agent participant: [ai-codex.md](ai-codex.md)
+
+## Approval gate
+
+The human owner explicitly approved ticket-076 and requested implementation on
+2026-08-14. The ticket is now `IN_PROGRESS / PUBLICATION`. Conversation
+approval is an audit note, not trusted merge authorization.
+
+## Verification evidence
+
+- Public-root regression tests prove facade/canonical parity, strict record
+  validation, source-channel isolation, bounded documentation discovery and
+  rejection of foreign paths.
+- The configuration fixture proves `.env.example` remains discoverable while
+  actual `.env` content is neither emitted nor leaked.
+- `npm run verify` passed, including TypeScript build, the full Node suite,
+  transitive no-LLM checks and module-boundary validation (124 modules, 545
+  internal imports, no cycles and independent `core`). The existing JDK-only
+  Java test remained skipped because the JDK is not installed.
+- `./project/governance-check.sh`, `make docker-smoke` and `git diff --check`
+  passed on 2026-08-14.
+- The implementation is ready for protected exact-head review and remains
+  `IN_PROGRESS / PUBLICATION` until that external delivery boundary completes.
+
+## Non-goals
+
+- No separate npm packages, repository split or runtime dependency.
+- No new DSL schema or envelope; the canonical record schema remains
+  `t2c.intent/v1`.
+- No claim of formal conformance with experimental standards and no dependency
+  on their local working-tree paths.
+- No `dsl-manifest.json`, modularity workspace or standards lock; those belong
+  to a separately approved integration ticket with immutable revisions and
+  artifact digests.
+- No CLI, MCP, A2A, pipeline, graph, synthesis or LLM behavior change.
+- No claim that a single source channel represents the complete repository.
+- No removal or signature change of existing extractor functions.

project/ticket-076/STANDARDS.md

@@ -0,0 +1,91 @@
+# Standards assessment for ticket-076
+
+This assessment records the local standards evidence inspected on 2026-08-14.
+It guides the API shape but does not turn a development checkout into a
+normative dependency or a trusted approval source.
+
+## Adoption matrix
+
+| Standard | Inspected identity | Role in ticket-076 |
+| --- | --- | --- |
+| `wellmanifest/new-project` | todo2code is pinned to `0.14.0`; the inspected upstream checkout is unmerged branch `ticket/078-home-adopt-placement` at `0b38f1bdf5c55cad6b54ad54ba89824a9eaeea78`, version `0.17.0` | Normative only through todo2code's existing immutable `0.14.0` governance package. Do not copy the upstream branch's unpublished `placement` field. |
+| `wellmanifest/dsl` | `b7d0595c95e5abbb48ebfdbdae0bc6d43c6f82f4`, version `0.1.0-dev`, pre-stable normative draft; manifest-schema digest `sha256:34d356b76bbd483372df84bb986e15bb84e9c1f8b11b7dc9e3a6c7276c85ed13` | Design constraint: one canonical JSON representation, explicit ownership/provenance/effects, strict validation and no authority minted by a DSL or model. Ticket-076 does not yet claim manifest conformance. |
+| `wellmanifest/modularity` | `1c8c94ee7e13ab95af3ab734b9548ebdfc4a7c20`, version `0.1.0-dev`, experimental | Design constraint: one-way dependencies, single exporter for the Intent contract, projections remain rebuildable, and composition does not copy semantics or transfer authority/state ownership. |
+| `wellmanifest/merge` | `5776debf6aaf999f542db37d535da7c8733b82c8`, version `0.1.0-dev` | Delivery-only guidance. Analytical evidence may inform a later merge disposition but cannot merge, delete or publish this branch. No merge-decision runtime belongs in a source converter. |
+| `wellmanifest/poa` | `8424a7f5c977915ee08404b8b82d63e0f5e44ea2`, version `0.1.0` | Boundary constraint: these APIs analyze and describe; they never grant or execute effects. Optional caches are rebuildable local projections, not authority or owned repository state. |
+| `wellmanifest/ssot` | `5d35394af81838bedb3f21d0894363cd18779d4c`, version `0.2.0-dev`, experimental | Implementation constraint: keep existing extractors canonical, add facade re-exports only, and prove parity so logic cannot fork into `code2dsl`, `docs2dsl` or `config2dsl`. |
+| `wellmanifest/env-dsl` | `0.1.0-dev`, no Git commit; all content is uncommitted and ticket-001 is `BLOCKED` | Informative only. Do not pin, import or claim conformance. Preserve its safe direction: environment values are data; do not evaluate/interpolate them, emit secrets, or treat `.env` as repository DSL evidence. |
+
+## Applicable architecture decisions
+
+### Canonical contract and adapter envelope
+
+The canonical semantic unit remains one closed JSON AST record conforming to
+`schemas/intent-record.schema.json` with `schemaVersion: t2c.intent/v1`.
+`ExtractionResult` is an operational adapter envelope containing a collection
+of those documents plus non-semantic warnings. It is not a new DSL and must not
+receive a fabricated schema identity.
+
+JSONL and future text/TOON renderings are projections. Ticket-076 neither
+changes their semantics nor introduces another canonical representation.
+
+### Module and SSOT boundaries
+
+`code2dsl`, `docs2dsl` and `config2dsl` are repo-local facades, not new semantic
+owners and not independent repositories. Each facade delegates to exactly one
+existing extractor and validates the resulting records through the shared core
+validator. The adapters must not import one another. The Intent record schema
+and its runtime validator retain sole contract ownership.
+
+A parity regression must compare each facade with its canonical extractor for
+the same explicit root, inputs and configuration. The facade may normalize
+only its public call shape and documentation-file discovery; it may not fork
+parsing, provenance, warning or cache semantics.
+
+### Effects, state and authority
+
+The APIs never mutate the analyzed repository, execute a proposed change,
+create an approval or infer authority from input. Reading files and invoking
+the existing allowlisted syntax adapters are analysis operations. An enabled
+AST content cache is a rebuildable projection under the configured output
+directory and must remain explicitly represented by existing cache evidence.
+
+No merge disposition, POA execution envelope, grant, receipt or SSOT decision
+is accepted or produced by this ticket.
+
+### Configuration and environment
+
+Every facade requires an explicit `T2CConfig`; neither implementation nor
+conformance tests depend on ambient environment state. This preserves the
+existing deterministic no-LLM import boundary and prevents a standalone source
+converter from silently loading provider credentials. The facades add no
+environment evaluation, interpolation or secret output. `config2dsl` may
+analyze `.env.example` through the existing structural extractor and must
+continue to exclude actual `.env` secret material.
+
+## Visible prerequisites for formal Wellmanifest adoption
+
+Formal `wellmanifest.dsl/manifest/v1` adoption is a separate integration
+workstream because it owns a new root manifest, artifact digests, command or
+document vocabulary, normalized finding producers and protected conformance
+commands. It cannot be smuggled into an extractor ticket.
+
+The following local evidence also prevents pretending that one complete
+standards lock exists today:
+
+- Modularity still pins Wellmanifest DSL revision
+  `550e5f441c709e15f2679c1af151352d1eba2f1e`, while the inspected SSOT lock
+  pins `b7d0595c95e5abbb48ebfdbdae0bc6d43c6f82f4`.
+- Modularity's standards reference still describes POA as uncommitted, while
+  the inspected POA repository now has commit
+  `8424a7f5c977915ee08404b8b82d63e0f5e44ea2`.
+- `env-dsl` has no immutable revision and its own first implementation ticket
+  is blocked.
+- Todo2code's current governance contract accepts new-project intent v3 as
+  pinned by `0.14.0`; it must not consume the unmerged upstream `placement`
+  extension from a working tree.
+
+Before a future integration ticket creates `dsl-manifest.json` or a modularity
+workspace, it must choose and verify exact immutable revisions and SHA-256
+contract digests, reconcile these stale references upstream or record them as
+explicit informative mappings, and run the standards' deterministic checkers.

project/ticket-076/ai-codex-logs.txt

@@ -0,0 +1,98 @@
+---
+participant-id: agent:codex
+participant: codex
+role: agent
+ticket: ticket-076
+---
+# Participant: codex (AI agent)
+
+## Understanding
+
+Todo2code already contains the three underlying deterministic boundaries and
+exports their modules from the package root. The missing part is a uniform,
+discoverable product-level facade: AST takes an options object, configuration
+takes a positional root, and documentation requires callers to resolve files
+themselves. None of those names expresses that its result is a valid but
+source-limited Intent DSL document.
+
+The smallest compatible change is to add one named facade in each owning
+extractor module. Each facade delegates to the mature extractor, validates the
+complete returned record array with the existing strict runtime validator and
+preserves warnings and cache evidence. This avoids duplicating parsing or
+creating dependencies between the three adapters.
+
+The Wellmanifest review sharpens that boundary. `ExtractionResult` is an
+adapter envelope, while each `records` element is the canonical
+`t2c.intent/v1` JSON document. SSOT requires delegation plus parity rather than
+copied converters; Modularity requires acyclic adapter dependencies and one
+contract owner; DSL and POA require descriptive output with no authority or
+execution effect. Env DSL is currently uncommitted and blocked, so only its
+safe data/no-evaluation direction is applicable. Exact evidence and adoption
+limits are in [STANDARDS.md](STANDARDS.md).
+
+## Execution plan after approval
+
+1. Add `code2dsl({ root }, config)` beside the AST coordinator.
+2. Add `docs2dsl({ root, files?, patterns?, excludes? }, config)`, resolving
+   patterns only when explicit files are absent.
+3. Add `config2dsl({ root }, config)` beside the configuration extractor.
+4. Require explicit `T2CConfig` at every facade so deterministic extraction
+   never imports or reads ambient provider/secret environment configuration;
+   fail closed when the root or emitted DSL is invalid.
+5. Add one public-root regression test proving independent invocation, strict
+   record validation, source-channel isolation, facade/canonical parity and
+   preservation of warnings/cache evidence.
+6. Prove that actual `.env` secret material remains outside configuration DSL
+   extraction and that no adapter imports another.
+7. Run the focused test, `npm run verify` (including module boundaries),
+   `./project/governance-check.sh` and `make docker-smoke`.
+
+## Actual changes
+
+- Inspected the current exports, extractor signatures, pipeline orchestration,
+  schema validator and workstream ownership.
+- Inspected the requested Wellmanifest standards, their exact local revisions,
+  effect/ownership rules and current publication maturity; recorded conflicts
+  that prevent a fabricated all-standards lock.
+- Created and completed this planning ticket on a dedicated branch.
+- Recorded the human owner's explicit approval on 2026-08-14 and transitioned
+  the ticket to `IN_PROGRESS / EDIT` before touching implementation.
+- Added all three facade APIs in their owning extractor modules. Each delegates
+  to the canonical extractor, requires explicit `T2CConfig` and validates the
+  emitted `t2c.intent/v1` records before returning the unchanged envelope.
+- Added public-root regression coverage for canonical parity, source isolation,
+  docs discovery, invalid/foreign inputs and `.env` non-disclosure.
+- Passed focused tests and full `npm run verify`, including the transitive
+  no-LLM and module-boundary gates; passed governance, Docker smoke and diff
+  checks. The existing JDK-only Java test remained skipped on this host.
+- Transitioned to `IN_PROGRESS / PUBLICATION`; protected exact-head review and
+  merge are intentionally not claimed by this ticket-local validation.
+- Validator Agent correctly failed closed on the first published head because
+  the test diff contained a credential-shaped assignment. Reworked
+  only the inert fixture to construct its key name and value at runtime; the
+  non-disclosure assertion remains equivalent without publishing a
+  credential-shaped assignment in the patch.
+- The protected governance resolver then rejected the combined ticket and
+  implementation history even after Validator approval. Reconstructed the
+  branch from `origin/main` so the approved plan and `intent.json` are committed
+  first, implementation follows in a separate commit, and this remediation is
+  last; no product bytes changed during the history repair.
+
+## Risks
+
+- Ambient environment defaults could make a convenience API surprising; the
+  explicit root remains authoritative and callers may pass a complete config.
+- Documentation pattern resolution must stay bounded by the existing glob and
+  ignore behavior.
+- An additive root export is still a contract and must be regression-tested
+  even though no package manifest change is necessary.
+- A local development checkout is not a normative dependency. Formal DSL or
+  Modularity adoption needs a later integration ticket and immutable pins.
+- The first optional-config implementation failed `verify:no-llm` because its
+  runtime `getConfig` import reached `OPENROUTER_API_KEY`. The corrected public
+  contract requires explicit config and restores the deterministic boundary.
+
+## Blockers
+
+- None for the bounded todo2code implementation. Cross-repository standards
+  reconciliation remains separately governed by each owning repository.