Skip to content

feat: add local model synthesis providers - #22

Merged
karngyan merged 3 commits into
mainfrom
feat/model-synthesis-providers
Jul 16, 2026
Merged

feat: add local model synthesis providers#22
karngyan merged 3 commits into
mainfrom
feat/model-synthesis-providers

Conversation

@karngyan

@karngyan karngyan commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Claim

autophagy mutations synthesize now works with a real local model. Two HTTP
providers — ollama (/api/chat with a JSON Schema format) and
openai-compatible (/v1/chat/completions with response_format json_schema)
— sit behind the existing synthesis boundary via a small blocking ureq client
(no async runtime). A model can enrich candidate wording but still cannot invent
evidence, widen a trigger, escalate permissions, or emit a candidate the
mutation contract rejects. The deterministic, offline, model-free path is
unchanged and remains the default. This is the "future PR" ADR 0004 anticipated,
so it versions the mutation contract (mutation/0.2) to record model provenance.

Evidence (test → behavior)

New crates/autophagy-synthesis/tests/http_providers.rs (hand-rolled
TcpListener mock, no new dep):

  • ollama_happy_path_produces_a_v0_2_candidate / openai_happy_path_... — valid
    model JSON → contract-valid mutation/0.2 candidate with provenance + token usage.
  • garbage_json_is_a_structured_decline_not_a_panic — unparseable model output →
    structured ProviderDeclined (usage still surfaced), never a panic.
  • fabricated_evidence_flows_into_the_rejection_path — invented evidence/selector/
    network → existing Rejected diagnostics (unknown_evidence, unknown_selector,
    excessive_permission).
  • timeout_surfaces_a_clean_provider_error / connection_refused_... — transport
    failures → clean ProviderError, never a panic.
  • non_loopback_endpoint_is_refused_without_the_flag / ..._allowed_with_the_flag
    loopback default enforced; remote only past the explicit opt-in.
  • api_key_is_read_from_env_and_sent_as_bearer_never_leaked /
    missing_api_key_env_is_a_clean_error_that_names_the_var_not_the_key — key read
    from env, sent only as Authorization: Bearer, never in the outcome; missing var
    names the variable, not a key.
  • prompt_is_deterministic_and_carries_only_template_fields /
    measured_prompt_size_over_the_fixture_corpus_is_bounded — prompt is built only
    from template fields and is small.

In boundary.rs: mutation_v0_2_schema_*, mutation_v0_1_fixtures_still_validate_*
(v0.1 round-trips byte-for-byte), manifest_v0_2_*, v0_1_manifest_rejects_v0_2_only_fields.
In remote.rs unit tests: loopback classification incl. IPv6 and userinfo-spoof.
In store migration.rs: provenance_migration_accepts_v0_2_and_preserves_v0_1.
All pre-existing boundary/store tests stay green; mise run check passes.

Offline smoke: deterministic provider registers candidates against a fixture DB;
an ollama provider against a dead loopback port surfaces
provider ollama error ... Connection refused per finding (clean, no panic).

Privacy

  • Default is offline. HTTP providers require a loopback endpoint
    (localhost, 127.0.0.0/8, ::1); a non-loopback host is refused unless
    --allow-remote-endpoint is passed, which also prints a warning.
  • What is transmitted, and to where: only the structured request — the
    deterministic template's baseline text, the hard constraints, and the cited
    event identifiers — to the configured endpoint. No raw event payloads,
    transcripts, or secrets. Measured on the fixture corpus at ~693 prompt tokens
    per candidate (max); response capped at 1024 tokens. A test asserts the prompt
    carries only template fields.
  • Under what flag: loopback needs no flag; a non-loopback endpoint requires
    --allow-remote-endpoint. API keys are never stored in the manifest/DB/logs/
    output — api_key_env names an environment variable; the key is read at call
    time and sent only as an Authorization: Bearer header (tested, including that
    error paths do not echo it).

Compatibility (mutation/0.2)

  • mutation/0.2 = mutation/0.1 + a required provenance block
    (provider, model_name, model_revision, optional model_digest,
    manifest_spec_version) and generated_by: model_synthesis_v1. New schema +
    valid/invalid fixtures at docs/specs/mutation/0.2/.
  • MutationPackage gains Option<Provenance> (serde default, skipped when absent),
    so every v0.1 package parses, validates, and round-trips byte-for-byte. A
    cross-field validator enforces the pairing (0.1 ⇔ deterministic ⇔ no provenance;
    0.2 ⇔ model synthesis ⇔ provenance). The deterministic provider and template
    generator still emit v0.1 unchanged — the offline default does not move to v0.2.
  • Store migration 0008_mutation_provenance.sql relaxes only the
    mutation_candidates.spec_version CHECK to ('mutation/0.1','mutation/0.2')
    via the established recreate-copy-rename procedure (new ordered migration; old
    ones untouched; every row copied verbatim; unknown versions still rejected).
    Schema version → 8; apps/macos knownSchemaVersion and store tests bumped.
  • Manifest gains synthesis-manifest/0.2 (optional timeouts, api_key_env) at
    docs/specs/synthesis/0.2/; the synthesis response shape is unchanged
    (synthesis/0.1). Rationale in docs/decisions/0007-model-synthesis-providers.md.

🤖 Generated with Claude Code

karngyan and others added 3 commits July 17, 2026 02:20
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015LY7xtercMc3NtLeX2z4YV
PR #21 landed docs/decisions/0006-claude-code-install-target.md, colliding
with the model-synthesis ADR. Renumber to 0007 and update all references.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015LY7xtercMc3NtLeX2z4YV
- remote.rs: disable HTTP redirects (max_redirects(0) + will_error(false)) and
  treat any 3xx as a transport error, so a loopback endpoint cannot 3xx-redirect
  evidence to a non-loopback host past the locality guard. Add mock-server test.
- Add a test asserting the transport-error path never echoes the Bearer key.
- ADR 0007: correct migration reference to 0008_mutation_provenance.
- Prompt test: assert a raw-payload corpus marker is absent from the prompt.
- Tighten the prompt-size bound to <=750 approx tokens to pin the documented figure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015LY7xtercMc3NtLeX2z4YV
@karngyan
karngyan merged commit 3eeb3f3 into main Jul 16, 2026
2 checks passed
@karngyan
karngyan deleted the feat/model-synthesis-providers branch July 16, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant