feat: add local model synthesis providers - #22
Merged
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015LY7xtercMc3NtLeX2z4YV
PR #21 landed docs/decisions/0006-claude-code-install-target.md, colliding with the model-synthesis ADR. Renumber to 0007 and update all references. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015LY7xtercMc3NtLeX2z4YV
- remote.rs: disable HTTP redirects (max_redirects(0) + will_error(false)) and treat any 3xx as a transport error, so a loopback endpoint cannot 3xx-redirect evidence to a non-loopback host past the locality guard. Add mock-server test. - Add a test asserting the transport-error path never echoes the Bearer key. - ADR 0007: correct migration reference to 0008_mutation_provenance. - Prompt test: assert a raw-payload corpus marker is absent from the prompt. - Tighten the prompt-size bound to <=750 approx tokens to pin the documented figure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015LY7xtercMc3NtLeX2z4YV
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claim
autophagy mutations synthesizenow works with a real local model. Two HTTPproviders —
ollama(/api/chatwith a JSON Schemaformat) andopenai-compatible(/v1/chat/completionswithresponse_formatjson_schema)— sit behind the existing synthesis boundary via a small blocking
ureqclient(no async runtime). A model can enrich candidate wording but still cannot invent
evidence, widen a trigger, escalate permissions, or emit a candidate the
mutation contract rejects. The deterministic, offline, model-free path is
unchanged and remains the default. This is the "future PR" ADR 0004 anticipated,
so it versions the mutation contract (
mutation/0.2) to record model provenance.Evidence (test → behavior)
New
crates/autophagy-synthesis/tests/http_providers.rs(hand-rolledTcpListenermock, no new dep):ollama_happy_path_produces_a_v0_2_candidate/openai_happy_path_...— validmodel JSON → contract-valid
mutation/0.2candidate with provenance + token usage.garbage_json_is_a_structured_decline_not_a_panic— unparseable model output →structured
ProviderDeclined(usage still surfaced), never a panic.fabricated_evidence_flows_into_the_rejection_path— invented evidence/selector/network → existing
Rejecteddiagnostics (unknown_evidence,unknown_selector,excessive_permission).timeout_surfaces_a_clean_provider_error/connection_refused_...— transportfailures → clean
ProviderError, never a panic.non_loopback_endpoint_is_refused_without_the_flag/..._allowed_with_the_flag—loopback default enforced; remote only past the explicit opt-in.
api_key_is_read_from_env_and_sent_as_bearer_never_leaked/missing_api_key_env_is_a_clean_error_that_names_the_var_not_the_key— key readfrom env, sent only as
Authorization: Bearer, never in the outcome; missing varnames the variable, not a key.
prompt_is_deterministic_and_carries_only_template_fields/measured_prompt_size_over_the_fixture_corpus_is_bounded— prompt is built onlyfrom template fields and is small.
In
boundary.rs:mutation_v0_2_schema_*,mutation_v0_1_fixtures_still_validate_*(v0.1 round-trips byte-for-byte),
manifest_v0_2_*,v0_1_manifest_rejects_v0_2_only_fields.In
remote.rsunit tests: loopback classification incl. IPv6 and userinfo-spoof.In store
migration.rs:provenance_migration_accepts_v0_2_and_preserves_v0_1.All pre-existing boundary/store tests stay green;
mise run checkpasses.Offline smoke: deterministic provider registers candidates against a fixture DB;
an
ollamaprovider against a dead loopback port surfacesprovider ollama error ... Connection refusedper finding (clean, no panic).Privacy
(
localhost,127.0.0.0/8,::1); a non-loopback host is refused unless--allow-remote-endpointis passed, which also prints a warning.deterministic template's baseline text, the hard constraints, and the cited
event identifiers — to the configured endpoint. No raw event payloads,
transcripts, or secrets. Measured on the fixture corpus at ~693 prompt tokens
per candidate (max); response capped at 1024 tokens. A test asserts the prompt
carries only template fields.
--allow-remote-endpoint. API keys are never stored in the manifest/DB/logs/output —
api_key_envnames an environment variable; the key is read at calltime and sent only as an
Authorization: Bearerheader (tested, including thaterror paths do not echo it).
Compatibility (mutation/0.2)
mutation/0.2=mutation/0.1+ a requiredprovenanceblock(
provider,model_name,model_revision, optionalmodel_digest,manifest_spec_version) andgenerated_by: model_synthesis_v1. New schema +valid/invalid fixtures at
docs/specs/mutation/0.2/.MutationPackagegainsOption<Provenance>(serde default, skipped when absent),so every v0.1 package parses, validates, and round-trips byte-for-byte. A
cross-field validator enforces the pairing (0.1 ⇔ deterministic ⇔ no provenance;
0.2 ⇔ model synthesis ⇔ provenance). The deterministic provider and template
generator still emit v0.1 unchanged — the offline default does not move to v0.2.
0008_mutation_provenance.sqlrelaxes only themutation_candidates.spec_versionCHECK to('mutation/0.1','mutation/0.2')via the established recreate-copy-rename procedure (new ordered migration; old
ones untouched; every row copied verbatim; unknown versions still rejected).
Schema version → 8;
apps/macosknownSchemaVersionand store tests bumped.synthesis-manifest/0.2(optionaltimeouts,api_key_env) atdocs/specs/synthesis/0.2/; the synthesis response shape is unchanged(
synthesis/0.1). Rationale indocs/decisions/0007-model-synthesis-providers.md.🤖 Generated with Claude Code