M6-9: Introduce severity classification + automation policy framework (no auto-approval)

[akserg]

Goal

Introduce a formal, repository-local severity and automation policy framework that classifies issues and runs by severity and risk, and enforces deterministic guardrails on what automation is allowed.

This milestone explicitly does NOT introduce autonomous behavior.

The purpose is to ensure that:

• high-impact changes receive appropriate scrutiny
• automation remains predictable and reviewable
• teams can apply consistent rules without relying on ad-hoc judgment

The system must encode process expectations, not bypass them.

Scope

This milestone defines policy, classification, and enforcement — not execution.

Included:

• severity classification
• risk classification
• policy definition and storage
• enforcement hooks during approval

Explicitly excluded:

• auto-approval
• auto-apply
• background execution loops
• unattended operation

Definitions

• Severity -> customer/user impact
  (e.g. Critical / High / Medium / Low or S0–S3)

• Risk -> engineering/change risk
  (e.g. Low / Medium / High)

• Policy -> deterministic rules defining:

  • what approvals are required
  • what checks must pass
  • what paths/context are allowed
  • what helper commands are permitted

Policy Configuration

Add a repository-local policy configuration file, one of:

• .llm-orchestrator/policy.json
• policies/default.policy.json

The policy defines rules such as:

• If severity -> Critical:

  • tests must be executed before approval
  • reviewer sign-off field is required
  • allowed paths are restricted

• If risk -> High:

  • manual context verification is required
  • patch apply dry-run is mandatory

Policies must be:

• explicit
• machine-readable
• version-controlled
• offline-capable

Run-Level Classification

Each run must capture:

• selected policy
• severity value
• risk value

Stored in:

• runs/<run_id>/manifest.json
  or
• a dedicated run metadata file

Values must be visible, inspectable, and auditable.

CLI Support

CLI must support setting and updating classification values:

orchestrator set-severity --run <id> --severity High
orchestrator set-risk --run <id> --risk Medium

(Command names may vary, but functionality must exist.)

Enforcement Rules

• approve-plan enforces policy requirements
• approve-task enforces policy requirements
• If required conditions are not met:
• approval is blocked
• error message explains what is missing
• Enforcement is deterministic and cannot be bypassed

Examples:

• required tests not executed -> approval blocked
• required sign-off missing -> approval blocked
• forbidden paths detected -> task invalid

Auditability

All classification changes and policy decisions are logged to:

• audit.log

Logs must include:

• previous value
• new value
• timestamp
• actor (CLI user)

Definition of Done (DoD)

1. Policy Framework Exists

• Policy config format defined and documented
• Policy file is loaded from the repository
• Policy schema is validated

2. Severity and Risk Are First-Class Run Properties

• Severity and risk stored in run state
• Values are visible via CLI/status
• Values persist across commands

3. Enforcement Hooks Are Active

• approve-plan enforces policy
• approve-task enforces policy
• Violations block progress deterministically

4. No Autonomous Behavior Introduced

• No auto-approval
• No auto-apply
• No background execution
• All actions remain human-initiated

5. Offline-First and Deterministic

• No network dependencies
• Same inputs produce same enforcement behavior
• Policy behavior is reproducible across machines

Success Signal

Teams can encode how careful they must be for a given change, and the system will enforce that discipline automatically — without ever approving or executing changes on its own.