Skip to content

[Tech Debt] The coding-agent skill's bundled YAML schema can drift from docs/yaml-schema.json unnoticed #968

Description

@weiqingy

Background

docs/yaml-schema.json is exported from the Pydantic specs (python -m flink_agents.api.yaml.specs), and two checks already keep it honest: python/flink_agents/api/yaml/tests/test_specs.py:377 asserts it matches the Pydantic export, and api/src/test/java/org/apache/flink/agents/api/yaml/spec/SchemaParityTest.java:52 asserts it matches the Java POJOs.

The coding-agent skill added in #918 ships a third copy so it can answer YAML questions offline, plus a record of where that copy came from:

  • dev/agent-skills/flink-agents-dev/assets/yaml-schema.json, byte-identical to docs/yaml-schema.json today
  • dev/agent-skills/flink-agents-dev/assets/yaml-contracts.yaml, whose main contract records blob_sha: 183cc7ac800cf97d030b75b166ea8146368747d5

At bc55d97e all three agree. git rev-parse HEAD:docs/yaml-schema.json returns that same SHA.

The gap

Neither existing check extends past docs/yaml-schema.json, and nothing regenerates the bundled copy. Regenerating the schema and forgetting the skill assets leaves no failing test, so the drift is only found by someone reading both files.

Two distinct things go stale, with different consequences.

The copy itself goes stale, and an agent consulting it is then taught a schema the repository no longer has. That is the failure the skill exists to prevent.

The recorded blob_sha also goes stale, which is the quieter half. SKILL.md:45 tells agents the bundled main schema "describes only the repository revision that published this skill", and references/local-development.md:62 says a source checkout should fall back to the bundled schema "only when they describe the same revision". Both instructions are decided by that SHA, so a stale value does not just lag. It asserts a provenance that is false, and it does so in the exact place an agent goes to decide whether the bundled copy is safe to trust.

Proposal

A dependency-free check under tools/, run from the existing lint job next to Check AGENTS.md freshness, asserting that the bundled copy and the recorded blob_sha both match docs/yaml-schema.json. This mirrors tools/check-agents-md.py, which guards the same class of hand-copied fact.

The failure message should print the two edits needed, since the fix is mechanical and the person hitting it is usually mid-way through an unrelated schema change.

Out of scope

dev/agent-skills/flink-agents-dev/assets/yaml-schemas/release-0.3.0.json pins ref release-0.3.0 and is expected to differ from the working tree. Only the unversioned main contract should be checked.

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority/majorDefault priority of the PR or issue.tech debt[Issue Type] User-unaware issues, such as code refactor and infrastructure maintenance.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions