Skip to content

Add an occasional real rhasspy/wyoming-piper integration check #23

Description

@sidick

Idea

tests/mock_wyoming_server.py (used by run_golden.sh, run_smoke.sh, run_nondist.sh) is a canned, deterministic stand-in for Piper -- it encodes our own assumptions about the Wyoming wire format (line-delimited JSON header, optional data_length block, payload_length binary payload, persistent multi-request connections). It's good for fast, offline, byte-exact regression testing, but it can't catch a case where those assumptions are subtly wrong and real Piper actually behaves differently.

Complement it with an occasional integration check against the real rhasspy/wyoming-piper Docker image:

  • Content-based, not byte-exact (real Piper output isn't guaranteed byte-identical run-to-run or host-to-host -- same reason we don't do byte-exact Copperline audio golden tests either).
  • Reuse tests/check_audio_wav.py's non-silence/duration approach for verification, and/or a basic wyomingtest/saytest run against it checking for a sane latency + non-empty PCM.
  • Not per-PR: pulling the image + a voice model and paying the one-time model-load latency (~1.8s per CLAUDE.md) is too slow/flaky for every push. Better as a manual workflow_dispatch job or a weekly cron, separate from ci.yml's existing jobs.

Context

Raised during a testing/CI improvement pass on the copperline-migration branch (see tests/run_golden.sh, tests/run_smoke.sh, tests/run_nondist.sh, tests/mock_wyoming_server.py, tests/check_audio_wav.py). Not implemented yet -- filed for later.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions