streamtts

StreamTTS uses Kokoro ONNX for speech and S2 Lite as the local record store. Each cast is a sequence of records with sentence text, duration, and MP3 bytes.

Requirements

• Jetson with CUDA
• Python 3.10
• uv
• ffmpeg
• s2 CLI
• Model files:
  • models/kokoro-v1.0.onnx
  • models/voices-v1.0.bin

Install

uv sync --frozen
uv pip install --force-reinstall onnxruntime-gpu \
  --extra-index-url https://pypi.jetson-ai-lab.dev/jp6/cu126

Run

uv run streamtts

Open http://localhost:7878.

By default this starts S2 Lite, creates a local basin, and runs the web, worker, and fanout roles in one process.

Useful flags:

uv run streamtts --lite-root var/s2lite-data
uv run streamtts --lite-url http://127.0.0.1:4002 --no-spawn
uv run streamtts --role web
uv run streamtts --role worker
uv run streamtts --role fanout

Production

Copy the example environment file:

cp deploy/systemd/streamtts.env.example deploy/streamtts.env

Run S2 Lite, web, worker, and fanout separately:

s2 lite --local-root var/s2lite-data --port 4002 --no-cors
uv run streamtts --role web --no-spawn --host 127.0.0.1 --port 7878
uv run streamtts --role fanout --no-spawn --host 127.0.0.1 --port 7879
uv run streamtts --role worker --no-spawn --host 127.0.0.1 --port 7880

Deployment files:

• deploy/systemd/*.service
• deploy/systemd/streamtts.env.example
• deploy/cloudflared-streamtts.yml
• deploy/observability/

S2 Lite has no auth. Keep it private:

sudo ./scripts/jetson_secure.sh

Expose only the web and fanout roles through your tunnel or reverse proxy.

Streams

stream	use
jobs	queued generation requests
jobs/_cursor	committed worker offset
progress/done	completed cast receipts
jobs/dead	failed jobs
catalog/<id>	private cast recipe
pub/casts/<id>	public cast records

Public cast records:

event	body
meta	title, voice, character count, created time
start	generation attempt marker
audio	sentence index, text, duration in headers; raw MP3 in the body
eos	terminal record

The browser reads /s2/records?stream=pub/casts/<id>&seq_num=0. Replay and live playback use the same read path.

CLI

echo "hello" | uv run python scripts/infer.py
uv run python scripts/infer.py "read this" --out out.mp3
uv run python scripts/infer.py --watch

Observability

cd deploy/observability
sudo docker compose up -d

Prometheus scrapes:

• http://127.0.0.1:7878/metrics/prom
• http://127.0.0.1:9101/metrics