Live at trysaathi.vercel.app. Try it in 30 seconds:
open the link, tap the mic, say "I need an income certificate" — or open /bridge and
speak a line in Kannada while the other tile is in Hindi. /w/rider-onboarding and both
/bridge scenarios run end to end with no Sarvam key configured anywhere — see
Try it with no key below.
Saathi is a voice-first workflow engine. A workflow is a JSON file, not code — the engine reads it and runs it as a spoken conversation in the speaker's own language. Two products come off the same runtime:
- One person against a form. Talk, and the form fills itself in, one question at a time, in Kannada, Hindi or English. Out comes a printable sheet or a spoken care card.
- Two people who do not share a language. A call bridge — customer speaks Kannada, delivery partner speaks Hindi — mediated turn by turn until both say the plan back.
The rule the whole system is built around: it refuses to guess. Below a confidence
floor of 0.75 it reads the value back and asks again instead of writing it down.
No chat window, no free-form Q&A. A declarative workflow file drives a person to an artifact: a filled sheet, a care card, a sentence spoken into another human's ear.
Kill test: the product depends on saaras:v3's mode=codemix, the output style
every quickstart leaves at the default. Default mode word-ifies every number; codemix
keeps digits as digits.
| Mode | Hindi output |
|---|---|
transcribe (default) |
…आपका चार हज़ार पाँच सौ का ड्यू डेट पंद्रह तारीख है… |
codemix |
…आपका ₹4500 का due date 15 तारीख है… |
A certificate form is close to 100% entities. Under default mode, the cascade's free layers (L0/L1) stop matching and the sheet stops filling. There is a live toggle in the app that flips this parameter mid-run so you can watch it happen.
Six endpoints, each doing a distinct job on the critical path — remove any row and a named capability breaks.
| Model | Job |
|---|---|
saaras:v3, mode=codemix |
speech → transcript with entities preserved as digits |
/transliterate |
per-token romanisation so Kannada speech matches a Roman-script list |
sarvam-105b-conversations |
L2 free-text extraction and the mediator agent's turns |
mayura:v1 |
live translation on the bridge |
bulbul:v3 |
text-to-speech, per party, in their language and voice |
/text-lid |
language identification per turn |
sarvam-translate:v1 was tried first, was 3× slower and hallucinated (Kannada in,
haircut out); mayura:v1 carries the bridge, sarvam-translate:v1 stays only as
fallback for languages outside Mayura's eleven.
9 page routes, 9 API routes, one job each:
| Route | For |
|---|---|
/ |
speak what you need, opens the right workflow |
/w/[id] → /w/[id]/done |
the form fills itself, then the artifact |
/bridge |
the live call bridge |
/new |
teach a workflow by voice, taps, or JSON |
/catalogue |
every workflow, marked rehearsed or not |
/passport |
which language a party actually speaks, measured not declared |
/how |
the cascade and models, laid out in full |
/import |
opens a workflow shared as a URL, no datastore |
API: /api/identify /fill /speak /compose /bridge /passport /handover
/teach /agent.
- 10 workflows, 5 rehearsed end to end and marked
live(income-certificate,delivery-handover,ride-pickup,medicine-checkin,rider-onboarding); 5 loaded as untested specs. The catalogue (/catalogue) marks each honestly. /bridge— two live call-bridge demos (delivery handover, ride pickup). Each call opens with raw, untranslated speech both sides fail to understand, flags a "Confusion Detected" moment, then goes permanently bilingual. Spoken lines are fuzzy-matched (token coverage + edit distance) against a scripted exchange; anything unscripted falls through to a live Saaras → Mayura → Bulbul pipeline./new— teach a workflow by voice, by tapping, or by pasting JSON.- Mediator agent (
AgentInstruction.md) is plain markdown, read at runtime — edit the file, change the agent's behaviour, no rebuild. - 101 unit tests pass across 7 files;
tsc --noEmitandeslintare both clean.
- Gig and mobility platforms (Swiggy, Zepto, Rapido, Ola, Uber) — language-and-address confusion is a priced, owned cost line; run the bridge against calls that currently end in cancellation, or the onboarding flow against a support queue full of first-week document questions, and count.
- Government form-filling / the ~5 lakh Common Service Centres already charging ₹300–2,000 per application — Saathi is a productivity tool for the operator and a trust layer for the citizen on the same engine.
- NGOs / e-governance departments write their own workflows as JSON files; nobody else has to deploy them.
- Growth loop: a workflow travels as a URL with no datastore behind it — a daughter can send one to her mother, or define the one her mother needs.
- Honest scope: removes preparation trips, not the legally mandated in-person verification. It does not submit to Seva Sindhu.
Engine and content are two different things. lib/workflow.ts (parseWorkflow, the
spec gate every definition passes), lib/registry.ts (loads workflows/*.json at boot),
lib/resolve.ts (the cascade below), lib/validate.ts (the rule interpreter), and
lib/handover.ts (templates the second party's sentence from validated fields, never
generates it) know nothing about certificates. workflows/*.json is content on top of
that engine — deleting every workflow file and adding one back is the whole cost of a new
form.
Resolver cascade, cheapest first:
| Layer | Resolves | Cost |
|---|---|---|
| L0 closed set | district, taluk, gender, purpose, area — fuzzy match + per-turn transliteration | 1 call/turn |
| L1 rules | date, money, time, mobile, Aadhaar (Verhoeff check digit) | free |
| L2 model | free text left over — names, buildings, occupations | paid |
| L3 human | anything under the floor — reads back, asks again | human |
Validation is declarative JSON (verhoeff, mobile-series, date-format, ceiling),
never an LLM call. Corrections are learned per device and sent up as a per-request alias
list — the server never stores them.
Defect-hunt harness: synthesises real speech with Bulbul TTS and drives it back
through /api/fill — 53 cases across every field kind, 3 languages. 46/53 resolved
above the floor; the rest are written up with mechanism and file:line in
docs/defects.md (local, not shipped — kept out of the repo on purpose). Measured, not
estimated: turn latency stayed 1.4–3.4s round trip across a six-turn Kannada run (ASR
0.5–0.7s, resolve 0.8–2.7s) and did not degrade as the transcript grew turn over turn;
/api/identify (homepage speak → workflow) came back correct on 4/4, 0.6–2.0s.
Three defects it caught in production code: a transcript/romanisation concatenation
that fabricated a street number nobody spoke; "female" matching "male" as a
substring at 0.95 confidence; and an L1 extractor that could never resolve a bare
numeric answer to "which flat?" because it required a cue word an answer doesn't carry.
All three are fixed. Also: speaker diarization is Batch-only on Sarvam's REST API,
which is why the bridge is two tap-to-talk lines rather than one shared call.
lib/sarvam.ts checks a committed fixture — keyed by a hash of the exact request —
before touching the network. On a hit it replays instantly with no key; on a miss it
falls through to a live call if SARVAM_API_KEY is set, or a plain error instead of a
crash if it isn't. Recording a fixture is a separate, local, opt-in step
(scripts/record_rider_onboarding.js, scripts/generate_audio.js) — the deployed app
never writes to it.
Two flows were rehearsed this way against the real API and are checked into the repo:
/bridge's two scripted call scenarios, and /w/rider-onboarding end to end (one real
Saaras transcript and one real Bulbul reply per step — lib/resolve.rider-onboarding.test.ts
asserts the resolver still clears the confidence floor on that exact recorded run). Both
run on the hosted link with zero configuration. Everything outside a rehearsed script —
free-form speech on the other workflows, teaching a new one by voice — needs a live
SARVAM_API_KEY.
git clone https://github.com/ashutosh887/saathi
cd saathi
npm install
npm run dev # http://localhost:3000 — the rehearsed flows work with no key
echo "SARVAM_API_KEY=sk_..." > .env # needed for everything else
npm test # 101 unit tests- The bridge is two tap-to-talk lines on one device — bridge logic, not a telephone network. Telephony is the next piece.
- Turn-taking is a fixed RMS silence threshold, not real VAD. Works in a quiet room, can misfire in a noisy hall. Streaming STT/VAD is the fix, not yet wired to the main path.
- TTS dominates latency on the live-fallback path (~5s of a multi-second turn).
Streaming TTS (
speakStreaminlib/sarvam.ts) exists but isn't on the main path yet. - 5 of 10 workflows are specs, not rehearsed runs — the catalogue says so on the page.
- Corrections and runs live in
localStorageper device — nothing is shared across phones yet. That's a datastore away, not a redesign. - The form artifact is a print-ready sheet, not a filled government AcroForm PDF.