Bahi.
Voice-first shop ledger · every speech and LLM vendor swappable by config · one eval suite for all
“Most voice-agent demos pick one vendor and hope. Bahi treats the vendors as replaceable parts and lets the evals choose.”
Speech stacks for Indian languages differ a lot by task — one vendor transcribes better, another speaks better, another reasons better — and the answer changes every few months.
Put every vendor behind an interface and pick per role in config. Then make the eval suite the judge: one YAML suite, run against every provider, scored against the ledger state the turn should have produced, plus word error rate, latency and cost.
The choices that mattered, with the reasoning at the time.
- Vendors live only in adapters
A test fails the build if any vendor SDK is imported outside its adapter, so swapping Sarvam for ElevenLabs is a config change, not a refactor.
decision · 01 - Score the ledger, not the transcript
A turn passes only if the resulting ledger rows match, which catches the agent that sounded right and wrote the wrong amount.
decision · 02
On the all-Sarvam 42-turn suite: 97.6% tool-call correctness, 95.2% task success, about ₹0.02 a turn, with WER and p50/p95 latency reported per provider.
- tool-call correctness
- 97.6%
- task success
- 95.2%
- cost per turn
- ~₹0.02