Four systems: what they had to do, how they were built, and what they cost to run.
Backend infrastructure so the client can stand up voice support for any company they onboard. My first plan was deliberately vendor-independent: Faster-Whisper Medium, a 4-bit Qwen, XTTS-v2, LangGraph. I costed and latency-tested it, then reversed my own position.
What shipped is Google Cloud STT on latest_short, Gemini 2.0 Flash, Cartesia Sonic and Pipecat, keeping BGE-M3 embeddings, Milvus and Silero VAD from the original design. Independence lost to latency and reliability, and I would make that trade again.
The per-minute number came out of tuning, not procurement: max_tokens=60 to hold answers to phone length, automatic punctuation off, 8 kHz PCM matching Twilio's native rate so nothing is resampled, temperature 0.3. Three profiles switch on an environment variable, so a faster stack can be A/B tested without a redeploy.
The failure worth reporting was a silent one. Retrieval was querying a Milvus collection before anything had been ingested: no crash, no error, just empty results and confident ungrounded answers. It now checks collection existence and row count at startup and, if empty, disables retrieval with a loud log line rather than failing mid-call.
result ✦ ₹2.29 per minute all in, at production quality
stack ✦ pipecat ✦ gemini 2.0 flash ✦ cartesia sonic ✦ bge-m3 ✦ milvus ✦ silero vad ✦ twilio ✦ redis
₹2.29 at ₹95/USD
The brief, in the client's own words, was to “optimise or find alternate to the existing Copilot functionality by improving contextual accuracy and interface responsiveness.” A facilities management platform was paying ₹95,000 a month to Microsoft Copilot Studio to answer questions about their own data, badly.
I built natural language to T-SQL against their Azure SQL schema. Intent classification routes each question to system docs, the database, or out-of-context. Semantic table selection puts only the relevant schemas in front of the model instead of the whole catalogue. Generated SQL carries correction logic, three cache tiers sit behind it — in-memory, disk, Redis — and every call is traced in LangSmith.
The figure on the right is a projection I built per layer before implementation. Spend after deployment tracked it. That is the part I care about: not the saving, but the fact that the model held.
the model totals ₹31,116; the balance to ₹35,000 is infrastructure and is unconfirmed. the two columns are not like-for-like — ₹95,000 was licensing, ₹35,000 is inference plus infra.
model tiering ✦ semantic table selection ✦ three-tier cache ✦ connection pooling
The reported distress rate was climbing, 21 percent to 27 percent year over year, and it read like a market signal. It was 187 spurious labels from companies that had simply not filed their accounts. I traced it to the label definition rather than to the model, and added a reportability gate that holds the rate inside a point.
It is XGBoost, and every prediction carries a SHAP attribution so an analyst can see which financials drove a flag before acting on it. The diagnostic engine sits at 92.155 percent on current-year distress flagging and the predictive engine at 87 percent on T+1 failure, both checked against company annual reports rather than a held-out split alone.
result ✦ 92.155% on current-year distress flagging ✦ 187 false labels removed
A Hinglish calling product for bus operators was hanging up mid-call, missing clear answers and talking over customers. The ask was for a better architecture. I took nine recordings and built a reusable diagnostic first, and scoped the paid deliverable as a labelled failure taxonomy with per-category rates rather than an architecture prescription.
All nine recordings were 8 kHz mono, so true barge-in cannot be measured at all — dual-channel capture was the first thing I asked for. Energy above 3.4 kHz ran between 0.003 and 0.6 percent, confirming hard narrowband. June files were PCM and July files 32 kbps MP3, so the capture format had changed mid-window. Four recordings carried a stall signature: an utterance under 1.2 s followed by more than 2.5 s of silence.
Two of my own numbers moved while I worked. The stall count went from nine to eleven once I tightened the VAD merge window from 300 ms to 80 ms. And I withdrew a claim that one stall cluster sat just before the call ended, after checking the file was 117 seconds long with the cluster in the middle. I would rather show that than a clean story.
result ✦ the build was stopped before the money went in
Three questions and I will tell you whether it is worth a call.