What is Tazo Labs, and why do we exist?
For years, people predicted that voice would become the universal way to talk to agents and software. In 2026, that prediction is starting to come true. AI now handles billions of business calls through companies like ElevenLabs, Cartesia and Sierra, and 800 products listed in G2’s AI voice assistants category. Consumers are adopting voice too: dictation tools like Wispr Flow, note-takers like Granola (and Abridge in the clinic), and most recently personal AI agents* like Muse and Instinct.
But adoption has outpaced quality. Builders are struggling with cost and reliability at scale, and only about one in five enterprises is very satisfied with its production voice agents. So most production voice agents stick to what is safe: outbound calls that follow a script, or inbound requests simple enough to predict.
Why? Nearly all of them are cascaded: speech-to-text (STT), then a large language model (LLM), then text-to-speech (TTS). Builders choose this design so they can pick the best model for each job, read every intermediate step, and keep tight control over what the agent says and which tools it calls. But inside each component, the builder has little control or visibility. Was the audio transcribed correctly? Why did the LLM do X when it was prompted to do Y? Did the voice come out the way it was meant to?
STT and TTS have improved dramatically and are now stable in cost, latency and accuracy for widely spoken languages.** With models like Fish Audio’s Drama-3, even the delivery of a voice, its emotion and its tone, is becoming controllable.
But today, engineers struggle at the LLM layer, and the tangle of models wired around it. This is where the agent works out what state of the conversation is in, who is speaking, and what to do next. That covers tool calls (web search, database reads and writes, running code), turn-taking, voice activity detection, language identification, speaker diarization, end-of-turn detection, emotion and sentiment, PII detection and data verification, all on top of writing the reply. It is also where most time goes: in the stacks we have seen, this layer accounts for roughly 70 to 80% of the agent’s response time.
Why is a pure LLM layer inefficient?
The obvious move is to hand everything to a frontier LLM like Opus 5.5 or Astra with one giant prompt. That is how most voice AI teams start. They soon find that these models are stateless, unpredictable and slow, and that a long, dense prompt loses instructions. In Distyl AI’s IFScale benchmark, the best frontier models, given 500 instructions in a single report-writing task, followed only 68% of them, and they favoured the instructions that came first.
On top of that, the LLM has to pull the right signal from every utterance, using nothing more than a system prompt and an STT transcript. To reply well, it needs to know:
- When should I respond? Has the user finished, or just paused?
- What language is the user speaking?
- What mood are they in? Are they curious, or angry about bad service?
- Is it a TV in the background, or children playing?
- Is the user talking to me, or to their partner?
- When should I hand this over to a human?
- What does the user want? A refund, a new account, a status check?
- Is the user distracted, and how do I bring them back?
A single LLM call can’t answer all of that well and still reply quickly.
A natural response is to build a system of several smaller LLMs (Haiku, Luna, Qwen-4B) that share a conversation’s memory, to capture signals from the transcript, run errands like tool calls and compile the response. This is the frontier stack today. But it is still a chain of language models reading text, and it still accounts for most of the agent’s response time. That is what keeps voice agents from being as reliable as a person on the phone.
We believe that there is a way to control agents and remove the bloat: decision models.
TypeSafe AI showed with Jev what a generalised classifier, or a decision model, can do. You give it the current state and a set of options, and it returns a choice with a calibrated probability attached, faster and far cheaper than an LLM. By our estimate, 30 to 40% of what an LLM does inside an agent is exactly this kind of work: routing, triage, choosing between bounded options. Hand that work to a decision model and you get a clear answer with measurable confidence, at a fraction of the cost.
But today’s decision models work best on text. TypeSafe itself reports that every model, Jev included, makes worse decisions when its inputs go beyond text, and Jev has no documented audio support.
That is the opening we see: the decision layer for voice AI agents, built for audio from the start. Many of the questions above turn on how something was said, not only on what was said: the pause before the user carries on, the edge in their voice, the TV behind them, the voice that turns away to talk to someone else. A transcript throws all of that away before any model sees it. The right decision model for voice is the one that listens.
At Tazo Labs, our early experiments confirmed that audio-native decision models outperform text-only decision models in numerous high-value use cases.
Graph agents.
Many teams now design an agent as a graph of states: greet, verify identity, handle the refund, escalate. At every turn, the agent has to answer two questions. Which state am I in, and where should I go next? A decision model answers both from the audio and the conversation so far, with a probability for each option, while every transition is logged. When a call goes wrong, you can see which step sent it there.
Speculative tool calling.
People often reveal what they want before they finish the sentence. “I’m calling about the order I placed last week” is already a status check. A decision model can recognise the intent mid-sentence and start a safe, read-only lookup while the user is still talking, so the answer is ready the moment they stop. If the guess is wrong, the results are discarded.
Read morePersonal AI agents.
Personal agents listen to voice notes and spoken commands all day, in kitchens, cars and busy streets. Before they act, they have to decide: is this a task, a reminder or just a passing thought? Was it even meant for me? Should I act now or ask first? These are fast, frequent decisions made on messy audio, exactly what an audio-native decision model is built for.
Read more