How audio decision models make graph agents reliable

Many teams now design an agent as a graph of states: greet, verify identity, handle the refund, escalate. At every turn, the agent has to answer two questions. Which state am I in, and where should I go next?

Those two answers decide whether the call goes well. This post is about what happens when they’re made by an audio decision model: a probability for each option, a log entry for every transition, and a call that you can debug instead of just re-read.

Why teams build agents as graphs

A single long prompt that has to handle every situation gets brittle. A graph breaks the job into steps. Each state has one task, a short prompt and only the tools it needs. LangGraph models an agent as nodes and edges, with conditional edges that pick the next node from the current state. Pipecat Flows and Retell’s conversation flows do the same for voice, as a graph where each step is a node.

That structure is the point. A refund node can’t be reached before the identity node has finished, because no edge allows it. The graph is the policy.

Where graph agents break: the routing

The graph is only as good as the choice of edge. Someone has to look at the conversation and decide which edge to take, and in most setups that someone is an LLM reading text. In Retell, for example, a prompt transition condition is “a natural-language condition evaluated by the LLM against the conversation.”

That works often, and fails in ways that are hard to see:

• The agent is in the wrong state. The caller already gave their order number three turns ago, but the agent is still asking for it, because a transition never fired.
• The agent takes the wrong edge. A caller says “yeah, that’s me” in a hesitant voice, and the agent moves to the refund node as if identity was verified.
• The agent moves too early or too late. It leaves the verify node while the caller is still reading out a number.
• No edge fits. The caller asks for something the graph never planned for, and the agent forces the closest match.

Each of these looks the same from outside: the call went badly. The usual way to find out why is to read the transcript and guess. A transcript shows what was said, not what the router believed, how sure it was or what else it considered. It also drops what only the audio carries: hesitation, a second voice in the room, a “yes” that sounds like a question.

Without that, you fix the symptom. You add a line to a prompt, the failure moves somewhere else, and you can’t tell if the change helped.

Two questions, answered from the audio

A decision model turns each turn into a pick from a known list. Both of the agent’s questions have that shape.

1. Which state am I in? The options are the nodes of your graph.
2. Where should I go next? The options are the edges out of the current node, plus staying put.

Tazo is an audio-native decision model. You pass in the options with the request, and it answers from the audio and the conversation so far, with a calibrated probability for each option. Because the options are an input, they come straight from your graph. Change the graph and the option list changes with it, with no retraining step in between. We cover how this works in Ask what the moment needs.

Here is one turn from a refund call, written out as the log entry it produces. The numbers are illustrative.

Field	Value
Current state	verify_identity
Options offered	stay, handle_refund, escalate
Probabilities	stay 0.31, handle_refund 0.64, escalate 0.05
Threshold for handle_refund	0.90
Result	Below threshold, so stay and ask again

A transition now has a score and a bar to clear. The same call with a text router would have gone to handle_refund, because 0.64 is the best option and nothing else was on the table.

Use the probabilities as guardrails

Calibrated probabilities mean 0.8 is right about 80% of the time, so you can set a threshold per edge and trust it. We explain why that matters in What the heck is an audio decision model?. In a graph, they give you three useful controls:

• A bar for each edge, set by cost. Moving from greeting to FAQ can be loose. Moving from identity to refund should need a high score. Escalating to a human might sit in between, depending on whether a false alarm or a miss costs more.
• A fallback for the middle. When no option clears its bar, the agent doesn’t have to guess. It can stay in the state and ask a short clarifying question, or hand off.
• Hard rules around the model. The decision model picks among edges that the graph allows. Business rules still apply on top: no edge into handle_refund unless identity_verified is true. The model can be wrong about the audio, but it can’t skip a step.

Log every transition

The log is where reliability comes from. Every transition records:

• the state the agent was in,
• the options it was given,
• the probability of each,
• the threshold, and what was chosen,
• the audio window the decision was made from.

That last field matters most. It means any decision can be replayed and listened to, and compared with what the model said.

This also pairs with post-call analysis. Post-call analysis asks questions about the whole recording. The transition log tells you what the agent decided at each point, so you can line the two up and see exactly where the call and the agent parted ways.

From a failed call to a fix

When a call goes wrong, the log answers the first question, which is where. Find the first transition that was wrong. Then what you see in its entry tells you what kind of fix it needs.

What the log shows	What it means	What to do
Confident and wrong: 0.95 on an edge that a listener says was incorrect	The model has learned a bad pattern for this audio	Collect clips like it, label the right edge, add them to the data
Spread out: probabilities split across two or three options	The model can’t separate those options, or the states overlap	Add labeled examples of the boundary, or merge or rename the states if people can’t tell them apart either
Low everywhere, including “stay”	No edge fits what the caller wanted	Add a node or an edge, and collect examples of what callers ask for
Right edge taken at low confidence	The threshold is too loose	Raise it for that edge. No new data is needed
Right edge, good score, bad call	The decision was fine, and the problem is in the node’s prompt, tool or voice	Fix it there. The decision model isn’t the cause
Right edge, wrong moment	A timing problem, such as leaving the state mid-sentence	Fix turn-taking separately, as described in Ask what the moment needs

The last two rows matter as much as the first four. Part of making an agent reliable is ruling out the decision model, so you stop tuning the wrong layer.

The data is specific

Notice what the first three rows ask for. It’s never “more data”. It’s examples from one state, around one edge, in one kind of audio: hesitant confirmations at verify_identity, callers asking about a partial refund, two people talking at once on the greeting. A patch is a small, targeted set of labeled clips, not a bigger prompt or a full retraining project.

Then you check it. Replay the logged audio from past calls against the updated model, confirm the failing transitions are fixed, and see whether any good ones changed.

How to start

1. Write your graph down as states and edges. If two states are hard to tell apart, fix that first.
2. Turn each state’s outgoing edges into an option list, including “stay”.
3. Set a threshold per edge from the cost of a wrong move. Start strict on anything that touches money, identity or compliance.
4. Log every transition with the options, scores, choice and audio window.
5. Review the low-confidence and escalated calls first. Tag each failure with one of the rows above.
6. Patch with labeled examples, then replay old calls to confirm the fix.

A graph tells your agent what it may do. A decision model, with a logged score for every choice, tells you what it chose and why. Tazo is built to be that layer.

Don’t ask why the agent went wrong. Look up the transition where it did.