Why reading transcripts is not testing
The most common way teams test a voice agent is to read a transcript and decide whether the words look right. This misses most of what goes wrong on a phone call. A transcript shows you what was eventually said; it does not show you that the agent took two and a half seconds to start speaking, that it talked over the caller, that it responded confidently to something the caller never actually said, or that it sat in silence while the caller waited for a prompt. All of those produce transcripts that read perfectly. Voice is a real-time medium, and the failure modes are timing failures, audio failures, and turn-taking failures far more often than they are wording failures. If your entire QA process happens in a text window, you are testing a chatbot that happens to have a phone number attached.
Start with the unhappy paths, not the demo script
The happy path — a cooperative caller who answers each question in order, in a quiet room, in a clear accent — is the one case that reliably works, and it is the one most teams spend their testing time on. Invert that. Build your first test set out of the situations that break agents: the caller who says nothing for eight seconds, the caller who starts talking before the agent has finished, two people talking in the room, a TV in the background, a caller on speakerphone in a car, someone who answers "yeah no" in a way that means no, and someone who volunteers three pieces of information at once when you asked for one. Add the caller who changes their mind mid-sentence, because real people do this constantly and scripted flows almost never expect it. If the agent survives that list, the happy path will take care of itself.
Turn every real failure into a saved scenario
The single highest-return habit in voice QA is refusing to fix a bug without first capturing it. When a call goes wrong in production, save that scenario — the audio conditions, the phrasing, the point in the flow where it broke — into a suite you re-run before every release. Without this, teams fix the same class of bug repeatedly, because a prompt change that repairs one conversation quietly breaks another one nobody thought to re-check. This matters more for voice agents than for most software, because the behaviour is driven by prompts and models rather than branches you can read, so a change anywhere can shift behaviour everywhere. A regression suite is how you get the confidence to keep editing prompts after launch instead of freezing them out of fear.
Test latency as a pass/fail, not a nice-to-have
Callers interpret silence as a broken line. Long before a delay becomes technically interesting, it has already made the agent feel unnatural — people start repeating themselves, or say "hello?", and once that happens the conversation rarely recovers. Treat response latency as a hard test criterion with a number attached, measured on real calls over a real network rather than on a developer machine sitting next to the server. Measure the full round trip the caller actually experiences, not just model inference, because transcription, reasoning, speech synthesis, and telephony each add their own share, and the last one is the one most teams forget to include. If you want the underlying architecture view of where that time goes, we wrote about it separately in our piece on zero-latency architecture.
Decide what "good enough to launch" means in advance
Teams routinely launch without agreeing on what success looks like, which means the first week becomes an argument about whether the agent is working. Set the bar before you go live and write it down. Three criteria cover most cases: containment, meaning the share of calls fully resolved without a human; escalation accuracy, meaning that when the agent does hand off, it hands off for the right reason and at the right moment; and a latency ceiling the agent must stay under. A fourth is worth adding for regulated work: whether required disclosures were actually delivered on every call. Note that a high containment number is not automatically good — an agent that never escalates can be failing callers silently, which is why containment and escalation accuracy have to be read together rather than separately.
Where this fits in Persistence
Testing is a first-class step in Persistence rather than something you assemble yourself. You can run simulated calls against an agent before it is connected to a live number, replay scenarios after a prompt change to see what moved, and inspect what the agent heard and why it responded the way it did — which is usually the fastest route to diagnosing a call that felt wrong but reads fine. Because building, testing, and monitoring live in one place, the scenario you capture from a bad production call becomes a test you can re-run, rather than a note in someone's backlog. If you are still choosing a platform, the practical question to ask any vendor is what happens between "the prompt looks right" and "this is safe to point at customers" — and whether they have an answer at all.
Key takeaways
- Reading a transcript is not testing. Most voice-agent failures are timing and audio failures that never appear in text.
- Test the unhappy paths first: silence, crosstalk, background noise, a caller who answers a different question than the one asked.
- Every bug you find on a real call should become a saved scenario you re-run before the next release, or you will ship it again.
- Set an explicit bar for what "passing" means — containment, escalation accuracy, and a hard latency ceiling — before launch, not after the first bad week.
