Blog · For developers
Voicemail detection: the awkward first three seconds of an AI call
A person says hello and waits. A machine says hello and keeps going. Telling them apart quickly is harder than it sounds.
By the CosVoice team · · 5 minute read
Hello. Hello?
Your agent dials a roofing company. Something picks up. A man's voice says "Hi, this is Dave." Your agent starts talking. Dave's voice carries on underneath: "I'm on a job right now, leave your name and number." Then a beep. Your agent is halfway through its second sentence, and the recording now holds the back half of an introduction and none of the reason for the call.
Run it the other way. Your agent is cautious, and waits to be sure. But this time Dave really did pick up, on a ladder, and what he hears after his hello is nothing. He says hello again. Still nothing. He hangs up, and he's right to.
Those are the two failures. Every approach to voicemail detection is a trade between them.
Why a machine answering a machine is hard
A voicemail greeting is a recording of a person trying to sound like themselves. For the first second or two it's the same signal as a live answer: a click, a breath, a name. There's no flag on the line that says "recording". The network tells you the call was answered, and that's all it tells you.
The old clues are about shape, not content. People who answer live tend to say something short and then stop to listen. Greetings run long and don't pause. Many end with a tone. Those clues are decent and not one of them is reliable. Some people answer with a whole sentence. Some greetings are three words long.
And some businesses answer with a phone menu, which is a machine but isn't voicemail, and wants a different response again.
Waiting to be sure costs you the call
The classic approach listens first and talks second. The call is answered, the agent is held back, the system measures the first thing it hears and then decides. It's tidy. It's also where the dead air comes from, because the person on the other end is waiting for a reply the whole time you're measuring.
People are quick to judge a silent line. A pause after hello sounds like a dialer connecting, and most of us hang up on that sound without thinking. So the careful approach makes a legitimate call feel like the kind nobody wants to take.
Deciding in the background instead
The alternative is to open the way a person would and keep deciding while you talk. The agent says who it is and who it's calling for, and listens at the same time. If what comes back is a reply, carry on. If what comes back is a greeting that doesn't react to anything, followed by a tone, it was a machine, and the agent changes plan.
This works best with a voice that can hear and speak at once. It helps to have a model that's listening to the words and not only the shape, too. "Please leave a message after the tone" isn't ambiguous. Neither is "press 1 for service".
It still makes mistakes. The agent may say its opening line to a recording. But that mistake is cheap. One wasted sentence costs a lot less than a roofer who said hello twice to nobody.
Once you know it's voicemail
There are three sensible things to do, and which is right depends on the call, not the technology. What's never sensible is a long message. Nobody listens to the end of one, and the callback number is usually at the end.
- Leave a short message. Who's calling, on whose behalf, one sentence on why, and the number to ring back. Wait for the tone before starting, and say the number slowly.
- Hang up and try later. Right when a message is no use: a table for tonight, a question that needs some back-and-forth.
- Hang up and report. Sometimes the owner would rather decide. The agent says it reached voicemail and asks what they'd like done.
Put the voicemail plan in the brief
This is the part you control as the person wiring up the agent. On our line a call starts from a brief in plain English, and the brief is where the voicemail decision belongs. We tell agents to cover four things: who to ask for, what you want, what to accept instead, and what to do on voicemail.
Something like: "If you reach voicemail, leave a short message asking them to call back on this number about the quote for Graham's water heater." The line gives its own number as the callback, not the owner's mobile, unless the owner has allowed otherwise. So when Dave rings back at six, the same number answers, with the owner's standing instructions in place, and takes the quote down.
There's more on writing these in how to write a call brief your AI can actually use.
Say what happened in the result
A call that reached a recording hasn't failed, and it hasn't succeeded either. It needs a status of its own, so your agent can choose the next action without reading a transcript to work out what went on.
Our reservation calls log voicemail and not_reached as outcomes in their own right, next to confirmed and the rest. One level down, a call's status can end as no-answer or busy as well as completed. Your agent should branch on those. Voicemail with a message left might mean wait until tomorrow for a callback. Not reached might mean try again after lunch.
In neither case should the agent tell the owner the job is done. We went further into that in structured call outcomes.
Common questions
What is voicemail detection?
It's how an automated caller works out whether a person or a recording answered the phone. It matters because the right behavior differs: talk with a person, leave a short message for a machine, or try again later.
Why is there a pause when some automated calls connect?
Often because the system is listening to the first second or two to decide whether a person answered before it lets the agent speak. The person on the other end hears that as dead air.
Should an AI agent leave a voicemail?
When a callback would help, yes, and it should be short: who is calling, for whom, why, and the number to ring. For anything time-sensitive, hanging up and trying again is usually better.
Can voicemail detection ever be fully accurate?
No. Greetings and live answers overlap too much in the first moments. The practical aim is to make mistakes cheap, which usually means opening politely and adjusting as soon as it's clear a recording is playing.
Read next: Write a call brief that covers voicemail · Full duplex vs turn-based voice · An AI phone call, from dial to summary
Hear it for yourself.
Call our own AI agent and ask her anything, or get a number in your area code in about a minute.