GuidesUnderstanding AI
AI voice assistant latency: where the wait comes from, and how to cut it
The steps
- 1End of speechKnowing the person has finished.
- 2TranscriptionVoice becomes text.
- 3ThinkingThe model prepares the answer.
- 4SynthesisText becomes a voice.
In sequence or streaming
In sequence
Streaming
The levers
| Lever | Effect |
|---|---|
| Streamed model response | Synthesis starts on the first sentence |
| “Short sentences” instruction | The first chunk arrives sooner |
| Tuned end-of-speech detection | Neither cutting in nor needless waiting |
| Faster model for conversation | The largest kept for hard cases |
| Servers close to users | Less network time |
The technique: streaming AI responses. Monitoring: observing an AI application.
Frequently asked questions
- What delay feels natural?
- In conversation, people usually reply in under a second. Beyond that, the other person feels misunderstood. Measure the time between the end of the sentence and the first sound of the answer.
- Do speech-to-speech models solve the problem?
- They remove steps and cut the delay. In exchange, you have less fine control over each step (transcribed text, voice choice, logging).
- What about during a long tool call?
- Say a short holding phrase (“Let me look up your order”) while the tool works, rather than staying silent.