Talking to an AI Should Not Feel Like Leaving a Voicemail | Insights | Mecverse

In human conversation, the typical gap between one person finishing and the next starting is about 200 milliseconds, and that holds across languages (Stivers et al., PNAS 2009). A voice AI pipeline needs speech-to-text, a model reply and speech synthesis before it can say anything, so it will not match that. It can feel much closer than it is, though. These are the patterns we used in InterviewRoom.

Speak the First Sentence First

Do not wait for the full model reply before speaking. Stream tokens, cut the stream at sentence boundaries, and send each sentence to speech synthesis as soon as it is complete. The user hears the first sentence while the model is still writing the third.

Sentence splitting needs more care than splitting on full stops. "Dr.", "e.g." and "vs." end in a period but do not end a sentence. Very short fragments sound choppy when spoken alone, so hold them until there is more text. Whatever is left when the stream ends gets spoken as the final piece.

Let People Interrupt

People talk over each other, and an AI that cannot be interrupted feels like a recording. When the user starts speaking, cancel everything in flight: the model request, speech synthesis, and audio already queued in the browser.

Cancelling requests is not enough, because a response can arrive after you have moved on. Give each turn a generation number, increase it on every interruption, and throw away any audio or text tagged with an old number. This one counter removes a whole class of "the AI answered the question before last" bugs.

Cache the Lines You Say Every Time

Greetings, transitions and closing lines repeat across sessions. Cache the synthesized audio keyed by voice and exact text, and those lines play immediately at no synthesis cost. It is a small cache with a high hit rate on exactly the moments users notice: the start and the end.

Captions Buy Patience

Showing live captions while the user talks proves the system is listening, and that makes the wait for the reply feel shorter. If your speech-to-text provider has no streaming mode, you can still do this: re-transcribe the growing utterance every second or so. Allow only one caption request in flight per session, space requests by wall-clock time, and give captions a small share of your speech-to-text budget so they can never starve the real transcription.

Detect Speech on the Device

Run voice activity detection in the browser (we use the open-source Silero VAD model) and send audio only while the user is speaking. You stop paying to transcribe silence, and you can detect the end of a turn on the device instead of waiting for a server timeout. That end-of-turn decision is where most perceived lag hides.