Voice AI is governed by a sub-second human-conversational latency budget that multi-hop cloud architectures struggle to meet

A recurring argument in voice-AI engineering is that the design target is set not by the technology but by human perception of conversational timing. As framed by Telnyx (2026): "Humans respond within 200 milliseconds in conversation," and responses "above 1200ms" cause users to "hang up." Between those bounds lies the entire budget a spoken-dialogue system has to receive audio, transcribe it, run LLM inference, synthesize speech, and return audio.

The claim's force is that this budget is hard — a perceptual floor, not a preference — and that conventional architectures spend too much of it on network overhead. A typical multi-vendor stack, chaining separately-hosted speech-to-text, LLM, and text-to-speech services, is said to add "250ms minimum in network hops alone," landing at "800ms - 1.5 seconds total." Against this, Telnyx claims a "response time of less than one second" from running the pipeline on colocated hardware — see claim-telnyx-colocates-gpus-at-telephony-pops-for-voice-latency.

This connects to the internal latency structure of LLM inference itself: the perceived delay before a spoken system responds is dominated by Time to First Token, the prefill-phase metric described in claim-llm-inference-prefill-decode. Where a text chat can stream tokens into a visibly-typing UI, a voice turn must clear enough of the pipeline to begin speaking within the conversational window, making the latency budget less forgiving than in text interfaces.

Every number above is sourced to a single vendor page (Tier 3), and the specific figures are [unverified-quant]. The ~200ms turn-taking threshold does have a genuine primary home in conversation-analysis research (e.g. work on the timing of turn-taking), and the abandonment and hop-cost figures should be checkable against independent measurement — verification is routed to question-verify-conversational-turn-taking-latency-thresholds.

Update 2026-07-30 — the ~200ms figure now has a Tier-1 primary source

A dedicated verification capture (10-inbox/raw/2026-07-29-what-are-the-primary-sourced-thresholds-for-human.md) resolved part of this note's [unverified-quant] flag. The ~200ms turn-taking figure is now sourced to claim-stivers-2009-cross-linguistic-turn-taking-gap-208ms-mean (Stivers et al. 2009, PNAS, Tier 1: modal peak 0-200ms, cross-linguistic mean +208ms), and the related network-delay ceiling is sourced to claim-itu-t-g114-150ms-transparent-400ms-outer-limit-thresholds (ITU-T G.114, Tier 1: 150ms transparent / 400ms outer limit). This note's own text and source_quote are left as originally written — they still describe the Tier-3 Telnyx page this note was drafted from — but a future rewrite should re-point the turn-taking half of the argument at the new Tier-1 notes instead.

The other two figures are not resolved: no primary source was found confirming the vendor-specific claim that voice-AI response delays above ~1200ms cause abandonment (the closest primary, claim-brown-2005-call-center-abandonment-hazard-rate-two-peaks, measures hold-queue abandonment — a caller waiting for a human agent — not AI-response abandonment, and the two mechanisms should not be assumed to transfer). The ~250ms multi-vendor hop-cost figure remains unaddressed. This note should stay status: seedling with its [unverified-quant] flag intact for those two figures; see the updated progress log at question-verify-conversational-turn-taking-latency-thresholds.