voice TTS providers for low-latency conversational AI stacks

A voice agent lives or dies inside a single turn: it has to hear the caller, figure out what to say, and say it, all before the silence gets awkward. Miss that window and the illusion collapses; the caller starts talking to a machine instead of a conversation partner. Generating an audiobook chapter overnight is a fundamentally different engineering problem, and the text-to-speech providers built for one job are frequently terrible at the other. The Voice AI Agents market is scaling fast, with the majority of Fortune 500 companies already running voice AI somewhere in production, which means the infrastructure choices being locked in right now will be lived with for years. Most teams pick a provider on voice quality alone, or on a single latency number pulled from a marketing page, and both are how you end up rebuilding your stack in six months.

The metrics that actually determine whether a voice agent feels real

Time to First Audio, TTFA, is the number that maps to how a human perceives responsiveness. Model size, sample rate, and feature list are secondary to how fast the first chunk of sound hits the caller's ear.

Here's the trap: a median TTFA number by itself tells you almost nothing. Look at two hypothetical vendors from the Coval benchmark: one posts a 155ms P50 with a 2ms interquartile range, the other posts 188ms with a 100ms IQR. The first vendor is boringly, reliably fast. The second is fast on paper and unpredictable in practice, meaning every 50th call or so is going to sound like it's lagging behind a bad Zoom connection. At scale, with thousands of concurrent sessions, that variance becomes the actual product experience, not the average.

This is also where streaming architecture separates itself from batch generation. A provider that emits audio token-by-token can start playback while the rest of the sentence is still being synthesized. A provider that buffers the full utterance before sending anything is, by construction, adding a floor to your latency that no amount of clever prompting will fix. Word Error Rate matters too, but it's a secondary axis: a model tuned for offline accuracy can behave differently once you force it to commit to phonemes in real time, under streaming constraints it wasn't optimized for.

One caveat before naming a single provider: benchmark numbers published by the vendor selling you the product are marketing copy with decimal points. They're a starting point for shortlisting, not evidence you can take to a procurement committee. The only number that matters is the one you generate against your own script, your own languages, and your own concurrency profile.

How the TTS provider market has split into three distinct tiers

The market isn't one market. It's three, and confusing them is the single most common mistake I see teams make.

Tier 1 is expressive, offline-oriented models built for fidelity, emotional range, and production polish. TTFA runs from the hundreds of milliseconds into full seconds. Great for narration, dubbing, audiobooks. Wrong tool for a live agent, full stop.

Tier 2 is the real-time agent tier: purpose-built for streaming, targeting sub-100ms to low-200ms TTFA, with consistency treated as a design requirement rather than a nice-to-have. This is the tier that matters for conversational stacks, and it's the one this piece spends most of its time in.

Tier 3 is commodity volume. Cost is the entire pitch; quality and latency both take a back seat. Fine for bulk, non-interactive workloads where nobody's waiting on the other end of the line.

Here's the part that trips people up: the tier boundary is fixed by model, not by company. ElevenLabs is the clearest example. Flash v2.5 belongs in Tier 2. Eleven v3 and Multilingual v2 belong in Tier 1. Same company, two different products solving two different problems. Pick the model, not the logo.

Cartesia: architecture-first approach to consistent low latency

Cartesia built its case on architecture rather than tuning. Its models run on a State Space Model backbone instead of the transformer stack most of the industry uses, and the practical payoff shows up at P99, the tail end of the distribution, where transformer-based models tend to degrade under concurrent load and SSMs hold their line.

Sonic-3.5 streams first audio in 75 to 90 milliseconds over WebSocket from a US-East egress point, and Cartesia's own published benchmarks claim TTFA as low as 40ms. On the independent Coval benchmark, Sonic-3 posts a 188ms P50 with a 100ms IQR, a reminder that the architecture's real advantage lives in the tail, not necessarily the median you'd quote in a sales deck.

The product line is vertically built for agent developers: Sonic-3 for TTS, Ink-Whisper for STT, and Line as a full voice agent platform sitting on top. Language coverage sits around 42 languages with roughly 100 curated voices, plenty for most single-region deployments, a real constraint if you're building for a dozen markets at once. Pricing runs from a lower-tier Startup plan to a mid-tier Scale plan, with volume pricing landing around $0.02 to $0.03 per minute, among the cheaper Tier 2 options. The catch: promotional LLM pricing bundled into some plans has no committed timeline, which makes long-term cost modeling harder than it should be.

Deepgram Aura-2: the case for a unified STT-TTS runtime

Deepgram's argument centers on removing the network hop entirely. Aura-2 pairs natively with Deepgram's own Nova-3 and Flux speech-to-text models on one enterprise runtime, so the round trip that normally happens between separate STT and TTS vendors just doesn't exist.

On paper, that trade shows up as a loss: Aura-2 posts a 313ms P50 TTFA on the Coval benchmark with a 68ms IQR, slower on the median than Cartesia or ElevenLabs Flash. But TTFA measured on the TTS leg alone isn't the same thing as end-to-end pipeline latency, and a unified runtime can claw back the difference once you account for the hops a split-vendor stack has to make.

Language support currently covers seven languages, English, Spanish, French, German, Dutch, Italian, and Japanese, over WebSocket streaming, with no voice cloning offered. Pricing runs $0.030 per 1,000 characters, dropping to $0.027 at the Growth tier, while the bundled Voice Agent API covering STT, LLM, and TTS together runs roughly $0.075 to $0.080 per minute. The fit is obvious: teams already committed to Deepgram's STT stack, or anyone optimizing total pipeline latency instead of chasing a single leg's TTFA number.

OpenAI Realtime API: when collapsing the pipeline into one model changes the math

Realtime-2, shipped in May 2026, does something structurally different from everyone else on this list: it collapses speech-to-text and text-to-speech into a single speech-to-speech model running GPT-5-class reasoning. There is no separate TTS call to benchmark, because there's no separate TTS step.

That collapse buys simplicity and costs flexibility. No knowledge-base support, no custom voice cloning, preset voices only, and the newer Cedar and Marin voices are exclusive to Realtime-2, not portable elsewhere. On the Artificial Analysis ELO ranking, OpenAI's Realtime TTS lands at a mid-range ELO score priced at a per-unit rate, which works out to roughly 73.7 ELO per dollar, a cost-efficiency framing worth running against every other model on this list rather than taking at face value. For non-realtime needs, gpt-4o-mini-tts is still around at a relatively low per-minute rate, one of the cheaper rates among premium-tier models.

The trade is clean, if you're willing to make it: accept the fixed voice roster and the customization ceiling, and you get to delete an entire integration layer from your architecture. Bad fit for anyone who needs a licensed brand voice or serious multilingual depth.

Inworld and Rime: specialized entrants with narrow but real advantages

Two smaller players earn their spot here on genuinely distinct merits rather than general-purpose completeness.

Inworld's Realtime TTS-2 currently sits at the top of the Artificial Analysis Realtime TTS Arena at roughly 1,208 ELO, with sub-250ms P90 end-to-end latency on its Max model and sub-130ms on Mini. The architecture streams token-by-token over WebSocket, and the company publishes a 100ms TTFB figure for Realtime TTS-2, with production use already running in high-concurrency consumer apps, a real-world stress test that benchmark tables alone can't simulate.

Rime took a different bet entirely: its models were trained on full-duplex conversation data, including the hesitations and backchannels ("mm-hm," "right," the verbal filler that makes conversation sound like conversation) that most TTS training sets scrub out. The output sounds like someone talking rather than someone reading a script aloud, a genuinely different achievement. The caution is real, though: on the Coval benchmark, both Mist-v3 and Arcana were flagged for high variance, unsuitable for real-time use in their current form, a gap between the quality story and the production reliability story that shouldn't be waved away.

Neither has the language breadth or voice library depth to be a default choice for large global deployments. Both are worth a serious look if your use case matches their specific strength: Inworld for quality-per-dollar in realtime, Rime for conversational naturalness where that specific texture is the point.

ElevenLabs: navigating a platform that spans both tiers deliberately

ElevenLabs is the company most likely to confuse a buyer, because it's deliberately playing in both tiers at once, and the two products inside it should not be compared to each other as if they're competing.

Flash v2.5 is the real-time model. ElevenLabs claims 75ms on its dedicated real-time path; independent numbers from Openbenchmarks put it at a 288ms median TTFA, ranking seventh of 29 providers tested, matching the May 2026 Coval snapshot. That snapshot also shows a 28ms IQR, tight consistency for a Tier 2 model, though that particular IQR figure is attributable to Turbo v2.5 rather than Flash v2.5 itself, worth flagging before you quote it back in an internal deck.

Eleven v3, which went generally available in March 2026, is the flagship on the other side of the house: inline audio tags for emotional delivery, multi-speaker dialogue in a single generation, coverage across 70-plus languages. TTFA lands in the 500 to 800 millisecond range, which puts it squarely in Tier 1. Do not point it at a live agent; it wasn't built for that job and it will show.

The voice library runs past 5,000 voices, with Instant Voice Cloning working off a short sample and Professional Voice Cloning requiring 30-plus minutes of consent-recorded audio to produce a result nearly indistinguishable from the source speaker. Audio output tops out at 44.1kHz PCM on higher tiers, a spec that matters for content work where Cartesia and Deepgram simply aren't in the conversation. At volume, the cost gap gets real: at 100,000 minutes a month, Flash v2.5 runs roughly $0.04 to $0.06 per minute, and independent analysis puts the annual gap between Cartesia's pricing (around $2,500 to $3,000 a month at that volume) and ElevenLabs Turbo (around $5,000 to $8,000) at $20,000 to $60,000 a year. That gap is real money at scale and rounding error below it, where cloning quality and voice depth end up driving the decision instead. ElevenLabs also ships Conversational AI 2.0, an end-to-end platform unifying STT, LLM routing, and TTS rather than leaving you to bolt TTS onto someone else's pipeline. The practical rule: Flash v2.5 or the Conversational AI platform for live agents, Eleven v3 or Multilingual v2 for narration and dubbing where fidelity is the actual requirement.

What the benchmark numbers don't capture: voice quality, language depth, and cloning under real conditions

ELO rankings from arenas like Artificial Analysis measure human preference, a different signal from WER or TTFA entirely, and it's the signal that matters when brand voice or naturalness is the actual constraint rather than raw speed.

Accuracy and speed don't move together the way you'd hope. Eleven v3 posts a 5.0% WER on the Coval benchmark, good enough for 14th of 29 on that specific axis, proof that a model can sound excellent and still trail on transcription-adjacent accuracy under real-time pressure.

Language count is a vanity metric if it's not paired with language depth. Supporting 70-plus languages with prosody that sounds native is an entirely different engineering achievement than running text through a translation layer and calling it multilingual, and the difference is exactly what a global call center deployment will expose within the first week of production traffic.

Voice cloning fidelity deserves its own line item in procurement, not a footnote. Sample length requirements, whether micro-prosody (the tiny pitch shifts and breath patterns that make a voice recognizable) survives the cloning process, and identity verification standards all matter once you're dealing with a licensed voice or a real person's likeness, and none of that shows up in a benchmark table.

Worth watching at the edge: on-device models like Neuphonic's NeuTTS Air, a 748-million-parameter model that runs on mid-tier CPUs with no GPU required, open up privacy-sensitive or offline-capable deployments that cloud-only benchmarks never account for. Put it together and the real evaluation checklist has five parts: TTFA paired with IQR, language coverage measured for native fidelity rather than raw count, voice cloning requirements, on-prem or edge needs, and cost modeled at your actual projected volume.

How to structure a provider selection process that won't fail in production

Lock the tier before you evaluate a single feature. Real-time conversational, high-fidelity content, or high-volume commodity: pick one, and don't let a Tier 1 model's voice quality seduce you into a Tier 2 decision.

Define your concurrency profile next. A provider with a beautiful P50 and a wide IQR is a liability the moment you're running hundreds of simultaneous sessions, and the only way to catch that is testing at projected peak load, not the calm average traffic your demo environment sees.

Run your own benchmark, against your own text, your own target languages, before you sign anything. Vendor numbers get you a shortlist; they don't get you a defensible procurement decision.

Price the thing at the volume you'll actually run, not the volume in the pilot. The cost delta between providers is nothing at low volume and tens of thousands of dollars a year at 100,000-plus minutes a month, so model that curve before the architecture gets locked in and expensive to unwind.

Treat the pipeline as a single latency budget, not a stack of independent numbers. A slower TTS model sitting on a unified STT-TTS runtime can beat a faster TTS model that has to hop across a network boundary to get there, and Deepgram's Aura-2 is the clearest existing proof of that trade-off.

Audit everything that isn't latency early, because it'll eliminate candidates faster than any speed test will. Voice cloning consent and identity verification, language coverage at genuinely native quality, on-prem or edge requirements: any one of these can rule out a provider before you've measured a single millisecond.

The providers that hold up in production stay fast under real concurrent load, in the languages your users actually speak, at a cost your business can carry past the pilot, rather than simply posting the prettiest median TTFA in a controlled benchmark. Everything else is a demo.