🧭 Guides & comparisons

Speech-to-speech or STT, LLM and TTS: choosing how your AI voice agent works

Updated 2026-10-22 · 4 min read · By the Convo360 team

In short

A classic AI voice agent chains three models: speech to text hears the caller, a language model decides the reply, and text to speech says it. A speech-to-speech model does all three in one, which usually feels faster and handles turn-taking more naturally. The chain gives you more control over the voice, language and cost of each step. Test both on real Indian calls.

Key takeaways

The classic chain lets you pick the best Indian-language voice and the best brain separately.
Speech-to-speech models cut hand-offs between models, which usually means quicker, more natural replies.
In 2026 OpenAI released GPT-Realtime-2 and GPT-Live 1, and Google made Gemini 3.8 Live generally available.
Whichever design you choose, judge it on a real phone line in your customers' language, not in a demo.
On Convo360 you can run the classic chain or Gemini Live, chosen per agent.

Two ways to build an AI voice agent

The classic chain (often called a cascaded pipeline) runs three separate jobs in a loop:

Speech to text (STT) turns the caller's words into text.
A language model (LLM) reads that text, with your instructions and knowledge base, and writes a reply.
Text to speech (TTS) turns the reply into a voice.

A speech-to-speech model listens to audio and answers in audio directly. There is no separate transcription and synthesis step that you configure; the model handles listening, thinking and speaking together, and usually returns a transcript as a by-product.

If the three-step idea is new to you, our explainer on text to speech and speech to text covers the basics.

What changed in 2026

Speech-to-speech is no longer a preview curiosity. The providers' own changelogs, read on 21 October 2026, record these releases:

OpenAI, 7 May 2026: GPT-Realtime-2, which OpenAI's model page calls its most capable realtime voice model, with configurable reasoning effort and more reliable tool use.
OpenAI, 6 July 2026: GPT-Realtime-2.1 and a faster, cheaper 2.1 mini, with improved alphanumeric recognition, silence and noise handling and interruption behaviour.
OpenAI, 10 September 2026: GPT-Live 1 generally available, for full-duplex voice conversations that continue while a backend model handles reasoning and tools. Voice sessions are priced at $0.05 per minute, with backend model and tool use charged separately.
Google, 15 September 2026: Gemini 3.8 Live generally available. Google describes it as the default option for most low-latency voice agent experiences.

Meanwhile the classic chain kept improving too, with new Indian-language voices and transcription models. Neither design has "won". They suit different jobs.

The trade-offs side by side

Factor	Classic chain	Speech-to-speech
Speed of reply	Three hand-offs add delay; tuning helps	Usually quicker, fewer hops
Turn-taking and interruptions	Depends on how the pipeline detects speech	Built into the model
Choice of voice	Any TTS provider, including India-native voices	Limited to the model's own voices
Choice of brain	Any LLM you like	Tied to that provider's model
Debugging	Clear text at every step	Fewer intermediate steps to inspect
Cost visibility	Four bills: telephony, STT, LLM, TTS	Fewer lines, priced by token or minute

Speed matters more on a phone than almost anywhere else. Our note on voice latency explains why half a second changes how a call feels.

The Indian-language question

For most Indian businesses the deciding factor is how the agent sounds in Hindi, Marathi, Tamil or Hinglish, and how well it hears names, addresses and amounts.

The classic chain lets you pair a strong reasoning model with a voice built for Indian languages. Sarvam's docs, for example, list Bulbul v3 with Hindi, Bengali, Tamil, Telugu, Gujarati, Kannada, Malayalam, Marathi, Punjabi, Odia and English. You can swap the voice without touching the brain.

Speech-to-speech models are adding Indian voices as well. On 16 October 2026, OpenAI's changelog listed a Hindi voice, Nira, among nine new voices for GPT-Live. Coverage of other Indian languages still varies by provider, so check the model's own documentation for the languages you need.

Whatever the spec sheet says, run the same test script through both designs: a customer name like Venkatesh, a locality like Kandivali, an amount like ₹4,750 and one sentence that switches from Hindi to English halfway.

Which one to choose

Pick the classic chain when the voice and language matter most, when you want a specific brain for reasoning, or when you need to see exactly what the agent heard at each step. Collections, regional-language support lines and long scripted calls usually fit here.
Pick speech-to-speech when quick, natural back-and-forth is the priority: receptionist calls, short qualification calls, and conversations where callers interrupt a lot.
Keep a fallback either way. A model can be restricted or retired at short notice, so keep a second provider's key ready.

There is no need to decide once for the whole business. Different agents can use different designs.

A simple way to decide is a one-week trial. Put the same script on two agents, one on each design, and send half of your test calls to each. Compare three things from the transcripts: how often the agent misheard a name or number, how often callers had to repeat themselves, and how many calls ended with the job done. Those three numbers tell you more than any benchmark table.

Running either design on Convo360

Convo360 is bring your own keys, so each model bills your own provider account with no markup on tokens or minutes.

Classic chain: choose the agent's brain (OpenAI, Anthropic Claude or Google Gemini) and a voice from Sarvam, Smallest.ai, ElevenLabs or OpenAI.
Speech-native: set Gemini Live as the agent's brain to let one model listen and speak.
Connect your Exotel, Plivo, Twilio or SIP number under Phone Numbers and assign the agent.
Place real test calls with both designs and compare transcripts and call outcomes.

More detail is in Gemini Live for voice agents and the voice docs. AI voice calls are on the Pro plan at ₹2,999 per month, excluding 18% GST.

Example conversation

Hamare Chennai customers Tamil aur English mix karte hain. Kaunsa setup better rahega?

Start with the classic chain using a Tamil-capable voice, then run the same ten test calls on Gemini Live. Compare how each handles names, amounts and mid-sentence language switches before you decide.

Try both voice designs on your own number

Start free in minutes — no code, no credit card. Bring your own AI keys and pay no per-message markup.

Start 14-day free trial →See pricing

Frequently asked questions

What is a speech-to-speech AI model?

It is a model that takes the caller's audio and replies in audio directly, rather than chaining separate speech to text, language model and text to speech steps. Examples released in 2026 include OpenAI's GPT-Realtime-2 and Google's Gemini 3.8 Live.

Is speech-to-speech always faster than the classic chain?

Usually it responds sooner because there are fewer hand-offs between models, but real-world speed also depends on telephony, network and settings. Measure the gap between the caller finishing and the agent starting on real calls before you decide.

Can I use an Indian-language voice with a speech-to-speech model?

Only the voices the model itself offers. OpenAI added a Hindi voice for GPT-Live on 16 October 2026, but coverage of other Indian languages varies. The classic chain lets you pick any TTS voice, including India-native providers.

Which design is cheaper?

It depends on call length and providers. The classic chain has four separate costs: telephony, transcription, tokens and voice. Speech-to-speech models bill by audio token or minute. Price a typical three-minute call both ways using current provider rates.

Does Convo360 support speech-to-speech voice agents?

Yes. You can set Gemini Live as an agent's brain so one model listens and speaks, or run the classic chain with your choice of language model and voice provider. The choice is per agent, using your own keys.