Real-Time Call Translation: A Guide for Contact Centres

Summary

Real-time call translation allows two people to speak different languages during the same live phone call. Each participant speaks in their own language, while an AI system translates the conversation in both directions.

Unlike an AI voice agent, which responds to the caller itself, real-time call translation supports a conversation between people.

For contact centres, the important questions go beyond how many languages a solution supports. Translation accuracy, latency, dialect coverage, telephony integration, call transfers, data handling and reliability all affect how the system performs in a live environment.

SentiVue Translate is designed to integrate real-time translation into existing telecommunications and contact-centre infrastructure.

Real-time call translation uses speech AI to allow people who speak different languages to communicate during the same phone call, without relying on a human interpreter for every interaction.

For example, a Portuguese-speaking contact-centre agent could receive a call from an Arabic-speaking customer. The customer speaks Arabic and hears the agent’s responses translated into Arabic. The agent speaks Portuguese and hears the customer’s responses translated into Portuguese.

Neither person needs to learn the other’s language. The translation system handles the language exchange while both people remain responsible for the conversation.

What Is Real-Time Call Translation?

Real-time call translation is the continuous translation of spoken language between participants in a live call. The system receives speech from one participant, processes its meaning and produces spoken output in the language of the other participant.

The process works in both directions: Caller speaks Language A → Agent hears Language B → Agent replies in Language B → Caller hears Language A

Unlike post-call transcription or text translation, the translated speech is delivered while the call is in progress. A real-time translation system may begin producing translated audio before the speaker has finished an entire sentence. How early it can do so depends on the language pair, the translation model and the amount of context needed to preserve meaning.

This is also what separates real-time translation from an AI voice agent. An AI voice agent interprets a request and decides how to respond, potentially using business rules, external data or other software tools.

A translation system has a different objective: convey what one person said to the other person as accurately as possible. The translator may need context to resolve ambiguity, but its role is to preserve the speaker’s intended meaning, not independently decide the content of the conversation.

How Does Real-Time Call Translation Work?

A real-time call translation system combines speech processing with live audio delivery and telephony integration. There are different technical approaches.

Cascaded speech translation

A conventional speech translation pipeline contains three main processing stages:

1. Speech recognition: Automatic speech recognition (ASR) converts incoming audio into text.

2. Machine translation: Machine translation (MT) converts that text into the target language.

3. Speech synthesis: Text-to-speech (TTS) generates the translated audio that the other participant hears.

These stages can operate incrementally, processing partial speech rather than waiting for the entire utterance. Language identification may also be used when the system does not already know which language each participant is speaking. In other deployments, languages can be configured in advance, avoiding repeated identification.

The central engineering challenge is coordination. Speech recognition, translation and audio generation must exchange information quickly enough to support a continuous conversation. But translating too early can introduce errors when later words change the meaning of an earlier phrase.

Direct speech-to-speech translation

Not every modern translation system uses a separately exposed text pipeline. Direct speech-to-speech models can receive audio and generate translated audio without requiring an application to connect independent ASR, MT and TTS services.

Some systems also use hybrid architectures, combining direct audio models with other processing components. These approaches differ in how they manage latency, translation quality, language coverage, controllability and deployment requirements.

For a contact centre, the important distinction is not simply which model architecture is used. It is whether the complete system can deliver accurate translated speech while maintaining the connection and managing changes in the call.

The telephony layer

Translation processing is only part of the system. The audio must also be received, processed and delivered to the correct participant.

In SIP-based environments, SIP manages call signalling, while media is commonly transported using RTP. Contact-centre platforms may expose additional APIs and Computer Telephony Integration (CTI) interfaces for call control, agent states and workflow events.

The translation service must remain connected to the relevant audio streams as the call changes. For example, a call transfer may change which agent receives the audio. A conference may introduce another participant. Hold and resume can alter the media flow. Whether translation survives these events depends on the complete integration, not the translation model alone.

Why Does Latency Matter?

Latency determines how long participants wait to hear translated speech. The longer the delay, the harder it can become to maintain a natural exchange. Participants may repeat questions, interrupt each other or start speaking before the translated response is complete.

However, there is no single universal latency threshold that determines whether a translated call is usable. A short delay may be manageable in a structured customer-service interaction, while the same delay may disrupt rapid exchanges or conversations involving frequent interruptions.

The language pair also matters. Some translations require information that arrives later in the source sentence. There is therefore a trade-off between translating quickly and waiting for enough context to translate accurately.

When evaluating latency, contact centres should distinguish between:

• Time to first translated audio: How long it takes before the listener starts hearing translated speech.

• Translation completion delay: When the relevant translated segment finishes playing.

• Translation lag: How far translated output falls behind the corresponding source speech.

• Latency distribution: Typical performance and slower cases, including median (p50) and 95th-percentile (p95) measurements.

These measurements should be taken across actual calls or representative test conditions, not inferred from model inference speed alone. The total experience also depends on network transmission, audio buffering, media conversion and playback.

For a deeper technical discussion, see our guide to low-latency speech translation pipelines.

Real-Time Translation vs Other Ways to Support Multiple Languages

Real-time call translation is one of several approaches available to multilingual contact centres. Each has different operational advantages and limitations.

Multilingual human agents

Recruiting agents who speak customers’ languages remains a practical option, particularly for high-volume language groups and interactions requiring specialised expertise. The limitation is staffing flexibility. Language demand, shift coverage and specialist availability can make it difficult to provide consistent support across every required language.

Human interpreters

A professional interpreter joins the interaction and translates between participants. This remains important where interpretation accuracy, professional standards or the consequences of misunderstanding justify specialist human involvement. The main operational considerations are interpreter availability, workflow integration and cost.

AI voice agents

AI voice agents can manage suitable customer requests without involving a human agent. They are useful for defined tasks such as routine enquiries, information gathering and structured service workflows. Their limitation is that some interactions still need a human to make a judgement, manage an exception or resolve a complex issue.

Real-time call translation

The human agent remains responsible for the interaction, but the language barrier is handled by the translation system. This can allow contact centres to serve customers across languages without requiring an agent fluent in each customer’s language. The trade-off is dependence on translation quality and the reliability of the real-time audio infrastructure.

Hybrid multilingual support

A hybrid model combines these approaches. For example, an AI voice agent may handle an initial request before transferring the interaction to a human agent. If the customer and agent speak different languages, real-time translation can support the conversation after the transfer.

The critical requirement is continuity. When the AI agent leaves, the translation service must remain connected to the call, and relevant conversation context must be passed to the human agent. This requires deliberate integration between the voice agent, translation system and telephony platform.

We’ve covered that architecture in more detail in our article on AI-to-human handoffs.

What Should Contact Centres Evaluate?

A language list and a successful demonstration are useful starting points. Neither is sufficient evidence that a translation system will perform reliably in production. The following areas should be part of a technical evaluation.

1. Translation accuracy and dialect coverage

Supporting a language does not mean supporting every dialect or regional variation equally well. Portuguese provides a useful example. European Portuguese and Brazilian Portuguese differ in pronunciation, rhythm and vocabulary. A system optimised for one may not perform equally well on the other.

Accuracy should be assessed using the specific languages and dialects spoken by customers and agents. Contact centres should also test industry terminology, names, addresses, numbers, account references and instructions where small errors can change meaning.

Speech recognition accuracy alone is not enough. A system can transcribe the original speech correctly and still translate it incorrectly. For production evaluation, bilingual reviewers should assess whether the translated output preserves the intended meaning, including information omitted, added or changed.

2. Bidirectional translation and interruptions

A translated call involves two directions of speech, each of which may have different performance characteristics. Translation from Language A into Language B should therefore be tested separately from translation from Language B into Language A.

Natural conversations also contain interruptions, pauses, incomplete sentences and overlapping speech. When both participants speak at once, the system must manage incoming audio and translated playback without confusing the speakers or delivering audio to the wrong participant.

3. Existing telephony and contact-centre integration

Determine exactly how the service connects to the existing environment. Possible integration approaches include SIP-based media handling, contact-centre platform interfaces, media streaming and external conferencing services.

What matters is how the complete integration affects latency, audio routing, deployment complexity, security and supported call operations. The technical evaluation should cover inbound and outbound calls, hold and resume, blind and attended transfers, conferences, supervisor functions and reconnections where relevant.

For telecom-specific considerations, see our guide to carrier-grade voice translation.

4. AI-to-human handoff

If the customer initially speaks with an AI agent, establish what happens when the call is escalated. Does the translation service continue when the human agent joins? Does it follow the correct media streams if the call is transferred? Does the agent receive the customer’s request, previous actions and reason for escalation?

These are separate requirements. Conversation context may be passed successfully while the translation service disconnects. Translation may continue while the agent receives no information about the earlier interaction. Both need to be tested.

5. Data processing and deployment

Contact centres should know where voice data is processed, what information is generated and whether any information is retained. This includes more than audio recordings. Depending on the architecture, the system may process transcripts, temporary audio buffers, diagnostic logs, metadata and other derived information.

Data residency, retention policies, access controls, third-party processors and deployment arrangements should be documented. Cloud, regional, on-premises and customer-controlled deployments can offer different levels of operational control.

6. Reliability and failure handling

Translation systems should be tested for failures as well as successful calls. A temporary network problem, media-stream interruption or translation-service failure should trigger a defined response.

Depending on the deployment, that response may involve retrying translation, reconnecting the service, alerting the agent, transferring to an appropriate queue or allowing the underlying call to continue. The correct behaviour depends on the service requirements and the consequences of an untranslated or incorrectly translated interaction.

7. Scalability under real call volumes

A demonstration involving a few calls does not establish performance under contact-centre load. Testing should reflect expected simultaneous calls, language distribution, call duration and peak demand. That includes the impact of concurrency on translation latency, model capacity, audio quality and recovery behaviour.

Questions to Ask a Real-Time Call Translation Vendor

Before selecting a vendor, ask for clear answers to the following questions:

1. Which language directions, dialects and regional accents are supported in our intended deployment?

2. How is latency measured, and can we review median and higher-percentile results for complete translated audio delivery?

3. How do you evaluate meaning preservation, particularly for terminology, names, numbers and instructions?

4. Can the system handle interruptions and overlapping speech in both translation directions?

5. Which call operations are supported, including transfers, conferences, hold and reconnection?

6. What happens when translation fails, becomes unavailable or produces an uncertain result?

7. Where is data processed, what information is retained and which external processors are involved?

8. Can the system be tested on our own infrastructure and representative call traffic, including peak concurrency?

How SentiVue Approaches Real-Time Call Translation

SentiVue Translate is designed to integrate live, bidirectional speech translation into existing telecommunications and contact-centre infrastructure. The aim is to allow people to make and receive ordinary phone calls while speaking their own languages, without requiring a separate translation application on their devices.

Depending on the infrastructure, integration can take place at different points in the network or call environment. Deployment options include cloud and customer-controlled infrastructure. SentiVue also develops dialect-specific speech capabilities for regional language variants, including European and Brazilian Portuguese.

In a published pilot with a European telecom operator, SentiVue demonstrated bidirectional translation between Portuguese and English over a standard phone call. The case study reports translation latency of approximately one second, with processing inside EU data zones and no retained call audio in that deployment.

For organisations combining voice automation with human support, SentiVue Talk is also designed to preserve conversation context and multilingual continuity during supported AI-to-human handoffs.

Explore SentiVue Translate or book a technical discussion to evaluate the service against your call environment.

Frequently Asked Questions

Can two people speak different languages on the same phone call?

Yes. Real-time call translation can process speech in both directions, allowing each participant to speak in their own language and hear translated audio during the call. Actual quality and supported language combinations depend on the translation system and deployment.

Is real-time call translation the same as an AI voice agent?

No. An AI voice agent is a participant in the conversation that generates its own responses. A real-time translation system conveys speech between human participants. The two technologies can also be combined in workflows where an AI agent transfers an interaction to a person.

How much latency is acceptable for live call translation?

There is no universal threshold. Lower latency generally improves conversational responsiveness, but the acceptable delay depends on the interaction, language pair and translation quality.

Does real-time call translation work with existing contact-centre platforms?

It can, but integration requirements vary. Some solutions use native platform interfaces or SIP-based integration. Others rely on external media services, applications or conferencing arrangements.

Does a caller need to download an app?

Not necessarily. Translation integrated into the telephony or call environment can operate without a separate application for the caller. Other services may require a dedicated application or joining method.

Can AI translation completely replace human interpreters?

Not in every situation. Real-time AI translation can support a range of multilingual conversations, but decisions about replacing or supplementing human interpretation should account for translation accuracy, operational requirements and the consequences of errors.

Can real-time call translation run on-premises?

Some translation systems support on-premises or customer-controlled deployment. Availability depends on the vendor, models, hardware requirements and selected language capabilities.