Conversational Voice AI

The problem

Menu IVR frustrates callers, and every language needs staffing

"Press 1 for..." menus push callers toward agents for things self-service should handle, and offering support in every language your customers speak means staffing agents around the clock across timezones.

This solution replaces menu trees with natural-language intent capture and real-time voice-to-voice translation — so more calls resolve without an agent, and the ones that need one arrive with full context, in any language.

Primary objective

Resolve more calls in natural language, in the caller's language, without adding headcount.

How it works

Capture intent → translate & respond → resolve or route → learn

Capture intent

Lex + Transcribe on the live call

→

Translate & respond

Translate + Polly, in real time

→

Resolve or route

Self-serve, or handoff with context

→

Learn

Transcripts tune future intents

What it does

✓ Natural-language intent capture
✓ Real-time voice-to-voice translation
✓ Self-service for common intents
✓ Context-rich handoff to agents
✓ Continuous tuning from transcripts

Built with

✓ Amazon Lex
✓ Amazon Transcribe
✓ Amazon Translate
✓ Amazon Polly
✓ AWS Lambda + Amazon Connect

Guardrails

✓ Confidence-threshold handoff
✓ No irreversible actions unconfirmed
✓ PII redaction before storage
✓ Full transcript logging

Under the hood

Low-confidence intents hand off, they don't guess

The bot only resolves what it's confident about. Below threshold, it routes to a queue with the transcript and detected intent already attached — no dead air, no repeating yourself.

connect/lex_fulfillment.py PYTHON

import boto3

translate = boto3.client("translate")
polly = boto3.client("polly")

CONFIDENCE_THRESHOLD = 0.75

def handle_intent(event: dict) -> dict:
    intent = event["sessionState"]["intent"]
    lang = event["bot"]["localeId"]

    if intent["nluConfidence"]["score"] < CONFIDENCE_THRESHOLD:
        # Don't guess — hand off with full context attached
        return route_to_agent(event, reason="low_confidence")

    reply_en = resolve(intent["name"], intent["slots"])
    reply = translate.translate_text(
        Text=reply_en, SourceLanguageCode="en", TargetLanguageCode=lang
    )["TranslatedText"]

    return speak(polly, reply, lang)

Outcomes

24/7

Multilingual coverage

Real-time

Voice-to-voice translation

Fewer

Agent-handled calls

Full

Context on handoff

Give every caller a language-fluent first line

Deploy standalone, or pair it with Contact Lens QA to score every AI and human interaction the same way.

Start a conversation →