HomeThe problem
Generic text-to-speech can flatten the regional character of a message. I explored how playback could preserve more of that character in Hindi and Punjabi, including contexts where the user cannot look at a screen.
What I built
A WhatsApp-style interface sends text to Gemini for dialect detection and transliteration, then to ElevenLabs with a pinned voice ID for synthesis. The prototype also includes an experimental in-car playback surface.
The frontend and backend are separate so the model pipeline can change without replacing the messaging interface.
The tradeoff
I chose pinned regional voice IDs rather than cloning a sender. The output represents a regional voice, not the actual speaker’s identity.
The repository includes the integration pipeline. The hosted frontend is a prototype; live synthesis depends on a configured backend and model-service credentials. It is not a WhatsApp integration or a deployed CarPlay application.
What needs evaluation
Before calling the voices authentic, I would compare pronunciation and dialect fidelity with native speakers, including mixed-language messages. I would also measure generation latency, failure rate, and cost per audio minute.
Consent, retention, fallback playback, and on-device feasibility belong in production planning. No benchmark or listening-study results are presented here.
Prototype outcomes
What this prototype demonstrates
- An integration prototype combining dialect detection, transliteration, and synthesized audio.
- A separate frontend/backend boundary for model iteration.
- A defined evaluation surface around pronunciation, latency, consent, and cost.
Technical context
- Next.js
- FastAPI
- Gemini
- ElevenLabs
Up next
Google Maps: Meet Halfway ↗Let’s talk about the product decisions behind the work.
Get in touch ↗