Urdu Text-to-Speech & ASR System
An end-to-end Speech AI system for Urdu, a low-resource language, covering automated speech corpus curation, Whisper-based transcription pipelines, LoRA/Q-LoRA TTS model fine-tuning, and production FastAPI inference APIs for real-time audio synthesis.
The problem
Urdu is spoken by over 230 million people, but the speech AI ecosystem for it is sparse. Pre-trained ASR and TTS models perform poorly on Urdu due to limited training data and the language's complex morphology. Building a reliable speech system requires starting almost from scratch: collecting and curating a high-quality speech corpus, building automated transcription pipelines, and fine-tuning models specifically for the language.
What I built
This project was developed during my AI/ML Engineering Internship at PureLogics (Sep–Nov 2025) and covers the full stack of a production Speech AI system.
Dataset curation pipeline. I engineered automated pipelines to collect, segment, and clean over 100 hours of high-quality Urdu speech data. The pipeline uses Silero VAD for voice activity detection to isolate speech segments, audio enhancement techniques for noise reduction and normalisation, and Whisper for automated transcription to generate aligned text labels for each audio segment. Metadata generation and quality filtering steps ensure only clean, well-transcribed data enters the training pipeline.
Automatic Speech Recognition (ASR). I built and benchmarked Urdu ASR pipelines using OpenAI Whisper fine-tuned with LoRA on the curated corpus. Word Error Rate (WER) evaluation against held-out Urdu test sets guided iteration on model selection and training hyperparameters.
Text-to-Speech (TTS) synthesis. I fine-tuned open-source neural TTS models including architectures compatible with VITS and XTTS-v2 using LoRA and Q-LoRA techniques. These parameter-efficient fine-tuning methods allow adapting large pre-trained speech models on constrained hardware without full retraining. The result is a model capable of generating natural, intelligible Urdu speech from input text.
Inference API. The trained TTS models are served via a FastAPI-based inference service supporting text normalisation (handling numerals, punctuation, and Urdu script specifics) and returning audio in WAV and MP3 formats with low latency. The API is designed to be scalable and decoupled from the training pipeline.
Tech stack
Related services & writing
Building a Speech AI system?
If you're working on ASR, TTS, voice cloning, or a speech system for a low-resource language, I can help from dataset curation through model training to deployment.