LREC 2026 · 2026 · Conference

M Kaab Bin Shahid et al. (University of Stuttgart / Aligarh Muslim University) · → Paper · Demo: ? · Code: ✓

Releases URDUTTS, an 89-hour, four-speaker Urdu TTS corpus with aligned Urdu-script, Romanized, and Phonemized transcriptions, and trains VITS and Glow-TTS from scratch on each input representation to show that phoneme input beats the paper’s Romanized input on MOS, WER, CER, and MCD.

Problem

Urdu has a large speaker population but few public, high-quality TTS corpora: earlier Urdu datasets are a few hours long, were collected for unit-selection or HMM synthesis, or were never released. The authors also argue that most neural Urdu TTS work feeds models Romanized Urdu, which has no standard spelling and drops contrasts that Urdu depends on (aspiration, vowel length, retroflexion, nasalization, pharyngeals). The Perso-Arabic script itself omits short vowels, so grapheme input is ambiguous too. No previous study had directly compared Romanized and phonemized input for Urdu TTS on the same data.

Method

Dataset. URDUTTS is built from publicly available long-form recordings (LibriVox, Internet Archive, Loyal Books), resampled to 22.05 kHz 16-bit PCM. Recordings are cut automatically with pydub silence detection (pauses over 400 ms below -15 dB; clips kept only if 2 to 15 s long). A 100 ms “synchronization” heuristic moves the tail of each clip onto the start of the next to avoid truncated word boundaries. Urdu-script transcripts come from a faster-whisper pipeline. Romanized text then comes from a deterministic grapheme-to-Latin transliteration module, and phonemized text from espeak-ng (§4.2). The corpus has 55,450 clips (88.59 h) from four male speakers. Speakers 1 to 3 have 21.6 to 30.0 h each and Speaker 4 has 9.7 h (Table 2). The dataset’s own quality figures are a VAD-based SNR of 48.52 dB and a reference MOS of 4.79 ± 0.12 from 40 native listeners (§4.5, Table 5).

Representations. Table 1 shows what the two input formats actually look like. The phonemized form is an IPA string with inferred vowels and stress marks, for example “ɟɪs ki mɪsˈaːl …”. The Romanized form is an upper-case consonant skeleton with no short vowels, for example “JS KY MSAL BNDGAN …”. It is not the informal vowelled Romanization (“khwaab”, “sahib”) that the introduction uses to motivate the study.

Models. The authors ran preliminary experiments with Tacotron 2, TransformerTTS, Neural HMM, VITS, Glow-TTS, DelightfulTTS, and FastSpeech, then kept VITS and Glow-TTS for their quality, convergence speed, and inference efficiency. These preliminary results are not reported. Each model is trained from scratch in Coqui TTS twice, once on Romanized and once on phonemized input. Training uses the Speaker 1 subset only, with batch size 64 for 3,000 epochs on a single 48 GB GPU, taking 5 to 8 days per run (§5.1). The vocoder used with Glow-TTS and the model sizes are not reported.

Evaluation. MOS on a combined pronunciation-and-naturalness scale (Table 3) comes from 40 native Urdu speakers rating 50 samples per system, reported with 95% confidence intervals. Objective metrics are computed on 500 samples per system: MCD in three variants (plain, DTW, slope-limited DTW), VAD-SNR, and WER/CER from faster-whisper transcription (§5.2). A word-level analysis (§7) lists the hardest and easiest words, the most frequent substitutions, and recognition accuracy by word length.

Key Results

  • MOS (Table 5): VITS-Ph 4.38 ± 0.24 vs VITS-Rm 3.86 ± 0.36; GlowTTS-Ph 3.34 ± 0.28 vs GlowTTS-Rm 2.64 ± 0.31; reference 4.79 ± 0.12. Phonemes beat Romanized input for both architectures, and VITS beats Glow-TTS for both representations. The Glow-TTS gap (0.70) matches the abstract’s 26.5% relative improvement. The VITS gap (0.52, about 13.5% relative to VITS-Rm by the same calculation) does not match the abstract’s 9.6% figure. The two VITS confidence intervals overlap slightly (4.14 to 4.62 vs 3.50 to 4.22), and no significance test is reported.
  • Intelligibility (Table 4): WER/CER is 12.3/5.3 (VITS-Ph) vs 15.1/6.5 (VITS-Rm), and 12.5/5.8 (Glow-Ph) vs 25.0/11.8 (Glow-Rm). Romanized input costs Glow-TTS far more than it costs VITS.
  • Spectral distortion (Table 4): phonemized input gives lower MCD in all three variants for both models. The gaps are modest (VITS DTW 7.97 vs 8.53; Glow DTW 6.74 vs 7.07). Glow-TTS has lower DTW and DTW-SL MCD than VITS even though its MOS is roughly one point lower, so MCD and MOS rank the architectures in opposite order.
  • SNR (Table 4): VITS-Ph 61.86 dB vs VITS-Rm 35.39 dB, while the two Glow-TTS systems are nearly identical (10.40 vs 10.27 dB). VITS-Ph’s SNR is higher than the 48.52 dB measured on the natural recordings. The architecture drives SNR far more than the input representation does.
  • Word-level analysis (Tables 6 to 8): phonemized models do better on the retroflex minimal pair “baṛī/barī” and on word-initial alef in function words. The by-word-length accuracies are mixed rather than uniformly favouring phonemes. VITS-Rm beats VITS-Ph on 5-character words (94.3 vs 93.9), and GlowTTS-Rm beats GlowTTS-Ph on 6-character words (91.5 vs 86.2). Phonemized models lead at lengths 2 to 4, and also at length 5 for Glow-TTS.

On the full set of tables, then, phonemized input wins on MOS, WER, CER, and MCD for both architectures. That does not support the paper’s statement that it is better “across all metrics” (§6): SNR is a tie for Glow-TTS, and the word-length breakdown is mixed.

Novelty Assessment

The main contribution is the dataset: a public, 89-hour, studio-quality Urdu TTS corpus with three aligned text representations is larger than prior released Urdu TTS resources described in §2.3. The modelling side is standard. Off-the-shelf Coqui VITS and Glow-TTS recipes are trained on one speaker, with no architectural changes. That phonemes outperform orthographic or transliterated input is a well-understood principle in TTS, and here it is applied to Urdu. The novel element is that the two representations are compared directly on the same data with human ratings. The Romanized condition is a vowelless consonant skeleton rather than conventional Romanized Urdu, so the experiment mostly measures the cost of discarding vowel information. That is a weaker baseline than the paper’s framing implies. The transcripts are Whisper output with no documented manual correction, despite the “post-editing” heading in §4.2.3.

Field Significance

Moderate. URDUTTS gives Urdu TTS a sizeable open training resource with parallel Urdu-script, Romanized, and phonemized text, and it can serve as a common benchmark for an under-resourced language. The representation study adds low-resource evidence that phoneme input matters more for a flow-based acoustic model (Glow-TTS) than for end-to-end VITS. Methodological caveats limit how strongly that comparison generalises.

Claims

  • supports: Phoneme-level input improves naturalness and intelligibility over lossy grapheme or transliterated input for neural TTS in languages whose script under-specifies pronunciation.

    Evidence: Trained from scratch on the same single-speaker Urdu data, espeak-ng phonemes beat a vowelless Romanized transliteration for both VITS (MOS 4.38 vs 3.86; WER 12.3 vs 15.1%) and Glow-TTS (MOS 3.34 vs 2.64; WER 12.5 vs 25.0%). (§6, Table 4, Table 5)

  • refines: How much an under-specified text representation costs depends on the acoustic model architecture: end-to-end VAE-GAN models degrade less than two-stage flow-based acoustic models.

    Evidence: Moving from phonemized to Romanized input raises WER by 2.8 points for VITS but by 12.5 points for Glow-TTS, and CER by 1.2 vs 6.0 points. The MOS drop is 0.52 vs 0.70. (§6, Table 4, Table 5)

  • complicates: Mel-cepstral distortion can rank TTS architectures in the opposite order from human naturalness judgements.

    Evidence: Glow-TTS systems have lower DTW and slope-limited DTW MCD than the corresponding VITS systems (e.g., 6.74 vs 7.97 DTW for phonemized input) while scoring roughly one MOS point lower (3.34 vs 4.38). (§6, Table 4, Table 5)

  • complicates: Signal-to-noise ratio measured on synthetic speech is a poor proxy for quality, because synthesized outputs can exceed the SNR of the natural recordings they imitate.

    Evidence: VITS-Phonemized reaches 61.86 dB VAD-SNR, above the 48.52 dB measured on the URDUTTS recordings. Glow-TTS systems sit near 10 dB regardless of input representation (10.40 vs 10.27). (§4.5, §6, Table 4)

  • supports: Retroflex and other language-specific phonemic contrasts are lost when a TTS model is trained on transliterated input lacking explicit phonetic encoding.

    Evidence: The retroflex minimal pair “baṛī/barī” (‘big’ vs ‘bad’) is among the top substitution errors for both Romanized models (5 occurrences each) but does not appear among the phonemized models’ top confusions. The dominant Romanized-model error, with 11 occurrences each, is a se/use function-word confusion that the authors label initial-alef deletion. (§7.2, Table 7)

Limitations and Open Questions

Warning

The “Romanized” condition is an upper-case consonant skeleton with no short vowels (Table 1), not the vowelled informal Romanization the introduction motivates. The headline phonemized-vs-Romanized gap therefore largely measures the cost of removing vowel information. Its relevance to conventional Romanized Urdu TTS input is untested.

  • Single speaker, single recipe. All models are trained only on Speaker 1 (a male audiobook narrator), so the multi-speaker portion of the corpus is never benchmarked. Preliminary results for the other five architectures are not reported.
  • Transcript quality. Urdu transcripts are produced by faster-whisper, and no manual correction or transcript error rate is reported. The same faster-whisper model is then used to compute WER/CER on synthesized speech, so recognizer biases enter both the training labels and the intelligibility metric.
  • Phonemizer reliability. espeak-ng must infer short vowels the Perso-Arabic script omits. Its accuracy on Urdu is not evaluated, so phonemization errors are an unmeasured confound in the “phonemized” condition.
  • MOS design. One scale merges pronunciation and naturalness (Table 3), with 50 samples per system and no significance testing. The VITS confidence intervals overlap slightly.
  • Duplicated rows in the word-level analysis. In Tables 6 and 8 (checked against the rendered tables), the VITS-Rm and GlowTTS-Rm rows are identical in every cell, and in Table 7 they share the same first three confusions and counts (11, 5, 5). This is despite the two systems’ very different WER (15.1 vs 25.0%). The per-model linguistic analysis for the Romanized systems should be treated with caution.
  • Data provenance and licensing. The audio comes from public audiobook sites. The exact license is deferred to the repository (“will be provided”), and all four speakers are male, which limits use for voice diversity.

Wiki Connections

  • GAN Vocoder — VITS, with its adversarially trained waveform decoder, is trained from scratch on URDUTTS and is the best system on MOS for both input representations.
  • Subjective Evaluation — a 40-listener native-speaker MOS test with 95% confidence intervals, on a scale that merges pronunciation accuracy and naturalness, provides the primary comparison.
  • Naseem et al. (Punjabi and Urdu TTS) — cited as prior Urdu Tacotron work with an unreleased dataset, which URDUTTS’s public release is positioned against.
  • Whisper — faster-whisper generates the Urdu-script transcripts for the corpus and also serves as the ASR for WER/CER evaluation.
  • Tacotron 2 — evaluated in preliminary experiments and discarded in favour of the non-autoregressive models, and cited as the backbone of earlier Urdu TTS systems.
  • Tacotron — cited as the starting point of neural Urdu TTS research reviewed in the prior-work section.