arXiv · 2026 · Preprint
Abhijit Mohanta et al. (Indian Institute of Technology Guwahati) · → Paper · Demo: ✓ · Code: ?
Builds and evaluates the first reported TTS system for Mizo, a low-resource tonal Tibeto-Burman language, comparing Tacotron2 and VITS on under 6 hours of data and finding that VITS achieves substantially better tone accuracy and naturalness despite tones never being explicitly annotated in training.
Problem
Mizo is a low-resource, tonal, Tibeto-Burman language (Kuki-Chin subfamily) spoken by over half a million people, primarily in Mizoram, India, with four lexical tones (High, Low, Rising, Falling) that are not consistently marked in its Roman-script orthography. No TTS system existed for the language prior to this work, and building one is constrained by both data scarcity (no existing speech database) and the absence of explicit tone annotation in any available text. The paper asks whether an end-to-end, non-autoregressive TTS architecture can produce acceptable and tonally accurate Mizo speech under these constraints, without adding any explicit tone-marking mechanism to the text or training pipeline.
Method
The authors recorded 10.01 hours of single-speaker Mizo speech (narration/news-reading style, 44.1kHz/16-bit) from a female native speaker, of which 5.18 hours (2,252 sentences, ~6,646 unique words) were curated for this work after sentence-level segmentation in Praat, numeral-to-word conversion, special-character normalization, and abbreviation expansion. No tone annotation was added at any stage; the Mizo orthography’s tone markings remain implicit in the text as written.
Two TTS architectures were trained on this corpus using ESPnet2. Tacotron2 uses a convolutional encoder with a BLSTM, location-sensitive attention, a two-LSTM-layer decoder with prenet and postnet, guided-attention and stop-token losses, and a separately trained Parallel WaveGAN vocoder to convert predicted mel-spectrograms to waveform. VITS is trained as a fully end-to-end alternative: a Conformer-based text encoder, a posterior encoder, normalizing flows for latent modeling, and a HiFi-GAN-style decoder that generates waveform directly, trained with a combination of adversarial (multi-scale multi-period discriminator), mel-spectrogram, feature-matching, duration, and KL-divergence losses.
Evaluation combined objective and subjective measures on 25 held-out sentences (75 total audio samples across ground truth, Tacotron2, and VITS). Objective metrics were DNSMOS (a non-intrusive perceptual quality predictor), Mel-Cepstral Distortion (MCD, segmental spectral similarity), and F0 RMSE and F0 correlation (pitch-contour similarity to ground truth), each tested for significance with paired t-tests. Subjective evaluation used 35 native Mizo listeners rating naturalness (real-vs-synthesized judgment) and a 5-point MOS scale, analyzed with a linear mixed-effects model and Bonferroni-corrected pairwise comparisons. A separate tone error rate (TER) analysis had an expert phonetician (also a native speaker) mark incorrectly realized tones per tone-bearing unit across the 25-sentence evaluation set.
Key Results
VITS outperformed Tacotron2 on every metric reported. DNSMOS was comparable between VITS (3.90) and natural speech (3.84), both nominally above Tacotron2 (3.81), though the DNSMOS difference between VITS and Tacotron2 was not statistically significant (p = 0.72). MCD favored VITS significantly (2.32dB vs. 2.53dB, p < 0.0001). Neither F0 RMSE (40.6Hz vs. 41.34Hz, p = 0.61) nor F0 correlation (0.25 vs. 0.24, p = 0.67) differed significantly between the two systems. Subjectively, native listeners rated ground-truth speech as natural 76% of the time, versus 41% for VITS and 20% for Tacotron2; scaled MOS was significantly higher for VITS (3.46) than Tacotron2 (2.63, pairwise estimate -0.53, p < 0.0001), though both trailed natural speech (4.18). On tone accuracy, VITS’s Tone Error Rate (5.67%) was less than half of Tacotron2’s (12.93%), with High and Low tones (the more frequent tone categories in Mizo) accounting for most errors in both systems.
Novelty Assessment
The contribution is entirely empirical and infrastructural, not architectural: both Tacotron2 and VITS are used as established ESPnet2 implementations with no modification. The value is in being a first-of-kind TTS system for an unserved low-resource tonal language, and in the specific finding that tone realization can emerge with reasonable accuracy from a non-autoregressive end-to-end architecture even without any explicit tone-marking in text or training targets, which the authors attribute to the diversity of the underlying text data producing an implicit language model that captures tonal patterns. The evaluation methodology (paired objective metrics with significance testing, LME-modeled subjective ratings, and an independent expert tone-error annotation) is more rigorous than is typical for a small low-resource-language TTS paper, and its finding that DNSMOS and F0-based metrics fail to track the large, significant differences in tone accuracy and human-judged naturalness is a genuinely useful methodological caveat for anyone evaluating tonal-language TTS.
Field Significance
Moderate, the paper’s architectural contribution is nil, but it provides a concrete, statistically grounded data point on two under-studied questions: whether non-autoregressive end-to-end TTS can learn lexical tone without explicit annotation in very low-resource settings, and whether standard automatic TTS quality metrics (DNSMOS, F0 RMSE/correlation) are adequate for tonal languages. Both findings are directly reusable by other low-resource and tonal-language TTS work, even though the underlying training data is not itself released.
Claims
- supports: A non-autoregressive, end-to-end TTS architecture can achieve substantially lower lexical tone error rates than a conventional autoregressive two-stage (acoustic model plus vocoder) architecture in a low-resource tonal language, even without explicit tone annotation in the training data.
Evidence: VITS reduces Tone Error Rate from 12.93% (Tacotron2) to 5.67% on the Mizo evaluation set, despite tones not being explicitly marked anywhere in the training transcripts or database. (§3.3, Table 6)
- supports: Acceptable subjective naturalness for a tonal, low-resource language can be achieved with under 6 hours of single-speaker training data using an end-to-end non-autoregressive framework.
Evidence: Native Mizo listeners judged VITS output natural in 41% of cases (vs. 20% for Tacotron2, 76% for ground truth) and gave VITS a significantly higher scaled MOS (3.46) than Tacotron2 (2.63, p < 0.0001), from a training corpus of only 5.18 hours. (§3.2, Table 2, Table 5)
- complicates: A general-purpose non-intrusive perceptual quality metric (DNSMOS) does not reliably track prosodic or tonal correctness in low-resource tonal-language TTS.
Evidence: DNSMOS scores for VITS (3.90) and Tacotron2 (3.81) were not statistically distinguishable (p = 0.72), even though the same two systems differed sharply and significantly in tone error rate (12.93% vs. 5.67%) and human-rated naturalness. (§3.1, §4, Table 2, Table 3)
- complicates: Objective pitch-contour metrics (F0 RMSE, F0 correlation) can remain statistically indistinguishable between systems that native listeners judge to be substantially different in naturalness and tonal accuracy.
Evidence: Neither F0 RMSE (p = 0.61) nor F0 correlation (p = 0.67) differed significantly between Tacotron2 and VITS, despite VITS’s large, statistically significant advantage in both tone error rate and subjective MOS. (§3.1, Table 2, Table 3)
Limitations and Open Questions
- The training and evaluation data (5.18 curated hours from a single female speaker) is not publicly released; the paper is explicitly framed as a work in progress, with only audio samples (not the corpus) made available on a demo page.
- All evaluation is single-speaker; no multi-speaker or multi-style (teaching, news, storytelling) variation is tested, though the authors list this as future work.
- Tone accuracy is assessed by a single expert annotator (also a native speaker) rather than multiple independent raters, so inter-annotator reliability for the TER measure is not established.
- The authors explicitly conclude that DNSMOS should not be treated as a sufficient benchmark for TTS evaluation in low-resource, tonal-language settings, since it fails to surface the segmental and prosodic quality differences that MCD, tone-error analysis, and subjective ratings do detect.
Wiki Connections
- GAN Vocoder — trains VITS end-to-end with a HiFi-GAN-style decoder and multi-scale multi-period discriminator, following established GAN vocoder practice for direct waveform generation.
- Subjective Evaluation — conducts a 35-listener native-speaker study combining naturalness (real-vs-synthesized) judgments and 5-point MOS ratings, analyzed with a linear mixed-effects model.
- Evaluation Metrics — reports that DNSMOS and F0-based objective metrics fail to track large, statistically significant differences in tone accuracy and subjective naturalness between compared systems, a methodological finding relevant to evaluating tonal-language TTS.
- WaveNet — cited as the DNN-based TTS precedent that established end-to-end, vocoder-free waveform generation, contextualizing the paper’s choice between two-stage (Tacotron2) and end-to-end (VITS) architectures.
- Tacotron — Tacotron2, the paper’s baseline system, is the direct successor architecture to this end-to-end mel-spectrogram TTS framework.
- FastSpeech 2 — discussed in related work as a non-autoregressive alternative noted for better prosody retention, motivating the paper’s interest in non-autoregressive architectures for a tonal language.