ACL · 2025 · Conference
Yizhong Geng et al. (Beijing Logic Intelligence Technology) · → Paper · Demo: ✓ · Code: ?
A production TTS system for Thai that combines a curated 540-hour multilingual dataset, LLM-supervised preprocessing, a Phoneme-Tone BERT prosody encoder, and a GAN-based decoder to achieve state-of-the-art Thai synthesis with zero-shot voice cloning, outperforming Google TTS and Microsoft TTS on domain-specific tasks.
Problem
Low-resource TTS for tonal languages like Thai faces three compounding challenges: (1) Thai has a five-tone system where tone-consonant interactions critically affect meaning, (2) Thai orthography lacks word boundaries, requiring complex segmentation, and (3) sparse high-quality annotated data means LLM-scale approaches (SeedTTS, CosyVoice) are not practically deployable. Existing Thai TTS systems produce incorrect pauses and unnatural intonation.
Method
The system has two main components:
Preprocessing Pipeline:
- A fine-tuned Typhoon2-3B-Instruct LLM predicts prosodic pause positions in unpunctuated Thai text (trained on 15,000 annotated sentences via instruction-following).
- A tokenizer extending PyThaiNLP’s lexicon from 60K to 100K words handles unspaced orthography.
- A hybrid rule/transformer G2P converter maps graphemes to IPA phonemes with five tone markers, using a 40,000-word Phoneme-Tone Annotation Dataset.
TTS Model:
- Three pretrained feature extractors (trained on AiShell, LibriSpeech, JVS, KsponSpeech, then fine-tuned on Thai) supply forced-alignment, pitch, and energy ground truth.
- A style encoder extracts a latent style vector from the input waveform for zero-shot voice cloning.
- A GAN-based waveform decoder reconstructs speech from phoneme embeddings, duration/pitch/energy features, and the style vector. Loss is L1 (time) + mel spectrogram (frequency) + GAN perceptual.
- A Phoneme-Tone BERT (12-layer, 768 hidden, trained on 1M Thai sentences with masked LM) provides contextual prosody representations for three predictors (duration, pitch, energy). Predictors are first trained independently, then co-trained with the decoder.
Key Results
On the general-domain TSync2 benchmark, the proposed system (WER 6.3%, NMOS 4.4, PESQ 4.3) outperforms all open-source baselines (best competitor: Typhoon2-Audio at WER 7.8%) and matches or exceeds proprietary systems (Google TTS WER 6.5%, Microsoft TTS WER 7.1%). On domain-specific tasks (finance, healthcare, education, law), the advantage is even clearer: proprietary systems degrade sharply (Google WER jumps to 14.5%), while the proposed system shows much smaller degradation (domain WER 6.5%).
For zero-shot TTS, the system achieves SIM 0.91 and SMOS 4.5, vs. OpenVoice’s 0.85 / 4.0 on a 10-second reference.
Ablation confirms the G2P module contributes most (removing it raises WER from 6.3% to 22.5%), followed by tokenization (WER 10.2%) and pause prediction (WER 6.5%, NMOS 3.8).
Novelty Assessment
The contribution is primarily a pragmatic data-engineering and system-integration achievement for an under-served language. The Phoneme-Tone BERT for tonal prosody modeling is a meaningful technical contribution. The overall architecture (GAN decoder + FastSpeech-style predictors + BERT prosody encoder) follows established patterns (FastSpeech 2, VITS). The dataset construction (500h multi-domain + 40h vertical domain) and annotation pipeline are the most differentiating aspects for industrial deployment.
Field Significance
Moderate — This paper demonstrates that low-resource TTS for tonal languages can reach parity with commercial proprietary systems through careful data curation and language-specific preprocessing, without requiring LLM-scale data. Its primary value is practical: the domain-specific evaluation against real-world industrial deployments provides evidence that generalisation gaps in proprietary TTS are real and addressable with targeted dataset construction. The Phoneme-Tone BERT approach provides a concrete template for similar work in other tonal or morphologically complex low-resource languages.
Claims
- Accurate grapheme-to-phoneme conversion is the single most important preprocessing component for intelligibility in tonal-language TTS, contributing more to WER reduction than tokenisation or pause prediction combined. (§6, Table 3)
- Proprietary general-purpose TTS systems degrade substantially on domain-specific vocabulary, while systems trained with targeted vertical-domain data show much smaller performance drops in specialised settings. (§6, Table 2)
- Low-resource TTS systems can achieve competitive speaker similarity in zero-shot voice cloning relative to voice conversion models when trained with a style embedding module, even without speaker-specific fine-tuning. (§6, Table 4)
- Contextual prosody modelling via a tonal phoneme-aware BERT encoder improves naturalness scores in tonal-language TTS beyond what standard duration and pitch predictors achieve. (§5)
Limitations and Open Questions
- The approach has been validated only on Thai; generalization to other tonal or low-resource languages is claimed but not demonstrated.
- Model size and full architecture details are not reported, limiting reproducibility.
- The zero-shot comparison is against OpenVoice (a VC model) rather than other zero-shot TTS systems.
- Data collection from TikTok and YouTube raises questions about license and long-term data availability.
Wiki Connections
- multilingual-tts — demonstrates a practically successful low-resource tonal language TTS system
- prosody-control — Phoneme-Tone BERT provides contextual tonal prosody encoding as a conditioning signal
- zero-shot-tts — style encoder enables voice cloning from reference audio without speaker-specific training
- gan-vocoder — GAN-based waveform decoder is the synthesis backbone
- speaker-adaptation — the preprocessing pipeline extends standard adaptation principles to language-level adaptation in a low-resource setting