arXiv · 2026 · Preprint
Jiaxu He et al. (Xingchen AGI Lab, China Telecom / Xizang University) · → Paper · Demo: ✗ · Code: ✗
Adapts an in-house autoregressive LM plus flow-matching TTS model, pretrained on Chinese-English and Chinese dialect speech, to Ü-Tsang Tibetan by cleaning multi-source data, replacing the text tokenizer with syllable-level or Tibetan BPE units, and lightly fine-tuning, reaching MOS 4.28 to 4.35 against 3.74 for a commercial Tibetan interface.
Problem
Tibetan TTS has scarce, inconsistent speech data, a conservative orthography whose mapping to modern spoken pronunciation is not one-to-one, strong dialect variation (tonal Ü-Tsang, non-tonal Amdo, mixed Kham), and weak NLP tooling for front-end processing. Earlier Tibetan systems trained Tacotron 2 or FastSpeech 2 style models from scratch and struggled with stability at this data scale. The authors also argue that the default tokenizers of large pretrained TTS models, built for Chinese and English, split Tibetan into fine-grained sub-character units, which lengthens sequences and blurs alignment with the syllable-centred Tibetan writing system (§1, §2, §3.3).
Method
The system has three parts (§3.2, Figure 2):
- Data quality pipeline (§3.1, Figure 1): multi-source Tibetan recordings get audio cleaning (noise, loudness, silence trimming, outlier removal), Tibetan text normalization (symbols, numbers, non-standard forms), and speech-text consistency checks combined with manual inspection. No tools, thresholds, or before/after data sizes are given.
- Tibetan-oriented tokenizer adaptation (§3.3, Figure 3): two alternatives replace the backbone’s default segmentation. (a) Syllable-level modeling treats Tibetan syllables or characters as units with explicit separators. (b) A Tibetan-specific BPE tokenizer trained on Tibetan text replaces the original tokenizer. The authors present this as an input-side change that leaves the backbone architecture as it is. How the new token embeddings are initialized is not described.
- Cross-lingual adaptive training (§3.4): the backbone is an in-house “large speech synthesis model” from Xingchen AGI Lab with a two-stage design, an autoregressive language model for high-level semantic modeling followed by a flow-matching acoustic generator. It is pretrained on about 200,000 hours of Chinese-English mixed speech and 5,000 hours of multi-dialect speech (§2). Supervised fine-tuning on Tibetan speech-text pairs updates “only a limited portion of the model parameters” (lightweight adaptation). Figure 2 marks the flow-matching block with a frozen (snowflake) icon, so adaptation appears to target the LM and tokenizer side. The specific parameter-efficient method, the amount of Tibetan data, model size, speech-token type, vocoder, and training hyperparameters are all not reported.
The evaluated system covers the Ü-Tsang dialect only (§5).
Key Results
All results are in Table 1 (values checked against table-1.png) (§4):
| System | MOS | Syllable accuracy (%) |
|---|---|---|
| X-API (commercial Tibetan TTS interface) | 3.74 | 93.8 |
| Syllable-based tokenization | 4.28 | 97.6 |
| BPE-based tokenization | 4.35 | 96.6 |
- MOS was rated by 10 native Tibetan speakers. The number of test sentences, confidence intervals, and the rating scale’s exact definition are not reported.
- BPE scores 0.07 MOS higher, while syllable-level tokenization scores 1.0 point higher on syllable accuracy. The authors read this as a fluency versus pronunciation-precision trade-off. Without confidence intervals, it is unclear whether either gap is significant.
- How syllable accuracy was measured (human transcription, ASR, or another method) is not described.
- The comparison against X-API is a black-box commercial system from an unnamed vendor, so the comparison shows that the system is usable but does not isolate any of the three proposed components.
- The paper says no repetition, omission, or sequence anomalies were observed across text lengths and domains, but gives no metric for this (§4).
Novelty Assessment
The contribution is an engineering adaptation recipe: data cleaning, a tokenizer swap, and light fine-tuning of an existing AR-LM plus flow-matching TTS backbone. None of the three components is new in itself. The one comparison that tests an idea is syllable-level versus Tibetan BPE tokenization. There is no ablation against the backbone’s default tokenizer, against training from scratch, or against fine-tuning without the data pipeline, so the contribution of each proposed component is not measured. The backbone is proprietary and not described beyond its two-stage structure and pretraining data volume.
Field Significance
low — A low-resource adaptation report showing that a large Chinese-English pretrained AR-LM plus flow-matching TTS model can be fine-tuned to produce Ü-Tsang Tibetan rated above 4.2 MOS by native listeners. Its most reusable observation is that syllable-aligned and BPE tokenizations of Tibetan land close together, with a small, untested hint of a naturalness versus pronunciation-accuracy trade-off. The missing data sizes, ablations, and backbone details make it a weak data point for cross-paper comparison.
Claims
- supports: Fine-tuning a large TTS model pretrained on unrelated high-resource languages can produce natural, intelligible speech in a low-resource language with a distinct script.
Evidence: The Xingchen AR-LM plus flow-matching backbone, pretrained on Chinese-English and Chinese dialect speech, reaches MOS 4.28 (syllable tokenization) and 4.35 (BPE) from 10 native Tibetan raters after lightweight fine-tuning, versus 3.74 for a commercial Tibetan interface; the amount of Tibetan data is not reported. (§4, Table 1)
- refines: When adapting a pretrained TTS model to a new script, units aligned with the language’s syllable structure and corpus-trained subword units trade off slightly differently between perceived naturalness and pronunciation accuracy.
Evidence: Tibetan BPE gives higher MOS (4.35 vs 4.28) while syllable-level units give higher syllable accuracy (97.6% vs 96.6%); the authors attribute the accuracy gap to BPE units not matching pronunciation units, with no significance testing. (§3.3, §4, Table 1)
- complicates: Low-resource adaptation reports that bundle data cleaning, tokenizer changes, and fine-tuning without ablations cannot show which component drives the gains.
Evidence: The only baseline is an unnamed commercial API; there is no default-tokenizer, from-scratch, or uncleaned-data condition, so the gains over X-API cannot be attributed to the data pipeline, the tokenizer adaptation, or the backbone’s pretraining. (§4, Table 1)
Limitations and Open Questions
Critical details are not reported
The amount of Tibetan training data, the test set, model size, the fine-tuned parameter subset, the tokenizer vocabulary size, and how syllable accuracy was measured are all not reported. The backbone is proprietary, and the paper gives no code or demo, so the results cannot be reproduced.
- Only Ü-Tsang is evaluated, although the motivation and title cover multi-dialect Tibetan; Amdo and Kham are deferred to future work (§5).
- No ablation separates the effect of data cleaning, tokenizer adaptation, and cross-lingual pretraining. The default-tokenizer condition the paper argues against in §3.3 is never evaluated.
- The X-API baseline is a black-box commercial system with an unknown model, training data, and dialect coverage.
- Robustness claims (no repetition or omission across lengths and domains) are qualitative only.
- Zero-shot voice cloning and cross-speaker generalization are listed as future work and not evaluated (§5).
- The reference list includes an entry ([9]) with placeholder venue text (“Example venue; please replace with accurate source”) and an example.org URL, which indicates the references were not fully checked.
Wiki Connections
- Autoregressive Codec TTS — The fine-tuned component is the backbone’s autoregressive LM stage, which the paper adapts to Tibetan through tokenizer replacement and lightweight supervised fine-tuning.
- Flow Matching — The backbone’s second stage is a flow-matching acoustic generator, shown as frozen in Figure 2 during Tibetan adaptation.
- Subjective Evaluation — The main results come from a MOS test with 10 native Tibetan raters comparing two tokenization variants and a commercial system.
- TMD-TTS — A prior Tibetan multi-dialect TTS system cited as related work; it is not used as a baseline here.
- CosyVoice 2 — Cited as an example of the two-stage LLM plus flow-matching design that the in-house backbone follows.
- F5-TTS — Cited among flow-matching TTS backbones considered in the technical-selection discussion; not compared against.
- FastSpeech 2 — Representative of the from-scratch architectures used in earlier Tibetan TTS work, which the paper argues are unstable at low resource.
- Tacotron 2 — The other from-scratch architecture family used by earlier Tibetan TTS systems that the paper contrasts with large-model adaptation.
- VALL-E — Cited as an example of the autoregressive speech-LM paradigm the backbone’s first stage belongs to.