NAACL · 2025 · Conference
Shenran Wang et al. (University of British Columbia) · → Paper · Demo: ? · Code: ✓
Flow-matching TTS systems for three Indigenous North American languages (Ojibwe, Mi’kmaq, Maliseet), showing that multilingual training on typologically related low-resource languages outperforms monolingual training, while also comparing attention-free architectures as memory-efficient alternatives and raising concerns about the cultural suitability of MOS evaluation for Indigenous communities.
Problem
TTS for most Indigenous languages remains undeveloped, largely due to severe data scarcity and the practical difficulty of studio recording with speakers who are predominantly senior community members. Prior work had produced systems for a handful of such languages but left Mi’kmaq and Maliseet entirely unaddressed. Beyond the data problem, no work had examined whether standard MOS-based subjective evaluation is culturally appropriate for Indigenous communities, whose norms around negative judgement can invalidate standard listener study designs.
Method
The system extends Matcha-TTS, a flow-matching TTS model comprising a text encoder, a duration predictor, and a flow-matching decoder that iteratively denoises mel spectrograms conditioned on upsampled text hidden states. For multilingual synthesis, learnable speaker and language embeddings are concatenated with encoder output before being passed into the decoder, following standard multilingual conditioning practice. The decoder uses 10 inference steps. A character-level tokenizer is used throughout, as all three languages employ phonetically transparent Latin orthographies.
Beyond the standard self-attention MatchaTTS (~40M parameters), the paper evaluates three attention-free alternatives as drop-in sequence mixers: Mamba2 (~38M params), a selective state-space model with subquadratic complexity; Hydra (~39M params), a bidirectional extension of Mamba2 using quasiseparable matrices; and FNet (~31M params), which replaces attention with a 2D Fast Fourier Transform and carries no learned parameters in the mixing operation. To compensate for the larger parameter footprint of SSM layers, encoder and decoder hidden dimensions are reduced by 3/4 in the Mamba2 and Hydra variants.
Data collection followed a community-centered protocol, allowing speakers to record at home using SpeechRecorder. Datasets range from ~1h41m (Mi’kmaq) to ~11h49m (Ojibwe JJ). During multilingual training, oversampling is applied to balance each speaker’s contribution by duration. Vocos, a frequency-domain vocoder, was trained on all speakers and used for waveform generation across all conditions.
Key Results
Multilingual models consistently outperform their monolingual counterparts across objective metrics (F0-RMSE, MCD, PESQ, VUV F1) for all four speakers and all three languages (§5, Table 2). The benefit is clearest for data-scarce speakers: the Ojibwe JJ speaker, with the largest dataset (~11h49m), shows a MOS improvement from 2.71 (monolingual self-attention) to 3.42 (multilingual FNet) against a natural speech reference of 4.16. For Ojibwe NJ (~1h41m), multilingual attention reaches 4.77 MOS, slightly above the natural speech reference of 4.74, suggesting that cross-lingual knowledge transfer can nearly close the naturalness gap for very low-resource speakers (§5, Table 2).
Among sequence mixing alternatives, Hydra achieves objective scores closest to self-attention across all speakers. No architecture dominates every metric. FNet lags on most objective scores due to its parameter-free nature, but its MOS for Ojibwe JJ (3.42) is the highest of any model for that speaker. In memory efficiency, all attention-free models use roughly half the GPU memory of self-attention at batch size 400 (2.3–2.5G vs. 4.6G in float32), but throughput advantages are not realised at this model scale (§5, Table 3).
MOS ratings were collected only for Ojibwe (two completed surveys); human evaluation was not possible for Mi’kmaq and Maliseet due to the small speaker community. The authors note that one rater assigned a score of 5 to all voices regardless of naturalness, which they attribute to cultural reluctance to judge negatively, raising questions about the validity of MOS results in this setting.
Novelty Assessment
The technical contribution is an applied adaptation: Matcha-TTS with multilingual conditioning and four alternative sequence mixing layers, trained from scratch on community-collected recordings. The multilingual-over-monolingual finding has solid precedent in multilingual TTS literature. The architecture comparison (Mamba2, Hydra, FNet in a flow-matching TTS encoder/decoder) is modestly novel in combination, though each mixer had been studied independently prior to this work. The primary novelty is coverage: Mi’kmaq and Maliseet had received no TTS attention previously.
The methodological observation about MOS evaluation and cultural norms is the paper’s most distinct contribution. It provides concrete evidence, grounded in field experience, that standard listener study designs can fail systematically when cultural norms around judgement diverge from those assumed by the protocol — a point that generalises beyond Indigenous languages.
Field Significance
Moderate — The paper provides the first TTS systems for Mi’kmaq and Maliseet and demonstrates that pooling typologically related low-resource languages during training yields consistent objective improvements. The cultural evaluation critique offers a practically grounded challenge to the universality of MOS-based evaluation, which can inform how researchers design listening studies for underrepresented communities. The technical methods are established; the contribution is primarily applied and empirical.
Claims
-
supports: Multilingual training on typologically related low-resource languages improves TTS objective quality over monolingual training for all participating languages.
Evidence: Multilingual models consistently outperform monolingual baselines across F0-RMSE, MCD, PESQ, and VUV F1 for all four speakers (Ojibwe JJ, NJ; Mi’kmaq MJ; Maliseet AT); MOS for Ojibwe JJ improves from 2.71 (monolingual) to 3.42 (multilingual FNet). (§5, Table 2)
-
supports: Attention-free sequence mixing architectures can match self-attention performance in flow-matching TTS while halving GPU memory requirements.
Evidence: Hydra achieves objective metrics close to self-attention MatchaTTS across all four speakers, and attention-free models consume 2.3–2.5G vs. 4.6G GPU memory at batch size 400 (float32). (§5, Tables 2–3)
-
complicates: Subquadratic complexity in attention-free architectures does not translate to inference latency gains in small TTS models.
Evidence: Self-attention has higher throughput than Mamba2, Hydra, and FNet in float32 on an A100 at batch size 400, because the model is small enough that architectural complexity advantages are not observable. (§5, Table 3)
-
complicates: Standard MOS-based subjective evaluation protocols can produce unreliable results when applied to Indigenous language TTS due to cultural norms around negative judgement.
Evidence: One of two Ojibwe raters assigned the highest score (5/5) to all voices including synthetic ones; MOS surveys could not be completed for Mi’kmaq and Maliseet due to lack of available raters; the authors argue the task paradigm itself is culturally misaligned. (§5)
Limitations and Open Questions
Warning
The custom datasets are not publicly released due to lack of consent from voice donors, preventing replication of the reported results. MOS evaluation is available only for one Ojibwe speaker (with two raters), making subjective quality assessment effectively impossible for three of the four speakers.
Recording in home environments introduces ambient noise that limits overall synthesis quality. The community-centered evaluation observation is preliminary and anecdotal rather than a systematic study of cultural evaluation norms. Model deployment for educational use, which the paper motivates, is planned but not yet demonstrated, leaving a gap between system performance and real-world utility.
Wiki Connections
- Flow Matching — the system is built on Matcha-TTS, a conditional flow-matching TTS architecture; this paper extends it to multilingual low-resource settings.
- Multilingual TTS — the paper directly contributes to multilingual TTS by demonstrating that pooling typologically related Indigenous languages improves performance over monolingual training.
- Evaluation Metrics — the paper uses F0-RMSE, MCD, PESQ, STOI, VUV F1, and FID as objective metrics, plus MOS, and challenges the validity of MOS in culturally distinct evaluation contexts.
- Subjective Evaluation — the paper provides concrete evidence that standard MOS listener studies can fail for Indigenous communities due to cultural norms, calling for community-centered evaluation design.