LoResLM 2026 · 2026 · Workshop
Fayçal S. A. Ouedraogo et al. (CITADEL / University of Luxembourg) · → Paper · Demo: ? · Code: ?
Builds a cascaded French-Mooré speech-to-speech translation system by collecting a 150-hour Mooré audio-text corpus and fine-tuning TTS and ASR models for a language with almost no prior speech technology.
Problem
Mooré is the most widely spoken local language in Burkina Faso (52.9% of the population), yet it remains an oral-first language with little standardized text data, while French dominates formal institutions such as banking, healthcare, and government administration. This asymmetry excludes Mooré speakers with low literacy from services that assume text-based interaction. Prior NLP work on African low-resource languages has concentrated on text-based translation, leaving speech-to-speech translation, which does not require reading or writing skills, largely unaddressed for Mooré. No prior ASR, TTS, or S2ST system existed for the language at the scale this paper targets.
Method
The authors build a cascaded French-Mooré speech-to-speech translation (S2ST) system rather than a direct end-to-end one, motivated by the absence of paired French-Mooré speech data and the availability of an existing French-Mooré machine translation model. The pipeline chains French ASR (Whisper Turbo), French-to-Mooré machine translation (a fine-tuned NLLB-200-distilled-600M model from CITADEL), and Mooré text-to-speech, with the reverse direction using Mooré ASR and French TTS (XTTS-v2 base).
To obtain training data, the authors scraped verse-aligned Bible audio and text from jw.org and bible.com, plus a small legacy monospeaker corpus from CITADEL, then used the fairseq toolkit to segment untranscribed chapter-length audio into verse-level utterances. This produced three datasets totaling roughly 88,000 utterances and 150 hours of paired Mooré audio-text (Table 1), none of which existed for Mooré before this work.
For Mooré speech synthesis, two TTS models were fine-tuned independently and compared: Orpheus-3B (a Llama-3B-based autoregressive TTS model pretrained on English) was adapted via LoRA fine-tuning (rank 64) on the smallest, single-speaker dataset, since it has no native multilingual support; XTTS-v2 (a GPT-style autoregressive model with convolutional encoders, pretrained on 27,000 hours across 17 languages excluding any African language) had its vocabulary extended to 4,000 tokens with Mooré-specific tokens and was fully fine-tuned for 20 epochs on the multispeaker JW.org data followed by 20 more epochs on the Bible.com data. For Mooré transcription, Wav2Vec-BERT-2.0 (a self-supervised speech representation model pretrained on 4.5 million hours across 140+ languages) was fine-tuned on a 90/10 split of the JW.org dataset, using a Mooré-alphabet-only vocabulary after lowercasing and stripping non-alphabetic symbols from the training text.
Key Results
Thirty-six Mooré native speakers evaluated the two fine-tuned TTS models on out-of-domain text (news, sport, tales, proverbs) not drawn from the religious training data. XTTS-v2 scored a MOS of 4.36/5 versus 3.47/5 for Orpheus-3B, and a UTMOS of 3.47 versus 2.80 (Table 5). In paired A/B comparisons on identical utterances, evaluators preferred XTTS-v2 audio 77.8% of the time versus 22.2% for Orpheus-3B. The authors note Orpheus-3B’s 3.47 MOS is notable given it was fine-tuned on only 6.48 hours of data, a fraction of what XTTS-v2 used. For transcription, the fine-tuned Wav2Vec-BERT-2.0 achieved 4.24% WER and 1.11% CER on its held-out split (§4.2). No comparison against another Mooré ASR system is reported, since none previously existed; the paper does not compare its S2ST pipeline’s end-to-end output against a direct (non-cascaded) alternative.
Novelty Assessment
The contribution is primarily a dataset and an engineering integration rather than a new architecture or training method: all three component models (Orpheus-3B, XTTS-v2, Wav2Vec-BERT-2.0) are pretrained systems adapted through standard fine-tuning or LoRA, and the cascaded S2ST design follows established ASR-MT-TTS pipeline structure. The genuinely new artifacts are the ~150-hour paired Mooré speech corpus assembled from religious websites and a working French-Mooré S2ST system, both previously unavailable for this language. The out-of-domain evaluation design (testing on non-religious text after training only on religious text) is a useful, if modest, methodological choice that supports generalization claims beyond the training domain.
Field Significance
moderate — This paper extends TTS, ASR, and cascaded speech-to-speech translation to a tonal African language with no prior speech technology, demonstrating that a religious-text speech corpus can transfer to out-of-domain use cases for a severely low-resource language. Its contribution is concentrated in data collection and system integration rather than modeling technique, so its significance to the broader field lies mainly in extending language coverage and providing a released Mooré corpus and reproducible fine-tuning recipe for a specific pair.
Claims
- supports: Vocabulary extension combined with full fine-tuning can adapt a multilingual autoregressive TTS model to a language absent from its pretraining data more effectively than lightweight adaptation of a monolingual model with far less target-language data.
Evidence: XTTS-v2, fully fine-tuned on ~144 hours of Mooré data after vocabulary extension to 4,000 tokens, scored 4.36 MOS and won 77.8% of A/B comparisons against Orpheus-3B, which was LoRA-adapted on only 6.48 hours (§4.1, Table 5).
- supports: A large self-supervised multilingual speech encoder can be fine-tuned into an accurate ASR system for a tonal, low-resource language using a comparatively modest amount of paired data.
Evidence: Wav2Vec-BERT-2.0, pretrained on 4.5M hours across 140+ languages, reached 4.24% WER and 1.11% CER after fine-tuning on roughly 63 hours (90% of Dataset 2) of Mooré speech (§4.2, Table 4).
- complicates: Speech corpora drawn from a single narrow domain can still support generalization to unrelated domains, but this should be verified with out-of-domain evaluation rather than assumed from in-domain scores alone.
Evidence: Both TTS models were trained exclusively on Bible audio-text (jw.org, bible.com) yet were evaluated by native speakers on unrelated non-religious texts (news, sport, tales, proverbs), which the authors highlight as evidence the approach transfers, though no in-domain comparison score is reported to quantify any domain gap (§4.1).
- complicates: Cascaded speech-to-speech translation pipelines built from independently fine-tuned components inherit each component’s error rate, which can compound end-to-end without being independently measured for the final translation output.
Evidence: The system chains fine-tuned French ASR (Whisper Turbo), machine translation, and TTS models, but no end-to-end S2ST quality metric (e.g., translated speech intelligibility or meaning preservation) is reported, only component-level MOS/UTMOS/A-B and WER/CER scores (§3, §4).
Limitations and Open Questions
The paper reports no end-to-end evaluation of the assembled French-Mooré S2ST system itself, only isolated scores for its ASR and TTS components; overall translation quality and error propagation across the cascade remain unmeasured.
The cascaded design, chosen due to the lack of paired French-Mooré speech data, is explicitly noted by the authors as prone to error propagation across ASR, MT, and TTS stages and as computationally demanding for training and inference, which the authors themselves flag as unsuitable for low-resource technological environments (§Limitations). The training corpus is drawn entirely from religious sources (Bible translations), so its linguistic register may not represent the full range of everyday spoken Mooré despite the out-of-domain test design. The paper also does not report code or demo availability, and it does not compare against a direct (non-cascaded) S2ST baseline, which the authors identify as future work.
Wiki Connections
- Speech-to-Speech — contributes a genuine cascaded French-Mooré speech-to-speech translation system, extending the S2S-translation sub-paradigm to a severely low-resource African language pair.
- Multilingual TTS — adapts two existing multilingual/monolingual TTS models (XTTS-v2, Orpheus-3B) to a previously unsupported tonal language through vocabulary extension and fine-tuning.
- Self-Supervised Speech — fine-tunes the self-supervised Wav2Vec-BERT-2.0 encoder as the core Mooré ASR component of the translation cascade.
- Subjective Evaluation — evaluates both TTS systems with MOS, UTMOS, and paired A/B listening tests conducted by 36 Mooré native speakers.
- XTTS-v2 — fine-tuned as the primary Mooré TTS model after vocabulary extension, and reused unmodified as the French TTS component in the reverse translation direction.
- SeamlessM4T — cited as a self-supervised, discrete-unit framework demonstrating direct S2ST feasibility, contrasted with this paper’s choice of a cascaded approach.