LREC 2026 · 2026 · Conference
Mohammadamini et al. (Le Mans Université (LIUM)) · → Paper · Demo: ? · Code: ✓
Builds three single-speaker Central Kurdish TTS voices by fine-tuning an English F5-TTS checkpoint, two of them on newly curated audiobook data, shows the audiobook voices rate on par with a studio-recorded voice in a large listening test, and uses the voices to generate about 514 hours of synthetic speech for training a Central Kurdish to English speech-to-text translation model.
Problem
Central Kurdish has few public TTS resources: the one prior studio-recorded dataset (Gigant-KTTS) had not been used to build or evaluate a TTS system, an earlier Tacotron 2 system and its data were not released, and Meta’s MMS voices lack detailed evaluation, public training data, and voice cloning. The paper asks whether web-available audiobook recordings can substitute for purpose-recorded studio data in a low-resource language, and whether the resulting voices are good enough to synthesize training data for a downstream task (speech-to-text translation) where no parallel Kurdish speech exists at scale.
Method
Data. Two new datasets are curated from roughly 30-minute audiobook podcasts (one male, one female narrator, both biography domain, about 11 hours each). Audio is resampled from 44 kHz to 24 kHz mono, segmented with energy-based voice activity detection (segments of 1 to 15 s kept), transcribed with a Seamless model fine-tuned for Central Kurdish (about 8% WER on the clean Asosoft benchmark), and then manually revised to fix errors and drop noisy or distorted segments. The third voice uses Gigant-KTTS (labelled studio-M), 13.5 hours of a professional male dubber reading multi-domain sentences (Table 1).
Model. The authors fine-tune the English F5-TTS-base checkpoint (F5-TTS), a non-autoregressive flow-matching diffusion-transformer model that takes a reference audio prompt plus its transcript and the target text, outputs mel-spectrograms, and relies on a pretrained Vocos vocoder (Vocos) that is not retrained. Because F5-TTS’s character vocabulary lacks many Kurdish characters, input text is first normalized (Unicode, punctuation, numbers to words, via the Asosoft library) and then converted with a rule-based Central Kurdish G2P so the symbols fit the existing vocabulary. F5-TTS was preferred over VITS and XTTS for data efficiency, speaker flexibility through prompting, and robustness to non-studio data, based on the authors’ earlier Blizzard work. With too few speakers for a multi-speaker model, three mono-speaker models are fine-tuned independently, each on one RTX8000 (48 GB) for 2 days, holding out 500 utterances per dataset for testing. Model size is not reported.
Evaluation protocol. Training is monitored with an automatic protocol (released as the tts4all_eval package) covering naturalness (Torchaudio Squim MOS predictor, MOS_p; AASIST pseudo bona-fide probability, DF), signal quality (PESQ, WSNR, SRMR) and intelligibility (CER from a fine-tuned SeamlessM4T v2 ASR). Final checkpoints are then rated in a MOS test (“How close is the following voice to real human speech?”) across seven sentence categories: code-switching, prosodic variability, idioms and expressions, semantically unpredictable sentences, news, in-domain test text, and natural recordings, with 20 synthetic and 20 natural sentences per category and system.
Synthetic data for S2TT. Kurdish source sentences from bitext corpora (Kuvost and a multi-source parallel text collection) are synthesized by randomly choosing one of the three voices and a random reference prompt from 5,000 Central Kurdish Common Voice utterances, giving 340k utterances (about 514 h). Whisper v3 large is fine-tuned on these synthetic utterances paired with English translations.
Key Results
- Listening test (Table 2). From 3,101 ratings by 88 participants, the three voices score MOS 4.08 (audiobook-F), 4.06 (studio-M) and 4.01 (audiobook-M), against 4.42, 4.40 and 4.34 for the corresponding natural recordings. The audiobook-F voice is marginally ahead of the studio voice, and voice ranking follows the ranking of the natural training recordings. The differences between systems are small relative to the reported standard deviations (0.98 to 1.06), and no significance test between systems is reported, so “competitive with studio data” is the supportable reading.
- Per-category results (Table 3). CER stays at or below 0.09 for every system and category (the 0.09 maximum is on studio-M natural recordings). Studio-M has clearly the best signal quality (PESQ 3.78 to 4.16 vs 2.39 to 3.74 for the audiobook voices), so the audiobook voices match it perceptually while trailing it on this telecom-quality metric. Prosodic-variability and expression sentences are the hardest categories for subjective MOS across systems (for example audiobook-M 3.88 and 3.66, audiobook-F 4.01 and 3.61).
- Objective vs subjective metrics (§4.2). MOS_p sits at about 4.3 to 4.4 almost everywhere and does not separate natural recordings from in-domain synthesis, which subjective MOS does (for example audiobook-F 4.42 vs 4.26, studio-M 4.40 vs 4.20). DF has the highest correlation with subjective MOS among the objective metrics, at Spearman 0.42.
- Data-synthesis scenarios (Table 4). In a diagnostic setup where the FLEURS CKB→EN test set is both training and test data, Whisper fine-tuned on the real recordings reaches 92.42 BLEU (the authors’ upper bound), while fine-tuning on synthetic versions of the same utterances reaches 79.95 with all three voices and random prompts. Single voices give 76.87 (studio-M), 74.64 (audiobook-F) and 73.92 (audiobook-M). For studio-M, a random prompt (76.87) and a fixed same-speaker prompt (76.89) give essentially the same BLEU, so the gain comes from mixing voices rather than from prompt randomization alone.
- S2TT (Table 5). Whisper fine-tuned only on 514 h of synthetic speech reaches 27.23 BLEU on Asosoft and 18.50 on FLEURS, up from 2.31 and 1.51 zero-shot. No system trained on real or pseudo-labelled Kurdish speech is reported as a reference point, so the cost of using synthetic speech in place of real speech outside the diagnostic setup is not quantified.
- Speaker diversity (Figure 3). A t-SNE plot of ECAPA-TDNN embeddings shows that utterances sometimes cluster by TTS system rather than by prompt speaker, which the authors read as training-speaker identity leaking through and limited prompt-based voice cloning in the fine-tuned mono-speaker models.
Novelty Assessment
There is no architectural contribution. The model is an off-the-shelf F5-TTS fine-tune, and the G2P plus normalization front end is a standard adaptation for a script outside the pretrained vocabulary. The contributions are resource and evaluation work: two released audiobook-derived Central Kurdish TTS datasets with a semi-automatic curation recipe, the first systematic evaluation of the Gigant-KTTS studio data, a reasonably large listening test (88 raters) organized by linguistically motivated sentence categories, and a TTS-for-S2TT data-augmentation case study. The S2TT experiments are informative about voice mixing, but the main result has only a zero-shot baseline and the scenario search uses a train-equals-test setup, so the evidence on how useful the synthetic data is remains partial.
Field Significance
Low — The paper is a careful low-resource TTS resource paper. Its broader value is a data point that a few hours of curated audiobook speech, run through a fine-tuned English flow-matching model, can match studio-recorded data perceptually, together with evidence that a naturalness MOS predictor trained on English saturates on Central Kurdish while a deepfake-classifier score tracks listeners somewhat better.
Claims
- supports: Curated audiobook recordings can match purpose-recorded studio data as fine-tuning material for low-resource TTS on perceived naturalness, even when trailing it on signal-quality metrics.
Evidence: F5-TTS fine-tuned on about 11 h of curated audiobook speech scores MOS 4.08 (female) and 4.01 (male) against 4.06 for the 13.5 h studio voice, while the studio voice keeps clearly higher PESQ (4.00 vs 3.04 and 2.41 in-domain). (§4.2, Table 2, Table 3)
- supports: A pretrained English flow-matching TTS model can be adapted to a new low-resource language by fine-tuning after mapping the target script into the existing input vocabulary with a G2P front end.
Evidence: The English F5-TTS-base checkpoint, fine-tuned per voice for 2 days on one GPU with G2P-converted Central Kurdish input, reaches CER at or below 0.09 in every evaluation category and subjective MOS of 4.01 to 4.08. (§3, Table 2, Table 3)
- complicates: Automatic MOS predictors trained on English synthetic speech saturate on low-resource languages and fail to separate synthetic from natural speech that human listeners distinguish.
Evidence: Squim MOS_p stays at about 4.3 to 4.4 and does not separate natural from in-domain synthetic speech, while subjective MOS does (for example 4.42 vs 4.26 for audiobook-F); the AASIST pseudo bona-fide score correlates best with subjective MOS, but only at Spearman 0.42. (§4.2, Table 3)
- supports: Mixing several TTS voices when synthesizing training data improves downstream speech-to-text translation more than prompt variation within a single voice.
Evidence: In the train-equals-test FLEURS setup, three voices with random prompts reach 79.95 BLEU vs 73.92 to 76.87 for single voices, while random and fixed prompts with the studio voice give 76.87 and 76.89 BLEU; real speech reaches 92.42. (§5.2, Table 4)
- complicates: Prompt-based voice cloning in mono-speaker fine-tuned TTS models only partly transfers the prompt speaker’s identity, which limits how much speaker diversity they add to synthetic training data.
Evidence: ECAPA-TDNN embeddings of synthetic utterances from three fine-tuned voices and five prompt speakers sometimes cluster by TTS system rather than by prompt, which the authors read as training-speaker identity leaking into the output. (§5.3, Figure 3)
Limitations and Open Questions
- The S2TT evaluation (Table 5) compares only against zero-shot Whisper. There is no reference model trained on real or pseudo-labelled Kurdish speech, so the gap between synthetic and real training data outside the diagnostic setup is not measured.
- The scenario search in Table 4 trains and tests on the same FLEURS sentences. The authors frame it as an acoustic upper-bound probe, but it cannot show how the configurations would rank with held-out sentences.
- The MOS comparisons between systems carry standard deviations near 1.0 and no significance test, so the “audiobook ≥ studio” ordering is not statistically established. Each voice also uses a different speaker and domain, which confounds data source with speaker.
- The listening test uses a single human-likeness question. Intelligibility, speaker similarity to the prompt, and code-switching pronunciation are not rated separately by listeners.
- The speaker-diversity analysis is qualitative (t-SNE of 20 utterances), with no SPK-SIM numbers against the prompts.
- The authors note that Central Kurdish is close to phonemic with a high-precision G2P, which may make this recipe easier to apply than for languages with less regular orthographies.
Wiki Connections
- Flow Matching — Fine-tunes a pretrained flow-matching TTS model for a new low-resource language and evaluates it in detail.
- Zero-Shot TTS — Uses prompt-based voice cloning with unseen Common Voice speakers to diversify synthetic data, and finds that the training speaker’s identity partly leaks into the output.
- Subjective Evaluation — Reports an 88-listener MOS test across seven linguistically motivated sentence categories, with natural recordings mixed in as hidden anchors.
- Evaluation Metrics — Compares MOS predictor, deepfake-classifier, signal-quality and ASR-based metrics with listener MOS on a low-resource language, and finds the MOS predictor saturated.
- F5-TTS — Its English base checkpoint is the model fine-tuned for all three Central Kurdish voices.
- Vocos — Used as the frozen vocoder that turns F5-TTS mel-spectrograms into waveforms.
- XTTS — Considered as a multilingual alternative, but F5-TTS was chosen instead.