arXiv · 2025 · Preprint
Zirui Li et al. · → Paper · Demo: ✓ · Code: ?
Introduces Nord-Parl-TTS, an open TTS dataset built from Nordic parliamentary recordings that yields 900 hours of Finnish and 5090 hours of Swedish speech, together with unified evaluation sets for both languages.
Problem
TTS quality tracks the availability of large, high-quality speech corpora, but conventional studio-recorded datasets (audiobooks, voice-actor recordings) exist almost exclusively for a handful of high-resource languages. Finnish and Swedish illustrate the gap directly: prior Finnish datasets (Perso Synteesi, CSS10, FinSyn) top out at 60 hours and are not conveniently distributed through open data hubs, and no public Swedish TTS dataset exists at all. Existing large ASR corpora built from the same parliamentary sources (the Finnish Parliament ASR corpus, RixVox-v2) are not directly usable for TTS: their transcripts are grammatically corrected and disfluency-edited, and their segmentation follows word or sentence boundaries rather than clean utterance boundaries, both of which degrade pronunciation and prosody learning.
Method
Nord-Parl-TTS adapts the Emilia in-the-wild data processing pipeline separately for Finnish and Swedish, since the two languages start from different raw sources (parliament video/audio for Finnish, the pre-segmented RixVox-v2 ASR corpus for Swedish).
For Finnish, audio is extracted and resampled to 24 kHz mono, denoised with UVR-MDX-Net, diarized with Pyannote’s speaker-diarization-3.1, and segmented with Silero VAD. Because Whisper-large-v3 underperforms on Finnish, transcripts are cross-validated against a second, Finnish-only fine-tuned Wav2Vec2-large model: a segment is kept only if the two models’ normalized transcripts agree within 5% WER, after which Whisper’s transcript (which restores casing and punctuation) is retained as the label. Segments are finally filtered by DNSMOS P.835 OVRL score (threshold 3.0), yielding 900 hours.
For Swedish, RixVox-v2 audio is resampled to 24 kHz, denoised with the same separation model, and re-segmented using the corpus’s existing sentence-level timestamps (splitting at silence gaps longer than 2 seconds). The same diarization model excludes multi-speaker segments. Transcript quality is checked with a Swedish-specific Whisper-large model (trained in part on RixVox-v2), keeping segments with WER below 10% against the corpus’s provided transcripts (a looser threshold than Finnish’s, chosen because RixVox-v2’s reference transcripts carry similar written-language corrections). The same DNSMOS 3.0 cutoff is applied, yielding 5090 hours.
Two curated evaluation sets accompany the training data: 500 gender-balanced prompt-target pairs (3-20s, >10 characters) for Finnish drawn from Perso Synteesi, and 500 pairs for Swedish drawn from Common Voice 22.0, manually vetted by a native speaker for clarity, volume, accent, and background noise.
To validate the dataset, the authors train two existing non-autoregressive, flow-matching-based TTS systems from scratch in a monolingual setting on each language: Matcha-TTS, which uses explicit monotonic alignment search (MAS) between text and mel-spectrogram plus a WeSpeaker speaker-embedding encoder and classifier-free guidance, and F5-TTS (v1 Base configuration), which learns alignment implicitly through a stack of diffusion transformers conditioned on a speech prompt. No autoregressive baseline is included because no open-source implementation could be found. Finnish inputs to Matcha-TTS are raw characters (given the language’s near one-to-one grapheme-phoneme mapping); Swedish inputs are phonemized. Matcha-TTS trains for 500k updates on a single A100 GPU; F5-TTS trains for 1.2M updates on 24 AMD MI250X GPUs at a global batch size of 308,400 mel-spectrogram frames, matching the original F5-TTS recipe.
Key Results
Evaluated with CER (via Wav2Vec2-large for Finnish, Swedish Whisper-large for Swedish), speaker cosine similarity (SIM, via a WavLM-large speaker verification model), and human-rated CMOS and SMOS (22 Finnish and 20 Swedish Prolific participants, native-language screened), the two systems trained on Nord-Parl-TTS show a consistent trade-off pattern:
- Intelligibility and hallucination: Matcha-TTS’s explicit alignment yields far lower CER than F5-TTS-Base in both languages (Finnish: 2.55% vs 6.72%; Swedish: 4.66% vs 13.64%) and far fewer hallucinated utterances (1/500 vs 4/500 Finnish, 1/500 vs 18/500 Swedish).
- Human-likeness: F5-TTS-Base scores higher (less negative) CMOS in both languages (Finnish: -1.42 vs -1.93; Swedish: -1.46 vs -2.25), despite fewer training updates in relative model scale and worse intelligibility.
- Speaker similarity: results flip by language — Matcha-TTS’s fixed speaker-embedding conditioning beats F5-TTS-Base’s prompt-based conditioning on Finnish SIM (0.566 vs 0.538) and SMOS (2.49 vs 3.12, F5-TTS actually higher on SMOS despite lower SIM), but F5-TTS-Base wins on Swedish SIM (0.53 vs 0.442) and SMOS (3.41 vs 2.27).
Both systems reach what the authors characterize as “acceptable” performance on both languages, which the paper treats as evidence that the dataset supports functional TTS model training rather than as a claim of state-of-the-art quality.
Novelty Assessment
The contribution is a dataset and accompanying evaluation protocol, not a new architecture or training method: both benchmarked TTS systems (Matcha-TTS, F5-TTS) are used as published, with only routine adaptations (character vs. phoneme input choice, speaker-encoder swap for Matcha-TTS). The genuine novelty is the data engineering: adapting the Emilia pipeline to two under-served languages, solving language-specific transcript reliability problems (dual-ASR cross-validation for Finnish, a looser WER threshold informed by RixVox-v2’s known transcript corrections for Swedish), and providing the first public large-scale Swedish TTS corpus and the first Finnish corpus an order of magnitude larger than prior alternatives. The benchmark experiments are a validation exercise for the dataset, not an independent architectural comparison; sample sizes for subjective evaluation (20-22 raters) are modest.
Field Significance
Moderate — This paper closes a specific and previously unaddressed resource gap: no public Swedish TTS dataset existed before this work, and the Finnish corpus is roughly 15 times larger than the next-largest available alternative. Its contribution is data-scale and pipeline-engineering rather than architectural, and its immediate benchmarking value is demonstrating that the extracted data is usable for training rather than establishing new state-of-the-art TTS quality.
Claims
- supports: In-the-wild recordings can be converted into usable TTS training corpora at scale for languages lacking studio-recorded resources, provided the processing pipeline applies dedicated noise, diarization, VAD, and perceptual-quality filtering.
Evidence: An adapted Emilia-style pipeline (denoising, Pyannote diarization, Silero VAD, DNSMOS P.835 OVRL ≥ 3.0 filtering) applied to parliamentary recordings yields 900 hours of Finnish and 5090 hours of Swedish TTS-ready speech, where previously only 60 hours (Finnish) or zero hours (Swedish) of public data existed. (§3.1, §3.2, Table 1)
- supports: Cross-validating transcripts from two independently trained ASR models is a practical way to filter noisy automatic transcripts for corpus construction when a single ASR system is unreliable for the target language.
Evidence: Because faster-whisper-large-v3 underperformed on Finnish, the pipeline retained only segments where Whisper and a Finnish-specific Wav2Vec2-large model’s transcripts agreed within 5% WER, using Whisper’s transcript (with restored casing and punctuation) as the final label. (§3.1)
- complicates: Explicit duration/alignment modeling in non-autoregressive TTS improves intelligibility and reduces hallucination relative to implicit alignment learned via diffusion transformers, but this comes at some cost to perceived naturalness.
Evidence: Matcha-TTS (explicit MAS alignment) achieves substantially lower CER (2.55% Finnish, 4.66% Swedish) and far fewer hallucinated utterances than F5-TTS-Base (6.72% Finnish, 13.64% Swedish CER; up to 18/500 hallucinations), yet F5-TTS-Base scores higher (better) CMOS in both languages. (§5, Table 2)
- complicates: The relative speaker-similarity performance of prompt-based zero-shot conditioning versus fixed speaker-embedding conditioning is not consistent across languages or training corpora, and results from one language should not be assumed to transfer to another.
Evidence: On the same two model architectures, fixed speaker-embedding conditioning (Matcha-TTS) outperforms prompt-based conditioning (F5-TTS-Base) on Finnish SIM (0.566 vs 0.538) but underperforms it on Swedish SIM (0.442 vs 0.53). (§5, Table 2)
Limitations and Open Questions
The corpus is drawn entirely from formal parliamentary proceedings, a narrow speech register (planned, often scripted political speech), and the paper does not evaluate whether models trained on it generalize to conversational or expressive speech styles. The Swedish transcript-quality filter is deliberately looser (10% WER) than the Finnish one (5% WER) because RixVox-v2’s sentence-level transcripts carry corrections similar to those the paper criticizes in the Finnish Parliament ASR corpus, meaning some residual transcript-audio mismatch likely persists in the Swedish subset. No autoregressive TTS baseline was evaluated due to lack of available open-source implementations, so the benchmarking results reflect only two non-autoregressive, flow-matching-based systems. Subjective evaluation used 20-22 raters per language, a modest panel size for CMOS/SMOS testing. The authors’ stated future work includes extending the pipeline to Danish and Norwegian.
Wiki Connections
- Zero-Shot TTS — the paper’s evaluation protocol (synthesizing target content prompted by a reference speech clip) follows the standard zero-shot TTS setup, applied here as a benchmarking exercise for a new dataset rather than as a novel zero-shot method.
- Flow Matching — both TTS systems used to validate the dataset (Matcha-TTS with explicit alignment, F5-TTS with implicit DiT-based alignment) are flow-matching architectures, letting the paper contrast explicit and implicit alignment strategies under matched data conditions.
- Evaluation Metrics — the paper defines unified CER/SIM objective evaluation protocols and paired evaluation sets specifically to support standardized future benchmarking on Finnish and Swedish TTS.
- Subjective Evaluation — the paper runs CMOS and SMOS listening tests with native-language-screened Prolific participants, explicitly adopting comparative MOS instead of absolute MOS to avoid known measurement pitfalls.
- Emilia — the Nord-Parl-TTS pipeline is a direct adaptation of Emilia’s in-the-wild data processing pipeline to Finnish and Swedish parliamentary sources.
- F5-TTS — used as one of two systems trained from scratch on Nord-Parl-TTS to validate the dataset, with results contrasted against Matcha-TTS’s explicit-alignment approach.