arXiv · 2026 · Preprint

Semin Kim et al. (KRAFTON / Ludo Robotics / Seoul National University / KAIST / University of Wisconsin-Madison / Stanford University) · → Paper · Demo: ? · Code: ✓

Releases a 615K-hour open English TTS data pool, a 510K-hour filtered training subset, a four-regime robustness benchmark, and unmodified F5-TTS-architecture models (0.3B, 1B) trained only on open data that match or beat open-weight systems trained on proprietary multi-million-hour corpora on Seed-TTS-Eval and CV3-Eval.

Problem

Most strong zero-shot TTS systems (Seed-TTS, CosyVoice 3, Qwen3-TTS) are trained on undisclosed corpora of up to several million hours, so their data and curation cannot be studied or reproduced. Fully open-data systems such as F5-TTS and MaskGCT train on roughly 100K hours of a single source (Emilia) and trail the proprietary-data systems. The paper asks whether a large, multi-source, openly licensed corpus can close that gap with a fixed, published architecture, in the way open corpora served LLM, CLIP, and Whisper reproduction efforts. It also argues that existing zero-shot TTS benchmarks draw about 1,000 prompts from a single read-speech source and therefore say little about robustness to noisy, conversational, or expressive prompts.

Method

The contribution is mainly data and evaluation; the model is held fixed on purpose.

Raon-OpenTTS-Pool (615K h, 240M segments). The pool aggregates 11 public English corpora with more than 500 hours each, mixing TTS-oriented and ASR-oriented sources: LibriTTS-R, SPGISpeech and SPGISpeech 2.0, HiFiTTS-2, LibriHeavy, GigaSpeech, VoxPopuli (English, with transcripts recovered from MoSEL), People’s Speech, the English splits of Emilia and Emilia-YODAS, and YouTube-Commons. Segments longer than 30 s are excluded, and Common Voice is left out to avoid overlap with Seed-TTS-Eval. The largest share (335K h) is Raon-YouTube-Commons, rebuilt from the original URLs through an Emilia-style pipeline: loudness normalization at 16 kHz, UVR-MDX source separation, PyAnnote 3.1 diarization, Silero VAD segmentation into 3 to 30 s clips, and Whisper-large-v3 transcription, followed by removal of segments whose transcript is implausibly short for their duration. Licenses are mixed and include non-commercial terms (Kensho User Agreement for SPGISpeech, CC BY-NC 4.0 for Emilia-YODAS) (§3.1, Table 2).

Raon-OpenTTS-Core (510K h, 194M segments). Each segment is scored on three model-based signals: Whisper-small WER against the existing transcript, DNSMOS, and Silero-VAD speech ratio. The combined filter ranks every segment on each signal, averages the ranks, and drops the lowest 15th percentile. Per-signal 15th-percentile cutoffs fall at DNSMOS 2.24, speech ratio 0.79, and WER 0.35. Overall retention is 84.7%: studio read-speech corpora keep over 93% of their segments, People’s Speech (Dirty) keeps 48.2%, and Raon-YouTube-Commons keeps 82.9% (§3.2, Figure 2, Table 4).

Raon-OpenTTS models. The models reuse the F5-TTS DiT flow-matching architecture (the authors call it a diffusion transformer) without modification, so that data effects can be isolated. The 0.3B variant (336M) uses the original F5-TTS configuration: 22 layers, 16 heads, width 1024. The 1B variant (1048M) scales to 28 layers, 22 heads, width 1408, and FFN 5632. Inputs are character-level text (vocabulary 5,512) and 80-band log-mel at 16 kHz with hop 256. Waveforms come from a HiFi-GAN vocoder pretrained on LibriTTS at 16 kHz. Training runs for 225K steps (0.3B, about 1K B200 GPU-hours) and 550K steps (1B, about 9K GPU-hours), and inference uses 32 NFE ODE steps for both Raon-OpenTTS and F5-TTS (§3.3, Appendix A, Table 11).

Raon-OpenTTS-Eval. The benchmark has 6,000 prompt-text pairs: 500 stratified prompts from each of 12 public datasets, each paired with a disjoint target sentence from the same dataset. The datasets are grouped into four regimes. Clean is LibriSpeech-clean, ST American English, CMU-ARCTIC, L2-ARCTIC, and VCTK. Noisy is LibriSpeech-other and TED-LIUM 3. Wild is AMI-IHM and AMI-SDM, with AMI-SDM restricted to segments that Whisper transcribes at 0% WER. Expressive is Expresso, CREMA-D, and EmoV-DB. Metrics are Whisper-normalized WER and WavLM speaker cosine similarity (§4.2). Human evaluation collects CMOS against Raon-OpenTTS-1B and SMOS from US Amazon Mechanical Turk raters: 30 items per regime, 6 raters per item (§5.4, Appendix B).

Key Results

Seed-TTS-Eval EN (Table 1). Raon-OpenTTS-1B reaches WER 1.78 and SIM 0.749. Its WER is second only to Qwen3-TTS (1.46, about 5M proprietary hours), and its SIM is the highest among open-weight models (VoxCPM is next at 0.730). The closed Seed-TTS still has higher SIM (0.762). The 0.3B model, which has the same architecture and size as F5-TTS, improves on it slightly (WER 1.95 vs 2.04, SIM 0.687 vs 0.671). The paper does not show that its training schedule matches the original F5-TTS run, so the gain cannot be credited to data alone.

CV3-Eval (Table 5). Raon-OpenTTS-1B has the lowest WER on CV3-EN (3.92, vs 4.52 for Qwen3-TTS) and on CV3-Hard-EN (6.15, vs 6.44 for VoxCPM), plus the highest Hard-EN SIM (0.775, vs 0.740 for CosyVoice 3 0.5B). Its DNSMOS (3.85) is below CosyVoice 3 (3.98), CosyVoice 2 (3.95), and Qwen3-TTS (3.87). F5-TTS fails on most CV3-Hard-EN inputs because they are too long.

Raon-OpenTTS-Eval (Table 6). The 1B model has the best overall WER (2.81) and SIM (0.695), and the 0.3B model has the second-best overall WER (2.93). The regime breakdown matters most. Every system stays within about 1.4 to 6.5% WER on Clean, Noisy, and Expressive, but the Wild regime separates them sharply. Raon models stay at 5.6 to 5.8% Wild WER and CosyVoice 3 at 8.31%, while MaskGCT (28.00), VoxCPM (43.83), CosyVoice 2 (49.73), Qwen3-TTS (79.14), and F5-TTS (136.03) degrade badly. Baselines win some cells: VoxCPM has the lowest Noisy WER (3.42), and VoxCPM (2.66) beats Raon-1B (2.77) on Expressive WER.

Human evaluation (Tables 7, 8). CMOS is reported relative to Raon-OpenTTS-1B, so negative values favour the 1B model. Overall, the closest systems are MaskGCT and Raon-OpenTTS-0.3B (both -0.01), effectively a tie. Baselines are preferred in some regimes: MaskGCT on Noisy (+0.31), Qwen3-TTS on Expressive (+0.25), and the 0.3B model on Clean (+0.54) and Wild (+0.16). The 1B model has the highest overall SMOS (3.70, vs 3.60 for the 0.3B and 3.59 for Qwen3-TTS), but the spread across systems is narrow (3.53 to 3.70). Confidence intervals are mentioned in the protocol but are not shown in the tables.

Data ablations (0.3B ablation variants, shorter schedules). Combined 15% filtering has the best average rank (3.4) against no filtering (4.9). Every filter does worse at 50% removal than at 15%, although DNSMOS 50% gives the best CV3-Hard-EN WER (7.20) (Table 3). At a matched 47K hours, a Pool subset beats English Emilia on overall WER (3.09 vs 3.33) and SIM (0.605 vs 0.558) but loses on Clean WER (1.53 vs 1.28) (Table 9). Adding Raon-YouTube-Commons (280K to 615K h) helps Clean, Wild, and Expressive WER but raises Noisy WER from 4.21 to 6.79 (Table 10).

Novelty Assessment

There is no architectural novelty: the model is F5-TTS, unchanged at 0.3B and widened at 1B. The contribution is a data-scale and curation result. The paper shows that a reproducible, permissively sourced (partly non-commercial) 510K-hour English corpus, built with standard tooling (an Emilia-style pipeline and Whisper/DNSMOS/VAD filters), can train a model that is competitive on standard English zero-shot benchmarks with systems trained on 2 to 10 times more proprietary data. The filtering recipe is simple rank averaging over three common quality signals, and its measured benefit is modest (Seed-TTS WER 2.00 vs 2.19 unfiltered). The more original evaluation contribution is Raon-OpenTTS-Eval, whose Wild regime exposes failure modes that single-source read-speech benchmarks miss. The paper is framed as the TTS analogue of open-data reproduction efforts such as OLMoASR and OpenCLIP, and its value lies in the released artifacts and controlled comparisons rather than in new modeling.

Field Significance

Moderate: this is a fully open-data, open-weight English zero-shot TTS system that matches or beats proprietary-data open-weight models on Seed-TTS-Eval and CV3-Eval with an unmodified published architecture. It releases the corpus, filtering pipeline, and checkpoints needed to study TTS data scaling and curation reproducibly, and its in-the-wild benchmark regime shows that conventional clean-speech benchmarks can hide large robustness gaps between otherwise comparable systems. Like other dataset and benchmark releases in this wiki, it is rated moderate until uptake of the corpus and benchmark is visible; the architecture itself is unchanged F5-TTS.

Claims

  • supports: Scaling a multi-source open training corpus, with the architecture held fixed, can bring open-data zero-shot TTS to parity with systems trained on far larger proprietary corpora on standard English benchmarks.

    Evidence: The unmodified F5-TTS DiT trained on 510K hours of filtered open data reaches Seed-TTS-Eval WER 1.78 / SIM 0.749 at 1B. That is second on WER behind Qwen3-TTS (about 5M h) and first on SIM among open-weight models. The same model has the lowest CV3-EN and CV3-Hard-EN WER and the highest Hard-EN SIM. (§5.1, Table 1; §5.2, Table 5)

  • complicates: Zero-shot TTS benchmarks built from clean read-speech prompts can hide severe robustness failures that appear when prompts come from spontaneous, in-the-wild recordings.

    Evidence: Qwen3-TTS has the best Seed-TTS-Eval WER (1.46) but reaches 79.14% WER on the Wild (AMI) regime of Raon-OpenTTS-Eval. CosyVoice 2 (49.73), VoxCPM (43.83), and F5-TTS (136.03) also degrade sharply there while staying within about 2 to 6.5% on Clean, Noisy, and Expressive. (§5.3, Table 6)

  • refines: Model-based quality filtering of large speech corpora helps most when it removes only the low-quality tail, and combining several quality signals gives the most consistent gains across benchmarks.

    Evidence: In 0.3B ablations at a fixed step budget, combined DNSMOS/WER/speech-ratio rank filtering at 15% removal has the best average rank (3.4) against no filtering (4.9). Every filter ranks worse at 50% removal than at 15%, and WER-50% ranks worst (7.6). Absolute gains are modest (Seed-TTS WER 2.00 vs 2.19). (§3.2, Table 3)

  • refines: At a fixed data budget, source diversity in TTS training data improves average robustness and speaker similarity but can cost accuracy on clean read speech.

    Evidence: A 47K-hour subset of the multi-source pool beats the same amount of English Emilia on overall WER (3.09 vs 3.33), Wild WER (5.97 vs 7.35), and SIM (0.605 vs 0.558), but has higher Clean WER (1.53 vs 1.28). (§6.1, Table 9)

  • complicates: Adding large volumes of in-the-wild web speech to TTS training does not improve every acoustic condition uniformly.

    Evidence: Adding 335K hours of processed YouTube-Commons lowers Clean (2.17 to 1.72), Wild (7.62 to 6.15), and Expressive (3.43 to 2.63) WER but raises Noisy WER from 4.21 to 6.79. The comparison confounds scale with domain. (§6.2, Table 10)

Limitations and Open Questions

Everything is English-only; the authors name multilingual extension as future work (§7). The headline comparisons mix data, model scale, and compute (the 1B model trains for 550K steps, about 9K GPU-hours). The 0.3B vs F5-TTS comparison is the closest to a controlled data test, but the paper does not show that its training schedule matches the original F5-TTS run.

  • Ablations use separate, shorter-trained 0.3B variants. The filtering, source, and YouTube-Commons ablations (Tables 3, 9, 10) do not use the released models, and the YouTube-Commons ablation confounds added scale with domain shift (the authors acknowledge this, §6.2).
  • Baseline protocol. Closed-weight results (Seed-TTS, CosyVoice 3 1.5B) are copied from the original papers. The CosyVoice 3 run on CV3-Eval and Raon-OpenTTS-Eval is the smaller 0.5B open checkpoint.
  • Benchmark construction. The authors built Raon-OpenTTS-Eval themselves. AMI-SDM is restricted to segments Whisper transcribes perfectly, which may favour ASR-friendly audio. SIM uses WavLM embeddings and WER uses Whisper, so the benchmark inherits those models’ biases.
  • Human evaluation is small. It uses 30 items per regime and 6 MTurk raters per item. CMOS is measured only against Raon-OpenTTS-1B, overall differences to the closest systems are 0.01, and confidence intervals are not shown in the tables.
  • Licensing. Parts of the pool (SPGISpeech under the Kensho agreement, Emilia-YODAS under CC BY-NC 4.0) are non-commercial, which limits how the released data and any model trained on the full Core subset can be used.
  • Quality ceiling. The 1B model’s DNSMOS on CV3-Hard-EN trails several proprietary-data systems, so the parity claim holds for intelligibility and similarity more clearly than for perceived quality.

Wiki Connections

  • Flow Matching — The models are unmodified F5-TTS flow-matching DiTs, which makes this a data-scaling study of a fixed flow-matching architecture.
  • Zero-Shot TTS — All evaluation is prompt-based zero-shot voice cloning, and the paper reports competitive Seed-TTS-Eval and CV3-Eval results trained only on open data.
  • Evaluation Metrics — Introduces Raon-OpenTTS-Eval, a 6K-prompt, 12-dataset benchmark split into Clean, Noisy, Wild, and Expressive regimes for robustness analysis.
  • Subjective Evaluation — Reports CMOS and SMOS from MTurk raters per acoustic regime, with a protocol in Appendix B.
  • F5-TTS — Its architecture is reused without change, and it is the same-size baseline that the 0.3B model beats on every headline benchmark.
  • DiTTo-TTS — An earlier KRAFTON DiT latent-diffusion TTS from overlapping authors, cited as a continuous-generation precursor; related work, not a duplicate.
  • Emilia — The source of the English Emilia and Emilia-YODAS portions of the pool, the template for the YouTube-Commons preprocessing pipeline, and the 47K-hour single-source comparison in the matched-scale ablation.
  • Qwen3-TTS — The strongest proprietary-data baseline on Seed-TTS-Eval WER, which degrades sharply on the Wild regime.
  • CosyVoice 3 — A proprietary-data baseline (0.5B open checkpoint evaluated) and the source of the CV3-Eval benchmark.
  • MaskGCT — An open-data baseline trained on Emilia that ties for closest overall CMOS but has 28% Wild WER.
  • Seed-TTS — The source of the Seed-TTS-Eval benchmark and a closed baseline with the highest Seed-TTS SIM (0.762).
  • VoxCPM — An open-weight baseline that is strong on Seed-TTS-Eval and Noisy/Expressive WER but degrades on Wild prompts and CV3-Hard SIM.