arXiv · 2026 · Preprint

Renhe Sun et al. (Ant Group / The Chinese University of Hong Kong / Zhejiang University) · → Paper · Demo: ? · Code: ?

A WildSpoof 2026 TTS Track technical report that fine-tunes F5-TTS on in-the-wild TITW data with an exponential moving average of the weights and picks reference prompts with two LLM scoring stages (audio scoring by an audio language model, text alignment by a text LLM); the resulting system obtained the highest a-DCF of all submissions against the challenge’s three spoofing-aware speaker verification systems.

Problem

TTS systems are usually developed on curated studio recordings, and they struggle when trained on or prompted with noisy, acoustically diverse in-the-wild speech. The WildSpoof 2026 Challenge TTS Track asks participants to build TTS from in-the-wild data (the TITW corpus) and scores submissions both on standard quality metrics and on how hard their output is for spoofing-aware speaker verification (SASV) systems to reject. The authors target two failure points in this setting: unstable fine-tuning on noisy, heterogeneous data, and poor or misaligned reference prompts degrading prompt-based synthesis.

Method

The system, F5-TTS-DPS, starts from the public F5-TTS v1 base checkpoint (a non-autoregressive flow-matching model with a diffusion-transformer backbone, pre-trained on Emilia; see F5-TTS) and is not modified architecturally. Two additions are made.

EMA-stabilized fine-tuning. The checkpoint is fully fine-tuned on the challenge’s TITW-easy and TITW-hard subsets while keeping an exponential moving average (EMA) of the parameters, with EMA beta 0.99. The motivation is to damp unstable gradients and overfitting to recording-specific artifacts in noisy data. Training runs 10 epochs at learning rate 1e-6 with 20,000 warmup updates, up to 64 sequences and 38,400 frames per device per batch, no gradient accumulation, and gradient clipping at max norm 1.0. Model size is not reported.

Dual-scoring prompt selection (DPS). F5-TTS synthesizes by infilling masked frames after a reference speech prompt, so the choice of prompt matters. DPS filters candidate prompts in two stages. First, Qwen2.5-Omni scores candidate audio prompts for emotional richness, voice expressiveness, and prompt suitability, and low-scoring clips are discarded. Second, the text LLM Qwen3-30B-A3B (Qwen3) compares the target text with the remaining candidates’ reference texts on prosodic alignment, emotional congruence, linguistic compatibility, and overall suitability (each scored 0 to 10 in the prompt reproduced in the appendix) and outputs the chosen reference. The paper does not describe the size or composition of the candidate pool.

Evaluation follows the challenge protocol with VERSA: UTMOS and DNSMOS for naturalness, WER from Whisper ASR, and speaker similarity from ESPnet2 speaker embeddings. The authors add their own Spoofing Detection Score (SDS), the confidence of an AASIST countermeasure trained on ASVspoof 2019 LA (higher means more likely synthetic). The official evaluation also reports a-DCF against three SASV systems (T01, T02, T08) from the challenge’s SASV track.

Key Results

Development set ablation (Table 1). Starting from the F5-TTS baseline (UTMOS 3.06, DNSMOS 2.91, WER 12.31%, SPK-SIM 0.450, SDS 0.283), each addition changes the metrics as follows:

ConfigurationUTMOSDNSMOSWERSPK-SIMSDS
CosyVoice23.652.797.760.4030.343
F5-TTS baseline3.062.9112.310.4500.283
+ SFT3.062.5410.600.4890.226
+ SFT + EMA3.182.619.320.4920.181
+ SFT + EMA + DPS3.202.618.650.5080.108

The full system has the best SPK-SIM and the lowest SDS in the table. It does not lead on the other metrics: CosyVoice 2 has higher UTMOS (3.65 vs. 3.20) and lower WER (7.76% vs. 8.65%), and the un-fine-tuned F5-TTS baseline has the highest DNSMOS (2.91 vs. 2.61). Fine-tuning lowered DNSMOS (2.91 to 2.54), and EMA and DPS recovered only part of that drop. The step from SFT to SFT+EMA improves UTMOS (3.06 to 3.18) and WER (10.60% to 9.32%) with almost no SPK-SIM change; DPS, which only changes inference-time prompts, gives the largest single SDS reduction (0.181 to 0.108).

Official challenge evaluation, seen speakers (Table 2). Team T05 (this system) obtained a-DCF of 0.1582 / 0.5233 / 0.2562 against SASV systems T01 / T02 / T08, the highest of all seven submissions in each column (the next highest, T06, scored 0.1527 / 0.3786 / 0.2292; the remaining teams scored at most 0.0471 / 0.1782 / 0.1125). The authors read higher a-DCF as speech that is harder for the SASV systems to reject. On the other metrics the system is mid-pack or below:

  • WER 8.65%, third behind T02 (5.50%) and T01 (6.48%).
  • UTMOS 3.2016, sixth of seven submissions (T01 3.9559, T02 3.7390, T07 3.5292, T06 3.4909, and T03 3.4540 all score higher; only T04 is lower).
  • DNSMOS 2.6078, the lowest of the seven submissions (only the B01 baseline is lower).
  • SPK-SIM 0.2798, fifth of seven (T03 0.4782, T06 0.4775, T02 0.3511, and T07 0.2895 are higher).

SPK-SIM on the official evaluation (0.2798) is much lower than on the development set (0.508). The paper does not discuss the gap, so it cannot be attributed to protocol differences or to a drop in system quality from the text alone. No human listening test is reported, and all comparisons are single runs without variance estimates.

Novelty Assessment

The contribution is engineering and recipe-level. The architecture is F5-TTS unchanged, EMA weight averaging is a long-established optimization technique (the paper cites Polyak and Juditsky and mean-teacher work), and full-parameter fine-tuning on the challenge data is the obvious adaptation step. The most distinctive element is DPS, which uses an audio language model and a text LLM as zero-training judges to choose reference prompts for prompt-based TTS. The selection criteria are hand-written LLM prompts, there is no comparison against simpler selection heuristics (signal-quality filters, random choice, text-length matching), and the two DPS stages are not ablated separately, so how much of DPS’s effect comes from each stage is unknown.

Field Significance

low: This report adds a data point that prompt selection and EMA-stabilized fine-tuning can make an off-the-shelf flow-matching TTS model harder for spoofing-aware verification systems to reject when trained on in-the-wild data, and it shows that detectability rankings can diverge sharply from naturalness and similarity rankings in the same challenge. It is a short challenge report with no new architecture, no human evaluation, and no ablation of the selection design.

Claims

  • supports: Selecting reference prompts at inference time, without retraining, can measurably improve speaker similarity, intelligibility, and resistance to spoofing detection in prompt-based TTS on noisy in-the-wild data.

    Evidence: Adding LLM/audio-LM dual-scoring prompt selection on top of the fine-tuned F5-TTS raises SPK-SIM from 0.492 to 0.508, cuts WER from 9.32% to 8.65%, and lowers the AASIST-based SDS from 0.181 to 0.108 on the WildSpoof development set. (§3.1, Table 1)

  • supports: Maintaining an exponential moving average of the weights during fine-tuning on noisy in-the-wild speech can improve naturalness and intelligibility over plain fine-tuning.

    Evidence: SFT+EMA vs. SFT alone improves UTMOS from 3.06 to 3.18 and WER from 10.60% to 9.32%, with SPK-SIM nearly unchanged (0.489 to 0.492). (§3.1, Table 1)

  • complicates: Fine-tuning a clean-data TTS foundation model on in-the-wild speech can trade signal-quality scores for speaker similarity and intelligibility.

    Evidence: SFT lowers DNSMOS from 2.91 to 2.54 while raising SPK-SIM and lowering WER; the full system ends at DNSMOS 2.61, still below the un-fine-tuned baseline. (§3.1, Table 1)

  • complicates: Being hardest for spoofing-aware verification systems to reject does not imply the highest naturalness or speaker similarity under automatic metrics.

    Evidence: The system has the highest a-DCF against all three SASV systems among seven submissions, yet ranks sixth on UTMOS, last on DNSMOS, and fifth on SPK-SIM in the same official evaluation. (§3.1, Table 2)

Limitations and Open Questions

  • The paper equates high a-DCF with “the highest degree of naturalness and authenticity,” but its own automatic naturalness scores (UTMOS, DNSMOS) and speaker similarity on the official evaluation are below several other teams, and no human listening test is reported. Detectability and perceived naturalness are measured separately and do not agree here.
  • The official evaluation table covers seen speakers only, so performance on unseen-speaker voice cloning is not reported.
  • The development-set SPK-SIM (0.508) and official SPK-SIM (0.2798) differ widely, and the report does not explain why.
  • The DPS candidate pool, the score thresholds used to discard prompts, and the cost of running two large models per synthesis are not described. The two DPS stages are not ablated individually, and there is no comparison with non-LLM prompt filters.
  • SDS uses an AASIST model trained on ASVspoof 2019 LA, an older countermeasure, so lower SDS may reflect that detector’s blind spots rather than general undetectability.
  • Code and demo availability are not stated.

Wiki Connections

  • Flow Matching — The system is an F5-TTS flow-matching model fully fine-tuned on in-the-wild data with EMA, with gains coming from training recipe and prompt selection rather than architecture.
  • F5-TTS — Base model and starting checkpoint (v1 base, Emilia-pretrained); F5-TTS-DPS keeps its architecture unchanged and serves as the ablation baseline.
  • CosyVoice 2 — Development-set comparison system; it beats the proposed system on UTMOS and WER but has lower speaker similarity and higher SDS.
  • TITW Database — Source of the TITW-easy and TITW-hard in-the-wild training subsets used for fine-tuning.
  • Qwen2.5-Omni — Used as the audio language model judge in the first DPS stage to score candidate audio prompts.
  • Qwen3 — Qwen3-30B-A3B is the text LLM judge in the second DPS stage, checking alignment between target and reference texts.
  • UTMOS — Naturalness metric used in both the development ablation and the official challenge evaluation.
  • Self-Purifying Flow Matching (WildSpoof) — Another WildSpoof 2026 TTS Track submission built on flow matching, addressing the same in-the-wild training setting.
  • BSC WildSpoof Submission — Another WildSpoof 2026 TTS Track submission, focused on enhancing audio prompts for zero-shot TTS, a related angle to this paper’s prompt selection.