arXiv · 2025 · Preprint
Di Bernardo et al. (Universidad Nacional de Tres de Febrero) · → Paper · Demo: ✗ · Code: ✓
Introduces a composite, metric-driven methodology for comparing preprocessing pipelines for in-the-wild TTS corpus creation without training a TTS model for every candidate configuration.
Problem
Building TTS training corpora from in-the-wild audio (Internet-harvested or crowdsourced recordings) requires a preprocessing pipeline, typically some combination of voice activity detection, denoising or speech enhancement, and quality-based filtering, before the data is usable for training. Many such pipelines have been proposed, each reporting downstream TTS results with its own choice of denoiser, quality predictor, and threshold, but the paper argues that the literature lacks a systematic acoustic comparison quantifying how individual preprocessing choices affect measurable audio properties. Because full pipeline evaluation normally means training a TTS model on each candidate subset, this comparison is expensive, and it becomes difficult to identify which pipeline components (denoiser choice, filtering threshold, quality predictor) are actually responsible for a given quality outcome.
Method
The paper proposes a composite objective score for ranking preprocessing configurations directly from audio measurements, computed before (raw) and after (processed) each candidate pipeline run. Four sub-scores are combined: dataset reduction (relative loss of duration), signal quality (PESQ, SI-SDR via Torchaudio-Squim, and WADA-SNR), acoustic parameters (T30 reverberation time and C50 clarity, from a CNN model validated for Argentine Spanish), and speech differences (deviation in F0 standard deviation via PESTO, and percentage increase in mean mel-cepstral distortion relative to a 5 dB reference, computed only for denoised audio). The four sub-scores are summed unweighted into a single objective that is minimized to select the best configuration, though the paper notes weights could be introduced to prioritize particular criteria.
Alongside the evaluation methodology, the authors build a CPU-friendly preprocessing chain from existing components: Silero VAD for voice activity detection, extended with a Whisper-timestamp-based classification of segments into slow/normal/fast speech rate and a Tree-Structured Parzen Estimator (TPE) search for per-category VAD hyperparameters; DeepFilterNet or Demucs for optional denoising; NISQA or DNSMOS as non-intrusive quality filters; and Whisper Large for transcription, chosen over faster CPU-oriented alternatives because those proved less reliable for transcription accuracy. No TTS model is trained or evaluated anywhere in the paper; the chain’s output is a curated, transcribed audio corpus intended as TTS training material for others to consume downstream.
As a testbed, the authors apply this pipeline to a newly collected 24-hour, 59-speaker in-the-wild Argentine Spanish corpus spanning diverse regional accents, which the paper positions as the first public wild-harvested corpus for this variety of Spanish. They evaluate 24 configurations crossing three denoising conditions (DeepFilterNet, Demucs, no denoising) with four NISQA thresholds and four DNSMOS thresholds.
Key Results
Table 1 breaks down one representative configuration (DeepFilterNet + NISQA threshold 3.8) stage by stage: relative to the 24.3-hour raw corpus, the accepted subsets show consistent gains in PESQ, SNR, and SI-SDR alongside lower T30 and higher C50, with reduced variance in every metric. Across all 24 configurations, no-denoising variants achieve the largest PESQ/SI-SDR/SNR gains under filtering (more than 10%) but retain the fewest hours, while denoised variants improve by no more than about 8% but keep proportionally more data, illustrating a direct quality-versus-quantity trade-off. Demucs consistently yields larger signal-quality and acoustic-parameter gains than DeepFilterNet (about a 5% advantage on C50), while the no-denoising variant best preserves F0 variability and is not penalized on MCD (Table 2). Ranking five representative configurations by the composite score, Demucs paired with the most permissive DNSMOS threshold (2.7) scores best overall, since for this corpus’s relatively good starting conditions the marginal quality gains from stricter filtering do not offset the data lost.
A secondary result concerns quality-predictor choice itself: DNSMOS scores over the raw corpus have a lower median and smaller variance than NISQA scores on the same recordings, so nominally similar threshold adjustments on DNSMOS admit or reject a substantially larger share of utterances than the same adjustment on NISQA. No comparison to a trained TTS model’s output quality is made anywhere in the paper; all reported numbers are signal- and acoustic-level measurements on raw versus processed recordings, not synthesized speech.
Novelty Assessment
The contribution is primarily methodological and infrastructural rather than architectural. The composite before/after scoring scheme (dataset reduction, signal quality, acoustic parameters, speech preservation) is a genuinely new way to compare preprocessing configurations without incurring the cost of training a TTS model per candidate, and it directly targets a gap the authors identify in prior pipeline papers, which typically report only downstream TTS results for a single chosen configuration. The preprocessing chain itself, however, is an engineering integration of existing off-the-shelf components (Silero VAD, DeepFilterNet, Demucs, NISQA, DNSMOS, Whisper); its adaptive VAD hyperparameter search via TPE is a modest addition to the VAD stage rather than a new algorithm. The Argentine Spanish corpus is a real dataset contribution, though it is a 24-hour testbed used to validate the methodology rather than a large-scale public release (the paper states the corpus will be prepared for release only after further pipeline work). Because no TTS model is trained on any of the curated subsets, the paper’s central premise, that the composite score is a valid proxy for downstream TTS training utility, is asserted rather than demonstrated.
Field Significance
Moderate — the paper fills a real methodological gap by proposing a reproducible, model-free way to compare preprocessing pipelines for in-the-wild TTS corpus creation, and packages a low-cost, CPU-only reference pipeline for research groups with limited hardware. Its contribution is narrower than a full TTS-facing evaluation, however: the composite score is validated only against itself across 24 configurations on a single small, single-language corpus, and the paper does not yet show that its ranking correlates with actual downstream TTS quality.
Claims
- supports: Composite objective audio-quality metrics computed before and after preprocessing can rank candidate data-curation pipelines for TTS corpus creation without requiring a full TTS training run for each candidate, offering a lower-cost model-selection substitute for pipeline comparison studies.
Evidence: A composite score combining dataset reduction, signal quality, acoustic parameters, and speech preservation is computed for 24 denoiser × filter-threshold configurations and used to rank them purely from raw/processed audio measurements, identifying a Demucs + permissive-DNSMOS configuration as the best trade-off without training any TTS model. (§2, §4, Table 2)
- complicates: Preprocessing choices for in-the-wild TTS corpora trade dataset retention against measured signal quality, so no single denoiser or filter setting is unambiguously best independent of how much data loss is tolerated.
Evidence: No-denoising variants show the largest PESQ/SI-SDR/SNR gains (>10%) under filtering but discard the most hours, while denoised variants gain at most about 8% but retain proportionally more audio; stricter thresholds always yield fewer, more uniform samples at the cost of corpus size. (§4, Figure 2)
- complicates: Non-intrusive quality predictors used to threshold in-the-wild speech corpora are not interchangeable, so a filtering threshold tuned on one predictor does not transfer directly to another.
Evidence: DNSMOS scores over the raw corpus show a lower median and smaller variance than NISQA scores on the same recordings, so nominally equivalent DNSMOS threshold adjustments admit or reject a larger share of utterances than the same adjustment applied to NISQA. (§3.3, Figure 1)
- complicates: Generative speech-enhancement models applied to in-the-wild recordings for TTS corpus preparation risk altering speaker timbre and prosody even as they raise perceptual quality scores, so preprocessing pipelines need explicit voice-preservation checks alongside signal-quality metrics.
Evidence: The paper reports that certain generative restoration methods increase perceived quality while making audio sound more robotic, toneless, or broadcast-like, motivating the explicit F0-STD and MCD speech-preservation metrics tracked alongside signal-quality scores for the two denoisers evaluated. (§3.2)
Limitations and Open Questions
Warning
The paper’s central premise, that its composite objective score can substitute for training a TTS model on each candidate subset, is not validated against any actual TTS training outcome. The authors themselves state this remains essential future work: measuring the correlation between the composite score and downstream TTS performance by training representative TTS models on metric-selected subsets.
Beyond that open validation question, the evaluation testbed is a single 24-hour, single-language (Argentine Spanish) corpus with 59 speakers, so the ranking of denoisers and thresholds may not generalize to other languages, recording conditions, or corpus scales. The transcription stage remains an operational bottleneck: Whisper Large is used for accuracy despite being CPU-slow, and faster CPU-oriented STT alternatives tested by the authors proved less reliable. The pipeline also lacks a speaker diarization component, planned as future work, and the Argentine Spanish corpus itself is not yet publicly released; the authors state it will be prepared for release only after integrating STT and diarization improvements.
Wiki Connections
- Evaluation Metrics — introduces a composite objective score (dataset reduction, signal quality, acoustic parameters, speech preservation) as a model-free proxy for ranking TTS data-preprocessing pipelines, an evaluation methodology rather than a listening-test or system-level metric.
- Emilia — cited as a contrasting design choice: the paper notes Emilia relies on source separation (UVR-MDXNet) alone for enhancement, whereas this pipeline evaluates dedicated denoisers (DeepFilterNet, Demucs) and explicitly tracks voice-preservation metrics to avoid the timbre and prosody drift that generative enhancement can introduce.
- TITW — cited alongside this paper’s own corpus as part of the broader motivation for in-the-wild TTS data collection, though TITW addresses TTS-in-the-wild evaluation rather than preprocessing-pipeline comparison.