PROPOR · 2026 · Workshop

Gustavo Evangelista Araújo et al. (University of São Paulo (USP)) · → Paper · Demo: ? · Code: ✓

Releases a 70-hour, 190-speaker corpus of spontaneous, noisy Brazilian Portuguese radio dialogue from 1980s-90s analog tapes, with manually verified transcripts, speaker labels, and artifact annotations, and reports single-speaker YourTTS and F5-TTS fine-tuning baselines as a robustness testbed for TTS.

Problem

Brazilian Portuguese TTS resources are dominated by clean, read, studio-style speech (Multilingual LibriSpeech, BRSD v2, Common Voice, CML-TTS, TTS-Portuguese). The spontaneous Portuguese corpora that do exist (CORAA ASR, NURC-SP, MuPe Life Stories) are interview-based and acoustically comparatively homogeneous. The authors argue that no Portuguese resource captures broadcast conversational speech with overlapping turns, music and jingle interference, and analog tape artifacts, which leaves TTS robustness under real-world acoustic and interactional variability largely untested for the language (§1, §2, Table 1).

Method

The corpus is drawn from Certas Palavras, a radio program on books and ideas, digitized from the CEDAE archive. Transcription starts from WhisperX (Whisper large-v2) with Pyannote-Audio VAD and diarization; audio is resampled to 16 kHz mono. Twelve annotators then manually corrected transcripts and replaced generic diarization IDs with real speaker identities over six months, following 11 annotation rules shared with MuPe Life Stories and NURC-SP: orality markers, filled pauses, repeated hesitations, numbers and letters, acronyms, foreign terms, punctuation, expressive sounds (e.g., laughter), misunderstood words, segmentation corrections, and segments with music or sound effects (§3.1). Diarization agreement among the 12 raters, measured with Fleiss’ kappa on two programs, is 0.809 and 0.962 (§3.1).

The released corpus totals 70.98 hours, 689,574 words, and 42,486 segments (mean 5.95 s), with 190 speakers (two main hosts, one recurring co-host, and 187 guests). Artifact annotations include 8,968 sentences with filled pauses, 1,807 with music/SFX, 972 with unintelligible words, 609 with segmental errors, and 594 with emotional content. Birthplace metadata covers roughly two thirds of speakers, which the authors explicitly describe as auxiliary rather than a balanced accent inventory (§3.2, Table 2). The license is CC BY-NC 4.0 (Table 1).

For baselines, the authors restrict training to the most represented voice, host Claudiney Ferreira (a 9-hour subset), to avoid confounding results with speaker data scarcity (§4.4). Two existing models are fine-tuned: YourTTS (a VITS-style end-to-end model with a VAE posterior encoder, flow-based decoder, monotonic alignment search, and stochastic duration predictor), pre-trained on CML-TTS; and F5-TTS, a non-autoregressive conditional flow-matching model with a Diffusion Transformer backbone that predicts mel-spectrograms vocoded by Vocos (§4.1, §4.2). Training uses fp16 mixed precision, AdamW (β1 = 0.8, β2 = 0.99), a constant 2e-4 learning rate, and batch size 32 on 8 A100 GPUs, with hyperparameters shared across regimes (§4.3). Selection scripts, seeds, split manifests, and a Docker image are released (§4).

Evaluation combines a naturalness MOS test (1-5 ACR scale) with 27 expert listeners split across two balanced lists, where no listener hears both the natural and synthetic version of the same utterance, and natural samples containing music or sound effects are excluded (§4.5, §4.7). Inter-rater agreement is reported with Kendall’s W. Intelligibility is measured by ASR back-evaluation (WER, CER) using an ASR model trained on spontaneous Brazilian Portuguese (from the MuPe Life Stories work), with two-sample t-tests between original and synthetic audio (§4.6, §5).

Key Results

Ground-truth audio receives a MOS of 4.56 (σ 0.77), while fine-tuned YourTTS reaches 3.28 (Table 4). MOS was collected only for YourTTS, so no subjective comparison between the two baselines exists (§5). Kendall’s W is 0.079 for ground truth and 0.278 for YourTTS (Table 4), meaning agreement is low in both conditions and nominally higher for the synthetic samples.

On ASR back-evaluation, ground-truth audio already yields 13.70 WER and 7.45 CER, which reflects the difficulty of the spontaneous, noisy material for the ASR model itself. Fine-tuned YourTTS reaches 23.86 WER and 11.79 CER; fine-tuned F5-TTS reaches 18.12 WER and 10.49 CER. All differences from ground truth are statistically significant (p < 0.01, except F5-TTS CER at p < 0.05) (Table 5). F5-TTS is better than YourTTS on both intelligibility metrics, but only one target speaker and one ASR system are tested (§6).

The reporting has some internal inconsistencies. The synthetic MOS standard deviation is 1.77 in Table 4 but 1.17 in the §5 text. The abstract and conclusion describe higher inter-rater variability for synthetic speech, which matches the larger standard deviation but not the Kendall’s W values, where synthetic samples show more rank agreement than ground truth. The paper also presents no model trained on a clean Portuguese corpus as a control, so the conclusion that exposure to the corpus “promotes increased robustness” (§7) is not directly tested by the experiments.

Novelty Assessment

The contribution is the dataset itself: a broadcast-conversational Brazilian Portuguese corpus with manually verified diarization, speaker identity, and a detailed artifact annotation layer covering music, noise, disfluencies, and segmentation errors. This combination is not present in the other Portuguese corpora the paper lists (Table 1). The annotation protocol is reused from the same group’s earlier MuPe and NURC-SP work rather than new. The TTS experiments are baselines only: two off-the-shelf architectures fine-tuned on one speaker, with no ablation of the annotated artifact categories, no clean-data control, and MOS for only one model. The listening-test design (split lists, no paired exposure, randomized order) is careful but follows prior practice the authors cite.

Field Significance

Moderate — The paper provides a rare publicly licensed resource for studying TTS trained on spontaneous, multi-speaker, acoustically degraded broadcast speech in a non-English language, together with fixed splits and an evaluation protocol that later work can reuse. Its baseline experiments are confirmatory evidence that current fine-tuned TTS systems keep intelligibility but lose naturalness on such data, not a new modeling result.

Claims

  • supports: TTS models fine-tuned on spontaneous, noisy archival broadcast speech retain usable intelligibility but show a clear loss of perceived naturalness relative to the source recordings.

    Evidence: YourTTS fine-tuned on a 9-hour single-host subset of Certas Palavras reaches a MOS of 3.28 against 4.56 for ground truth, while its ASR back-evaluation WER rises from 13.70 on natural audio to 23.86 on synthetic audio. (§5, Table 4, Table 5)

  • supports: Non-autoregressive flow-matching TTS preserves linguistic content better than a VAE-based end-to-end model when both are fine-tuned on noisy, spontaneous single-speaker data.

    Evidence: F5-TTS fine-tuned on the same Certas Palavras host subset obtains 18.12 WER and 10.49 CER against 23.86 WER and 11.79 CER for fine-tuned YourTTS, under the same ASR scoring protocol; only one speaker and one ASR model are tested. (§5, Table 5, §6)

  • complicates: ASR back-evaluation of TTS trained on spontaneous or degraded speech must be read against the ASR error on ground-truth audio, because the recognizer itself struggles on the source material.

    Evidence: Even with an ASR model trained on spontaneous Brazilian Portuguese, ground-truth Certas Palavras audio yields 13.70 WER and 7.45 CER, so absolute synthetic WER values conflate synthesis errors with recognizer difficulty on the domain. (§5, Table 5)

  • complicates: Subjective naturalness ratings for natural recordings from noisy archival sources show low listener agreement, making ground-truth MOS a less stable ceiling than for clean studio corpora.

    Evidence: Ground-truth broadcast audio receives MOS 4.56 with σ 0.77 and a Kendall’s W of only 0.079 across 27 expert listeners, even after excluding natural samples containing music or sound effects; the synthetic condition shows a W of 0.278. (§4.7, §5, Table 4)

Limitations and Open Questions

Warning

All TTS experiments use a single target speaker (the main host), MOS is collected only for YourTTS, and no model trained on clean Portuguese data is included as a control, so the corpus’s value for multi-speaker dialogue synthesis and the claim that training on it improves robustness remain untested.

  • Audio is standardized to 16 kHz, which caps spectral bandwidth and makes the corpus better suited to robustness studies than high-fidelity synthesis; a 48 kHz release is planned (§6).
  • Intelligibility relies on a single ASR system trained on spontaneous Brazilian Portuguese from the same research group (§6).
  • The effect of the specific annotated artifact categories (music, hiss, laughter, overlaps) on synthesis is not analyzed; this is left to future stress-test studies (§6).
  • Reported statistics are partly inconsistent (synthetic MOS σ 1.77 in Table 4 vs 1.17 in §5), and the stated interpretation of inter-rater variability does not match the Kendall’s W direction.
  • Training used fp16 mixed precision; the authors plan to replicate with fp32 (§6).
  • The CC BY-NC 4.0 license restricts commercial use (Table 1).
  • Speaker birthplace metadata is incomplete (about a third of speakers lack it) and not accent-balanced (§3.2, Table 2).

Wiki Connections

  • Evaluation Metrics — Releases fixed splits and a WER/CER ASR back-evaluation protocol for TTS on noisy, spontaneous Portuguese, and shows that ground-truth ASR error must be reported alongside synthetic error.
  • Subjective Evaluation — Runs a naturalness MOS test with 27 expert listeners on split lists with no paired exposure, and reports Kendall’s W showing low agreement on archival ground-truth audio.
  • Flow Matching — Fine-tunes a flow-matching TTS model on noisy broadcast speech as a baseline, where it gives better intelligibility than a VAE-based alternative.
  • F5-TTS (flow-matching TTS) — Used as one of the two fine-tuned baselines on the single-host subset.
  • HiFiTTS-2 (large-scale high-bandwidth TTS corpus) — Listed in the corpus comparison as an example of large but clean, read-speech TTS data that Certas Palavras complements.
  • XTTS (multilingual zero-shot TTS) — Cited as context for the naturalness of modern TTS that, the authors argue, still depends on clean, homogeneous training data.