arXiv · 2025 · Preprint
Nomo Sudro et al. (The University of Sheffield) · → Paper · Demo: ✓ · Code: ?
The first comparative study of adult-to-child voice conversion using real acted dubbing speech from professional voice talents, evaluating GAN, VAE, flow-based, and diffusion generative model families, and introducing a trainable frequency-warping post-processing step that consistently improves speaker similarity across all four.
Problem
Generative voice conversion is well studied for adult-to-adult conversion, but its usefulness for adult-to-child conversion, needed for dubbing children’s media (translating dialogue while preserving tone, emotion, and lip-sync) has not been investigated. Dubbing specifically requires acted speech from professional voice talents rather than the read speech typically used in VC research, and children’s voice resources are limited, making this a genuinely under-studied, low-resource scenario. Prior adult-to-child VC work has been sparse and limited in scope: one study used only read speech and did not evaluate target-child speaker similarity, another performed dubbing for a different language using only a handful of words, and a third applied VC only to synthesized rather than real speech.
Method
The paper compares four generative model families for adult-to-child voice conversion, each following an established architecture from adult-to-adult VC literature: CycleGAN-VC2 (adversarial and cycle-consistency losses over Mel-cepstral coefficients, WORLD vocoder), a VAE-based system with separate speaker and content encoders trained with reconstruction plus KL-divergence loss (Griffin-Lim reconstruction), a Blow-style flow-based model that learns an invertible, exact-likelihood mapping between source and target speech frames end-to-end on raw audio conditioned on speaker identity labels (Griffin-Lim reconstruction), and a diffusion-based VC system (following a Grad-TTS-style forward/reverse diffusion process with a Transformer-based speaker-independent encoder, HiFi-GAN vocoder). To address a lack of target-speaker similarity the authors observed across all four models, they introduce a lightweight post-processing step: a trainable frequency-warping matrix, applied to Mel cepstral coefficients after conversion, that learns a per-frequency warping function minimizing the mismatch between the warped source spectrum and a reference target spectrum, trained with mean squared error.
Key Results
Across both an internal, professionally-acted adult-to-child dubbing corpus (5 hours, 3 speakers) and the public multi-speaker CMU Kids corpus (9.1 hours, 76 child speakers, paired with a re-recording by a single adult actor), the diffusion-based VC model achieves the lowest Mel-cepstral distortion (MCD) and F0 RMSE of the four families, with CycleGAN-VC2 second-best despite requiring separate training per speaker pair and lacking disentangled speaker/content representations. Applying the frequency-warping post-processing step reduces both MCD and F0 RMSE for every model on both corpora. Across different adult-child speaker pair combinations (varying in pitch, vocal tract characteristics, and speaking style), results vary substantially by pairing, and diffusion and flow-based models show the largest MCD reduction as training data increases from 1 to 5 hours, though flow-based MCD remains comparatively highest overall, which the authors attribute to end-to-end raw-audio synthesis needing more than 1-5 hours of data. In a 12-listener subjective evaluation (naturalness and similarity MOS), warping improves rated similarity for all four models while naturalness remains comparable to unwarped output.
Novelty Assessment
The comparison itself, evaluating four established VC architectures on a genuinely novel target domain (adult-to-child conversion using real acted dubbing speech from professional talents) rather than proposing a new architecture, is the paper’s primary contribution, and the authors explicitly position it as the first such study using acted (rather than read) speech data. The frequency-warping post-processing step is a smaller, secondary contribution: the warping technique itself is adapted from prior VC and speech-synthesis post-processing work rather than newly invented, but its systematic validation across all four model families and two corpora is a genuine empirical addition.
Field Significance
moderate — This is a narrowly-scoped but genuinely novel empirical study addressing a real, previously under-examined gap (adult-to-child VC using authentic acted dubbing speech) with a systematic four-model comparison and a validated, simple post-processing fix. Its practical value is centered on the dubbing application specifically; the underlying architectures are all established rather than novel, and evaluation is limited to a single research group’s internal dataset plus one small public child-speech corpus with a 12-listener subjective study.
Claims
- supports: Among established generative-model families, a diffusion-based voice conversion system can achieve better spectral and pitch fidelity than GAN-, VAE-, and normalizing-flow-based alternatives when converting adult speech to child-like target voices.
Evidence: Across both the internal adult-to-child dubbing corpus and the public CMU Kids corpus, the diffusion-based VC system achieves the lowest MCD and F0 RMSE of the four compared model families. (§4.4, Tables 1-2)
- supports: A lightweight, trainable frequency-warping post-processing step applied after voice conversion can measurably reduce spectral and pitch mismatch between converted and target-child speech, consistently across architecturally different VC model families.
Evidence: Applying the learned frequency-warping step after conversion reduces both MCD and F0 RMSE for every one of the four generative model families tested, on both the internal dubbing corpus and the CMU Kids corpus. (§4.4, Tables 1-2)
- complicates: A voice conversion model’s architectural ability to generalize across multiple speakers does not automatically translate into better adult-to-child conversion quality than a simpler, per-pair-trained alternative.
Evidence: CycleGAN-VC2, despite requiring separate training per speaker pair with no disentangled speaker/content representations, performs competitively with or better than the multi-speaker-capable VAE and flow-based models on MCD, while showing minimal further improvement from additional training data compared to diffusion and flow-based models. (§4.4, Table 3, Figure 5)
- complicates: A post-processing step that improves objective spectral-distortion metrics after voice conversion does not necessarily improve perceived naturalness, even when it improves perceived speaker similarity.
Evidence: Subjective evaluation shows frequency warping improves listener-rated similarity for all four VC model families, but naturalness MOS remains comparable to or slightly lower than unwarped output. (§4.5, Table 4)
Limitations and Open Questions
The internal dubbing corpus is small (5 hours, 3 speakers, single speaker pair per model in the main comparison) and not publicly released, limiting reproducibility and generalization beyond the specific speakers studied. The paper notes that CycleGAN-VC2’s one-to-one speaker-pair training design would require separate training for each new speaker pair, an impractical constraint for multi-speaker dubbing at scale, unlike the other three model families. The authors identify prosody and intonation modeling for child voice conversion, not addressed in this study, as future work.
Wiki Connections
- Voice Conversion — the first comparative study of adult-to-child voice conversion using real acted dubbing speech, evaluating GAN, VAE, flow-based, and diffusion generative model families.
- Subjective Evaluation — reports naturalness and similarity MOS from a 12-listener study comparing converted speech with and without frequency-warping post-processing.