concept: zero-shot-tts last_updated: '2026-07-26' paper_count: 203 papers: - id: '1712.05884' published_date: "2017-12-16" entry_date: '2026-07-26' year: 2017 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: historical_context method_family: - encoder_decoder_voice_cloning claims: - claim_id: tacotron2_mel_intermediate_bridges_e2e role: supports claim: A compact low-level acoustic intermediate representation can bridge text encoding and neural vocoder synthesis without requiring hand-crafted linguistic features, enabling fully end-to-end neural TTS at human-quality levels. source: §2.1, Table 1 evidence: Tacotron 2 reaches MOS 4.526 against a ground-truth MOS of 4.582, closing nearly the entire gap to human recordings on the internal single-speaker test set. confidence: high relevance: medium - claim_id: tacotron2_location_sensitive_attention role: supports claim: Location-sensitive attention, which incorporates cumulative attention weights as a conditioning signal, reduces failure modes such as repetition and omission compared to standard additive attention in autoregressive TTS decoders. source: §2.2 evidence: The paper adopts location-sensitive attention specifically to address repetition/omission failure modes observed with plain additive attention in the original Tacotron. confidence: medium relevance: low - claim_id: tacotron2_prenet_bottleneck_essential role: supports claim: A pre-net information bottleneck in the autoregressive decoder is essential for stable attention alignment during training. source: §2.2 evidence: The two-layer 256-unit pre-net bottleneck is described as necessary for stable attention learning in the decoder design. confidence: medium relevance: low - claim_id: tacotron2_vocoder_train_on_predicted role: supports claim: Training the vocoder on predicted rather than ground-truth intermediate features is necessary because predicted features are systematically oversmoothed; vocoders trained on clean features degrade when exposed to predicted inputs. source: §3.3.1, Table 2 evidence: Training WaveNet on ground-truth spectrograms and then synthesising from predicted (oversmoothed) spectrograms is the worst configuration in the paper's ablation (MOS 4.362 vs. 4.526 for the matched train/inference condition). confidence: high relevance: low - claim_id: tacotron2_mel_shrinks_vocoder_receptive_field role: supports claim: Neural vocoders conditioned on compact mel spectrograms can operate with substantially smaller receptive fields than those conditioned on linguistic features, enabling significant architecture simplification without quality loss. source: §3.3.4, Table 4 evidence: A 12-layer WaveNet with a 10.5 ms receptive field matches the quality of the full 30-layer 256 ms model when conditioned on mel spectrograms. confidence: medium relevance: low limitations: - All experiments use a single proprietary speaker and internal, unreleased dataset, leaving generalisation to multiple speakers, languages, or recording conditions untested. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '1904.02882' published_date: "2019-04-05" entry_date: '2026-07-26' year: 2019 venue: arXiv task: - TTS architecture: [] relevance: medium evidence_role: - infrastructure current_role: influential method_family: [] claims: - claim_id: libritts_24khz_higher_mos_than_16khz role: supports claim: Training TTS models at 24 kHz rather than 16 kHz produces measurably higher subjective naturalness scores even when the underlying model architecture is held constant. source: §5.2, Table 4 evidence: GMVAE-Tacotron trained on LibriTTS at 24 kHz outperforms the 16 kHz LibriTTS models by an average of 0.175 MOS for female speakers and 0.133 MOS for male speakers. confidence: high relevance: medium - claim_id: libritts_segmentation_effect_confounded role: complicates claim: Sentence-boundary segmentation and preserved punctuation are theoretically beneficial for TTS training, but their effect is difficult to isolate empirically because corpus size and acoustic quality covary when comparing corpus versions. source: §5.2 evidence: Comparing LibriSpeech (16 kHz, 460h) with LibriTTS (16 kHz, 245h) at matched sampling rate shows a 0.15 MOS gain for female speakers but no consistent gain for male speakers, and the authors explicitly acknowledge in §5.2 that the corpora differ in size, confounding attribution to text-processing quality. confidence: medium relevance: medium - claim_id: libritts_snr_filtering_tradeoff role: supports claim: 'SNR-based filtering of audiobook recordings improves corpus quality at the cost of significantly reducing yield: approximately 25% of clean-subset sentences are removed by a 20 dB threshold.' source: §4, Table 3 evidence: A WADA-SNR threshold of 20 dB on the clean subsets discards roughly a quarter of candidate sentences during corpus construction. confidence: high relevance: low - claim_id: libritts_gender_disparity_in_mos role: supports claim: Gender disparities in TTS naturalness persist under multi-speaker training on large open corpora, with male voices showing larger gaps to natural speech than female voices. source: §5.2, Table 4 evidence: The naturalness gap to natural speech is 0.61 MOS for male speakers versus 0.16 MOS for female speakers in the LibriTTS 24 kHz evaluation. confidence: high relevance: medium limitations: - Experimental validation uses a single model architecture (GMVAE-Tacotron) and evaluates only six held-out speakers, limiting the generality of the corpus-quality findings. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2010.05646' published_date: "2020-10-12" entry_date: '2026-07-26' year: 2020 venue: arXiv task: - TTS architecture: - GAN relevance: low evidence_role: - infrastructure current_role: minor method_family: - gan_based_voice_synthesis claims: - claim_id: hifigan_periodic_structure_quality role: supports claim: Explicitly modeling periodic structure in speech at multiple timescales is necessary for GAN-based vocoders to match the perceptual quality of autoregressive models. source: §2.3, Table 2 evidence: The multi-period discriminator reshapes waveforms into 2D grids indexed by period to model periodicity explicitly; removing it drops MOS from 4.10 to 2.28 in the ablation study. confidence: high relevance: low - claim_id: hifigan_compact_realtime_quality role: supports claim: A compact GAN vocoder can achieve CPU real-time synthesis with quality comparable to autoregressive models when discriminator design, rather than generator depth, is the primary quality bottleneck. source: §4.1, Table 1 evidence: V3 (1.46M parameters) achieves MOS 4.05 at 13.4x faster than real-time on CPU, comparable to WaveNet MoL at a fraction of the compute cost. confidence: high relevance: low - claim_id: hifigan_speaker_generalization_mel_only role: supports claim: Vocoders trained on single-speaker data generalise to unseen speakers when the generator is conditioned only on mel-spectrograms, with quality exceeding flow-based and autoregressive alternatives. source: §4.3, Table 3 evidence: On VCTK's nine held-out speakers, all three HiFi-GAN variants outperform WaveNet MoL, WaveGlow, and MelGAN despite training only on single-speaker mel-spectrogram conditioning, indicating the mel-spectrogram representation itself generalises across speaker identity without explicit speaker conditioning. confidence: medium relevance: high - claim_id: hifigan_finetuning_predicted_spectrograms role: supports claim: Fine-tuning a mel-spectrogram vocoder on predicted (rather than ground-truth) spectrograms substantially improves end-to-end TTS quality, while flow-based vocoders do not benefit from the same adaptation. source: §4.4, Table 4 evidence: In the end-to-end experiment with Tacotron2-predicted mel-spectrograms, fine-tuned HiFi-GAN variants all exceed MOS 4.0, while fine-tuned WaveGlow does not improve over its pre-fine-tuning baseline. confidence: high relevance: low - claim_id: hifigan_discriminator_over_generator role: supports claim: Discriminator architecture choices have a larger impact on GAN vocoder quality than generator architecture choices. source: §4.2, Table 2 evidence: The ablation shows removing MPD causes a catastrophic MOS drop from 4.10 to 2.28, a far larger effect than varying generator capacity across the V1/V2/V3 variants. confidence: high relevance: low limitations: - Evaluation is limited to LJSpeech (single speaker) and VCTK (multi-speaker English); performance on noisy, spontaneous, or cross-lingual speech is not assessed. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2206.04658' published_date: "2022-06-09" entry_date: '2026-07-26' year: 2022 venue: arXiv task: - TTS architecture: - GAN relevance: medium evidence_role: - architecture_variant current_role: active_evidence method_family: - gan_based_voice_synthesis claims: - claim_id: bigvgan_periodic_activation_improves_ood_robustness role: supports claim: Periodic activation functions with learnable frequency parameters improve GAN vocoder robustness to out-of-distribution audio, including unseen recording environments, languages, and non-speech signals. source: §3.2, §4.5, §4.6 evidence: On out-of-distribution real-world Korean speech, BigVGAN reaches SMOS 4.38 versus 4.21 for HiFi-GAN. confidence: high relevance: low - claim_id: bigvgan_antialiasing_reduces_artifacts role: supports claim: Anti-aliasing via low-pass filtering around nonlinear activations reduces high-frequency artifacts in neural vocoders and is complementary to periodic inductive bias. source: §3.3, §4.7, Table 4 evidence: Ablation shows removing the low-pass filter degrades OOD performance, and removing both filter and Snake activation degrades it further still, with differences statistically significant (p<0.01). confidence: high relevance: low - claim_id: bigvgan_scaling_benefits_with_stabilization role: supports claim: GAN vocoders benefit significantly from scaling model capacity when paired with appropriate architectural improvements and training stabilisation techniques, whereas naive scaling without those changes leads to early collapse. source: §3.4, §4.7 evidence: Scaling from 14M to 112M parameters requires gradient clipping at global norm 10^3 and a halved learning rate (1e-4) to avoid early training collapse caused by amplified discriminator gradients. confidence: medium relevance: low - claim_id: bigvgan_smos_more_sensitive_than_mos role: refines claim: Similarity MOS (SMOS), which requires direct comparison with ground truth, is a more sensitive evaluation metric than standard MOS for distinguishing speaker fidelity under diverse and out-of-distribution conditions. source: §4.3 evidence: BigVGAN-base (14M params) already outperforms HiFi-GAN on SMOS (+0.05) even though the MOS gap between these small models is comparatively less diagnostic, and the SMOS gap over HiFi-GAN widens further under noisy and OOD conditions (e.g. +0.13 SMOS on real-world Korean). confidence: high relevance: medium - claim_id: bigvgan_diverse_training_data_necessary role: supports claim: Training on diverse, multi-condition speech data (including noisy recordings) is a necessary complement to architectural improvements for achieving universal vocoding performance. source: §4.7, Table 5 evidence: All models are trained on the full LibriTTS train set including the noisy train-other-500 subset, which the ablations in Table 5 identify as contributing to OOD robustness. confidence: medium relevance: low limitations: - Trained and evaluated on 24 kHz audio only; generalisation to 44.1 kHz or 48 kHz audio common in music and broadcast is not tested. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2209.03143' published_date: "2022-09-07" entry_date: '2026-07-26' year: 2022 venue: arXiv task: - TTS - SCA architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence current_role: foundational method_family: - autoregressive_speech_token_lm claims: - claim_id: combining_self_supervised_semantic_tokens_with_codec_acoustic role: supports claim: Combining self-supervised semantic tokens with codec acoustic tokens in a hierarchical language model resolves the quality-versus-coherence tension that affects single-tokenizer audio language models. source: §III-B, §III-C, Table I evidence: AudioLM treats audio generation as a language modeling problem over a hybrid vocabulary of discrete tokens, and solves the quality-versus-coherence tension by separating the representation into two complementary token streams. **Semantic tokens** are extracted by applying k-means clustering (K=1024) to the intermediate representations of w2v-BERT XL (0.6B parameters), a self-supervised model trained with contrastive and masked language modeling objectives. confidence: high relevance: low - claim_id: semantic_and_acoustic_tokens_in_speech_carry_complementary role: supports claim: 'Semantic and acoustic tokens in speech carry complementary information: semantic tokens primarily encode linguistic content and prosody, while acoustic tokens primarily encode speaker identity and recording conditions.' source: §IV-C, §IV-D, Tables II–III evidence: This shows semantic tokens faithfully carry linguistic content through the generation pipeline. *(§IV-C, Table II)* **Disentanglement of content and speaker:** Speaker classification accuracy on acoustic generations conditioned on the same semantic tokens is 3.2% (versus 100% for SoundStream reconstructions), confirming that speaker identity is encoded in the acoustic tokens and not the semantic tokens. *(§IV-D, Table III)* **Speaker-consistent continuation:** When continuing 3-second prompts from unseen speakers in LibriSpeech test-clean, the speaker classifier identifies the original speaker in 92.6% of generated continuations, demonstrating that prompting with both semantic and acoustic tokens preserves voice identity. *(§IV-F, Table III)* **Human perception test:** In a forced-choice listening test where raters judged whether a 7-second continuation was real or synthesized (first 3 seconds known-real), raters achieved only 51.2% accuracy, which a binomial test (p=0.23) confirms is not statistically distinguishable from chance. *(§IV-G)* **Linguistic knowledge:** On the ZeroResource Challenge 2021 benchmarks, AudioLM achieves 71.5% sWUGGY (all) / 83.7% (in-vocab) and 64.7% sBLIMP, improving over GSLM (68.7% in-vocab, 57.1% sBLIMP) and surpassing non-causal models and even a supervised forced-alignment topline on sBLIMP. *(§IV-E, Table IV)* **Piano continuation:** A model retrained on 40k hours of internal piano data generates continuations preferred over acoustic-token-only baselines in 83.3% of pairs in a subjective test, demonstrating the framework generalises beyond speech. *(§IV-I)* **Anti-spoofing:** A companion classifier trained to detect AudioLM-generated speech achieves 98.6% accuracy, despite human listeners being at chance. *(§IV-H)* confidence: high relevance: high - claim_id: autoregressive_language_modeling_over_discrete_audio_tokens_can role: supports claim: Autoregressive language modeling over discrete audio tokens can produce speech continuations indistinguishable from real speech to human listeners in an unpaired forced-choice test. source: §IV-G evidence: This shows semantic tokens faithfully carry linguistic content through the generation pipeline. *(§IV-C, Table II)* **Disentanglement of content and speaker:** Speaker classification accuracy on acoustic generations conditioned on the same semantic tokens is 3.2% (versus 100% for SoundStream reconstructions), confirming that speaker identity is encoded in the acoustic tokens and not the semantic tokens. *(§IV-D, Table III)* **Speaker-consistent continuation:** When continuing 3-second prompts from unseen speakers in LibriSpeech test-clean, the speaker classifier identifies the original speaker in 92.6% of generated continuations, demonstrating that prompting with both semantic and acoustic tokens preserves voice identity. *(§IV-F, Table III)* **Human perception test:** In a forced-choice listening test where raters judged whether a 7-second continuation was real or synthesized (first 3 seconds known-real), raters achieved only 51.2% accuracy, which a binomial test (p=0.23) confirms is not statistically distinguishable from chance. *(§IV-G)* **Linguistic knowledge:** On the ZeroResource Challenge 2021 benchmarks, AudioLM achieves 71.5% sWUGGY (all) / 83.7% (in-vocab) and 64.7% sBLIMP, improving over GSLM (68.7% in-vocab, 57.1% sBLIMP) and surpassing non-causal models and even a supervised forced-alignment topline on sBLIMP. *(§IV-E, Table IV)* **Piano continuation:** A model retrained on 40k hours of internal piano data generates continuations preferred over acoustic-token-only baselines in 83.3% of pairs in a subjective test, demonstrating the framework generalises beyond speech. *(§IV-I)* **Anti-spoofing:** A companion classifier trained to detect AudioLM-generated speech achieves 98.6% accuracy, despite human listeners being at chance. *(§IV-H)* confidence: high relevance: high - claim_id: the_semantic_to_acoustic_hierarchical_generation_pattern_transfers role: supports claim: 'The semantic-to-acoustic hierarchical generation pattern transfers across audio domains: a model trained on piano music without symbolic notation also benefits from the two-tier tokenization.' source: §IV-I evidence: This shows semantic tokens faithfully carry linguistic content through the generation pipeline. *(§IV-C, Table II)* **Disentanglement of content and speaker:** Speaker classification accuracy on acoustic generations conditioned on the same semantic tokens is 3.2% (versus 100% for SoundStream reconstructions), confirming that speaker identity is encoded in the acoustic tokens and not the semantic tokens. *(§IV-D, Table III)* **Speaker-consistent continuation:** When continuing 3-second prompts from unseen speakers in LibriSpeech test-clean, the speaker classifier identifies the original speaker in 92.6% of generated continuations, demonstrating that prompting with both semantic and acoustic tokens preserves voice identity. *(§IV-F, Table III)* **Human perception test:** In a forced-choice listening test where raters judged whether a 7-second continuation was real or synthesized (first 3 seconds known-real), raters achieved only 51.2% accuracy, which a binomial test (p=0.23) confirms is not statistically distinguishable from chance. *(§IV-G)* **Linguistic knowledge:** On the ZeroResource Challenge 2021 benchmarks, AudioLM achieves 71.5% sWUGGY (all) / 83.7% (in-vocab) and 64.7% sBLIMP, improving over GSLM (68.7% in-vocab, 57.1% sBLIMP) and surpassing non-causal models and even a supervised forced-alignment topline on sBLIMP. *(§IV-E, Table IV)* **Piano continuation:** A model retrained on 40k hours of internal piano data generates continuations preferred over acoustic-token-only baselines in 83.3% of pairs in a subjective test, demonstrating the framework generalises beyond speech. *(§IV-I)* **Anti-spoofing:** A companion classifier trained to detect AudioLM-generated speech achieves 98.6% accuracy, despite human listeners being at chance. *(§IV-H)* confidence: high relevance: high - claim_id: self_supervised_speech_representations_trained_with_masked_language role: supports claim: Self-supervised speech representations trained with masked language modeling objectives encode sufficient lexical and syntactic information to outperform earlier causal spoken language models on zero-resource linguistic benchmarks. source: §IV-E, Table IV evidence: AudioLM treats audio generation as a language modeling problem over a hybrid vocabulary of discrete tokens, and solves the quality-versus-coherence tension by separating the representation into two complementary token streams. **Semantic tokens** are extracted by applying k-means clustering (K=1024) to the intermediate representations of w2v-BERT XL (0.6B parameters), a self-supervised model trained with contrastive and masked language modeling objectives. confidence: high relevance: low limitations: - 'AudioLM is a continuation model only: it generates continuations of an audio prompt but cannot synthesise speech from a specified transcript.' - The paper explicitly frames TTS integration (encoder-decoder with text conditioning) as future work, which means the WER/CER results reflect acoustic fidelity to a given semantic token sequence, not instruction-following capability. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2301.02111' published_date: "2023-01-05" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS - codec architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence current_role: foundational method_family: - autoregressive_speech_token_lm claims: - claim_id: treating_tts_as_conditional_language_modeling_over_discrete role: supports claim: Treating TTS as conditional language modeling over discrete codec tokens enables zero-shot speaker generalisation as in-context learning, without speaker-specific fine-tuning or engineered speaker encoders. source: §4.1, §5.2 evidence: VALL-E treats TTS as a conditional language modeling problem over discrete acoustic tokens. confidence: high relevance: high - claim_id: training_on_large_scale_semi_supervised_speech_data role: supports claim: Training on large-scale semi-supervised speech data, even with noisy transcriptions and diverse acoustic conditions, yields stronger generalisation to unseen speakers than training on smaller clean corpora. source: §1, §5.2 evidence: On LibriSpeech test-clean (zero-shot, 3-second prompt, 40 unseen speakers), VALL-E achieves SMOS 4.38 ± 0.10 versus YourTTS's 3.45 ± 0.09, a +0.93 gap that closes most of the distance to ground truth (4.50 ± 0.10). confidence: high relevance: high - claim_id: the_hierarchical_structure_of_residual_vector_quantization_supports role: supports claim: The hierarchical structure of residual vector quantization supports a two-stage AR+NAR generation pipeline in which first-codebook tokens carry speaker identity and subsequent codebooks refine fine acoustic detail. source: §4.2 evidence: 'Audio is tokenised using EnCodec, a convolutional encoder-decoder with residual vector quantization (RVQ): the encoder produces embeddings at 75 Hz, and each frame is modelled by eight hierarchical quantizers with 1024 entries each, corresponding to a 6K bitrate.' confidence: high relevance: high - claim_id: speaker_similarity_in_zero_shot_codec_tts_improves role: supports claim: Speaker similarity in zero-shot codec TTS improves monotonically with acoustic prompt length, suggesting that speaker identity modelling does not saturate within a few seconds. source: §5.3, Table 6 evidence: Speaker similarity improves monotonically with longer prompts, and the gap between systems widens when YourTTS is evaluated on its truly unseen 11 speakers (Table 6). confidence: high relevance: high - claim_id: stochastic_sampling_in_autoregressive_codec_generation_introduces_output role: supports claim: Stochastic sampling in autoregressive codec generation introduces output diversity — varying speech rate, prosody, and accent realisation — that is both a feature for data augmentation and a complication for deterministic evaluation. source: §4.3, §5.4 evidence: The paper presents evidence that stochastic sampling in autoregressive codec generation introduces output diversity — varying speech rate, prosody, and accent realisation — that is both a feature for data augmentation and a complication for deterministic evaluation. confidence: high relevance: low limitations: - 'Synthesis robustness is a material constraint: the autoregressive first-stage LM exhibits attention alignment failures that cause word deletions, insertions, and repetitions.' - WER on LibriSpeech test-clean is 5.9%, nearly three times the ground-truth rate of 2.2%. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2303.03926' published_date: "2023-03-07" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: large_scale_multilingual_codec_language_models_can_transfer role: supports claim: Large-scale multilingual codec language models can transfer speaker identity, emotion, and acoustic environment across languages from a single source utterance without paired bilingual data. source: §3, §5.3, Table 3 evidence: A multilingual autoregressive codec language model (φ_MAR) is a 12-layer Transformer decoder (attention dimension 1024, FFN dimension 4096) that generates first-layer acoustic tokens from a concatenated prompt of source-language phonemes, target-language phonemes, and source first-layer acoustic tokens. confidence: high relevance: high - claim_id: language_id_conditioning_is_essential_for_native_sounding role: supports claim: 'Language ID conditioning is essential for native-sounding accent in cross-lingual codec TTS: removing it significantly degrades accent scores even while modestly improving speaker similarity.' source: §5.5, Table 6 evidence: 'Ablation of the language ID module confirms its importance: removing it drops accent scores from 4.10 to 2.98 (scale 1-5, higher is more native-sounding) and degrades translation quality by roughly 1.5 BLEU (Table 6).' confidence: high relevance: high - claim_id: in_context_learning_with_acoustic_token_prompts_provides role: supports claim: In-context learning with acoustic token prompts provides stronger cross-lingual voice preservation than speaker embedding approaches across both TTS and speech-to-speech translation tasks. source: §5.3, §5.4, Tables 3, 5 evidence: 'On zero-shot cross-lingual TTS (English generation prompted by Chinese speech), VALL-E X substantially outperforms YourTTS across all metrics: speaker similarity improves from 0.30 to 0.36 ASV-Score, WER drops from 8.53 to 4.07, and NISQA naturalness rises from 3.36 to 3.54 (Table 2).' confidence: high relevance: high - claim_id: the_ar_nar_two_stage_codec_language_model role: supports claim: The AR/NAR two-stage codec language model architecture extends naturally to cross-lingual generation by treating bilingual phoneme sequences as concatenated prompts. source: §3.2, §3.4 evidence: VALL-E X treats cross-lingual speech synthesis as a conditional multilingual language modelling task over discrete acoustic tokens, extending the VALL-E architecture to handle bilingual prompts. confidence: high relevance: medium limitations: - Evaluations cover only English and Chinese, and test sets are small (40 speakers, 1373 samples for TTS; 14 speakers, 350 utterances for S2ST on EMIME). - Generalisation to typologically distant language pairs, more than two languages, or lower-resource settings is untested. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2304.09116' published_date: "2023-04-18" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS - VC - singing architecture: - diffusion - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - diffusion_voice_cloning claims: - claim_id: ns2_latent_diffusion_avoids_ar_errors role: supports claim: Latent diffusion models operating on continuous codec vectors avoid the word-skipping and repetition errors that arise from autoregressive generation over long discrete token sequences. source: §2.3, §5.3, Table 7 evidence: NaturalSpeech 2 achieves 0% error rate on 50 adversarially hard sentences versus 24% for Tacotron, 34% for Transformer-TTS, and known failures for VALL-E. confidence: high relevance: low - claim_id: ns2_speech_prompting_zeroshot_adaptation role: supports claim: Speech prompting via in-context learning during training enables zero-shot speaker adaptation without requiring speaker embeddings or multi-step speaker encoding pipelines. source: §3.3, §5.5 evidence: Removing the diffusion prompt entirely causes convergence failure, confirming the speech prompting mechanism (rather than an explicit speaker embedding) is the load-bearing adaptation signal. confidence: medium relevance: high - claim_id: ns2_prosody_scales_with_prompt_length role: supports claim: Prosody adherence in zero-shot TTS improves monotonically with the length of the reference speech prompt, at least up to 10 seconds. source: §5.5, Table 10 evidence: Longer (10-second) prompts substantially reduce pitch mean difference compared to 3-second prompts on LibriSpeech. confidence: medium relevance: high - claim_id: ns2_nonAR_robust_hard_phonemes role: supports claim: Non-autoregressive TTS architectures maintain near-zero error rates on adversarially difficult phoneme sequences where autoregressive models degrade significantly. source: §5.3, Table 7 evidence: NaturalSpeech 2 achieves 0% error on 50 hard sentences versus substantial failure rates for autoregressive Tacotron and Transformer-TTS baselines. confidence: high relevance: low - claim_id: ns2_crossmodal_timbre_transfer role: supports claim: A system trained jointly on speech and singing data can synthesise singing in novel timbres using only a speech reference prompt, demonstrating cross-modal timbre transfer within a shared latent space. source: §5.6 evidence: The model synthesises singing in speaker timbres taken from speech-only prompts, using continuous pitch and duration predictors rather than a discrete singing-specific token structure. confidence: medium relevance: medium limitations: - The model is described as still underfitting at 300K training steps; inference requires 150 diffusion steps (1000 for singing), which is slow for real-time deployment. Training and evaluation are English-only. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2305.07243' published_date: "2023-05-12" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS architecture: - autoregressive-LM - diffusion - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_diffusion_acoustic claims: - claim_id: conditioning_a_diffusion_decoder_on_the_continuous_latent role: supports claim: Conditioning a diffusion decoder on the continuous latent activations of an autoregressive model rather than its discrete token outputs substantially improves output quality in a cascaded AR-diffusion TTS pipeline. source: §2.2.2, Appendix B.4 evidence: 'The "TorToise Trick" is the most consequential design choice: after training the diffusion decoder on discrete VQVAE tokens to convergence, it is fine-tuned to decode the final hidden-state activations of the autoregressive model rather than the discrete tokens.' confidence: high relevance: low - claim_id: contrastive_re_ranking_of_multiple_autoregressive_candidates_using role: supports claim: Contrastive re-ranking of multiple autoregressive candidates using a text-speech discriminator measurably improves the final output quality of a TTS system without requiring the expensive decoder to process every candidate. source: §2.3, §4 evidence: Third, CLVP (Contrastive Language-Voice Pretrained Transformer) is a CLIP-style dual encoder trained contrastively on text/speech pairs; it scores autoregressive output candidates during re-ranking before the expensive diffusion step, allowing many candidates to be filtered cheaply. confidence: high relevance: medium - claim_id: applying_image_generation_scaling_techniques_large_scale_self role: supports claim: Applying image-generation scaling techniques (large-scale self-supervised data, generalist transformer architectures, multi-stage AR-then-diffusion generation) to speech synthesis yields high-expressiveness multi-speaker TTS even when trained by a single researcher on commodity hardware. source: §7 evidence: Second, a GPT-2-style autoregressive decoder (30 transformer layers, dim 1024, 16 heads) is trained to predict sequences of these speech tokens conditioned on text and a speaker conditioning vector. confidence: high relevance: medium - claim_id: building_a_large_scale_tts_training_corpus_by role: supports claim: Building a large-scale TTS training corpus by scraping and filtering internet audio (audiobooks, podcasts) with automatic transcription is a viable path to tens-of-thousands-of-hours datasets without manual labelling. source: §5, Appendix A evidence: The extended dataset of 49,000 hours was self-assembled from audiobooks and podcasts, transcribed with a fine-tuned wav2vec2-large model that was adapted to predict punctuation. confidence: high relevance: low limitations: - No formal listening test or MOS table is reported. - The primary quality claim rests on informal sample comparisons; the paper's own evaluation suite (CLVP-FID) is not a standard benchmark. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2305.09636' published_date: "2023-05-16" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS - SCA architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: parallel_iterative_masked_decoding_adapted_to_rvq_structure role: supports claim: Parallel, iterative masked decoding adapted to RVQ structure enables acoustic token generation two orders of magnitude faster than autoregressive generation at matched perceptual quality. source: §4.3, Figure 3 evidence: 'The runtime gain is two orders of magnitude: SoundStorm generates 30 seconds of audio in 0.5 seconds on a TPU-v4 versus AudioLM''s acoustic stages which scale prohibitively with sequence length.' confidence: high relevance: low - claim_id: non_autoregressive_rvq_level_by_level_decoding_maintains role: supports claim: Non-autoregressive RVQ-level-by-level decoding maintains better voice and acoustic consistency over long sequences than autoregressive chunk-and-prompt approaches. source: §4.2, Table 1, Figure 2 evidence: On long continuations, SoundStorm maintains higher prompt-to-generation acoustic consistency than AudioLM's slide-and-prompt chunking strategy. confidence: high relevance: medium - claim_id: fine_level_rvq_tokens_are_conditionally_independent_given role: supports claim: Fine-level RVQ tokens are conditionally independent given coarser tokens and can be decoded greedily in a single pass without measurable quality loss. source: §3.3, §4.3 evidence: For subsequent levels, greedy decoding in a single pass suffices, exploiting the conditional independence of fine-level tokens given coarser ones. confidence: high relevance: low - claim_id: confidence_based_iterative_decoding_provides_a_meaningful_quality role: supports claim: Confidence-based iterative decoding provides a meaningful quality gain over greedy decoding at the coarsest RVQ level, but additional iterations at finer levels yield no significant improvement for speech. source: §4.3, Figure 4 evidence: An ablation on decoding iterations (Figure 4) shows that 16 iterations in the first RVQ level provides approximately 0.1-0.2 quality gain over greedy decoding; additional iterations on finer levels yield no statistically significant improvement, validating the greedy-at-fine-levels strategy. confidence: high relevance: low - claim_id: coupling_a_text_to_semantic_token_model_with role: supports claim: Coupling a text-to-semantic token model with an efficient acoustic generator enables real-time synthesis of controllable multi-speaker dialogue at 30-second horizons. source: §5 evidence: SoundStorm is a 350M-parameter Conformer trained on the acoustic generation task, taking as conditioning the time-aligned semantic tokens from AudioLM (originating from w2v-BERT quantised with k-means) and producing SoundStream codec tokens (50 frames/second, 12 RVQ levels, codebook size 1024 per level). confidence: high relevance: medium limitations: - Evaluation uses a DNSMOS-style estimator rather than human listening tests for audio quality comparisons, and the subjective baseline is carried over from earlier AudioLM papers rather than re-run. - Direct perceptual comparisons between SoundStorm and AudioLM on the same conditions by human raters are not reported. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2306.12925' published_date: "2023-06-22" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS - SCA architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: initializing_a_speech_text_llm_from_a_pretrained role: supports claim: Initializing a speech-text LLM from a pretrained text-only checkpoint substantially outperforms training from scratch at equivalent model scale. source: §5.4.2, Table 6 evidence: At the same 8B model scale, fine-tuning from a pretrained PaLM-2 checkpoint substantially outperforms training the speech-text model from scratch on the reported translation evaluation. confidence: high relevance: low - claim_id: audio_tokenizer_quality_is_a_primary_bottleneck_in role: supports claim: 'Audio tokenizer quality is a primary bottleneck in LLM-based speech generation and understanding: stronger semantic tokenizers yield large downstream gains independent of LM scale.' source: §5.4.3, Table 7 evidence: 'Ablation over tokenizers shows a large gap: USM-v2 tokens reach 30.5 BLEU on CoVoST2 AST while w2v-BERT tokens yield only 15.2 BLEU at the same model scale, establishing audio tokenizer quality as the primary performance bottleneck (§5.4.3, Table 7).' confidence: high relevance: low - claim_id: a_unified_multimodal_vocabulary_that_interleaves_text_and role: supports claim: A unified multimodal vocabulary that interleaves text and audio tokens enables zero-shot speech translation to language pairs not seen during speech training, by inheriting translation capability from text pretraining. source: §5.2, Table 3 evidence: Zero-shot AST on FLEURS shows AudioPaLM-2 achieving 20.7 BLEU on language pairs where only ASR data was seen during training, outperforming Whisper (19.6) despite Whisper having seen speech-to-text translation data for all those languages. confidence: high relevance: high - claim_id: training_on_combined_tasks_that_decompose_complex_speech role: supports claim: Training on combined tasks that decompose complex speech operations into intermediate text steps improves performance over direct end-to-end decoding. source: §5.4.4, Table 8 evidence: For complex tasks, the model can also be trained on combined tasks, emitting intermediate outputs (transcription, then translation, then translated audio) in a single autoregressive pass, which mirrors chain-of-thought prompting and improves performance relative to direct decoding. confidence: high relevance: medium - claim_id: voice_identity_preservation_in_cross_lingual_speech_synthesis role: supports claim: Voice identity preservation in cross-lingual speech synthesis can exceed high-quality TTS-based references when an audio LM is conditioned on a short spoken prompt. source: §5.3, Table 4 evidence: Voice conditioning is provided as a 3-second reference clip encoded as both audio tokens and SoundStream tokens, enabling cross-lingual voice transfer. confidence: high relevance: medium limitations: - The entire system depends on the quality of the audio tokenizer, which is not released and requires access to Google-internal USM models. - The best-performing configuration (AudioPaLM-2 with USM-v2 tokens) is not reproducible externally; the published ablations use the multilingual w2v-BERT tokenizer as the weakest condition, suggesting that reported performance at USM-v2 quality cannot be independently verified. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2308.16692' published_date: "2023-08-31" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS - codec architecture: - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - gan_based_voice_synthesis claims: - claim_id: separating_semantic_content_from_paralinguistic_information_across_rvq role: supports claim: Separating semantic content from paralinguistic information across RVQ layers within a single codec improves both reconstruction quality and speech language model coherence compared to undifferentiated acoustic tokenisation. source: §4.4, Tables 2 and 4 evidence: 'The paper then demonstrates utility by building a Unified Speech Language Model (USLM) on top of SpeechTokenizer: a 12-layer transformer autoregressive model produces first-layer (semantic) tokens from phoneme input, while a 12-layer non-autoregressive model generates layers 2-8 (paralinguistic) conditioned on the first-layer tokens and an acoustic speaker prompt.' confidence: high relevance: medium - claim_id: acoustic_tokens_from_standard_neural_codecs_encode_content role: supports claim: Acoustic tokens from standard neural codecs encode content and speaker identity in an entangled form that causes systematic word errors in autoregressive language model generation. source: §2.3, Table 3 evidence: 'The paper then demonstrates utility by building a Unified Speech Language Model (USLM) on top of SpeechTokenizer: a 12-layer transformer autoregressive model produces first-layer (semantic) tokens from phoneme input, while a 12-layer non-autoregressive model generates layers 2-8 (paralinguistic) conditioned on the first-layer tokens and an acoustic speaker prompt.' confidence: high relevance: high - claim_id: a_distillation_objective_computed_per_feature_dimension_d role: supports claim: A distillation objective computed per feature dimension (D-axis) produces stronger semantic guidance to a codec's first quantizer than the conventional per-timestep (T-axis) formulation. source: Appendix C, Table 7 evidence: 'During training, a HuBERT model acts as a frozen semantic teacher guiding the first RVQ quantizer via two distillation objectives: a continuous cosine-similarity loss computed dimension-wise (not timestep-wise) between the first-layer quantized output and HuBERT layer-9 or average-layer representations, and a pseudo-label prediction loss over HuBERT unit targets.' confidence: medium relevance: low - claim_id: the_first_layer_tokens_of_a_hierarchically_disentangled role: supports claim: The first-layer tokens of a hierarchically disentangled codec can serve as a zero-shot voice conversion mechanism by swapping higher-layer tokens from a reference speaker, without requiring a separate conversion model. source: §5.2, Table 5 evidence: 'The paper then demonstrates utility by building a Unified Speech Language Model (USLM) on top of SpeechTokenizer: a 12-layer transformer autoregressive model produces first-layer (semantic) tokens from phoneme input, while a 12-layer non-autoregressive model generates layers 2-8 (paralinguistic) conditioned on the first-layer tokens and an acoustic speaker prompt.' confidence: high relevance: high - claim_id: codec_tokens_trained_without_explicit_content_supervision_exhibit role: supports claim: Codec tokens trained without explicit content supervision exhibit poor codebook utilisation and weak phoneme-code correspondence, increasing the modelling burden on downstream language models. source: Appendix F, Table 8 evidence: On SLMTokBench (Table 3), SpeechTokenizer RVQ-1 tokens achieve mutual information of 31.6-32.9 (depending on teacher) versus 16.5 for EnCodec RVQ-1, confirming the content-focused design. confidence: medium relevance: low limitations: - SpeechTokenizer is trained solely on English LibriSpeech. - While preliminary results in Appendix G suggest cross-lingual token transfer is plausible, the codec is not validated for multilingual speech language models and the text-alignment properties of RVQ-1 may not hold for typologically distant languages. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2310.00704' published_date: "2023-10-01" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS - VC - singing architecture: - autoregressive-LM - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: training_a_single_audio_language_model_across_diverse role: supports claim: Training a single audio language model across diverse generation tasks (TTS, voice conversion, sound synthesis, music, singing) produces consistent performance improvements over task-specific models trained on the same data. source: §3.4.1, Appendix C.1, Table 17 evidence: Multi-task training consistently outperforms single-task training across all 11 tasks (Appendix C.1, Table 17), confirming that cross-task sharing is beneficial rather than neutral. confidence: high relevance: medium - claim_id: the_autoregressive_property_is_critical_for_audio_generation role: supports claim: 'The autoregressive property is critical for audio generation quality: parallel and delay-based codec prediction approaches yield measurably lower naturalness than fully autoregressive methods when codec quantization levels are held constant.' source: §3.4.2, Tables 4–5 evidence: With codec quantization levels held constant, fully autoregressive prediction receives higher naturalness ratings than the parallel and delayed prediction variants in the architecture ablation. confidence: high relevance: low - claim_id: hierarchical_factorisation_of_rvq_codec_token_sequences_into role: supports claim: Hierarchical factorisation of RVQ codec token sequences into inter-frame and intra-frame modeling substantially reduces training memory and time relative to flat-sequence autoregressive prediction, with comparable generation quality. source: §2.3, §3.4.2, Table 4 evidence: The multi-scale Transformer matches flat-sequence autoregressive prediction in TTS quality while reducing the memory and training-time burden of modeling RVQ sequences. confidence: high relevance: low - claim_id: pre_training_on_a_broad_multi_task_audio role: supports claim: Pre-training on a broad multi-task audio corpus enables strong adaptation to unseen audio generation tasks via fine-tuning on small datasets, outperforming task-specific models trained from scratch on those tasks. source: §3.3, Appendix B.5–B.8, Table 17 evidence: 'Training proceeds in two stages: joint pretraining on 7 tasks (165K hours), followed by fine-tuning to add 4 new tasks (audio editing, speech dereverberation, instructed TTS, speech editing).' confidence: high relevance: low - claim_id: signal_level_metrics_such_as_pesq_are_poorly role: supports claim: 'Signal-level metrics such as PESQ are poorly suited for evaluating generative audio models: systems achieving higher perceptual MOS scores routinely score lower on PESQ than discriminative baselines.' source: §3.2, §3.4.2, Table 11 evidence: On speech enhancement, UniAudio obtains higher DNSMOS and human MOS than the compared systems while its PESQ remains below discriminative baselines. confidence: high relevance: low limitations: - Model checkpoints are not released due to misuse concerns, limiting reproducibility. - Only code and demos are public. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2312.01479' published_date: "2023-12-03" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS architecture: - GAN - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: openvoice_decoupled_tone_color_control role: supports claim: Decoupling tone color transfer from style and language generation enables flexible post-cloning control over emotion, accent, and rhythm without requiring paired multi-attribute training data. source: §2.1, §2.2 evidence: The tone color converter uses a KL-divergence tone color elimination objective in a normalising flow to strip speaker identity while preserving phonetic/style content extracted from the base speaker model, allowing independent post-hoc control of each attribute. confidence: medium relevance: high - claim_id: openvoice_ipa_crosslingual_generalization role: supports claim: Using IPA as a universal phoneme representation in a normalising flow allows voice cloning to generalise to unseen languages without retraining on massive multilingual speaker data. source: §2.3, §3 "Importance of IPA" evidence: The IPA-aligned flow representation enables cross-lingual cloning for languages absent from the MSML training set, demonstrated qualitatively across multiple target languages. confidence: medium relevance: medium - claim_id: openvoice_feedforward_realtime role: supports claim: Feed-forward voice cloning pipelines (encoder-flow-decoder without autoregressive or diffusion components) achieve real-time or faster inference, making large-scale commercial deployment feasible. source: §3 "Fast Inference with Low Cost" evidence: The full pipeline achieves 12x real-time inference on a single A10G GPU with an estimated upper bound of 40x. confidence: medium relevance: medium - claim_id: openvoice_crossstudy_comparison_confounded role: complicates claim: Cross-study quantitative comparison of voice cloning systems is systematically confounded by differences in test set composition, training data scale, and evaluation protocol. source: §3 evidence: The authors explicitly decline to report numerical MOS scores, arguing that cross-study comparisons with VALL-E, XTTS, YourTTS, or Voicebox using different raters and test sets would be unfair; all evaluation in the paper is qualitative. confidence: medium relevance: medium limitations: - The paper reports no quantitative metrics anywhere; all evaluation is qualitative via a demo website. Style preservation through the tone color converter is imperfect for highly expressive inputs, with emotion occasionally being neutralised. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2312.15821' published_date: "2023-12-25" entry_date: '2026-07-26' year: 2023 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - nonautoregressive_flow_matching claims: - claim_id: audiobox_unified_model_matches_specialists role: supports claim: A single generative model trained across speech, sound, and music modalities can match or surpass modality-specific models on dedicated benchmarks. source: §5.4, §6.4, §7.5, Tables 1, 5, 13, 14 evidence: Audiobox Speech surpasses Voicebox and UniAudio on LibriSpeech zero-shot TTS speaker similarity while the unified Audiobox model outperforms all unified text-to-audio baselines on AudioCaps FAD. confidence: high relevance: high - claim_id: audiobox_ssl_pretraining_improves_generalization role: supports claim: Self-supervised pre-training on large-scale unlabeled audio substantially improves multi-domain style generalisation in subsequent supervised fine-tuning, with gains most pronounced on out-of-domain test sets. source: §5.5, Table 3 evidence: Audiobox Speech improves over Voicebox by 0.096-0.156 absolute speaker similarity on CommonVoice, Switchboard, Expresso, and internal Accent datasets after SSL pre-training on 185K hours. confidence: high relevance: high - claim_id: audiobox_general_clap_fails_speech_attributes role: supports claim: General audio-language embedding models trained primarily on sound events fail to capture fine-grained speech attributes, rendering them unreliable for evaluating description-conditioned speech generation. source: §7.3.1, Table 8 evidence: Stock CLAP achieves text-to-audio retrieval recall@10 of only 2.29 on speech description data and Pearson correlation of 0.028 with human relevance scores, versus 22.01 and 0.727 respectively for the paper's proposed Joint-CLAP. confidence: high relevance: low - claim_id: audiobox_bespoke_solvers_speedup_flow_matching role: supports claim: Flow-matching models admit post-training inference optimisation via learned ODE reparameterisation (Bespoke Solvers) that reduces function evaluations by 25x without measurable quality degradation. source: §8, Table 15 evidence: Bespoke Solvers at 4 steps (8 NFE) match the midpoint solver at 32 steps (64 NFE) exactly on speaker similarity and WER across five test sets. confidence: high relevance: high - claim_id: audiobox_silence_removal_improves_speaker_similarity role: supports claim: Removing trailing silence from audio context prompts substantially improves speaker similarity in zero-shot TTS, particularly on datasets with long prompt silences. source: §5.5, Table 3 evidence: Table 3's ablation attributes part of the speaker-similarity gain over Voicebox to removing trailing silence from voice prompts. confidence: medium relevance: high limitations: - The model and training data are not released publicly, limiting reproducibility, and the description-based conditioning pipeline depends on LLM-generated captions from a limited categorical attribute vocabulary. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2401.07333' published_date: "2024-01-14" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: interleaving_phoneme_tokens_with_their_corresponding_acoustic_frames role: supports claim: Interleaving phoneme tokens with their corresponding acoustic frames in the training sequence substantially reduces phoneme-level alignment errors in autoregressive codec LM TTS, including repetitions, transpositions, and omissions. source: §3.2, §4.2, Table 3 evidence: ELLA-V reorders the hybrid token sequence so that phonemes are interleaved with their corresponding acoustic tokens rather than prepended en bloc. confidence: high relevance: low - claim_id: autoregressive_codec_language_models_that_concatenate_all_phoneme role: supports claim: Autoregressive codec language models that concatenate all phoneme tokens ahead of all acoustic tokens are prone to infinite-silence generation, with failure rates exceeding 80% under greedy decoding. source: §1, Table 1 evidence: 'The stability advantage is especially pronounced under near-greedy decoding: VALL-E''s infinite-silence rate (INF%) reaches 87% at greedy decoding, while ELLA-V''s forced-truncation rate (CUT%) remains low across the entire top-p range.' confidence: high relevance: low - claim_id: explicit_forced_alignment_supervision_at_training_time_enables role: supports claim: Explicit forced-alignment supervision at training time enables fine-grained phoneme-level control at inference, allowing deterministic truncation of abnormal synthesis and making greedy decoding viable. source: §3.2.1, §3.3, Figure 5 evidence: 'The stability advantage is especially pronounced under near-greedy decoding: VALL-E''s infinite-silence rate (INF%) reaches 87% at greedy decoding, while ELLA-V''s forced-truncation rate (CUT%) remains low across the entire top-p range.' confidence: high relevance: low - claim_id: structural_alignment_constraints_in_the_token_sequence_provide role: supports claim: Structural alignment constraints in the token sequence provide larger accuracy gains than naturalness or speaker similarity improvements, suggesting that intelligibility and speaker identity are relatively easy to preserve while alignment robustness remains the primary challenge. source: §4.2, Table 2 evidence: The alignment-guided sequence reordering produces its largest improvement in recognition accuracy, while speaker-similarity and naturalness changes are comparatively smaller. confidence: high relevance: high limitations: - Results are compared only against a VALL-E baseline reproduced on LibriSpeech 960h, not against the original VALL-E trained on 60k hours of LibriLight. - The restricted training data limits direct comparison to the published VALL-E numbers and leaves open whether the gains hold at scale. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2402.01912' published_date: "2024-02-02" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: automatic_acoustic_labeling_can_substitute_for_human_annotations role: supports claim: Automatic acoustic labeling can substitute for human annotations in training large-scale instruction-conditioned speech language models without a loss in attribute control accuracy relative to human-labeled systems. source: §3.1, §3.2, §4.1 evidence: Objective attribute control is validated using the same automatic classifiers that labeled the training data. confidence: high relevance: low - claim_id: including_a_small_proportion_of_high_fidelity_audio role: supports claim: Including a small proportion of high-fidelity audio (approximately 1%) in a predominantly noisy training corpus, combined with explicit recording-quality labels, enables a speech LM to generate professional-sounding speech on demand from text prompts alone. source: §3.1.2, §4.2, Table 1 evidence: 'High audio fidelity is achieved through a combination of: (1) using DAC rather than EnCodec, which the authors report provides subjectively and objectively superior audio quality, and (2) including LibriTTS-R in training, which provides approximately 500 hours (roughly 1%) of clean, professionally enhanced speech alongside the crowd-sourced data.' confidence: high relevance: medium - claim_id: the_choice_of_neural_audio_codec_has_a role: supports claim: The choice of neural audio codec has a measurable effect on perceptual audio quality in autoregressive TTS; higher-fidelity codecs translate directly to higher MOS and objective quality scores. source: §3.3, §4.2, Table 1–2 evidence: On samples prompted for excellent recording quality, the higher-fidelity codec system outperforms Audiobox on PESQ, STOI, and SI-SDR and approaches the ground-truth values. confidence: high relevance: medium - claim_id: natural_language_conditioning_on_accent_can_be_achieved role: supports claim: Natural language conditioning on accent can be achieved in a single TTS model covering dozens of accents, though classifier accuracy reflects the noise and imbalance inherent in automatic accent labeling of crowd-sourced data. source: §3.1.1, §4.1 evidence: A large language model (Stable Beluga 2) then converts keyword tuples into natural language sentences, e.g. "a woman with a deep voice speaking slowly and somewhat monotonously with a Hungarian accent in an echoey room with background noise." This approach is applied to the 45k-hour English Multilingual LibriSpeech corpus and the smaller 585-hour LibriTTS-R corpus (included for its higher audio fidelity from the Miipher speech-enhancement model). confidence: high relevance: medium limitations: - The evaluation compares only against Audiobox. - No standard TTS baselines (reference-based zero-shot systems, encoder-decoder models) are included, making it impossible to assess whether the MOS gains arise from the conditioning approach, the codec choice, or the training data mix. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2402.08093' published_date: "2024-02-12" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: scaling_autoregressive_codec_tts_to_500m_parameters_and role: supports claim: Scaling autoregressive codec TTS to 500M+ parameters and 10K+ hours of training data produces qualitatively different prosody rendering on linguistically complex inputs compared to smaller models trained on less data. source: §4.3, Figure 4, Table 5 evidence: The WavLM-based speechcodes match VQ-VAE on English (MUSHRA 74.7 vs 74.8, non-significant) but outperform it on Spanish despite Spanish comprising only 2% of training data (MUSHRA 74.7 vs 73.3, significant for some speakers), supporting the hypothesis that better representations matter more in lower-resource languages (Table 3). confidence: high relevance: medium - claim_id: ssl_based_speech_representations_with_explicit_speaker_disentanglement role: supports claim: SSL-based speech representations with explicit speaker disentanglement outperform purely acoustic codec representations for zero-shot TTS, particularly in lower-resource languages. source: §4.1, Table 3 evidence: The WavLM-based speechcodes match VQ-VAE on English (MUSHRA 74.7 vs 74.8, non-significant) but outperform it on Spanish despite Spanish comprising only 2% of training data (MUSHRA 74.7 vs 73.3, significant for some speakers), supporting the hypothesis that better representations matter more in lower-resource languages (Table 3). confidence: high relevance: high - claim_id: a_streamable_convolutional_decoder_can_match_or_exceed role: supports claim: A streamable convolutional decoder can match or exceed a diffusion-based spectrogram decoder in subjective naturalness while reducing synthesis compute by approximately 3x and enabling low-latency streaming. source: §4.2, §4.5, Table 4 evidence: The speechcode decoder matches or improves over the diffusion decoder on 4 of 6 evaluated speakers, while offering a 3x compute reduction and enabling streaming (Table 4). confidence: high relevance: medium - claim_id: applying_bpe_to_discrete_speech_tokens_reduces_autoregressive role: supports claim: Applying BPE to discrete speech tokens reduces autoregressive sequence length by approximately 40% without degrading downstream synthesis quality, enabling longer-context training. source: §2.2.3 evidence: A speech tokenizer converts waveforms into a compact discrete sequence; an autoregressive Transformer ("SpeechGPT") predicts those tokens from text and a reference speech clip; and a convolutional decoder renders the predicted tokens to audio. confidence: high relevance: low - claim_id: autoregressive_tts_trained_at_scale_generalises_to_a role: supports claim: Autoregressive TTS trained at scale generalises to a wide range of textual phenomena without any explicit prosody annotation or task-specific supervision. source: §4.3, §6 evidence: Baseline comparisons (Tortoise, Bark, YourTTS) use publicly available checkpoints not trained on comparable data volumes; the gains partly reflect data scale rather than architecture alone. confidence: high relevance: medium limitations: - Model weights are not released, and evaluation uses proprietary test speakers. - The MUSHRA baselines (YourTTS, Bark, TortoiseTTS) are not trained on comparable data or compute, making architecture-level conclusions difficult to separate from scale effects. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2402.13236' published_date: "2024-02-20" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS - SCA - codec architecture: [] relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: [] claims: - claim_id: residual_vector_quantisation_is_the_dominant_quantisation_strategy role: supports claim: Residual vector quantisation is the dominant quantisation strategy across neural audio codec models, with variation concentrated in discriminator design, bitrate, and semantic token integration rather than in the core compression mechanism. source: §II.A, Table II evidence: 'Key observations surfaced through comparison include: most codec-based LMs depend on Encodec or SoundStream as their tokeniser backbone; task coverage is expanding beyond TTS and music generation toward speech editing, enhancement, translation, and instruction-conditioned generation; and the integration of semantic tokens into codecs is an active design axis with differing approaches (distillation from HuBERT vs. additive fusion).' confidence: high relevance: low - claim_id: codec_based_audio_language_models_increasingly_target_multi role: supports claim: Codec-based audio language models increasingly target multi-task coverage rather than single-task specialisation, with several systems spanning TTS, voice conversion, speech editing, speech enhancement, and translation in a single framework. source: §III.B, Table III evidence: 'Key observations surfaced through comparison include: most codec-based LMs depend on Encodec or SoundStream as their tokeniser backbone; task coverage is expanding beyond TTS and music generation toward speech editing, enhancement, translation, and instruction-conditioned generation; and the integration of semantic tokens into codecs is an active design axis with differing approaches (distillation from HuBERT vs. additive fusion).' confidence: high relevance: medium - claim_id: integrating_semantic_tokens_from_self_supervised_speech_representations role: supports claim: Integrating semantic tokens from self-supervised speech representations into the codec quantisation process improves audio quality at low bitrates, with HuBERT-guided RVQ being the most common approach. source: §II.B evidence: 'Key observations surfaced through comparison include: most codec-based LMs depend on Encodec or SoundStream as their tokeniser backbone; task coverage is expanding beyond TTS and music generation toward speech editing, enhancement, translation, and instruction-conditioned generation; and the integration of semantic tokens into codecs is an active design axis with differing approaches (distillation from HuBERT vs. additive fusion).' confidence: high relevance: low - claim_id: discrete_units_derived_from_self_supervised_representations_enable role: supports claim: Discrete units derived from self-supervised representations enable textless speech language modelling but sacrifice speaker and paralinguistic information relative to codec-based approaches. source: §III.A evidence: 'Key observations surfaced through comparison include: most codec-based LMs depend on Encodec or SoundStream as their tokeniser backbone; task coverage is expanding beyond TTS and music generation toward speech editing, enhancement, translation, and instruction-conditioned generation; and the integration of semantic tokens into codecs is an active design axis with differing approaches (distillation from HuBERT vs. additive fusion).' confidence: high relevance: medium limitations: - The survey covers only open-source codec models and does not include proprietary codecs used in industry systems. - 'Evaluation methodology is not addressed: the paper does not compare codecs on shared benchmarks or report reproduction numbers, making it difficult to assess quality claims from the original papers in a unified way.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2403.03100' published_date: "2024-03-05" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS - VC architecture: - diffusion - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - diffusion_voice_cloning claims: - claim_id: ns3_explicit_disentanglement_reduces_complexity role: supports claim: Explicit disentanglement of speech attributes in the codec representation reduces the complexity of zero-shot generation and improves speaker similarity, quality, and prosody simultaneously. source: §3, §4.2, Table 1, Table 2 evidence: 'Ablation confirms factorisation is the primary driver of quality: removing it degrades Sim-O from 0.67 to 0.55, CMOS by 0.25, and SMOS by 0.42.' confidence: high relevance: high - claim_id: ns3_grl_supervised_losses_suppress_leakage role: supports claim: Gradient reversal combined with attribute-specific supervised losses is an effective mechanism for suppressing cross-attribute information leakage in neural codec quantization. source: §3.2.2, Appendix B.4 evidence: FACodec applies gradient reversal layers alongside F0/phoneme/speaker supervised auxiliary losses on the prosody, content, and timbre subspaces respectively to actively suppress cross-attribute leakage during codec training. confidence: high relevance: medium - claim_id: ns3_factorization_architecture_agnostic role: supports claim: The factorization paradigm for codec representations is architecture-agnostic and improves both autoregressive and non-autoregressive generators when applied. source: §4.3.2, Table 6 evidence: Substituting VALL-E for the diffusion modules while retaining FACodec still improves results over standard VALL-E, demonstrating the factorized representation benefits transfer across generator architectures. confidence: high relevance: low - claim_id: ns3_discrete_diffusion_faster_than_ar role: supports claim: Discrete masked diffusion over disentangled codec tokens is faster than autoregressive LM-based codec generation at comparable or better quality. source: Appendix A.5, Table 10 evidence: NaturalSpeech 3 achieves RTF of 0.296 compared to 4.52 for VALL-E on a single V100. confidence: high relevance: low - claim_id: ns3_scales_with_data_and_size role: supports claim: Performance on zero-shot TTS scales predictably with both training data volume and model size when the underlying speech representation captures disentangled attributes. source: §4.4, Tables 7, 8 evidence: Scaling to 1B parameters and 200K hours of training data improves Sim-O from 0.73 to 0.78 and WER from 2.11% to 1.71% on an internal test set. confidence: medium relevance: high limitations: - FACodec requires phoneme-level transcriptions for content supervision, constraining applicability to languages/settings without reliable alignments. The zero-shot TTS evaluation is English-only. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2403.16973' published_date: "2024-03-25" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: autoregressive_codec_language_models_can_perform_speech_infilling role: supports claim: Autoregressive codec language models can perform speech infilling with naturalness approaching that of the original unedited recording when masked spans are relocated to the end of the sequence, enabling bidirectional context conditioning within a causal framework. source: §3.1, §5.3, Table 5 evidence: 'The core mechanism is a two-step token rearrangement procedure applied before autoregressive modeling: In the first step (causal masking), the spans to be generated are relocated from their original position in the codec matrix to the end of the sequence.' confidence: high relevance: low - claim_id: zero_shot_tts_and_speech_editing_can_be role: supports claim: Zero-shot TTS and speech editing can be unified as a single autoregressive infilling operation without task-specific architectural components, at no cost to performance on either task. source: §3.4, §5.4, Table 6 evidence: VoiceCraft reframes both speech editing (infilling) and zero-shot TTS (continuation) as left-to-right language modeling over rearranged neural codec token sequences. confidence: high relevance: high - claim_id: wer_measured_by_asr_systems_is_an_unreliable role: supports claim: 'WER measured by ASR systems is an unreliable proxy for perceptual intelligibility when evaluating speech synthesis quality: systems can achieve lower WER than ground truth recordings while receiving substantially lower intelligibility ratings from human listeners.' source: §5.3, §5.4 evidence: 'A recurrent observation across both tasks: FluentSpeech achieves lower WER than the ground truth (3.5% vs 3.8% for TTS; 4.5% vs 5.4% for editing), yet receives substantially lower intelligibility MOS from human listeners.' confidence: high relevance: low - claim_id: evaluation_of_speech_synthesis_exclusively_on_audiobook_data role: supports claim: Evaluation of speech synthesis exclusively on audiobook data underestimates the performance gap between systems when applied to in-the-wild recordings with diverse accents, noise, and speaking styles. source: §5.3, §5.4 evidence: This suggests that ASR-based WER can be a misleading proxy for perceptual intelligibility when systems produce dull or robotic speech that ASR models handle well. confidence: high relevance: medium - claim_id: scaling_autoregressive_codec_lm_parameters_consistently_improves_objective role: supports claim: Scaling autoregressive codec LM parameters consistently improves objective metrics across intelligibility and acoustic fidelity measures, with larger gaps between larger model sizes suggesting further gains from continued scaling. source: §5.2, Table 3 evidence: Ablation over model sizes (120M, 430M, 830M) shows consistent gains from scaling across WER, MCD, F0, and energy distance metrics. confidence: high relevance: low limitations: - The inference-time artifact mitigation (generating 10 candidates and discarding the 4 longest) adds significant latency and compute cost. - The strategy is acknowledged as inelegant, and the underlying cause (repetitive loop generation) is unresolved. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2404.03204' published_date: "2024-04-04" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: autoregressive_codec_language_model_tts_can_be_substantially role: supports claim: Autoregressive codec language model TTS can be substantially stabilised by predicting prosody tokens as explicit intermediate targets before speech token generation, without requiring reranking or a separate alignment model at inference. source: §3.2, Table 2 evidence: 'Rather than having the AR Transformer jump directly from phoneme input to speech token generation, RALL-E inserts an explicit intermediate prediction stage: the model first generates a phoneme-level prosody token sequence encoding discretised pitch (quantised to 256 buckets) and duration (capped at 32 frames) for the full input utterance, then uses those prosody tokens as an additional conditioning signal when generating the first-layer RVQ speech tokens autoregressively.' confidence: high relevance: low - claim_id: duration_guided_attention_masking_which_restricts_each_speech role: supports claim: Duration-guided attention masking, which restricts each speech token to attend only to a local phoneme window based on predicted alignment, provides significant robustness improvements beyond prosody conditioning alone. source: §3.3, Table 4 evidence: RALL-E uses the predicted duration to restrict each speech token's attention to a local phoneme window of width 2k+1 centered on the corresponding phoneme (k=1 in the final system), masking all other phoneme and prosody positions. confidence: high relevance: low - claim_id: the_robustness_deficit_of_ar_codec_tts_relative role: supports claim: The robustness deficit of AR codec TTS relative to non-autoregressive methods is most pronounced on structurally unusual inputs (repetitive patterns, numeric sequences, code strings) where learned implicit alignment is most likely to fail. source: §4.4, Table 1 evidence: On 50 hard sentences containing repetitive numbers, file paths, and code strings, RALL-E reduces the error rate from 68% to 4%, closely approaching non-autoregressive NaturalSpeech 2 (0%) (Table 1, §4.4). confidence: high relevance: low - claim_id: reranking_over_multiple_samples_and_explicit_intermediate_prosody role: supports claim: Reranking over multiple samples and explicit intermediate prosody prediction address the same underlying alignment problem and can be combined for additive gain, but prosody CoT prompting reduces the dependency on reranking by improving single-sample quality. source: §4.2, Table 2 evidence: 'Rather than having the AR Transformer jump directly from phoneme input to speech token generation, RALL-E inserts an explicit intermediate prediction stage: the model first generates a phoneme-level prosody token sequence encoding discretised pitch (quantised to 256 buckets) and duration (capped at 32 frames) for the full input utterance, then uses those prosody tokens as an additional conditioning signal when generating the first-layer RVQ speech tokens autoregressively.' confidence: high relevance: medium limitations: - The paper relies on an internal proprietary alignment tool for extracting phoneme-speech alignments during training. - Alignment quality directly affects duration-guided masking effectiveness (§3.3 notes that alignment errors required loosening the masking window from k=0 to k=1). caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2406.00654' published_date: "2024-06-02" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: standard_supervised_training_objectives_for_tts_produce_a role: supports claim: Standard supervised training objectives for TTS produce a systematic mismatch with human perceptual evaluation metrics such as MOS and WER, and correcting this mismatch through preference-aware fine-tuning yields large performance gains. source: §1, §4.2, Table 1 evidence: The framework is applied as a fine-tuning pass on a pre-trained VoiceCraft model (330M or 830M parameters), which uses SpeechTokenizer (EnCodec-based, 4 RVQ codebooks, vocabulary of 2048) for discrete acoustic tokenisation. confidence: high relevance: medium - claim_id: existing_rlhf_methods_requiring_pairwise_preference_data_from role: supports claim: Existing RLHF methods requiring pairwise preference data from the same input (DPO) are difficult to apply directly to autoregressive codec TTS because these models lack sufficient output diversity to form meaningful preference pairs from a fixed transcript-prompt combination. source: §4.2, Appendix B evidence: In the sampling stage, a fixed target transcript is paired with a batch of distinct speech prompts drawn from an unseen speaker pool, yielding a diverse set of synthesised outputs; varying the prompt (rather than stochastic decoding) provides much richer quality variation across samples. confidence: high relevance: high - claim_id: uncertainty_in_human_speech_quality_annotations_is_not role: supports claim: Uncertainty in human speech quality annotations is not noise to be discarded but an informative signal that, when incorporated into the optimization objective, improves the consistency of generated speech across listeners. source: §4.2, §6.3, Table 3 evidence: An ablation without uncertainty weighting (UNO-null) achieves similar mean MOS but higher output variance, confirming that uncertainty modelling specifically improves generation consistency rather than average quality. confidence: high relevance: low - claim_id: rlhf_style_alignment_for_tts_can_be_achieved role: supports claim: RLHF-style alignment for TTS can be achieved with a small number of self-generated samples (hundreds) without access to ground truth speech, making it practical for post-training fine-tuning at low computational cost. source: §4.1, §5, Appendix D evidence: Only 400 self-generated samples (balanced 200 positive / 200 negative) are needed, requiring approximately 10 minutes of fine-tuning on a single A100 GPU. confidence: high relevance: low - claim_id: alignment_objectives_designed_for_naturalness_mos_transfer_to role: supports claim: Alignment objectives designed for naturalness MOS transfer to other perceptual dimensions such as emotion by substituting the selection criterion, demonstrating that preference-based fine-tuning generalises beyond a single quality axis. source: §6.4, Table 4 evidence: 'The method extends naturally to emotion TTS by substituting the selection criterion in the sampling stage: using emotional ESD dataset prompts and sorting by valence or arousal score instead of MOS drives the model to generate speech with target affective properties.' confidence: high relevance: medium limitations: - 'The comparison with SpeechAlign is acknowledged by the authors to be partially unfair: SpeechAlign-DPO requires ground truth speech as positive samples during optimization, which is additional supervision not available to UNO.' - Presenting both as baselines without fully separating this distinction may understate SpeechAlign's performance under matched conditions. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2406.02430' published_date: "2024-06-04" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS - VC architecture: - autoregressive-LM - diffusion - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_diffusion_acoustic claims: - claim_id: autoregressive_codec_language_models_trained_at_sufficient_scale role: supports claim: Autoregressive codec language models trained at sufficient scale can generate zero-shot speech with naturalness and speaker similarity indistinguishable from human speech as measured by CMOS. source: §3.1, Table 1 evidence: Seed-TTS ICL receives CMOS scores of -0.07 in English and -0.08 in Chinese relative to human speech while retaining strong speaker-similarity scores. confidence: high relevance: high - claim_id: self_distillation_on_perturbation_generated_pairs_is_an role: supports claim: Self-distillation on perturbation-generated pairs is an effective method for timbre disentanglement in TTS systems, achieving large speaker similarity gains in voice conversion without modifying model architecture or loss functions. source: §4.1, Table 6 evidence: 'ICL: +0.14 preference win - RL-SER improves emotion control from (0.46, 0.44, 0.53, 0.13) to (0.91, 0.80, 0.78, 0.82) across angry/happy/sad/surprise **Voice conversion (Table 6, self-distillation):** - Without self-distillation: SIM 0.491 EN, 0.636 ZH - With self-distillation: SIM 0.753 EN, 0.791 ZH — far exceeding DiffVC (0.311) and HierSpeech++ (0.387) **Speaker fine-tuning (Table 3):** - SFT CMOS +0.37 over ICL baseline **Deployed model (Table 5):** - Latency 0.028× offline, RTF 0.132×, with CMOS only -0.02 vs. offline model **ASR data utility (Table 2):** ASR model trained on synthetic Seed-TTS LibriSpeech data achieves WER within 0.31% absolute of real-data baseline on clean sets.' confidence: high relevance: high - claim_id: reinforcement_learning_post_training_with_objective_speech_quality role: supports claim: Reinforcement learning post-training with objective speech quality rewards improves robustness and speaker similarity in autoregressive TTS while introducing reward hacking that must be carefully managed. source: §4.2, Tables 7, 8 evidence: 'For post-training, REINFORCE is used with two reward configurations: Seed-TTS-RL-SIM-WER (WER + speaker similarity as rewards) and Seed-TTS-RL-SER (speech emotion recognition accuracy as reward).' confidence: high relevance: high - claim_id: end_to_end_diffusion_models_that_predict_total role: supports claim: End-to-end diffusion models that predict total duration and learn local text-audio alignment implicitly can match or exceed the quality of autoregressive codec LM systems without per-phoneme duration supervision. source: §4.3, Table 10 evidence: This enables zero-shot voice conversion without architectural changes. **Seed-TTSDiT (NAR variant):** A fully diffusion-based architecture (Diffusion Transformer / DiT) that directly maps Gaussian noise to vocoder latents conditioned on text, audio prompt, and a total duration signal. confidence: high relevance: high - claim_id: synthetic_speech_generated_by_a_high_quality_zero role: supports claim: Synthetic speech generated by a high-quality zero-shot TTS system can train ASR models that approach real-data performance on clean test sets, though a gap persists on noisy sets due to background noise smoothing. source: §3.1, Table 2 evidence: 'ICL: +0.14 preference win - RL-SER improves emotion control from (0.46, 0.44, 0.53, 0.13) to (0.91, 0.80, 0.78, 0.82) across angry/happy/sad/surprise **Voice conversion (Table 6, self-distillation):** - Without self-distillation: SIM 0.491 EN, 0.636 ZH - With self-distillation: SIM 0.753 EN, 0.791 ZH — far exceeding DiffVC (0.311) and HierSpeech++ (0.387) **Speaker fine-tuning (Table 3):** - SFT CMOS +0.37 over ICL baseline **Deployed model (Table 5):** - Latency 0.028× offline, RTF 0.132×, with CMOS only -0.02 vs. offline model **ASR data utility (Table 2):** ASR model trained on synthetic Seed-TTS LibriSpeech data achieves WER within 0.31% absolute of real-data baseline on clean sets.' confidence: high relevance: high limitations: - '- Model weights not released; no reported parameter count. - Zero-shot ICL struggles with speakers having strong accents or unique styles not captured in a short (15s) prompt — the fine-tuned model is still preferred for ''hard'' speakers. - Synthesized speech slightly over-smooths background noise (ASR degrades on noisy test sets). - Does not handle singing or speech with background music well. - Reward hacking (over-slow, over-articulated speech) in RL post-training requires careful tuning. - Multi-shot ICL for long-form generation not yet demonstrated.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2406.04904' published_date: "2024-06-07" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: multilingual_zero_shot_tts_training_degrades_speaker_similarity role: supports claim: Multilingual zero-shot TTS training degrades speaker similarity compared to monolingual training on the same data, reflecting a fundamental trade-off in cross-lingual speaker conditioning. source: §4.1, Table 2, Table 3 evidence: The multilingual training penalty on speaker similarity is acknowledged by the authors as an expected trade-off. confidence: high relevance: high - claim_id: a_perceiver_resampler_based_speaker_conditioning_encoder_producing role: supports claim: A Perceiver Resampler-based speaker conditioning encoder, producing multiple fixed-length embeddings from variable-length reference audio, improves voice cloning robustness in massively multilingual autoregressive TTS over single-embedding approaches. source: §2 evidence: 'The key architectural departure from Tortoise is the Conditioning Encoder: rather than compressing a reference audio clip into a single 1,024-dim embedding, XTTS uses six scaled dot-product attention layers followed by a Perceiver Resampler to produce 32 fixed-size 1,024-dim embeddings per reference clip, independent of clip length.' confidence: high relevance: high - claim_id: evaluating_multilingual_tts_models_against_monolingual_baselines_on role: supports claim: Evaluating multilingual TTS models against monolingual baselines on the same language produces misleading comparisons, because the multilingual model's per-language training data is substantially reduced. source: §3.2, §4.1 evidence: English comparisons against StyleTTS 2 and HierSpeech++ are against monolingual models, whereas XTTS is trained on 16 languages simultaneously. confidence: high relevance: low - claim_id: a_small_amount_of_target_speaker_fine_tuning role: supports claim: A small amount of target-speaker fine-tuning data (approximately 10 minutes) substantially improves speaker similarity in cross-lingual zero-shot synthesis, including extreme prosody styles such as whispering. source: §5 evidence: The paper also demonstrates that ten minutes of adaptation data for a target speaker improves SECS from 0.585 to 0.717 in cross-lingual transfer, including a whispered voice style transferred to all 16 languages. confidence: high relevance: high - claim_id: low_frequency_codec_codebook_entries_can_be_pruned role: supports claim: Low-frequency codec codebook entries can be pruned without quality loss and improve expressiveness in multilingual discrete-token TTS. source: §2 evidence: The first component is a VQ-VAE (13M parameters) that encodes mel-spectrograms into discrete codes using a single codebook of 8,192 entries at 21.53 Hz. confidence: high relevance: low limitations: - Speaker similarity lags behind monolingual specialists in English, and the multilingual evaluation uses cross-lingual prompting (English speaker references for non-English languages), which may understate true within-language similarity. - No human listening test was conducted for multilingual outputs beyond subjective English comparisons. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2406.05370' published_date: "2024-06-08" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: adaptive_sampling_that_detects_and_breaks_token_repetition role: supports claim: Adaptive sampling that detects and breaks token repetition loops can stabilise autoregressive codec LM decoding without requiring forced-alignment auxiliary data. source: §3.4.1, Table 1 evidence: The system is trained on Libriheavy (50k hours of labelled English audiobooks) using only utterance-level speech-transcription pairs, with no forced alignment or per-speaker auxiliary data. confidence: high relevance: medium - claim_id: grouping_codec_codes_into_multi_token_ar_steps role: supports claim: Grouping codec codes into multi-token AR steps reduces effective sequence length and simultaneously improves long-context modelling quality at moderate group sizes. source: §3.1, §4.2.1, Table 1 evidence: This reduces sequence length by a factor of G, cutting both inference latency and the context length that the Transformer must attend over, which the paper shows improves long-context modelling. confidence: high relevance: low - claim_id: autoregressive_codec_tts_can_match_or_exceed_ground role: supports claim: Autoregressive codec TTS can match or exceed ground-truth speech on robustness and speaker similarity metrics when evaluated on clean English audiobook benchmarks. source: §4.2.2, Table 2; §4.3.2, Table 5 evidence: On LibriSpeech test-clean with a reference utterance as prompt, VALL-E 2 (G=1) achieves SMOS of 4.61 vs. ground truth SMOS of 4.13, and CMOS of +0.033 relative to ground truth (Table 2). confidence: high relevance: high - claim_id: prompt_availability_in_both_the_ar_and_nar role: supports claim: Prompt availability in both the AR and NAR stages is independently necessary for preserving speaker identity; removing either prompt degrades speaker similarity substantially. source: §4.2.3, Table 3; §4.3.3, Table 6 evidence: Ablations confirm that prompt input to both AR and NAR models is critical for speaker similarity (removing it collapses SIM by 50–80%), and that the NAR model's explicit acoustic condition splitting substantially improves speaker identity capture. confidence: high relevance: high - claim_id: inference_time_multiple_sampling_followed_by_metric_based role: supports claim: Inference-time multiple sampling followed by metric-based selection can substantially close the single-sample robustness gap, but at proportional computational cost. source: §4.1.3, Table 1 evidence: On the same dataset with a 3s prefix prompt, single-sampling WER reaches 1.6% against ground truth WER of 1.6%, matching it with no multiple-sampling overhead (Table 1). confidence: high relevance: medium limitations: - Human parity is claimed solely from results on LibriSpeech test-clean and VCTK; both benchmarks are read speech from controlled or semi-controlled recording conditions. - Generalisation to spontaneous, noisy, or low-resource speech is undemonstrated and the authors explicitly flag this caveat. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2406.05551' published_date: "2024-06-08" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: ardit_continuous_tokens_avoid_codec_tradeoff role: supports claim: Autoregressive generation of continuous speech tokens preserves the quality advantages of high-bitrate representations while avoiding the length constraints of high-bitrate discrete codecs. source: §1, §3.2 evidence: VAE-based continuous tokens at 1.7 kbps yield near-lossless reconstruction vs. discrete codecs at similar bitrates; ARDiT avoids the bitrate-fidelity tradeoff of VQ. confidence: high relevance: low - claim_id: fm_distillation_improves_naturalness role: supports claim: Diffusion distillation into single-step generators can improve perceptual naturalness of TTS samples relative to their multi-step teacher models, not only match it. source: §4.2, Table 1 evidence: ARDiT(DMD, B=4) achieves MUSHRA naturalness 79.3 vs. teacher model's lower naturalness score; DMD distillation improves over the multi-step ODE teacher on perceptual quality. confidence: medium relevance: low - claim_id: ardit_ar_order_improves_quality role: supports claim: Autoregressive decoding order materially improves speech intelligibility and speaker similarity in TTS compared to fully parallel non-autoregressive diffusion generation at the same total computation budget. source: §4.4, Figure 4 evidence: Block-size ablation shows AR models consistently outperform fully non-autoregressive limit (B=INF) on intelligibility and speaker similarity, with gap widening with more training. confidence: high relevance: high - claim_id: ardit_fim_speech_editing role: supports claim: Fill-in-the-middle training enables near-ground-truth speech editing quality with continuous-token autoregressive models, bringing MUSHRA scores within statistical confidence intervals of human recordings. source: §4.3, Table 2 evidence: ARDiT(DMD, R=1) achieves MUSHRA naturalness 77.8 on Test Set C vs. ground truth 78.1; significantly outperforms VoiceCraft (64.6) at p < 0.01. confidence: high relevance: medium limitations: - All training and evaluation is on LibriTTS (reading-style English audiobook speech); generalization to in-the-wild or noisy speech is unverified. - Evaluation uses MUSHRA rather than MOS, making direct numerical comparison with most TTS literature harder. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2406.07855' published_date: "2024-06-12" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: phoneme_monotonic_alignment_in_decoder_only_autoregressive_tts role: supports claim: Phoneme monotonic alignment in decoder-only autoregressive TTS can close most of the robustness gap caused by unconstrained attention, achieving near-ground-truth WER without encoder-decoder architectural changes. source: §3.2, Table 1 evidence: Unlike encoder-decoder monotonic attention methods, this approach is compatible with the decoder-only Transformer architecture. confidence: high relevance: low - claim_id: downsampling_only_the_first_rvq_layer_of_a role: supports claim: Downsampling only the first RVQ layer of a neural codec at inference time reduces autoregressive steps and latency by more than half, with negligible impact on PESQ and STOI. source: §3.1, Table 5 evidence: 'Applying merging only to the first RVQ layer (rather than multiple layers) preserves audio quality: PESQ and STOI drop negligibly when only layer 1 is merged at 2x, while merging four or eight layers causes significant degradation.' confidence: high relevance: low - claim_id: robustness_improvements_that_route_additional_phoneme_tokens_through role: supports claim: Robustness improvements that route additional phoneme tokens through the autoregressive stream (as in ELLA-V) improve WER but increase inference time, illustrating a robustness-efficiency trade-off in codec LM TTS. source: §5.3, Table 4 evidence: For inference efficiency (Table 4), VALL-E R (2x merging) generates 10 seconds of speech in 3.67 seconds (375 AR steps), compared to VALL-E's 10.27 seconds (750 AR steps) — a 64% reduction. confidence: high relevance: low - claim_id: explicit_phoneme_level_alignment_in_a_codec_lm role: supports claim: Explicit phoneme-level alignment in a codec LM enables independent control of prosody and timbre by substituting preset phoneme sequences at inference, enabling a form of voice conversion. source: §3.2.3, Table 3 evidence: 'Because VALL-E R explicitly tracks phoneme progress during inference, it also supports prosody control: replacing the self-predicted phoneme sequence with a preset aligned phoneme sequence from a reference utterance allows timbre and prosody to be controlled independently, enabling a form of voice conversion.' confidence: high relevance: high limitations: - All evaluations use LibriSpeech (clean English read speech). - Robustness gains from monotonic alignment and codec-merging quality preservation have not been tested on noisy, expressive, or multilingual speech. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2406.18009' published_date: "2024-06-26" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: explicit_phoneme_alignment_supervision_in_non_autoregressive_tts role: supports claim: Explicit phoneme alignment supervision in non-autoregressive TTS, while useful for early convergence, imposes a naturalness ceiling that joint character-level training can exceed. source: §3.4, Table 2 evidence: 4) shows E2 TTS takes longer to converge than Voicebox (which benefits from explicit alignment supervision early in training) but surpasses it by end of training, consistent with the hypothesis that joint learning of duration and acoustic modeling yields a better end state. confidence: high relevance: medium - claim_id: flow_matching_mel_spectrogram_generators_can_learn_alignment role: supports claim: Flow-matching mel spectrogram generators can learn alignment implicitly from raw characters and filler tokens, eliminating the need for grapheme-to-phoneme converters, phoneme aligners, and duration models. source: §2.1, §2.3 evidence: 'The CMOS comparison directly identifies phoneme alignment as the primary bottleneck to naturalness: replacing it with joint character-and-filler modeling closes the gap.' confidence: high relevance: low - claim_id: unsupervised_pretraining_on_unlabeled_speech_improves_downstream_zero role: supports claim: Unsupervised pretraining on unlabeled speech improves downstream zero-shot TTS performance in both intelligibility and speaker similarity. source: §3.4, Table 1 evidence: On LibriSpeech-PC test-clean (1,132 samples, 39 speakers), E2 TTS trained on Libriheavy achieves WER 2.0% and speaker similarity (SIM-o) 0.675 from random initialization, improving to WER 1.9% and SIM-o 0.708 with unsupervised pretraining. confidence: high relevance: high - claim_id: zero_shot_tts_speaker_similarity_scores_at_inference role: supports claim: Zero-shot TTS speaker similarity scores at inference can exceed those of ground-truth recordings on standard speaker verification metrics, suggesting the metrics reward consistency within a generation rather than perceptual identity. source: §3.4, Table 2 evidence: On LibriSpeech-PC test-clean (1,132 samples, 39 speakers), E2 TTS trained on Libriheavy achieves WER 2.0% and speaker similarity (SIM-o) 0.675 from random initialization, improving to WER 1.9% and SIM-o 0.708 with unsupervised pretraining. confidence: high relevance: high - claim_id: non_autoregressive_tts_systems_trained_jointly_on_duration role: supports claim: Non-autoregressive TTS systems trained jointly on duration and acoustics scale predictably with training data volume without requiring architectural changes. source: §3.4, Table 1 evidence: Training on 200,000 hours of proprietary data reaches WER 1.9% and SIM-o 0.707 from random initialization. confidence: high relevance: low limitations: - The CMOS evaluation covers only 39 samples from 39 LibriSpeech speakers read English. - 'The naturalness finding ("indistinguishable from ground truth") is thus narrow in scope: spontaneous speech, non-native speakers, expressive or prosodically complex content, and cross-lingual settings are untested.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2407.05361' published_date: "2024-07-07" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS - evaluation architecture: [] relevance: medium evidence_role: - infrastructure current_role: active_evidence method_family: [] claims: - claim_id: emilia_itw_data_matches_formal_improves_spontaneous role: supports claim: Training speech generation models on large-scale in-the-wild data, after automated quality filtering, can match audiobook-derived training data on formal-domain benchmarks while improving performance on spontaneous speech generation. source: §V.A, Table V evidence: AR+SoundStorm and VoiceBox trained on Emilia-En (46k hours) match MLS-trained counterparts on LibriSpeech-Test WER/S-SIM/FSD, but improve WER, S-SIM, and FSD on Emilia-Test (spontaneous speech) for both architectures. confidence: high relevance: medium - claim_id: emilia_scaling_law_diminishing_returns role: supports claim: Increasing in-the-wild training-set size continues to improve zero-shot TTS quality at fixed model capacity, with the largest gains at smaller scales and diminishing but still-positive returns as scale grows. source: §V.B, Figure 5 evidence: Scaling training data from 5k to 134k hours steadily reduces AR+SoundStorm WER (5.2% to 4.2% on LibriSpeech-Test; 5.7% to 4.9% on Emilia-Test), with gains concentrated below 46k hours and convergence emerging around 100k hours. confidence: high relevance: high - claim_id: emilia_multilingual_pooling_tradeoff role: complicates claim: Training a single model on pooled multilingual in-the-wild data enables crosslingual generation but trades off same-language quality relative to a monolingual model trained on comparable per-language data volume. source: §V.C evidence: The Emilia-Large-trained multilingual AR+SoundStorm model scores WER 4.9%, S-SIM 0.636, and FSD 15.31 on English, all slightly worse than an English-only 134k-hour model (WER 4.5%, S-SIM 0.65, FSD 14.8). confidence: medium relevance: low - claim_id: emilia_filtering_discards_majority_of_raw_audio role: complicates claim: Automated quality-filtering pipelines for in-the-wild speech data necessarily discard a large majority of raw source audio to reach a training-quality bar. source: §III, Table 2 evidence: On a 666.94-hour benchmark sample, Emilia-Pipe's six stages retain only 38.75% of the input (258.44 hours, 96,117 clips) after DNSMOS, language-confidence, and duration-outlier filtering. confidence: high relevance: low - claim_id: emilia_crosslingual_transfer_remains_harder role: complicates claim: Crosslingual voice transfer in zero-shot TTS remains substantially harder than same-language transfer even when the training corpus itself is multilingual and balanced across the target languages. source: §V.C, Table VI evidence: Crosslingual WER and speaker similarity degrade relative to matched-language pairs across all six languages, e.g. VoiceBox S-SIM falls from 0.588 En-En to 0.386 En-Zh. confidence: high relevance: high limitations: - Subjective evaluation in the audiobook-vs-in-the-wild comparison uses only sixteen samples and twelve English-speaking evaluators, limiting statistical reliability of reported CMOS/SMOS differences. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2407.05407' published_date: "2024-07-07" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: inserting_a_vector_quantizer_into_a_supervised_asr role: supports claim: Inserting a vector quantizer into a supervised ASR encoder yields discrete speech tokens that preserve significantly stronger text-semantic alignment than unsupervised alternatives such as HuBERT or EnCodec tokens. source: §2.1, §5.1, Table 7 evidence: The supervised VQ-inserted ASR encoder obtains 3.18% and 7.56% WER on LibriSpeech test-clean and test-other, outperforming the compared unsupervised token representations. confidence: high relevance: low - claim_id: in_autoregressive_codec_lm_tts_both_the_text role: supports claim: In autoregressive codec-LM TTS, both the text tokenizer and the speech tokenizer independently contribute to content consistency, while speaker similarity is primarily controlled by the speaker embedding and acoustic model conditioning. source: §5.2, Table 7 evidence: 2.89%/6.57% for the vanilla ASR model (only a minor degradation). **LLM:** An autoregressive transformer models the sequence [SOS, speaker-embedding, text-encodings, speech-turn-token, speech-tokens, EOS]. confidence: high relevance: high - claim_id: asr_re_ranking_is_an_effective_post_hoc role: supports claim: ASR re-ranking is an effective post-hoc method for improving content consistency in autoregressive TTS without any model retraining, at the cost of increased inference-time compute. source: §5.3, Tables 8, 9 evidence: Generating multiple candidates and selecting them with ASR scoring improves content consistency without retraining the speech generator, while increasing inference cost. confidence: high relevance: low - claim_id: instruction_fine_tuning_on_a_modest_amount_of role: supports claim: Instruction fine-tuning on a modest amount of labelled data (556h) enables a TTS system to control fine-grained paralinguistic features — including laughter, breath, and word emphasis — with substantially improved accuracy over the base model. source: §2.4, §5.4, Table 10 evidence: For cross-lingual cloning, the prompt text and tokens are omitted to prevent source-language prosodic leakage; only speaker embedding and prompt mel are passed to the flow model. **CosyVoice-instruct:** Built on CosyVoice-base with instruction fine-tuning on 556h of curated data covering speaker identity descriptions, speaking style (emotion, gender, rate, pitch), and fine-grained paralinguistics (laughter, breath, emphasis). confidence: high relevance: high - claim_id: high_quality_tts_synthesized_speech_can_serve_as role: supports claim: High-quality TTS-synthesized speech can serve as effective training data augmentation for ASR, with text diversity of the synthesis prompts contributing more to downstream ASR gains than the raw duration of the synthetic corpus. source: §5.5, Table 11 evidence: 2.79%/5.97% on real data; combining real + synthetic on MLS text yields 2.04%/4.53%, a strong improvement. confidence: high relevance: medium limitations: - '- Only a single VQ codebook (4096 codes) is used; multi-level RVQ and its effect on quality vs. compression is left for future work. - The choice of VQ insertion layer (after layer 6 of 12) is not ablated — optimal placement is unresolved. - Cross-lingual cloning omits prompt prosody to prevent leakage, which may reduce naturalness in target language. - Instruction fine-tuning data amounts (556h) are modest; broader paralinguistic coverage remains open. - No subjective (MOS) evaluation in the main paper; relies entirely on objective WER/CER/SS metrics.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2407.08551' published_date: "2024-07-11" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: continuous_mel_spectrogram_representations_preserve_more_speaker_relevant role: supports claim: Continuous mel-spectrogram representations preserve more speaker-relevant acoustic information than vector-quantized codec codes at standard compression rates. source: §5.1, Table 1 evidence: This replaces the top-p discrete sampling of codec language models with a learned continuous sampling mechanism, providing output diversity without requiring manual sampling configuration. confidence: high relevance: medium - claim_id: autoregressive_tts_models_trained_to_predict_continuous_frames role: supports claim: Autoregressive TTS models trained to predict continuous frames can achieve naturalness comparable to human speech while avoiding the silence and repetition failures endemic to discrete codec language models. source: §5.2, Table 3 evidence: This replaces the top-p discrete sampling of codec language models with a learned continuous sampling mechanism, providing output diversity without requiring manual sampling configuration. confidence: high relevance: low - claim_id: variational_sampling_in_the_continuous_latent_space_is role: supports claim: Variational sampling in the continuous latent space is more effective than top-p discrete sampling for improving output diversity and speaker similarity in autoregressive TTS. source: §5.3, Table 4 evidence: This replaces the top-p discrete sampling of codec language models with a learned continuous sampling mechanism, providing output diversity without requiring manual sampling configuration. confidence: high relevance: high - claim_id: a_reduction_factor_that_predicts_multiple_frames_per role: supports claim: A reduction factor that predicts multiple frames per autoregressive step can substantially reduce inference time with only modest degradation in speaker similarity. source: §5.4, Table 5 evidence: 'The reduction factor r provides a throughput knob: r=2 halves AR steps and inference time with modest speaker similarity loss, while r=4 reduces inference time to one quarter of the baseline and still outperforms most VALL-E variants on WER.' confidence: high relevance: high limitations: - The subjective evaluation rests on only 40 samples from a single English corpus (LibriSpeech test-clean). - The naturalness and speaker similarity advantages may not generalize to noisier prompts, non-native accents, or other languages. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2409.00750' published_date: "2024-09-01" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS - VC architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: non_autoregressive_masked_generative_transformers_can_achieve_human role: supports claim: Non-autoregressive masked generative transformers can achieve human-level speaker similarity in zero-shot TTS without requiring explicit text-speech alignment or phone-level duration supervision. source: §4.2.1, Table 2 evidence: A flow-matching duration predictor estimates total speech length from text and prompt duration, but phone-level durations are only used as intermediates to derive the total, not as direct conditioning. confidence: high relevance: high - claim_id: replacing_k_means_quantisation_of_ssl_features_with role: supports claim: Replacing k-means quantisation of SSL features with VQ-VAE vector quantisation reduces information loss in tonal languages and improves downstream acoustic token prediction. source: §3.2.1 evidence: Semantic tokens are not extracted via k-means (which loses tonal information, problematic for Chinese) but from a custom VQ-VAE trained on W2v-BERT 2.0 hidden states (17th layer). confidence: high relevance: low - claim_id: masked_generative_tts_substantially_outperforms_autoregressive_tts_on role: supports claim: Masked generative TTS substantially outperforms autoregressive TTS on hard-text robustness (tongue twisters, repeating phrases) while maintaining competitive naturalness on standard benchmarks. source: §4.2.2, Appendix J, Table 13 evidence: In multilingual settings (Japanese, Korean, French, German), MaskGCT substantially outperforms XTTS-v2 and Emilia baseline systems on SIM-O across all four languages, with lower WER in three of four. confidence: high relevance: low - claim_id: parallel_iterative_decoding_in_masked_generative_models_yields role: supports claim: Parallel iterative decoding in masked generative models yields constant inference cost regardless of output length, in contrast to autoregressive decoding whose cost scales linearly with utterance duration. source: §4.2.2 evidence: During inference, the T2S model decodes in 25-50 parallel iterative steps (constant regardless of output length). confidence: high relevance: low - claim_id: zero_shot_style_cloning_via_in_context_learning role: supports claim: Zero-shot style cloning via in-context learning extends to accent and emotion transfer without task-specific architectural changes. source: §4.3, Tables 4–5 evidence: Text and prompt semantic tokens are prepended as a prefix, enabling in-context learning without alignment supervision. confidence: high relevance: high limitations: - Speech content editing is acknowledged as "not very robust" by the authors, who attribute this to a training objective mismatch (mask-and-predict vs. fill-in-mask). - The editing capability is demonstrated qualitatively only, with no quantitative evaluation reported. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2409.03283' published_date: "2024-09-05" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: separating_the_waveform_generation_stage_into_a_low role: supports claim: Separating the waveform generation stage into a low-sampling-rate Mel decoder and a super-resolution vocoder allows a system trained predominantly on low-sampling-rate data to produce high-fidelity 48kHz output. source: §3.3 evidence: In both cases, a BigVGAN-V2-based super-resolution vocoder upsamples the 16kHz Mel spectrogram to 48kHz waveform, trained on a curated 294-hour high-sampling-rate subset. confidence: high relevance: low - claim_id: few_shot_fine_tuning_of_a_large_foundation role: supports claim: Few-shot fine-tuning of a large foundation TTS model substantially outperforms zero-shot in-context learning for highly expressive, distinctive target voices, even with only one hour of data. source: §5.2.1, Table 5 evidence: 'Two downstream applications are demonstrated: (1) zero-shot and few-shot voice cloning for video dubbing, where UGC scenarios use in-context learning and PUGC scenarios use 1-hour supervised fine-tuning of the LM and flow-matching decoder; (2) instruction-tuned human-like speech generation adding four emotion categories via a dedicated embedding layer and 13 paralinguistic behaviour types via token insertion or embedding injection.' confidence: high relevance: high - claim_id: prompt_audio_enhancement_improves_voice_cloning_quality_for role: supports claim: Prompt audio enhancement improves voice cloning quality for noisy prompts but can slightly degrade performance when prompts are already clean. source: §5.2.2, Table 6 evidence: Prompt enhancement before voice cloning is beneficial at SNR 0dB and 10dB but can slightly degrade performance at SNR 20dB (Table 6), recommending selective application based on noise level. confidence: high relevance: medium - claim_id: instruction_tuning_with_a_small_domain_specific_dataset role: supports claim: Instruction tuning with a small domain-specific dataset dramatically improves emotion controllability in a pre-trained TTS language model, raising accuracy from near-chance to near-ceiling. source: §5.3, Table 7 evidence: Instruction tuning raises emotion-classification accuracy from 45–87% before tuning to 97–100% across the four evaluated emotion categories. confidence: high relevance: low - claim_id: autoregressive_tts_systems_trained_on_predominantly_one_language role: supports claim: Autoregressive TTS systems trained on predominantly one language show markedly higher pronunciation error rates on under-represented languages, even at large data scales. source: §5.1.2, Table 3 evidence: CosyVoice's 5.68% in Chinese, but both systems show substantially higher rates in English and code-switch conditions (12% and 8.5% for FireRedTTS), attributed to the limited English proportion in training data (Table 3). confidence: high relevance: medium limitations: - All evaluations are conducted on proprietary internal test sets with no publicly released benchmarks, data, or model weights. - This makes direct comparison with other systems difficult to reproduce and limits the generalisability of the reported numbers. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2411.01156' published_date: "2024-11-02" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: eliminating_grapheme_to_phoneme_conversion_by_directly_feeding role: supports claim: Eliminating grapheme-to-phoneme conversion by directly feeding raw text to an LLM backbone is viable for multilingual TTS and can improve handling of context-dependent polyphonic words. source: §1, §3 evidence: The first component is the Dual Autoregressive (Dual-AR) language model, which replaces G2P preprocessing by leveraging an LLM backbone for direct linguistic feature extraction from raw text. confidence: high relevance: low - claim_id: hierarchical_decomposition_of_autoregressive_token_generation_into_semantic role: supports claim: Hierarchical decomposition of autoregressive token generation into semantic-level and acoustic-level stages improves codebook stability in grouped scalar quantization. source: §3.1, §3.1.1 evidence: This hierarchical decomposition is intended to improve the stability of the GFSQ codebook during autoregressive generation, which the authors report as a persistent challenge in token-based TTS when codebook dimensions are large. confidence: high relevance: low - claim_id: grouped_finite_scalar_vector_quantization_achieves_higher_codebook role: supports claim: Grouped Finite Scalar Vector Quantization achieves higher codebook utilisation than residual vector quantization alternatives, mitigating dead-code collapse. source: §3.2.2, §3.2.3 evidence: The quantization scheme, GFSQ (Grouped Finite Scalar Vector Quantization), combines Finite Scalar Quantization (FSQ) with Group Vector Quantization (GVQ), partitioning the feature space into groups and applying scalar quantization per group. confidence: high relevance: low - claim_id: real_time_tts_inference_with_low_first_packet role: supports claim: Real-time TTS inference with low first-packet latency is achievable on consumer GPU hardware through standard inference optimisations without architectural compromise. source: §4.2 evidence: Inference uses KV caching, torch compile, and other optimisations, achieving a first-packet latency of 150ms and a real-time factor of approximately 1:5 on an RTX 4060 mobile GPU and 1:15 on an RTX 4090. confidence: high relevance: low limitations: - The entire experimental evaluation is conducted on a proprietary test set with undisclosed composition and size. No public benchmark is used, making it impossible to independently verify the claimed superiority over CosyVoice and F5-TTS or to compare against the broader literature. - The MOS evaluation uses "naive listeners" rather than trained raters or crowdsourced panels following standard listening test protocols (e.g. ITU-T P.800), which may inflate scores relative to conventional evaluations. The paper does not report model size, training compute, or inference memory requirements in full, limiting reproducibility. DPO training details are omitted from the main training description. The paper does not evaluate cross-lingual transfer or accent preservation, which are claimed motivations for the non-G2P design. It is also unclear how the system handles low-resource languages beyond the eight listed in the training data. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2411.09943' published_date: "2024-11-15" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - VC architecture: - diffusion - transformer-enc-dec relevance: high evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - diffusion_voice_cloning claims: - claim_id: seedvc_timbre_shifter_reduces_leakage role: supports claim: Using an imperfect external VC model as a timbre-shifting augmentation device during training reduces timbre leakage in content representations without requiring discrete bottlenecking, preserving intelligibility. source: §3.3, Table 1 evidence: Seed-VC achieves SECS 0.8676 with the lowest WER (11.99%) and CER (2.92%) among compared systems on zero-shot VC, by extracting Whisper semantic features from timbre-shifted (rather than original) source audio during training. confidence: high relevance: high - claim_id: seedvc_incontext_reference_improves_similarity role: supports claim: Conditioning a generative VC model on the full reference utterance via in-context learning yields higher speaker similarity than conditioning on a single speaker embedding alone. source: §3.4, Table 1 evidence: Removing full reference enrollment and using only the timbre vector drops SECS from 0.8676 to 0.7948. confidence: high relevance: high - claim_id: seedvc_mitigates_similarity_intelligibility_tradeoff role: supports claim: The speaker similarity versus intelligibility trade-off that limits bottleneck-based zero-shot VC approaches can be mitigated by separating timbre perturbation from content feature extraction. source: §1, §3.3.1 evidence: By perturbing timbre before content extraction rather than quantising content features to remove speaker information, Seed-VC avoids the intelligibility loss typically associated with aggressive bottlenecking (k-means, VQ). confidence: medium relevance: high - claim_id: seedvc_f0_conditioning_singing_vc role: supports claim: F0 conditioning enables diffusion transformer VC models to extend to zero-shot singing voice conversion with competitive pitch preservation and improved lyric intelligibility over per-speaker trained baselines. source: §3.5, Table 2 evidence: On M4Singer, Seed-VC outperforms RVCv2 on SECS (0.7405 vs. 0.7264) and CER (19.70% vs. 28.46%) in a zero-shot setting, while RVCv2 requires per-character training. confidence: medium relevance: high limitations: - No subjective listening tests (MOS or MUSHRA) are reported; all quality comparisons rely on DNSMOS, an automatic metric. The training data advantage (Emilia-101k, ~100k hours) versus baselines is substantial and not dataset-controlled. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2412.04724' published_date: "2024-12-06" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - VC architecture: - flow-matching relevance: high evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: stablevc_independent_timbre_style_transfer role: supports claim: Independent timbre and style transfer from distinct unseen speakers can be achieved without degrading either attribute when conditioning streams are separated via parallel cross-attention with adaptive gating. source: §DualAGC, Table 2 evidence: On the style transfer task with timbre and style references from different unseen speakers, StableVC achieves F0 correlation 0.75 and RMSE 12.87, versus 0.69/15.68 for the next-best diffusion baseline. confidence: high relevance: high - claim_id: stablevc_flow_matching_beats_diffusion_speed_quality role: supports claim: Flow matching enables zero-shot voice conversion that surpasses diffusion baselines in both sample quality and inference speed, with quality stabilising in as few as 10 ODE steps. source: §Conditional Flow Matching, Table 1, Table 3 evidence: StableVC achieves nMOS 3.96 and RTF 0.146, a 1.65x speedup over the diffusion baseline DDDM-VC (RTF 0.287, nMOS 3.67); quality stabilises at 10 Euler steps. confidence: high relevance: high - claim_id: stablevc_grl_reduces_timbre_leakage_into_style role: supports claim: Timbre leakage into style representations is a failure mode in jointly trained VC systems, and adversarial disentanglement via gradient reversal measurably reduces this cross-contamination. source: §Training Objectives, Table 4 evidence: Ablation removing the GRL adversarial loss degrades both style and timbre metrics, confirming the gradient reversal layer actively suppresses timbre information from leaking into the style encoder. confidence: high relevance: high - claim_id: stablevc_multiref_speaker_prior_critical_for_intelligibility role: supports claim: Using multiple reference utterances alongside a pre-trained speaker verification prior as the timbre attention key dramatically improves intelligibility in cross-attention-based timbre modeling, as its removal causes WER to collapse from 2% to over 22%. source: §DualAGC, Table 4 evidence: Ablation removing the timbre prior and multiple references causes severe intelligibility collapse, with WER jumping from 2.03% to 22.16%. confidence: high relevance: high - claim_id: stablevc_nonAR_constant_latency role: supports claim: Non-autoregressive VC generation maintains near-constant latency regardless of utterance length, a qualitative advantage over token-by-token autoregressive decoding that is not captured by fixed-length RTF comparisons. source: §Experimental Results on Zero-shot VC evidence: StableVC runs at RTF 0.146, a 25x speedup over autoregressive LM-VC (RTF 3.891), attributed to its non-autoregressive flow-matching decoding scheme. confidence: medium relevance: low limitations: - The factorized codec used as the style extractor is referenced as a public tool but not described in detail in the paper, restricting exact reproducibility. The evaluation covers English speakers only (VCTK for timbre, ESD for style). caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2412.10117' published_date: "2024-12-13" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence - acceleration_evidence current_role: influential method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: finite_scalar_quantization_achieves_full_codebook_utilization_in role: supports claim: Finite scalar quantization achieves full codebook utilization in supervised speech tokenizers, capturing substantially more semantic content than vector quantization at equivalent bitrates. source: §2.2, §4.1, Table 4 evidence: FSQ achieves 100% codebook utilization vs. 23% for VQ, and reduces ASR error rate from 18.26% to 10.67% on CommonVoice EN — indicating substantially more semantic content is preserved. confidence: high relevance: medium - claim_id: replacing_a_randomly_initialized_custom_lm_with_a role: supports claim: Replacing a randomly initialized custom LM with a pre-trained LLM backbone improves content consistency in hybrid TTS systems without requiring a separate text encoder. source: §2.3, §4.3, Table 7 evidence: 'Unified Text-Speech LM: Uses Qwen2.5-0.5B as the backbone, removing the text encoder and speaker embedding from the sequence entirely.' confidence: high relevance: medium - claim_id: streaming_and_non_streaming_synthesis_can_be_unified role: supports claim: Streaming and non-streaming synthesis can be unified in a single autoregressive model through interleaved text-speech token sequences, with virtually lossless quality on typical inputs relative to offline mode. source: §2.3, §4.2, Table 8 evidence: 'Two training modes share one model: (1) Non-streaming: [S, text tokens, T, speech tokens, E]; (2) Streaming: text and speech tokens are interleaved at ratio N:M=5:15 — every 5 text tokens are followed by 15 speech tokens, with "filling tokens" inserted when text tokens are not yet available.' confidence: high relevance: low - claim_id: training_a_flow_matching_model_simultaneously_on_multiple role: complicates claim: Training a flow matching model simultaneously on multiple causal mask types — from non-causal to full-causal — enables a single model to span the latency-quality trade-off continuum at inference time, with masks providing implicit self-distillation. source: §2.4, §4.3, Table 8 evidence: 'Chunk-aware Causal Flow Matching: The flow matching model is trained with four mask types simultaneously: non-causal (offline, best quality), full-causal (lowest latency), Chunk-M (past + M future frames), and Chunk-2M (near-offline quality with moderate latency).' confidence: high relevance: low - claim_id: differentiable_asr_reward_optimization_generalizes_better_to_out role: supports claim: Differentiable ASR reward optimization generalizes better to out-of-domain and hard-case inputs than preference-based DPO in TTS speaker fine-tuning. source: §2.8, §4.7, Table 11 evidence: A differentiable ASR reward (Gumbel-softmax sampling through FSQ indices → frozen ASR decoder) avoids the four-forward-pass overhead of standard DPO and generalizes better to out-of-domain/hard cases. confidence: high relevance: medium limitations: - '- EN quality still lags CosyVoice 2 behind Seed-TTS and F5-TTS on SEED test-en (WER 2.57% vs. 2.25% and 1.83%), reflecting data imbalance toward Chinese. - Japanese synthesis degrades due to character set overlap with Chinese (CER 18.79% test-ja vs. 7.98% test-ko). - Cannot control timbre through text instructions. - Singing not supported. - Streaming still incurs a hard degradation on test-hard, suggesting that contextual information from future text is important for difficult patterns.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2412.15649' published_date: "2024-12-20" entry_date: '2026-07-26' year: 2024 venue: arXiv task: - SCA architecture: - autoregressive-LM - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: decoupling_speaker_identity_from_semantic_content_in_spoken role: supports claim: Decoupling speaker identity from semantic content in spoken dialogue systems enables zero-shot timbre control without modifying the language model or adding speaker-conditioning layers. source: §3.4 evidence: Because the semantic tokens encode only linguistic content (not speaker identity), a conditional flow-matching vocoder conditioned on a speaker audio prompt converts semantic tokens to mel spectrograms, which HiFi-GAN then synthesizes to waveform. confidence: high relevance: high - claim_id: grouping_audio_tokens_to_reduce_the_frequency_mismatch role: supports claim: Grouping audio tokens to reduce the frequency mismatch between text and speech representations substantially improves speech-text alignment and training efficiency in parallel audio-text dialogue models. source: §3.3, Table 5 evidence: With G = 3, the audio sequence length is reduced by 3x, enabling the model to converge with less than half the GPU hours of the G = 1 baseline while achieving substantially better speech-text alignment (4.54% ASR-WER vs. 18.23% at G = 1). confidence: high relevance: low - claim_id: single_stage_fine_tuning_on_dialogue_data_can role: supports claim: Single-stage fine-tuning on dialogue data can match or outperform multi-stage pipelines that include ASR or TTS pre-training, because modality-specific pre-training degrades instruction-following and general knowledge retention. source: §5.3.2, Table 6 evidence: The entire system is trained in a single stage on dialogue data (primarily VoiceAssistant-400K, ~664 hours of instruction audio), with full fine-tuning of all components except the Whisper encoder, which is kept frozen. confidence: high relevance: medium - claim_id: replacing_audio_token_history_with_text_only_history role: supports claim: Replacing audio-token history with text-only history in multi-turn spoken dialogue models improves the system's ability to handle longer conversation contexts while leveraging pre-trained LLM in-context learning. source: §3.5 evidence: For multi-turn dialogue, Historical Text Prompting replaces the typical approach of maintaining interleaved audio-text token histories. confidence: high relevance: high - claim_id: current_spoken_dialogue_models_consistently_underperform_text_only role: supports claim: Current spoken dialogue models consistently underperform text-only LLMs of similar scale on semantic content quality, even after dialogue fine-tuning. source: §5.1, Table 3 evidence: At 0.5B scale, SLAM-Omni substantially outperforms Mini-Omni and Mini-Omni2 on ChatGPT-based content scoring (39.32 vs. 22.58 and 26.56 overall), achieves the highest UTMOS across all SDMs including the larger 7-9B models (4.45 vs. 4.37 for Freeze-Omni), and the lowest ASR-WER (4.54% vs. 6.05% for Mini-Omni and 10.42% for LLaMA-Omni). confidence: high relevance: low limitations: - Historical text prompting discards all non-verbal information from prior dialogue turns (prosody, emotion, paralinguistic cues). In scenarios where voice-level context matters for dialogue coherence, this strategy may reduce response quality in ways not captured by the text-based evaluation metrics used. - The system is evaluated exclusively at 0.5B scale. The single-stage training advantage may not hold for larger LLMs, where the data volume required for joint audio-text modeling grows substantially. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2409.09098' published_date: "2025-01-09" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - encoder_decoder_voice_cloning claims: - claim_id: accentbox_speaker_accent_entanglement_hurts_generalization role: supports claim: Speaker-accent entanglement in accent identification models causes poor generalisation to unseen speakers and limits the utility of accent embeddings for conditioning TTS systems. source: §III.A, §IV.A, Table IV evidence: The CommonAccent baseline achieves 0.96 accuracy on seen speakers but only 0.43 on unseen speakers (gap 0.53), and has a high Silhouette Coefficient for Speaker Clusters (SCSC) of 0.236, indicating it memorises speaker-to-accent mappings; GenAID with information bottleneck and adversarial training reduces the gap to 0.06 and SCSC to 0.079. confidence: high relevance: high - claim_id: accentbox_continuous_embeddings_beat_onehot role: supports claim: Continuous accent embeddings extracted from a speaker-agnostic model provide stronger accent conditioning for zero-shot TTS than discrete one-hot accent labels. source: §IV.B, Tables V–VII evidence: AccentBox conditioned on continuous GenAID embeddings achieves higher accent cosine similarity than the Accent_ID system using one-hot accent embeddings in both inherent and cross accent generation, and is preferred by listeners in subjective accent similarity tests. confidence: high relevance: high - claim_id: accentbox_accent_naturalness_tradeoff_data_dependent role: complicates claim: Zero-shot accent generation introduces a trade-off between accent fidelity and naturalness that varies with accent data coverage. source: §IV.B, Table VI evidence: AccentBox shows higher naturalness preference for American accent (60.0% preferred over Baseline, p=0.011) but lower preference for Irish accent (33.9%), where limited training data and sensitivity to monotonic prosody in reference speech cause degradation. confidence: medium relevance: high - claim_id: accentbox_objective_subjective_speaker_similarity_mismatch role: complicates claim: Objective speaker similarity metrics may not align with subjective listener perception when accent and speaker identity are jointly manipulated. source: §IV.B, Tables V–VI evidence: For inherent accent generation, AccentBox achieves lower SpkCos (0.8293) than Baseline (0.8413) and VALL-E X (0.8605) in objective evaluation, yet listeners subjectively prefer AccentBox for speaker similarity (70.0%, p=0.002 for American accent). confidence: medium relevance: high - claim_id: accentbox_standard_evals_miss_accent_hallucination role: refines claim: Standard ZS-TTS evaluations based on naturalness and speaker similarity fail to detect accent hallucination, underrepresenting the accent fidelity gap between TTS systems trained predominantly on American English and target accented speakers. source: §I, §III.B evidence: The paper demonstrates that current SOTA ZS-TTS systems (including VALL-E X) generate a default American-English accent regardless of the target speaker's accent, an artefact not captured by conventional MOS or speaker-similarity metrics. confidence: medium relevance: high limitations: - Subjective listening tests are restricted to two accents (American and Irish) due to budget constraints. The TTS backbone (YourTTS, VITS-based) is several generations behind current LLM-based and flow-matching ZS-TTS systems. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2501.06282' published_date: "2025-01-10" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - SCA architecture: - autoregressive-LM - flow-matching - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: large_scale_multi_task_training_across_heterogeneous_speech role: supports claim: Large-scale multi-task training across heterogeneous speech tasks substantially improves both comprehension and generation in aligned multimodal speech LMs without catastrophic forgetting of the base LLM's text capabilities. source: §3.4, §4.1–4.5 evidence: MinMo adopts an aligned multimodal architecture that adapts a pretrained text LLM by adding audio input and output pathways without fundamentally replacing the LLM's operating regime. confidence: high relevance: low - claim_id: aligned_multimodal_architectures_can_achieve_instruction_controlled_speech role: supports claim: Aligned multimodal architectures can achieve instruction-controlled speech style (emotion, dialect, speaking rate, voice identity) when trained with appropriate instruction data, contradicting prior claims that this capability is limited to native multimodal models. source: §3.2, §4.4, Table 18 evidence: End-to-end instruction control (emotion, dialect, speaking rate, voice identity) is achieved by conditioning on hidden embeddings from the LLM, which carry the user instruction context through to the audio output. confidence: high relevance: medium - claim_id: full_duplex_spoken_dialogue_at_sub_second_latency role: supports claim: Full-duplex spoken dialogue at sub-second latency is achievable with a modular aligned architecture combining a semantic predictor with a streaming autoregressive decoder, without requiring joint speech-text pre-training. source: §3.1, §4.5, Table 21 evidence: The end-to-end full-duplex response latency is ~600ms in theory (250ms duplex decision + 150ms text generation + 70ms speech token generation + 130ms token-to-wave), and ~800ms in practice on L20 GPUs. confidence: high relevance: low - claim_id: an_autoregressive_streaming_voice_decoder_that_interleaves_text role: supports claim: An autoregressive streaming voice decoder that interleaves text hidden states with speech tokens outperforms non-autoregressive CTC-based decoders in naturalness and content consistency for aligned speech LMs. source: §3.2, §4.4, Table 17 evidence: 'The decoder operates on interleaved sequences of semantic vectors and speech tokens in a fixed 5:15 ratio: every five text tokens produce five semantic vectors via a linear output projector, and these drive fifteen autoregressive speech tokens from the Voice Token LM.' confidence: high relevance: medium limitations: - The instruction-following voice generation evaluation uses a single in-house Chinese test set of 122 turns, making external validation of the 98.4% accuracy figure impossible. Code and model weights had not been released at time of publication. - LoRA-only updates to the text LLM during training limit the model's ability to follow diverse instructions; more comprehensive LLM updates with higher-quality text data remain unexplored. Long-tail pronunciation errors persist in end-to-end audio generation, partly due to special symbols that the decoder cannot reliably convert to speech. The full-duplex module still relies on external acoustic echo cancellation (AEC) and voice activity detection (VAD) modules, meaning a truly end-to-end duplex system has not been achieved. Performance on speech emotion recognition in low-resource languages shows mixed results (e.g., Polish at 55.9% F1), indicating that cross-lingual generalization is uneven despite the broad multilingual training. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.coling-main.685 published_date: "2025-01-19" entry_date: '2026-07-26' year: 2025 venue: COLING task: - TTS architecture: [] relevance: medium evidence_role: - infrastructure - control_evidence current_role: active_evidence method_family: [] claims: - claim_id: voxpopulitts_naturalistic_corpora_enable_crosslingual role: supports claim: Large-scale, naturally-spoken multilingual corpora enable cross-lingual zero-shot TTS capabilities in models that lack them when trained on single-language or audiobook-derived data. source: §4, Table 2 evidence: XTTS_v2 trained on the VoxpopuliTTS large subset (10,000h per language) achieves cross-lingual synthesis in EN, FR, and ES with MOS scores of 3.37, 3.52, and 3.31, respectively. confidence: medium relevance: high - claim_id: voxpopulitts_naturalness_intelligibility_tradeoff role: complicates claim: Naturalness and conversational style in training data trades off against intelligibility metrics during TTS model training. source: §4 evidence: The authors attribute relatively high WER values (8.25-9.20%) to the naturalness and conversational complexity of VoxpopuliTTS, suggesting models trained on naturalistic in-the-wild data face greater transcript-fidelity challenges than those trained on clean studio recordings. confidence: medium relevance: low - claim_id: voxpopulitts_asr_confidence_filters_hallucination role: supports claim: ASR confidence scores provide an effective proxy for filtering hallucinated transcripts in automatic speech processing pipelines for TTS data curation. source: §2.2 evidence: Per-word logit thresholding (confidence < 0.7 discarded) is used to remove FasterWhisper hallucinations, validated as effective in the pipeline design alongside punctuation-based segment merging. confidence: medium relevance: low - claim_id: voxpopulitts_quality_scale_tradeoff role: complicates claim: Quality-stratified corpus subsets introduce a trade-off between data scale and audio quality, requiring practitioners to select the appropriate tier for their task. source: §3, Table 1 evidence: VoxpopuliTTS is split into small (~1,000h, DNSMOS ~3.87), medium (~5,000h, DNSMOS ~3.71), and large (~10,000h, DNSMOS ~3.39) subsets per language, with quality decreasing as scale increases. confidence: medium relevance: low limitations: - Does not compare VoxpopuliTTS against Emilia or WenetSpeech4TTS in a controlled training experiment, making it impossible to assess whether the pipeline or dataset composition offers measurable advantages over alternatives. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2502.04128' published_date: "2025-02-06" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: single_stage_autoregressive_tts_trained_with_next_token role: supports claim: Single-stage autoregressive TTS trained with next-token prediction over discrete speech tokens is competitive with multi-stage AR+NAR pipelines on intelligibility and speaker similarity in continuation mode, though SIM-o gaps remain due to codec acoustic reconstruction limits. source: §3.2.4, Table 3 evidence: However, speaker similarity (SIM-o) for direct inference is notably weaker than these baselines, reflecting the acoustic reconstruction ceiling of a single-VQ codec versus multi-layer RVQ or mel-based vocoders. confidence: high relevance: high - claim_id: both_model_scale_and_training_data_volume_independently role: supports claim: Both model scale and training data volume independently improve TTS quality across naturalness, prosody, and text comprehension, consistent with scaling laws observed in text LLMs. source: §2.3, §3.2.2, Tables 2, 4 evidence: 'Scaling both model size and training data consistently improves all metrics: WER on test-en drops from 3.71% (1B-80k) to 2.97% (8B-250k), and emotion similarity on ESD rises from 0.753 to 0.778 (English) across model sizes (Table 4).' confidence: high relevance: low - claim_id: inference_time_compute_scaling_via_speech_understanding_verifiers role: complicates claim: Inference-time compute scaling via speech understanding verifiers can substantially improve speaker similarity and emotional expressiveness beyond what train-time scaling alone achieves, at the cost of additional inference compute. source: §2.4, §3.2.3, Figure 2, Table 2 evidence: Inference-time scaling uses off-the-shelf speech understanding models as verifiers. confidence: high relevance: high - claim_id: pure_process_reward_model_beam_search_for_tts role: supports claim: Pure process reward model beam search for TTS is prone to mode collapse that degrades content accuracy (WER), and a hybrid partial-PRM strategy is needed to preserve both speaker similarity and intelligibility. source: §3.2.3, Figure 2 evidence: 'Two search strategies are compared: Best-of-N (output reward model, ORM) and beam search with a process reward model (PRM) that evaluates speaker similarity at each 0.5-second step.' confidence: high relevance: high - claim_id: single_vq_codecs_can_achieve_intelligibility_and_naturalness role: complicates claim: Single-VQ codecs can achieve intelligibility and naturalness competitive with multi-layer RVQ codecs at the same token rate, but acoustic fidelity (speaker similarity) remains the limiting factor for single-VQ reconstruction. source: §3.1.2, Table 1 evidence: However, speaker similarity (SIM-o) for direct inference is notably weaker than these baselines, reflecting the acoustic reconstruction ceiling of a single-VQ codec versus multi-layer RVQ or mel-based vocoders. confidence: high relevance: high limitations: - 'The SIM-o gap between Llasa and RVQ-based baselines is intrinsic to the single-VQ design: acoustic reconstruction from a 65,536-entry single codebook at 50 Hz is weaker than 8-layer RVQ codecs, and this gap is only partially recovered by inference-time search. Systems requiring high timbre fidelity in a single inference pass would need a different codec design.' - Inference-time compute scaling requires running multiple candidates (beam search or Best-of-N) with auxiliary verifier models, which increases latency and compute cost substantially and makes the approach unsuitable for real-time or low-resource applications. The paper does not characterize latency or wall-clock overhead of the search strategies. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2502.05512' published_date: "2025-02-08" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: speaker_conditioning_via_a_multi_reference_conformer_perceiver role: supports claim: Speaker conditioning via a multi-reference Conformer Perceiver improves zero-shot voice cloning stability and timbre consistency over single-vector speaker embeddings. source: §2.3, Table 4 evidence: Speaker conditioning uses a Conformer-based Perceiver encoder with a subsampling rate of 2, replacing the single-vector speaker embeddings used by Tortoise and CosyVoice. confidence: high relevance: high - claim_id: direct_waveform_decoding_from_lm_hidden_states_via role: supports claim: Direct waveform decoding from LM hidden states via a GAN vocoder achieves competitive audio quality with faster inference than diffusion-based intermediate representation decoding. source: §2.4, Table 5 evidence: The speech decoder directly converts the LM's last hidden state to waveform via BigVGAN2, with a 4x interpolation step from 25 Hz to 100 Hz before decoding. confidence: high relevance: low - claim_id: fsq_reaches_near_100_codebook_utilisation_with_less role: supports claim: FSQ reaches near-100% codebook utilisation with less training data than VQ, though VQ converges to similar utilisation with sufficient data scale. source: §3.3.2, Figure 2 evidence: The paper compares Vector Quantization (VQ) against Finite Scalar Quantization (FSQ) with levels [8, 8, 8, 6, 5], finding that VQ collapses to around 55% codebook utilisation at 6k hours of training data whereas FSQ reaches near-100% utilisation; at 34k hours both converge. confidence: high relevance: low - claim_id: hybrid_character_pinyin_tokenisation_enables_reliable_correction_of role: supports claim: Hybrid character-pinyin tokenisation enables reliable correction of Chinese polyphonic character mispronunciations at inference time without requiring a separate grapheme-to-phoneme module. source: §3.3.1, Table 2 evidence: 'To handle polyphonic characters, a character-pinyin hybrid modelling approach is used at training time: 50% of training samples have 20% of their non-polyphonic Chinese characters randomly replaced with pinyin, training the model to accept pinyin corrections at inference.' confidence: high relevance: low limitations: - Model size is not reported, and the training data pipeline uses proprietary internet-sourced audio with pseudo-labels from commercial ASR; neither the data nor the code is released, limiting reproducibility. - The system is limited to Chinese and English, with acknowledged weak emotional expression replication. Instruction-based voice generation is explicitly unsupported. The MOS evaluation relies on 100 samples from an unspecified test set distribution, and the SPK-SIM metric uses ERes2Net rather than a standardised model, making direct comparison with published baselines difficult. The paper does not report streaming latency or real-time factor, despite positioning the hybrid architecture as streaming-capable. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2502.07243' published_date: "2025-02-11" entry_date: '2026-07-26' year: 2025 venue: ICLR task: - TTS - VC architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: the_vq_vae_codebook_vocabulary_size_can_function role: complicates claim: The VQ-VAE codebook vocabulary size can function as a self-supervised information bottleneck for progressive disentanglement of timbre, style, and linguistic content in self-supervised speech representations. source: §3.1, Table 2 evidence: 'At its core, the approach treats the VQ-VAE codebook vocabulary size as a controllable information bottleneck applied to HuBERT-Large 18th-layer features: a large vocabulary (4,096 entries) produces content-style tokens that preserve style and content but strip out timbre; a small vocabulary (32 entries) produces content tokens that strip out both timbre and most style, retaining primarily linguistic content.' confidence: high relevance: medium - claim_id: zero_shot_style_imitation_accent_and_emotion_conversion role: supports claim: Zero-shot style imitation (accent and emotion conversion) without annotation can match or exceed supervised baselines that rely on parallel corpora and style labels. source: §4.3, Table 4 evidence: For zero-shot style imitation (Table 4), Vevo-Style outperforms accent conversion baselines (ASR-AC, VoiceShop, Conv-Speak) and emotion conversion baseline (Emovox) in N-CMOS and style accuracy metrics (A-ACC, A-CMOS, E-ACC, E-CMOS) despite using no parallel corpus, no style labels, and no textual transcriptions. confidence: high relevance: high - claim_id: hybrid_two_stage_pipelines_combining_autoregressive_style_modeling role: supports claim: Hybrid two-stage pipelines combining autoregressive style modeling with flow-matching acoustic generation can decouple style and timbre control more effectively than single-stage approaches that use in-context learning to mimic all speech attributes jointly. source: §3.4, Tables 3–5 evidence: The second stage, acoustic modeling, uses a 334M-parameter flow-matching transformer to generate mel spectrograms from content-style tokens, conditioned on a timbre reference mel spectrogram via a span-masking strategy inspired by Voicebox. confidence: high relevance: high - claim_id: autoregressive_models_in_zero_shot_tts_consistently_trade role: supports claim: Autoregressive models in zero-shot TTS consistently trade intelligibility (higher WER) for stronger style imitation compared to non-autoregressive alternatives trained on the same data. source: §4.4, Tables 5, 9 evidence: For zero-shot TTS (Table 5, ACCENT+EMOTION samples), Vevo-TTS achieves emotion similarity MOS of 4.03 and accent similarity MOS of 4.12, surpassing CosyVoice (trained on 171K hours of in-the-wild data) and MaskGCT (trained on 100K hours) on these style dimensions, while being trained only on 60K hours of audiobooks. confidence: high relevance: high - claim_id: duration_reduction_on_content_tokens_improves_style_transfer role: supports claim: Duration reduction on content tokens improves style transfer fidelity by removing unit-level duration patterns that encode source speaking style. source: §4.5, Table 6 evidence: A duration reduction step collapses consecutive duplicate content tokens before feeding them to the model, which further suppresses unit-level duration patterns that encode style, and also shortens inference sequence length. confidence: high relevance: low limitations: - Style imitation evaluations (Table 4) use demo website samples from baseline systems as the test set, meaning evaluation conditions (recording environment, speaker demographics, utterance content) differ between Vevo and baselines. These comparisons are suggestive but not controlled, and the reported improvements should be treated as approximate. - Training is restricted to English audiobook speech (clean, single-domain), and no multilingual or expressive speech experiments are reported. The content-style token vocabulary size (K_s = 4096) and content token vocabulary size (K_c = 32) are empirically selected; the authors note these may not be globally optimal. The AR content-style model has 463M parameters and requires sequential decoding, introducing latency that could be problematic for streaming applications. The self-supervised disentanglement quality depends on HuBERT-Large features, requiring a large pre-trained SSL model as a prerequisite. Style controllability through a single reference utterance may be brittle for rare or highly expressive speaking styles not represented in the audiobook training distribution. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2502.18924' published_date: "2025-02-26" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - diffusion - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - diffusion_voice_cloning claims: - claim_id: megatts3_sparse_alignment_naturalness_robustness role: supports claim: Providing coarse stochastic phoneme anchors rather than fully expanded forced alignments improves both naturalness and robustness simultaneously in latent diffusion TTS. source: §3.2, §4.3, Table 4, Table 7 evidence: Sparse alignment anchors improve prosodic naturalness metrics (GPE 0.44→0.31, VDE 0.33→0.29) over forced alignment while cutting hard-sentence WER from 8.49% (E2-TTS, fully implicit) to 3.95%. confidence: high relevance: low - claim_id: megatts3_compact_continuous_latent_quality role: supports claim: Compact continuous latent representations at very low token rates enable higher zero-shot TTS quality than discrete codecs at higher bit rates when used as the target space for diffusion. source: §4.5, Table 5, Table 6 evidence: WaveVAE's 25 vectors/second continuous latent space outperforms discrete codec targets at higher bitrates on downstream TTS quality, per the paper's codec-choice ablation. confidence: high relevance: high - claim_id: megatts3_perflow_distillation_25_to_8_steps role: supports claim: Piecewise rectified flow distillation reduces inference steps from 25 to 8 with negligible degradation in speaker similarity and intelligibility. source: §3.2, §4.2, Table 1 evidence: The PeRFlow-distilled MegaTTS 3 (8 steps) retains SIM-O 0.70 and WER 1.86% versus the 25-step teacher's SIM-O 0.71 and WER 1.82%, while RTF drops from 0.188 to 0.124. confidence: high relevance: high - claim_id: megatts3_decoupled_cfg_accent_control role: supports claim: Decoupled text and speaker guidance scales in classifier-free guidance provide a continuous accent intensity control axis without requiring accent labels. source: §3.2, §4.4, Table 3 evidence: Varying alpha-txt independently of alpha-spk shifts pronunciation systematically from distorted to accented to standard English on L2-ARCTIC, with no accent labels used in training. confidence: high relevance: medium - claim_id: megatts3_scaling_behaviour role: supports claim: Latent diffusion TTS systems exhibit strong data and model scaling behaviour, with both speaker similarity and intelligibility improving consistently as training data grows from 2k to 600k hours and model size grows from 0.5B to 7B parameters. source: Appendix D, Table 8 evidence: SIM-O improves from 0.52 at 2k training hours to 0.66 at 600k hours, and a 7B-parameter model reaches 0.74 SIM-O and 1.90% WER. confidence: medium relevance: high limitations: - Scaling and cross-domain results use an internal 400-sample test set, limiting external reproducibility for those specific claims. - The sparse alignment mechanism still depends on an external forced aligner (MFA) at training time, relaxing rather than eliminating the alignment-tooling dependency. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2503.01710' published_date: "2025-03-03" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: disentangling_speech_tokens_into_linguistic_content_and_speaker role: supports claim: Disentangling speech tokens into linguistic content and speaker attributes within a single-stream codec enables a standard LLM to perform zero-shot TTS without a multi-stage pipeline. source: §3, §4.1 evidence: 'Speaker similarity in zero-shot TTS is Spark-TTS''s relative weakness: it scores 0.672 SIM on test-zh and 0.584 SIM on test-en, outperforming Llasa (0.684, 0.574) but falling behind multi-stage models such as MaskGCT (0.774, 0.714) and Seed-TTS (0.796, 0.762).' confidence: high relevance: high - claim_id: small_llm_backbones_can_achieve_competitive_zero_shot role: supports claim: Small LLM backbones can achieve competitive zero-shot TTS intelligibility when the codec reduces per-token modeling complexity through semantic alignment. source: §6.4, Table 4 evidence: 'The Spark-TTS language model fine-tunes Qwen2.5-0.5B in a decoder-only configuration, directly predicting the concatenated token sequence: for zero-shot TTS, the model conditions on text and global tokens extracted from a reference audio, then predicts the semantic token sequence.' confidence: high relevance: high - claim_id: single_stage_autoregressive_tts_consistently_trails_multi_stage role: supports claim: Single-stage autoregressive TTS consistently trails multi-stage or non-autoregressive methods on speaker similarity metrics, even when intelligibility is comparable. source: §6.4, Table 4, Limitation evidence: 'Speaker similarity in zero-shot TTS is Spark-TTS''s relative weakness: it scores 0.672 SIM on test-zh and 0.584 SIM on test-en, outperforming Llasa (0.684, 0.574) but falling behind multi-stage models such as MaskGCT (0.774, 0.714) and Seed-TTS (0.796, 0.762).' confidence: high relevance: high - claim_id: fsq_based_global_token_quantization_with_learnable_cross role: supports claim: FSQ-based global token quantization with learnable cross-attention queries produces better speaker attribute representation than group-VQ at equivalent token lengths. source: §6.2, Table 2 evidence: The global tokenizer processes the Mel spectrogram through an ECAPA-TDNN encoder followed by a cross-attention module with learnable queries, producing a fixed-length sequence of 32 tokens that represent time-invariant speaker attributes. confidence: high relevance: medium - claim_id: attribute_controllable_tts_benefits_from_hierarchical_coarse_to role: supports claim: Attribute-controllable TTS benefits from hierarchical coarse-to-fine prediction within the LM inference loop rather than requiring separate conditioning modules. source: §4.1, §6.3 evidence: 'For attribute-controlled voice creation, a chain-of-thought (CoT) mechanism is employed: the model first predicts fine-grained attribute values (pitch in Hz rounded to integer, speaking rate in syllables-per-second) from coarse-grained labels (gender, 5-level pitch, 5-level speed), then generates global tokens, then semantic tokens.' confidence: high relevance: medium limitations: - Speaker similarity in zero-shot cloning is meaningfully lower than multi-stage methods (SIM 0.672 vs. 0.774 for MaskGCT on test-zh). The paper attributes this to AR variability without explicit disentanglement constraints between semantic and global tokens, and no solution is evaluated in this work. - The VoxBox training data and BiCodec codec are trained on separate, relatively limited datasets (3k hours for BiCodec; 102.5k hours for the LM). The BiCodec training data is English-only (LibriSpeech + Emilia EN/CN), which may limit acoustic reconstruction quality for languages outside this distribution. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2503.14345' published_date: "2025-03-18" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: spontaneous_scripting_from_an_llm_is_roughly_as role: supports claim: Spontaneous scripting from an LLM is roughly as important as the acoustic modeling choice for perceived spontaneity in long-form dialogue TTS. source: §4.2.2, Table 3 evidence: The paper demonstrates empirically that spontaneous scripting alone accounts for a large fraction of the final audio spontaneity. confidence: high relevance: low - claim_id: full_sequence_interleaving_of_text_and_speech_codes role: supports claim: Full-sequence interleaving of text and speech codes, extended to 40,000-token context windows with speaker-change tokens, enables coherent long-form zero-shot multi-speaker synthesis that turn-level concatenation cannot match. source: §3.2.1, Tables 1–2 evidence: The sequence design interleaves full podcast text with full speech codes (rather than per-turn interleaving), using speaker-change tokens to signal speaker transitions. confidence: high relevance: high - claim_id: curriculum_learning_progressively_exposing_a_codec_lm_to role: supports claim: Curriculum learning, progressively exposing a codec LM to increasing dialogue complexity, is an effective strategy for developing long-context and spontaneous generation capability without requiring matched long-context data from the outset. source: §3.2.1 evidence: 'Training uses curriculum learning across three stages: single-turn single-speaker zero-shot TTS; long-context two-speaker non-conversational data (audiobooks, up to 40,000 tokens / 800 seconds); and long-context two-speaker conversational data (podcasts).' confidence: high relevance: high - claim_id: automatic_speaker_similarity_metrics_cosine_embedding_similarity_can role: supports claim: Automatic speaker similarity metrics (cosine embedding similarity) can disagree with subjective speaker similarity ratings in long-form generation settings, particularly when the acoustic model attends to prosodic rather than purely timbral features. source: §4.2.1 evidence: However, SIM-O (automatic speaker cosine similarity) is notably lower for MoonCast on English (0.53 vs. 0.75 for the concatenation baseline), attributed to insufficient audiobook data in the English training curriculum, which the authors acknowledge as a limitation. confidence: high relevance: high - claim_id: chunk_wise_autoregressive_decoding_with_a_causal_chunk role: supports claim: Chunk-wise autoregressive decoding with a causal chunk mask provides a practical solution to the continuity and memory constraints of mel-spectrogram reconstruction from long semantic code sequences. source: §3.2.2 evidence: A chunk-wise causal attention mask allows each chunk to attend to all previously generated chunks, preserving continuity across boundaries without loading the full sequence. confidence: high relevance: low limitations: - All evaluation is conducted on a small internal test set (4 knowledge sources for podcast, 7 podcasts for the script ablation), and training data is entirely proprietary (~515K hours). Results cannot be independently reproduced, and generalisability to other domains or languages beyond Chinese and English is unverified. - The notably lower SIM-O score for English (0.53 vs. 0.75 baseline) reveals unresolved speaker consistency challenges in the English long-context setting, which the authors attribute to an imbalanced training curriculum favouring audiobook over conversational English data. Hallucinations in speaker attribution (utterances assigned to the wrong speaker) emerge from the interplay of semantic token timbre leakage, diarization errors in training data, and ambiguous filler-word interpretations; no mitigation is proposed beyond discussion. The two-speaker restriction (host + guest) is a deliberate scope limitation; extension to three or more speakers is left as future work. Evaluation is subjective-only for multi-speaker interactions; no standardised zero-shot TTS benchmark (LibriSpeech, VCTK) is used, limiting direct comparison to single-speaker zero-shot systems. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2504.02407' published_date: "2025-04-03" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: reformulating_the_output_of_a_flow_matching_tts role: supports claim: Reformulating the output of a flow-matching TTS model as a Gaussian distribution is sufficient to enable standard policy-gradient RL algorithms without architectural redesign. source: §2.2 evidence: The model is then trained under a modified flow-matching objective that maximises the log-likelihood of the target velocity under the predicted Gaussian, making the pretraining phase functionally equivalent to the original flow-matching loss while rendering the outputs probabilistic. confidence: high relevance: low - claim_id: grpo_with_wer_and_speaker_similarity_rewards_consistently role: supports claim: GRPO with WER and speaker-similarity rewards consistently reduces word error rate and increases speaker similarity in non-autoregressive TTS post-training, across diverse Mandarin datasets. source: §3.2.2, Table 1, Table 2 evidence: Speaker similarity rises from 0.698 to 0.730 (4.6% relative). confidence: high relevance: high - claim_id: rl_post_training_that_optimises_speaker_similarity_reward role: supports claim: RL post-training that optimises speaker-similarity reward also improves robustness to noisy reference audio, not only clean-reference speaker cloning. source: §3.2.2, Table 1 evidence: 'A noise-robustness subset using noisy reference audio shows an even larger WER benefit (33.6% relative reduction: 2.32% to 1.54%), suggesting GRPO also improves robustness to reference audio quality.' confidence: high relevance: high - claim_id: hard_text_tongue_twisters_and_repetitive_phrasing_degrades role: supports claim: Hard text (tongue twisters and repetitive phrasing) degrades all flow-matching TTS variants substantially, and RL training with WER rewards provides smaller relative gains in this regime than on plain text. source: §3.2.2, Table 1 evidence: On the hard set (tongue twisters and repetitive text), WER drops from 11.30% to 10.63% (6.1% relative) and SIM improves from 0.673 to 0.711 (5.6% relative). confidence: high relevance: low - claim_id: the_grpo_phase_requires_only_a_small_fraction role: supports claim: The GRPO phase requires only a small fraction of the pretraining data (100 h out of 7,226 h) and update steps to achieve measurable objective improvements, suggesting that RL fine-tuning is data-efficient for this task. source: §3.1 evidence: The entire GRPO phase uses only 100 hours of data from the same Mandarin corpus and runs for 1,100 update steps on 8 A100 GPUs, compared to 1 million steps in pretraining. confidence: high relevance: low limitations: - No subjective evaluation (MOS or MUSHRA) is reported. All claimed improvements are based on automatic WER and speaker cosine similarity, which are proxy metrics. Whether GRPO post-training preserves or alters naturalness remains untested. - Evaluation is restricted to Mandarin (Seed-TTS-eval test-cn). Generalisation to English, multilingual, or cross-lingual settings is not studied. The model builds directly on F5-TTS architecture and the WenetSpeech4TTS corpus; adapting to other flow-matching backbones or languages would require separate experiments. The hard-set WER (10.63% for F5R-TTS vs. 11.30% for F5) remains high in absolute terms, indicating that difficult phonetic sequences are not resolved by RL post-training alone. Future directions identified by the authors include PPO and DDPO integration, improved reward design for challenging scenarios, and larger-scale training experiments. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: iclr-2025-868masI331 published_date: "2025-04-24" entry_date: '2026-07-26' year: 2025 venue: ICLR task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - autoregressive_speech_token_lm claims: - claim_id: reducing_the_first_layer_rvq_frame_rate_of role: supports claim: Reducing the first-layer RVQ frame rate of a neural audio codec through hierarchical multi-resolution distillation enables autoregressive TTS to generate minute-long speech with stable intelligibility. source: §4, §7.3, Table 3, Table 4 evidence: MReQ-Encodec at 8 Hz achieves WER 9.79% on MinutesSpeech test-90s where standard Encodec at 8 Hz produces 100% WER and naive VALL-E at 48 Hz with long training data yields 16.14%. confidence: high relevance: low - claim_id: lowering_the_codec_frame_rate_in_autoregressive_tts role: complicates claim: Lowering the codec frame rate in autoregressive TTS improves temporal coherence and intelligibility for long utterances but degrades speaker similarity. source: §7.3, Table 4, Table 6 evidence: HALL-E consistently lags VALL-E by 0.025-0.042 SIM points on MinutesSpeech tests; the paper attributes this to acoustic information loss when compressing from 48 Hz to 8 Hz in the first RVQ layer. confidence: high relevance: high - claim_id: post_training_from_a_pre_trained_lm_checkpoint role: supports claim: 'Post-training from a pre-trained LM checkpoint is essential for hierarchical codec TTS: training from scratch without pre-trained weights collapses quality.' source: §7.4, Table 10 evidence: Removing VALL-E pre-training from HALL-E increases WER from 9.79% to 49.8% on MinutesSpeech test-90s, and removing MReQ pre-training similarly degrades codec reconstruction. confidence: high relevance: low - claim_id: the_frame_rate_at_which_autoregressive_speech_token role: refines claim: The frame rate at which autoregressive speech token generation becomes unstable is approximately 8 Hz, consistent with average phoneme durations of around 100 ms. source: Appendix C.1, Table 18 evidence: Ablation at 4 Hz first-layer rate raises WER to 20.07%, while 8 Hz yields 9.79%; the paper notes that phoneme duration averaging ~100 ms corresponds to ~10 Hz, making 4 Hz fundamentally insufficient. confidence: high relevance: low - claim_id: length_balanced_training_data_covering_the_target_synthesis role: supports claim: Length-balanced training data covering the target synthesis duration is necessary for autoregressive models to generalize to long-form speech. source: §7.3, Table 4 evidence: VALL-E trained only on segments up to 28 seconds achieves WER 39.77% on test-90s, and training on longer data decreases but does not eliminate the gap; HALL-E's 8 Hz formulation resolves instability that remains even with longer VALL-E training data. confidence: high relevance: low limitations: - 'Speaker similarity consistently degrades with frame rate reduction: HALL-E''s SIM scores are 0.025-0.042 lower than VALL-E across test conditions. This reflects a fundamental trade-off between temporal compression and acoustic fidelity preservation that the paper does not resolve.' - The current NAR model processes audio at full 48 Hz resolution, which limits NAR input length to around 28-54 seconds during training and requires a sliding window at inference. Integrating cross-attention text conditioning addresses alignment but does not eliminate the NAR length bottleneck. The paper also does not compare against streaming or chunked autoregressive synthesis approaches, which represent a practical alternative. MinutesSpeech training data consists entirely of English podcast speech, and generalization to other languages, domains, or reading styles is untested. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: iclr-2025-cuFzE8Jlvb published_date: "2025-04-24" entry_date: '2026-07-26' year: 2025 venue: ICLR task: - TTS architecture: - autoregressive-LM - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: continuous_latent_representations_can_replace_discrete_vector_quantization role: supports claim: Continuous latent representations can replace discrete vector quantization in autoregressive TTS without sacrificing generation quality. source: §5.1, Table 1, Table 2 evidence: GMM-LM trained on continuous GMM-VAE encoder features outperforms VALL-E (RVQ-based) on WER, speaker similarity, Q-MOS, and S-MOS on LibriSpeech test-clean across all prompt lengths, while using 10.3% of VALL-E's parameter count. confidence: high relevance: high - claim_id: longer_audio_prompts_do_not_uniformly_improve_zero role: complicates claim: Longer audio prompts do not uniformly improve zero-shot speaker cloning across AR architectures. source: §5.1, Table 2 evidence: VALL-E's WER increases monotonically with prompt length (6.04% at 3s, 7.54% at 8s, 9.68% at 15s), suggesting that simple cross-attention cannot leverage extended speaker context in AR decoding; the proposed GMM-LM shows the opposite trend, consistently benefiting from longer prompts. confidence: high relevance: high - claim_id: strict_monotonic_alignment_substantially_reduces_word_error_rate role: supports claim: Strict monotonic alignment substantially reduces word error rate in autoregressive TTS compared to standard cross-attention and soft monotonic variants. source: Appendix A.1, Table 6 evidence: Among alignment strategies tested on the same GMM-LM architecture, stochastic monotonic alignment with ST-Gumbel achieves WER 2.72% vs. 6.6% for cross-attention alone; even monotonic attention with Gumbel (without the stochastic binary forward pass) scores 3.34%. confidence: high relevance: low - claim_id: continuous_speech_representations_improve_downstream_autoregressive_model_perfor role: supports claim: Continuous speech representations improve downstream autoregressive model performance relative to discrete counterparts, independent of the alignment mechanism. source: Appendix A.6, Table 10 evidence: A head-to-head ablation comparing GMM-LM (continuous) against discrete AR models (VQ-VAE single codebook and DAC multi-codebook with delayed prediction), all using the proposed monotonic alignment, shows GMM-LM achieves WER 2.72% vs. 5.35% and 5.87% for the discrete variants. confidence: high relevance: low - claim_id: increasing_the_number_of_gaussian_components_in_continuous role: complicates claim: Increasing the number of Gaussian components in continuous AR modeling yields diminishing returns and can reduce quality through overfitting. source: §5.4, Table 4, Table 5, Appendix A.3 evidence: GMM-LM with 6 diagonal-covariance Gaussians (WER 2.72%, SIM 0.91) outperforms 3-Gaussian (WER 2.89%, SIM 0.85), but 10-Gaussian degrades to WER 5.21%, SIM 0.71; the 6-mixture GMM-VAE also shows worse evaluation-set reconstruction than the 3-mixture model despite better training-set fit. confidence: high relevance: low limitations: - The evaluation compares against VALL-E (2023), a dated AR baseline. Stronger AR systems published by the time of this paper's submission are not included, limiting the strength of the superiority claim for continuous AR over discrete AR in general. - The model is evaluated exclusively on English audiobook speech (LibriLight training, LibriSpeech evaluation). Generalisation to conversational speech, noisy in-the-wild data, or other languages is not demonstrated, though noise robustness experiments (Appendix A.4) show the method degrades gracefully under additive noise in prompts. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: iclr-2025-hQvX9MBowC published_date: "2025-04-24" entry_date: '2026-07-26' year: 2025 venue: ICLR task: - TTS architecture: - diffusion - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - diffusion_voice_cloning claims: - claim_id: diffusion_transformer_backbones_are_better_suited_to_tts_than_u_net_back role: supports claim: Diffusion Transformer backbones are better suited to TTS than U-Net backbones once domain-specific conditioning factors (phonemes, durations) are removed. source: §5.2, Table 4 evidence: Under matched training conditions, replacing the DiT backbone with a U-Net (and a U-Net variant without down/up-sampling) increases WER from 2.93 to 3.7 and drops SIM-r from 0.588 to 0.389 on the English cross-sentence task. confidence: high relevance: low - claim_id: predicting_total_target_length_and_generating_variable_length_sequences role: supports claim: Predicting total target length and generating variable-length sequences outperforms fixed-length generation with padding in diffusion-based TTS. source: §5.2, Table 5 evidence: Fixed-length modeling with padding reaches WER 6.81-8.89, while a learned speech length predictor with variable-length generation reaches WER 5.36-5.58 under otherwise identical settings. confidence: high relevance: low - claim_id: aligning_text_and_speech_latent_representations_improves_cross_attention role: supports claim: Aligning text and speech latent representations improves cross-attention-conditioned generation quality, independent of model or training-data scale. source: §5.2, Tables 6-7 evidence: A speech codec fine-tuned with an auxiliary language-modeling loss against a frozen text encoder (Mel-VAE++) improves WER/SIM over the unaligned codec regardless of which text encoder (ByT5 or SpeechT5) is paired with it, and a jointly text-speech-trained text encoder (SpeechT5, 85M params) outperforms a larger text-only encoder (ByT5-base, 415M params) trained on more data. confidence: high relevance: low - claim_id: removing_domain_specific_alignment_factors_from_ldm_based_tts_narrows_bu role: complicates claim: Removing domain-specific alignment factors from LDM-based TTS narrows but does not eliminate the gap to phoneme-duration-based systems in speaker similarity. source: §5.1, Table 2 evidence: DiTTo-en-XL reaches SIM-r 0.6554 on the cross-sentence task, below Voicebox's reported 0.681 (a phoneme/duration-based non-autoregressive model), even though DiTTo-en-XL is faster and matches or exceeds Voicebox on WER. confidence: medium relevance: high - claim_id: codec_compression_ratio_not_codec_reconstruction_quality_alone_determine role: complicates claim: Codec compression ratio, not codec-reconstruction quality alone, determines suitability as a diffusion target for variable-length TTS. source: §5.2, Table 7 evidence: DAC achieves higher PESQ and ViSQOL codec-reconstruction scores than Mel-VAE, but its 7-8x longer latent sequences make training and inference substantially less efficient and degrade end-to-end WER/SIM relative to the more compressed but lower-fidelity Mel-VAE. confidence: medium relevance: low limitations: - See the source paper page for the complete limitations and evaluation scope. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.findings-naacl.279 published_date: "2025-04-29" entry_date: '2026-07-26' year: 2025 venue: NAACL task: - TTS architecture: - autoregressive-LM - GAN - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: few_shot_fine_tuning_on_a_small_amount role: supports claim: Few-shot fine-tuning on a small amount of target-speaker studio audio substantially improves naturalness, clarity, and speaker similarity over a zero-shot pretrained multilingual TTS model, even when the zero-shot model was pretrained on thousands of hours of in-language speech. source: §4, Table 1 evidence: Fine-tuning on 20 minutes per speaker (4 speakers, 10 epochs) raises SMOS from 4.456 (BnTTS-0, zero-shot) to 4.624 (BnTTS-n, few-shot) and SECS (reference) from 0.529 to 0.548 on BnStudioEval. confidence: high relevance: high - claim_id: expressive_conversationally_prosodic_tts_output_can_score_worse role: complicates claim: Expressive, conversationally-prosodic TTS output can score worse on ASR-based intelligibility metrics than flatter-sounding commercial systems, even when subjective listening scores favor the expressive system. source: §4, "High CER in Text Generation", Tables 1–2 evidence: BnTTS-n and BnTTS-0 show higher CER than Google Cloud TTS and Azure TTS on both BnStudioEval and BnTTSTextEval despite BnTTS-n achieving the highest SMOS, Naturalness, and Clarity scores overall; the authors attribute this to ASR systems being better suited to standardized, less expressive speech patterns. confidence: high relevance: low - claim_id: autoregressive_codec_based_tts_models_trained_primarily_on role: complicates claim: Autoregressive codec-based TTS models trained primarily on longer utterances can fail to generate short sequences reliably under default sampling settings. source: §4, "Effect of Sampling and Prompt Length on Short Speech Generation", Table 3 evidence: Under default generation settings (T=0.85, TopK=50), the model achieves only 0.699 DurationEquality and 0.081 CER on a 23-sample short-text subset (under 30 characters); adjusting prompt length and sampling temperature/top-K improves this to 0.827 DurationEquality and 0.015 CER. confidence: high relevance: medium - claim_id: adapting_a_multilingual_zero_shot_tts_architecture_to role: refines claim: Adapting a multilingual zero-shot TTS architecture to a new low-resource language via continual pretraining can match or exceed existing language-specific commercial and open-source systems without requiring training from scratch. source: §3 "BnTTS model", §7 "Limitations", Tables 1–2 evidence: BnTTS-n, initialized from the pretrained XTTS checkpoint and continually pretrained on 3.85k hours of Bengali data, outperforms IndicTTS, Google Cloud TTS, and Azure TTS on SMOS, Naturalness, and Clarity across both evaluation datasets, while the authors note training from scratch was not attempted due to resource constraints. confidence: high relevance: high limitations: - The system struggles to adapt to speakers with unique vocal traits absent from training, and fails to generate short sequences (under two words or 20 characters) in some cases even after mitigation; these failures stem from limitations inherited from the underlying XTTS foundation model rather than being fully resolved by the Bangla adaptation. - The authors did not train XTTS from scratch for Bangla, relying instead on continual pretraining due to resource constraints, leaving the ceiling of a from-scratch Bangla model untested. A substantial portion of the pretraining corpus (3636 of 3856 hours) is pseudo-labeled and not manually reviewed, which may introduce label noise not fully captured by the reported metrics. The evaluation is limited to four in-house speakers for the few-shot speaker-adaptation setting, and dialectal variation within Bangla is not addressed. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.naacl-demo.12 published_date: "2025-04-29" entry_date: '2026-07-26' year: 2025 venue: NAACL task: - TTS - SCA architecture: - autoregressive-LM relevance: low evidence_role: - infrastructure current_role: minor method_family: - autoregressive_speech_token_lm claims: - claim_id: a_speech_language_model_can_be_initialized_from role: supports claim: A speech language model can be initialized from a pre-trained text LLM and jointly trained on speech recognition, speech synthesis, text continuation, and audio continuation without substantially degrading the text-only capability of the base model. source: §4.3, Table 5 evidence: The 1.7B multi-task model trained on ASR, TTS, TextLM, and AudioLM objectives scores MMLU 30.5, ARC-C 41.3, and HellaSwag 50.4, close to the text-only LLaMA-3.2-1B baseline (32.2, 32.8, 41.2) despite carrying three additional speech tasks. confidence: high relevance: low - claim_id: concatenating_neural_codec_tokens_with_self_supervised_speech role: supports claim: Concatenating neural codec tokens with self-supervised speech representations frame-by-frame is a viable tokenization strategy for both speech understanding and generation tasks within a single sequential model. source: §3.3 evidence: The "Codec_SSL" scheme (ESPnet-Codec combined with XEUS SSL tokens) is used for the headline ASR and TTS experiments and the paper reports it "behaves well in both speech understanding and generation" *(§3.3)*. confidence: medium relevance: low - claim_id: a_decoder_only_autoregressive_speech_language_model_can role: supports claim: A decoder-only autoregressive speech language model can match or exceed dedicated, larger ASR-only systems on English benchmarks while using substantially fewer parameters. source: §4.2, Table 3 evidence: A 442M-parameter ESPnet-SpeechLM ASR model reaches average WER 5.4% across six English test sets, matching OWSM v3.1-medium (1.02B, 5.4%) and beating Whisper-small (244M, 6.4%) and Whisper-medium (769M, 5.7%). confidence: high relevance: low - claim_id: cross_system_comparisons_of_speech_language_models_reported role: complicates claim: Cross-system comparisons of speech language models reported in the literature are frequently not run under matched conditions, limiting how much can be concluded from any single performance table. source: §4.3, Table 5 evidence: In the multi-task comparison (Table 5), competitor numbers for Moshi, VITA, GLM-4-Voice, and others are taken directly from their own published reports rather than reproduced by the authors, and the paper explicitly flags this with footnote markers. confidence: high relevance: medium - claim_id: combining_a_codec_tokenizer_with_a_self_supervised role: complicates claim: Combining a codec tokenizer with a self-supervised tokenizer frame-by-frame is reported as an effective design choice but is not validated against single-tokenizer ablations in the same controlled setting. source: §3.3 evidence: The claim that Codec_SSL tokenization "behaves well" rests on a single line of justification without a paired ablation against codec-only or SSL-only tokenization on the same task and dataset. confidence: high relevance: low limitations: - The multi-task model's TTS quality (Proxy MOS 3.99, WER 6.0%) is noticeably weaker than the single-task TTS model trained on the same architecture (Proxy MOS 4.03, WER 3.1%), indicating a capacity or interference cost to joint multi-task training that the paper reports but does not analyze further. Most training and evaluation data is English-only (the multilingual text corpus is used only for the TextLM objective, not for speech tasks), so the demonstrated speech capabilities are not evidence of multilingual generalization. Competitor numbers in the multi-task comparison table are drawn from third-party reports under unmatched training data and conditions rather than reproduced by the authors, which the paper itself notes. As a system/demo paper, the contribution is the toolkit and its reference recipes rather than a novel architecture or training method; the headline numbers serve to validate functionality rather than push state of the art on any individual benchmark. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.naacl-demo.21 published_date: "2025-04-29" entry_date: '2026-07-26' year: 2025 venue: NAACL task: - SCA architecture: [] relevance: low evidence_role: - infrastructure current_role: minor method_family: [] claims: - claim_id: cascaded_pipelines_higher_output_quality_than_e2e role: supports claim: Cascaded spoken dialogue pipelines built from dedicated, task-specific ASR and TTS components can produce substantially higher audio output quality than current end-to-end spoken dialogue systems that generate speech directly from audio input. source: §7, Table 4 evidence: On Switchboard Eval 2000, the directly-synthesized speech from the E2E system Mini-Omni scores UTMOS 2.88, versus 3.97-4.07 for cascaded pipelines using LJSpeech, LibriTTS, or VCTK VITS as the TTS module. confidence: medium relevance: low - claim_id: e2e_coherent_but_repetitive_across_turns role: complicates claim: End-to-end spoken dialogue systems that respond directly to audio input without explicit dialogue-context modeling can produce coherent individual responses while still exhibiting unnatural conversational behavior such as repeated or templated phrasing across turns. source: §7, Table 3 evidence: The text dialogue evaluation finds that Mini-Omni "produces highly coherent and contextually relevant responses" but "exhibits significant overlap in its responses, often generating template-like outputs," which the authors note could make conversations feel less natural in deployment. confidence: medium relevance: low - claim_id: domain_mismatched_asr_produces_misleading_wer role: complicates claim: Automatic ASR module benchmarking on conversational telephone speech corpora can produce highly misleading word error rates when the candidate ASR model's training domain does not match the conversational, disfluent register of the test data. source: §7, Table 2 evidence: An ESPnet LibriSpeech-trained ASR model evaluated on Switchboard Eval 2000 yields a WER of 342.2% and CER of 298%, far worse than domain-matched models like OWSM 3.1 (14.4% WER), illustrating a severe train/test domain mismatch effect rather than a genuine architecture comparison. confidence: high relevance: low - claim_id: transcript_quality_affects_downstream_tts_intelligibility role: supports claim: The choice of input transcript quality substantially affects downstream TTS intelligibility metrics independent of the TTS model itself, because more grammatically well-formed text is easier for TTS systems to synthesize accurately. source: §A.4, Table 8 evidence: Replacing ground-truth (disfluent, spontaneous) Switchboard transcripts with LLM-generated text responses as TTS input improves intelligibility across all tested TTS models, e.g. LJSpeech VITS WER drops from 16.1% to 6.0% under OWSM 3.1 hypotheses. confidence: high relevance: low - claim_id: dialogue_lm_sensitivity_to_asr_noise_varies_by_size role: complicates claim: Text dialogue response generators integrated into a cascaded spoken dialogue pipeline are not strongly degraded by upstream automatic speech recognition errors relative to using ground-truth transcripts, though smaller language models may be more sensitive to this noise than larger ones. source: §A.3, Table 7 evidence: When substituting OWSM 3.1 ASR transcripts for ground-truth transcripts as dialogue context, LLaMA-3.2-1B perplexity changes only slightly (48.2 to 47.5) while SmolLM v2-1.7B perplexity degrades substantially (113.7 to 285.1). confidence: medium relevance: low limitations: - The headline comparative analysis is conducted on a single dataset (11 hours of Switchboard Eval 2000) with only one example E2E system (Mini-Omni) and a 4-participant human pilot drawn from the author group; findings should be read as a toolkit demonstration rather than a generalizable benchmark. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.naacl-long.242 published_date: "2025-04-29" entry_date: '2026-07-26' year: 2025 venue: NAACL task: - TTS architecture: - diffusion - GAN - VAE - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - diffusion_voice_cloning claims: - claim_id: styletts_zs_compressed_prosody_enables_distillation role: supports claim: Compressing prosodic variation into a fixed-length discrete latent before diffusion modeling makes one-step distillation feasible with a small simulation budget, whereas bootstrapping-based distillation methods require significantly more samples or fail entirely on variable-length latents. source: §3.3, Appendix A.3, Table 6 evidence: The RVQ-quantized fixed-length (K=50) prosody latent is small enough that simulation-based one-step distillation succeeds with only 10k samples, achieving a 90% inference-time reduction with no statistically significant naturalness degradation. confidence: high relevance: low - claim_id: styletts_zs_codecfree_competitive_with_codec_systems role: supports claim: Codec-free end-to-end waveform synthesis achieves competitive or superior human-rated naturalness and similarity compared to codec-dependent systems, while adversarial training with multimodal discriminators compensates for the quantisation reconstruction gap. source: §3.1, §3.4, §4.2, Table 1 evidence: StyleTTS-ZS (LibriLight) achieves CMOS-N of 0.00, outperforming NaturalSpeech 3 (-0.28) and FlashSpeech (-0.42), without relying on any neural audio codec. confidence: high relevance: low - claim_id: styletts_zs_joint_prompt_text_encoding_helps role: supports claim: Jointly encoding the speech prompt and input text into prompt-aligned embeddings alongside a global style vector provides larger naturalness and similarity gains than using either conditioning alone in zero-shot TTS. source: §4.3, Table 3 evidence: Ablation shows removing prompt-aligned text embeddings (PATE) or the global style vector each independently drops CMOS-N (by 0.24 and 0.31 respectively), confirming both the timbre (global style) and content-alignment signals are separately necessary. confidence: high relevance: high - claim_id: styletts_zs_automatic_metrics_diverge_from_human role: complicates claim: Automatic speaker embedding similarity metrics can diverge from human-rated similarity in zero-shot TTS when codec-free systems are compared to codec-dependent ones, suggesting that codec representations may be tuned to neural-network perceptions rather than human perception. source: §4.2, Table 1 evidence: StyleTTS-ZS's automatic speaker embedding SIM (0.56) trails NaturalSpeech 3 (0.66), but human CMOS-S shows no significant difference between the two systems. confidence: medium relevance: high limitations: - The model is English-only (LibriTTS and LibriLight are English audiobook corpora). Speaker embedding SIM is somewhat lower than NaturalSpeech 3, suggesting the codec-free approach may leave some neural-network-measured timbre fidelity on the table. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.naacl-short.65 published_date: "2025-04-29" entry_date: '2026-07-26' year: 2025 venue: NAACL task: - TTS architecture: - hybrid - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - gan_based_voice_synthesis claims: - claim_id: ssl_feature_spaces_from_pre_trained_models_encode_cross_speaker_structur role: supports claim: SSL feature spaces from pre-trained models encode cross-speaker structure that enables zero-shot voice transfer through nearest-neighbor retrieval, without speaker-specific training data. source: §2.1, Table 1 evidence: kNN-TTS uses WavLM-Large layer 6 features, where frames from different speakers that are linearly close share phonetic information while preserving speaker identity; kNN retrieval over these features achieves SECS 0.72 and competitive MOS scores trained only on 24h of single-speaker LJSpeech data. confidence: high relevance: high - claim_id: zero_shot_multi_speaker_tts_competitive_with_large_multi_speaker_end_to role: supports claim: Zero-shot multi-speaker TTS competitive with large multi-speaker end-to-end systems can be achieved with single-speaker transcribed training data by delegating speaker identity to inference-time retrieval. source: §4, Table 1 evidence: GlowkNN-TTS (24h training, single speaker) achieves N-MOS and S-MOS within the confidence intervals of HierSpeech++ (2,796h, 7299 speakers) and XTTS (27,282h, multi-speaker) on LibriSpeech test-clean. confidence: high relevance: high - claim_id: retrieval_based_zero_shot_tts_requires_substantially_more_reference_audi role: complicates claim: Retrieval-based zero-shot TTS requires substantially more reference audio from the target speaker than embedding-based approaches to achieve sufficient quality. source: §Limitations, Figure 3b evidence: kNN-TTS requires approximately 30 seconds of target speaker audio for suitable intelligibility and around 1 minute for speaker similarity to plateau, whereas competing embedding-based systems show diminishing returns beyond 10-30 seconds of reference audio. confidence: medium relevance: high - claim_id: frame_level_knn_speaker_transfer_does_not_address_speaker_specific_durat role: complicates claim: Frame-level kNN speaker transfer does not address speaker-specific duration and rhythm, leaving prosodic timing patterns fixed to the training speaker. source: §Limitations "Rhythmic variations" evidence: In kNN-TTS, utterance duration is determined entirely by the single-speaker Text-to-SSL model; frame-by-frame retrieval substitutes voice quality but does not adapt speaking rate or rhythm to the target speaker. confidence: medium relevance: medium limitations: - See the source paper page for the complete limitations and evaluation scope. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.naacl-srw.6 published_date: "2025-04-29" entry_date: '2026-07-26' year: 2025 venue: NAACL task: - TTS - codec architecture: - autoregressive-LM - hybrid relevance: medium evidence_role: - infrastructure - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: non_overlapping_encoder_receptive_fields_in_rvq_codecs role: supports claim: Non-overlapping encoder receptive fields in RVQ codecs improve downstream language model likelihood and end-to-end TTS metrics relative to the standard causal overlapping setup. source: §3.1, Table 1 evidence: Replacing the overlapping causal encoder with a framewise encoder on a DAC-based codec reduces LM NLL by more than 8% and improves WER, NISQA, and speaker similarity on LibriTTS-R test-clean, despite slightly worsening Mel-L1 reconstruction. confidence: high relevance: high - claim_id: better_codec_audio_reconstruction_quality_does_not_reliably role: complicates claim: Better codec audio reconstruction quality does not reliably predict better end-to-end speech generation quality in codec-LM systems. source: §4, Table 1 evidence: The framewise encoder achieves higher NLL and better TTS metrics than the causal baseline while scoring slightly worse on Mel-spectral L1 reconstruction distance, demonstrating that reconstruction-optimised codecs can be suboptimal for downstream LM training. confidence: high relevance: low - claim_id: increasing_rvq_codec_frame_duration_can_substantially_reduce role: supports claim: Increasing RVQ codec frame duration can substantially reduce codec-LM inference latency with little or no degradation in TTS intelligibility and speaker similarity, provided the bitrate is held approximately constant by adjusting codebook depth. source: §3.3, §4, Table 2 evidence: Doubling frame duration from 11ms to 22ms yields a 1.94x inference speedup with WER 4.21%, NISQA 4.42, and speaker similarity 81.0%, matching or improving on the 11ms framewise baseline. Quadrupling to 44ms further accelerates inference (3.2-3.8x) but substantially degrades WER and speaker similarity. confidence: high relevance: high - claim_id: a_single_lm_trained_with_codebook_level_dropout role: supports claim: A single LM trained with codebook level dropout can efficiently approximate the performance profile of training one LM per candidate RVQ level count. source: §3.2, §4, Figure 2 evidence: Training with 90%-full CL drop on a 12-level codec produces per-level performance curves for WER, NISQA, and speaker similarity that closely track those of 12 independently trained LMs across all Q' values 1-12. confidence: high relevance: high - claim_id: the_optimal_number_of_rvq_codebook_levels_for role: refines claim: The optimal number of RVQ codebook levels for end-to-end codec-LM TTS differs across evaluation dimensions, and more levels are not universally better for end-to-end performance even when they monotonically improve codec reconstruction. source: Appendix B, Figure 3 evidence: End-to-end FAD reaches a global minimum at 9 levels before degrading, while WER reaches its best at 3-4 levels and NISQA and speaker similarity peak at approximately 9 levels, in contrast to codec Mel-L1 which improves monotonically. confidence: high relevance: high limitations: - The evaluation is conducted on a single English TTS corpus (LibriTTS-R test-clean) using automatic metrics only (WER via Whisper, NISQA, cosine speaker similarity). No human listening test is reported, so perceptual quality gains are estimated rather than directly validated. - The codec is trained on proprietary in-house podcast data (1.7K hours), which limits reproducibility for the codec training stage specifically. The LM training does use the public LibriTTS-R dataset. The codebook size hyperparameter (|V|) remains outside the scope of CL drop, requiring separate trial-and-error search. The paper does not investigate multilingual or noisy speech settings. The optimal frame duration finding (22ms being the sweet spot) is specific to this codec architecture and training data and may not generalise to other codec families. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2505.07916' published_date: "2025-05-12" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: jointly_training_a_speaker_encoder_with_an_autoregressive role: supports claim: Jointly training a speaker encoder with an autoregressive TTS model yields better intelligibility and competitive speaker similarity compared to using a fixed speaker verification embedding. source: §3.6, Table 4 evidence: Ablation on speaker conditioning (§3.6, Table 4) confirms the learnable speaker encoder outperforms both fixed speaker embedding (pre-trained speaker verification model) and prompt-only conditioning in WER-SIM balance. confidence: high relevance: high - claim_id: zero_shot_voice_cloning_conditioned_on_untranscribed_reference role: complicates claim: Zero-shot voice cloning conditioned on untranscribed reference audio produces lower word error rates than one-shot conditioning with a paired text-audio exemplar, at the cost of slightly reduced speaker similarity. source: §3.2, Table 1; §3.5, Table 3 evidence: 'The system supports two voice cloning paradigms: zero-shot (conditioning only on the speaker encoder''s output from an untranscribed reference) and one-shot (additionally providing a paired text-audio exemplar as an in-context prompt).' confidence: high relevance: high - claim_id: augmenting_a_vae_with_a_normalizing_flow_on role: supports claim: Augmenting a VAE with a normalizing flow on the latent space (Flow-VAE) improves both waveform reconstruction quality and downstream TTS speaker similarity over a standard VAE with the same architecture. source: §3.7, Tables 5–6 evidence: The Flow-VAE extends a standard VAE by inserting a normalizing flow between the encoder's posterior distribution and the standard normal prior, enabling a more expressive latent space. confidence: high relevance: high - claim_id: speaker_encoder_representations_trained_without_text_dependencies_support role: supports claim: Speaker encoder representations trained without text dependencies support cross-lingual synthesis with high intelligibility, outperforming prompt-based systems that require transcribed reference audio in cross-lingual scenarios. source: §3.5, Table 3 evidence: Its core innovation is the jointly-trained speaker encoder, which takes a variable-length reference audio segment (without transcription) and compresses it to a fixed-size conditional vector. confidence: high relevance: high - claim_id: disentangled_speaker_embeddings_from_a_task_specific_encoder role: supports claim: Disentangled speaker embeddings from a task-specific encoder enable parameter-efficient per-speaker adaptation by fine-tuning only the speaker embedding vector, preserving base model generalization. source: §4.3 evidence: 'Three downstream extensions are documented: LoRA-based emotion control (separate LoRA modules per emotion category loaded at inference), text-to-voice generation (a compact model predicts speaker embeddings from natural language descriptions and discrete attribute tags), and professional voice cloning (PVC), which fine-tunes only the speaker embedding vector for a target speaker while keeping the AR Transformer frozen.' confidence: high relevance: medium limitations: - The training dataset is proprietary and the model size is not reported, which makes independent replication impossible. All results are from internal or public leaderboard evaluations only; no code release is indicated. - The leaderboard evaluation (Artificial Analysis Arena) uses preference judgements that may be influenced by sample selection; the Arena snapshot is from a single date (May 12, 2025) and rankings can shift as more models are added. The claim of "top position" is time-bound. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2505.13000' published_date: "2025-05-19" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - codec - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - infrastructure - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: directly_encoding_ssl_features_into_the_first_rvq role: supports claim: Directly encoding SSL features into the first RVQ layer preserves significantly more semantic content than distilling SSL representations into codec tokens, particularly for tonal languages where pitch information is phonemically critical. source: §4.2, Table 2 evidence: DualCodec addresses the three-way tension through a dual-stream architecture that directly assigns SSL features to the first RVQ layer rather than distilling into it. confidence: high relevance: low - claim_id: neural_audio_codecs_operating_at_lower_frame_rates role: supports claim: Neural audio codecs operating at lower frame rates with more quantization layers achieve superior audio quality at equivalent bitrates compared to higher frame rate codecs with fewer quantization layers. source: §4.3, Table 3 evidence: Dual encoding adds perceptual quality gains over plain DAC at the same frame rate (MUSHRA 79.5 vs 68.8 at 25Hz). confidence: high relevance: low - claim_id: semantic_enhancement_of_the_first_rvq_layer_improves role: supports claim: Semantic enhancement of the first RVQ layer improves downstream TTS speaker similarity as well as intelligibility, because higher semantic fidelity in RVQ-1 enables the waveform stream's quantizers to focus on acoustic detail rather than recovering content information. source: §4.4, Table 4 evidence: This residual formulation concentrates acoustic detail (timbre, noise, fine structure) in the waveform stream's quantizers, while the SSL stream is responsible solely for content and prosody. confidence: high relevance: high - claim_id: the_quality_gap_between_distillation_based_and_direct role: complicates claim: The quality gap between distillation-based and direct-encoding semantic codecs is substantially larger in Mandarin than in English, revealing a systematic limitation of distillation approaches for tonal languages. source: §4.2, Table 2 evidence: The same setup with distillation (C2) gives 17.8/14.4%, confirming that direct encoding from SSL features substantially outperforms distillation. confidence: high relevance: low limitations: - The 12.5Hz DualCodec-based TTS systems consistently underperform their 25Hz counterparts on both WER and speaker similarity (Table 4, Table 6), indicating that the quality upper bound of the 12.5Hz variant is not yet competitive with the best open-source systems at 50Hz despite the frame rate reduction improving inference speed. - The paper evaluates TTS only on Seed-TTS-Eval; no subjective TTS listening tests are reported, so the MUSHRA gains in codec reconstruction may not fully translate to perceived TTS naturalness. Speaker similarity scores with DualCodec-VALLE remain below those of MaskGCT baselines that use separate semantic and acoustic tokenizers, suggesting the unified approach has not yet matched the best-performing two-stage pipeline design. The DualCodec encoder is substantially heavier than baselines (628M vs 38M for Mimi) due to the frozen w2v-BERT-2.0 model, increasing training-time compute, though the decoder remains lightweight for inference. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2505.17589' published_date: "2025-05-23" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: differentiable_reward_optimization_over_discrete_speech_tokens_rather role: supports claim: Differentiable reward optimization over discrete speech tokens, rather than synthesized audio, enables scalable post-training for LLM-based TTS without the computational overhead of full audio generation. source: §2.2 evidence: The post-training method, Differentiable Reward Optimization (DiffRO), bypasses full audio synthesis by operating directly on discrete speech tokens. confidence: high relevance: low - claim_id: supervised_multi_task_training_of_a_speech_tokenizer role: supports claim: Supervised multi-task training of a speech tokenizer on tasks including emotion recognition, language identification, and speaker analysis captures richer paralinguistic content than ASR-only objectives, with measurable downstream gains in content consistency and speaker similarity. source: §2.1, §5.3 evidence: A Finite Scalar Quantization (FSQ) module is inserted into MinMo's voice encoder, and the entire tokenizer is then fine-tuned via supervised multi-task learning on 530K hours covering ASR, language identification (LID), speech emotion recognition (SER), audio event detection (AED), and speaker analysis. confidence: high relevance: high - claim_id: current_zero_shot_tts_systems_infer_emotional_tone role: complicates claim: Current zero-shot TTS systems infer emotional tone primarily from text semantics; expression of text-unrelated emotions from a reference prompt remains a significant unsolved challenge. source: §5.2.3, Table 9 evidence: 'Emotion cloning results are more mixed: text-related emotion accuracy is strong (0.84–0.98 for happy/angry/sad with DiffRO-EMO), but text-unrelated emotion accuracy drops substantially for sad and angry categories, revealing that current models infer emotional tone primarily from text semantics rather than from the reference prompt.' confidence: high relevance: high - claim_id: scaling_training_data_and_model_size_in_two role: supports claim: Scaling training data and model size in two-stage codec TTS produces consistent gains up to a point of diminishing returns, with data scaling showing stronger marginal returns than model size scaling beyond 0.5B parameters on current benchmarks. source: §4.2, §5.1 evidence: The only model that exceeds CosyVoice 3's speaker similarity in this evaluation is Seed-TTS, which the authors attribute to larger speaker diversity in Seed-TTS pretraining data. confidence: high relevance: high - claim_id: existing_tts_benchmarks_based_on_studio_quality_audiobook role: supports claim: Existing TTS benchmarks based on studio-quality audiobook data underestimate the difficulty of in-the-wild voice cloning, where noisy reference audio and domain diversity expose robustness gaps not visible in standard evaluations. source: §4.4 evidence: The only model that exceeds CosyVoice 3's speaker similarity in this evaluation is Seed-TTS, which the authors attribute to larger speaker diversity in Seed-TTS pretraining data. confidence: high relevance: high limitations: - Speaker similarity lags Seed-TTS across all SEED-TTS-Eval conditions, and the authors attribute this to data volume differences rather than architectural factors. Since pretraining data is not released, the degree to which CosyVoice 3's improvements are reproducible without Alibaba's internal 1M-hour corpus is unclear. - 'The DiffRO reward mechanism exhibits a trade-off between intelligibility and speaker similarity: RL post-training slightly reduces speaker similarity while improving WER. Multi-reward DiffRO with emotion targets further degrades pronunciation, suggesting that reward balancing in multi-objective post-training for TTS is an open engineering problem. Timbre control via textual instructions is absent and noted as future work. Singing voice generation remains unsatisfactory due to insufficient training data coverage in the tokenizer and LM stages. Performance on Korean and Japanese lags other languages due to data volume limitations and language-specific encoding challenges (kana conversion, multiple pronunciations).' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2506.13053' published_date: "2025-06-16" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - nonautoregressive_flow_matching claims: - claim_id: zipvoice_compact_matches_larger_models role: supports claim: Compact flow-matching TTS models can match the speech quality of models two to eight times larger when architectural components are selected for per-parameter efficiency rather than chosen by default from the diffusion literature. source: §V.A, Table I evidence: ZipVoice (123M params) matches or exceeds the speech quality of F5-TTS (336M) and other baselines 2-8x larger by repurposing the Zipformer ASR encoder as the flow-matching vector field estimator backbone. confidence: high relevance: medium - claim_id: zipvoice_uniform_upsampling_beats_filler_padding role: supports claim: In non-autoregressive TTS without explicit duration prediction, a simple uniform-duration upsampling assumption provides substantially better initial alignment than filler-token padding, yielding large intelligibility improvements without additional parameters. source: §II.D, §V.C, Table IV evidence: Removing average upsampling (reverting to filler-token-only padding, the E2-TTS approach) raises WER from 1.69% to 20.19% on LibriSpeech-PC test-clean, with the parameter-free uniform-duration heuristic matching or exceeding F5-TTS's learned ConvNeXt refinement. confidence: high relevance: low - claim_id: zipvoice_cfg_distillation_eliminates_double_pass role: supports claim: Flow distillation that conditions the student on CFG strength eliminates the doubled forward-pass cost of classifier-free guidance while preserving its quality benefit, outperforming consistency distillation and ReFlow at 4 NFEs. source: §II.E, §V.E, Table VI evidence: ZipVoice-Distill, conditioned on CFG strength via a Fourier embedding, internalizes CFG behavior to bypass the doubled evaluation cost at runtime, outperforming consistency distillation and ReFlow baselines at 4 NFEs in the paper's distillation-method comparison. confidence: high relevance: medium - claim_id: zipvoice_zipformer_inductive_bias_transfers role: supports claim: The U-Net-style multi-resolution structure and convolutional modules in Zipformer transfer the inductive biases beneficial for diffusion-family models from ASR into TTS vector field estimation, with ablation evidence showing each structural element independently contributes to intelligibility and naturalness. source: §II.C, §V.D, Table V evidence: Component ablation confirms the U-Net-style structure, convolutional modules, and attention-weight reuse mechanism each independently improve intelligibility and naturalness metrics relative to a standard Transformer flow-matching backbone. confidence: high relevance: low limitations: - Speaker similarity (SIM-o) for ZipVoice-Distill is consistently below F5-TTS and larger NAR baselines despite quality and intelligibility advantages, and the cause is not mechanistically isolated. - Average upsampling assumes uniform token durations within a sentence, a poor model of natural prosody that may limit expressiveness for prosodically varied speech. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.02176' published_date: "2025-07-02" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - evaluation architecture: [] relevance: medium evidence_role: - evaluation_caution current_role: active_evidence method_family: [] claims: - claim_id: asv_embeddings_miss_dynamic_identity_markers role: supports claim: ASV embeddings encode static anatomical speech features but systematically fail to represent dynamic behavioral identity markers such as rhythm and timing patterns. source: §3.2, Figure 1 evidence: Lasso regression predicting handcrafted features from seven ASV models' embeddings shows high r² for mean pitch, HNR, shimmer, and α-ratio, but near-zero r² for speech rate, voiced/unvoiced segment lengths, and pitch standard deviation. confidence: high relevance: medium - claim_id: eer_confounded_by_non_identity_factors role: complicates claim: EER-based speaker similarity measurements in speech synthesis evaluation are susceptible to confounding factors unrelated to voice identity, which can invalidate comparisons between synthesis systems. source: §3.3, Table 2 evidence: Duration-sorting same-speaker utterances depresses EER from an expected 50% to 30-39% across all ASV models tested; SNR 20dB noise reduces EER to 15-38%; equalization mismatches cause near-total failure in GE2E, all correctable via re-equalization and duration matching. confidence: high relevance: high - claim_id: rhythm_identity_requires_phoneme_duration_distributions role: supports claim: Characterizing speaker rhythm for identity assessment requires modeling phoneme-duration distributions rather than aggregate measures such as mean speech rate. source: §3.4, Figure 2, Table 3 evidence: U3D Wasserstein distances clearly separate same-speaker pairs (avg. 2.15) from nearest-by-speech-rate pairs (18.40) and random pairs (21.53), while many L2-ARCTIC speakers sharing similar syllable rates show markedly different voiced-segment duration patterns. confidence: high relevance: medium - claim_id: ssl_units_substitute_for_phoneme_labels_in_rhythm_analysis role: refines claim: Self-supervised speech unit representations serve as language-agnostic substitutes for phoneme labels in rhythm analysis, avoiding the need for forced alignment. source: §3.4, Table 3 evidence: U3D using HuBERT-derived unsupervised clusters achieves Wasserstein distance separation between speaker conditions (2.15 same, 18.40 nearest, 21.53 random) nearly identical to forced-alignment-based phoneme rhythm analysis (2.48, 18.37, 24.43). confidence: medium relevance: medium limitations: - U3D is validated as a discriminative metric (separating speaker pairs) but has not been validated against human perceptual judgments of rhythm similarity. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.02380' published_date: "2025-07-03" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - SCA - TTS architecture: - autoregressive-LM - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: routing_llm_hidden_states_into_the_tts_module role: supports claim: Routing LLM hidden states into the TTS module's embedding space can enable voice cloning in an end-to-end spoken chatbot without a dedicated speaker encoder. source: §1.2 evidence: JoyTTS projects Qwen-7B hidden states via an MLP into the CosyVoice2-based LLM-TTS embedding, achieving SS of 0.73 on seed-tts-zh using prompt audio as the only speaker reference. confidence: high relevance: high - claim_id: jointly_training_an_llm_chat_module_with_a role: complicates claim: Jointly training an LLM-Chat module with a TTS module in a chatbot pipeline can degrade intelligibility relative to running the TTS component standalone, even when speaker similarity improves. source: §4, Table 1 evidence: JoyTTS achieves WER of 5.09 compared to 1.45 for standalone CosyVoice2 on seed-tts-zh, despite closing the speaker similarity gap (JoyTTS SS 0.73 vs. CosyVoice2 SS 0.748). confidence: high relevance: high - claim_id: end_to_end_spoken_chatbot_systems_pairing_a role: supports claim: End-to-end spoken chatbot systems pairing a 7B-parameter LLM with an autoregressive TTS module can achieve sub-2-second response latency on a single consumer GPU without specialised inference optimisations. source: §4 evidence: JoyTTS reports 1.8-second end-to-end latency on a single NVIDIA 4090D with no engineering optimisation applied. confidence: high relevance: low limitations: - The evaluation is limited to a single Chinese benchmark (seed-tts-zh), leaving performance on English, multilingual, or spontaneous conversational speech uncharacterised. The WER gap between JoyTTS (5.09) and standalone CosyVoice2 (1.45) is large and unexplained; the paper does not ablate whether the regression originates from the joint training procedure, the conversational data distribution, or the MLP projection coupling. No subjective listening tests are reported, making it impossible to assess naturalness or perceived quality beyond intelligibility and speaker similarity metrics. Emotion control, identified as a target for future work, is not implemented in the current system. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.03887' published_date: "2025-07-05" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: joint_training_improves_ood_attribution role: supports claim: Joint training of a TTS generator and a speech discriminator with aligned optimization objectives improves out-of-domain generalization of model-specific attribution compared to training the discriminator independently. source: §5.4, Table 3 evidence: Jointly training F5-TTS and a wav2vec 2.0 + LCNN discriminator improves out-of-domain AUC from 0.8823 to 0.9421 and EER from 18.99% to 11.50% relative to independent discriminator training. confidence: high relevance: medium - claim_id: watermark_free_traceability_preserves_quality role: supports claim: Watermark-free TTS traceability through joint training can preserve or slightly improve synthesis quality relative to the base model. source: §5.6, Table 5 evidence: The jointly trained F5-TTS achieves WER 2.033% (vs. 2.202% baseline), UTMOS 3.958 (vs. 3.926), and speaker similarity 0.661 (vs. 0.659) on LibriSpeech-PC test-clean. confidence: medium relevance: high - claim_id: attribution_vulnerable_to_noise_and_pitch_manipulation role: complicates claim: TTS attribution discriminators based on audio feature patterns are vulnerable to additive noise and pitch manipulation, but robust to common audio processing operations. source: §5.5, Table 4 evidence: Out-of-domain AUC drops by 0.1233 under additive noise (MUSAN) and 0.0901 under pitch shifting, while remaining stable within 0.025 AUC under resampling, time-stretching, reverb, volume scaling, and MP3 compression. confidence: medium relevance: low - claim_id: differentiability_required_for_watermark_free_traceability role: complicates claim: Differentiability of the full synthesis pipeline from text input to output waveform is a necessary precondition for watermark-free traceability via joint generator-discriminator training, excluding discrete token-based TTS architectures. source: §6 evidence: The paper explicitly notes VALL-E-type models using non-differentiable discrete token generation are incompatible with this framework because the discriminator's loss cannot backpropagate through the discrete generation step. confidence: high relevance: low limitations: - The method is evaluated on a single TTS backbone (F5-TTS) with a single discriminator architecture, and the out-of-domain test uses only three other TTS systems. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.04349' published_date: "2025-07-06" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: ttsctrlnet_frozen_adapter_less_data role: supports claim: Frozen-backbone adapter training can add fine-grained, time-varying conditioning to pretrained flow-matching TTS with substantially less data than full fine-tuning. source: §1, §4.4, Table 5 evidence: ControlNet blocks trained on ~400 hours of public emotion data achieve higher Emo-SIM and Aro-Val SIM than EmoCtrl-TTS, which requires 87k hours including 27k hours of in-house emotion data, while preserving the frozen backbone's zero-shot voice cloning capability. confidence: high relevance: high - claim_id: ttsctrlnet_emotion_intelligibility_tradeoff role: complicates claim: Stronger emotion conditioning in flow-matching TTS introduces a trade-off with text intelligibility. source: §4.3.4, Table 4 evidence: Increasing the ControlNet control scale lambda from 0 to 1 consistently improves emotion similarity metrics but raises WER from 2.9% to 9.58% on the EMO-Change benchmark. confidence: high relevance: low - claim_id: ttsctrlnet_dit_blocks_unequal_contribution role: supports claim: Transformer blocks in DiT-based TTS models contribute unequally to speaker identity and text intelligibility, and block selection is important for conditional control. source: §4.3.1, Figure 2, Table 3 evidence: Layer-wise skip analysis on F5-TTS shows removing specific blocks dramatically increases WER and reduces speaker similarity; excluding those critical blocks from ControlNet connections yields WER 0% versus 8.9% WER when all blocks are connected. confidence: high relevance: high - claim_id: ttsctrlnet_emotion_concentrated_early_flow_steps role: supports claim: Emotion in a flow-matching trajectory is concentrated at early denoising steps, and restricting conditioning to this interval improves both efficiency and intelligibility. source: §4.3.2, Table 1, Table 8 evidence: Training with flow step interval [0, 0.1] achieves Emo-SIM 0.565 and 1.9% WER, whereas training on the full [0, 1] interval degrades to Emo-SIM 0.389 with 0% WER; restricting ControlNet to early steps also reduces per-sample inference time from 5.4s to 4.2s. confidence: high relevance: low - claim_id: ttsctrlnet_ser_needs_temporal_smoothing role: complicates claim: Frame-level emotion features from self-supervised SER models require temporal smoothing to serve as effective conditioning signals for TTS. source: §4.3.3, Table 2 evidence: Using emotion window size of 1 (no smoothing) produces Emo-SIM 0.500 and WER 4.7%; a window size of 30 achieves Emo-SIM 0.565 and WER 1.9%, showing token-level SER features without pooling lose emotional coherence. confidence: high relevance: low limitations: - Most baseline comparisons use values reported in a prior paper (EmoCtrl-TTS) rather than independently reproduced, limiting reliability of direct numerical comparisons. - The underlying SER model cannot reliably recognize non-verbal vocalizations (laughing, crying), restricting expressiveness relative to systems with dedicated non-verbal encoders. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.01348' published_date: "2025-07-08" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - VC - TTS architecture: - autoregressive-LM - VAE - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: ctc_regularization_applied_before_vector_quantization_improves_the role: supports claim: CTC regularization applied before vector quantization improves the temporal locality and temporal robustness of discrete speech content tokens. source: §5.2, Table 2 evidence: SpeechCodeVAE achieves 59% higher De-duplication Efficiency and approximately 9 times better Speed Robustness than CosyVoice-50Hz; ablation without CTC loss collapses Speed Robustness from 0.219 to 0.009, identifying CTC as the critical factor. confidence: high relevance: medium - claim_id: multi_task_learning_with_tts_as_an_auxiliary role: supports claim: Multi-task learning with TTS as an auxiliary objective compensates for data scarcity in foreign accent conversion, improving both convergence and output quality. source: §3, §5.1, Table 1 evidence: Joint FAC+TTS training on 370 hours of TTS data alongside 9.3 hours of FAC data yields 25% accentedness reduction and WER improvement from 14.4% to 9.1% relative to a standalone FAC baseline. confidence: high relevance: medium - claim_id: bert_style_masked_token_restoration_can_correct_stochastic role: supports claim: BERT-style masked token restoration can correct stochastic local substitution errors introduced by autoregressive speech token decoding, improving acoustic continuity. source: §5.3, Table 4 evidence: Removing SpeechRestorer decreases TTS CMOS from 3.850 to 3.629, with the paper attributing the gain to error-correction of spurious token substitutions that cause acoustic discontinuities. confidence: high relevance: low - claim_id: token_level_post_processing_for_llm_speech_generation role: complicates claim: Token-level post-processing for LLM speech generation can correct local substitution errors but fails to address higher-level failure modes such as word skipping and repetition. source: §6 evidence: The paper explicitly states that SpeechRestorer cannot fix skipped words or repetitions, as these require sequence-level rather than token-level correction. confidence: high relevance: low - claim_id: training_data_scale_rather_than_architectural_design_is role: refines claim: Training data scale, rather than architectural design, is the primary driver of quality gaps between LLM-based TTS systems at different performance levels. source: §5.3, Table 4 evidence: SpeechAccentLLM trails NaturalSpeech2 in TTS naturalness (CMOS 3.850 vs. 3.944) despite comparable architecture; the gap is attributed to NS2 training on approximately two orders of magnitude more data. confidence: high relevance: medium limitations: - The FAC evaluation is restricted to four L1 backgrounds from L2-ARCTIC and uses a single TTS model (LJSpeech-trained VITS) to generate native accent counterparts. Generalisation to other accents, speaking styles, or higher-quality native reference speech is untested. - Prosody modelling is acknowledged as incomplete; the Variance Adapter models f0 only and does not capture rhythm, duration patterns, or prosodic phrasing beyond pitch. Timbre reconstruction quality is bounded by the frozen ECAPA-TDNN speaker encoder, which was not trained for the L2/accented-speech domain. SpeechRestorer cannot resolve sequence-level decoding failures (word skipping, repetition), leaving a category of LLM-generated errors unaddressed. The paper does not report total parameter counts for any module, which complicates direct comparison with other systems. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2506.09874' published_date: "2025-07-10" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: joint_generation_beats_post_hoc_mixing role: supports claim: Joint generative modelling of speech and environmental audio produces more naturally integrated output than post-hoc mixing of separately generated speech with background sounds. source: §4.2 evidence: A/B tests with 20+ raters per sample showed 87.89% preference for UmbraTTS over F5-TTS plus post-hoc background mixing across 30 AudioSet samples. confidence: high relevance: low - claim_id: self_supervised_extraction_sufficient_for_environmental_tts role: supports claim: Self-supervised extraction of speech and background audio from unlabeled mixed recordings is sufficient to train environmentally conditioned TTS to state-of-the-art performance. source: §3, §4.1, Table 1 evidence: UmbraTTS uses VAD-based silence extraction and learned source separation applied to AudioSet to construct training triplets without manual annotation, achieving best WER and FAD among all baselines on AudioCaps. confidence: medium relevance: low - claim_id: flow_matching_beats_diffusion_for_environmental_tts role: supports claim: Flow matching outperforms diffusion-based architectures in environmentally aware speech synthesis on both automatic and human evaluation metrics. source: §4.1, §4.2, Tables 1-3 evidence: UmbraTTS (CFM-based) achieves WER 6.89% and FAD 4.14 versus VoiceDiT (diffusion, 7.09%, 4.60) and VoiceLDM (diffusion, 10.39%, 5.56) on AudioCaps, with 78.54-81.91% human preference over VoiceLDM. confidence: medium relevance: medium - claim_id: explicit_ser_control_needed_for_background_balance role: complicates claim: Controlling speech-to-environment balance in jointly generated audio scenes requires an explicit scalar conditioning parameter; standard text and speaker conditioning are insufficient for this degree of output control. source: §3, §4.2 evidence: UmbraTTS introduces a dedicated SER signal injected via adaLN-zero; a separate controlled listening test confirms 96.6% of participants correctly distinguish the intended background volume direction in pairwise comparisons. confidence: medium relevance: medium limitations: - The evaluation protocol is borrowed from VoiceLDM and covers only the AudioCaps and MusicCaps test subsets, so generalisation to other environmental conditions is untested. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.07799' published_date: "2025-07-10" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: instruction_tts_enables_anonymization_without_source_audio role: supports claim: Instruction-conditioned TTS enables speaker anonymization without relying on source speaker audio or speaker embeddings. source: §3.3, §3.4.1, Table 1 evidence: Parler-TTS with natural language speaker descriptions achieves FAR=0% on SLUE-VoxPopuli, matching zero-shot TTS baselines (VALL-E, XTTS-v2) that require reference audio. confidence: high relevance: high - claim_id: far_unreliable_under_disjoint_speaker_evaluation_design role: complicates claim: False acceptance rate is an unreliable discriminator of speaker anonymization quality when evaluation design guarantees disjoint training and test speakers. source: §3.4.1 evidence: FAR=0% is achieved by every TTS-based system regardless of architecture or conditioning, because TTS models are trained on speakers absent from the evaluation set; the metric cannot distinguish the proposed method from any TTS-based pipeline in this setting. confidence: high relevance: medium - claim_id: asr_ner_pipeline_content_privacy_limited_by_errors role: complicates claim: ASR + NER pipeline-based content privacy is limited by recognition and detection errors that allow a meaningful share of sensitive content to pass through. source: §3.4.1, §4 evidence: The ASR component achieves 19.00% WER on original speech and DeBERTa-L NER achieves only 71.80% F1 on predicted transcriptions, meaning many named entities are missed at the detection stage. confidence: medium relevance: low - claim_id: prompt_based_tts_variability_limits_identity_consistency role: complicates claim: Generative variability in prompt-based TTS limits consistent voice identity reproduction across utterances sharing the same speaker description. source: §4 evidence: Parler-TTS outputs may sound like different persons across generations even with an identical speaker description, undermining multi-utterance anonymization coherence. confidence: low relevance: medium - claim_id: speaker_attribute_choice_affects_intelligibility role: supports claim: Speaker attribute choice in instruction-conditioned TTS introduces measurable variation in downstream speech intelligibility that practitioners must account for. source: §3.4.2, Tables 2 and 3 evidence: WER varies from 12.07% (Slovak accent) to 23.76% (Italian accent) across 38 accent descriptions, and from 12.39% (normal rate) to 18.56% (very fast) across speaking rate settings, while speaker privacy (FAR=0%) remains constant. confidence: medium relevance: medium limitations: - FAR=0% is observed for all TTS-based systems including both baselines, because the evaluation speakers are disjoint from TTS training speakers; the privacy metric cannot distinguish the proposed approach from any TTS pipeline in this experimental design. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.08319' published_date: "2025-07-11" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: minor method_family: - encoder_decoder_voice_cloning claims: - claim_id: active_learning_more_data_efficient_than_static_selection role: supports claim: Active learning with model-informed informativeness criteria produces more data-efficient TTS corpora than model-agnostic static data selection at equal corpus size. source: §III.B, Table I evidence: The proposed method achieves 71.0% synthesizable speaker ratio at 3,943 utterances versus 66.6% for the static baseline of the same size, measured by pseudo-MOS exceedance rate. confidence: high relevance: medium - claim_id: informativeness_criterion_trades_off_top_quality_speakers role: complicates claim: Data selection methods that optimize synthesizable speaker coverage may reduce synthesis quality for speakers already near the quality threshold. source: §III.B, Figure 3 evidence: The proposed method underperforms the static baseline for speakers with pseudo-MOS above 2.6, because the informativeness criterion explicitly excludes speakers the current model can already synthesize well. confidence: medium relevance: medium - claim_id: quality_estimator_effective_on_small_training_fraction role: supports claim: TTS corpus quality estimators remain effective when trained on a small fraction of candidate data rather than requiring full candidate set pre-processing. source: §III.B, Figure 7 evidence: The data quality estimator trained on 10% of the YouTube candidate pool shows a strong correlation with the estimator trained on 100% of the data, with no significant change in selection behavior. confidence: medium relevance: low - claim_id: diffusion_better_captures_speaker_embedding_distributions role: supports claim: Diffusion models better capture complex speaker embedding distributions than Gaussian mixture models for speaker generation tasks with limited training samples. source: §III.B, Figure 5 evidence: The diffusion model achieves Wasserstein distance to held-out test speakers that is more than twice the standard deviation lower than GMMs across all cluster sizes (M=1..10). confidence: medium relevance: medium limitations: - The evaluation uses pseudo-MOS (an automatic quality predictor) rather than subjective listening tests, so the absolute synthesis quality of the final corpora is not independently verified. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.09282' published_date: "2025-07-12" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - diffusion - VAE relevance: low evidence_role: - historical_context current_role: minor method_family: - hybrid_ar_diffusion_acoustic claims: - claim_id: zero_shot_tts_normalisation_filters_atypical_speech role: supports claim: The normalising tendency of zero-shot TTS models can be deliberately exploited to filter atypical speech characteristics while preserving general voice timbre. source: §2.2, §4, Table 1 evidence: ClaritySpeech applies XTTSv2 to dementia-affected speech without fine-tuning, reducing adversarial dementia detection F1 by 16% on ADReSS while maintaining speaker similarity of 0.50, exploiting the model's tendency to smooth prosodic irregularities learned from diverse fluent training data. confidence: medium relevance: high - claim_id: speaker_identity_naturalness_tradeoff_zero_shot_tts role: complicates claim: Speaker identity preservation and speech naturalness trade off against each other when zero-shot TTS is applied to atypical speech. source: §4, Table 1 evidence: StyleTTS2 achieves the best naturalness (UTMOS 2.86) and lowest WER (0.04) on ADReSS but the worst speaker similarity (0.36) among three TTS backends, while XTTSv2 achieves a more balanced UTMOS 2.15 and speaker similarity 0.50, showing that more aggressive normalisation sacrifices voice fidelity. confidence: medium relevance: high - claim_id: text_acoustic_obfuscation_complementary role: supports claim: Text-level content correction and acoustic re-synthesis provide complementary privacy protection that neither provides alone in disordered speech obfuscation. source: §4, Table 2 evidence: Ablation on ADReSS shows removing text obfuscation raises text-modality detection F1 by 10 points (0.59 to 0.68), while removing zero-shot TTS collapses speaker similarity to 0.12, showing the full pipeline needs both components. confidence: medium relevance: high - claim_id: pipeline_latency_precludes_realtime_obfuscation role: complicates claim: Pipeline-based speech obfuscation systems that chain ASR, text processing, and TTS carry latency that precludes real-time deployment on standard hardware. source: §4, Table 4 evidence: ClaritySpeech achieves a mean real-time factor of 6.07 on CPU for 3-6 second audio samples, with ASR alone contributing RTF 3.01 and TTS contributing RTF 1.71. confidence: high relevance: low limitations: - 'Evaluation uses small, low-diversity datasets (ADReSS: 690 samples, ADReSSo: 947 samples) from a single cookie-theft description task; generalisability to other dementia types, languages, or conversational speech is untested.' caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2506.18296' published_date: "2025-07-15" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - VC - evaluation architecture: [] relevance: medium evidence_role: - infrastructure current_role: active_evidence method_family: [] claims: - claim_id: nonanonymous_corpora_enable_rigorous_speaker_similarity_eval role: supports claim: Non-anonymous speaker corpora with publicly identifiable voices enable more rigorous subjective evaluations of speaker similarity in TTS and VC systems. source: §1, §3.1 evidence: JIS assigns stage names to 169 Japanese live idol speakers, allowing experiment designers to recruit listeners familiar with the speakers, enabling discrimination of subtle voice characteristics that anonymous corpus evaluations cannot capture. confidence: high relevance: high - claim_id: automatic_mos_underestimates_nonprofessional_quality role: complicates claim: Automatic MOS predictors trained on TTS-generated speech may underestimate audio quality when applied to recordings of non-professional speakers, even under studio conditions. source: §4.2.1 evidence: JIS Speech A (studio-recorded) achieves a mean UTMOS of 3.4 compared to JVS parallel100's 3.7, with the gap attributed to linguistic mismatch in UTMOS training data and speech hesitations inherent to non-professional speakers rather than recording quality differences. confidence: medium relevance: medium - claim_id: speaking_style_shifts_speaker_embeddings role: supports claim: Speaking style and communicative context introduce systematic variation in speaker embeddings that interacts with speaker identity, presenting a challenge for style-robust speaker representation. source: §4.2.2 evidence: ECAPA-TDNN embeddings of JIS speakers show that specific speaking styles (energetic post-performance greetings, intimate photo-session speech) produce cross-speaker clusters in t-SNE, partially overriding individual speaker identity, while speech expressing personal individuality is more dispersed. confidence: medium relevance: high limitations: - The corpus is small (17 hours, 169 speakers) and its non-anonymous design is restricted to non-commercial basic research, which is a general corpus limitation rather than one specific to its evaluation-methodology contribution. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.10985' published_date: "2025-07-15" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: [] relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: acoustic_deviation_correlates_mispronunciation role: supports claim: Acoustic deviation between a learner utterance and a personalized voice-cloned reference correlates with mispronunciation at the word level. source: §4, Table 2 evidence: MFCC-DTW distances are consistently higher for mispronounced words than correctly pronounced words across all four L2-ARCTIC speakers tested (mean deviation of 0.027-0.028 units on average), confirming the core deviation hypothesis. confidence: medium relevance: medium - claim_id: deviation_metric_high_speaker_variability role: complicates claim: Acoustic deviation metrics derived from voice cloning exhibit high per-speaker variability, making universal calibration insufficient for reliable mispronunciation detection. source: §4, Table 1 evidence: Classification accuracy ranges from 0.461 to 0.646 across four speakers; the authors note that per-individual models would likely improve results. confidence: medium relevance: high - claim_id: ambiguity_class_needed_for_threshold_decisions role: complicates claim: Mispronunciation detection systems based on acoustic distance alone require an explicit ambiguity class when threshold-based decision rules cannot confidently separate correct from incorrect pronunciations. source: §3.6 evidence: The system defines an AMBIGUOUS output for samples falling between class-specific KDE thresholds, but the paper does not report how frequently this category occurs in practice. confidence: low relevance: low limitations: - No comparison against established CAPT baselines (e.g., GOP scoring) is provided, so it is unclear whether the proposed MFCC-DTW deviation score is competitive as an evaluation methodology. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.12197' published_date: "2025-07-16" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - singing architecture: - autoregressive-LM - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: scaling_the_number_of_rvq_codebooks_in_a role: supports claim: Scaling the number of RVQ codebooks in a discrete speech codec reduces information loss and improves reconstruction quality for expressive and challenging vocal content. source: §3.2, Table 5 evidence: QDAC reconstruction improves monotonically from 1 to 16 codebooks across PESQ, STOI, and SI-SDR; 16-codebook QDAC at 50Hz achieves PESQ 3.83 versus PESQ 2.98 for 8 codebooks, with similar gains on Mel distance and speaker similarity. confidence: high relevance: high - claim_id: higher_multi_codebook_reconstruction_fidelity_in_a_codec role: complicates claim: Higher multi-codebook reconstruction fidelity in a codec does not necessarily translate into better speaker identity preservation in downstream zero-shot TTS. source: §3.2, Table 6 evidence: QTTS achieves Spk Sim 0.75 on SeedTTS-Easy compared to 0.82 and 0.81 for single-codebook CosyVoice v1 and v2, despite QDAC's superior reconstruction metrics in Table 5. confidence: high relevance: high - claim_id: semantic_disentanglement_in_audio_codecs_can_be_achieved role: supports claim: Semantic disentanglement in audio codecs can be achieved by backpropagating ASR loss exclusively through the first RVQ codebook, enforcing content isolation without relying on general-purpose self-supervised representations. source: §2.1.2, Table 5 evidence: QDAC trains an AR-ASR module conditioned only on first-codebook tokens; the resulting WER at reconstruction (6.42 for 8cb/25Hz) is close to ground truth (6.01), indicating the first codebook encodes phoneme-level content while residual codebooks capture acoustic detail. confidence: high relevance: low - claim_id: multi_codebook_autoregressive_tts_admits_a_principled_speed role: refines claim: Multi-codebook autoregressive TTS admits a principled speed-quality trade-off by choosing between strict hierarchical inter-codebook conditioning and a delayed multi-head parallel prediction scheme. source: §2.2, §2.3, Tables 1, 3, 4 evidence: Hierarchy Parallel (200M, dual-AR) and Multihead Delay (120M, parallel with fixed delay) achieve comparable TTFT at 512 tokens (26ms vs 24ms) but differ substantially in decode throughput; the Multihead variant reaches over 196K codebook tokens/s at short output lengths versus 105K for Hierarchy. confidence: high relevance: low - claim_id: mos_evaluations_in_zero_shot_tts_can_yield role: complicates claim: MOS evaluations in zero-shot TTS can yield above-reference scores for synthesised speech, undermining direct absolute comparisons across studies. source: §3.2, Table 6 evidence: Ground truth speech achieves MOS 2.7 while QTTS, CosyVoice, and CosyVoice2 all score between 3.01 and 3.03 on the same test set, producing a ranking inconsistent with naturalness expectations. confidence: high relevance: high limitations: - Training data is not disclosed, the PGC-hard benchmark is proprietary, and no code or demo is available, making results difficult to reproduce or build upon. - The evaluation compares only against the CosyVoice v1/v2 family, leaving open how QTTS performs relative to flow-matching systems, other multi-codebook approaches, or stronger autoregressive baselines. Speaker similarity is lower than both single-codebook baselines (0.75 vs 0.81-0.82), suggesting the multi-codebook generation pipeline needs further work to fully leverage the improved codec for speaker transfer. The paper positions singing and music synthesis as motivating use cases but does not evaluate on these tasks. Only the 8-codebook QTTS model is evaluated for TTS synthesis, leaving open whether 16-codebook generation would further improve or introduce new training challenges. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.14988' published_date: "2025-07-20" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: grpo_discrete_upstream_subcomponent_gain role: supports claim: Applying GRPO to a discrete upstream subcomponent of a distilled TTS pipeline achieves perceptual metric gains at substantially lower computational cost than whole-system RL. source: §3.2.2, §4.1, Table 1 evidence: Duration predictor GRPO fine-tuning uses 4-step student inference for reward computation and converges in 1.5K steps; on Seed-TTS-en, WER improves from 3.750 (no RL) to 1.752 and SIM from 0.672 to 0.698 without modifying the synthesis model. confidence: high relevance: low - claim_id: duration_prediction_bottleneck role: supports claim: Duration prediction quality independently constrains intelligibility and speaker similarity in non-autoregressive TTS regardless of synthesis model capacity. source: §A.1, Table 3 evidence: With the same flow-matching generator, WER on Seed-TTS-en ranges from 1.723 (best-of-8 oracle duration sampling) to 3.750 (unoptimized predictor) to 1.752 (RL-optimized), establishing duration as the primary bottleneck. confidence: high relevance: high - claim_id: distillation_reduces_prosodic_diversity role: complicates claim: Distribution matching distillation in flow-matching TTS reduces prosodic diversity disproportionately relative to acoustic quality, concentrating degradation in temporal and structural dimensions. source: §3.3.1, §4.3.3, Table 1, Figure 3 evidence: Step reduction from 32 to 4 lowers pitch CV-f0 by 30.3% (from 0.666 to 0.464) while mean-centered UTMOS distributions remain consistent across teacher and student, showing diversity loss is prosodic rather than spectral. confidence: high relevance: low - claim_id: best_of_n_practical_ceiling role: refines claim: Best-of-N sampling over discrete duration predictions establishes a practical performance ceiling for RL-based duration optimization in non-autoregressive TTS. source: §A.1, Table 3 evidence: Best-of-8 duration sampling achieves WER 1.723 and SIM 0.724 on Seed-TTS-en; the RL-optimized system reaches WER 1.752 and SIM 0.698 in a single pass, approaching but not fully closing the gap to this oracle upper bound. confidence: medium relevance: low - claim_id: teacher_guided_sampling_restores_diversity role: supports claim: A hybrid inference strategy using a high-diversity teacher for early denoising and a distilled student for acoustic refinement can restore prosodic diversity without sacrificing metric optimization gains. source: §3.3.2, §4.3.3, Table 1 evidence: Teacher-guided sampling (t_switch=0.25, 16 steps) recovers CV-f0 to 0.593 (89.1% of teacher diversity) while achieving WER 1.738 and SIM 0.699, outperforming the teacher on both objective metrics at 1.8x faster RTF. confidence: high relevance: low limitations: - Reward signals used for RL optimization are limited to WER and speaker similarity; explicit naturalness or prosody reward metrics are not explored, leaving the reward design incomplete relative to full perceptual quality. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.15272' published_date: "2025-07-21" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - diffusion relevance: high evidence_role: - core_evidence - control_evidence current_role: minor method_family: - diffusion_voice_cloning claims: - claim_id: conditioning_duration_prediction_on_reference_audio_from_the_same_speake role: supports claim: Conditioning duration prediction on reference audio from the same speaker can improve speaker similarity in zero-shot diffusion TTS without modifying the training objective. source: §2.2, Table 2 evidence: Cross-attention over an unrelated reference mel-spectrogram (same speaker, different content) improves SIM-O over a speaker-embedding-conditioned Grad-TTS baseline across all seven Indian languages tested. confidence: high relevance: high - claim_id: classifier_free_guidance_with_a_dataset_mean_unconditional_condition_imp role: supports claim: Classifier-free guidance with a dataset-mean unconditional condition improves speaker consistency at inference time in diffusion-based TTS for unseen speakers. source: §2.3, Table 2 evidence: A2TTS applies CFG at inference, following UnitSpeech, to amplify speaker conditioning without changing the training procedure; speaker similarity gains are observed across all languages evaluated. confidence: high relevance: high - claim_id: diffusion_based_zero_shot_tts_can_achieve_reasonable_speaker_similarity role: complicates claim: Diffusion-based zero-shot TTS can achieve reasonable speaker similarity, but intelligibility on out-of-domain benchmarks degrades substantially relative to ground-truth speech. source: §Appendix A, Tables 3-9 evidence: A2TTS TTS WER on FLEURS, CommonVoice, and IndicTTS is 1.5-4x higher than the corresponding ASR baseline on the same test sets across Hindi, Marathi, Punjabi, Bengali, Gujarati, Tamil, and Malayalam. confidence: medium relevance: high - claim_id: language_specific_diffusion_tts_models_can_be_trained_on_standard_benchm role: supports claim: Language-specific diffusion TTS models can be trained on standard benchmark speech corpora to achieve multi-speaker zero-shot synthesis in low-resource settings. source: §3.1, Table 2 evidence: A2TTS trains separate models for seven Indian languages using only the IndicSUPERB dataset (except Malayalam and Tamil which also use IndicTTS), achieving SIM-O above 0.70 for all languages on the IndicSUPERB evaluation set. confidence: high relevance: high limitations: - See the source paper page for the complete limitations and evaluation scope. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.16875' published_date: "2025-07-22" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: duration_strategy_language_dependent_tradeoffs role: complicates claim: Duration prediction strategy choice in zero-shot TTS presents language-dependent trade-offs between intelligibility and speaker similarity that are not consistent across multilingual settings. source: §5, Table 3, Table 5 evidence: The speaker-prompted (PFlow-style) predictor improves Tamil intelligibility (WER 0.295 vs. 0.312) and speaker similarity (SMOS 4.56 vs. 4.41) but degrades Marathi intelligibility substantially (WER 0.365 vs. 0.224 for infill), while Bengali human evaluation favours infill on both dimensions. confidence: medium relevance: high - claim_id: speaker_prompted_duration_no_forced_alignment role: supports claim: Speaker-conditioned audio prompting enables duration prediction without forced alignment, yielding competitive or superior speaker similarity in low-resource multilingual TTS. source: §3.4.2, §5.1, Table 4, Table 5 evidence: The 84M-parameter speaker-prompted predictor, conditioning on a 3-second mel spectrogram via cross-attention, achieves higher Sim-o scores than the infill predictor for Tamil, Telugu, and Bengali, and substantially higher human SMOS for Hindi (4.14 vs. 3.57), without using explicit duration context at inference. confidence: medium relevance: high - claim_id: pooled_corpora_quality_variance_requires_filtering role: complicates claim: Pooled low-resource training corpora for multilingual TTS contain significant quality variance that requires aggressive filtering before flow-matching models can be reliably trained. source: §4.2.2 evidence: WER-based filtering using IndicWhisper reduced the Hindi training set from 1.3M to 1M utterances; a partial recovery via CTC alignment scores added 30K back, indicating that roughly 23% of the raw data was too noisy or misaligned to be usable. confidence: medium relevance: low - claim_id: speaker_prompting_benefit_depends_on_phonology role: refines claim: The relative benefit of speaker-prompted duration conditioning over text-and-alignment conditioning is not uniform and depends on the phonological and prosodic characteristics of the target language. source: §5.1, §5.2, Table 3, Table 4, Table 5 evidence: Tamil, with complex phoneme structures and high prosodic variability, benefits consistently from speaker prompting across both objective WER and human SMOS; Marathi shows a clear regression; Hindi and Bengali exhibit mixed patterns depending on test subset. confidence: medium relevance: medium limitations: - All experiments are conducted on speech infilling (50% sentence masking) rather than full-sentence TTS generation, so intelligibility and speaker-similarity results may not transfer directly to the standard single-utterance TTS evaluation setting. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.21138' published_date: "2025-07-22" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: rl_alignment_with_composite_perceptual_rewards_improves_multiple role: supports claim: RL alignment with composite perceptual rewards improves multiple speech quality dimensions simultaneously over SFT-only baselines in autoregressive codec TTS. source: §3.5, Table 8 evidence: GRPO with combined WER + speaker similarity + DNSMOS rewards achieves gains in all three metrics over the SFT baseline, while single-reward models improve only their target dimension; TTS-1 WER drops from 7.9% (SFT) to 6.3% (RL-aligned). confidence: high relevance: high - claim_id: audio_pre_training_on_large_scale_raw_speech role: supports claim: Audio pre-training on large-scale raw speech substantially improves subsequent SFT quality in LLM-based TTS beyond what instruction-tuned LLM initialisation provides. source: §3.4, Figure 5 evidence: Initialising SFT from an audio pre-trained LLaMA-3.2-1B checkpoint yields lower SFT loss, approximately 15% lower WER, and approximately 3% higher speaker similarity than initialising from the base LLaMA-3.2-1B-Instruct checkpoint. confidence: high relevance: high - claim_id: scaling_speechlm_parameter_count_in_autoregressive_codec_tts role: supports claim: Scaling SpeechLM parameter count in autoregressive codec TTS consistently improves intelligibility and speaker fidelity across languages. source: §4, Figure 8, Table 8 evidence: TTS-1-Max (8.8B) achieves 5.1% overall WER and higher SIM across all 11 evaluated languages compared to TTS-1 (1.6B) at 6.3% WER, with performance correlating with pre-training loss differences. confidence: high relevance: medium - claim_id: style_conditioning_via_discrete_text_tags_conflicts_with role: complicates claim: Style conditioning via discrete text tags conflicts with speaker identity preservation in single-codebook codec TTS architectures. source: §3.6 evidence: Direct prepending of style markup tags during SFT produced no effect on synthesized speech because the single-codebook design entangles acoustic and semantic information; successful style control required constructing paired neutral/stylized utterances from the same speaker and applying LoRA fine-tuning. confidence: high relevance: high - claim_id: streaming_audio_delivery_in_autoregressive_tts_introduces_audible role: complicates claim: Streaming audio delivery in autoregressive TTS introduces audible artifacts and volume inconsistencies at segment boundaries that require specific engineering mitigations independent of the generative model's quality. source: §5.1 evidence: Without concatenation restricted to non-voicing regions and context-aware decoder decoding with extended audio prompt context, segment boundaries introduce clicks and volume drops; these are engineering-layer problems independent of SpeechLM quality. confidence: high relevance: medium limitations: - Model weights are not publicly released, making independent benchmarking and replication impossible. All evaluations use proprietary or internal test sets; the internal TTS arena covers only English and uses approximately 20 annotators with a modest vote count per pair. - The training data is drawn from a proprietary mixture of public and licensed sources whose exact composition is not disclosed, limiting reproducibility. The evaluation framework does not include standard public TTS benchmarks (e.g., LibriTTS or VCTK test sets), making direct numerical comparison with published systems that do report on these benchmarks difficult. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-demo.37 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - VC architecture: - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: minor method_family: - modular_hybrid_systems claims: - claim_id: rtvc_articulatory_disentanglement_preserves_intelligibility role: supports claim: Articulatory feature spaces enable interpretable content-speaker disentanglement in voice conversion without sacrificing intelligibility relative to SSL-based approaches. source: §5.3, Table 1 evidence: RT-VC's SPARC-derived articulatory pipeline achieves WER 6.69% and CER 2.12% on LibriTTS to VCTK zero-shot conversion, close to StreamVC's SSL-based WER of 6.22% and CER 2.17%, while relying on interpretable EMA features rather than an abstract latent space. confidence: medium relevance: high - claim_id: rtvc_realtime_cpu_below_70ms role: supports claim: Real-time zero-shot voice conversion on CPU hardware is achievable below 70 ms end-to-end latency while maintaining naturalness MOS above 3.8. source: §3.6, Table 1 evidence: RT-VC achieves 61.4 ms CPU latency (13.3% faster than StreamVC's 70.8 ms) with MOS 3.87 and SMOS 3.59 on LibriTTS test-clean to VCTK conversion. confidence: high relevance: high - claim_id: rtvc_ddsp_competitive_with_gan role: supports claim: Causal DDSP vocoders conditioned on articulatory features provide competitive synthesis quality compared to GAN-based alternatives at substantially lower computational cost. source: §2.3, §3.5 evidence: The causal harmonic-plus-noise DDSP vocoder, conditioned on articulatory features and speaker embedding via FiLM layers, achieves MOS 3.87 while running within the 61.4 ms end-to-end CPU latency budget. confidence: medium relevance: medium - claim_id: rtvc_noise_robustness_floor role: complicates claim: Voice conversion systems trained with static noise augmentation degrade gracefully down to approximately 20 dB SNR input but fail at 10 dB, suggesting a practical noise floor for real-time deployment. source: §5.4 evidence: RT-VC remains robust to static noise down to approximately 20 dB SNR (WER stays near 10%), but white noise at 10 dB SNR degrades quality substantially. confidence: medium relevance: medium limitations: - EMA representation omits nasal cavity and laryngeal dynamics, limiting modeling of nasal sounds and vocal fry; pseudo-EMA labels come from a WavLM model pretrained on English, limiting cross-lingual performance. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-industry.42 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS architecture: - GAN - transformer-enc-dec relevance: medium evidence_role: - architecture_variant - control_evidence current_role: minor method_family: - encoder_decoder_voice_cloning claims: - claim_id: g2p_conversion_most_important_for_tonal_intelligibility role: supports claim: Accurate grapheme-to-phoneme conversion is the single most important preprocessing component for intelligibility in tonal-language TTS, contributing more to WER reduction than tokenisation or pause prediction combined. source: §6, Table 3 evidence: Ablation shows removing the G2P module raises WER from 6.3% to 22.5%, far more than removing tokenization (10.2%) or pause prediction (6.5%), confirming G2P as the dominant contributor to intelligibility in this Thai TTS system. confidence: high relevance: low - claim_id: domain_specific_data_reduces_proprietary_degradation role: supports claim: Proprietary general-purpose TTS systems degrade substantially on domain-specific vocabulary, while systems trained with targeted vertical-domain data show much smaller performance drops in specialised settings. source: §6, Table 2 evidence: On domain-specific tasks (finance, healthcare, education, law), Google TTS's WER rises from 6.5% (general domain) to 14.5%, while the proposed system trained with vertical-domain data shows only a small degradation to 6.5% domain WER. confidence: medium relevance: low - claim_id: zero_shot_speaker_similarity_competitive_with_vc role: supports claim: Low-resource TTS systems can achieve competitive speaker similarity in zero-shot voice cloning relative to voice conversion models when trained with a style embedding module, even without speaker-specific fine-tuning. source: §6, Table 4 evidence: The style-encoder-based zero-shot cloning system achieves SIM 0.91 and SMOS 4.5 on a 10-second reference clip, exceeding OpenVoice's 0.85 SIM and 4.0 SMOS, without speaker-specific fine-tuning. confidence: medium relevance: high - claim_id: tonal_bert_prosody_improves_naturalness role: supports claim: Contextual prosody modelling via a tonal phoneme-aware BERT encoder improves naturalness scores in tonal-language TTS beyond what standard duration and pitch predictors achieve. source: §5 evidence: The Phoneme-Tone BERT prosody encoder feeding duration, pitch, and energy predictors contributes to a general-domain NMOS of 4.4, exceeding all compared open-source Thai TTS baselines. confidence: medium relevance: low limitations: - Model size and full architecture details are not reported, limiting reproducibility of the reported metric values. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.1043 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS architecture: - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: replacing_gaussian_noise_with_a_learned_prior_as role: supports claim: Replacing Gaussian noise with a learned prior as the starting point for flow matching reduces the number of required inference steps to one without needing a separate distillation stage. source: §3.1, §3.3 evidence: 'Vector Field Estimator (OT-CFM): Rather than regressing vectors from Gaussian noise x_0 to data x_1, the estimator regresses from the learned prior x_pr to x_1.' confidence: high relevance: low - claim_id: flow_matching_tts_systems_trained_on_traditional_ot role: complicates claim: Flow-matching TTS systems trained on traditional OT-CFM are data-hungry and fail to generalise when retrained on limited data, while neural codec-based systems remain effective with as few as 500 hours. source: §4.2, Table 1 evidence: 'OZSpeech consists of three components: FACodec (frozen): NaturalSpeech 3''s factorized neural codec decomposes speech into 6 discrete token sequences encoding prosody (1 quantizer), content (2 quantizers), and acoustic details (3 quantizers), plus a separate timbre embedding.' confidence: high relevance: high - claim_id: zero_shot_tts_systems_trained_exclusively_on_clean role: supports claim: Zero-shot TTS systems trained exclusively on clean prompts degrade substantially in intelligibility when given noisy reference audio, with autoregressive codec models being especially vulnerable. source: §4.4, Table 4 evidence: Under noisy prompts (Table 4), all baselines degrade dramatically. confidence: high relevance: high - claim_id: factorised_codec_representations_that_balance_acoustic_and_semantic role: supports claim: Factorised codec representations that balance acoustic and semantic attributes trade perceptual naturalness (UTMOS) for intelligibility (WER) relative to codecs that prioritise acoustic fidelity. source: §4.2 evidence: UTMOS is lower than F5-TTS and VALL-E (3.15 vs. 3.7+) due to codec trade-offs (FACodec balances acoustic and semantic representations). confidence: high relevance: low - claim_id: fine_tuning_a_zero_shot_tts_model_on role: supports claim: Fine-tuning a zero-shot TTS model on noise-augmented prompts preserves intelligibility and substantially recovers acoustic quality metrics under low-SNR conditions. source: §4.4, Table 4 evidence: Fine-tuning with noisy prompts further improves acoustic quality metrics. confidence: high relevance: high limitations: - '- UTMOS and speaker similarity lag behind F5-TTS and VoiceCraft; the prior-based approach trades acoustic naturalness for intelligibility and speed. - Duration predictor rounding errors (integer quantization of phoneme durations) introduce temporal domain artifacts. - FACodec dependency means reproduction requires NaturalSpeech 3''s codec infrastructure. - Noise robustness fine-tuning improves non-WER metrics but was validated only on the QUT-NOISE database. - Future work: multilingual extension, adaptive noise filtering integration.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.1252 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS - SCA architecture: - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: acoustic_accent_match_improves_chatbot_reception role: supports claim: Incorporating an acoustic accent that matches a user's linguistic community improves chatbot warmth, similarity, and engagement preference even when the underlying text remains in standard dialect. source: §4.4.2, Figure 4 evidence: The AA-accent + SAE-text spoken chatbot configuration outperforms the SAE baseline across all evaluated Likert dimensions (comprehension, communication ease, similarity, role appropriateness, engagement preference) among 8 AAE-speaking evaluators. confidence: medium relevance: medium - claim_id: high_intensity_text_dialect_produces_caricature role: complicates claim: Text-based dialect generation at high intensity produces caricature-like representations that degrade perceived inoffensiveness and role appropriateness with speakers of that dialect. source: §4.4.1, Figure 3 evidence: High-AAE text chatbots receive the worst inoffensiveness scores among all 9 AAE text variants, with near-neutral or negative scores for trustworthiness and role appropriateness, which the authors attribute to caricature-like over-modification rather than authentic AAE. confidence: medium relevance: medium - claim_id: phonetic_modification_dominates_llm_generated_aae role: supports claim: Phonetic modification dominates LLM-generated AAE output, while semantic feature generation remains sparse across all tested models and intensity levels. source: §4.3, Figure 2 evidence: Claude-Sonnet-3.5-based AAE feature tagging (91% accuracy) shows phonetic features dominate across all three LLMs' generated AAE text, while semantic AAE features remain sparse regardless of intensity level. confidence: medium relevance: low - claim_id: tts_trained_on_standard_dialect_degrades_on_minority_accent role: complicates claim: TTS models trained predominantly on standard-dialect data exhibit degraded speech clarity when reproducing minority accent features, limiting the fidelity of accent-conditioned spoken chatbots. source: §5 evidence: F5-TTS, trained predominantly on Standard American English data, shows decreasing speech clarity ratings as AAE dialect intensity increases in the spoken chatbot condition, even though it is applied only via a single CORAAL reference clip for accent. confidence: medium relevance: medium limitations: - The evaluator pool is small (12 text, 8 spoken) from a single university demographic, limiting the statistical reliability of the 15-metric Likert evaluation framework. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.1498 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS - codec architecture: - autoregressive-LM - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: context_dependent_encoding_in_rvq_based_neural_audio role: supports claim: Context-dependent encoding in RVQ-based neural audio codecs causes perceptually equivalent audio segments to produce divergent discrete token sequences, increasing prediction uncertainty in downstream codec language models. source: §2, §2.3 evidence: 'The paper first defines a consistency accuracy metric: given a full audio clip and a random slice extracted from it, consistency accuracy measures what fraction of the slice''s RVQ token positions are identical between encoding the slice in isolation and encoding the same frames within the full-length clip.' confidence: high relevance: low - claim_id: codec_token_consistency_and_downstream_autoregressive_tts_quality role: supports claim: 'Codec token consistency and downstream autoregressive TTS quality are monotonically correlated: improvements in consistency accuracy reliably reduce word error rate and increase speaker similarity.' source: §5.2, Figure 4 evidence: The reported correlation between consistency accuracy and WER (Figure 4) shows a clear monotonic relationship — improving consistency reliably reduces downstream WER. confidence: high relevance: high - claim_id: auxiliary_consistency_losses_applied_during_codec_training_can role: supports claim: Auxiliary consistency losses applied during codec training can substantially increase token consistency with negligible impact on reconstruction quality. source: §5.1, Table 1 evidence: 'To mitigate DRI without shrinking the encoder''s receptive field (which would harm reconstruction quality), the authors add two auxiliary MSE losses during codec training: Slice-consistency: A random 20% segment is carved out of each training audio clip and encoded in isolation to obtain Z_slice.' confidence: high relevance: low - claim_id: consistency_constraint_methods_applied_to_codec_training_generalize role: supports claim: Consistency constraint methods applied to codec training generalize across neural codec architectures and autoregressive LM backbones, as demonstrated by cross-system experiments on both EnCodec-VALL-E and FunCodec-UniAudio pipelines. source: §5.2, Table 3 evidence: 'Cross-codec generalizability (Table 3): Applying consistency constraint to EnCodec (as input to VALL-E) reduces WER from 9.02% to 5.13% at 960 h.' confidence: high relevance: low - claim_id: in_rvq_codecs_deeper_codebook_layers_suffer_disproportionately role: supports claim: In RVQ codecs, deeper codebook layers suffer disproportionately from context-induced inconsistency, because they encode fine-grained acoustic detail that is more sensitive to contextual perturbation than the semantic information stored in shallow layers. source: §2.3, Appendix A.3, Table 6 evidence: 'The analysis shows that: (1) all existing codecs suffer from low consistency — FunCodec 8 kbps has only 3.58% consistency across all 16 codebook layers; (2) consistency deglines substantially with deeper codebook layers in every codec examined, because shallow layers encode semantic (context-independent) information while deeper layers encode fragile fine-grained acoustic detail; and (3) the large receptive field of the RVQ encoder (up to 2718 samples at layer 18, per Table 5 in the paper) is the structural cause.' confidence: high relevance: low limitations: - The DRI fix is applied at codec training time; it does not address the underlying structural cause (large receptive fields in the convolutional encoder). A follow-up could explore whether directly reducing receptive field size with compensating distillation achieves similar or better consistency. - The method is validated on English speech (LibriTTS and MLS, which is multilingual but dominated by English). Whether DRI and its mitigation generalize to tonal languages (where fine-grained acoustic context is phonemically contrastive) is unstudied. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.313 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - nonautoregressive_flow_matching claims: - claim_id: explicit_alignment_not_required_for_robust_naf_tts role: supports claim: Explicit phoneme-level alignment is not required for robust non-autoregressive zero-shot TTS; modality-specific representation refinement before feature concatenation can resolve text-speech alignment failures in alignment-free architectures. source: §3.2, §5.1 evidence: ConvNeXt V2 text refinement prior to concatenation resolves the alignment-failure mode observed in E2 TTS without reintroducing phoneme-level forced alignment or a duration predictor, achieving WER 2.42% on the newly released LibriSpeech-PC test-clean benchmark. confidence: high relevance: high - claim_id: entangled_semantic_acoustic_features_cause_alignment_failure role: complicates claim: Deep entanglement of semantic and acoustic features in a shared transformer produces systematic alignment failures in non-autoregressive TTS that persist across training scale and cannot be addressed by re-ranking. source: §3.2, §5.1, Table 4 evidence: Removing the ConvNeXt text-refinement branch (F5-TTS -Conv2Text) causes complete alignment failure, and applying the same refinement to E2 TTS's U-Net backbone instead degrades WER to 18.1%, showing the entanglement problem is backbone-specific and not resolved by re-ranking alone. confidence: high relevance: low - claim_id: sway_sampling_improves_faithfulness_without_retraining role: supports claim: Inference-time flow step scheduling that densifies early integration steps improves faithfulness and speaker similarity in flow-matching TTS without any retraining. source: §3.2, §5.2, Table 5 evidence: Sway Sampling improves WER from 2.84% to 2.41% and UTMOS from 3.70 to 3.89 at 32 NFE on LibriSpeech-PC test-clean without any retraining, and the same inference-time schedule also improves E2 TTS's WER from 2.95% to 2.84%, confirming portability across CFM-based TTS models. confidence: high relevance: high - claim_id: naf_flowmatching_matches_ar_at_100k_hour_scale role: supports claim: Non-autoregressive flow-matching TTS trained at 100K-hour scale can match autoregressive systems on intelligibility while achieving substantially lower real-time factor. source: §5, Table 1 evidence: At 16 NFE, F5-TTS achieves WER 2.53% with an RTF of 0.15, compared to Voicebox's RTF of 0.64 and DiTTo-TTS's 0.162, while remaining competitive with autoregressive systems on intelligibility across Seed-TTS test-en and test-zh. confidence: medium relevance: medium - claim_id: missing_standardized_benchmarks_barrier_to_comparison role: supports claim: Absence of standardised benchmark subsets with released prompt lists is a structural barrier to fair cross-paper comparison in zero-shot TTS; community-released evaluation sets directly reduce this problem. source: §4 evidence: The paper notes that comparisons with VALL-E 2 and MELLE rely on scores from a different, incompatible LibriSpeech test-clean subset reported in prior work, motivating the release of the fixed-prompt-list LibriSpeech-PC benchmark to enable fair cross-paper comparison in zero-shot TTS. confidence: high relevance: high limitations: - Comparisons against VALL-E 2 and MELLE reuse numbers from a different LibriSpeech test-clean subset reported in prior work, which cannot be directly compared to the LibriSpeech-PC results introduced by this paper. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.346 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS architecture: - transformer-enc-dec - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - encoder_decoder_voice_cloning claims: - claim_id: ctrlspeech_decoupling_necessary_for_joint_control role: supports claim: Disentangling speaker timbre and speaking style into separate codec representations is a necessary condition for simultaneous zero-shot speaker cloning and style control; without explicit decoupling, the two conditioning signals interfere and controllability collapses. source: §4.3, Table 4 evidence: The codec decoupling ablation shows that without decoupling, controllability collapses on test set A (pitch accuracy drops to 0.492, speed to 0.517, emotion to 0.237), confirming that independently disentangling the speech prompt and style prompt is necessary to prevent cross-channel interference. confidence: high relevance: high - claim_id: ctrlspeech_style_description_many_to_many role: supports claim: Natural language style descriptions have an inherent many-to-many relationship with audio that cannot be resolved by timbre conditioning alone, requiring a probabilistic model of style variation such as a mixture density network. source: §3.3 evidence: ControlSpeech identifies the many-to-many mapping between style descriptions and acoustic realizations as an inherent ambiguity of natural language itself, distinct from PromptTTS 2's attribution of one-to-many ambiguity to missing timbre information, motivating the SMSD mixture-density module. confidence: medium relevance: high - claim_id: ctrlspeech_zeroshot_cloning_on_disentangled_codec role: supports claim: Zero-shot speaker cloning capability in style-controllable TTS can be achieved by building on a large-scale pre-trained disentangled codec without sacrificing audio quality relative to dedicated zero-shot TTS systems. source: §4.2, Table 2 evidence: On out-of-domain speakers (test set B), ControlSpeech achieves WER 3.3% vs. VALL-E's 6.7%, with MOS-Q 3.95 comparable to MobileSpeech's 3.94, despite additionally providing independent style control via the frozen FACodec disentangled representation. confidence: high relevance: high - claim_id: ctrlspeech_probabilistic_sampling_improves_generalization role: supports claim: Probabilistic sampling from a mixture density model of style representations improves both style diversity and generalization to out-of-domain style descriptions compared to deterministic style encoding. source: §4.2, §4.3, Table 3 evidence: The ablation without SMSD drops MOS-SA from 3.84 to 3.59 and MOS-SD from 4.05 to 3.66 on the many-to-many style control test set, and ControlSpeech degrades much less than baselines on out-of-domain style descriptions (test set C). confidence: high relevance: medium - claim_id: ctrlspeech_pitch_harder_under_joint_control role: complicates claim: Pitch control is measurably harder to preserve when timbre and style are controlled simultaneously, suggesting that pitch conditioning interacts with speaker identity in ways that speed, volume, and emotion do not. source: §4.2, Table 1 evidence: ControlSpeech's pitch accuracy (0.833) is the only style-control metric where it trails style-only baselines InstructTTS (0.849) and PromptTTS 2 (0.867), attributed by the authors to the difficulty of simultaneously controlling timbre and style. confidence: medium relevance: high limitations: - The training dataset is still limited in scale for style-controllable TTS; the exploration of generative model architectures is narrow (only non-autoregressive parallel decoding is tried). caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.598 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: cross_model_preference_pairs_yield_larger_gains role: supports claim: Preference alignment with cross-model preference pairs yields larger intelligibility improvements than single-model self-comparison pairs for TTS post-training. source: §5.2, Table 5 evidence: Ablation shows that combining intra-model and inter-model (cross-model) preference pairs yields larger WER reductions than intra-model self-comparison pairs alone, with inter-pairs and intra-pairs shown to be complementary. confidence: medium relevance: low - claim_id: dpo_derivable_for_flow_matching_and_masked_generative role: supports claim: DPO objectives can be derived for flow-matching and masked generative TTS architectures by operating in velocity space and masked token distributions, respectively, without requiring explicit reward models. source: §4.2, §4.3 evidence: The paper derives DPO-FM by showing the log-ratio of learned versus reference velocity fields at each flow timestep is equivalent to an MSE difference in predicted velocities, and derives DPO-MGM operating on masked token distributions, enabling principled preference alignment for F5-TTS and MaskGCT respectively. confidence: high relevance: low - claim_id: intelligibility_gains_generalize_to_unseen_languages role: supports claim: Intelligibility gains from preference alignment on English and Chinese data generalize to unseen languages (Japanese, Korean, German, French) without degrading speaker similarity. source: §5.1, Table 10 evidence: INTP alignment trained only on English and Chinese data reduces WER substantially when evaluated on Japanese, Korean, German, and French via GTSinger, without measurable loss in speaker similarity. confidence: high relevance: high - claim_id: weak_to_strong_generalization_in_speech_alignment role: supports claim: Preference datasets constructed from weaker TTS generators effectively improve stronger models outside the training distribution, demonstrating weak-to-strong generalization in speech alignment. source: §5.1, Table 4 evidence: Preference data built from the weaker ARS, F5-TTS, and MaskGCT generators improves the stronger CosyVoice 2 (average WER 13.09 to 10.56) and the authors' own Ints model, which were not part of INTP's construction. confidence: high relevance: medium - claim_id: iterative_alignment_diminishing_returns role: refines claim: Iterative preference alignment yields diminishing returns across rounds, suggesting that base model capability is the primary ceiling on alignment effectiveness. source: §5.3, Table 6 evidence: Ints v1 (INTP-aligned) generates new preference data to train Ints v2, reducing average WER from 7.96 to 7.10, a smaller relative gain than the initial alignment round, suggesting base model capability bounds further iterative improvement. confidence: medium relevance: low limitations: - All evaluation uses the SeedTTS test sets (English and Chinese); no evaluation on genuinely out-of-distribution natural speech (e.g., radio or phone-call recordings) is provided. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.65 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS architecture: - autoregressive-LM - VAE - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: discrete_codec_representations_introduce_a_quantifiable_fidelity_loss role: supports claim: Discrete codec representations introduce a quantifiable fidelity loss relative to continuous mel-spectrogram representations even at high codebook counts, measurable in both WER and speaker similarity. source: §5, Table 1 evidence: 'Objective (Table 1, LibriSpeech test-clean, continuation task): - MELLE WER_C/WER_H: 1.47/1.98, SIM: 0.508 - VALL-E 2 WER_C/WER_H: 1.6/2.32, SIM: 0.504 - VALL-E WER_H: 3.8, SIM: 0.508 - Ground truth WER_H: 2.15, SIM: 0.668 (mel resynthesis: WER_H 2.24, SIM 0.617; EnCodec 8CB: WER_H 2.33, SIM 0.593) — confirming codec fidelity loss hypothesis MELLE achieves 47.9% relative WER reduction vs. VALL-E and 8.1% vs. VALL-E 2 on continuation.' confidence: high relevance: high - claim_id: continuous_valued_autoregressive_speech_synthesis_can_achieve_robustness role: supports claim: Continuous-valued autoregressive speech synthesis can achieve robustness and naturalness on par with codec-based two-stage systems when paired with appropriate regularization objectives. source: §5.1, §5.2, Table 1, Table 3 evidence: 'This module replaces top-p sampling for codec codes: it provides continuous diversity without the instability of discrete token sampling.' confidence: high relevance: low - claim_id: a_variational_latent_sampling_module_applied_to_continuous role: supports claim: A variational latent sampling module applied to continuous spectrogram prediction provides diversity and robustness benefits analogous to top-p sampling for discrete tokens, without the instability caused by the high similarity of consecutive acoustic codes. source: §3.2.2, §5.3, Table 4 evidence: 'This module replaces top-p sampling for codec codes: it provides continuous diversity without the instability of discrete token sampling.' confidence: high relevance: low - claim_id: bypassing_the_non_autoregressive_second_stage_in_codec role: supports claim: Bypassing the non-autoregressive second stage in codec language model pipelines reduces inference time while maintaining competitive output quality. source: §5.4, Table 5 evidence: 'Subjective (Table 3, 40 samples cross-sentence): - MELLE MOS: 4.20 ± 0.20 vs. GT 4.29 ± 0.16 (CMOS -0.032, p > 0.1 → not significantly different) - MELLE SMOS: 4.40 ± 0.22 vs. GT 3.94 ± 0.25 — higher than ground truth, suggesting the model''s speaker reproduction is more consistent than inter-utterance variation in the reference - VALL-E 2 MOS: 4.08, CMOS: -0.085 Efficiency (Table 5): - MELLE: 5.49s for 10s speech vs. VALL-E 2: 7.32s (no NAR second pass) - MELLE-R2: 2.76s, MELLE-R4: 1.40s — 5x speedup at r=4 while still competitive Ablation (Table 4): Removing both LSM and SFL degrades cross-sentence WER_C from 1.47 to 23.21 — catastrophic failure.' confidence: high relevance: medium - claim_id: prediction_quality_in_continuous_valued_autoregressive_tts_degrades role: complicates claim: Prediction quality in continuous-valued autoregressive TTS degrades gracefully with reduction factor increases, enabling a controllable quality-efficiency trade-off unavailable in discrete-token systems. source: §5.1, Table 1, Table 2 evidence: 'This module replaces top-p sampling for codec codes: it provides continuous diversity without the instability of discrete token sampling.' confidence: high relevance: low limitations: - '- English-only evaluation; multilingual extension not attempted. - Vocoder quality bottleneck: uses open-source HiFi-GAN trained on 585h LibriTTS; Voicebox''s proprietary vocoder trained on 60Kh provides higher quality ceiling. - Mel-spectrogram as the only continuous representation explored; VAE latent states suggested as future work. - SMOS exceeding ground truth may partly reflect the test setup''s limitation (inter-speaker/inter-session variation in the reference set rather than genuine quality superiority). - No streaming or low-latency inference analysis.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.654 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS - codec architecture: - GAN - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: codec_token_distributions_in_the_first_rvq_channel role: complicates claim: Codec token distributions in the first RVQ channel are a meaningful bottleneck for autoregressive generation from text, independent of reconstruction quality. source: §1, §3.3 evidence: The net effect is that no single early channel dominates information content, making the first-channel token distribution lower-entropy and therefore more tractable for a text-conditioned autoregressive model to predict. confidence: high relevance: low - claim_id: redistributing_information_load_uniformly_across_the_first_few role: supports claim: Redistributing information load uniformly across the first few RVQ codebook channels via parallel masked quantization consistently improves speaker similarity in downstream autoregressive TTS. source: §4.4, Table 3 evidence: 'In the first Nq=3 layers the quantizers operate in parallel: each receives only 1/Nq of the latent frame (the rest is masked), so each of the first three codebooks carries an equal, reduced share of the total information.' confidence: high relevance: high - claim_id: a_fourier_based_decoder_with_a_self_attention role: supports claim: A Fourier-based decoder with a self-attention module achieves better codec reconstruction quality than a transposed-convolution upsampler, without length extrapolation issues. source: §3.2, Appendix G, Table 9 evidence: The decoder replaces EnCodec's transposed-convolution upsampler with a Vocos-style Fourier-based decoder. confidence: high relevance: low - claim_id: codec_reconstruction_quality_does_not_scale_substantially_with role: supports claim: Codec reconstruction quality does not scale substantially with training data volume beyond a few hundred hours, while domain generalization does benefit from larger and more diverse datasets. source: Appendix A, Table 5 evidence: 'Codec reconstruction (LibriTTS Test-Clean, Table 1): - Language-Codec with 4 codebooks (3.0 kbps): UTMOS 3.79, PESQ 3.27, STOI 0.949 — outperforms all 4-codebook baselines and several 8-codebook baselines (eg., Encodec-8 PESQ 2.72, STOI 0.939). - Language-Codec with 8 codebooks (6.0 kbps): UTMOS 4.04, PESQ 3.88, STOI 0.972 — best across all models on all metrics. - Results hold under noisy conditions (LibriTTS Test-Other, Table 6) and out-of-domain (LJSpeech, Table 7).' confidence: high relevance: medium - claim_id: the_choice_of_underlying_codec_has_a_larger role: supports claim: The choice of underlying codec has a larger impact on zero-shot TTS speaker similarity than on intelligibility, with codec swaps producing 10–15% SPK-SIM gains while WER differences remain within noise. source: §4.3, Table 2 evidence: 'Downstream zero-shot TTS (LibriSpeech Test-Clean, Table 2): - Replacing EnCodec with Language-Codec in VALL-E raises SPK-SIM from 0.612 to 0.700 (+14%) with negligible WER change. - MobileSpeech with Language-Codec: SPK-SIM 0.771, WER 2.9%, MOS-Q 4.20 vs. EnCodec baseline MOS-Q 3.91.' confidence: high relevance: high limitations: - '- Language-Codec is trained and evaluated exclusively on speech; audio, music, and environmental sound domains are explicitly left as future work. The codec''s suitability for general audio language models is therefore unvalidated. - MCRVQ prediction accuracy drops when more than 4 codebook channels are used in downstream models, suggesting the information-redistribution benefit weakens at higher bitrates. The mechanism for this degradation is not fully explained. - The paper evaluates downstream quality only with VALL-E and MobileSpeech; it is unclear whether the SPK-SIM gains extend to flow-matching or diffusion-based TTS backends. - No demo page is linked in the paper, making it difficult to subjectively verify the quality claims beyond the crowd-sourced MOS. - The internal 20,000-hour Chinese dataset is not publicly available, limiting full reproducibility of the 50k-hour training run.' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-long.790 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - VC architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: rvc_shortcut_fm_two_step_matches_full role: supports claim: Self-consistency training enables shortcut flow matching to match full-step quality in voice conversion with as few as two inference steps. source: §4.2, Table 1 evidence: R-VC at NFE=2 matches NFE=10 across all quality metrics (SECS 0.930 vs 0.931, UTMOS 4.1 vs 4.1, QMOS 4.03 vs 4.05, SMOS 4.11 vs 4.12) while reducing inference time by 2.83x; vanilla CFM degrades sharply below 10 steps. confidence: high relevance: medium - claim_id: rvc_duration_model_improves_emotion_transfer role: supports claim: Explicit rhythm modeling via a target-conditioned duration model substantially improves emotion style transfer in zero-shot VC. source: §4.3, §4.5, Table 2, Table 4 evidence: Removing the duration module from R-VC drops the emotion score from 0.59 to 0.425 on the ESD dataset, while baselines that preserve source rhythm score 0.395-0.489. confidence: high relevance: high - claim_id: rvc_fine_grained_duration_instability role: complicates claim: Fine-grained duration prediction in non-autoregressive models introduces instability in voice conversion that coarser duration strategies do not fully resolve. source: §7, Table 4 evidence: R-VC's masked transformer duration model produces occasional over-extended pronunciations; sentence-level duration as a fallback yields worse WER (9.86 vs 6.95) and UTMOS (3.58 vs 3.85), offering no stability improvement in practice. confidence: medium relevance: medium - claim_id: rvc_perturbation_reduces_leakage role: supports claim: Data perturbation before discrete content tokenisation reduces timbre leakage more effectively than relying on the self-supervised representation alone. source: §4.5, Table 4 evidence: Removing pitch perturbation before HuBERT token extraction degrades WER from 3.51 to 7.28 and speaker similarity from 0.930 to 0.869, confirming that perturbation actively suppresses content-irrelevant speaker information. confidence: high relevance: high limitations: - The system is evaluated only on English (MLS, LibriSpeech, ESD) and English Seed-TTS subsets; generalisation to cross-lingual or multilingual VC is untested. - The masked transformer duration model has a known instability causing over-extended pronunciations; sentence-level duration as an alternative proved worse in both stability and quality. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.acl-short.81 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS - evaluation architecture: - autoregressive-LM - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: long_form_training_audio_10_20_seconds_per role: supports claim: Long-form training audio (10–20 seconds per segment) with explicit speaker identities improves zero-shot TTS quality for low-resource tonal languages compared to training on short-segment corpora. source: §3.1, §4 evidence: 'Three zero-shot TTS models are adapted to Vietnamese by fine-tuning on the 1,494-hour augmented set (940h long-form + 554h short-clip augmentation): - VALL-E: An autoregressive codec language model operating on EnCodec discrete tokens at the phoneme level.' confidence: high relevance: high - claim_id: multilingual_voice_cloning_models_such_as_xtts_v2 role: supports claim: Multilingual voice-cloning models such as XTTS-v2 exhibit architectural failure modes on short input sequences that are not corrected by data augmentation with short clips. source: §4 evidence: Manual inspection reveals that XTTS-v2-based models generate rambling or redundant speech on short inputs, which the authors attribute to an architectural issue in XTTS-v2 rather than a data problem. confidence: high relevance: medium - claim_id: autoregressive_codec_language_models_vall_e_voicecraft_generalize role: supports claim: Autoregressive codec language models (VALL-E, VoiceCraft) generalize better than Tortoise-based models to short-sentence inputs in low-resource language fine-tuning. source: §4, Table 2 evidence: 'Three zero-shot TTS models are adapted to Vietnamese by fine-tuning on the 1,494-hour augmented set (940h long-form + 554h short-clip augmentation): - VALL-E: An autoregressive codec language model operating on EnCodec discrete tokens at the phoneme level.' confidence: high relevance: high - claim_id: a_dataset_curation_pipeline_based_on_dual_asr role: supports claim: A dataset curation pipeline based on dual-ASR agreement filtering produces higher-quality transcriptions for audiobook audio than single-model transcription alone, enabling more reliable TTS training. source: §2.1 evidence: The authors construct PhoAudiobook via an automated pipeline applied to ~23K hours of Vietnamese audiobook audio scraped from sachnoiviet.net. confidence: high relevance: low limitations: - The paper does not evaluate code-switching scenarios (mixed Vietnamese-English input), which is relevant in practice. The dataset is audiobook domain only, so speaking style coverage is narrower than general-purpose datasets. All models are fine-tuned rather than trained from scratch, which means performance is bounded by the pre-trained model's multilingual capacity. The architecture issue observed with XTTS-v2 on short sentences is identified but not resolved. The dataset is released for non-commercial use only, which limits industrial adoption. It is also unclear how the system handles tonal phonology beyond phonemizer outputs, and no ablation on augmented data proportion is presented. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.findings-acl.115 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - SCA - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: slamomni_grouped_tokens_reduce_mismatch role: supports claim: Grouping discrete semantic tokens during autoregressive generation reduces training and inference costs by alleviating the frequency mismatch between text and audio token streams. source: §5.4.1, Table 6 evidence: Semantic Group Modeling with G=3 reduces SLAM-Omni training from 126 to 60 GPU hours and ASR-WER from 18.23% (G=1) to 4.54%. confidence: high relevance: low - claim_id: slamomni_pretraining_hurts_instruction_following role: refines claim: Multi-stage pre-training on modality-specific tasks for spoken dialogue systems can degrade instruction-following ability despite improving modality alignment metrics. source: §5.4.2, Table 7 evidence: ASR pre-training reduces ChatGPT Score from 39.32 to 34.02 and TTS pre-training to 27.22, while ASR-WER improves only marginally. confidence: high relevance: low - claim_id: slamomni_timbre_control_data_constrained role: complicates claim: Zero-shot timbre control in spoken dialogue systems achieves competitive speaker similarity but remains constrained by training data volume relative to dedicated TTS systems. source: §5.2, Table 5 evidence: SLAM-Omni reaches SIM-o of 0.517, comparable to FireRedTTS (0.486) but below CosyVoice2 (0.684), with the gap attributed to approximately 50x less training data. confidence: medium relevance: high - claim_id: slamomni_text_history_sacrifices_paralinguistics role: complicates claim: Compressing multi-turn dialogue history to text representation sacrifices non-verbal paralinguistic context that may be important for maintaining dialogue coherence across turns. source: §3.5, §6 Limitations evidence: Historical Text Prompting stores only text history, explicitly trading away emotional and prosodic signals from previous dialogue turns for computational efficiency. confidence: medium relevance: medium - claim_id: slamomni_semantic_tokens_tighter_alignment role: supports claim: Semantic token-based speech generation in spoken dialogue systems provides tighter speech-text alignment than acoustic codec-based approaches, as measured by word error rate between generated speech and corresponding text. source: §5.1, Table 3 evidence: SLAM-Omni achieves the lowest ASR-WER (4.54%) among evaluated spoken dialogue models, outperforming Moshi (7.18%), GLM-4-Voice (12.71%), and Freeze-Omni (16.32%), using single-layer semantic tokens. confidence: high relevance: medium limitations: - The single-stage training finding is demonstrated only at 0.5B scale with limited data (400K utterances); generalisation to larger backbones is untested. - The custom 8-task benchmark measures general spoken interaction but not domain-specific or emotion-aware dialogue quality. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.findings-acl.1226 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - TTS architecture: - hybrid relevance: low evidence_role: - historical_context current_role: minor method_family: - modular_hybrid_systems claims: - claim_id: multiagent_llm_produces_more_informative_scripts role: supports claim: Parallel multi-agent LLM architectures can produce more informative and engaging conversational scripts than single-prompt generation for long-form spoken content. source: §5.1, §5.3, Table 1, Table 2 evidence: Host-Guest-Writer system outperforms direct GPT-4 baseline on all five quantitative metrics and all five LLM-as-a-Judge dimensions across four topic categories; the single-agent ablation also shows substantially lower scores on all metrics. confidence: high relevance: medium - claim_id: llm_predicted_style_improves_speech_expressiveness role: supports claim: LLM-predicted speaking style instructions improve speech expressiveness in zero-shot TTS when used for long-form conversational audio. source: §5.2, Figure 7 evidence: All CMOS scores (0.2-0.9) are positive and preference tests favour LLM-instructed synthesis over uninstructed synthesis across all four topic categories in a 9-evaluator study. confidence: medium relevance: high - claim_id: automatic_voice_role_matching_cannot_guarantee_alignment role: complicates claim: Automatic voice-role matching from a speaker pool cannot guarantee perceptual alignment between voice characteristics and speaker roles in conversational audio generation. source: §5.2, §7 evidence: 87.4% average pass rate implies roughly 1 in 8 voice assignments is judged inappropriate by participants; the voice pool is also limited to read-speech data (LibriTTS-R), and the paper notes the need for more conversational-style voices. confidence: medium relevance: medium - claim_id: more_participants_not_monotonically_better role: supports claim: Increasing the number of participants in multi-agent dialogue generation does not monotonically improve content quality, and smaller groups with more focused interactions outperform larger ensembles. source: §5.3, Table 2 evidence: Ablation across 1-5 guest configurations shows that 2-guest setups score highest on Distinct-N, MATTR, and Semantic-Div; quality degrades progressively from 3 to 5 guests. confidence: medium relevance: low limitations: - Content evaluation uses GPT-4-as-a-Judge to evaluate GPT-4-generated outputs, introducing well-documented self-preference and position biases that the paper only partially mitigates by averaging comparison orders. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.findings-acl.687 published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: ACL task: - singing - TTS architecture: - flow-matching - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: tcsinger2_boundary_masking_improves_naturalness role: supports claim: Boundary masking during training improves zero-shot singing synthesis naturalness by forcing the model to learn smooth phoneme and note transitions without relying on precise alignment annotations. source: §3.2, Table 4 evidence: The BBC Encoder masks 8 tokens at each phoneme/note boundary before its output conditions the flow-matching transformer; ablating this masking causes CMOS-Q to drop -0.36 in style transfer, the largest single-component quality degradation. confidence: high relevance: high - claim_id: tcsinger2_contrastive_style_alignment_unifies_modalities role: supports claim: Contrastive alignment of cross-modal style embeddings (singing, speech, natural language) enables a single encoder to support style transfer, speech-to-singing, and instruction-conditioned synthesis without modality-specific architectures. source: §3.3, Table 4 evidence: The Custom Audio Encoder trained with CLIP-style triplet contrastive loss achieves a unified style space feeding the flow-matching transformer; removing it drops CMOS-S by -0.37 (style transfer) and CMOS-C by -0.41 (style control). confidence: high relevance: low - claim_id: tcsinger2_moe_routing_improves_multilingual_quality role: supports claim: Language-conditioned mixture-of-experts routing in a flow-matching transformer improves multilingual singing synthesis quality by directing language-family-specific token processing to specialised experts. source: §3.4, Table 4, Table 7 evidence: Lingual-MOE ablation (replacing the routed experts inside the flow-matching transformer with a standard FFN) causes CMOS-Q to drop -0.29 in style transfer; four experts per group is the optimal configuration before diminishing returns. confidence: high relevance: low - claim_id: tcsinger2_f0_supervision_improves_quality role: supports claim: F0 supervision applied to intermediate transformer representations improves both synthesis quality and style fidelity in zero-shot singing voice synthesis. source: §3.4, Table 4 evidence: Removing F0 supervision from the flow-matching transformer's first block output causes CMOS-Q -0.33 and CMOS-S -0.24 in style transfer, and CMOS-Q -0.31 and CMOS-C -0.27 in style control. confidence: high relevance: high - claim_id: tcsinger2_style_annotation_costly role: complicates claim: Multi-level style control via natural language prompts in singing synthesis requires costly manual annotation, limiting dataset scale and introducing labelling errors. source: §6, Appendix B evidence: Style labels were annotated by music experts at $300/hour then converted to natural language via GPT-4o; the paper cites labelling cost and annotation errors as primary limitations constraining generalisation. confidence: high relevance: medium limitations: - Training and test sets share the same underlying singer pool; 30 'unseen' singers are held out from a pool drawn from the same data collection, limiting assessment of true out-of-distribution generalisation. - Generation speed does not meet real-time industrial requirements despite 25-step ODE inference; streaming inference is deferred to future work. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.17527' published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: e2e_s2s_translation_rl_beats_cascaded_commercial role: supports claim: End-to-end speech-to-speech translation trained with reinforcement learning can achieve substantially lower latency than cascaded commercial systems while surpassing them in translation quality. source: §3.3, Table 1 evidence: On RealSI (zh-en), the system achieves VIP 79.5 and SVIP 67.8 at AL 5.18 seconds, versus the best commercial competitor at VIP 53.2 and SVIP 3.0 at AL 48.21. confidence: medium relevance: low - claim_id: twostage_rl_outperforms_single_strategy_streaming role: supports claim: Two-stage RL training that separates fine-grained process rewards from holistic sequence-level outcome rewards outperforms applying either strategy in isolation for streaming speech generation. source: §4.3, Table 6 evidence: The combined two-stage scheme achieves higher VIP and lower AL than both single-turn-only and multi-turn-only variants; multi-turn-only trades translation quality for latency, while single-turn-only cannot exploit global latency constraints. confidence: medium relevance: low - claim_id: rl_reward_design_prone_to_exploitation role: complicates claim: RL reward design for speech generation is prone to exploitation when individual reward signals are applied without complementary constraints. source: §4.2, Table 5 evidence: Training with the Time Compliance Reward alone caused the model to reduce audio duration by ~35% and text token count by ~15%, yielding an 8-point BLEURT drop while satisfying the temporal constraint; adding an adversarial quality reward restored translation fidelity. confidence: high relevance: low - claim_id: rl_improves_latency_more_than_quality role: supports claim: Reinforcement learning applied after SFT improves latency more substantially than translation quality in simultaneous speech translation. source: §4.1, Tables 3, 4 evidence: RL reduces FLAL from 3.90 to 2.37 and AL from 2.82 to 2.58 on RealSI, while VIP improves more modestly from 75.1 to 79.5 compared to the SFT baseline. confidence: medium relevance: low - claim_id: automated_mt_metrics_insufficient_for_simultaneous_speech role: complicates claim: Automated MT metrics such as BLEURT and COMET do not fully capture the quality of simultaneous speech interpretation in realistic long-form scenarios. source: §3.2, Appendix A evidence: The paper introduces SVIP as a composite human evaluation metric combining translation quality, delivery latency, speech rate, pronunciation, and fluency, arguing automated metrics fail to assess the pragmatic requirements of live spoken interpretation. confidence: medium relevance: low limitations: - Training data, model weights, and all commercial baseline identities are proprietary, so the headline results cannot be independently reproduced or verified. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.20140' published_date: "2025-07-27" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: unlearning_via_randomization_suppresses_speaker_identity role: supports claim: Machine unlearning via randomization-based training objectives can selectively suppress specific speaker identities in zero-shot TTS while preserving synthesis quality for retained speakers. source: §5.2, Table 1 evidence: TGU achieves SIM-F of 0.169 (within the measured cross-speaker similarity range of 0.02-0.17) and retain-set SPK-SIM of 0.631, a drop of only 2.8% from the original model's 0.649, with WER-F of 2.4% comparable to the original's 2.1%. confidence: high relevance: high - claim_id: data_exclusion_insufficient_for_voice_privacy role: complicates claim: Excluding target speakers from the fine-tuning dataset is insufficient for voice privacy protection in zero-shot TTS, because these models generalize at inference time to replicate unseen speakers via in-context learning. source: §5.2, Table 1 evidence: Exact Unlearning and Fine Tuning on the retain set yield SIM-F of 0.687 and 0.675 respectively, nearly identical to the original model's 0.708, confirming the model continues to clone forgotten speakers not present in the fine-tuning data. confidence: high relevance: high - claim_id: gradient_reversal_unlearning_degrades_intelligibility role: complicates claim: Gradient-reversal and KL-divergence-based unlearning methods degrade speech intelligibility rather than achieving genuine speaker forgetting in voice-conditioned generative models, due to entanglement between speaker style and linguistic content. source: §5.2, Table 1 evidence: Negative Gradient achieves WER-F of 5.0% and KL divergence maximization achieves WER-F of 47.2%, compared to 2.4% for TGU. confidence: high relevance: medium - claim_id: speaker_similarity_alone_insufficient_to_verify_unlearning role: supports claim: Speaker similarity metrics alone are insufficient to verify effective machine unlearning in generative speech models, as consistent failure patterns can yield low similarity scores without achieving the randomness needed to resist voice reconstruction. source: §4.3, §5.2, Table 1 evidence: Negative Gradient and KL methods achieve low SIM-F scores (0.402 and 0.179) but spk-ZRF-F values of 0.842 and 0.810, at or below the original model's baseline of 0.846, revealing patterned outputs that could be reverse-engineered; TGU achieves both low SIM-F (0.169) and elevated spk-ZRF-F (0.871). confidence: high relevance: high - claim_id: pretrained_model_as_own_teacher_for_unlearning role: supports claim: A pre-trained generative model can serve as its own teacher for unlearning by providing diverse speaker-identity targets, eliminating the need for aligned cross-speaker audio pairs. source: §4.2 evidence: TGU generates per-step training targets by running VoiceBox conditioned only on text (no audio prompt), producing a different speaker identity at each Gaussian initialization; these unconditional outputs replace paired cross-speaker audio as forget-set targets. confidence: medium relevance: high limitations: - All experiments are conducted on a single TTS architecture (VoiceBox with mel-spectrogram representation); transferability of TGU to codec-based autoregressive ZS-TTS systems is untested. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2503.11026' published_date: "2025-07-30" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: mavflow_multimodal_conditioning_improves_speaker_consistency role: supports claim: Conditioning a flow matching mel-spectrogram generator on rich multimodal speaker representations produces more consistent speaker identity in zero-shot cross-lingual speech synthesis than injecting a single speaker embedding at the vocoder stage. source: §5.4, Table 1 evidence: MAVFlow achieves an average 36% improvement in speaker similarity over the AV2AV baseline across four language pairs on MuAViC, using an OT-CFM Transformer decoder conditioned on x-vector speaker embeddings plus facial emotion embeddings, while AV2AV uses d-vector conditioning in the vocoder only. confidence: high relevance: high - claim_id: mavflow_mel_quality_propagates_to_face_gen role: supports claim: Higher-quality intermediate mel-spectrogram synthesis propagates benefits to downstream talking-face generation even when the face decoder itself is unchanged. source: §5.5, Table 5 evidence: MAVFlow improves LSE-C by +0.87 and FID by -0.61 relative to AV2AV on LRS3 visual evaluation despite using the same unchanged Wav2Lip face decoder. confidence: medium relevance: low - claim_id: mavflow_visual_alone_insufficient_for_emotion role: complicates claim: Visual emotion conditioning is insufficient on its own to improve emotion expression in synthesized speech and requires concurrent audio speaker conditioning to be effective. source: §5.6, Table 8 evidence: Adding only visual guidance to the CFM model reduces emotion accuracy from 28.66% to 26.83% on CREMA-D; the combination of audio and visual guidance is needed to reach 36.46%. confidence: high relevance: medium - claim_id: mavflow_paralinguistic_linguistic_compatible role: supports claim: Paralinguistic and linguistic generation objectives are compatible in zero-shot cross-lingual speech synthesis; improving speaker fidelity does not require sacrificing translation accuracy. source: §5.4, Tables 1 and 3 evidence: MAVFlow maintains competitive ASR-BLEU scores (26.97 Es-En vs 26.57 for AV2AV) while substantially improving speaker similarity, using the same unit translation module as AV2AV. confidence: high relevance: high - claim_id: mavflow_emotion_recognition_gap_unsolved role: complicates claim: Emotion recognition accuracy in synthesized cross-lingual speech remains far below ground-truth levels even with multimodal conditioning, indicating that paralinguistic preservation is an unsolved challenge. source: §5.4, Table 2; §5.6, Table 7 evidence: MAVFlow achieves 36.46% audio emotion accuracy versus a ground-truth upper bound of 81.95% on CREMA-D, even with dual audio-visual guidance in the CFM decoder. confidence: high relevance: low limitations: - The unit extractor and unit-to-unit translation modules are adopted unchanged from AV2AV; improvements in semantic translation quality require addressing those upstream components separately. - Evaluation is limited to five European languages with English as the target; generalization to typologically distant language pairs is untested. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.ccl-1.77 published_date: "2025-08-01" entry_date: '2026-07-26' year: 2025 venue: workshop task: - TTS architecture: - diffusion relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - diffusion_voice_cloning claims: - claim_id: hfsdv2c_face_features_sufficient_speaker_signal role: supports claim: Cross-modal facial features can provide sufficient speaker identity signal for zero-shot voice generation without reference audio. source: §3.2, Table 1, Table 2 evidence: The cross-modal biometric unseen speaker modelling module maps a face image to a speaker embedding via ResNet50 and a trainable visual network; Id.Acc of 66.87/68.99 on LRS2/GRID exceeds all baselines that require reference audio at inference. confidence: medium relevance: high - claim_id: hfsdv2c_diffusion_improves_prosodic_diversity role: supports claim: Diffusion-based denoising in a multimodal TTS/dubbing pipeline improves prosodic diversity relative to deterministic autoregressive or attention-based baselines. source: §4.2.4, Figure 4 evidence: Running HFSD-V2C 10 times per speaker produces diverse F0 contours capturing individual accent patterns, while Neural Dubber and HPMDubbing produce fixed prosodic distributions. confidence: medium relevance: medium - claim_id: hfsdv2c_hierarchical_prosody_improves_emotion role: supports claim: Hierarchical multimodal prosody modelling (video, audio, and text levels jointly) improves emotion accuracy over methods relying on fewer modalities. source: §4.2.1, §4.2.5, Table 1, Table 2 evidence: Emo.Acc reaches 65.63 on LRS2 and 66.32 on GRID, against 61.46 and 63.66 for HPMDubbing; the ablation (w/o UP) drops Emo.Acc to 22.08/27.64, confirming the prosody module's role. confidence: high relevance: low - claim_id: hfsdv2c_biometric_embeddings_dont_fully_replace_reference role: complicates claim: Zero-shot visual voice cloning accuracy remains significantly below ground-truth speaker identity, indicating that cross-modal biometric embeddings do not fully replace reference audio. source: §4.2.1, Table 1 evidence: GT Id.Acc on LRS2 is 91.52 vs. HFSD-V2C's 66.87; GT MOS is 4.72 vs. 4.29 for HFSD-V2C, a gap of 0.43 that persists after hierarchical multimodal conditioning. confidence: high relevance: high limitations: - The evaluation is restricted to LRS2 and GRID, both containing constrained speaking styles; the subjective MOS is collected on only 20 clips rated by 20 evaluators, a limited sample for drawing robust conclusions. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: 2025.icnlsp-1.34 published_date: "2025-08-01" entry_date: '2026-07-26' year: 2025 venue: workshop task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: preference_alignment_via_dpo_can_match_the_perceptual role: supports claim: Preference alignment via DPO can match the perceptual quality of supervised fine-tuning on human-labeled TTS data when preference pairs are constructed automatically from model-generated samples. source: §3.5, Table 1, Table 2, Table 5 evidence: 'On VCTK, DPO fine-tuning with LLM-augmented texts (V4) achieves WER 0.013, SS 0.482, and UTMOS 4.108 from the base xTTSv2 checkpoint, outperforming classical fine-tuning on human-labeled data (V1: WER 0.041, SS 0.500, UTMOS 3.685) on intelligibility and naturalness. CMOS evaluation confirms no statistically significant perceptual difference (all CM p > 0.30).' confidence: high relevance: low - claim_id: the_initialisation_point_before_dpo_fine_tuning_critically role: complicates claim: The initialisation point before DPO fine-tuning critically determines outcome quality, with checkpoint diversity being a more important factor than label quality of the preceding supervised stage. source: §3.6, Table 1, Table 2 evidence: DPO applied from the ASR-fine-tuned checkpoint (L5/V5) consistently underperforms DPO from the base model (L3/V3), with V5 reaching WER 0.273 versus V3's WER 0.014. The explanation offered is that standard fine-tuning narrows generation diversity, leaving insufficient variation for DPO preference signal. confidence: high relevance: low - claim_id: augmenting_unlabeled_tts_training_data_with_llm_generated role: supports claim: Augmenting unlabeled TTS training data with LLM-generated texts improves synthesis naturalness when the dataset's own texts are structurally mismatched with natural language. source: §3.5, Table 1, Table 2 evidence: On VCTK, which contains newspaper sentences well-aligned with LLM output, adding 15,000 LLM-generated texts (V4 vs. V3) raises UTMOS from 4.009 to 4.108. On LJSpeech, which contains audiobook fragments (partial sentences), the improvement is smaller and WER increases, attributed to domain mismatch. confidence: high relevance: low - claim_id: speaker_similarity_is_the_most_difficult_quality_dimension role: complicates claim: Speaker similarity is the most difficult quality dimension to preserve when fine-tuning TTS on ASR-transcribed or automatically constructed data, even when naturalness and intelligibility are maintained. source: §3.5, §3.7, Table 4, Table 5 evidence: Human-labeled fine-tuning (L1/V1) achieves the highest SS scores across both datasets (SS 0.481 on LJSpeech, 0.500 on VCTK). DPO from the base model recovers intelligibility and naturalness but trails on SS. Statistical significance testing shows SS degradation is significant for checkpoint-initialised DPO (Exp 6 vs. Exp 1/2, p < 0.001), while naturalness differences are not significant. confidence: high relevance: high limitations: - All experiments use xTTSv2 as the sole base model; it is unknown whether the same training recipe and checkpoint-initialisation finding generalise to other LM-based TTS architectures. Evaluation covers English only despite xTTSv2 being multilingual. - The comparison between fine-tuning from scratch on ASR-labeled versus human-labeled data is not performed; the human-labeled baseline starts from the same pre-trained xTTSv2 checkpoint, not a randomly initialised model, which limits conclusions about the true cost of label-free training. CMOS subgroup sample sizes (as few as 5 per accent in some comparisons) are too small for reliable accent-level conclusions. The bias implications of using ASR and LLM components for underrepresented speaker groups are acknowledged but not empirically quantified. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.22746' published_date: "2025-08-01" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: composing_autoregressive_generation_across_fixed_length_token_blocks role: supports claim: Composing autoregressive generation across fixed-length token blocks with parallel flow-matching denoising within each block can simultaneously provide KV-cache efficiency and bidirectional contextual refinement. source: §3.1, Table 3 evidence: Within each chunk, flow-matching denoising runs in parallel with bidirectional self-attention, exploiting full chunk context to refine all 25 tokens simultaneously. confidence: high relevance: low - claim_id: neural_codecs_using_finite_scalar_quantisation_can_preserve role: supports claim: Neural codecs using finite scalar quantisation can preserve speaker similarity and intelligibility at frame rates (12.5 Hz) where STFT-based vocoders suffer significant quality degradation. source: §4.3.3, Table 4 evidence: This demonstrates that FSQ-based discrete tokens can match or exceed STFT-based reconstruction at rates where the stationarity assumption underlying STFT begins to break down. confidence: high relevance: high - claim_id: continuous_denoising_models_can_implicitly_classify_discrete_token role: supports claim: Continuous denoising models can implicitly classify discrete token targets through appropriate embedding design, without requiring a separate discrete language model head. source: §3.1 evidence: This unification of discrete and continuous feature modeling avoids the instability of low-frame-rate mel spectrograms and simplifies downstream training without requiring a separate discrete language model head. confidence: high relevance: low - claim_id: reducing_the_token_frame_rate_is_a_more role: supports claim: Reducing the token frame rate is a more tractable path to low-latency hybrid AR-diffusion TTS than increasing diffusion step efficiency alone, given the quadratic scaling of self-attention with sequence length. source: §3.1, §4.3.2 evidence: For diversity, Dragon-FM variants achieve Fréchet Audio Distance (FAD) scores of 1.9–2.6 across chunk size and flow-matching step configurations (Table 2), with shorter chunks (1s) and more FM steps generally yielding lower FAD. confidence: high relevance: low limitations: - No MOS or SMOS listening test results are reported. All quality comparisons use SPK-SIM, WER, and FAD on an internal podcast dataset. The absence of subjective evaluation and fair comparison against published baselines (VALL-E 2, E2 TTS, NaturalSpeech 3) on a public benchmark makes it impossible to independently verify naturalness claims. - The model is trained and evaluated on English podcast data only. Generalisation to other languages, controlled studio-quality TTS, and expressive speech domains is untested. The podcast use-case naturally emphasises diversity and disfluency tolerance over precise prosody control, so the evaluation protocol may not transfer to production TTS settings. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.02038' published_date: "2025-08-04" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - VC architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: marco_diffmeans_emotion_embedding_generalizes role: supports claim: Difference-in-means emotion embeddings, computed from paired emotional and neutral speech, enable speaker-independent emotion representations that generalise across speakers. source: §2.2 evidence: Marco-Voice's Rotational Emotion Embedding aggregates the normalized difference between emotional and neutral speech from the same speaker over 10 pairs to produce a speaker-independent emotion direction vector, which is then injected into both the LM and flow-matching module for any speaker. confidence: medium relevance: medium - claim_id: marco_orthogonality_enables_independent_control role: supports claim: Enforcing orthogonality between speaker and emotion embedding spaces via a cross-dot-product loss enables independent control of voice identity and emotional style without degrading either. source: §2.3, Table 1 evidence: Marco-Voice v4 (with cross-orthogonal constraint) achieves speaker similarity 0.8275 vs CosyVoice2's 0.605 and emotional expression MOS 4.225 vs CosyVoice2's 3.240, indicating both identity and style are preserved simultaneously under the orthogonality regularization. confidence: high relevance: high - claim_id: marco_unified_modeling_beats_separate_systems role: supports claim: Unified joint modeling of voice cloning and emotion control produces higher speaker similarity and emotional naturalness than systems that address the two capabilities separately. source: §4.1, §4.2, Tables 1-2 evidence: Marco-Voice achieves overall satisfaction 4.430 vs CosyVoice2's 3.330 and is preferred over CosyVoice1/CosyVoice2 in 60%/65% of A/B comparisons. confidence: medium relevance: high - claim_id: marco_expressive_fillers_inflate_wer role: complicates claim: Expressive emotional TTS systems that include naturalistic vocalized fillers exhibit higher word error rates on clean-transcript benchmarks, even when perceived speech quality is high. source: §4.4, Table 4-5 evidence: Marco-Voice v4 shows WER 17.6 on AISHELL-3 (higher than CosyVoice1's 3.0) despite high subjective quality, attributed to vocalized fillers and prosodic elements not present in the reference transcript. confidence: medium relevance: medium - claim_id: marco_gender_imbalance_emotion_recognition role: complicates claim: Gender imbalance in emotional speech training data produces systematic differences in emotion recognition accuracy, with male speakers underperforming female speakers across most emotion categories. source: §4.4, Figure 6 evidence: Emotion recognition accuracy analysis on Marco-Voice v4 output shows systematic accuracy gaps between male and female speakers across most of the 7 emotion categories tested. confidence: medium relevance: medium limitations: - Requires paired emotional/neutral speech data from the same speaker for emotion embedding extraction, which limits scalability and language coverage; model size not reported. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2504.10352' published_date: "2025-08-05" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: pseudo_autoregressive_generation_which_commits_spans_left_to role: supports claim: Pseudo-autoregressive generation, which commits spans left-to-right within a bidirectional masked transformer, achieves constant inference steps regardless of target speech duration while maintaining temporal coherence. source: §3, §5.4 evidence: This creates span-level causal ordering (earlier spans committed before later ones) with O(1) inference steps regardless of target duration, since the number of steps equals 1/r' (a constant ratio of the target length). confidence: high relevance: low - claim_id: a_model_trained_on_580_hours_of_english role: supports claim: A model trained on 580 hours of English speech can match or exceed the intelligibility of NAR flow-matching systems trained on 100,000+ hours when temporal ordering is explicitly enforced during generation. source: §5.3, Table 1 evidence: PALLE two-stage achieves WER-H 2.83 and SIM-o 0.716 on the cross-sentence task of LibriSpeech test-clean — the best WER among all systems evaluated, outperforming F5-TTS (WER-H 2.77, SIM-o 0.705), E2-TTS (WER-H 2.92, SIM-o 0.756), and MaskGCT (WER-H 4.22, SIM-o 0.756), all of which are trained on 100k+ hours of data. confidence: high relevance: low - claim_id: confidence_guided_iterative_nar_refinement_of_an_initial role: supports claim: Confidence-guided iterative NAR refinement of an initial PAR generation substantially reduces word error rate with only a small number of additional inference steps. source: §5.5, Figure 4 evidence: This creates span-level causal ordering (earlier spans committed before later ones) with O(1) inference steps regardless of target duration, since the number of steps equals 1/r' (a constant ratio of the target length). confidence: high relevance: low - claim_id: temporally_unordered_nar_generation_produces_higher_alignment_errors role: supports claim: Temporally unordered NAR generation produces higher alignment errors than span-level causal generation across both continuation and cross-sentence evaluation tasks. source: §5.4, Table 3 evidence: This creates span-level causal ordering (earlier spans committed before later ones) with O(1) inference steps regardless of target duration, since the number of steps equals 1/r' (a constant ratio of the target length). confidence: high relevance: low - claim_id: separate_model_capacity_for_each_generation_stage_is role: supports claim: Separate model capacity for each generation stage is necessary; unifying PAR and NAR refinement into a single multitask model degrades cross-sentence intelligibility by approximately 20%. source: §5.5 evidence: PALLE two-stage achieves WER-H 2.83 and SIM-o 0.716 on the cross-sentence task of LibriSpeech test-clean — the best WER among all systems evaluated, outperforming F5-TTS (WER-H 2.77, SIM-o 0.705), E2-TTS (WER-H 2.92, SIM-o 0.756), and MaskGCT (WER-H 4.22, SIM-o 0.756), all of which are trained on 100k+ hours of data. confidence: high relevance: low limitations: - PALLE is evaluated only on English (LibriTTS). The 100-step inference (with 7 refinement steps) may still be too slow for the most latency-sensitive streaming applications despite the 10x speedup. Duration estimation for the cross-sentence task relies on a simple linear heuristic; errors in duration estimation lead to modest quality degradation (WER-H 2.83 vs. 2.62 with GT duration). The shared architecture between stage one and stage two (joint multitask fine-tuning) causes stage two loss to degrade stage one performance, suggesting that separate model capacity is required. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.03543' published_date: "2025-08-05" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: emosteer_trainingfree_steering_competitive role: supports claim: Training-free inference-time activation steering can achieve emotion control in flow-matching TTS models that is competitive with or superior to systems explicitly trained for emotion conditioning. source: §4.2, Table 1 evidence: EmoSteer-TTS + F5-TTS achieves E-SIM 0.280 and WER 2.79 in-distribution, comparing favourably to unsteered CosyVoice2 (E-SIM 0.240) and the trained baseline EmoVoice (E-SIM 0.205, WER 5.31), while requiring no additional training. confidence: high relevance: medium - claim_id: emosteer_emotion_linear_in_dit_activations role: supports claim: Pre-trained flow-matching TTS models encode emotion as approximately linear directions in DiT activation space, enabling vector arithmetic over multiple emotion categories. source: §3, §4.6 evidence: Steering vectors are constructed as normalized differences between emotional and neutral activation means at a specific DiT layer, and adding, subtracting, or interpolating these vectors during inference reliably produces emotion conversion, interpolation, erasure, and multi-emotion blending across F5-TTS, E2-TTS, and CosyVoice2. confidence: high relevance: medium - claim_id: emosteer_emotion_concentrated_in_topk_tokens role: supports claim: Emotion-bearing information in TTS DiT activations is concentrated in a subset of token positions identifiable via speech emotion recognition scores, and selective steering of these positions outperforms global activation modification. source: §3.3, §4.6 evidence: An emotion2vec model scores each token position by emotional relevance and only top-k positions are steered; this topk selective steering avoids distorting prosodically neutral segments like silences or unstressed phonemes. confidence: medium relevance: low - claim_id: emosteer_generalizes_across_architectures role: supports claim: Activation steering for emotion control generalises across model architectures and scales without retuning, producing consistent results on F5-TTS (22 layers), E2-TTS (8 layers), and CosyVoice2 (56 layers). source: §4.2, §4.5 evidence: The same steering methodology, applied every 3-5 layers depending on model depth, produces consistent emotion conversion results across three DiT-based flow-matching TTS models of substantially different layer counts. confidence: high relevance: medium - claim_id: emosteer_steering_costs_speaker_similarity role: complicates claim: Training-free emotion manipulation via activation steering incurs a moderate speaker similarity cost relative to the unsteered baseline, indicating residual speaker-emotion entanglement in DiT representations. source: §4.2, Table 1 evidence: Speaker similarity drops from S-SIM 0.66 (unsteered F5-TTS) to 0.59 (EmoSteer + F5-TTS), suggesting some speaker-emotion coupling in the activations being steered. confidence: medium relevance: high limitations: - The steering vectors are derived from difference-in-means, which assumes linearity of emotional representations in DiT activations, a plausible but unverified assumption. - The method requires access to DiT intermediate activations, ruling out fully black-box deployment. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.14049' published_date: "2025-08-05" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: large_multilingual_tts_systems_built_on_semantic_token role: supports claim: Large multilingual TTS systems built on semantic token intermediaries transfer to low-resource languages more readily than end-to-end spectrogram models. source: §2.1, §5.1 evidence: For semantic representations, the system uses a pretrained XLS-R wav2vec2.0 model (1B parameters, trained on 436K hours across 128 languages) with k-means clustering to produce a 10,000-token discrete phoneme dictionary at approximately 25 tokens per second. confidence: high relevance: low - claim_id: decoupling_the_text_to_semantic_and_semantic_to role: supports claim: Decoupling the text-to-semantic and semantic-to-acoustic stages enables independent training and simplifies the addition of new languages without full system retraining. source: §2, §4.1 evidence: 'MahaTTS-v2 follows a two-stage pipeline: M1 converts text to semantic tokens, and M2 converts semantic tokens to an audio waveform — the two stages are trained independently with no joint optimisation required.' confidence: high relevance: low - claim_id: flow_matching_is_a_viable_replacement_for_diffusion role: supports claim: Flow matching is a viable replacement for diffusion in the acoustic generation stage of two-stage TTS pipelines, maintaining competitive quality at lower training complexity. source: §2.3, §5.1 evidence: M2 is a 300M parameter conditional flow matching model (Matcha-TTS-inspired) that maps semantic tokens to mel spectrograms. confidence: high relevance: low - claim_id: intelligibility_in_low_resource_languages_with_limited_training role: complicates claim: Intelligibility in low-resource languages with limited training data remains markedly worse than high-resource languages within the same multilingual system. source: §5.1, Table 2 evidence: 'Performance is uneven: results on Assamese (42%) and Odia (73%) are substantially weaker than the leading baselines, suggesting that low-resource languages with limited training hours remain difficult even within this framework.' confidence: high relevance: low limitations: - The evaluation relies exclusively on WER over 10 sentences per language with no MOS, SMOS, or naturalness scores. This makes it impossible to assess audio quality, expressiveness, or speaker similarity relative to baselines — core dimensions for a TTS system. - 'English dominates the training set at 58%, which may explain strong English results but raises questions about whether true cross-lingual transfer or data dominance is responsible. The system lacks prosody and pace control conditioning in M1, which the authors flag as future work. Zero-shot speaker fidelity for M2 is acknowledged as limited compared to infilling-based approaches like Seamless. Fine-tuning introduces hallucination that requires careful intervention (freezing classification heads only), suggesting the LM component is sensitive to distribution shift. Languages with fewer training hours (Assamese: 48h, Dogri: 8h, Rajasthani: 20h) show substantially weaker results.' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.04141' published_date: "2025-08-06" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: generating_semantic_and_acoustic_tokens_simultaneously_in_a role: supports claim: Generating semantic and acoustic tokens simultaneously in a single autoregressive forward pass, rather than cascading semantic prediction before acoustic prediction, reduces word error rate and improves naturalness in zero-shot TTS. source: §V.A, Table I evidence: Crucially, both token types are generated simultaneously in a single forward pass, maintaining temporal alignment while preserving content independence. confidence: high relevance: high - claim_id: combining_specialist_ssl_models_for_distinct_speech_attributes role: supports claim: Combining specialist SSL models for distinct speech attributes (semantic content, acoustic texture, speaker identity) as frozen feature extractors enables more effective token-level disentanglement than using a single encoder for all attributes. source: §III.A, Tables III–IV evidence: 'A custom tokenizer combines three frozen SSL models: Wav2Vec 2.0 (semantic features), BEATs (acoustic features), and CAM++ / Campplus (speaker features).' confidence: high relevance: high - claim_id: a_hybrid_ar_nar_design_that_enforces_independence role: supports claim: A hybrid AR+NAR design that enforces independence at the coarse token level and interdependence at the fine-grained level distributes modeling complexity more effectively than an all-AR or all-NAR approach. source: §V.B, Tables III–IV evidence: Crucially, both token types are generated simultaneously in a single forward pass, maintaining temporal alignment while preserving content independence. confidence: high relevance: low - claim_id: parallel_semantic_acoustic_modeling_improves_naturalness_and_intelligibility role: supports claim: Parallel semantic-acoustic modeling improves naturalness and intelligibility without fully closing the speaker similarity gap relative to systems with dedicated speaker embedding refinement. source: §V.A, Table I evidence: On speaker similarity (SMOS), Parallel GPT (4.08/3.92) is closely competitive with MaskGCT (4.05/3.91) but trails CosyVoice (4.23/4.11) — a gap the authors attribute to CosyVoice's specialized speaker embedding refinement. confidence: high relevance: high limitations: - Speaker similarity lags behind CosyVoice (SMOS gap ~0.15–0.2 on English), suggesting the parallel architecture does not yet fully leverage speaker conditioning. UTMOS scores, while competitive, do not reach ground-truth levels. Model size is not reported, making compute comparisons difficult. The internal Chinese dataset and preprocessing pipeline (Emilia + NCSSD) are not publicly released, limiting reproducibility on that front. Extending the framework to prosody control, emotion conditioning, or cross-lingual voice conversion is not explored. The subjective decoupling evaluation (Section V.C) relies on 90% evaluator agreement rather than a standardized metric, leaving quantitative disentanglement assessment as an open question. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.04195' published_date: "2025-08-06" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: paralinguistic_tokens_enable_joint_asr_transcription role: supports claim: Treating paralinguistic vocalizations as inline vocabulary tokens in ASR decoding enables joint transcription of lexical and non-verbal content within a single unified sequence, without requiring a separate event detection module. source: §Paralinguistic Aware Speech Recognition evidence: SenseVoice-Small, fine-tuned with paralinguistic events as special output tokens, achieves in-domain F1 0.83 and open-domain F1 0.85 on the paralinguistic-aware ASR task, outperforming Paraformer, Whisper, and Qwen-Audio. confidence: medium relevance: medium - claim_id: autolabeled_data_matches_human_annotated_for_paralinguistic_tts role: supports claim: Automatically annotated data at scale can match or exceed the training effectiveness of smaller human-annotated sets for paralinguistic-enhanced TTS, measured by CER and perceptual quality. source: §Main Results, Table 5 evidence: Fine-tuning CosyVoice2 on the full auto-labeled corpus (573h) achieves 7.51% in-domain CER (vs. 8.61% on human data alone) and UTMOS 2.67 (vs. 2.54), with speaker similarity preserved at 0.700. confidence: medium relevance: high - claim_id: tts_finetuning_enables_tokenlevel_nonverbal_control role: supports claim: Fine-tuning zero-shot TTS models with paralinguistic vocabulary extensions enables token-level control over non-verbal vocalization placement without material degradation of lexical quality or speaker similarity. source: §Paralinguistic-enhanced TTS Experiments, Table 5 evidence: CosyVoice2 fine-tuned with paralinguistic vocabulary extensions maintains 7.51% CER and 0.700 speaker similarity while inserting non-verbal tags at arbitrary token positions in the target sequence. confidence: medium relevance: high - claim_id: listener_preference_strong_despite_low_tag_recall role: complicates claim: Listener preference for para-enhanced TTS is strong even when paralinguistic tag recall remains below 65%, suggesting that the mere presence of contextually appropriate non-verbal cues improves perceived naturalness. source: §Human Evaluation, Table 6 evidence: Human evaluation finds listener preference for para-enhanced output at win rates of 78.7% (CosyVoice) and 75.4% (CosyVoice2), with naturalness MOS of 3.9-4.0, even though paralinguistic tag recall reaches only 61.9%. confidence: medium relevance: medium limitations: - Paralinguistic tag recall in TTS reaches only 61.9%, leaving substantial room for improvement in ensuring specified non-verbal cues are faithfully rendered; the paper evaluates TTS on its own in-domain/open-domain splits rather than a shared benchmark. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.04585' published_date: "2025-08-06" entry_date: '2026-07-26' year: 2025 venue: ACM MM task: - TTS - SCA architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: matching_the_token_rates_of_speech_and_facial role: supports claim: Matching the token rates of speech and facial landmark codecs enables frame-level synchronisation between synthesised speech and talking-face animations without post-hoc alignment. source: §4.1.3, §4.2 evidence: LmkCodec operates at 25 Hz with a codebook of size 1000, matching the speech tokenizer's rate and enabling frame-level alignment. confidence: high relevance: low - claim_id: llm_based_joint_prediction_of_interleaved_speech_and role: supports claim: LLM-based joint prediction of interleaved speech and visual tokens in dialogue context outperforms cascaded speech-then-video generation on both emotional accuracy and lip synchronisation. source: §6.2, §6.3, Table 2, Table 3 evidence: 'Built on Qwen2.5-0.5B, EVSLM takes the serialized multimodal context and sequentially predicts: (1) the target emotion token, (2) interleaved facial expression and speech tokens via a bimodal hard alignment strategy.' confidence: high relevance: low - claim_id: including_visual_dialogue_history_talking_face_animations_of role: supports claim: Including visual dialogue history (talking-face animations of prior turns) improves emotion accuracy and speaker consistency in conversational speech synthesis relative to audio-text-only context. source: §6.4, Table 4 evidence: 'UniTalker frames the problem as Conversational Speech-Visual Synthesis (CSVS): given the multimodal history of a dialogue (text, speech, talking-face animations, speaker identity, and emotion labels for each prior turn) and the current turn''s speaker and text, predict the target utterance''s emotion, speech, and talking-face animations jointly.' confidence: high relevance: high - claim_id: low_rate_discrete_tokenisation_of_facial_landmarks_1 role: supports claim: Low-rate discrete tokenisation of facial landmarks (1 token per frame) is more effective for LLM contextual modelling than higher-rate representations, even at a cost in geometric reconstruction fidelity. source: §6.1, §6.2, Table 1, Table 2 evidence: The LmkCodec's FSQ design with 1 token per frame also proves better than the 8-token-per-frame GRVQ variant, demonstrating that shorter sequences aid LLM contextual modeling even at some reconstruction cost. confidence: high relevance: low - claim_id: emotion_guided_conditioning_of_the_speech_renderer_including role: supports claim: Emotion-guided conditioning of the speech renderer, including predicted emotion labels as explicit conditioning, improves measured emotional expressiveness over systems that rely on implicit contextual inference alone. source: §6.4, Table 4 evidence: In speech emotion expressiveness (MOS_SE), the margin is +0.111 over Empatheia. confidence: high relevance: low limitations: - The Talking-face Animations Renderer (EchoMimic) is a pre-trained third-party module that receives no additional fine-tuning in this pipeline. Its outputs are constrained by the quality ceiling and biases of the EchoMimic base model, limiting the paper's ability to attribute animation quality gains to UniTalker specifically vs. the renderer. - 'Rendering latency is notable: speech generation takes approximately 2 seconds and animation rendering takes approximately 5 seconds per 25 frames on an RTX 4080 with 32 GB RAM, making the system unsuitable for real-time interaction in its current form. The paper acknowledges this and lists streaming optimisation as future work.' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.04996' published_date: "2025-08-07" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - VC architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: refvc_ssl_expressiveness_needs_mitigation role: complicates claim: SSL features improve paralinguistic expressiveness in voice conversion but introduce timbre leakage and noise sensitivity that require explicit mitigation. source: §I, §II.B evidence: WavLM-derived SSL features carry prosody and emotion but are shown to leak source timbre and degrade under noise unless mitigated by the paper's random erasing strategy. confidence: high relevance: high - claim_id: refvc_random_erasure_reduces_reliance role: supports claim: Random feature erasure at training time can reduce a model's over-reliance on information-rich but noise-sensitive representations without information bottleneck machinery. source: §II.B evidence: Removing random erasing collapses SECS to 0.5248 (clean) / 0.4384 (noisy) and raises CER to 18.64% on the noisy set, versus 0.8253/0.8031 SECS and 8.03% CER with erasing applied. confidence: high relevance: low - claim_id: refvc_implicit_alignment_improves_noise_robustness role: supports claim: Implicit alignment borrowed from non-autoregressive TTS can improve noise robustness in voice conversion by preventing the model from over-reconstructing noise-carrying source frames. source: §II.C evidence: Removing the E2TTS-inspired implicit alignment (blank-frame padding fusion feeding the flow-matching DiT) degrades SMOS substantially, from 3.98 to 3.51 on the clean set. confidence: high relevance: medium - claim_id: refvc_shortcut_models_reduce_fm_steps role: supports claim: Shortcut Models reduce flow-matching inference steps by an order of magnitude with only marginal quality loss in voice conversion. source: §II.D, Table I evidence: REF-VC's 4-step Shortcut Model configuration achieves NMOS 3.89, CER 8.79, and SECS 0.7919 on the noisy set, remaining competitive with the 32-step baseline (NMOS 3.68, CER 8.03, SECS 0.8031) and with Seed-VC's 32-step baseline. confidence: high relevance: medium - claim_id: refvc_asr_ssl_complementary role: supports claim: 'ASR-based bottleneck features and SSL representations are complementary: the former provides noise-robust linguistic content, the latter contributes paralinguistic fidelity that ASR training suppresses.' source: §I, §II.A evidence: REF-VC fuses Wenet ASR bottleneck features (noise-robust, content-only) with WavLM SSL features (paralinguistic-rich, noise-sensitive) as dual conditioning to the flow-matching DiT decoder. confidence: high relevance: low limitations: - 'Evaluation scale is small: 100 clean and 50 noisy test utterances is insufficient to draw strong conclusions about generalisation across noise types or speaking styles.' - The model cannot synthesise arbitrarily long utterances due to a maximum-length constraint from the implicit alignment mechanism. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.06262' published_date: "2025-08-08" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: token_verification_is_necessary_for_multi_token_prediction role: supports claim: 'Token verification is necessary for multi-token prediction to be effective in autoregressive TTS: without it, WER increases from 3.07% to 14.37% and speaker similarity drops from 0.570 to 0.463.' source: §V, Table III evidence: Multi-Token Prediction (MTP) Modules. confidence: high relevance: high - claim_id: plug_and_play_mtp_modules_trained_on_a role: supports claim: Plug-and-play MTP modules trained on a modest dataset can accelerate a frozen autoregressive TTS backbone by up to 1.48x without sacrificing generation quality on standard benchmarks. source: §IV.B, Table I evidence: The backbone is never updated, ensuring no degradation of the original generation quality. confidence: high relevance: low - claim_id: under_quality_maximizing_inference_settings_mtp_with_verification role: supports claim: Under quality-maximizing inference settings, MTP with verification can improve intelligibility beyond the backbone baseline, likely due to extended look-ahead context from the cascaded hidden states. source: §IV.A, Table I evidence: At topk=100 (quality-maximizing setting), Llasa+ achieves WER 2.499% and SIM 0.575 on Seed-TTS-eval-en — both better than the Llasa baseline (WER 3.22, SIM 0.572) — with 1.42x speedup. confidence: high relevance: low - claim_id: converting_a_non_causal_codec_decoder_to_a role: supports claim: Converting a non-causal codec decoder to a causal streaming architecture via lightweight fine-tuning preserves approximately 95% of reconstruction quality, making streaming reconstruction viable without full retraining. source: §IV.C, Table II evidence: The original XCodec2 transformer decoder is converted to a causal architecture to support streaming waveform reconstruction. confidence: high relevance: low - claim_id: attention_based_mtp_modules_substantially_outperform_mlp_based role: supports claim: Attention-based MTP modules substantially outperform MLP-based equivalents of similar parameter count in both intelligibility and speaker similarity for TTS acceleration. source: §IV.B, Table I evidence: 'Ablation studies confirm that: (1) attention-based MTP layers substantially outperform MLP replacements (WER 2.745 vs. 4.375 for MTP1 at topk=500); (2) the DeepSeek-V3-style input (feeding ground-truth tokens to MTP) underperforms the proposed hidden-state-only input due to train/inference mismatch; (3) a third MTP module provides less than 10% additional speedup at disproportionate cost.' confidence: high relevance: high limitations: - Evaluation is English-only (LibriTTS training, Seed-TTS-eval-en test), and speaker generalization to out-of-distribution languages or accents is untested. The verification overhead (one additional LM forward pass for verification at each step) partially offsets the MTP speedup, especially at strict topk values. The 1.48x figure assumes topk=500, which allows some quality degradation; the fully lossless speedup (topk=100) is closer to 1.42x. Scaling MTP to larger models (Llasa-3B, Llasa-8B) is not investigated. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.07302' published_date: "2025-08-10" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: xemorag_language_agnostic_embeddings_retrieval_signal role: supports claim: Language-agnostic emotional embeddings from pre-trained models can serve as a reliable cross-lingual retrieval signal for zero-shot emotion transfer in TTS. source: §III.C, §IV.C evidence: Emo2Vec embeddings extracted from a Chinese reference utterance are used with K-means clustering over a pool of 8K Thai utterances to retrieve emotionally matched Thai prompts, achieving 86.3% retrieval accuracy. confidence: medium relevance: high - claim_id: xemorag_rag_reduces_accent_artefacts role: supports claim: Retrieval-augmented prompting reduces foreign-accent artefacts in cross-lingual emotional speech synthesis more effectively than direct prosody transfer between typologically distant languages. source: §III.C, §IV.E evidence: The RAG module retrieves native Thai emotional prompts rather than transferring Chinese prosody directly; removing the RAG module drops EMOS from 4.65 to 4.25. confidence: medium relevance: medium - claim_id: xemorag_fm_alignment_improves_speaker_prosody role: supports claim: Flow-matching alignment between discrete codec tokens and mel-spectrograms improves speaker identity preservation as well as prosodic naturalness in multilingual synthesis. source: §III.B, §IV.E evidence: Removing the flow-matching alignment module (a transformer-based 1D U-Net mapping X-Codec2 tokens to mel-spectrograms) drops EMOS from 4.65 to 4.20 and speaker similarity sharply from 0.89 to 0.74. confidence: high relevance: high - claim_id: xemorag_clustering_retrieval_scales_better role: supports claim: Clustering-based retrieval strategies over large emotional speech pools maintain higher accuracy and lower latency than exhaustive cosine similarity search as pool size grows. source: §IV.D, Table II evidence: Clustering-based retrieval at 8K utterances achieves 86.3% accuracy at 1.67s latency, outperforming direct cosine similarity search which degrades to 82.4% accuracy at 2.85s latency at the same scale. confidence: medium relevance: low - claim_id: xemorag_twostage_finetuning_effective role: supports claim: Two-stage fine-tuning, first on phonetics, then on expressiveness, enables effective emotion adaptation in low-resource target languages from a strong multilingual foundation model. source: §III.D, §IV.A evidence: The system fine-tunes on 1K hours of non-emotional Thai data before fine-tuning on 60 hours of Thai emotional data, following a standard low-resource adaptation curriculum. confidence: medium relevance: low limitations: - All evaluation is conducted on internal, non-public datasets with a narrow test configuration (one Chinese speaker, proprietary Thai data); there is no standard benchmark or released code. - The evaluation covers only the Chinese-to-Thai direction; the claimed language-agnostic generalisability is stated as future work rather than demonstrated. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.07426' published_date: "2025-08-10" entry_date: '2026-07-26' year: 2025 venue: ASRU task: - TTS architecture: - transformer-enc-dec - GAN - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - encoder_decoder_voice_cloning claims: - claim_id: accenttts_label_quality_over_volume role: supports claim: Accent label quality exerts a larger effect on accent synthesis fidelity than raw data volume, and filtering noisy self-reported labels consistently outperforms using them directly. source: §V-C, Table VI evidence: The geolocation-filtered system achieves DCF 0.571 averaged across all accents, outperforming the unfiltered system's DCF 0.735, which uses the same underlying data volume but with noisy self-reported labels. confidence: high relevance: medium - claim_id: accenttts_vc_augmentation_aids_disentanglement role: supports claim: Speaker-accent disentanglement in low-resource settings benefits from voice conversion-based timbre augmentation, even when the augmented data introduces acoustic degradation. source: §V-D, Table VIII evidence: Applying kNN-VC to diversify speaker timbre within each accent class (converting training utterances to randomly sampled LibriTTS speakers while preserving accent) improves objective accent similarity, though the augmented data itself introduces some acoustic degradation. confidence: medium relevance: high - claim_id: accenttts_metric_human_divergence role: complicates claim: Automatic accent identification metrics and human perceptual evaluations of accent plausibility frequently diverge, limiting the reliability of either metric alone as a benchmark. source: §V-B, §V-E evidence: kNN-VC augmentation improves objective DCF accent-similarity scores but shows unclear or mixed effects in human evaluation, and human evaluators found no clear benefit for augmented versus non-augmented systems in Australian and Scottish evaluations despite objective improvements. confidence: medium relevance: medium - claim_id: accenttts_zeroshot_style_copying_competitive role: supports claim: Large zero-shot TTS systems can produce competitive accented speech through style copying without explicit accent conditioning, often matching or exceeding dedicated accent-controllable models on naturalness. source: §V-B, Table VII evidence: CosyVoice2, using zero-shot style copying without explicit accent conditioning, ranks highest across all four human-evaluated accent sets, including above real utterances in the US accent test. confidence: medium relevance: high - claim_id: accenttts_geolocation_discovery_variable_precision role: complicates claim: Geolocation-based accent label discovery is extensible to languages and accent varieties without existing labeled speech, but precision varies substantially across accent regions depending on broadcast training data coverage. source: §III-A, Table III evidence: Precision on Canada (13.3%), Scotland (9.4%), Germany (15.9%), Ireland (20.4%), and Malaysia (7.1%) is poor, limiting practical applicability of the geolocation-based discovery approach to a subset of the 11 accents studied. confidence: high relevance: medium limitations: - The paper evaluates English accent synthesis only, despite XTTS-v2 supporting multiple languages; the WER gap between the proposed systems and CosyVoice2 (13.1% vs. 7.1%) is notable and unexplained in depth. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.09702' published_date: "2025-08-13" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - evaluation architecture: [] relevance: low evidence_role: - infrastructure current_role: minor method_family: [] claims: - claim_id: quality_mismatch_degrades_zeroshot_generation role: supports claim: Quality mismatch between clean training corpora and real-world inference prompts is a meaningful source of degradation in zero-shot speech generation systems. source: §1 evidence: The paper motivates M3PDB by observing that zero-shot TTS systems trained on clean studio recordings degrade against noisy, incomplete, or out-of-domain inference prompts. confidence: medium relevance: high - claim_id: crosslingual_proxy_prompts_can_outperform_native role: supports claim: Cross-lingual proxy prompts selected by linguistic proximity can produce lower synthesis error than native-language prompts of lower audio quality. source: §3.2, Table 3 evidence: Cross-lingual prompts selected by M3PDB for Italian synthesis via XTTS-v2 yield CER of 2.78%, compared to 4.32% for native Italian prompts and 5.21% for randomly selected French prompts. confidence: medium relevance: medium - claim_id: multimodal_annotation_improves_age_gender_accuracy role: supports claim: Multi-modal annotation that fuses visual and acoustic signals yields substantially higher age and gender labelling accuracy than audio-only or text-only annotation strategies. source: §5.1, Table 2 evidence: The multi-agent annotation system reduces age MAE from 12.54 (SpeechCraft) to 6.22 years and raises gender accuracy from 80.69% to 91.22%. confidence: medium relevance: low - claim_id: curated_database_outperforms_generative_face_to_speech role: supports claim: A curated, high-quality audio database outperforms generative face-to-speech synthesis as a source of speaker prompts when audio quality is the binding constraint. source: §5.3, Table 4 evidence: M3PDB-retrieved audio achieves comparable speaker similarity to Imaginary Voice face-to-speech synthesis (15.14% vs. 16.18% SS) while substantially improving UTMOSv2 (2.69 vs. 2.10). confidence: medium relevance: high - claim_id: cascaded_retrieval_viable_for_latency_constraints role: supports claim: Cascaded, interruptible similarity retrieval is a viable approach to prompt selection under strict latency constraints without dedicated hardware. source: §4.2, §5.4, Table 5 evidence: The online two-stage selection strategy improves SS from 39.53% to 44.88% and CER from 4.21% to 1.17% over using degraded input directly, without specialized hardware. confidence: medium relevance: medium limitations: - No subjective listening tests are reported anywhere in the paper; all quality metrics (UTMOSv2, automatic speaker similarity via cosine embeddings, Emotion2Vec emotion similarity) are automatic predictors. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0047 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: mixing_high_quality_audio_only_data_with_audio role: supports claim: Mixing high-quality audio-only data with audio-visual corpora via contrastive modality alignment substantially improves speech naturalness in face-driven TTS systems. source: §2.3, Table 1 evidence: Ablation confirms that mixing high-quality LibriTTS-R audio data contributes the largest MOS gain (+0.50), followed by style augmentation (+0.26 in MOS), with contrastive learning primarily benefiting face-voice association (SIM metric) rather than naturalness. confidence: high relevance: medium - claim_id: neural_style_transfer_augmentation_of_training_face_images role: supports claim: Neural style transfer augmentation of training face images reduces the domain gap between photorealistic faces and artistic portraits, improving both naturalness and face-matching scores. source: §2.2, Table 1 evidence: 'Style augmentation: During training, input face images are randomly passed through CAST neural style transfer and augmented with gray-scaling and blurring (50% chance).' confidence: high relevance: low - claim_id: in_context_prompting_with_a_user_selected_speech role: supports claim: In-context prompting with a user-selected speech sample enables consistent voice generation across multiple utterances from the same face, addressing the inherent one-to-many ambiguity of face-to-voice mapping. source: §2.4 evidence: For diverse but consistent voice generation, the model uses sampling-based decoding to produce multiple candidate voices from the same face, then accepts a user-selected sample as an in-context prompt to enforce voice consistency across subsequent utterances. confidence: high relevance: high - claim_id: contrastive_pre_training_of_face_and_audio_encoders role: supports claim: Contrastive pre-training of face and audio encoders primarily improves face-voice association rather than overall speech naturalness. source: §3.4.1, Table 1 evidence: Ablation confirms that mixing high-quality LibriTTS-R audio data contributes the largest MOS gain (+0.50), followed by style augmentation (+0.26 in MOS), with contrastive learning primarily benefiting face-voice association (SIM metric) rather than naturalness. confidence: high relevance: medium - claim_id: objective_acoustic_metrics_speaking_rate_si_sdr_c50 role: supports claim: Objective acoustic metrics (speaking rate, SI-SDR, C50, pitch standard deviation) shift predictably with natural language descriptors, confirming that text-driven style control transfers to measurable acoustic properties. source: §3.4.3, Table 3 evidence: Controllability is verified using objective metrics (speaking rate, SI-SDR, C50, pitch std) that shift predictably with descriptive text changes. confidence: high relevance: low limitations: - '- Voice consistency score (VCS) for RV-TTS (3.96) is below the audio-driven YourTTS (4.42), indicating residual inconsistency in pure face-conditioned generation. - The artistic portrait test uses only 20 copyright-free images; broader and more controlled evaluation is needed. - Descriptive text label generation for LRS3/VoxCeleb2 relies on Data-Speech auto-annotation rather than human labels. - Face encoder is ResNet50; more powerful vision-language models may improve face-voice matching. - No multilingual evaluation.' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0305 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - singing - VC architecture: - flow-matching - transformer-enc-dec relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: flow_matching_beats_gan_decoder_svc role: supports claim: Flow matching decoders produce higher audio quality than GAN-based decoders in singing voice conversion when conditioning signal quality is held constant. source: §4.1, §4.2, Table 1, Table 2 evidence: Replacing NeuCoSVC's GAN-based FastSVC decoder with a CFM module improves MCD from 8.634 to 7.220 and MOS-Naturalness from 3.47 to 3.80 on OpenSinger. confidence: high relevance: medium - claim_id: ssl_matching_insufficient_alone_for_timbre_similarity role: refines claim: SSL feature matching prevents timbre leakage in singing voice conversion but is insufficient on its own for high timbre similarity, because target timbre is distributed across the full reference utterance rather than captured by sparse nearest-neighbour retrieval. source: §4.2, Table 1, Table 2 evidence: The SSL-matching-only ablation achieves SSIM 0.709 versus 0.692 for NeuCoSVC (marginal improvement), while adding speaker embeddings with the full DCAM raises SSIM to 0.754. confidence: medium relevance: high - claim_id: crossattention_fusion_improves_timbre_similarity role: supports claim: Cross-attention fusion of speaker embeddings and melody features with shared content queries improves timbre similarity and audio coherence over simple feature concatenation in conditional singing voice conversion. source: §4.2, Table 2 evidence: Removing the DCAM module while retaining speaker embeddings drops SSIM from 0.754 to 0.710 and MCD from 7.220 to 8.129. confidence: high relevance: high - claim_id: oneshot_svc_evaluation_remains_narrow role: complicates claim: One-shot singing voice conversion evaluations remain narrow in scope, limiting the generalisability of reported gains. source: §3.1, §3.4, §5 evidence: Experiments use a single Chinese singing dataset (OpenSinger), 20 samples for subjective evaluation with 15 listeners, and four unseen target speakers; cross-language, multi-domain, or noisy-environment generalisation is explicitly deferred to future work. confidence: medium relevance: medium limitations: - The system is trained and evaluated exclusively on high-quality Chinese singing recorded in a professional studio; performance in noisy environments and cross-language settings is untested. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0319 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: llm_based_zero_shot_tts_systems_are_more role: supports claim: LLM-based zero-shot TTS systems are more sensitive to noise in audio prompts than speaker-embedding-based approaches, because their in-context learning mechanism preserves the acoustic environment of the prompt. source: §1 evidence: 'Noise-robust zero-shot TTS task (subjective + objective): | System | Prompt | MOS | SMOS | SECS | WER | +FLOPs | |--------|--------|-----|------|------|-----|--------| | LauraTTS | Clean | 4.03 | 3.99 | 0.827 | 2.33% | — | | LauraTTS | Noisy | 3.46 | 2.80 | 0.642 | 21.37% | — | | LauraTTS + MP-SENet | Noisy | 4.01 | 3.85 | 0.811 | 2.54% | 38.93G | | NR-LauraTTS | Noisy | 4.02 | 3.98 | 0.827 | 2.44% | 1.10G | NR-LauraTTS achieves SECS of 0.827 — matching LauraTTS with clean prompts exactly — while LauraTTS+MP-SENet reaches only 0.811 due to SE artifacts corrupting speaker identity.' confidence: high relevance: high - claim_id: performing_speech_enhancement_in_the_discrete_acoustic_token role: supports claim: Performing speech enhancement in the discrete acoustic token domain outperforms waveform-domain SE methods in both speech quality and computational efficiency, achieving higher DNSMOS scores at roughly one-third the FLOPs. source: §4.1, Table 1 evidence: 'Speech enhancement task (DNSMOS P.835, 1s signals): | Method | SIG | BAK | OVRL | FLOPs | |--------|-----|-----|------|-------| | CMGAN | 3.54 | 4.05 | 3.26 | 31.68G | | MP-SENet | 3.56 | 4.09 | 3.30 | 38.93G | | Codec Denoiser | 3.62 | 4.11 | 3.36 | 9.96G | The codec denoiser outperforms both waveform-domain baselines in all DNSMOS dimensions at 3x lower compute.' confidence: high relevance: low - claim_id: waveform_domain_speech_enhancement_introduces_artifacts_that_degrade role: supports claim: Waveform-domain speech enhancement introduces artifacts that degrade speaker identity in the enhanced prompt, resulting in lower speaker similarity in downstream zero-shot TTS compared to codec-domain denoising. source: §4.2, Table 3 evidence: 'Speech enhancement task (DNSMOS P.835, 1s signals): | Method | SIG | BAK | OVRL | FLOPs | |--------|-----|-----|------|-------| | CMGAN | 3.54 | 4.05 | 3.26 | 31.68G | | MP-SENet | 3.56 | 4.09 | 3.30 | 38.93G | | Codec Denoiser | 3.62 | 4.11 | 3.36 | 9.96G | The codec denoiser outperforms both waveform-domain baselines in all DNSMOS dimensions at 3x lower compute.' confidence: high relevance: high - claim_id: predicting_only_the_first_two_rvq_groups_of role: supports claim: Predicting only the first two RVQ groups of clean acoustic tokens is sufficient for effective token-domain speech enhancement; predicting more groups increases complexity without improving quality. source: §4.1, Table 2 evidence: Prediction of only the first two groups is motivated by LauraTTS's architecture (first two groups encode essential speaker properties) and by the empirical finding that predicting more groups increases complexity without improving SE metrics. 2. confidence: high relevance: medium - claim_id: the_vq_bottleneck_of_neural_codecs_acts_as role: complicates claim: The VQ bottleneck of neural codecs acts as an implicit noise filter during quantization, providing a structural advantage for denoising in the token domain relative to signal-domain methods. source: §4.1 evidence: 'Codec Denoiser (the novel component): A two-stage model operating entirely in the discrete acoustic token domain: 1.' confidence: high relevance: low limitations: - '- Specific to LauraTTS + Encodec (FunCodeec); adaptation to other codecs (DAC, SoundStream, EnCodec original) requires retraining. - Training uses synthetic noise (DNS Challenge 2022 + WHAM!); real-world noise types (music, babble, room impulse responses) may not be fully covered. - No ablation on LauraTTS model size or the effect of codec VQ count K on denoiser difficulty. - Intelligibility (WER 2.44%) is slightly worse than clean-prompt LauraTTS (2.33%), though close. - No evaluation of the codec denoiser on other TTS or SE tasks beyond the paired experiment.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0406 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - GAN relevance: low evidence_role: - architecture_variant current_role: minor method_family: - gan_based_voice_synthesis claims: - claim_id: pretrained_vocoder_refines_warped_speech_to_binaural role: supports claim: Pretrained monaural denoising vocoders can refine geometrically warped speech into perceptually natural binaural audio without exposure to any binaural training data. source: §3.3, §4.4 evidence: ZeroBAS applies a pretrained WaveFit vocoder iteratively to geometrically warped and amplitude-scaled mono audio, achieving MOS 4.07 on BSD versus 4.01 for the supervised BinauralGrad baseline. confidence: high relevance: low - claim_id: supervised_binaural_models_degrade_ood role: supports claim: Supervised mono-to-binaural synthesis models trained on small room-specific datasets degrade significantly under out-of-distribution acoustic conditions, including different rooms and languages. source: §4.5, Table 2 evidence: On the out-of-distribution TUT Mono-to-Binaural dataset (different rooms, French speakers), supervised baselines WarpNet and BinauralGrad drop to MOS 3.60 and 3.27 respectively, with BinauralGrad producing Gaussian noise artifacts on OOD samples. confidence: high relevance: medium - claim_id: gtw_and_as_both_necessary_for_zeroshot_binaural role: supports claim: Geometric interaural time delay warping and amplitude scaling based on the inverse-square law are both necessary components for zero-shot binaural synthesis; removing either one substantially degrades perceptual quality. source: §5, Table 3 evidence: Removing GTW degrades MOS to 3.64 and removing AS degrades MOS to 2.93, versus 4.07 for the full ZeroBAS pipeline. confidence: high relevance: high - claim_id: in_distribution_benchmarks_insufficient_for_binaural_generalisation role: complicates claim: Standard in-distribution benchmarks for binaural synthesis are insufficient to assess generalisation; evaluating on held-out room conditions reveals large performance gaps that in-distribution results conceal. source: §4.2, §4.5 evidence: Supervised methods that are competitive with ZeroBAS on the in-distribution BSD dataset (differences not statistically significant by MUSHRA) fall significantly behind on the newly introduced out-of-distribution TUT Mono-to-Binaural dataset. confidence: high relevance: medium limitations: - The method does not model room impulse responses or listener head shape; HRTF is implicit and generic rather than personalized. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0455 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - flow-matching - VAE - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: aptts_adversarial_posttraining_step_reduction role: supports claim: Adversarial post-training applied to a pre-trained flow matching decoder can reduce the required ODE sampling steps by 5-8x while maintaining competitive naturalness and improving intelligibility relative to the full-step baseline. source: §3.3, Table 1, Table 2 evidence: APTTS (4 steps) achieves WER 1.73% on the full LibriSpeech test-clean set, the lowest of all baselines, with RTF 0.052 vs. F5-TTS's 32-step RTF 0.217, a 4x speedup. confidence: high relevance: low - claim_id: aptts_latent_fm_low_data_parity role: supports claim: Latent flow matching TTS trained on approximately 1K hours can match or surpass the intelligibility of zero-shot systems trained on 60-100K hours, though speaker similarity remains lower at this data scale. source: §4.2, Table 2 evidence: APTTS achieves WER 1.73%/CER 0.40% on full LibriSpeech test-clean, below VALL-E (WER 5.9%) and comparable to Voicebox (WER 1.9%), but SIM-o is 0.55 vs. 0.66 for Voicebox and 0.64 for F5-TTS. confidence: medium relevance: high - claim_id: aptts_infilling_framing_no_speaker_encoder role: supports claim: Framing zero-shot TTS as a speech infilling task within a latent space enables prompt-conditioned generation without requiring a separate speaker encoder or d-vector lookup. source: §3.2 evidence: Zero-shot TTS is framed as masked-frame infilling with a binary mask covering 70-100% of the latent, letting the FM decoder condition on the speech prompt directly rather than via an explicit speaker embedding. confidence: medium relevance: high - claim_id: aptts_hybrid_cfg_reduces_error_accumulation role: supports claim: Hybrid classifier-free guidance, decoupling the unconditional branch to a frozen base model checkpoint, reduces error accumulation in few-step generators compared to applying a fixed large guidance scale. source: §3.3 evidence: The unconditional term used at inference is derived from the pre-adversarial-post-training base model rather than the few-step generator, enabling stable guidance without the error accumulation seen with a fixed large guidance scale. confidence: medium relevance: low - claim_id: aptts_ap_generalizes_across_domains role: supports claim: 'Adversarial post-training generalises across flow matching representation domains: the same post-training procedure improves both latent-domain (APTTS) and mel-spectrogram-domain (Matcha-TTS) models.' source: §4.3, Table 3 evidence: AP at 2 steps applied to Matcha-TTS surpasses MAT-10 in both MOS (3.96 vs. 3.71) and WER (4.8% vs. 6.81%), confirming the strategy generalizes beyond the latent-domain backbone. confidence: medium relevance: low limitations: - SIM-o scores lag behind large-scale trained models; the system is English-only and the fixed ODE time steps are empirically chosen rather than derived per input. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0464 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC - codec architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: explicit_mutual_information_minimisation_at_the_codec_embedding role: supports claim: Explicit mutual information minimisation at the codec-embedding level provides effective prosody-timbre disentanglement for voice conversion. source: §3.5, Table 3 evidence: Removing the MI loss (L_MI) from the full system leads to a notably higher normalised F0 distance in the prosody-from-source scenario (3.28 vs. 2.82), while quality and timbre metrics change only modestly, isolating prosody control as the primary benefit of the MI objective. confidence: high relevance: high - claim_id: in_context_learning_codec_lms_can_serve_as role: supports claim: In-context learning codec LMs can serve as controllable VC backbones when augmented with prosody-disentangling encoder modules. source: §3.4, §3.5, Table 2, Table 3 evidence: The proposed system builds on VALL-E X's ICL mechanism and outperforms VALL-E X in speaker similarity (ASV 0.91 vs. 0.84), intelligibility (WER 0.101 vs. 0.115), naturalness (MOS 4.36 vs. 4.19), and prosody alignment (F0 distance 2.70 vs. 3.10) in the prompt-based scenario. confidence: high relevance: high - claim_id: prosody_disentanglement_at_the_codec_level_introduces_a role: complicates claim: Prosody disentanglement at the codec level introduces a small trade-off in absolute codec reconstruction fidelity compared to the unmodified encoder. source: §3.3, Table 1 evidence: PACE's ASV score (0.662) and NISQA score (3.98) are lower than the baseline EnCodec encoder (0.681, 4.17), though the gap does not substantially affect system-level VC performance. confidence: high relevance: low - claim_id: prosody_from_source_and_prosody_from_prompt_are role: refines claim: Prosody-from-source and prosody-from-prompt are distinct capability axes in voice conversion; systems strong at one do not automatically handle the other. source: §3.5, Table 3 evidence: VALL-E X supports only prosody-from-prompt and is excluded from the source-prosody evaluation; TriAAN-VC and ProsoVC support only source-prosody and are excluded from the prompt-prosody evaluation. Only the proposed system is evaluated in both modes. confidence: high relevance: medium limitations: - All evaluation is conducted on LibriTTS-clean-100 and test-clean, a relatively clean single-domain corpus with 247 speakers. Generalisation to noisy environments, expressive or emotional speech, or cross-lingual settings is not tested. - The 54-hour training dataset is modest for a codec language model approach; it is unclear whether the disentanglement quality degrades with longer or more expressive source utterances. No code or demo is reported, limiting reproducibility. The paper does not ablate the number of RVQ codebooks or the sensitivity of the MI-minimisation trade-off weight (lambda_MI), leaving the robustness of the disentanglement objective undercharacterised. Prosody is operationalised solely through f0 and UV binary flags; richer prosodic dimensions such as energy, speaking rate, and phrase-level structure are not captured. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0468 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - codec architecture: - VAE - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: directly_encoding_ssl_features_as_a_first_class role: supports claim: Directly encoding SSL features as a first-class codec stream produces stronger semantic preservation in RVQ-1 tokens than distillation from an SSL model, particularly for tonal languages where pitch fidelity is critical. source: §4.2, Table 2 evidence: The SSL encoding stream takes 16th-layer features from a frozen w2v-BERT-2.0 model (600M parameters), downsamples them via 1D average pooling to the target frame rate (factor 2 for 25 Hz, factor 4 for 12.5 Hz), and passes them through a ResNet encoder (ConvNeXt blocks, 13M parameters) before VQ quantisation. confidence: high relevance: low - claim_id: operating_a_neural_codec_at_lower_frame_rates role: supports claim: Operating a neural codec at lower frame rates with more RVQ layers at fixed token rate improves audio quality over higher-frame-rate codecs with fewer layers at the same bitrate. source: §4.3, Table 3 evidence: The waveform encoding stream follows the DAC architecture — a CNN encoder with strided convolutions reaches the same frame rate, then N-1 RVQ layers quantise the residual between the waveform feature and the RVQ-1 feature; the decoder sums both streams back before upsampling to audio. confidence: high relevance: low - claim_id: semantic_quality_of_rvq_1_tokens_is_a role: supports claim: Semantic quality of RVQ-1 tokens is a primary determinant of downstream TTS intelligibility in autoregressive codec-based systems, independent of codec audio reconstruction quality. source: §4.4, Table 4 evidence: On codec audio reconstruction quality (LibriSpeech-test-clean, normalised to 75 tokens/s, ~0.75 kbps), DualCodec 25Hz with 16384 codebook (G2) achieves MUSHRA 86.2 and UTMOS 4.08, exceeding WavTokenizer-large (MUSHRA 81.0, UTMOS 3.87) and Mimi (MUSHRA 72.8). confidence: high relevance: low - claim_id: an_ssl_based_semantic_stream_in_a_codec role: supports claim: An SSL-based semantic stream in a codec encoder can improve perceptual audio quality beyond what waveform-only codecs achieve, even when using the same decoder architecture. source: §4.3, Table 3 evidence: The waveform encoding stream follows the DAC architecture — a CNN encoder with strided convolutions reaches the same frame rate, then N-1 RVQ layers quantise the residual between the waveform feature and the RVQ-1 feature; the decoder sums both streams back before upsampling to audio. confidence: high relevance: low limitations: - The 12.5 Hz DualCodec-based TTS lags behind the 25 Hz variant in both WER and speaker similarity, indicating that the more aggressive downsampling introduces a ceiling on semantic accuracy that affects TTS quality. The paper acknowledges this gap as the primary remaining challenge. - The SSL model (w2v-BERT-2.0, 600M parameters, frozen) is required at TTS training time but not inference. This makes the training pipeline heavier than pure waveform codec approaches. It is also unclear whether the approach generalises to SSL models other than w2v-BERT-2.0, or whether the chosen 16th layer feature is optimal across languages beyond English and Mandarin. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0551 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: selectively_applying_monotonic_attention_constraints_to_automatically_identified role: supports claim: Selectively applying monotonic attention constraints to automatically identified alignment heads in decoder-only LLMs reduces word repetition and omission errors without degrading performance on general test sets. source: §3.1, §3.2, §5.1, Table 1 evidence: A recursive formula aggregates these probabilities into monotonic attention scores — analogous to monotonic multihead attention in encoder-decoder models (Raffel et al. 2017, Ma et al. 2019) but adapted for decoder-only self-attention. confidence: high relevance: low - claim_id: forced_aligner_based_monotonic_alignment_methods_for_llm role: supports claim: Forced-aligner-based monotonic alignment methods for LLM TTS suffer from out-of-domain generalization failure when training data is diverse, while alignment-free approaches are more robust. source: §5.1, Table 1 evidence: 'VALL-E R (via Phn-Cosy-S-Ali) significantly raises general test WER (Seed-ZH CER: 4.03%→5.08%, Seed-EN WER: 5.32%→6.40%) indicating poor out-of-domain generalization, while SMA avoids this degradation.' confidence: high relevance: medium - claim_id: in_decoder_only_llms_used_for_tts_alignment role: supports claim: In decoder-only LLMs used for TTS, alignment characteristics are concentrated in a small subset of attention heads (typically 2–3 out of the full multi-head set), and applying monotonic constraints only to those heads is critical for preserving generation quality. source: §3.2, §5.2, Figure 2 evidence: A recursive formula aggregates these probabilities into monotonic attention scores — analogous to monotonic multihead attention in encoder-decoder models (Raffel et al. 2017, Ma et al. 2019) but adapted for decoder-only self-attention. confidence: high relevance: low - claim_id: monotonic_attention_fine_tuning_for_llm_based_tts role: supports claim: Monotonic attention fine-tuning for LLM-based TTS scales to large model sizes and large-scale multilingual training corpora without degrading speaker similarity. source: §5.1, Table 1 evidence: 'Stepwise Monotonic Attention (SMA): At each acoustic token generation step i, the model computes selection probabilities over phoneme positions using sigmoid-activated energies from the self-attention mechanism.' confidence: high relevance: high limitations: - The method fine-tunes a pre-trained model; it remains unclear whether SMA should be applied from scratch or only as a fine-tuning step. Hard-case CER/WER remains substantially above general test performance even after SMA (e.g., Seed-ZH-Hard CER 8.53% vs. Seed-ZH CER 2.33%), meaning severe repetition/omission cases are not fully resolved. The selection of alignment heads via the diagonal ratio heuristic is validated empirically but not theoretically. Extension to streaming/online inference (where the monotonic state must be maintained across steps) is not discussed. No subjective (MOS/SMOS) evaluation is reported, making naturalness trade-offs unclear. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0575 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC - TTS architecture: - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: voicemark_latent_watermark_survives_vc role: supports claim: Watermarks embedded in the speaker-specific latent space of a neural codec survive zero-shot voice cloning synthesis, whereas waveform-level watermarks do not. source: §1, §3.5, Table 1 evidence: VoiceMark achieves bit-wise accuracy of 0.964/0.979/0.957 across CosyVoice, F5-TTS, and MaskGCT zero-shot VC, while AudioSeal, WavMark, and Timbre all fall to ACC approximately 0.5 (near-random) after the same conversions. confidence: high relevance: high - claim_id: voicemark_effectiveness_depends_on_latent_preservation role: refines claim: The effectiveness of latent-space watermarking in zero-shot VC scenarios depends on the VC model preserving speaker-specific latents to achieve high speaker similarity. source: §1, §2.1 evidence: The method relies on the pretrained SpeechTokenizer RVQ scheme's own disentanglement (VQ layer 1 = content, layers 2-8 = speaker-specific timbre/pitch/prosody), reasoning that any zero-shot VC model must transfer those speaker-specific latents to produce a convincing clone. confidence: medium relevance: high - claim_id: voicemark_simulated_augmentation_sufficient role: supports claim: VC-simulated augmentation during training, without exposure to actual VC model outputs, is sufficient to achieve robust watermark recovery from synthesized audio. source: §2.3, §3.6, Table 2 evidence: The watermark decoder never sees actual VC model outputs during training, achieving robustness entirely through simulated distortions (frame masking, segment shuffling/replacement, EnCodec encode/decode, speed/amplitude/filter perturbations). confidence: medium relevance: low - claim_id: voicemark_codec_watermarking_quality_tradeoff role: complicates claim: Codec-based watermarking pipelines introduce perceptible audio quality degradation compared to waveform-level methods, representing a trade-off between VC resistance and transparency. source: §3.7, Table 3 evidence: VoiceMark achieves PESQ 2.2 and SMOS 4.25, lower than AudioSeal's PESQ 4.32 and SMOS 4.67, though comparable to SpeechTokenizer's own reconstruction quality (SMOS 4.63). confidence: high relevance: medium limitations: - The model is trained only on VCTK, a small clean dataset, limiting robustness to out-of-distribution audio editing; coverage is limited to English and the approach does not address adversarial attacks designed specifically to remove latent-space watermarks. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0596 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - encoder_decoder_voice_cloning claims: - claim_id: dysarthric_teacher_student_reduces_articulation_errors role: supports claim: A teacher-student speaker encoder architecture, where the teacher conditions the TTS backbone during training and the student handles inference-time pathological inputs, substantially reduces phoneme articulation errors compared to single-encoder approaches in dysarthric TTS. source: §4.2, Table 3 evidence: The proposed method achieves PER 14.254% vs. 31.017% for the best baseline (Hybrid, FastPitch+FreeVC), a greater than 50% relative reduction; removing the student model raises PER from 14.254 to 22.846 in the ablation. confidence: high relevance: medium - claim_id: dysarthric_curriculum_learning_outperforms_random_cropping role: supports claim: Progressive curriculum learning, which gradually reduces the duration of student encoder inputs during training, outperforms random cropping and substantially improves phoneme error rate over training without structured audio augmentation. source: §4.3, Figure 3 evidence: Removing curriculum learning raises PER further to 26.428 in the ablation, and the paper reports progressive cropping (75% to 50% to 25% of full length over three stages) is superior to random cropping. confidence: high relevance: low - claim_id: dysarthric_zeroshot_generalization_from_normal_speech role: supports claim: Zero-shot multi-speaker TTS trained on normal speech can generalize to dysarthric speakers at inference time without any dysarthric training data, provided the speaker encoder is made robust to articulation distortions and short reference lengths. source: §1, §3 evidence: The system trains exclusively on LibriSpeech (normal speech) and is evaluated by zero-shot synthesis on UASpeech dysarthric recordings, with the teacher-student curriculum bridging the domain and length mismatch. confidence: medium relevance: high - claim_id: dysarthric_similarity_intelligibility_jointly_improved role: supports claim: Speaker similarity and phoneme intelligibility can be simultaneously improved through knowledge anchoring, avoiding the trade-off seen in prior hybrid approaches where intelligibility gains came at the cost of speaker identity. source: §4.1, Table 1 evidence: Speaker similarity is 0.619 (comparable to Conditional at 0.647, better than Hybrid at 0.534) while PER simultaneously drops to 14.254%, whereas the Hybrid baseline (FastPitch + FreeVC) achieves intelligibility only by sacrificing speaker identity. confidence: medium relevance: high limitations: - The system is English-only and non-autoregressive (FastSpeech2, mel-spectrogram output); speaker similarity (0.619) remains below what might be needed for truly personalized assistive use, and the approach has not been tested on languages with non-Latin scripts. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0656 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC architecture: - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: eegvc_crossmodal_alignment_enables_vc role: supports claim: Cross-modal feature alignment between neural signals and speaker embeddings can enable voice conversion without any target-speaker voice data. source: §3.1.2, §4.4 evidence: The EEG-voice feature alignment module, trained with embedding MSE and speaker classification losses, produces speaker embeddings from EEG that drive FreeVC-based conversion to zero-shot quality (Naturalness MOS 4.00, Consistency obj 0.8026 for unseen speakers). confidence: medium relevance: high - claim_id: eegvc_nonspeech_biometric_encodes_identity role: supports claim: Non-speech biometric signals can encode speaker-identity information sufficient to guide voice timbre conversion. source: §4.4.1, Figure 2 evidence: t-SNE visualisation shows synthesised speech clusters align with reference audio per speaker, and Homogeneity scores (0.9437-0.9465) exceed the FreeVC speech-only baseline (0.9371), indicating EEG features encode timbre-discriminative information. confidence: medium relevance: high - claim_id: eegvc_requires_speech_pretraining_stage role: complicates claim: Zero-shot voice conversion from EEG signals requires a large-scale speech-only pre-training stage to compensate for the scarcity and noise of paired EEG-speech data. source: §3.2 evidence: The three-stage curriculum first pre-trains on VCTK (Stage I, speech only), then aligns EEG to pre-trained speaker embeddings (Stage II), before joint fine-tuning (Stage III); the authors explicitly attribute feasibility to leveraging the pre-trained VC model's representations. confidence: medium relevance: high - claim_id: eegvc_evaluation_limited_by_paired_data role: complicates claim: Evaluation of EEG-driven voice conversion is fundamentally limited by the availability of paired EEG-speech corpora at the scale needed for generalisation. source: §4.1, §5 evidence: The entire EEG evaluation uses the Single-Word-Production Dutch-iBIDS dataset (10 speakers, single-word utterances); seen-speaker training uses 80% of this data, and the authors note that more data will improve performance. confidence: medium relevance: medium limitations: - The evaluation dataset contains only 10 speakers producing single words in Dutch; the baseline comparison (FreeVC) is not equivalent since it uses a target-speaker voice prompt rather than EEG, and no intelligibility metrics (WER/CER) are reported. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0787 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: low evidence_role: - historical_context current_role: minor method_family: - autoregressive_speech_token_lm claims: - claim_id: speaker_embedding_manipulation_can_transfer_a_distinctive_speaking_style role: supports claim: Speaker embedding manipulation can transfer a distinctive speaking style (Lombard speech) to unseen speakers using only a small feedforward network trained on minimal paired data, without retraining the underlying TTS model. source: §2.3, §4.1 evidence: Speaker embedding manipulation can transfer a distinctive speaking style (Lombard speech) to unseen speakers using only a small feedforward network trained on minimal paired data, without retraining the underlying TTS model. confidence: medium relevance: high - claim_id: perceptual_plausibility_of_artificially_generated_lombard_speech_depends role: supports claim: Perceptual plausibility of artificially generated Lombard speech depends substantially on modelling pitch and duration changes, not only level amplification as recommended by ITU-T P.1150. source: §4.3, Table 1 evidence: Perceptual plausibility of artificially generated Lombard speech depends substantially on modelling pitch and duration changes, not only level amplification as recommended by ITU-T P.1150. confidence: medium relevance: low - claim_id: spherical_interpolation_between_plain_and_lombard_speaker_embeddings_whe role: supports claim: Spherical interpolation between plain and Lombard speaker embeddings, when calibrated to a physical noise-level scale, enables continuous and controllable Lombard intensity without requiring recordings at each intermediate level. source: §2.5 evidence: Spherical interpolation between plain and Lombard speaker embeddings, when calibrated to a physical noise-level scale, enables continuous and controllable Lombard intensity without requiring recordings at each intermediate level. confidence: medium relevance: medium - claim_id: embedding_space_style_transfer_trained_on_in_distribution_data_can_gener role: supports claim: Embedding-space style transfer trained on in-distribution data can generalise to out-of-distribution speakers with a speaker similarity loss of approximately 0.5 MOS relative to an ideal upper bound. source: §4.3 evidence: Embedding-space style transfer trained on in-distribution data can generalise to out-of-distribution speakers with a speaker similarity loss of approximately 0.5 MOS relative to an ideal upper bound. confidence: medium relevance: high limitations: - See the source paper page for the complete limitations and evaluation scope. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0815 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC architecture: - VAE - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: expvc_discrete_units_reduce_leakage_hurt_intelligibility role: complicates claim: Discrete speech unit representations reduce source speaker leakage in voice conversion but introduce pronunciation artefacts that degrade intelligibility compared to continuous feature counterparts. source: §4.2, Table 1 evidence: The proposed model using mHuBERT-147 discrete units improves emotion transfer (ECA 37.0% to 78.9%) and reduces source leakage (EER 14.6% to 7.4% on LibriTTS) relative to Hierspeech++, but at a WER cost (7.98% vs. 5.01% on ESD, 8.84% vs. 3.48% on LibriTTS). confidence: high relevance: medium - claim_id: expvc_mixln_mitigates_train_inference_mismatch role: supports claim: Mix-style layer normalisation mitigates the train-inference mismatch caused by content-style dependence in style encoders, improving zero-shot generalisation on unseen speakers. source: §4.3, Table 2 evidence: Removing Mix-LN causes the most severe WER degradation on LibriTTS (19.83%) in the ablation, consistent with the hypothesis that Mix-LN reduces train-inference content-style mismatch. confidence: high relevance: high - claim_id: expvc_enriched_style_embeddings_improve_transfer role: supports claim: Enriching global style embeddings with explicit pitch and energy features improves emotion transfer fidelity in expressive voice conversion beyond mel-spectrogram-only style encoding. source: §3.5, §4.3, Table 2 evidence: The style encoder fuses separate embeddings from mel spectrograms, F0 contours, and energy contours via a trainable weighted sum; ablation shows this enriched embedding contributes to the emotion transfer gains over Hierspeech++. confidence: medium relevance: medium - claim_id: expvc_cross_attention_f0_beats_additive role: supports claim: Cross-attention fusion of local F0 contours with content embeddings produces stronger prosodic alignment to the target than additive F0 injection in non-autoregressive voice conversion. source: §3.1, §4.3, Table 2 evidence: Removing F0 cross-attention degrades SECS by 0.6pp and ECA by 12.5pp in the ablation, and the cross-attention design allows target F0 to be directly used regardless of source/target length differences. confidence: medium relevance: medium - claim_id: expvc_crosslingual_vc_without_multilingual_training role: refines claim: Zero-shot cross-lingual voice conversion is achievable with a monolingual training corpus when content representations are extracted from a multilingual speech model, though intelligibility degrades for unseen source languages. source: §4.4, Table 4 evidence: The system achieves 76.1% ECA for English-to-German conversion vs. 61.7% for Hierspeech++, though German-to-English WER is 30.84%, indicating intelligibility degrades when the source language is absent from training. confidence: medium relevance: high limitations: - The evaluation uses a small subjective panel (15 listeners); cross-lingual results are restricted to English and German, and model size and computational cost are not reported. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-0989 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: - autoregressive-LM relevance: low evidence_role: - infrastructure current_role: minor method_family: - autoregressive_speech_token_lm claims: - claim_id: speaker_diversity_in_training_data_is_a_stronger role: complicates claim: Speaker diversity in training data is a stronger driver of zero-shot TTS generalization than audio quality or dataset size alone, as a 10-speaker high-quality dataset fails catastrophically on unseen speakers despite controlled recording conditions. source: §4.3, Table 3 evidence: The model trained solely on HiFiTTS (10 speakers) fails badly on unseen speakers (WER 10.38%, SSIM 0.059), confirming that speaker diversity rather than audio quality alone drives zero-shot performance. confidence: high relevance: high - claim_id: mixed_bandwidth_audio_in_large_scale_speech_corpora role: supports claim: Mixed-bandwidth audio in large-scale speech corpora degrades codec and vocoder training, making bandwidth estimation and filtering an essential step in high-bandwidth TTS data preparation. source: §1, §2.3 evidence: 'Bandwidth Estimation: Following HiFiTTS methodology, the maximum frequency with ≥-50 dB power relative to peak is estimated from the first 30 seconds of each audiobook.' confidence: high relevance: low - claim_id: restoring_punctuation_and_capitalization_to_asr_derived_transcripts role: supports claim: Restoring punctuation and capitalization to ASR-derived transcripts is feasible at scale via text matching (87% coverage) with neural prediction for remaining cases, and meaningfully improves transcript quality for TTS prosody modeling. source: §2.1 evidence: 'Text Preprocessing: Punctuation and capitalization (PC) are restored either by matching MLS transcripts to original audiobook text (successful for 87%) or by predicting PC using NeMo DistilBERT (remaining 13%).' confidence: high relevance: low - claim_id: providing_per_utterance_quality_metadata_wer_cer_bandwidth role: supports claim: Providing per-utterance quality metadata (WER, CER, bandwidth, speaker count) rather than applying fixed thresholds increases dataset utility by allowing downstream researchers to select quality-volume trade-offs appropriate to their application. source: §2.5, §2.6 evidence: The model trained solely on HiFiTTS (10 speakers) fails badly on unseen speakers (WER 10.38%, SSIM 0.059), confirming that speaker diversity rather than audio quality alone drives zero-shot performance. confidence: high relevance: high limitations: - The dataset is English-only. Audio quality is not filtered by SNR, accepting noise present in LibriVox recordings; this is intentional (modern TTS can handle noise) but may affect some applications. The 44.1 kHz subset's bandwidth ranges from 13–22 kHz (mixed-bandwidth within the subset), which may complicate training for systems requiring uniform bandwidth. No listening-test based MOS evaluation is reported; evaluation relies on automatic metrics (SQUIM-MOS, SSIM). The Koel-TTS codec details are not specified in this paper. Future work on non-English and non-audiobook sources is mentioned but not pursued. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1081 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC architecture: - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - gan_based_voice_synthesis claims: - claim_id: sncr_content_restoration_compensates_in_degradation role: supports claim: Explicit content-restoration mechanisms can compensate for linguistic information degraded by instance normalization in speaker disentanglement, preserving intelligibility without SSL feature extractors. source: §2.1, Table 2 evidence: The SNCR dual restoration loss, which decomposes the instance-normalization residual into content-relevant and content-irrelevant parts via phoneme-entropy constraints, reduces WER from 5.57% (without SNCR) to 3.78% compared to a baseline trained with neither IN nor dual restoration loss. confidence: high relevance: medium - claim_id: sncr_gan_matches_diffusion_lower_cost role: supports claim: GAN-based zero-shot voice conversion systems can match the speaker similarity of diffusion-based systems at substantially lower parameter count and faster inference. source: §3.2, Table 1 evidence: The proposed 50M-parameter GAN system achieves SECS 0.25 versus DDDM-VC's SECS 0.27, while running at RTF 0.11 versus 0.29 and using 366M fewer parameters; naturalness (nMOS 3.86 vs. 3.52) also favors the GAN system. confidence: high relevance: high - claim_id: sncr_attention_discriminator_improves_quality role: supports claim: Integrating self-attention layers into CNN-based discriminators improves adversarial training quality for mel-spectrogram reconstruction in voice conversion. source: §3.3, Table 2 evidence: Ablation with the discriminator's self-attention layers removed (replaced with a standard CNN discriminator) degrades nMOS and increases WER compared to the full system. confidence: medium relevance: medium - claim_id: sncr_speaker_similarity_inflated_by_overlap role: complicates claim: Speaker similarity metrics in zero-shot VC evaluations can be inflated when the test set speakers overlap with a baseline system's training set, making fair cross-system comparison difficult. source: §3.2 evidence: VQMIVC achieves SECS 0.23 (close to the proposed system's 0.25) but was trained on VCTK, which overlaps with the VCTK test speakers; its WER of 21.4% reveals conversions are not actually intelligible despite the seemingly competitive SECS. confidence: high relevance: high limitations: - The system trains and evaluates on English only (LibriLight and VCTK); the ablation evaluates components one at a time, leaving interactions between SNCR and consistency loss unexplored. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1084 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: mamba_based_sequence_models_can_match_or_exceed role: supports claim: Mamba-based sequence models can match or exceed the quality of larger Transformer-based TTS systems while enabling real-time streaming inference on CPU hardware. source: §4.5, §4.6, Table 1 evidence: SMAM+MLM (26M params) achieves MOS 4.02 and CER 2.73%, matching Lee et al. (2024) at 263M params (MOS 4.00, CER 4.01%) while reducing first-token latency from 26.5s to 0.065s on a single-threaded CPU. confidence: high relevance: low - claim_id: iterative_depthwise_refinement_of_rvq_tokens_substantially_improves role: supports claim: Iterative depthwise refinement of RVQ tokens substantially improves codec TTS quality over single-pass parallel depth prediction. source: §4.7, Table 1 evidence: Replacing MLM depthwise decoding with a single-pass no-masking baseline (SMAM+noMLM) causes a significant drop in all quality metrics (MOS from 4.02 to 3.89, CER from 2.73% to 4.12%, UTMOS from 4.13 to 3.83) with negligible change in RTF and latency. confidence: high relevance: low - claim_id: objective_speaker_similarity_metrics_based_on_embedding_cosine role: complicates claim: Objective speaker similarity metrics based on embedding cosine distance do not reliably predict subjective speaker similarity as judged by human listeners. source: §4.6, Table 1 evidence: SMAM+MLM scores SECS 0.816 (below Lee et al.'s 0.863) but achieves higher SMOS of 3.36 vs. 3.27, indicating a divergence between embedding-space distance and perceptual similarity that has practical implications for zero-shot TTS evaluation. confidence: high relevance: high - claim_id: depthwise_decoding_strategies_for_rvq_present_an_explicit role: supports claim: Depthwise decoding strategies for RVQ present an explicit quality-speed trade-off that system designers can exploit based on deployment constraints. source: §3.3, §4.5, §4.6, Table 1 evidence: SMAM+MLM (iterative, 3 passes) achieves MOS 4.02 and RTF 0.701, while SMAM+INR (single forward pass) achieves MOS 3.97 and RTF 0.568, demonstrating a consistent quality-speed trade-off across both objective and subjective evaluations. confidence: high relevance: low limitations: - Evaluation is limited to LibriTTS test-clean (English, read speech), leaving performance on spontaneous speech, noisy environments, and non-English languages uncharacterized. The SECS speaker similarity scores for the proposed models fall below the strongest baseline (Lee et al. 2024), indicating room for improvement in speaker faithfulness despite strong subjective SMOS scores. The paper does not release code, limiting reproducibility and adoption. RTF comparisons are not fully apples-to-apples since baselines generate complete utterances in batch mode while SMAM operates incrementally. Future directions mentioned include a fully streaming pipeline covering codec processing and applying depthwise decoding strategies to decoder-only speech language models. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1101 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC architecture: - diffusion relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - diffusion_voice_cloning claims: - claim_id: zsdevc_diffusion_mi_guidance_zeroshot_emotion_control role: supports claim: Diffusion-based voice conversion systems can achieve strong emotion controllability in zero-shot settings when combined with mutual-information disentanglement and inference-time guidance. source: §3.2, §3.3, Table 2 evidence: ZSDEVC achieves subjective emotion classification accuracy (ECA) of 0.53 vs. 0.256 for EMOCONV-DIFF, with the disentanglement mechanism alone raising ECA by 16.8-21.1% and expressive guidance adding a further 34.4-40.0% relative gain over the backbone. confidence: high relevance: high - claim_id: zsdevc_mi_minimisation_no_parallel_data role: supports claim: Disentangling speaker identity and emotion via mutual information minimisation improves emotion controllability in voice conversion without requiring parallel or speaker-specific training data. source: §2.1.4, §3.3, Table 2 evidence: The vCLUB contrastive upper-bound estimator minimises MI between speaker and emotion projections during training on MSP-Podcast, an in-the-wild corpus with no parallel emotional recordings, and this alone raises ECA by 16.8% on MSP-Podcast and 21.1% on ESD over the backbone. confidence: high relevance: high - claim_id: zsdevc_autoencoder_higher_accuracy_lower_naturalness role: complicates claim: In emotional voice conversion, autoencoder-based methods tend to achieve higher emotion accuracy than GAN-based methods, but at the cost of substantially lower naturalness and higher speech distortion. source: §3.2, Table 1 evidence: The paper notes that autoencoder-disentanglement approaches accurately separate emotion but tend to produce distorted, low-naturalness outputs, motivating the diffusion-plus-guidance approach instead of a pure autoencoder disentanglement pipeline. confidence: medium relevance: medium - claim_id: zsdevc_cfg_style_guidance_tradeoff role: complicates claim: Classifier-free-style guidance applied to emotion representations at inference time provides a direct lever for trading naturalness against emotion controllability in diffusion-based EVC. source: §2.1.3, §3.3, Table 2 evidence: Adding EG_emo guidance boosts ECA by 40.0% on MSP-Podcast and 37.4% on ESD relative to the backbone, with a slight cost in naturalness (UTMOS drops from 2.427 to 2.353 on MSP-Podcast). confidence: medium relevance: low - claim_id: zsdevc_large_scale_itw_training_generalizes role: supports claim: Training on large-scale in-the-wild emotional corpora enables zero-shot generalisation to speakers absent from training, even when evaluation is conducted on acted-speech datasets with different recording conditions. source: §3.4, §4 evidence: The model is trained on 48,389 utterances from 1,381 speakers in MSP-Podcast (in-the-wild podcast recordings) and evaluated on entirely held-out speakers from both MSP-Podcast and the acted ESD dataset with consistent trends. confidence: medium relevance: high limitations: - The comparison between ZSDEVC and EMOCONV-DIFF in Table 1 is not fully fair, as EMOCONV-DIFF is evaluated in a seen-speaker scenario while ZSDEVC operates zero-shot; evaluation covers only five emotion categories and excludes neutral-to-emotional conversion. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1115 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - diffusion - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_diffusion_acoustic claims: - claim_id: mpetts_unified_emotion_encoder_control role: supports claim: Unified multi-modal emotion encoders that align different prompt modalities to a shared emotion representation space enable flexible emotion control in zero-shot TTS without sacrificing speaker similarity. source: §2.2, §3.4, Table 1 evidence: MPEE ablation shows removing the multi-modal encoder drops MOS from 3.73 to 3.20 and emotion accuracy from 48% to 35% in speech-prompt mode; SSMOS (3.73) is maintained even when emotion and timbre are drawn from different speakers. confidence: high relevance: high - claim_id: mpetts_ecl_prosody_emotion_alignment role: supports claim: Auxiliary emotion consistency losses applied to autoregressive prosody prediction improve emotion alignment in generated speech. source: §2.3, §3.4, Tables 1–3 evidence: ECL ablation reduces ESMOS from 4.05 to 3.83 and ACC from 48% to 45% in speech-prompt mode; the gain is larger for text and image prompts, where the emotion signal is less direct. confidence: high relevance: medium - claim_id: mpetts_emotion_accuracy_gap role: complicates claim: Emotion accuracy in expressive zero-shot TTS remains substantially below ground truth even with explicit emotion conditioning and auxiliary training objectives. source: §3.4, Tables 1–3 evidence: Best system ACC is 48% vs 54% for ground-truth mel-spectrograms; the 6-point gap persists despite MPEE and ECL, indicating that fine-grained emotion control is not yet solved at this data scale. confidence: high relevance: high - claim_id: mpetts_baseline_reproduction_limitation role: complicates claim: Evaluating multi-modal TTS systems against prior work is complicated by the absence of official open-source implementations for key baselines. source: §3.4 evidence: MM-TTS has no official open-source release; all comparisons in this paper are against a reproduction based on the original paper's settings, which the authors acknowledge as a limitation of the experimental evaluation. confidence: medium relevance: low - claim_id: mpetts_hierarchical_disentanglement_granularity role: supports claim: Hierarchical disentanglement of speech attributes at different granularity levels enables fine-grained independent control over timbre and emotion in zero-shot TTS. source: §2.1, §3.4, Tables 1–3 evidence: The disentangling strategy separates coarse-grained features (timbre, emotion as global vectors) from fine-grained features (content, prosody at frame level), using distinct bottleneck designs; SSMOS remains stable (3.73–3.76) across all three prompt modalities despite using different emotion sources from the timbre reference. confidence: high relevance: high limitations: - MPE-TTS is fine-tuned on only 36 hours of MEAD-TTS (48 actors, 8 emotions), evaluates only English speech, and only one emotion modality can be provided at inference (no multi-modal fusion); WER is noticeably worse than ground-truth reconstructed mel (23.4% vs. 18.8%). caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1192 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - encoder_decoder_voice_cloning claims: - claim_id: voiceimpression_grl_adversarial_disentanglement role: supports claim: Adversarial disentanglement via a gradient reversal layer can separate voice impression information from speaker identity in a zero-shot TTS speaker encoder, enabling independent modulation of perceived voice characteristics. source: §2.2, §4.1, Figure 5 evidence: The control module applies GRL + 0.8 dropout to the speaker embedding to remove impression signal, then reintroduces it via an 11-dim impression vector; cosine similarity to the target speaker remains above the inter-speaker distribution at all tested modulation levels. confidence: high relevance: high - claim_id: voiceimpression_llm_style_vector_generation role: supports claim: LLMs can generate low-dimensional speech style parameter vectors from free-form natural language descriptions, providing a usable zero-shot interface for voice characteristic control. source: §2.3, §4.3, Table 4 evidence: ChatGPT-4o prompted with dimension definitions and pre-modulation values produces impression vectors preferred over unmodulated speech in 94.6% of "sleepy" trials and 74.6% of "urgent, attention-grabbing" trials (n=436, crowdsourced). confidence: medium relevance: high - claim_id: voiceimpression_naturalness_tradeoff role: complicates claim: 'Fine-grained impression control in TTS involves a trade-off: stronger modulation produces more perceptually distinct impressions but degrades naturalness, particularly when the target impression is socially incongruent with the source speaker.' source: §4.2, Table 3 evidence: MOS naturalness at maximum modulation (±3) falls to 2.71-2.88 for the Powerful-Weak and Dark-Bright dimensions, compared to 3.61-3.72 at zero modulation; low-scoring samples correspond to combinations such as "feminine male" or "strongly dark/bright." confidence: high relevance: medium - claim_id: voiceimpression_labeling_pipeline_error role: complicates claim: Automatic annotation of high-dimensional perceptual voice attributes at training scale requires indirect labeling pipelines that introduce estimation error, limiting the precision of supervision. source: §3.2 evidence: Crowdsourced subjective ratings were collected for only 1,154 of 20,270 speakers; a HuBERT-based estimator extrapolated labels to the remaining data with an RMSE of 0.338 on held-out utterances. confidence: medium relevance: medium limitations: - All training and evaluation data is a proprietary in-house Japanese corpus (no public datasets), and the evaluation spans only two held-out speakers; the eleven impression dimensions exhibit substantial inter-correlation (up to 0.8), so independent perceptual dimensions are not fully captured by the vector design. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1229 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: icl_flow_matching_enables_joint_timbre_and_background_preservation role: supports claim: In-context learning with flow matching can enable voice conversion systems to simultaneously transform speaker timbre and preserve background sounds without an explicit separation step. source: §4.2, Table 2 evidence: E2E-BPVC achieves BS-MOS 4.60 and SS-MOS 4.02, comparable to the cascaded Denoise-VC II (BS-MOS 4.65, SS-MOS 4.07), using a single model without a denoising module, validated by 12 human raters. confidence: high relevance: high - claim_id: standard_vc_metrics_insufficient_for_background_preserving_systems role: complicates claim: Standard voice conversion evaluation metrics (speaker similarity, character error rate, speech quality) are insufficient for assessing systems that operate on speech with background sounds. source: §4.1, Table 1 evidence: ECAPA-TDNN speaker similarity and ASR-based CER are degraded by background sound, causing clean-output systems to appear relatively stronger on objective metrics despite failing entirely on background preservation (ICL-VC BS-MOS 0.70); the authors explicitly state objective metrics 'do not adequately reflect the capabilities' of background-preserving systems. confidence: high relevance: high - claim_id: noise_robust_ssl_representations_improve_content_disentanglement role: supports claim: Noise-robust self-supervised speech representations improve content disentanglement in voice conversion systems trained on speech with background sounds. source: §4.3, Table 3 evidence: Replacing HuBERT with WavLM as the semantic token backbone reduces CER from 10.27 to 7.99 under noisy evaluation, and training k-means on noisy speech further reduces CER to 7.99 versus 9.22. confidence: high relevance: medium - claim_id: background_preservation_objective_trades_off_against_clean_speech_quality role: complicates claim: Designing a single voice conversion model to handle background preservation introduces a competing objective that slightly degrades clean-speech conversion quality relative to a clean-speech-only system. source: §4.1, Table 1 evidence: E2E-BPVC achieves SIM 0.849 and CER 3.18 on clean-speech conversion, modestly below the clean-only ICL-VC (SIM 0.873, CER 2.37). confidence: medium relevance: medium limitations: - Evaluation relies exclusively on LibriTTS with synthetically added noise and music at controlled SNRs, without testing on real-world recordings with natural, non-stationary backgrounds; the comparison set is limited to two baselines sharing the same ICL-VC foundation. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1236 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: adma_auxiliary_alignment_accelerates_fm_training role: supports claim: Auxiliary alignment losses applied at intermediate transformer layers can substantially accelerate convergence in flow-matching TTS training without modifying the inference pipeline. source: §2, §4.1, Table 1 evidence: A-DMA reduces WER from 2.68% to 1.97% and improves SIM from 0.60 to 0.62 for F5-TTS on LibriSpeech-PC in a low-resource (0.6kh) setting, with A-DMA matching the baseline's terminal WER in roughly half the training steps. confidence: high relevance: low - claim_id: adma_modality_specific_layer_depth role: refines claim: In transformer-based generative models, text-semantic and speaker-acoustic alignment supervision are best applied at different network depths rather than at the same layer. source: §4.2, Table 2 evidence: Layer-wise ablation on F5-TTS Small (18 DiT layers) shows CTC text alignment at layer 8 combined with HuBERT speech alignment at layer 12 (WER 2.226%) outperforms same-layer dual alignment at either layer 8 (3.063%) or layer 12 (2.688%). confidence: high relevance: medium - claim_id: adma_ssl_teacher_improves_sim role: supports claim: Frozen self-supervised speech encoders such as HuBERT are effective perceptual teachers for improving speaker similarity in flow-matching TTS through cosine-similarity distillation. source: §4.3, Table 3 evidence: Negative cosine similarity to HuBERT-large features at the 12th transformer layer improves SIM from 0.578 to 0.609 and WER from 7.474% to 3.521%; WavLM achieves similar gains. confidence: high relevance: high - claim_id: adma_training_acceleration_not_naturalness role: complicates claim: Training-time alignment acceleration for diffusion TTS primarily improves text intelligibility and speaker similarity rather than overall speech naturalness, as measured by automatic MOS prediction. source: §4.2, Table 2, §6 evidence: UTMOS scores across all A-DMA ablation conditions range from 4.01 to 4.12, close to the 4.04 baseline, while WER and SIM show large improvements; no human listening test is reported. confidence: medium relevance: high limitations: - All experiments use the low-resource 0.6kh training regime; whether doubled convergence speed holds at the 100kh scale used by high-resource baselines is untested. - No human listening evaluation is reported; naturalness gains rest entirely on UTMOS as an automatic proxy. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1344 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: peft_tts_preserves_zeroshot_better_than_full_ft role: supports claim: Adapter-based fine-tuning preserves zero-shot TTS capability in multilingual models more reliably than full parameter fine-tuning on single-speaker low-resource data. source: §4.3, Table 2 evidence: On the multi-speaker Korean evaluation, full fine-tuning of F5-TTS loses zero-shot capability entirely (NMOS 2.24 ± 0.21), while the adapter-based PEFT-TTS retains it (NMOS 3.06 ± 0.28). confidence: high relevance: high - claim_id: peft_tts_cross_lingual_low_data_feasible role: supports claim: Cross-lingual adaptation of TTS models is feasible with fewer than 15 hours of single-speaker data when a multilingual pre-trained foundation is available. source: §4.1.1, §4.2 evidence: PEFT-TTS adapts F5-TTS to Korean using only the 12.65-hour single-speaker KSS dataset and 1.72% of model parameters, achieving CER 4.906 versus 17.345 for F5-TTS trained from scratch on the same data. confidence: high relevance: medium - claim_id: peft_tts_text_conditioning_needs_more_capacity role: supports claim: In flow-matching TTS fine-tuning, the text conditioning module requires more parameters to adapt to a typologically distant language than the acoustic generation module. source: §4.4.1 evidence: The ablation shows that a compressed Conv-Adapter on the ConvNeXt V2 text encoder (compression factor 0.25, full adaptation) is necessary for reliable Korean pronunciation, whereas the DiT acoustic blocks use a much smaller LoRA rank (16) without degrading pronunciation. confidence: medium relevance: low - claim_id: peft_tts_lora_rank_tradeoff role: supports claim: Low-rank adaptation in DiT-based TTS models exhibits a rank-dependent trade-off between linguistic accuracy and speaker generalisation, with optimal rank dependent on dataset scale. source: §4.4.3 evidence: The ablation shows increasing DiT LoRA rank improves pronunciation accuracy but reduces zero-shot speaker adaptation capability, motivating the paper's choice of rank 16 over larger ranks. confidence: medium relevance: high limitations: - Reproducibility is limited to the Korean-specific KSS dataset; generalisation of the parameter-allocation findings to other low-resource languages is assumed but not tested. - Multi-speaker WER (21.6%) remains well above a large-scale multilingual baseline (10.9%), with hallucination artefacts not fully resolved by the proposed adapters. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1440 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - codec architecture: - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: freecodec_ssl_disentangle_match_supervised role: supports claim: Self-supervised disentanglement of speech into content, speaker, and prosody streams can match or exceed supervised codec quality at significantly lower bitrate. source: §4.1, Table 1, Table 2 evidence: FreeCodec-v1 at 0.45 kbps achieves UTMOS 4.034, STOI 0.918, and speaker cosine similarity 0.919 on VCTK, matching or exceeding SpeechTokenizer at 3 kbps and FACodec at 2.4 kbps on most metrics despite using unsupervised content/speaker/prosody factorisation via dedicated frame-level encoders. confidence: high relevance: medium - claim_id: freecodec_factorization_over_capacity role: supports claim: Codec coding efficiency is more sensitive to information factorisation than to raw model capacity or bitrate allocation. source: §4.1, Table 1 evidence: FreeCodec-v2 (fully discrete, 0.45 kbps) scores MUSHRA 87.44, beating DAC, TiCodec, FACodec, and SpeechTokenizer despite operating at 2-7x lower bitrate, indicating the three-way factorisation rather than added model capacity drives the efficiency gain. confidence: medium relevance: low - claim_id: freecodec_decoder_supervision_routing role: supports claim: Routing WavLM supervision to the decoder rather than the encoder during training improves speaker-content disentanglement for voice conversion. source: §2.5, §4.2 evidence: FreeCodec-v3 moves the WavLM cosine-similarity content loss from the content encoder to the decoder branch specifically to prevent speaker leakage into the content encoder, enabling any-to-any VC with SECS 0.847 versus 0.553 for FACodec at 5x higher bitrate. confidence: high relevance: medium - claim_id: freecodec_ultralow_bitrate_competitive role: supports claim: Ultra-low-bitrate codecs (below 0.5 kbps) can achieve MUSHRA scores competitive with codecs operating at 2-3 kbps when disentanglement is applied to reduce frame-level redundancy. source: §4.1, Table 2 evidence: FreeCodec-v2 at 0.45 kbps (87.44 MUSHRA) outperforms FACodec at 2.4 kbps (80.44) and SpeechTokenizer at 3 kbps (82.0), attributing the gain to the three-encoder factorisation reducing redundant information encoded per frame. confidence: medium relevance: low limitations: - Evaluation is restricted to English (LibriSpeech and VCTK); generalisation to other languages, accents, or spontaneous-speech domains is untested. The prosody encoder's separation is validated only via t-SNE visualisation without a formal mutual information analysis, and partial speaker clustering suggests the separation is not complete. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1531 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - singing - VC architecture: - VAE - GAN relevance: low evidence_role: - historical_context current_role: minor method_family: - continuous_latent_vae claims: - claim_id: svc_dimreduction_timbre_leakage role: supports claim: In singing voice conversion, reducing the dimensionality of SSL embeddings through random channel selection proportionally reduces timbre leakage while preserving sufficient phonetic content for accurate reconstruction. source: §4. Results, Table 1 evidence: SSL-128-Emb (128 of 768 HuBERT dimensions) achieves SMOS 3.920 vs. SSL-Emb's 2.750 on the Chinese test set, while maintaining CMOS 4.244 vs. 4.366 for full embeddings. The pattern holds for SSL-Soft and ContentVec variants. confidence: high relevance: medium - claim_id: svc_discrete_token_generalization_failure role: complicates claim: Discrete token-based content representations for voice conversion cannot generalize to phonetic inventories outside the training distribution. source: §4. Results, Table 2 evidence: SSL-Token CMOS drops from 4.086 on Chinese to 3.210 on other languages (English, Korean, Vietnamese, Japanese, Cantonese), demonstrating that k-means quantization of HuBERT with up to 10,000 clusters still loses phonetic detail that is language-specific. Continuous dimension-reduced embeddings maintain CMOS above 4.17 across both conditions. confidence: high relevance: medium - claim_id: svc_ssl_dimension_uniform_distribution role: supports claim: Self-supervised speech embedding dimensions contain roughly proportional timbre and content signal, such that uniform random subsampling functions as an effective form of timbre disentanglement. source: §2.2, §4. Results evidence: The uniform-distribution assumption underlying random dimension selection is empirically supported by consistent improvements across three distinct SSL embedding types (HuBERT, SSL-Soft, ContentVec), each responding to dimension reduction in the same direction. confidence: medium relevance: medium - claim_id: svc_unsupervised_matches_supervised_disentangle role: refines claim: Supervised disentanglement methods for SSL content encoders in voice conversion can be matched or exceeded by unsupervised dimensionality reduction at equivalent embedding sizes. source: §4. Results, Table 1, Table 2 evidence: SSL-256-Emb surpasses SSL-Soft-Emb (which uses supervised soft target training) at the same 256-dimensional size on both SSIM and SMOS metrics in Chinese. ContentVEC-256-Emb matches token-based singer similarity while outperforming token-based CMOS on cross-lingual evaluation. confidence: medium relevance: medium limitations: - All training data is proprietary (200h internal Chinese singing corpus); no public benchmark is used, and the optimal dimension count d is determined by grid search with no principled selection criterion. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1536 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: zeroshot_tts_overnormalizes_dysarthric_intelligibility role: supports claim: Zero-shot TTS systems trained on typical speech systematically over-normalise intelligibility when cloning dysarthric speech, producing cleaner output than the reference at the cost of dysarthric feature preservation. source: §4.1, §4.2, Table 3 evidence: F5-TTS yields delta-WER DI of 0.59 for high-severity speakers and delta-CER DI of 0.75, both rated 'poor,' while SIM-o DI remains 0.85 and AutoPCP DI 0.90. confidence: high relevance: high - claim_id: biased_dysarthric_cloning_harms_severe_asr_augmentation role: complicates claim: Augmenting dysarthric ASR training data with zero-shot voice-cloned samples improves performance for mild cases but degrades it for severe cases when the cloning system is biased toward intelligibility normalisation. source: §4.3, Table 4 evidence: Adding F5-TTS synthetic data reduces WER by 44.6% for low-severity speakers but increases WER by 5.7% and 7.7% for mid- and high-severity speakers respectively. confidence: high relevance: high - claim_id: fairness_metrics_reveal_severity_disparities_masked_by_aggregates role: supports claim: Fairness metrics (Disparate Impact and Parity Difference) applied to objective TTS evaluation measures reveal severity-level disparities that aggregate quality scores would obscure. source: §2.3, §4.2, Table 3 evidence: Per-severity DI analysis on TORGO shows near-zero intelligibility bias for low severity (delta-WER DI=0.97) escalating to severe bias for high severity (DI=0.59), invisible in mean quality scores. confidence: high relevance: low - claim_id: dysarthric_cloning_bias_varies_by_gender_and_severity role: complicates claim: Bias in zero-shot dysarthric speech cloning varies across gender as well as severity, with male and female speakers affected on different dimensions. source: §4.2, Table 3 evidence: Male speakers exhibit higher intelligibility bias (delta-WER PD=0.37, DI=0.68) while female speakers show higher prosody bias (AutoPCP PD=0.36). confidence: medium relevance: high limitations: - The study uses a single TTS system (F5-TTS) and a single dataset (TORGO, restricted sentences) with a small speaker pool (8 dysarthric, 7 healthy); no human listening studies were conducted. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1638 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: eatsspeech_disentangled_emotion_branch role: supports claim: Treating emotion as a disentangled parallel synthesis branch improves emotional expressiveness over global style conditioning in zero-shot TTS. source: §3.6, Table 2 evidence: Ablation removing emotion decoupling reduces EMOS from 3.96 to 3.45 on LibriTTS test, with Emotion Discrepancy rising from 0.571 to 0.939, confirming that explicit emotion isolation drives the expressiveness gains. confidence: high relevance: high - claim_id: eatsspeech_llm_emotion_transform role: supports claim: LLM-based emotion transformation conditioned on target text semantics reduces emotion-content mismatch in zero-shot speech synthesis. source: §3.6, Table 2 evidence: Removing the emotion-aware LLM raises ED from 0.571 to 1.056 and drops EMOS from 3.96 to 3.73 on LibriTTS test, with both components contributing independently to emotional consistency. confidence: high relevance: high - claim_id: eatsspeech_direct_transfer_mismatch role: complicates claim: Direct reference emotion transfer in zero-shot TTS produces emotionally inconsistent speech when the reference and target texts differ in emotional valence. source: §3.3, §3.5, Table 1 evidence: All five comparison zero-shot TTS baselines (YourTTS, TransferTTS, VALL-E, E2-TTS, CosyVoice) show Emotion Discrepancy scores of 0.802 to 1.399 versus EATS-Speech at 0.571, suggesting that without explicit emotion adaptation, standard conditioning mechanisms misalign emotion to content. confidence: medium relevance: high - claim_id: eatsspeech_circular_metric_evaluation role: complicates claim: Evaluation of emotion expressiveness with metrics derived from the same model used in training introduces circularity that limits the reliability of reported Emotion Discrepancy scores. source: §2.1, §3.5 evidence: ED is computed using utterance-level emotion2vec embeddings, the same pre-trained model whose frame-level features the Emotion Tokenizer builds on; this means the metric and the system share the same representational basis, potentially inflating reported gains. confidence: medium relevance: low limitations: - Evaluation is limited to LibriTTS, an audiobook corpus with naturally constrained emotional variability; model size is not reported, and only English is evaluated. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1641 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: explicit_phoneme_position_supervision_during_autoregressive_codec_training role: supports claim: Explicit phoneme position supervision during autoregressive codec training eliminates alignment errors more effectively than phoneme identity prediction or monotonic decoding constraints. source: §4.2.1, Table 2; §4.3, Table 4 evidence: VALL-E R, the closest prior baseline, reduces deletion errors well (8.37% hard CER) but its greedy decoding variant degrades severely (19.33%), showing that phoneme identity prediction alone does not solve the fundamental alignment ambiguity in repeated-character contexts. confidence: high relevance: low - claim_id: alignment_failures_in_codec_language_model_tts_including role: supports claim: Alignment failures in codec language model TTS — including phoneme skipping, repetition, and one-to-many correspondence — are fundamentally a training-objective problem rather than an inference-time problem. source: §4.2.1, §4.3, Table 4 evidence: An alignment analysis on the hard set confirms that VALL-E makes 1,656 phoneme-skipping errors and 813 repetition errors; the proposed method eliminates all three error categories entirely (zero occurrences of skipping, repetition, and one-to-many alignment errors in Table 4). confidence: high relevance: low - claim_id: jointly_predicting_phoneme_identity_and_position_introduces_conflicting role: supports claim: Jointly predicting phoneme identity and position introduces conflicting signals that degrade pronunciation accuracy compared to position-only prediction. source: §4.2.1, Table 2 evidence: The ablation VALL-E BOTH (joint identity + position prediction) achieves 6.07% hard CER but introduces more substitution errors than position-only prediction, suggesting that conflicting phoneme identity and position signals interfere with pronunciation accuracy. confidence: high relevance: low - claim_id: robustness_improvements_in_autoregressive_codec_tts_can_be role: supports claim: Robustness improvements in autoregressive codec TTS can be achieved without changes to inference-time decoding strategy or additional duration prediction stages. source: §3.3, §4.2.1, Table 1 evidence: VALL-E R, the closest prior baseline, reduces deletion errors well (8.37% hard CER) but its greedy decoding variant degrades severely (19.33%), showing that phoneme identity prediction alone does not solve the fundamental alignment ambiguity in repeated-character contexts. confidence: high relevance: low limitations: - The model is trained and evaluated exclusively on Mandarin using a proprietary G2P toolkit and character-level duration annotations from the WenetSpeech4TTS dataset. Generalisation to languages without character-aligned duration labels, or to datasets where forced-alignment quality is lower, is untested. - The approach requires phoneme duration annotations at training time to construct the position sequence, which constrains its applicability to datasets with reliable forced alignments. At inference, the enrollment speech prompt must include a duration estimate — the paper does not discuss what happens when this estimate is imprecise. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1684 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: saras_speaker_aware_retrieval role: supports claim: Incorporating speaker identity into the retrieval query improves style compatibility in RAG-based zero-shot TTS. source: §3.5, Table 1 evidence: Ablation removing the speaker encoder from Zero-shot CLAP drops S-SMOS from 3.556 to 3.168, while the full model outperforms CA-CLAP (3.117) on the same metric, confirming that speaker conditioning directs retrieval toward each speaker's stylistic range. confidence: high relevance: high - claim_id: saras_style_timbre_disentangle_retrieval role: supports claim: Style-timbre disentanglement enables retrieval in a style-specific embedding space that better captures expressive variation than entangled audio representations. source: §2.1, §3.3, §3.4 evidence: Split-TTS uses Seed-VC to generate timbre-free NTF features for training, yielding a GST encoder whose embeddings cluster by emotion in t-SNE; this style encoder, used in Zero-shot CLAP retrieval, outperforms the CA-CLAP audio encoder on style-similarity MOS (3.556 vs 3.117). confidence: high relevance: high - claim_id: saras_style_fidelity_tradeoff role: complicates claim: Optimizing for style-compatible reference retrieval in zero-shot TTS can trade off against acoustic fidelity metrics. source: §3.3, Table 1 evidence: SA-RAS achieves better S-SMOS and ST-MOS than CA-CLAP but slightly higher MCD (6.86 vs 6.678), suggesting that the reference best matched for style is not always the one that minimises acoustic distortion. confidence: medium relevance: high - claim_id: saras_oracle_gap role: complicates claim: Retrieval-augmented zero-shot TTS consistently falls short of oracle (text-matched) reference selection by a substantial margin across both naturalness and style metrics. source: §3.4, Table 1 evidence: The proposed SA-RAS scores NMOS 3.827 and S-SMOS 3.556, versus the "Self" oracle at NMOS 4.046 and S-SMOS 3.980, a gap that persists even after speaker-aware retrieval improvements. confidence: medium relevance: high limitations: - The subjective evaluation involves only 15 listeners rating 15 samples drawn from three unseen speakers; the paper does not compare against end-to-end zero-shot TTS systems beyond the retrieval method comparison. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1726 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: minimal_readaloud_finetuning_matches_hourscale_spontaneous role: supports claim: Fine-tuning large-scale TTS models on minimal scripted read-aloud data (order of minutes) produces personalised voices with perceived AAC suitability comparable to fine-tuning on hour-scale spontaneous recordings. source: §3.2, Table 1 evidence: FT-Read (12 min of read-aloud sentences, SECS 0.643) and FT-Spont-Short (1h spontaneous, SECS 0.706) show no significant difference in MUSHRA-rated listener preference (p=0.213). confidence: high relevance: medium - claim_id: dysarthric_zeroshot_prompt_matches_typical_aac_suitability role: supports claim: Zero-shot TTS adaptation from dysarthric reference audio achieves perceptual suitability for assistive communication comparable to adaptation from typical speech, enabling personalisation for users who lack pre-morbid recordings. source: §3.2 evidence: ZS-Atypical and ZS-Typical show no significant difference in MUSHRA-rated AAC suitability (p=0.683), despite a large objective speaker similarity gap (SECS 0.324 vs. 0.635). confidence: high relevance: high - claim_id: secs_may_not_predict_aac_perceived_suitability role: refines claim: Embedding-based objective speaker similarity metrics may not reliably predict perceived suitability in assistive communication contexts, where other voice qualities matter more than timbre fidelity. source: §3.1, §3.2, Table 1 evidence: ZS-Atypical achieves the lowest SECS (0.324) of all evaluated systems yet is rated comparably to ZS-Typical (SECS 0.635) by human listeners in the MUSHRA-like AAC suitability test (p=0.683). confidence: high relevance: high - claim_id: dysarthric_prompt_yields_measurable_prosodic_alignment role: supports claim: Using dysarthric speech as a prosodic reference input for a fine-tuned TTS system yields measurable alignment with the speaker's intended prosodic pattern while preserving intelligibility. source: §5.2 evidence: Dysarthric-prompted synthesis achieved 29.3% higher SSIM in scalogram comparison of prosodic peaks versus unprompted synthesis (0.0548 vs. 0.0424, p=0.0013); CER improved slightly (2.9% to 2.3%). confidence: medium relevance: medium limitations: - The entire evaluation rests on a single research participant (one Hungarian stroke survivor); generalisability across speakers, languages, and impairment types is untested. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1779 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - VC architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: reflowvc_rectified_flow_efficiency role: supports claim: Rectified flow achieves comparable or better sample quality to diffusion for voice conversion at substantially fewer inference steps. source: §4.2, Table 1 evidence: ReFlow-VC with the RK45 solver matches Diff-VC's naturalness (NMOS 3.78 vs 3.75) while substantially outperforming it on speaker similarity (SECS 0.843 vs 0.781) and content preservation (WER 4.84% vs 13.92%), and a one-step Euler solver matches Diff-VC's 30-step output at roughly 47x lower latency. confidence: high relevance: high - claim_id: reflowvc_content_pitch_speaker_fusion role: supports claim: Conditioning speaker embeddings on concurrent content and pitch features improves speaker similarity in zero-shot voice conversion. source: §3.1, §4.2, Table 1 evidence: The feature fusion module combines HuBERT-Soft content features, VQ-VAE speaker-independent pitch representations, and a speaker embedding via cross-attention and gated fusion; the ablation (NReFlow-VC without fusion) shows the gated fusion module accounts for most of the speaker similarity gain (SECS 0.751 to 0.830 at one step). confidence: high relevance: high - claim_id: reflowvc_single_step_quality role: supports claim: A single-step ODE solver can match the quality of dozens of diffusion steps when the flow trajectories are sufficiently linear. source: §4.2, Table 1 evidence: One-step Euler sampling achieves SECS 0.830 and WER 4.54%, comparable to Diff-VC's 30-step output (SECS 0.766, WER 13.8%), at 0.10s versus 4.68s inference time. confidence: medium relevance: low - claim_id: reflowvc_recursive_rectification_diminishing role: complicates claim: Recursive rectification (retraining on model-generated samples) yields only marginal improvements when the initial model is already well-trained. source: §4.2, Table 2 evidence: The 2-ReFlow-VC variant modestly improves one-step SECS (0.832 vs 0.830) but shows no advantage at 30 steps or with the RK45 solver, indicating diminishing returns from a second rectification pass on an already well-converged flow. confidence: medium relevance: low limitations: - Trained and evaluated on a 38-hour clean subset of LibriTTS with a single-GPU budget, evaluated only on same-domain test utterances; the pitch extraction step adds over 1 second of preprocessing latency for a ~9-second utterance. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-1993 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - VC architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: embedding_watermark_detection_directly_into_codec_encoder_training role: supports claim: Embedding watermark detection directly into codec encoder training is a viable alternative to post-hoc or hard-coded watermark gates for protecting open-source zero-shot TTS models. source: §2.2, §3.3.1 evidence: 'By modifying the codec encoder''s training objective, the paper embeds watermark detection implicitly: when the encoder encounters watermarked audio, it is trained to reconstruct a silent (all-zero) waveform rather than the original signal; for clean audio, reconstruction proceeds normally.' confidence: high relevance: high - claim_id: neural_codec_architectures_are_a_natural_intervention_point role: supports claim: Neural codec architectures are a natural intervention point for access-control in speaker-conditioned TTS because they mediate all speaker information transfer from prompt to synthesis. source: §1, §2.3 evidence: The key observation is that virtually all state-of-the-art zero-shot TTS systems route the speaker prompt through a pre-trained codec encoder before passing speaker information to the generative model. confidence: high relevance: high - claim_id: training_time_augmentation_with_common_audio_distortions_substantially role: supports claim: Training-time augmentation with common audio distortions substantially improves a codec's robustness to watermark removal attacks without degrading reconstruction quality on clean audio. source: §2.2, Table 1, Table 2 evidence: To make the codec robust against watermark removal attacks, 13 types of audio distortions (speed adjustment, resampling, echo, noise, low/high/band-pass filters, smoothing, dynamic range changes, and MP3/AAC/EnCodec compression) are applied to training utterances as augmentation. confidence: high relevance: low - claim_id: codec_level_defenses_for_voice_cloning_create_a role: supports claim: Codec-level defenses for voice cloning create a structural barrier to adaptation attacks because TTS models trained on modified codec distributions cannot be trivially swapped to unprotected codecs without retraining. source: §2.3, §3.3.2 evidence: The TTS model is retrained on clean data using the modified codec encoder, meaning that adapting it back to a standard codec would require substantial retraining effort — forming the primary barrier against adaptation attacks. confidence: high relevance: high limitations: - The defense is effective only against speech watermarked with the specific watermarking model (AudioSeal) used during codec training. A copyrighted voice that is unwatermarked — or that is protected with a different, unseen watermarking system — receives no protection. The attacker simply needs to avoid using an AudioSeal-watermarked prompt. - The evaluation is conducted only on the VALL-E architecture and EnCodec backbone; whether the approach generalizes to other zero-shot TTS architectures (e.g., flow-matching or diffusion-based systems) is not tested. All data is clean studio speech (LibriSpeech/LibriTTS-R); robustness in noisy or spontaneous speech conditions is unknown. The speed adjustment and low-pass filter attacks are reported as failures for the codec detector, though the authors argue these attacks also degrade clean prompts, partially neutralising them as practical bypass routes. No listening test compares watermark-rejected output against human expectations of what a "protection failure" looks and sounds like. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2031 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - encoder_decoder_voice_cloning claims: - claim_id: phonotactic_rule_assignment_improves_crossfamily_zeroshot_tts role: supports claim: For zero-shot TTS across language families, assigning G2P rules by phonotactic family rather than script family produces substantially more natural and intelligible synthesis. source: §5.3, Tables 1, 2 evidence: Sanskrit synthesis with Dravidian rules (Kannada system, MOS 4.12, WER 4.6%) substantially outperforms synthesis with Indo-Aryan rules (Hindi system, MOS 3.25, WER 28.0%). confidence: high relevance: high - claim_id: extended_shared_phoneme_inventory_enables_unseen_phone_zeroshot role: supports claim: A shared phoneme inventory extended via phonetic substitution can enable zero-shot synthesis for target languages with sounds absent from all source training sets. source: §4.2 evidence: The CLS was extended to represent nukta-diacritic consonants and borrowed aspiration in Kurukh by mapping them to nearest-neighbour base characters, enabling intelligible synthesis without Kurukh training data. confidence: medium relevance: high - claim_id: dialect_zeroshot_tts_requires_phonotactic_source_matching role: supports claim: Dialect-specific zero-shot TTS requires matching the source synthesiser to the dialect's phonotactic affiliation, not its surface script or nominal language grouping. source: §5.3, Table 1 evidence: Maharashtrian Konkani (IA phonotactics) is best with Marathi (MOS 3.64) while Canara Konkani (DR phonotactics) is best with Kannada (MOS 3.34), despite both dialects sharing a language label. confidence: high relevance: high - claim_id: crossfamily_synthesis_degrades_for_structurally_absent_features role: complicates claim: Cross-family zero-shot synthesis degrades for target languages with phonological features structurally absent from all available source synthesisers. source: §5.3 evidence: Kurukh synthesis struggles specifically with glottal stops across both Hindi and Kannada systems, and question intonation was misrendered as assertions in some utterances. confidence: medium relevance: high - claim_id: phonotactic_family_predicts_transfer_better_than_script role: refines claim: Linguistic proximity for zero-shot TTS transfer is captured more precisely by phonotactic family than by script family or geographic proximity. source: §5.3, Tables 1, 3 evidence: 'Sanskrit belongs to the IA family but achieves lower MOS with IA synthesisers (3.25) than with DR synthesisers (4.12), and MCD scores independently confirm the same pattern (IA: 8.48 vs. DR: 7.15/6.94).' confidence: high relevance: high limitations: - The evaluation relies on only four monolingual source synthesisers, and the anomalous MCD result for Maharashtrian Konkani (higher distortion with Marathi despite better MOS) is unresolved. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2189 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - encoder_decoder_voice_cloning claims: - claim_id: joint_acoustictextual_masking_beats_style_emotion_encoders role: supports claim: Combining acoustic context with text conditioning in a masked prediction framework produces more accurate prosody prediction than style encoders or emotion representations adapted to the task. source: §4.1, Table 1 evidence: ProMode outperforms StyleTTS2*, Wav2Vec2-SER*, and Emotion2Vec* across all F0 and energy metrics at frame and phoneme level on GigaSpeech Test. confidence: high relevance: low - claim_id: ser_representations_dont_transfer_to_prosody_prediction role: complicates claim: Self-supervised representations trained for emotion recognition do not transfer effectively to prosody prediction, even after full fine-tuning on prosody-labelled data. source: §4.1, Table 1 evidence: Emotion2Vec* and Wav2Vec2-SER* show markedly lower F0 RPA and higher RMSE than StyleTTS2* and ProMode. confidence: medium relevance: low - claim_id: dualdecoder_acousticonly_loss_prevents_encoder_collapse role: supports claim: A dual-decoder architecture with an auxiliary acoustic-only loss prevents the prosody encoder from collapsing onto text context in masked prediction, where removing the loss causes catastrophic degradation. source: §4.3, Table 1 evidence: Ablation removing AOL drops F0 RPA from 43.9% to 25.8%, the largest degradation of any ablation. confidence: high relevance: low - claim_id: taskagnostic_prosody_prediction_improves_downstream_tts role: supports claim: Task-agnostic prosody models that predict F0 independently can improve both naturalness and prosody preference in downstream TTS systems without requiring joint end-to-end training. source: §4.2, Table 2, §4.2.2 evidence: Replacing FluentSpeech's built-in pitch predictor with ProMode-predicted pitch improves UTMOS from 3.00 to 3.10, WER from 4.41% to 3.99%, and AutoPCP from 2.47 to 2.64. confidence: high relevance: low limitations: - All experiments are conducted on English GigaSpeech; ProMode requires ground-truth phoneme durations from a forced aligner during evaluation, and model size is not reported. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2283 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: [] relevance: low evidence_role: - evaluation_caution current_role: minor method_family: [] claims: - claim_id: pronunciation_metrics_correlate_better_with_accent_quality role: supports claim: Pronunciation-based objective metrics correlate more reliably with accent quality rankings than general intelligibility or naturalness proxies. source: §5.2, Table 2 evidence: PPG CosSim and PPG JS both achieved SRCC=0.9643 with hypothesised accent ranking (p=0.0005); VF RMSE achieved SRCC=0.9286 (p=0.0025). WER's SRCC was 0.6429 and not statistically significant (p=0.1194). confidence: high relevance: low - claim_id: wer_unreliable_for_underrepresented_accent_evaluation role: contradicts claim: WER is a reliable quality metric for evaluating TTS systems on underrepresented accents. source: §5.2, Table 2 evidence: WER ranked copysyn worse than xtts and xtts worse than corrupt30k, despite the hypothesised reverse quality ordering; WER's correlation with the quality ranking was non-significant (p=0.1194), attributed to ASR accent bias. confidence: high relevance: low - claim_id: transcriptions_improve_xab_statistical_efficiency role: supports claim: Providing reference transcriptions to listeners during accent similarity evaluation significantly improves statistical discriminability with fewer participants. source: §5.1, Figure 1 evidence: Baseline XAB with 15 listeners failed to reach p<0.05; XAB+trans+screen achieved significance within 10 valid submissions; the full design reached 64.1±6.7% preference for copysyn. confidence: high relevance: low - claim_id: highlight_tasks_improve_efficiency_but_double_completion_time role: complicates claim: Auxiliary highlight tasks in listening tests improve statistical efficiency but nearly double completion time, limiting benefit to settings with constrained listener pools. source: §5.1 evidence: XAB+trans+highlight took ~30.6 minutes vs. ~16.9 minutes for baseline XAB. confidence: medium relevance: low limitations: - The study evaluates a single accent variety (Edinburgh Scottish English) using two VCTK speakers; the high SRCC for speaker-similarity (WavLM) may be confounded by the corruption scheme affecting broader speaker modelling. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2447 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: speculative_decoding_adapted_for_speech_can_reduce_autoregressive role: supports claim: Speculative decoding adapted for speech can reduce autoregressive inference latency without measurable degradation in subjective naturalness or speaker similarity. source: §4.1, §4.2 evidence: WER rises from 3.67% to 5.70%, which the authors attribute to draft model training on data roughly 1/300th the size of CosyVoice 2's proprietary training corpus rather than to the speculative decoding mechanism itself — the draft model in isolation reaches 16.13% WER, whereas SSD's verification keeps it at 5.70%. confidence: high relevance: high - claim_id: speech_token_sequences_exhibit_many_to_one_mappings role: supports claim: Speech token sequences exhibit many-to-one mappings to perceived quality, enabling relaxed acceptance criteria that improve decoding throughput over strict token-distribution matching. source: §2.2, §4.3 evidence: The tolerance factor ablation (Table 3) shows that varying β from 0 to 0.4 consistently improves LM-RTF while quality metrics (NMOS, SS) remain stable, suggesting that speech token identity is weakly coupled to perceptual quality — a key property that makes the relaxed acceptance criterion viable. confidence: high relevance: low - claim_id: initialising_a_lightweight_draft_model_from_the_upper role: supports claim: Initialising a lightweight draft model from the upper layers of the target model provides immediate vocabulary alignment and reduces the data requirements for draft model training. source: §2.3 evidence: Initialisation uses the pre-trained upper layers (layers 19–24) of the target, giving immediate vocabulary alignment. confidence: high relevance: low - claim_id: inference_stage_acceleration_of_autoregressive_tts_is_achievable role: supports claim: Inference-stage acceleration of autoregressive TTS is achievable without fine-tuning the target model, preserving deployment flexibility for frozen production systems. source: §2, §4.1 evidence: 'This selective fine-tuning is roughly analogous to adapter tuning: the draft model learns speech-specific generation patterns in its trainable lower layers while the frozen upper layers preserve the semantic representations of the full target model.' confidence: high relevance: low limitations: - The WER increase (3.67% → 5.70%) is unexplained beyond a data-scale hypothesis. The draft model's limited training data (LibriTTS, ~580h vs. CosyVoice 2's proprietary corpus) is identified as the likely cause, but this is not verified experimentally — e.g., by scaling draft training data or by ablating with matched data. - Evaluation is restricted to a single target model (CosyVoice 2) on a single English benchmark. Generalisation of SSD to multilingual systems, streaming inference contexts, or multi-codebook AR models (e.g., VALL-E-style RVQ decoding) is not explored. The tolerance factor β is treated as a fixed hyperparameter tuned on objective metrics; its interaction with speaker diversity and domain shift is unexamined. The reported 1.4× speedup measures LM-RTF only and does not account for the draft model's own compute overhead in the total pipeline time. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2449 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: nonuniform_step_schedules_reduce_fm_tts_inference_steps role: supports claim: Non-uniform ODE time-step schedules can reduce flow-matching TTS inference to 7 steps with minimal quality degradation when steps are pruned from the later, near-linear portion of the trajectory. source: §3.2, §4.3, Table 1 evidence: Fast F5-TTS with 7-step EPSS scheduling achieves WER 1.74% and UTMOS 3.78 on Seed-TTS test-en, close to the 32-step baseline (WER 1.70%, UTMOS 3.76), while naive uniform 7-step sampling collapses to 7.96% WER on test-zh. confidence: high relevance: low - claim_id: early_curvature_phase_critical_for_fm_tts_quality role: supports claim: The early, high-curvature phase of flow-matching inference in TTS is disproportionately critical for quality, and uniform step reduction degrades performance primarily by under-sampling this phase. source: §3.1, §4.4, Table 3 evidence: Pruning later steps (schedule 6a) yields 2.41% WER at 6 steps versus 2.71% WER when pruning early steps (schedule 6b), confirming the trajectory-analysis motivation for EPSS. confidence: high relevance: low - claim_id: training_free_step_scheduling_generalises_across_fm_tts role: supports claim: Training-free step scheduling methods generalise across flow-matching TTS architectures without requiring model-specific retraining. source: §4.3, Table 1 evidence: EPSS generalises from F5-TTS to E2 TTS (333M params), recovering most of the quality lost by naive step reduction at the same NFE without any additional training. confidence: medium relevance: low - claim_id: low_nfe_quality_floor_in_fm_tts role: complicates claim: At very low NFE (below 6 steps), quality degradation in flow-matching TTS becomes abrupt rather than gradual, suggesting a practical floor for inference reduction without distillation. source: §4.4, Table 3 evidence: Ablation confirms quality degrades sharply below 6 NFE, and the paper does not explore combining EPSS with consistency or rectified-flow distillation to push further. confidence: medium relevance: low limitations: - Evaluation uses no subjective listening tests (MOS); quality assessment relies entirely on UTMOS, WER, and speaker cosine similarity, automatic proxies that may not fully capture perceptual degradation at low NFE. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2595 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: [] relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: zeroshot_tts_can_match_clinical_stimuli_intelligibility role: supports claim: Zero-shot TTS voice cloning models can replicate clinical speech assessment stimuli with psychometrically comparable intelligibility functions, meeting a necessary condition for their use as synthetic alternatives to recorded corpora. source: §3.1 evidence: Psychometric function thresholds for E2 (0.41 dB above original, p=0.053) and VALL-E X (0.05 dB, p=1.00) did not significantly differ from the original BKB corpus, and slopes were statistically indistinguishable across all three models. confidence: high relevance: high - claim_id: naturalness_quality_dont_guarantee_speaker_identity_replication role: complicates claim: High naturalness and speech quality in zero-shot TTS voice cloning do not guarantee accurate speaker identity replication, revealing a trade-off invisible to standard aggregate quality metrics. source: §3.2, §3.3 evidence: XTTSv2 achieved the highest MOS-X2 ratings and lowest intelligibility thresholds but received significantly fewer 'same speaker' responses; E2 achieved the best speaker replication consistency but lower naturalness, prosody, and social impression scores than XTTSv2. confidence: high relevance: high - claim_id: aggregate_quality_scores_mask_domain_critical_failures role: complicates claim: Aggregate TTS quality scores can mask domain-critical performance failures that only appear at specific signal-to-noise operating points. source: §3.1, §4 evidence: VALL-E X produced comparable psychometric function thresholds to the original BKB corpus but exhibited significant intelligibility deviations above -6 dB SNR, the range most critical for hearing-impaired listeners, a failure that MOS-X2 ratings did not capture. confidence: high relevance: low limitations: - The study evaluates models on a single target speaker with a specific accent (Southern Standard British English male), and results may not generalise across accents or speaking styles; the study population consisted exclusively of normal-hearing listeners. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2765 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: [] relevance: low evidence_role: - evaluation_caution current_role: minor method_family: [] claims: - claim_id: cmos_mushra_overestimate_naturalness_vs_deception_rates role: complicates claim: CMOS and MUSHRA scores can overestimate naturalness relative to human deception rates, because listeners in preference tests match stimuli to reference recordings rather than assess absolute human-likeness. source: §4.1, §4.2, Table 2 evidence: StyleTTS 2 (best open-source system on standard benchmarks) achieves only 50.89% mean HFR against a 74.11% human baseline despite reporting CMOS +0.28 on LJSpeech, and XTTS scores MUSHRA 76.58 (above the 74.78 human reference) yet fools listeners only 41.8% of the time. confidence: high relevance: low - claim_id: low_expressivity_benchmarks_set_artificially_low_evaluation_bar role: complicates claim: Evaluating TTS systems on benchmarks with low expressive variation sets an artificially low bar, as human recordings from such datasets are themselves infrequently identified as human. source: §4.1 evidence: The human reference recordings from LJSpeech/LibriTTS/LibriSpeech score a mean HFR baseline of only 74.11%, compared to 70.68% on the more expressive Expresso benchmark where commercial systems (PlayHT 71.49, ElevenLabs 69.85) match the human rate. confidence: high relevance: low - claim_id: commercial_tts_near_human_deception_rates_expressive_speech role: supports claim: Commercial TTS systems achieve near-human deception rates in zero-shot speaker adaptation to expressive conversational speech, while leading open-source systems remain substantially below this threshold. source: §4.2, Table 2 evidence: On Expresso, PlayHT (HFR 71.49) and ElevenLabs (HFR 69.85) approach the human reference rate (70.68), while the best open-source system F5-TTS reaches only 50.26 HFR, a 20-point gap. confidence: high relevance: high - claim_id: finetuning_narrows_but_doesnt_close_naturalness_gap role: refines claim: Fine-tuning on high-quality expressive data improves but does not close the gap between open-source TTS naturalness and human speech in deception-based evaluations. source: §4.3, Table 3 evidence: Fine-tuning on 40 hours of Expresso improved F5-TTS from 50.26 to 52.22 HFR and VoiceCraft from 30.52 to 43.45 HFR, but neither approached the human level. confidence: high relevance: medium - claim_id: digital_voice_quality_and_flat_prosody_drive_synthetic_detection role: supports claim: Digital voice quality artifacts and flat prosody are the primary perceptual cues that allow listeners to reliably identify synthetic speech from open-source systems. source: §4.4, Table 4 evidence: Granular HFR analysis attributes synthetic-speech detection primarily to digital voice quality (36.1%), unnatural pauses (22.8%), and flat/monotonic delivery (20.6%), markers on which commercial systems perform comparably to human speech. confidence: high relevance: medium limitations: - The study evaluates only US-English native listeners on English speech; findings about human deception rates may not generalise to other languages, accents, or listener populations with different familiarity with synthetic speech. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2787 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: vocal_effort_modulation_improves_intelligibility_in_noise role: supports claim: Vocal effort modulation in zero-shot TTS via audio prompt selection can produce meaningful intelligibility gains in adverse acoustic conditions. source: §2.1, §3.1 evidence: Llasa-synthesized speech at higher vocal effort levels showed up to 30% WER reduction in meeting room and cafeteria noise at low SNRs, with acoustic analysis confirming systematic increases in F0, intensity, and high-frequency spectral energy. confidence: high relevance: high - claim_id: vocal_effort_benefit_is_noise_and_snr_dependent role: complicates claim: The intelligibility benefit of vocal effort adaptation in TTS is noise-type and SNR-dependent, and becomes negligible in moderate-to-favorable acoustic conditions. source: §3.1 evidence: WER differences across effort levels converged above 6 dB SNR in most environments; most environments reached WER < 0.2 at SNR > 12 dB regardless of effort level. confidence: high relevance: low - claim_id: snr_and_noise_type_dominate_intelligibility_prediction role: complicates claim: SNR and noise type dominate intelligibility prediction; speaker identity, vocal effort, and text complexity add minimal predictive variance beyond those two factors. source: §3.2, Table 2 evidence: XGBoost regression (R2 = 0.65) dropped to R2 = 0.20 when SNR was removed and to R2 = 0.40 when noise type was removed, while removing vocal effort or voice type had near-zero effect (R2 = 0.64 in both cases). confidence: high relevance: high - claim_id: asr_wer_underestimates_perceptual_difficulty_of_noisy_speech role: complicates claim: ASR-based WER underestimates the perceptual difficulty of noisy speech, particularly in speech-like masking conditions and at threshold SNR regions. source: §4 evidence: Human listeners (n=35) exhibited non-monotonic WER patterns in cafeteria noise and sharp threshold effects not observed in wav2vec 2.0 ASR, and showed greater sensitivity to meeting room noise; human WER ranged 0.1-1.2 with qualitatively different SNR-response shapes. confidence: high relevance: low limitations: - The human listening experiment is small (n=35) and covers only a subset of the conditions evaluated by ASR, limiting statistical power for the human vs. ASR comparison. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-2815 published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: [] relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: zero_shot_tts_enables_personalized_voice_output_in_assistive_speech_appl role: supports claim: Zero-shot TTS enables personalized voice output in assistive speech applications by conditioning synthesis on short user-supplied reference audio, without requiring model retraining or large speaker-specific corpora. source: §2.1 evidence: XTTS is integrated into the AAC platform so that users record or upload a short voice clip; synthesis is then conditioned on this reference to produce speech in the target voice for any input text, with no fine-tuning step. confidence: high relevance: high - claim_id: integrating_cloud_based_generative_ai_into_assistive_applications_with_p role: complicates claim: Integrating cloud-based generative AI into assistive applications with privacy-sensitive users creates tension between expressive capability and data security when user speech or conversational content must be transmitted to third-party services. source: §2.3, §2.4 evidence: The paper flags that GPT-4o-based text expansion sends conversational content to third-party servers, and recommends replacing it with a locally run LLM before broad deployment; the same privacy concern applies to cloud-based image generation components. confidence: medium relevance: low limitations: - See the source paper page for the complete limitations and evaluation scope. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: interspeech-2025-raju25_interspeech published_date: "2025-08-17" entry_date: '2026-07-26' year: 2025 venue: Interspeech task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: dubbing_fm_multilingual_vocab_expansion role: supports claim: Multilingual adaptation of flow-matching TTS through vocabulary expansion and phoneme-based tokenisation can extend zero-shot speaker preservation to typologically diverse language families not covered by the base model. source: §2.2.2 evidence: F5-TTS fine-tuned on 1,626 hours across English and 9 Indian languages with an expanded vocabulary and phoneme-based tokenisation achieves zero-shot speaker-conditioned synthesis across all supported languages. confidence: low relevance: high - claim_id: dubbing_e2e_pipeline_reduces_effort role: supports claim: End-to-end automated dubbing pipelines integrating ASR, machine translation, and zero-shot TTS can substantially reduce manual effort in cross-lingual content localisation. source: §2.1, §3 evidence: The platform automates six sequential stages from audio extraction to output re-encoding, supports 9 Indian languages, and is reported in active use by content creators and educators. confidence: low relevance: high - claim_id: dubbing_duration_alignment_tradeoff role: complicates claim: Duration alignment for lip-synchronisation in automated dubbing requires a user-controlled trade-off between temporal precision and naturalness that cannot be resolved automatically. source: §2.3 evidence: Duration alignment is offered as a toggleable feature; when active it constrains flow-matching synthesis duration to source timestamps, potentially affecting naturalness, whereas disabling it produces sequential utterances with a fixed 1-second pause. confidence: low relevance: low limitations: - No quantitative evaluation (MOS, WER, or speaker similarity) is reported for the fine-tuned flow-matching model, making it impossible to verify synthesis quality against any baseline. - In-house training data covers only six of the ten supported languages; per-language performance is not broken out. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.15442' published_date: "2025-08-21" entry_date: '2026-07-26' year: 2025 venue: EMNLP task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: token_level_entropy_is_a_statistically_significant_predictor role: supports claim: Token-level entropy is a statistically significant predictor of hallucination in autoregressive LM-based TTS, enabling unsupervised detection of likely error regions before synthesis is complete. source: §2.1, §2.2 evidence: Uncertainty is measured via token-level entropy aggregated at character and utterance levels, adapting uncertainty-based hallucination detection from NLP to the speech modality. confidence: high relevance: low - claim_id: reformulating_autoregressive_speech_token_generation_as_trajectory_flow role: supports claim: Reformulating autoregressive speech token generation as trajectory flow optimization via GFlowNets enables reward-proportional sampling that reduces hallucination without modifying the inference procedure. source: §3.1, §3.2 evidence: GOAT treats the autoregressive generation of speech tokens as a trajectory flow optimization problem. confidence: high relevance: low - claim_id: an_intrinsic_reward_derived_from_the_backbone_lm role: supports claim: An intrinsic reward derived from the backbone LM's own probability distribution is sufficient to drive distribution alignment in post-training, without requiring human annotations or an external reward model. source: §3.2.2 evidence: 'Rather than relying on an external reward model or labeled data, GOAT uses an intrinsic reward: the cumulative log-probability of the generated token sequence under the backbone LM, sharpened by an inverse temperature T (0 < T < 1).' confidence: high relevance: low - claim_id: fine_grained_subtrajectory_optimization_is_essential_for_hallucination role: supports claim: Fine-grained subtrajectory optimization is essential for hallucination suppression in long and complex utterances; degrading to trajectory-level balance significantly reduces the error reduction. source: §4.4.5, Table 2 evidence: 'Ablation (Table 2): Enhanced SubTB reduces test-hard CER from 13.72% to 6.61%; degrading to plain Trajectory Balance yields 11.72–11.84%, demonstrating that fine-grained subtrajectory optimization is essential.' confidence: high relevance: low - claim_id: post_training_alignment_for_hallucination_suppression_in_one role: supports claim: Post-training alignment for hallucination suppression in one language generalizes to unseen languages, with cross-lingual models retaining meaningful gains over the untuned baseline. source: §4.4.4, Table 1 evidence: 'Cross-lingual generalization holds: models trained on one language and evaluated on the other still outperform the baseline.' confidence: high relevance: low limitations: - '- Evaluated on a single backbone (CosyVoice 2); generalization to other LM-based TTS architectures (VALL-E, BASE TTS, LLaSA) is asserted but not demonstrated. - The uncertainty-hallucination correlation explains only part of hallucination causes (correlation 0.636–0.649); prosody errors show no consistent uncertainty pattern and are not effectively addressed. - The internal reward assumes the backbone LM assigns higher probability to higher-quality sequences — an assumption the authors acknowledge is a hypothesis. - Training still requires H100 GPU access (~70 GB VRAM per GPU), which is non-trivial despite the small data requirement. - Reward temperature is a sensitive hyperparameter; the sweet spot (0.825) was found empirically and may differ for other backbones or domains. - No subjective listening evaluation (MOS/MUSHRA) is reported; quality improvements are measured entirely through ASR-based error rates and the automatic UTMOS predictor.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.16332' published_date: "2025-08-22" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - VC - singing architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: joint_pre_training_on_speech_and_singing_voice role: supports claim: Joint pre-training on speech and singing voice data improves generation quality in both domains, with singing data contributing to expressive speech prosody richness and speech data compensating for the limited size of singing corpora. source: §V-A, Table I evidence: On singing voice, Vevo2 substantially outperforms zero-shot TTS baselines in N-CMOS (gap > 1.5) and SS-CMOS (gap > 0.9), validating the benefit of joint training. confidence: high relevance: high - claim_id: chromagram_based_prosody_tokenization_provides_a_notation_free role: supports claim: Chromagram-based prosody tokenization provides a notation-free, octave-invariant representation that bridges the F0 distribution gap between speech and singing, enabling unified prosody control without expert MIDI annotations. source: §III-A evidence: 'Two custom VQ-VAE tokenizers are introduced: - Prosody tokenizer (6.25 Hz, 56.25 bps, codebook 512): encodes chromagram features, which are octave-free (bridging the F0 distribution gap between speech and singing) and notation-free (extractable without MIDI annotation).' confidence: high relevance: low - claim_id: autoregressive_models_initialized_from_llm_checkpoints_can_achieve role: supports claim: Autoregressive models initialized from LLM checkpoints can achieve competitive zero-shot TTS quality at low codec frame rates (12.5 Hz) when paired with strong post-training, partially compensating for the sequence granularity disadvantage relative to higher frame-rate systems. source: §V-A, Table VII evidence: On the SeedTTS benchmark (regular speech), post-trained Vevo2 achieves WER 3.64 (en) / 2.94 (zh) and SIM-o 0.693 / 0.754, competitive with CosyVoice 2 and MaskGCT despite using a lower frame-rate (12.5 Hz vs 25–50 Hz) content-style tokenizer. confidence: high relevance: high - claim_id: single_objective_preference_alignment_in_multi_capability_speech role: supports claim: Single-objective preference alignment in multi-capability speech models degrades complementary objectives; joint multi-objective optimization is necessary to avoid quality regressions on non-targeted capabilities. source: §V-F, Figure 5 evidence: Single-objective optimization degrades the complementary objective (intelligibility-only training drops melody accuracy from 65% to 50%); joint optimization achieves both gains. confidence: high relevance: low - claim_id: inference_time_pitch_shift_applied_to_prosody_tokens role: complicates claim: Inference-time pitch shift applied to prosody tokens improves speaker similarity in voice and singing conversion tasks, at the cost of a minor intelligibility degradation due to train-inference distribution mismatch. source: §V-E, Table VI evidence: 'Inference-time controllability: The system flexibly recombines text, prosody source, style reference, and timbre reference to perform TTS, VC, SVS, SVC, speech/singing editing, duration control (97%+ accuracy via chromagram length scaling), and pitch region control (via F0 shift before prosody token extraction).' confidence: high relevance: high limitations: - '- Training-inference mismatch when applying pitch shift: content-style tokens from pitch-shifted audio are out-of-distribution, causing slight WER degradation. The paper flags future work to include pitch shift augmentation during training. - Singing voice data used (7K hours source-separated from in-the-wild songs) may have variable quality. - The 12.5 Hz frame rate of the content-style tokenizer slightly limits VC intelligibility compared to Vevo-FM (50 Hz), as shown in Table VIII. - Melody-MOS for Vevo2 in SVC is slightly lower than the FM-only Vevo2-FM, suggesting a quality-style tradeoff when adding text input. - Generalization to tonal languages in singing is not specifically evaluated.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.16790' published_date: "2025-08-22" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - codec - TTS architecture: - diffusion - transformer-enc-dec - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_diffusion_acoustic claims: - claim_id: text_conditioning_in_the_codec_decoder_rather_than role: supports claim: Text conditioning in the codec decoder, rather than in the language model alone, is a viable lever for achieving extreme compression rates in speech tokenization without adversarial training. source: §3.1, Table 4 evidence: In speech LM contexts — TTS, spoken dialogue — the target text is always available, so conditioning on it effectively offloads content information from the compressed token to the conditioning signal, enabling much more aggressive compression. confidence: high relevance: low - claim_id: a_single_end_to_end_training_objective_flow role: supports claim: A single end-to-end training objective (flow-matching loss) is sufficient to jointly optimise quantization and reconstruction in a speech codec, eliminating the need for multi-stage pipelines. source: §3.1, §4.2.2 evidence: TaDiCodec reframes the codec problem as an end-to-end diffusion autoencoder trained with a single flow-matching loss. confidence: high relevance: low - claim_id: the_reconstruction_generation_gap_the_degradation_in_intelligibility role: supports claim: The reconstruction-generation gap — the degradation in intelligibility when tokens are predicted by a language model rather than encoding reference speech — varies substantially across tokenizer architectures and is not captured by reconstruction metrics alone. source: §4.3, Figure 3 evidence: The low token rate (6.25 Hz vs. 25–50 Hz for typical AR TTS) means each second of speech is predicted as only 6–7 tokens, drastically shortening the generation sequences. confidence: high relevance: low - claim_id: lower_token_rates_in_speech_tokenizers_can_improve role: supports claim: Lower token rates in speech tokenizers can improve autoregressive TTS intelligibility by shortening prediction sequences and reducing error accumulation, particularly on linguistically challenging inputs. source: §4.3, Table 5 evidence: The low token rate (6.25 Hz vs. 25–50 Hz for typical AR TTS) means each second of speech is predicted as only 6–7 tokens, drastically shortening the generation sequences. confidence: high relevance: low - claim_id: binary_spherical_quantization_without_a_commitment_loss_achieves role: supports claim: Binary Spherical Quantization without a commitment loss achieves stable end-to-end training of a speech codec and produces superior representations to standard VQ under equal codebook sizes. source: §4.2.2, Table 4 evidence: The encoder is an 8-layer Llama-style bidirectional Transformer operating on mel-spectrogram frames; its output is downsampled and projected to a 14-dimensional latent, then quantized using Binary Spherical Quantization (BSQ) — an implicit-codebook scheme that projects encoder embeddings onto a unit hypersphere and binarises each dimension, yielding a codebook of size 2¹⁴ = 16,384 tokens without a learnable codebook or commitment loss. confidence: high relevance: low limitations: - 'TaDiCodec''s text-aware decoder is not a general-purpose audio codec: it requires a transcript at both training and inference time. The strong reconstruction and TTS results are conditional on text availability; performance at 6.25 Hz without text conditioning is not competitive (WER exceeds 10% at 12.5 Hz without text, per Table 4). This limits applicability to codec-transmission, speech enhancement, or any scenario where transcriptions are unavailable.' - The diffusion decoder introduces multi-step inference latency. At 32 steps, decoding speed is acceptable for generation but higher than GAN vocoders; reducing to 5 steps degrades quality noticeably. The authors propose distillation as future work but have not yet demonstrated single-step performance. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.17031' published_date: "2025-08-23" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - encoder_decoder_voice_cloning claims: - claim_id: rephrasetts_crossmodal_beats_global_speaker role: supports claim: Cross-modal attention between phoneme and audio representations is more effective than global speaker embeddings for preserving localised prosodic and acoustic context in speech insertion tasks. source: §4.3, Table 3 evidence: Ablation experiments show that removing cross-modal attention and replacing it with a global speaker encoder worsens MCD on dev-clean, confirming fine-grained contextual style transfer outperforms speaker-level global conditioning. confidence: medium relevance: medium - claim_id: rephrasetts_nar_variable_length_no_alignment role: supports claim: Non-autoregressive TTS architectures with duration prediction can infer variable-length insertion segments without explicit phoneme-level alignment at inference time. source: §3.1, §3.2 evidence: The FastSpeech2-style variance adaptor predicts phoneme-level duration at inference; MFA-derived alignments are used only for training supervision, enabling variable-length output without alignment at inference time. confidence: medium relevance: low - claim_id: rephrasetts_multiloss_reduces_artefacts role: supports claim: Combining local and global adversarial losses with a triplet-based style matching loss reduces perceptual artefacts in synthesised speech segments that L1 reconstruction loss alone cannot eliminate. source: §3.3, §4.3 evidence: Phase two training adds local/global LSGAN losses plus a triplet-based style matching loss on top of phase one's L1 reconstruction; the ablation in Table 3 shows each component contributes to perceptual quality on dev-clean. confidence: medium relevance: low - claim_id: rephrasetts_local_context_graceful_degradation role: supports claim: Speech insertion quality degrades more gracefully with insertion length when the model conditions directly on local audio context rather than full-utterance speaker embeddings. source: §4.2, Table 1 evidence: On long insertions, RephraseTTS scores 3.97 MOS versus 2.80 for MetaStyleSpeech (which relies on global speaker embeddings), a much smaller degradation from the short-insertion MOS scores of both systems. confidence: medium relevance: medium limitations: - Evaluation is limited to clean LibriTTS dev-clean/dev-other read speech; the user study comprises only 15 utterances rated by 6 annotators; SpeechPainter, the closest neural inpainting baseline, is not evaluated quantitatively because its task formulation differs. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.19098' published_date: "2025-08-26" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: replacing_discrete_neural_codec_tokens_with_continuous_vae role: supports claim: Replacing discrete neural codec tokens with continuous VAE latents in an autoregressive TTS framework reduces the number of AR decoding steps and lowers inference RTF without sacrificing speech quality. source: §5.1, Table 2 evidence: 'On LibriSpeech-PC test-clean Subset-B (1,127 samples, F5-TTS evaluation protocol): - CLEAR-Large: WER 1.88%, SPK-SIM 0.59, UTMOS 4.22 - CLEAR-Base: WER 2.21%, SPK-SIM 0.59, UTMOS 4.22 - Compared to DiTAR (600M, Emilia-100k): CLEAR-Large achieves 0.51% absolute WER reduction (21.3% relative) at lower parameter count and with far fewer AR decoding steps On LibriSpeech test-clean Subset-A (40 samples, NaturalSpeech3 protocol): - CLEAR-Large: WER 1.74%, SPK-SIM 0.56, UTMOS 4.26 - CLEAR-Base: WER 1.83%, SPK-SIM 0.55, UTMOS 4.21 - Outperforms VALL-E (WER 6.11%), MegaTTS (WER 2.32%), NaturalSpeech2 (WER 1.94%) Inference efficiency (Table 2, 10-second generation on single GPU): - CLEAR-Base: RTF 0.18, avg. 78 AR decoding steps (vs. VALL-E 750 steps / RTF 1.03; MELLE 620 steps / RTF 0.55; F5-TTS RTF 0.31) - CLEAR-Large: RTF 0.29 Subjective evaluation on Subset-B vs. F5-TTS and CosyVoice-2: - CLEAR-Base: N-MOS 4.09, Q-MOS 4.14, S-MOS 4.02, CMOS +0.04 (outperforms ground truth on N-MOS and Q-MOS) Streaming synthesis (Table 4, chunk size 4): first-frame latency 96ms, WER 2.34%, UTMOS 4.27 — largely matching non-streaming performance.' confidence: high relevance: medium - claim_id: a_per_token_mlp_rectified_flow_head_that role: supports claim: A per-token MLP rectified flow head that operates independently on each AR hidden state enables streaming speech synthesis with latency competitive with non-streaming systems. source: §5.3, Table 4 evidence: Importantly, the MLP operates independently on each conditioning vector, so denoising can start as soon as the first AR hidden state is available — enabling streaming without the full-sequence wait imposed by DiT-based heads. confidence: high relevance: low - claim_id: high_compression_ratio_vae_latents_2048x_downsampling_can role: supports claim: High-compression-ratio VAE latents (2048x downsampling) can achieve reconstruction quality comparable to ground truth on standard benchmarks while producing sequences short enough to benefit AR sequence modeling. source: §3.4, Appendix C.1 evidence: To maintain reconstruction quality under this aggressive compression, parameter-free shortcut connections perform space-to-channel reshaping with channel averaging in each downsampling block and the reverse in each upsampling block. confidence: high relevance: low - claim_id: continuous_ar_tts_systems_trained_with_a_joint role: supports claim: Continuous AR TTS systems trained with a joint language model and flow-matching head in a single stage can match or exceed the WER of two-stage cascaded discrete AR systems trained on comparable data. source: §5.1, Table 1 evidence: 'The system has three components trained end-to-end in a single stage: Wav-VAE.' confidence: high relevance: low - claim_id: objective_speaker_similarity_metrics_underestimate_the_perceptual_voice role: supports claim: Objective speaker similarity metrics underestimate the perceptual voice fidelity of AR systems that rely entirely on in-context learning from an audio prompt, as shown by divergence between WavLM-TDNN scores and subjective S-MOS ratings. source: §5.2, Table 3, Appendix D.2 evidence: 'For zero-shot synthesis, a short audio prompt and its transcription are prepended as context: `[S, x_prompt, x_target, T, y_prompt]`; the model then autoregressively completes y_target.' confidence: high relevance: high limitations: - 'The paper explicitly acknowledges: (1) objective speaker similarity (WavLM-TDNN) lags behind models that inject speaker embeddings directly into the generator (e.g., CosyVoice, NaturalSpeech3 with FACodec); CLEAR relies entirely on in-context learning from the audio prompt with no explicit speaker conditioning, which is a design choice that limits speaker fidelity under high compression. (2) Evaluation is English-only; multilingual generalization is untested. (3) Sequential single-token AR generation; multi-token prediction could improve throughput further. Additionally, training data for VAE (LibriTTS, 585h) is much smaller than TTS training data (LibriHeavy, 50kh), and the interaction between VAE compression ratio and downstream TTS quality could be investigated more rigorously in full-scale settings.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2508.19205' published_date: "2025-08-26" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: extreme_acoustic_codec_compression_single_codebook_vae_at role: supports claim: Extreme acoustic codec compression (single-codebook VAE at 7.5 Hz) can achieve superior perceptual quality over multi-codebook discrete codecs operating at much higher frame rates. source: §3.3, Table 3 evidence: The acoustic tokenizer comparison (Table 3) shows that at 7.5 Hz — 40× fewer tokens than WavTokenizer at 40 Hz — the proposed tokenizer achieves PESQ 3.068 and UTMOS 4.181 on LibriTTS test-clean, beating all discrete codec baselines including DAC (UTMOS 3.433) and WavTokenizer 75 Hz (UTMOS 4.049). confidence: high relevance: low - claim_id: long_form_multi_speaker_tts_benefits_from_separate role: supports claim: Long-form multi-speaker TTS benefits from separate acoustic and semantic tokenizers trained with task-specific objectives rather than a single unified codec. source: §2.1 evidence: A separate semantic tokenizer mirrors the encoder architecture but is trained via an ASR proxy task, producing content-aligned representations; this component is discarded after pre-training and used only to provide hybrid acoustic-semantic context tokens during generation. confidence: high relevance: medium - claim_id: scaling_the_llm_backbone_in_a_next_token role: supports claim: Scaling the LLM backbone in a next-token diffusion speech system yields consistent gains in perceptual quality, speaker similarity, and expressiveness. source: §3.1, Table 1 evidence: 'VibeVoice (paper name: VIBEVOICE) resolves the context-length bottleneck through two innovations that work in tandem: an ultra-compressed acoustic tokenizer and a next-token diffusion generation framework.' confidence: high relevance: high - claim_id: token_level_diffusion_conditioned_on_llm_hidden_states role: supports claim: Token-level diffusion conditioned on LLM hidden states enables streaming speech generation without the codebook constraints of discrete autoregressive systems. source: §2.2 evidence: The LLM processes this hybrid context and produces hidden states that condition a lightweight token-level diffusion head (4 transformer layers) adapted from the MAR image generation framework. confidence: high relevance: low - claim_id: tts_systems_optimised_for_long_form_conversational_content role: supports claim: TTS systems optimised for long-form conversational content retain competitive performance on short-utterance benchmarks without dedicated fine-tuning. source: §3.2, Table 2 evidence: On the SEED short-utterance benchmark (CommonVoice English/Chinese), VibeVoice-1.5B achieves CER of 1.16% and SIM of 0.744 for Chinese, and WER of 3.04% and SIM of 0.689 for English, despite being primarily trained for long-form content. confidence: high relevance: medium limitations: - Training data is not disclosed. The paper is from Microsoft Research but does not specify the data composition, size, or any cleaning procedures, making it impossible to assess whether the reported gains are attributable to architecture or data advantage. - The model is limited to English and Chinese; other languages produce unpredictable outputs. The system does not model overlapping speech — a significant gap for realistic conversational audio. The subjective evaluation used only 8 long-form test conversations, which is a narrow sample; standard benchmark evaluations (SEED) are short-utterance only and do not capture the long-form quality the paper targets. Speaker similarity at 7.5 Hz remains below the best short-utterance systems (e.g., Seed-TTS at 0.762 SIM for English), suggesting the compressed representation sacrifices some speaker identity fidelity. The maximum of 4 speakers is a hard constraint imposed by the context design. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2411.19770' published_date: "2025-08-28" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - VC architecture: - diffusion relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - diffusion_voice_cloning claims: - claim_id: noro_contrastive_noise_invariant_speaker role: supports claim: Contrastive training with noise-augmented views enforces noise-invariant speaker representations and substantially improves one-shot VC robustness at low SNR without adding inference cost. source: §III.A, Tables I-II evidence: Noro's dual-branch reference encoding module and noise-agnostic contrastive speaker loss hold SECS at 80.09 and CER at 4.66 in 0-5 dB noise, versus 77.28 and 7.26 for the baseline; weight sharing means inference architecture is unchanged. confidence: high relevance: medium - claim_id: noro_baseline_degrades_under_noise role: complicates claim: Standard one-shot voice conversion systems degrade substantially when reference speech contains background noise, even after data augmentation training. source: §III.A.2, Table II evidence: The diffusion-based baseline trained on LibriLight with no noise-robustness mechanism shows CMOS dropping from 3.29 to 2.09 and SMOS from 3.02 to 2.75 under 0-5 dB noisy reference conditions. confidence: high relevance: medium - claim_id: noro_vc_encoder_competitive_speaker_verification role: supports claim: Voice conversion reference encoders trained on large-scale speech data develop speaker representations competitive with dedicated self-supervised speaker models. source: §III.B.2, Table III evidence: VC-SPK2VEC (the Noro baseline reference encoder repurposed as a speaker encoder, 72.4M params, trained on LibriLight) achieves 5.32% EER on VoxCeleb1 under SUPERB, outperforming wav2vec 2.0 Base (6.02%) and Large (5.65%) and HuBERT Large (5.98%). confidence: medium relevance: medium - claim_id: noro_explicit_alignment_beyond_augmentation role: refines claim: Speaker-noise disentanglement in voice conversion benefits from training objectives that explicitly align clean and noisy representations of the same speaker, beyond simple noise augmentation. source: §III.A.2, Figure 2 evidence: t-SNE visualisations show that the baseline (trained with augmentation but no contrastive alignment) produces clearly separated clean/noisy representation clusters, while Noro's contrastive loss causes them to mix, correlating with the performance gap under noise. confidence: medium relevance: medium limitations: - Evaluation uses VCTK studio-recorded English with synthetically added DEMAND noise; the test set is small (150 objective pairs, ten per condition for subjective evaluation); the contrastive loss requires speaker labels at training time, so the approach does not extend to fully unsupervised training; the secondary VC-SPK2VEC finding is evaluated under only one SUPERB protocol. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2507.14534' published_date: "2025-08-30" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - VC architecture: - GAN - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - gan_based_voice_synthesis claims: - claim_id: conan_distillation_recovers_causal_content role: supports claim: Streaming voice conversion quality degrades significantly when offline HuBERT representations are naively replaced by causal alternatives, but knowledge distillation into an Emformer backbone can recover content accuracy with acceptable latency. source: §III.B, Table III evidence: Removing the Stream Content Extractor (distilled from offline HuBERT via an Emformer backbone) raises WER to 26.92% and drops CMOS-Q by -0.33, confirming that naive causal substitution for offline SSL representations degrades content accuracy without distillation. confidence: medium relevance: medium - claim_id: conan_pixelshuffle_avoids_checkerboard role: supports claim: Causal temporal upsampling via pixel shuffle eliminates the checkerboard artifacts introduced by zero-padding non-causal vocoders, without sacrificing subjective quality. source: §III.D, Table III evidence: Replacing the Causal Shuffle Vocoder with a zero-padding causal HiFi-GAN variant drops CMOS-Q by -0.21, confirming pixel-shuffle upsampling avoids the artifacts of naive causal conversion. confidence: medium relevance: low - claim_id: conan_cvq_style_beyond_timbre_embeddings role: supports claim: Explicit style modeling with clustering-based vector quantization improves zero-shot speaker similarity in streaming VC beyond what timbre embeddings alone provide. source: §III.C, Table I evidence: Removing the Adaptive Style Encoder (CVQ-based) drops CMOS-S by -0.19; Conan (Full) achieves the highest speaker similarity (85.71%) among all compared systems, the only online system to exceed 85%. confidence: medium relevance: high - claim_id: conan_chunklevel_matches_offline_similarity role: supports claim: Online voice conversion systems can achieve speaker similarity comparable to offline systems when style transfer is modeled at chunk level rather than at the global utterance level. source: §IV.B, Table I evidence: Conan Full's chunk-level style representation achieves SIM 85.71%, exceeding the non-causal offline baselines BNE-PPG-VC (79.06%) and VQMIVC (62.54%). confidence: medium relevance: high limitations: - Evaluation is English-only; the reference speaker must be fully available before streaming begins, limiting real-time reference scenarios; model size is not reported; perceptual evaluation used only 15 listeners per pair on Amazon Mechanical Turk. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.02020' published_date: "2025-09-02" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - SCA architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: reducing_speech_tokenizer_frame_rate_to_12_5hz role: supports claim: Reducing speech tokenizer frame rate to 12.5Hz with explicit semantic supervision produces tokens that enable more stable text-to-token modelling over long dialogue sequences than higher-rate tokenizers without semantic injection. source: §2.1, §4.1, Table 1 evidence: Crucially, the quantised features are passed through a semantic decoder that predicts the original Whisper features (explicit semantic supervision), stabilising text-to-token modelling over long sequences. confidence: high relevance: low - claim_id: a_dual_transformer_architecture_for_multi_layer_rvq role: supports claim: A dual-transformer architecture for multi-layer RVQ prediction achieves lower first-packet latency than the delay-pattern while providing stronger contextual conditioning from prior turns. source: §2.2 evidence: This dual-transformer avoids the delay-pattern's drawbacks (weak contextual conditioning, high first-packet latency) and achieves under 100ms first-packet latency. confidence: high relevance: low - claim_id: autoregressive_tts_systems_trained_on_multi_speaker_dialogue role: supports claim: Autoregressive TTS systems trained on multi-speaker dialogue data with text-speech interleaved formatting can infer and adjust prosody and emotion from implicit conversational context without explicit emotion labels. source: §3.2, §4.3, Table 3 evidence: 'Emotion inference (interactive chat): 76.7–93.3% accuracy across 6 emotions (surprise, sadness, happiness, concern, apology, anger) inferred from implicit contextual cues without explicit emotion labels.' confidence: high relevance: medium - claim_id: sentence_by_sentence_multi_speaker_dialogue_tts_systems role: supports claim: Sentence-by-sentence multi-speaker dialogue TTS systems produce more coherent prosody across turns than approaches that concatenate monologue TTS outputs or model a mixed audio track. source: §4.4, Table 4 evidence: 'Three-stage curriculum training: pretraining on 1.1M hours monologue, post-training on 300k hours 2–5 speaker dialogue data, then task-specific SFT (eg. 15h emotion corpus for chat, 50h podcast hosts for podcast fine-tuning).' confidence: high relevance: medium - claim_id: fine_tuning_a_post_trained_dialogue_tts_model role: supports claim: Fine-tuning a post-trained dialogue TTS model on as little as 50 hours of speaker-specific data is sufficient to produce synthesis that is perceptually indistinguishable from human recordings in a majority of trials. source: §4.4, Figure 4 evidence: 'Three-stage curriculum training: pretraining on 1.1M hours monologue, post-training on 300k hours 2–5 speaker dialogue data, then task-specific SFT (eg. 15h emotion corpus for chat, 50h podcast hosts for podcast fine-tuning).' confidence: high relevance: medium limitations: - '- Currently limited to 3-minute dialogues with up to 4 speakers; scaling requires extending training corpus. - English speaker similarity (SIM 0.665) lags Mandarin (0.736), attributed to limited English voice diversity in training data — a data rather than architectural limitation. - Trails Mimi on PESQ metrics, likely because Mimi was trained on a massive English-only corpus closely matching LibriSpeech. - Emotion fine-tuning is demonstrated for a single distinctive female voice; generalisation to arbitrary voices and more nuanced emotional transitions is not evaluated. - No ablation of the semantic supervision contribution vs. the lower frame rate independently.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2502.11128' published_date: "2025-09-03" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: autoregressive_context_from_the_preceding_mel_spectrogram_frame role: supports claim: Autoregressive context from the preceding mel-spectrogram frame provides a more informative prior for per-token flow matching, enabling high-quality synthesis with fewer function evaluations than a standard Gaussian prior. source: §4.2, §6.2, Table 3 evidence: Rather than using a standard Gaussian prior, FELLE initialises the flow-matching ODE for frame i from the previous predicted frame x_{i-1} via a Gaussian centred at that frame (variance sigma^2 = 0.1). confidence: high relevance: low - claim_id: hierarchical_coarse_to_fine_spectral_decomposition_within_each role: supports claim: Hierarchical coarse-to-fine spectral decomposition within each flow-matching step improves speaker similarity over holistic or fully decoupled generation approaches in continuous-token autoregressive TTS. source: §4.2, §6.2, Table 3 evidence: Token-wise coarse-to-fine flow matching (C2F-FM). confidence: high relevance: high - claim_id: continuous_mel_spectrogram_autoregressive_models_can_match_discrete role: supports claim: Continuous mel-spectrogram autoregressive models can match discrete-token systems on intelligibility while substantially surpassing them on speaker similarity in zero-shot TTS. source: §6.1, Table 2 evidence: 'Evaluations on LibriSpeech test-clean use two zero-shot schemes: continuation (same utterance, first 3 s as prompt) and cross-sentence (different utterance from the same speaker as prompt).' confidence: high relevance: high - claim_id: the_number_of_flow_matching_function_evaluations_introduces role: complicates claim: 'The number of flow-matching function evaluations introduces a trade-off in autoregressive TTS: moderate NFE improves intelligibility but excessive NFE degrades both intelligibility and speaker similarity.' source: §6.3, Figure 3 evidence: 'Ablations confirm all three novel components contribute: removing the previous-frame prior degrades both WER and SIM; replacing C2F-FM with holistic (HFM) or decoupled (DFM) flow matching degrades one or both metrics.' confidence: high relevance: high limitations: - '- The model operates at 16 kHz with 80-band mel features; extension to 24 kHz or higher fidelity representations is not addressed. - Inference is significantly slower than non-autoregressive systems due to the frame-by-frame autoregressive loop compounded with 3 flow-matching NFEs per frame. Streaming or parallel decoding is not explored. - Training and evaluation are English-only (LibriSpeech); multilingual zero-shot capability is not tested. - The LM parameter count is not stated, making it difficult to assess compute efficiency relative to MELLE or VALL-E. - The MOS evaluation uses automatic prediction (RAMP+) rather than human listeners; discrepancies with human MOS are possible, especially when comparing systems at similar quality levels. - Coarse-to-fine decomposition uses a fixed downsampling strategy (even-indexed frames); more principled spectral decompositions (e.g., sub-band filterbanks) are unexplored.' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2506.21619' published_date: "2025-09-03" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - GAN - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: tying_the_duration_target_embedding_to_the_semantic role: supports claim: Tying the duration-target embedding to the semantic positional embedding table enables near-perfect token-count precision in autoregressive TTS without sequence-level supervision. source: § Proposed Method — Duration Control, Table 4 evidence: For duration control (Table 4), token number error rate is below 0.02% at 1× scaling and below 0.07% even at 0.75× compression on SeedTTS test-zh — near-perfect precision for an autoregressive model. confidence: high relevance: low - claim_id: gradient_reversal_layer_training_effectively_disentangles_emotional_prosody role: supports claim: Gradient Reversal Layer training effectively disentangles emotional prosody from speaker timbre in zero-shot TTS, enabling independent control of expression and identity from separate reference signals. source: § Proposed Method — Emotional Control, Table 2 evidence: A Gradient Reversal Layer (GRL) during training forces `e` to be invariant to speaker timbre, while a frozen speaker perceiver conditioner extracts `c` (timbre). confidence: high relevance: high - claim_id: gpt_latent_fusion_between_the_autoregressive_semantic_module role: supports claim: GPT-latent fusion between the autoregressive semantic module and the flow-matching acoustic module improves phonetic clarity under high emotional expressiveness, as measured by WER on the emotional test set. source: § Semantic-to-Mel Module, Table 2 evidence: 'IndexTTS2 is a three-module cascade: a Text-to-Semantic (T2S) autoregressive transformer, a Semantic-to-Mel (S2M) flow-matching model, and a BigVGANv2 vocoder.' confidence: high relevance: low - claim_id: a_three_stage_curriculum_that_separates_base_capability role: supports claim: A three-stage curriculum that separates base capability training from emotion-conditioner fine-tuning is essential for emotional expressiveness; removing it causes EMOS to collapse from 4.22 to 2.82. source: § Experiment Results — Emotional Performance, Table 2 evidence: 'Training uses a three-stage curriculum: (1) full dataset without emotion conditioning to establish base capability; (2) 135-hour emotional subset with emotion conditioner trainable and GRL active; (3) full dataset fine-tune with all conditioners frozen for robustness.' confidence: high relevance: low - claim_id: knowledge_distillation_from_a_large_reasoning_llm_into role: supports claim: Knowledge distillation from a large reasoning LLM into a small fine-tuned model provides a usable soft emotion routing interface for TTS that outperforms instruction-tuned baselines on naturalness and prosody. source: § Text-to-Emotion, Table 3/5 evidence: Natural language emotion control (T2E, Table 3/5) outperforms CosyVoice2 on all four MOS dimensions (SMOS 3.875 vs. 2.973). confidence: high relevance: medium limitations: - The emotional training data (135 hours across 361 speakers) is modest and partly commercial, raising questions about reproducibility and demographic coverage. The GRL-based disentanglement works for the 7 basic emotions defined, but the emotion space is discrete and limited — continuous or compositional emotion representations are not addressed. The paper notes WER degrades slightly on AISHELL-1 relative to IndexTTS, suggesting the emotional training may slightly hurt clean-speech intelligibility. Duration control is evaluated only at up to 1.25× scaling; extreme compression or expansion is untested. The code release is stated as planned but not confirmed at submission. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.04072' published_date: "2025-09-04" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: narrative_aware_segmentation_yields_higher_emotional_diversity role: supports claim: Narrative-aware segmentation of audiobook data into character quotation and narration subsets yields training material with measurably higher emotional diversity than standard sentence-level audiobook splits. source: §3.2, Table 5 evidence: The filtered high-expressivity subset (Q_f, 379 hours) retains quotations where Phi-4 extracts a speech adverb or explicit expressive speech verb from narrative context, yielding pseudo-labels such as "he whispered softly." confidence: medium relevance: low - claim_id: flow_matching_gains_more_from_expressive_finetuning_than_ar role: supports claim: Flow-matching TTS models show larger expressivity gains from fine-tuning on targeted expressive speech data than autoregressive models with equivalent training setups. source: §4.3, Table 2 evidence: Fine-tuning F5-TTS (flow-matching) on Q_f raises ContextMOS from 2.95 to 3.33, while fine-tuning SparkTTS (autoregressive) leaves ContextMOS roughly unchanged (2.89-2.97 vs. 2.94 baseline) despite substantial WER improvement. confidence: medium relevance: low - claim_id: narrative_context_conditioning_improves_contextual_appropriateness role: supports claim: Conditioning TTS synthesis on surrounding narrative context rather than only the target utterance text improves contextual appropriateness of synthesized speech at the cost of modest intelligibility degradation. source: §4.3, Table 2 evidence: Training SparkTTS from scratch with contextual narrative conditioning raises ContextMOS from 3.09 to 3.15 but degrades WER from 4.8 to 9.5. confidence: medium relevance: low - claim_id: opensource_tts_less_expressive_than_narrators_despite_comparable_mos role: complicates claim: Current open-source TTS systems are substantially less expressive than human audiobook narrators on contextual benchmarks, even when naturalness scores (MOS) are comparable. source: §5.2, Table 4 evidence: In zero-shot benchmarking, IndexTTS2-Context reaches the best ContextMOS (3.45) and Win-Rate (54%), matching human ground truth, while the other benchmarked systems (SparkTTS, F5-TTS, MaskGCT) show comparable naturalness MOS but negative CMOS (~-0.97 to -0.98) relative to ground truth, unlike IndexTTS2 (+0.25). confidence: medium relevance: high - claim_id: llm_extracted_delivery_pseudolabels_reliable_for_expressive_tts_training role: supports claim: LLM-extracted speech-delivery pseudo-labels (verbs and adverbs) from narrative prose are reliable enough to serve as training signals for expressive TTS, achieving high precision when confidence-filtered. source: §3.3, Figure 2 evidence: The confidence-filtered pseudo-label extraction pipeline (Phi-4 with self-reported confidence scoring) is used to build the 379-hour Q_f subset that drives the largest expressivity gains among all training variants. confidence: medium relevance: low limitations: - ContextMOS and Win-Rate are computed with Gemini-2.5 Pro as judge on only a 201-quotation subset, not the full test set; MOS and CMOS subjective scores use only 30 samples (2 per speaker), limiting statistical precision. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.04685' published_date: "2025-09-04" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - codec architecture: - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - gan_based_voice_synthesis claims: - claim_id: content_adaptive_token_allocation_in_acoustic_tokenisers_achieves role: supports claim: Content-adaptive token allocation in acoustic tokenisers achieves better reconstruction quality than fixed-rate designs at equal or lower token budgets. source: §4.2, Table 1 evidence: VARSTok at 30.95 Hz achieves UTMOS 3.8949 on LibriTTS test-clean, surpassing the 40 Hz WavTokenizer (3.6107) while using 23% fewer tokens; the 36.81 Hz configuration (UTMOS 4.000) nearly matches the 75 Hz WavTokenizer (4.025) with fewer than half the tokens. confidence: high relevance: low - claim_id: dynamically_segmented_speech_tokens_carry_more_semantically_discriminative role: supports claim: Dynamically segmented speech tokens carry more semantically discriminative information than uniformly sampled tokens at the same average rate. source: §4.3, Table 2 evidence: All VARSTok configurations outperform the 40 Hz WavTokenizer on all four ARCH benchmark classification tasks (emotion, digit recognition, intent), despite operating at lower average frame rates. confidence: high relevance: low - claim_id: encoding_token_duration_implicitly_in_the_vq_codebook role: supports claim: Encoding token duration implicitly in the VQ codebook index eliminates the need for auxiliary duration predictors and preserves compatibility with autoregressive speech language models. source: §3.4, §4.4, Table 3 evidence: The implicit duration coding scheme maps each cluster's content index k and duration d to a single token ID D = (d-1)*K + k, enabling a standard cross-entropy autoregressive model to generate variable-rate token sequences without modification; MOS and WER improve over the fixed-rate baseline in zero-shot TTS. confidence: high relevance: high - claim_id: more_aggressive_temporal_compression_in_variable_rate_tokenisers role: complicates claim: More aggressive temporal compression in variable-rate tokenisers trades reconstruction quality for token efficiency beyond a practical compression threshold. source: §4.2, Table 1 evidence: Increasing S_max from 2 to 8 reduces the average frame rate from 46.5 Hz to 22.38 Hz but degrades UTMOS from 4.038 to 3.647 and PESQ from 2.069 to 1.453 on LibriTTS test-clean; the optimal configuration (tau=0.7, S_max=4 at 30.95 Hz) sits at the knee of this trade-off curve. confidence: high relevance: low - claim_id: inference_speed_in_autoregressive_speech_lms_depends_primarily role: refines claim: Inference speed in autoregressive speech LMs depends primarily on sequence length rather than vocabulary size, so variable-rate tokenisers with expanded vocabularies still accelerate decoding. source: §J, Table 5 evidence: VARSTok (tau=0.6) achieves RTF 0.487 versus 0.766 for the 40 Hz WavTokenizer baseline (36% speedup) despite expanding the token vocabulary from K to K*S_max = 16,384 entries, because shorter sequences reduce the dominant cost of attention computation over more function evaluations. confidence: high relevance: low limitations: - Evaluation is restricted to English (LibriTTS). The clustering algorithm relies on cosine similarity in a WavTokenizer embedding space trained on English read speech; whether the density-peak boundaries remain meaningful for other languages, spontaneous speech, or emotionally expressive styles is untested. - Speaker similarity under more aggressive compression (tau=0.6, 26.29 Hz) does show a statistically modest decline in objective SIM (0.880 vs 0.918 for the baseline), and while subjective SMOS remains comparable, the long-tail impact on voices far from the training distribution is unknown. Codebook collapse becomes severe for K above 4096 in the expanded index space, suggesting that very large vocabulary configurations require dedicated regularisation strategies not addressed here. The clustering algorithm is not differentiable, so joint end-to-end training with a downstream TTS model is not straightforward. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.05863' published_date: "2025-09-06" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: preference_based_alignment_dpo_improves_intelligibility_in_multilingual role: supports claim: Preference-based alignment (DPO) improves intelligibility in multilingual autoregressive TTS systems beyond what supervised fine-tuning achieves. source: §4.3, §5.2, Table 2, Table 3 evidence: LatinX (DPO) reduces WER across nearly all 30 cross-lingual language pairs compared to the supervised fine-tuned baseline, and outperforms XTTSv2 in most pairs; Romanian-source conditions show particularly large gains (e.g., ro-to-es at 0.45% WER). confidence: high relevance: medium - claim_id: automated_speaker_similarity_metrics_based_on_speaker_encoder role: complicates claim: Automated speaker similarity metrics based on speaker encoder embeddings do not reliably reflect human perceptual judgments of voice identity in zero-shot TTS. source: §5.2, §6, Table 4, Table 5 evidence: XTTSv2 achieves higher Sim-O scores than both LatinX models, yet human evaluators strongly prefer LatinX speaker similarity (SMOS 3.63/3.54 vs. 3.24); the paper explicitly flags this as a divergence between objective and subjective evaluation. confidence: high relevance: high - claim_id: dpo_alignment_in_tts_involves_a_trade_off role: complicates claim: 'DPO alignment in TTS involves a trade-off: optimizing for intelligibility and objective similarity can reduce naturalness MOS and, in some language conditions, perceptual similarity relative to the fine-tuned baseline.' source: §5.2, §6, Table 5, Table 6 evidence: LatinX (DPO) improves WER and Sim-E over the fine-tuned model but achieves lower average MOS (3.35 vs. 3.41) and lower SMOS in several cross-lingual conditions; the paper attributes this partly to the codec introducing artifacts that cap perceptual quality. confidence: high relevance: medium - claim_id: lossy_neural_audio_codecs_set_a_perceptual_quality role: complicates claim: Lossy neural audio codecs set a perceptual quality ceiling in codec-based TTS that preference alignment cannot overcome, because the model learns to replicate codec artifacts introduced during reference encoding. source: §6 evidence: The paper notes that the VQ-VAE codec is lossy and the model learns to reproduce its artifacts, limiting the maximum perceptual quality achievable regardless of post-training alignment method. confidence: high relevance: low limitations: - The evaluation uses an internal test set of unseen speakers with no publicly named benchmark, and the human rating pool is predominantly English and Portuguese native speakers. Conclusions about multilingual naturalness and similarity, especially for French, Italian, and Romanian, should be treated with caution. - The DPO preference signal is constructed solely from WER and speaker similarity; no prosody, naturalness, or rhythm metric is incorporated, which likely explains the MOS regression relative to the fine-tuned baseline. The preference labeling is fully automated with no human verification of winner/loser assignments. The real-time factor of 4.85 makes the system unsuitable for real-time applications, and the authors note that non-autoregressive architectures are a necessary direction. The Romanian evaluation suffers from very small rater counts and predominantly non-native listeners, undermining the interpretation of the unusually high SMOS scores that exceed real audio. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.07376' published_date: "2025-09-09" entry_date: '2026-07-26' year: 2025 venue: EMNLP task: - TTS architecture: - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: hierarchical_facial_features_improve_voice_face_congruence role: supports claim: Hierarchical, locally-aware facial feature extraction substantially improves voice-face identity congruence compared to global visual representations in face-to-speech synthesis. source: §5.3, Table 5 evidence: Progressive Facial Granularity Aggregation (FGA), hierarchical patch decomposition with progressive convolutional aggregation, yields SECS of 79.96 (seen) vs. 66.25 when replaced with a single vanilla transformer over the full image, a gap of 13.71 points. confidence: high relevance: medium - claim_id: crossmodal_attribute_supervision_improves_face_voice_alignment role: supports claim: Cross-modal multi-task supervision over demographic attributes improves face-voice alignment in end-to-end synthesis systems. source: §5.2, Table 4 evidence: Bilateral attribute enhancement (predicting gender and ethnicity from both the facial embedding and synthesized audio) produces the highest SECS configuration at 79.96 seen / 71.39 unseen; applying supervision to only one modality yields lower scores (77.76 / 70.27 for visual-only, 75.84 / 69.52 for audio-only). confidence: high relevance: medium - claim_id: multiview_augmentation_critical_for_ftv_generalization role: supports claim: Multi-view face augmentation, pairing multiple facial frames per speaker across varying poses and lighting, is critical for robust speaker identity generalization in face-to-speech systems. source: §5.3, Table 5 evidence: Removing the multi-view augmentation strategy causes the largest single-component SECS drop in ablation studies (from 79.96 to 64.43, a decrease of 15.53 on seen speakers and 9.35 on unseen), exceeding the contribution of any individual architectural module. confidence: high relevance: high - claim_id: e2e_ftv_still_needs_crossmodal_semantic_supervision role: complicates claim: End-to-end face-to-voice synthesis without audio-space alignment targets still requires explicit cross-modal semantic supervision to maintain speaker identity coherence. source: §5.2, §5.3, Table 4, Table 5 evidence: Removing bilateral attribute supervision, even partially, consistently degrades SECS; visual attribute enhancement removal alone costs 5.23 SECS points. confidence: medium relevance: high - claim_id: ftv_systems_uneven_across_demographic_groups role: complicates claim: Face-to-voice synthesis systems trained on demographically skewed corpora may produce uneven speaker similarity across underrepresented groups. source: §Appendix B, Table 6, Table 7 evidence: The LRS3-TED training set is 74.66% male and 72.23% Caucasian; the proposed model achieves SECS of 75.97 for male and 75.03 for female speakers, while Middle Eastern (69.25) and African American (71.65) speakers show measurably lower scores despite the bilateral attribute supervision. confidence: high relevance: high limitations: - The evaluation uses 40 speakers and 21 annotators from LRS3-TED, which is heavily skewed toward Caucasian male speakers (74.66% male, 72.23% Caucasian), limiting statistical power for cross-demographic comparisons; comparison baselines do not include the most recent large-scale zero-shot TTS systems. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.08379' published_date: "2025-09-10" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - VC architecture: - diffusion - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: latentvoicegrad_latent_diffusion_fm_improves_quality_speed role: supports claim: Operating diffusion and flow-matching models in a compressed latent representation improves audio quality and inference speed in non-parallel voice conversion compared to operating directly on mel-spectrograms. source: §III.A, §IV.H, Table IV, Table X evidence: LatentVoiceGrad-DPM outperforms VoiceGrad-DPM in pMOS (3.93 vs. 3.86), subjective qMOS (4.09 vs. 3.83), sMOS (3.05 vs. 2.63), and GPU RTF (0.034 vs. 0.045) on VCTK zero-shot any-to-any conversion. confidence: high relevance: high - claim_id: latentvoicegrad_fm_matches_dpm_fewer_steps role: supports claim: Flow matching achieves comparable voice conversion quality to diffusion models with substantially fewer inference steps. source: §IV.G, §IV.J, Tables VII, IX evidence: VoiceGrad-FM with L=3 Euler steps achieves pMOS 3.85 (versus 3.86 for VoiceGrad-DPM at L=20), at GPU RTF 0.007 versus 0.045, a six-fold speedup with no quality penalty. confidence: high relevance: medium - claim_id: latentvoicegrad_adversarial_autoencoder_improves_quality role: supports claim: Adversarial autoencoder training substantially improves audio quality and speaker similarity in latent-domain voice conversion compared to reconstruction-only training. source: §IV.F, Table V evidence: Adding adversarial loss during autoencoder training improves pMOS from 3.78 to 3.93 and SECS from 0.829 to 0.844 on VCTK zero-shot VC with the DPM generative model. confidence: high relevance: high - claim_id: latentvoicegrad_discrete_tokens_underperform_continuous role: complicates claim: Discrete acoustic token sequences from neural audio codecs underperform continuous spectral representations as the conversion domain for iterative generative voice conversion models. source: §IV.E, Tables II, III evidence: VoiceGrad with DAT features (DAC codec, dimensionality 1024) achieves pMOS 2.41 versus 3.86 for mel-spectrograms; LatentVoiceGrad-DAT shows CER 9.08 versus 2.99 for the mel-spectrogram variant. confidence: medium relevance: medium - claim_id: latentvoicegrad_noise_ratio_controls_tradeoff role: refines claim: The noise injection ratio at the initial point of the ODE in flow-matching voice conversion controls a three-way trade-off between audio quality, intelligibility, and speaker similarity. source: §IV.G, Table VI evidence: Systematic sweeps of r from 0 to 1 in VoiceGrad-FM and LatentVoiceGrad-FM show increasing r raises SECS but degrades CER, with pMOS peaking around r=0.4-0.6; r=0.7 provides a practical balance. confidence: high relevance: high limitations: - Evaluation is limited to English speech (VCTK, LibriTTS) and a 12-listener subjective test that omits FACodec as a comparison. - The discrete-token (DAT) evaluation is acknowledged as potentially sub-optimal due to untuned hyperparameters, leaving the conclusion about discrete-token unsuitability tentative. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.08696' published_date: "2025-09-10" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: zero_training_layer_caching_accelerates_diffusion_tts role: supports claim: Zero-training transformer layer caching can accelerate diffusion-based TTS inference without retraining or architectural changes. source: §3.2, Table 1 evidence: Applying SmoothCache to F5-TTS with alpha=0.25 (caching approximately half of 32 NFE steps) reduces RTF from 0.46 to 0.26 while keeping WER, SIM-o, and UTMOS within measurement noise on LibriSpeech-PC test-clean, requiring no modification to model weights. confidence: high relevance: low - claim_id: caching_quality_benefit_depends_on_denoising_trajectory_length role: complicates claim: The inference-time quality benefit of layer caching in diffusion TTS depends on the length of the denoising trajectory. source: §4.1, Table 1 evidence: At 32 NFE, caching 50% of steps (alpha=0.25) preserves quality; at 16 NFE, the same proportional caching (alpha=0.50) degrades UTMOS from 3.90 to 3.79 and NISQA from 4.14 to 3.97 on LibriSpeech-PC test-clean, comparable to reducing NFE directly. confidence: high relevance: low - claim_id: independent_perlayer_cache_schedules_cause_artefacts role: complicates claim: Per-layer cache schedules derived independently for attention and feed-forward sublayers introduce inter-layer dependency artefacts in diffusion transformer TTS. source: §3.1 evidence: Applying separate Attn-only and FFN-only cache schedules to F5-TTS caused audible artefacts in informal listening tests; adopting a unified schedule derived from the attention layer pattern resolved this issue while maintaining cache utilisation. confidence: medium relevance: low - claim_id: calibrated_caching_marginally_superior_to_step_reduction role: supports claim: Calibrated layer caching provides a marginally superior quality-compute trade-off compared to equivalent step reduction at high NFE settings. source: §4.1, Table 2 evidence: A user study with 37 participants found 53% preference for 32 NFE with caching (24 effective compute steps) over a 24 NFE no-cache baseline using the same compute budget; the difference collapsed to 49% vs. 51% at the lower 16 NFE setting. confidence: medium relevance: low limitations: - Subjective evaluation relies on A/B preference tests rather than CMOS or SMOS ratings, limiting the quantitative characterisation of quality degradation; RTF measurements use a single GPU and a single TTS model. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2506.04077' published_date: "2025-09-11" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: low evidence_role: - historical_context current_role: minor method_family: - autoregressive_speech_token_lm claims: - claim_id: speaker_conditioned_tts_preserves_learner_disfluencies role: supports claim: Speaker-conditioned TTS synthesis, when conditioned on reference embeddings from real non-native learners, can produce augmentation data that preserves proficiency-level-appropriate disfluencies and prosodic variability. source: §2.2, Figure 1 evidence: XTTSv2 conditioned on 512-dimensional speaker embeddings from real learner recordings produces speech retaining speaking rate, rhythm, and characteristic disfluencies; this contrasts with standard single-speaker TTS, which generates uniformly fluent output incompatible with non-native learner characteristics. confidence: medium relevance: medium - claim_id: synthetic_speech_alone_insufficient_for_asa role: complicates claim: Synthetic speech from standard TTS pipelines cannot substitute for authentic learner recordings in isolation, even when speaker conditioning is applied to mimic non-native characteristics. source: §3.4, Table 2 evidence: Training exclusively on synthesised data achieves only 42.22% seen-test accuracy versus 73.33% for a real-data-only baseline, demonstrating that the acoustic domain gap between synthetic and authentic speech is not fully bridged by speaker-conditioned voice cloning alone. confidence: high relevance: high - claim_id: mixed_real_synthetic_reweighting_outperforms_real_only role: supports claim: Mixed real-plus-synthetic training with adaptive importance reweighting outperforms training on real data alone for low-resource speech understanding tasks. source: §3.3, Table 1 evidence: Adding synthesised augmentation with dynamic real-sample upweighting raises seen-test overall accuracy from 73.33% to 76.67% and pass-fail accuracy from 77.78% to 81.11% on the LTTC GEPT dataset, outperforming both single-modality and multimodal baselines trained on real data only. confidence: high relevance: low - claim_id: augmentation_gains_diminish_on_ood_prompts role: complicates claim: Gains from generative data augmentation in speech assessment diminish substantially on out-of-distribution prompt types, limiting the generalisation benefit of synthetic speech augmentation. source: §3.3, Table 1 evidence: Overall accuracy on unseen-prompt test data improves by only 0.67 percentage points (63.33% to 64.00%) compared to 3.34 points on seen prompts, indicating that augmentation-driven gains do not transfer robustly across different prompt domains. confidence: high relevance: medium limitations: - The paper does not report any measure of the TTS output's perceptual quality (e.g., MOS or speaker similarity scores), making it difficult to assess whether the voice cloning pipeline actually produces convincing non-native-sounding speech or whether the downstream scoring model is simply robust to low-fidelity synthesis; evaluation is restricted to a single proprietary English dataset. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.09631' published_date: "2025-09-11" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: discrete_fm_over_factorized_tokens_competitive_naturalness_prosody role: supports claim: Discrete flow matching defined directly over factorized speech token subspaces achieves competitive naturalness and superior prosody reconstruction compared to continuous-space flow and diffusion TTS baselines trained on comparable data. source: §4.2, Table 1, Table 2 evidence: DiFlow-TTS achieves UTMOS 3.98 vs. 3.83 (MaskGCT) and 3.76 (F5-TTS), and F0-RMSE 7.97 vs. 14.33 (MaskGCT) and 13.78 (F5-TTS), the best margin among all systems compared. confidence: high relevance: low - claim_id: factorized_multihead_velocity_prediction_improves_fidelity role: supports claim: Factorized multi-head velocity prediction, separate prediction heads for distinct speech attribute subspaces within a single discrete flow model, improves prosody and acoustic fidelity over a single-head alternative. source: §4.3, Table 4 evidence: Dedicated prediction heads process the split prosody (h^p) and acoustic (h^a) representations separately after the shared 12-layer DiT backbone, rather than a single unified head predicting all factorized attributes jointly. confidence: medium relevance: low - claim_id: adaln_speaker_conditioning_insufficient_simo_diverges_from_perceptual role: complicates claim: Global speaker embedding conditioning via AdaLN is insufficient for reliable zero-shot speaker similarity in discrete token-space TTS; perceptual similarity judgements diverge from embedding-based automatic metrics (SIM-O) in this setting. source: §4.2, Table 1, Table 2 evidence: SIM-O (automatic speaker embedding cosine similarity) ranks DiFlow-TTS third among five systems at 0.45, behind MaskGCT (0.67) and F5-TTS (0.52), yet subjective Similarity MOS ranks DiFlow-TTS best at 4.42, with the authors attributing the discrepancy to perceptual identity cues not captured by embedding cosine similarity. confidence: high relevance: high - claim_id: nar_discrete_flow_compact_dit_matches_single_step_latency role: supports claim: Non-autoregressive discrete flow models with compact DiT backbones can match the inference latency of single-step flow matching systems while operating at higher NFE, without compromising data efficiency. source: §4.2, Table 3 evidence: DiFlow-TTS (164M, 4 NFE) achieves RTF 0.03, matching OZSpeech's 1-NFE RTF, while DiFlow-TTS-Small (122M, 16 NFE) reaches RTF 0.05, 5.2x-34x faster than continuous-space baselines (VoiceCraft 1.70, NaturalSpeech 2 1.66). confidence: high relevance: medium limitations: - The evaluation uses a single language (English) and a moderately sized training set (470h); comparisons are made under heterogeneous training data conditions (DiFlow-TTS 470h vs. MaskGCT 100K h vs. VoiceCraft 9K h), which complicates interpreting quality differences as purely architectural. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.09748' published_date: "2025-09-11" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: training_free_caching_reduces_flops_without_intelligibility_loss role: supports claim: Training-free caching of redundant DiT computations can substantially reduce FLOPs in TTS without retraining and without significant intelligibility degradation. source: §4.2, Table 1 evidence: At the optimal T4 threshold, DiTReducio reduces F5-TTS FLOPs by 54.4% while keeping WER within 0.002 percentage points of the uncompressed baseline on LibriSpeech-PC test-clean. confidence: high relevance: low - claim_id: diagonal_attention_patterns_predict_temporal_redundancy role: supports claim: Diagonal-like self-attention patterns in DiT layers are a reliable proxy for identifying temporally redundant layer-timestep pairs during diffusion inference. source: §3.3, Figure 4 evidence: Cosine similarity between an attention heatmap and the identity matrix is strongly correlated with temporal redundancy; layer-step pairs scoring above 0.35 in diagonal similarity approach 100% redundancy. confidence: high relevance: low - claim_id: temporal_skipping_alone_causes_conditional_information_loss role: complicates claim: Temporal skipping alone in DiT TTS inference causes severe conditional information loss, making branch skipping a necessary complement for quality-preserving acceleration. source: §4.3, Figure 6 evidence: Ablation on F5-TTS shows that TS-only achieves greater speedup but reaches WER of 23.06% at maximum threshold; full DiTReducio (TS + BS) holds WER at 2.9% under the same threshold. confidence: high relevance: low - claim_id: training_free_dit_acceleration_has_threshold_dependent_quality_cliff role: complicates claim: 'Training-free DiT acceleration exhibits a threshold-dependent quality cliff: beyond a moderate compression ratio, marginal speed gains decrease while speaker similarity degradation accelerates.' source: §4.2, Table 1 evidence: For F5-TTS, moving from T4 to T6 yields only an additional 0.017 RTF improvement (0.129 to 0.112) but drops SIM-o from 0.618 to 0.590; the paper identifies T4 as the practical optimum. confidence: high relevance: high - claim_id: image_video_acceleration_transfers_but_needs_cfg_adaptation role: refines claim: Inference-time acceleration techniques from image and video diffusion transfer to speech synthesis but require domain-specific adaptation, particularly for classifier-free guidance structures. source: §3.2, §Appendix A.1 evidence: Branch skipping is introduced to handle speech CFG's conditional/unconditional branch redundancy, and the multi-condition CFG in MegaTTS 3 requires separate residuals for each branch pair. confidence: medium relevance: low limitations: - 'The paper reports no subjective listening test: quality claims rest entirely on SIM-o and WER, which may not capture perceptual artefacts introduced by aggressive caching; the evaluation scope is narrow (two models, one dataset, no comparison against other TTS-specific distillation methods).' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.11084' published_date: "2025-09-14" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: length_normalized_positional_encoding_improves_longform_alignment role: supports claim: Length-normalized positional encoding in cross-attention TTS substantially improves alignment accuracy for long-form speech synthesis, where standard absolute-index RoPE degrades. source: §4.2, Table 1 evidence: LARoPE with K_e=4 reduces WER on tc-long (10-30s) from 4.98% to 2.16%, while RoPE's WER on tc-long is nearly double that of tc-short (2.41%), indicating progressive degradation with utterance length. confidence: high relevance: low - claim_id: diagonal_inductive_bias_accelerates_alignment_learning role: supports claim: A diagonal inductive bias in cross-attention positional embeddings accelerates text-speech alignment learning during training. source: §4.3, Figure 2 evidence: At 200k training iterations without batch expansion, LARoPE reduces CER from 2.00% to 1.23% versus RoPE, with the performance gap maintained throughout 700k iterations. confidence: high relevance: low - claim_id: standard_rope_suboptimal_in_cross_attention_tts role: complicates claim: Standard RoPE is suboptimal in cross-attention TTS because relative positional distances based on absolute indices do not align diagonally when query and key sequence lengths differ, violating the monotonic text-speech alignment structure. source: §3, Figure 1 evidence: Analysis of relative upper bounds shows that RoPE's attention score map loses diagonal structure whenever L_q != L_k (e.g., L_q=64, L_k=256), whereas LARoPE preserves the diagonal regardless of length ratio. confidence: high relevance: low - claim_id: compact_tts_matches_larger_models_via_positional_embedding_design role: supports claim: Compact attention-based TTS systems can achieve pronunciation accuracy competitive with models more than an order of magnitude larger through improved positional embedding design rather than parameter scaling. source: §4.5, Table 3 evidence: The 19M-parameter LARoPE model achieves 2.25% WER on LibriSpeech test-clean, outperforming E2 TTS (333M params, 2.95% WER), F5-TTS (336M, 2.42%), and DiTTo-TTS (740M, 2.56%) at RTF 0.05. confidence: high relevance: low - claim_id: existing_rope_extensions_leave_cross_attention_alignment_unresolved role: refines claim: Existing RoPE extensions (position interpolation, YaRN, LongRoPE) address self-attention length extrapolation but leave the cross-attention alignment problem in TTS unresolved. source: §5.1 evidence: The paper identifies that all prior RoPE extensions are designed for self-attention and do not modify the relative distance formulation for cross-attention with differing query and key lengths; LARoPE specifically targets this gap. confidence: medium relevance: low limitations: - The evaluation is conducted entirely within the SupertonicTTS framework; generalisability to other cross-attention TTS architectures is asserted but not tested, and the paper does not benchmark LARoPE against alternative alignment strategies such as guided attention loss or diagonal priors in monotonic alignment search. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.11425' published_date: "2025-09-14" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - codec - TTS architecture: - GAN - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: injecting_multimodal_guidance_directly_into_the_encoder_latent role: supports claim: Injecting multimodal guidance directly into the encoder latent space of a neural codec improves reconstruction quality beyond similarity-based supervision of the quantized layer. source: §2.3.1, §3.2.1, Table 2 evidence: FuseCodec-Fusion, which fuses global semantic and contextual vectors into Z via additive fusion, achieves WER 3.99, ViSQOL 3.47, and PESQ 3.13 on LibriSpeech test-clean, outperforming FuseCodec-Distill (ViSQOL 3.43, PESQ 3.06) and FuseCodec-ContextAlign, both of which only supervise Q(1) without modifying the latent. confidence: high relevance: low - claim_id: neural_codecs_trained_with_joint_semantic_and_contextual role: supports claim: Neural codecs trained with joint semantic and contextual supervision generalize to unseen languages without multilingual training data. source: §3.3, Table 5 evidence: FuseCodec, trained exclusively on English LibriSpeech train-clean-100, achieves the best WER and ViSQOL in the majority of 7 tested languages from Multilingual LibriSpeech and outperforms all baselines on PESQ by at least 0.3 in most languages. confidence: high relevance: low - claim_id: codec_representations_enriched_with_semantic_and_contextual_signals role: supports claim: Codec representations enriched with semantic and contextual signals support stronger downstream task generalization (emotion recognition, audio event classification) than acoustic-only codecs. source: §3.2.2, Table 3 evidence: On CodecSUPERB at 4 kbps, FuseCodec-Fusion achieves emotion recognition accuracy of 73.96% and audio signal quality 0.785, versus 66.18% and 0.697 for EnCodec at 6 kbps. All FuseCodec variants exceed SpeechTokenizer, EnCodec, and DAC on audio signal quality. confidence: high relevance: low - claim_id: fine_grained_temporal_alignment_between_text_tokens_and role: complicates claim: Fine-grained temporal alignment between text tokens and acoustic frames improves local interpretability but is constrained relative to global supervision strategies. source: §2.3.3, §3.2.1, Table 2 evidence: FuseCodec-ContextAlign, which aligns contextual embeddings to RVQ tokens via a windowed similarity matching algorithm, achieves WER 4.15 and ViSQOL 3.18, lagging FuseCodec-Fusion (WER 3.99, ViSQOL 3.47) and FuseCodec-Distill (ViSQOL 3.43). The paper attributes this to constrained local alignment limiting global contextual guidance. confidence: high relevance: low - claim_id: distilling_both_semantic_self_supervised_speech_model_and role: supports claim: Distilling both semantic (self-supervised speech model) and contextual (language model) signals into codec token supervision outperforms semantic-only distillation for perceptual naturalness. source: §3.2.1, Table 2 evidence: FuseCodec-Distill achieves UTMOS 3.65 and Similarity 0.996 on LibriSpeech test-clean, while codecs using only semantic distillation (SpeechTokenizer, Mimi, X-Codec2) score below 3.55 UTMOS and fail to consistently match speaker similarity. confidence: high relevance: high limitations: - The codec is trained on LibriSpeech train-clean-100 (100 hours, English read speech), which limits conclusions about robustness to spontaneous speech, diverse accents, or noisy conditions. Multilingual generalization results are promising but the training data and BERT model are English-only, leaving the mechanism behind cross-lingual transfer unclear. The ASR transcription step (wav2vec 2.0) introduces an error-prone intermediate representation during training; the effect of ASR errors on contextual embedding quality is not quantified. Model size is not reported, making it difficult to assess computational cost relative to baselines. The TTS evaluation compares only against other codec-based systems trained on LibriTTS and does not benchmark against the strongest current TTS models. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.12171' published_date: "2025-09-15" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - SCA - evaluation architecture: - flow-matching relevance: low evidence_role: - historical_context current_role: minor method_family: - nonautoregressive_flow_matching claims: - claim_id: raw_accuracy_masks_demographic_fairness_failures role: supports claim: Raw accuracy metrics are insufficient to surface demographic fairness failures in speech-aware LLMs. source: §7, Table 1 evidence: Models achieve overall fairness scores of 0.43-0.83, but when constrained to tasks they can solve for at least one speaker, constrained fairness drops below 5% across all models, a discrepancy invisible to accuracy-only evaluation. confidence: high relevance: medium - claim_id: population_level_robustness_masks_speaker_group_inconsistency role: complicates claim: Speech-aware LLMs that perform robustly at the population level may still exhibit systematic speaker-group inconsistencies. source: §7, Table 1 evidence: Audio Flamingo 3 achieves an overall fairness of 0.83 but a constrained fairness of 0.002, and its cross-modal robustness trails overall fairness by 13 percentage points, indicating modality inconsistency even when all speaker groups receive the same answer. confidence: high relevance: medium - claim_id: voice_cloning_scales_diverse_speaker_benchmark_construction role: supports claim: Voice cloning provides a scalable alternative to human recording for constructing diverse-speaker evaluation benchmarks. source: §3, §6 evidence: C3T uses a Voicebox-based conditional flow matching TTS to synthesize 14-29 speaker variants per task instance, covering age, gender, and accent groups from ESLTTS and GLOBE, replacing a recording effort that would otherwise be infeasible at scale. confidence: high relevance: high - claim_id: crossmodal_degradation_reflects_speaker_dependent_response_variation role: refines claim: Cross-modal degradation in speech-aware LLMs reflects not only ASR errors but also speaker-characteristic-dependent response variation that survives correct transcription. source: §1, §5 evidence: The benchmark explicitly tests scenarios where transcription is correct but model output differs across speakers, isolating demographic sensitivity from recognition error as distinct failure modes. confidence: medium relevance: medium limitations: - The benchmark is English-only and the speaker pool for age and gender evaluation is limited to three accent regions, and the custom fairness and robustness metrics are not directly comparable to existing benchmark scores, limiting cross-study synthesis. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.12831' published_date: "2025-09-16" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - hybrid - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - gan_based_voice_synthesis claims: - claim_id: pre_trained_zero_shot_tts_and_gan_based_visual_synthesis_models_can_be_c role: supports claim: Pre-trained zero-shot TTS and GAN-based visual synthesis models can be chained in a modular pipeline without fine-tuning to produce lip-synced talking-head video from short noisy reference audio. source: §3.4, §5.2, §5.7 evidence: Tortoise TTS and Wav2Lip were connected in inference-only mode using a single 10-15 second audio clip and one frontal-face video, producing output that informal viewers assessed as perceptually coherent. confidence: high relevance: high - claim_id: diffusion_based_autoregressive_tts_systems_impose_inference_latency_that role: complicates claim: Diffusion-based autoregressive TTS systems impose inference latency that conflicts with real-time voice cloning requirements even on dedicated GPU hardware. source: §7.4 evidence: Tortoise TTS synthesis takes "several minutes for each sentence" on an NVIDIA Tesla T4, which the authors acknowledge prevents interactive or real-time deployment without model compression or server-side acceleration. confidence: medium relevance: high - claim_id: single_speaker_uncontrolled_evaluations_with_informal_raters_provide_ins role: complicates claim: Single-speaker, uncontrolled evaluations with informal raters provide insufficient evidence to support generalizability claims for zero-shot VC and talking-head pipelines. source: §6.2, §6.6, §7.1 evidence: The entire evaluation rests on one speaker (public recordings of Angelina Jolie), assessment by three non-expert viewers with no task protocol, and no computed metrics; the authors explicitly state that generalizability to other speakers, languages, or acoustics cannot be claimed. confidence: medium relevance: high limitations: - See the source paper page for the complete limitations and evaluation scope. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.13068' published_date: "2025-09-16" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - codec - VC architecture: - autoregressive-LM - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: cascaded_residual_codec_architectures_can_enforce_attribute_disentanglement role: supports claim: Cascaded residual codec architectures can enforce attribute disentanglement through structure rather than through adversarial training objectives. source: §2.1, §3.3.3, Table 3 evidence: MSR-Codec achieves clean separation of timbre, prosody, and semantic content by having each stream operate on residuals from the previous stage, without adversarial disentanglement loss; VC experiments confirm independent manipulation of each attribute. confidence: high relevance: medium - claim_id: explicit_prosodic_supervision_in_a_dedicated_codec_stream role: supports claim: Explicit prosodic supervision in a dedicated codec stream promotes measurable disentanglement of pitch from speaker identity. source: §2.1.2, §3.3.3, Table 3 evidence: VQ1 (prosody stream) is trained with MSE loss against ground-truth F0 and spectral energy; prosody-only VC achieves low ΔF0,tar (12.3-14.2 Hz) while maintaining high SIM-src (0.59-0.64), confirming that prosody and timbre are independently manipulable. confidence: high relevance: high - claim_id: disentangled_codec_designs_can_achieve_competitive_speaker_similarity role: supports claim: Disentangled codec designs can achieve competitive speaker similarity at lower bitrates than undifferentiated RVQ codecs. source: §3.3.1, Table 1 evidence: MSR-Codec-424 achieves SPK-SIM 0.80 at 424 bps, higher than WavTokenizer (0.67 at 900 bps) and X-Codec (0.72 at 1000 bps), attributed to the time-invariant timbre stream which preserves speaker identity without scaling with utterance length. confidence: high relevance: high - claim_id: data_efficient_tts_systems_built_on_factorized_codec role: supports claim: Data-efficient TTS systems built on factorized codec representations can achieve competitive intelligibility relative to larger models trained on more data. source: §3.3.2, Table 2 evidence: The 0.2B MSR-Codec-524 TTS model trained on 45k hours achieves WER 3.07% on Seed-TTS-eval English, outperforming Llasa-1B trained on 250k hours (WER 3.22%) and FireRedTTS-0.4B trained on 150k hours (WER 3.82%). confidence: high relevance: low - claim_id: signal_fidelity_codec_metrics_stoi_pesq_and_speaker role: complicates claim: Signal-fidelity codec metrics (STOI, PESQ) and speaker similarity diverge at low bitrates, making holistic quality assessment difficult. source: §3.3.1, Table 1 evidence: MSR-Codec-424 achieves the highest SPK-SIM (0.80) among codecs at comparable bitrates but lower STOI (0.84) and PESQ-WB (1.82) than some baselines, indicating that speaker preservation and signal-level fidelity are optimized differently by the multi-stream design. confidence: high relevance: high limitations: - No subjective listening test (MOS/MUSHRA) is reported for any condition; all quality comparisons rely on automatic metrics (UTMOS, STOI, PESQ, WER, SPK-SIM). Conclusions about perceived naturalness cannot be confirmed from the available data. - 'The VC evaluation protocol is small in scope: 8 target speakers from VCTK and 100 source utterances from LibriTTS. Generalisation to more diverse speakers, accents, or noisy conditions is not assessed. The FreGAN vocoder operates at 16 kHz, and the Mel-spectrogram-based pipeline may impose a quality ceiling relative to waveform-domain codecs. The TTS model is evaluated only on English; the codec was trained on Mandarin and English data but multilingual TTS capability is not demonstrated. Model size figures for the codec itself are not reported; only the TTS model size (0.2B) is provided.' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2412.16846' published_date: "2025-09-17" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: distributional_training_objectives_for_continuous_ar_speech_modeling role: supports claim: Distributional training objectives for continuous AR speech modeling achieve higher intelligibility than regression-based alternatives. source: §TTS Evaluation, Table 2; §Ablation Study, Table 5 evidence: KALL-E with KL divergence loss achieves WER 1.94 / CER 0.96 on Seed-TTS test sets, below all discrete-token and regression-based baselines; ablation replacing Flow-VAE with Stable Audio VAE (near-zero KL weight, approaching a plain autoencoder) collapses CER from 2.79 to 40.09 at the same latent dimension. confidence: high relevance: low - claim_id: low_frame_rate_continuous_representations_reduce_autoregressive_tts role: supports claim: Low frame-rate continuous representations reduce autoregressive TTS inference compute by over an order of magnitude without sacrificing synthesis quality. source: §TTS Evaluation, Table 3, Table 4 evidence: KALL-E at 12.5 Hz requires 7,947 GFLOPs to synthesize 10 seconds vs. 122,170 for Llasa-1B at 50 Hz, while achieving higher MOS (4.17 vs. 3.92) and lower WER (1.94 vs. 3.6) on the same test set. confidence: high relevance: low - claim_id: objective_speaker_similarity_metrics_for_zero_shot_tts role: complicates claim: Objective speaker similarity metrics for zero-shot TTS are unreliable for cross-system comparisons when decoder architectures differ in their use of reference audio. source: §TTS Evaluation, Table 2, Table 3 evidence: Discrete-token systems (Seed-TTS SIM 0.796, FireRedTTS SIM 0.635) score differently on objective SPK-SIM than KALL-E (SIM 0.646/0.568), but KALL-E receives higher listener naturalness ratings; the authors attribute the gap to those systems conditioning the waveform decoder on the reference utterance at decode time, which inflates the metric independent of perceived speaker fidelity. confidence: high relevance: high - claim_id: test_time_adaptation_from_a_single_reference_utterance role: supports claim: Test-time adaptation from a single reference utterance improves speaker similarity in continuous-representation AR TTS without requiring full model retraining. source: §Test Time Training; §TTS Evaluation, Table 2 evidence: KALL-E (TTT) improves SPK-SIM from 0.568 to 0.611 on test-en using N=200 latent sequences sampled from the reference utterance's Flow-VAE distribution, with WER remaining stable at 1.90. confidence: high relevance: high - claim_id: increasing_vae_kl_regularization_weight_trades_reconstruction_fidelity role: complicates claim: Increasing VAE KL regularization weight trades reconstruction fidelity for a latent space structure that is more suitable for downstream generative modeling. source: §VAE Evaluation, Table 1; §Ablation Study, Table 5 evidence: Flow-VAE uses KL weight 32 and scores PESQ-WB 3.26 at 512-dim/12.5 Hz, below Stable Audio VAE (3.11) at the same frame rate with near-zero KL weight; however, Stable Audio VAE's latent space causes CER to collapse when used as the AR LM encoder, demonstrating that reconstruction quality and generation compatibility impose conflicting constraints on VAE training. confidence: high relevance: low limitations: - 'Objective speaker similarity remains below discrete-token systems that condition their decoders on the reference audio, suggesting the Flow-VAE''s information bottleneck trades some speaker detail for a more LM-friendly latent space. The TTT procedure assumes the transcript of the reference utterance is available, which may not hold in all deployment settings. Overfitting risk in TTT is real: CER rises after N=200 in ablation, limiting the effective adaptation set size. Evaluations are conducted solely on the Seed-TTS test sets; generalization to other benchmarks, out-of-distribution speakers, or noisy acoustic conditions is not assessed. Training data composition differs from the most directly comparable system (Llasa-1B), making it difficult to fully isolate architecture from data quality as the source of WER gains.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2504.20581' published_date: "2025-09-17" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - evaluation architecture: [] relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: [] claims: - claim_id: automated_speaker_similarity_enables_reproducible_voice_cloning_eval role: supports claim: Automated speaker-similarity metrics can serve as architecture-agnostic proxies for voice cloning quality, enabling reproducible evaluation without human raters. source: §3, §4 evidence: ClonEval evaluates five open-source models as black boxes using WavLM cosine similarity, requiring no human listeners or model internals access, while revealing systematic performance differences across systems and conditions. confidence: high relevance: high - claim_id: neutral_read_speech_benchmarks_overestimate_voice_cloning_performance role: complicates claim: Benchmark evaluations using only neutral read-speech overestimate the practical voice cloning performance of current TTS systems. source: §7.1, Table 1 evidence: All five models achieved WavLM cosine similarity in the 0.88-0.91 range on LibriSpeech test-clean, but scores dropped substantially on emotional datasets, with OuteTTS falling to 0.54 on SAVEE. confidence: high relevance: high - claim_id: emotion_transfer_fidelity_varies_by_arousal_category role: complicates claim: Emotion transfer fidelity varies systematically by emotion category, with high-arousal negative emotions consistently harder to clone than neutral or positive-valence speech. source: §7.3, Table 3 evidence: 'Across all five evaluated models, fear produced the lowest or near-lowest per-emotion scores (e.g., XTTS-v2: 0.7929 for fear vs. 0.848 for neutral), and disgust and anger also ranked consistently below neutral for most models.' confidence: medium relevance: low limitations: - Speaker similarity via WavLM cosine similarity does not capture naturalness, intelligibility, or audio artefacts, so a system with high speaker similarity scores may still produce perceptually poor speech, and the paper acknowledges the current absence of human evaluation. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.14579' published_date: "2025-09-18" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - VC architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: clf5_transcript_free_crosslingual_cloning role: supports claim: Cross-lingual zero-shot voice cloning in flow-matching TTS can be achieved without audio prompt transcripts by using forced alignment to partition training data at word boundaries. source: §4.3, Table 3 evidence: CL-F5 achieves WER 2.496% on a cross-lingual English test set drawn from four unseen languages (German, French, Hindi, Korean), while the F5-TTS baseline cannot perform cross-lingual cloning at all due to its transcript dependency. confidence: high relevance: high - claim_id: clf5_speaking_rate_predictors_reliable_duration role: supports claim: Multi-granularity speaking rate predictors trained with ordinal-aware Gaussian loss provide reliable duration estimation for transcript-free flow-matching TTS. source: §4.1, §4.2, Table 1, Table 2 evidence: The phoneme-level predictor (M1) achieves MAE=0.759s on LibriSpeech-PC and enables WER 2.079%, improving over the length-ratio baseline (2.205%); the syllable-level predictor (M2) is preferred for Chinese synthesis. confidence: high relevance: low - claim_id: clf5_removing_transcripts_costs_speaker_sim role: complicates claim: Removing audio prompt transcripts from flow-matching TTS incurs a small but consistent cost in speaker similarity. source: §4.2, Table 2, §5 evidence: CL-F5 M1 achieves SIM-o 0.663 vs. 0.668 for the transcript-conditioned baseline on LibriSpeech-PC; the authors also report reduced capacity for transferring accent and emotion. confidence: high relevance: high - claim_id: clf5_optimal_granularity_language_dependent role: refines claim: The optimal linguistic granularity for acoustic speaking rate prediction is language-dependent rather than universal. source: §4.1, Table 1 evidence: Phoneme-level (M1) outperforms syllable-level (M2) for English duration prediction, while M2 outperforms M1 for Chinese on SeedTTS test-zh; word-level (M3) is consistently the weakest across all languages. confidence: high relevance: low limitations: - Speaking rate predictors are trained exclusively on English and Chinese; genuinely low-resource or typologically distant language generalization remains untested despite empirical cross-lingual demonstration on four additional languages. - Speaker characteristic transfer (accent, emotional cues) is degraded compared to the transcript-conditioned baseline. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.14684' published_date: "2025-09-18" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: daiental_crossattention_beats_concatenation role: supports claim: Injecting time-varying environment conditioning via cross-attention into flow-matching TTS backbones produces higher environment fidelity than direct concatenation of environment features. source: §4.2, Table 1 evidence: DAIEN-TTS with cross-attention achieves ESMOS 3.65 versus 3.49 for the concatenation-based ablated variant, with naturalness MOS also improving from 3.68 to 3.78. confidence: medium relevance: low - claim_id: daiental_separation_module_enables_environmental_prompts role: supports claim: A pretrained source separation module can enable zero-shot TTS to use environmental speech as a speaker prompt without degradation from background noise. source: §4.1, Table 1 evidence: DAIEN-TTS achieves WER 1.93% and MOS 3.84 under environmental speaker prompts, slightly outperforming F5-TTS with clean prompts (WER 2.30%, MOS 3.80) and substantially outperforming F5-TTS under the same environmental conditions (WER 2.87%, MOS 3.09). confidence: high relevance: high - claim_id: daiental_dual_cfg_independent_control role: supports claim: Dual classifier-free guidance enables independent control of speaker and environment attributes in a jointly trained flow-matching TTS model. source: §2.3.1, §4.2 evidence: DCFG with separate guidance strengths for the speech-text condition and the environment condition is the inference-time mechanism through which DAIEN-TTS independently controls timbre and acoustic background; ablating the cross-attention (which effectively collapses environment guidance) degrades environment fidelity without recovering speaker similarity. confidence: medium relevance: high - claim_id: daiental_no_baselines_for_timevarying_env role: complicates claim: The absence of established baseline systems for time-varying environment-aware TTS makes it difficult to assess absolute performance gains beyond ablation comparisons. source: §3.2 evidence: For the background environment synthesis condition, no existing systems match the task definition; all comparisons are between DAIEN-TTS and its own ablated variant, with no external reference point for environmental fidelity. confidence: medium relevance: low - claim_id: daiental_flowmatching_sensitive_to_prompt_environment role: complicates claim: Flow-matching TTS systems trained on clean speech are sensitive to acoustic environment in the speaker prompt, with degradation visible in both speaker similarity and naturalness. source: §4.1, Table 1 evidence: F5-TTS (trained on clean LibriTTS data) shows SIM-o dropping from 0.58 to 0.49 and MOS dropping from 3.80 to 3.09 when the speaker prompt contains background noise rather than clean speech. confidence: high relevance: high limitations: - Model size is not reported; the SES module introduces a separate pretrained component that must be trained and aligned with the TTS module, adding pipeline complexity; the SNR adaptation strategy assumes target SNR should match the environment prompt, which may not always align with creative intent. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.14784' published_date: "2025-09-18" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: aligning_autoregressive_decoder_representations_with_pre_trained_asr role: supports claim: Aligning autoregressive decoder representations with pre-trained ASR encoder semantics improves content consistency in continuous-feature TTS. source: §2.3, §3.2, Table 1, Table 2 evidence: Representation alignment reduces WER 6.3 to 5.3 on LibriTTS ablation and produces 25%, 40%, and 30% relative CER/WER reductions on seed-tts-eval test-zh, test-en, and test-hard at 170K-hour scale. Training convergence accelerates by over 3.3x. confidence: high relevance: low - claim_id: using_the_target_acoustic_domain_mel_spectrogram_as role: contradicts claim: Using the target acoustic domain (mel-spectrogram) as an intermediate alignment target helps continuous AR TTS systems. source: §3.2, Table 1 evidence: Aligning transformer decoder outputs to mel-spectrograms (Exp 2, Table 1) raises WER from 6.3 to 6.7 and degrades SS1/SS2, while alignment to SenseVoice-Large ASR representations (Exp 1) reduces WER to 5.3. confidence: high relevance: medium - claim_id: continuous_feature_ar_tts_achieves_competitive_content_accuracy role: complicates claim: Continuous-feature AR TTS achieves competitive content accuracy but lags behind discrete-token systems on speaker similarity in zero-shot voice cloning. source: §3.3, Table 2 evidence: MELA-TTS achieves best CER on seed-tts-eval test-zh but lower SS1 (0.59 vs. F5-TTS 0.65) and SS2 (0.68 vs. F5-TTS 0.74) on test-en; attributed to the diffusion module's local context constraint versus discrete-token systems that condition on all prior tokens and the full prompt. confidence: high relevance: high - claim_id: unified_interleaved_training_on_streaming_and_offline_sequences role: supports claim: Unified interleaved training on streaming and offline sequences enables continuous AR TTS to operate in both modes at comparable quality without separate model variants. source: §2.4, §3.2, §3.3, Table 1, Table 2 evidence: Streaming mode achieves WER 5.0 versus offline 5.2, and SS1/SS2 parity, on LibriTTS. At 170K-hour scale, streaming WER is 2.5 versus offline 2.4 on test-en. confidence: high relevance: low - claim_id: continuous_feature_tts_without_speech_tokenization_scales_effectively role: supports claim: Continuous-feature TTS without speech tokenization scales effectively with training data, reducing content error rates and approaching discrete-token system performance. source: §3.3, Table 2 evidence: Scaling from LibriTTS (585h) to 170K-hour in-house data reduces English WER from 5.2 to 2.4 and improves SS1 from 0.48 to 0.59, bringing MELA-TTS close to CosyVoice 2.0 (2.6 WER) and CosyVoice 3.0 (2.5 WER). confidence: high relevance: medium limitations: - The diffusion module operates on local context only (the immediately adjacent latent vector and the previous mel chunk), preventing it from attending to the full prompt speech. This is identified as the likely structural cause of the consistent speaker similarity gap versus discrete-token systems across test-en and test-hard. The limitation is left as future work with no proposed fix. - Main comparison results rely on a proprietary 170K-hour multilingual dataset that is not publicly available, limiting reproducibility of the top-line numbers. Ablations are conducted on LibriTTS only (585 hours), which may not reflect behavior at full scale. No comparison to DiTAR on test-hard is available, making it unclear whether MELA-TTS's robustness advantage on challenging inputs holds relative to the closest continuous AR baseline. Code and demo are not released with the preprint. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.15253' published_date: "2025-09-18" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: [] relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: [] claims: - claim_id: chained_pipelines_compound_error_across_stages role: complicates claim: End-to-end speech generation pipelines that chain multiple vision and language components suffer from compounding errors that substantially limit overall system accuracy. source: §3.5 evidence: Speaker attribution accuracy in the full pipeline is 64.8% and emotion classification micro-F1 is 42.9%, but joint speaker-and-emotion accuracy is only 20.4%, illustrating how independent module errors multiply at the pipeline level. confidence: high relevance: medium - claim_id: llm_dialogue_attribution_fails_on_spatially_ambiguous_layouts role: complicates claim: LLM-based dialogue attribution for TTS struggles in spatially ambiguous layouts where dialogue text is not co-located with the speaking character. source: §3.3, Table 1 evidence: The full pipeline achieves 79.2% speaker accuracy on easy cases but only 20.5% on hard cases where dialogue is spatially separated from the character, showing the LLM falls back to proximity heuristics that fail when layout breaks co-location assumptions. confidence: high relevance: medium - claim_id: imbalanced_emotion_datasets_undermine_finegrained_tts_conditioning role: complicates claim: Imbalanced emotion datasets make fine-grained emotion conditioning for TTS unreliable, particularly for the neutral category which dominates real-world speech. source: §2.4, §3.4, Table 3 evidence: KangaiSet's extreme class imbalance (neutral 35%, disgust 0.47%) required binary reformulation of 7-class emotion classification, and the neutral class still achieves only 34.6% recall in 5-way LLM-based emotion prediction. confidence: medium relevance: low - claim_id: accuracy_metrics_underestimate_perceptual_quality_under_label_ambiguity role: complicates claim: Standard accuracy metrics for emotion classification may underestimate the perceptual quality of emotion-conditioned speech when ground-truth emotion labels are inherently ambiguous. source: §3.4 evidence: Manual inspection of mispredictions in the full pipeline setting reveals that many misclassified emotions still produce perceptually appropriate voice outputs, which the authors attribute to inherent visual and contextual ambiguity in comic-style emotional expression. confidence: medium relevance: medium limitations: - 'No perceptual evaluation of synthesised speech is conducted: the pipeline''s audio output quality, naturalness, and emotional appropriateness are not assessed with listening tests or any objective metric such as MOS, UTMOS, or WER.' caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.15462' published_date: "2025-09-18" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - codec - VC architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: semantic_factorization_enables_lower_bitrate_task_relevant_transmission role: supports claim: Factorising speech into semantic components (content, style, timbre) enables task-relevant transmission at lower bitrates than general-purpose codecs that encode all features uniformly. source: §3.3, Table 1 evidence: Content-style tokens at 650 bps achieve WER 0.15 and sentiment accuracy 59%, within 1% of EnCodec at 1.5 kbps (WER 0.20, accuracy 60%), while using approximately half the bitrate. confidence: high relevance: medium - claim_id: reused_speaker_reference_maintains_similarity_at_lower_average_bitrate role: supports claim: Reusing a compressed speaker reference transmitted once per speaker can maintain speaker similarity at lower average bitrate than encoding full audio continuously. source: §3.4, Table 1 evidence: Vevo with Zonos speaker embedding (1 second timbre) achieves SpkrSim 0.54, matching EnCodec at 1.5 kbps (0.54), while long-run average bitrate approaches 650 bps as registered speakers accumulate. confidence: medium relevance: high - claim_id: generative_reconstruction_trades_perceptual_quality_for_signal_fidelity role: complicates claim: Generative reconstruction from semantic tokens improves perceptual quality scores but degrades low-level signal fidelity metrics relative to waveform-level codecs. source: §3.4, Table 1 evidence: Vevo configurations outperform EnCodec on UTMOS and NISQA across all bitrates, but score lower on PESQ and STOI because the flow-matching decoder was not trained to preserve signal-level characteristics. confidence: high relevance: low - claim_id: semantic_codecs_face_latency_error_propagation_tradeoff role: complicates claim: Semantic codec approaches face a latency and error-propagation trade-off that general-purpose codecs avoid. source: §2.2, §4 evidence: Timbre transmission introduces per-speaker latency, and errors in that one-time transmission cause permanent voice reconstruction inaccuracies until a correction is transmitted. confidence: medium relevance: medium - claim_id: semantic_tokens_moderately_robust_to_channel_bit_errors role: supports claim: Neural semantic token representations are moderately robust to channel bit errors, maintaining acceptable downstream task performance at realistic noise levels. source: §3.5, Table 2 evidence: At 0.1% bit-flip rate the system shows no measurable degradation, and at 1% BER sentiment classification (0.63) and speaker verification (0.77) still exceed Opus at 5 kbps, with performance collapsing only above 10% BER. confidence: high relevance: medium limitations: - The system does not handle overlapping speakers, evaluation is restricted to English on a modest 1,000-clip test set from a single corpus, and the real-time latency introduced by speaker embedding generation and transmission is uncharacterised across network conditions. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.15845' published_date: "2025-09-19" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: conditioning_a_flow_matching_model_to_generate_speaker role: supports claim: Conditioning a flow matching model to generate speaker embeddings from natural language descriptions enables automatic voice assignment for characters without requiring reference audio. source: §2.2, §3.3, Table 2 evidence: The TTT model, trained with OT-CFM to map text descriptions (age, gender, personality, identity) to speaker embeddings, achieves CMS of 2.866 and sex accuracy above 96% for adult demographics; Qwen3-Embedding-0.6B outperforms T5-Large and RoBERTa-Large on all character attribute metrics. confidence: high relevance: high - claim_id: providing_llm_derived_emotion_scene_instructions_as_an role: supports claim: Providing LLM-derived emotion-scene instructions as an additional conditioning signal to TTS improves emotional expressiveness without substantially degrading intelligibility. source: §3.3, Table 3 evidence: CA-Instruct-TTS achieves MOS-E of 4.15 vs. 3.67 and MOS-N of 3.33 vs. 3.10 compared to the instruction-free CA-TTS baseline, with WER increasing only from 2.39% to 2.54%. confidence: high relevance: low - claim_id: embedding_based_speaker_generation_from_text_degrades_for role: complicates claim: Embedding-based speaker generation from text degrades for child voices due to acoustic similarity and training data artifacts. source: §3.3, Figure 2 evidence: The TTT model shows notably lower age accuracy for the child category (74.38% with TTT-Qwen3-0.6B) compared to adult categories (90%+). The paper attributes this to pre-pubertal voice similarity and to adult speakers imitating child voices in available training data. confidence: high relevance: medium - claim_id: larger_semantically_richer_text_encoders_improve_text_conditioned role: supports claim: Larger, semantically richer text encoders improve text-conditioned speaker embedding generation across demographic categories. source: §3.3, Table 2 evidence: TTT-Qwen3-0.6B achieves substantially higher CMS (2.866) and age accuracy across most categories compared to TTT-T5-Large (CMS 2.375) and TTT-RoBERTa-Large (CMS 2.359), suggesting that language understanding capacity from pre-trained LLMs transfers to voice attribute generation. confidence: high relevance: medium - claim_id: the_flow_matching_framework_extends_naturally_from_speech role: refines claim: The flow matching framework extends naturally from speech waveform generation to speaker embedding synthesis. source: §2.2 evidence: The OT-CFM framework is applied directly to the generation of continuous speaker embedding vectors from noise, using the same DiT backbone and conditioning architecture as the speech-side flow matching decoder. The speaker embedding space is treated as the target data distribution. confidence: high relevance: medium limitations: - All evaluations use proprietary internal data. No comparisons to published expressive audiobook TTS systems are included, limiting the ability to independently verify the reported gains in emotional expressiveness. - The TTT model struggles with child voice generation, an acknowledged limitation tied to data scarcity and label noise in child speech corpora. Future work is noted toward collecting authentic child speech. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.15969' published_date: "2025-09-19" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: fully_autoregressive_streaming_tts_can_achieve_first_packet role: supports claim: Fully autoregressive streaming TTS can achieve first-packet latencies under 150 ms without sacrificing intelligibility relative to non-streaming operation. source: §4, Table 1, Table 3 evidence: On LibriSpeech long (Table 2, full-stream evaluation against CosyVoice2 under matched conditions), VoXtream achieves substantially lower WER (3.24% vs. 6.11%) and wins the naturalness preference study (57% vs. 31%, p < 5e-10), though CosyVoice2 retains higher SPK-SIM (0.685 vs. 0.564) due to its non-autoregressive flow-matching decoder. confidence: high relevance: medium - claim_id: training_data_scale_is_the_primary_driver_of role: supports claim: Training data scale is the primary driver of speaker similarity in zero-shot TTS, and systems trained on an order-of-magnitude less data show measurable SPK-SIM gaps even when naturalness scores are competitive. source: §4, Table 1, Table 2 evidence: Among mid-scale systems (comparable training data), VoXtream-NS (non-streaming variant) achieves the best SPK-SIM (0.537 on SEED) and UTMOS (3.89 on SEED) and attains the second-best WER on LibriSpeech test-clean (2.99%) across all systems evaluated. confidence: high relevance: high - claim_id: borrowing_frozen_depth_transformer_weights_from_a_large role: supports claim: Borrowing frozen depth transformer weights from a large-scale pretrained model provides substantial quality improvement for a mid-scale system without requiring additional large-scale training. source: §3, Table 4 evidence: Critically, the DT weights are borrowed frozen from the CSM model trained on a large-scale dataset — this cross-model distillation substantially improves output quality without requiring large-scale training. confidence: high relevance: low - claim_id: full_stream_input_processing_introduces_only_marginal_quality role: supports claim: Full-stream input processing introduces only marginal quality degradation relative to output-streaming when a bounded phoneme look-ahead is used, suggesting that input latency and output quality are largely decoupled in autoregressive codec TTS. source: §4, Table 1 evidence: 'The streaming variant VoXtream (full-stream) introduces only minor degradations: WER rises from 2.99% to 3.15% on LibriSpeech test-clean; SPK-SIM and UTMOS are essentially unchanged.' confidence: high relevance: low - claim_id: non_autoregressive_flow_matching_decoders_used_in_chunk role: supports claim: Non-autoregressive flow-matching decoders used in chunk-based streaming systems incur first-packet latencies exceeding 1.5 seconds on standard hardware, which is prohibitive for real-time spoken conversational agents. source: §4, Table 3 evidence: With `torch.compile`, the full system achieves a first-packet latency of 102 ms and an RTF of 0.17, meaning it runs more than 5 times faster than real time. confidence: high relevance: low limitations: - Speaker similarity remains lower than large-scale systems trained on hundreds of thousands of hours, particularly in full-stream mode (SPK-SIM 0.458 on LibriSpeech test-clean vs. 0.587 for CosyVoice2 trained on 167k hours). The system is English-only; multilingual extension is not addressed. Prosody and speaking rate are not explicitly controllable at inference. The use of a frozen CSM depth transformer introduces a dependency on an external large-scale model. Long-form streaming beyond 10-15 second utterances is identified as future work. Performance in adverse or spontaneous speech conditions (outside the training domain) is untested. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.17021' published_date: "2025-09-21" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: exposure_bias_produces_measurable_quality_degradation_in_autoregressive role: supports claim: Exposure bias produces measurable quality degradation in autoregressive codec TTS, with the effect scaling with output sequence length. source: §3.4, Figure 3; §3.2, Table 1 evidence: Token prediction accuracy under free-running inference is consistently lower than under teacher forcing, with the gap widening over sequence position; WER improvement from hybrid training is larger on long-form LibriSpeech (6.23→4.21%) than on short Seed-TTS utterances (4.83→4.64%). confidence: high relevance: low - claim_id: hybrid_training_that_mixes_teacher_forced_and_self role: supports claim: Hybrid training that mixes teacher-forced and self-generated tokens reduces WER and improves speaker similarity in autoregressive LM-based TTS relative to standard teacher-forcing fine-tuning. source: §3.2, Table 1; §3.3, Figure 2 evidence: Prompt-guided hybrid fine-tuning of CosyVoice2 on LibriSpeech reduced WER from 6.23% to 4.21% and raised speaker similarity from 0.74 to 0.78 on LibriSpeech test-clean (Table 1); human MOS scores approach ground-truth quality on 30-sample evaluation. confidence: high relevance: high - claim_id: the_benefits_of_exposure_bias_mitigation_in_autoregressive role: complicates claim: The benefits of exposure-bias mitigation in autoregressive TTS diminish for short utterances where prediction errors have less opportunity to accumulate. source: §3.2, Table 1 evidence: On Seed-TTS utterances under 10 seconds, WER improvement is smaller (4.83→4.64) compared to LibriSpeech, where longer sequences amplify the compounding effect of distributional mismatch. confidence: high relevance: low - claim_id: eos_misprediction_rate_serves_as_a_reliable_proxy role: supports claim: EOS misprediction rate serves as a reliable proxy for training-time exposure bias severity, enabling adaptive control of self-conditioning intensity. source: §3.5; §3.6, Table 2 evidence: EOS-guided adaptive scheduling tracks premature termination events across iterations; ablation shows removing this component raises WER from 4.21% to 4.98% and drops speaker similarity from 0.80 to 0.72 on LibriSpeech test-clean. confidence: high relevance: high limitations: - The evaluation is limited to LibriSpeech and Seed-TTS with a single speaker conditioning setup; generalisation to multilingual or highly expressive TTS domains is untested. The method is applied only as fine-tuning on top of existing CosyVoice models, leaving open whether hybrid training from scratch would yield similar or greater benefits. The MOS evaluation uses only 30 samples, which provides limited statistical power for assessing naturalness improvements. The 1.5x training overhead, while modest, is an additional cost relative to standard fine-tuning. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.17516' published_date: "2025-09-22" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: decoupling_the_training_pairing_between_the_speech_prompt role: supports claim: Decoupling the training pairing between the speech prompt and the synthesis target (rather than using identical prompt/target audio) improves text-prosody alignment in style-controllable TTS, at the cost of an inflated speaker-similarity score under the coupled setup. source: §3.3, Table 2 evidence: Switching from non-decoupled to decoupled prompt/target training raises S-MOS from 3.45±0.09 to 3.82-3.93±0.06-0.07 while reducing speaker-similarity (SS) from 0.87 (interpreted as over-similar timbre/prosody) to 0.69-0.8. confidence: high relevance: medium - claim_id: providing_an_autoregressive_speech_language_model_with_explicit role: supports claim: Providing an autoregressive speech language model with explicit surrounding-sentence text context improves narrative coherence in long-form speech generation beyond what sentence-isolated synthesis achieves. source: §2.1, §3.3, Table 1 evidence: Adding pre/post-context text sequences raises Chapter M-MOS from 3.88±0.07 (CosyVoice2 baseline, sentence-level synthesis) to 4.13±0.09 (Infer-ctx), and a qualitative example shows the model generating contextually appropriate laughter for a sentence with no explicit emotional markers when given the preceding sentence's context. confidence: high relevance: medium - claim_id: the_strength_of_the_prompt_target_decoupling_threshold role: complicates claim: The strength of the prompt/target decoupling threshold used during training trades off speaker-identity stability against naturalness, rather than improving both simultaneously. source: §3.3, Table 2 evidence: Sweeping the voiceprint-similarity clustering threshold shows a lower threshold reduces speaker-similarity (with occasional timbre discontinuities) but yields slightly higher S-MOS, while a higher threshold approaches the non-decoupled model's inflated similarity at the cost of lower S-MOS. confidence: high relevance: medium - claim_id: self_distillation_from_a_pretrained_emotional_tts_model role: supports claim: Self-distillation from a pretrained emotional TTS model, filtered by objective quality metrics, can mitigate the scarcity of high-intensity emotional training data and improve controllable emotional expressiveness. source: §2.2, §3.3, Table 3, Table 4 evidence: On the CV3-Eval text-unrelated test set, the model trained with 500 hours of emotional data plus 5,300 hours of self-distilled augmentation achieves larger high- vs. low-intensity F1 discriminability than a CosyVoice2-instruct baseline (e.g. angry ΔF1 0.31 vs. 0.07) and higher mixed-emotion S-MOS (3.87±0.09 vs. 3.35±0.07). confidence: high relevance: medium limitations: - All training data (1M-hour audiobook corpus, 100K-hour context-aware corpus, 500-hour emotional corpus, 5K-hour augmentation set) and all three test sets (Test-NAR, Test-DIA, Test-CHAP) are proprietary and internally constructed; no code, model weights, or evaluation data are released, which limits independent verification of the reported gains. - The evaluation is confined to Chinese-language audiobook content with a single subjective rater pool (50 native Chinese speakers); generalization to other languages or narration styles is untested. The decoupling-threshold ablation reports trends across tested values without identifying a clearly optimal setting, and the paper notes the risk of persona instability at high thresholds is hypothesized rather than directly measured across the full range. The authors flag future work on scaling chapter-level context data and applying reinforcement learning for further quality improvement. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.17765' published_date: "2025-09-22" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - SCA - TTS architecture: - autoregressive-LM - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: qwen3omni_causal_convnet_vocoder_latency role: supports claim: Replacing a block-wise diffusion vocoder with a lightweight causal convolutional decoder, driven by a multi-codebook autoregressive token predictor, can substantially reduce first-packet latency in streaming speech generation without sacrificing competitiveness on content-consistency metrics. source: §2.4, §2.5, Table 1, Table 13 evidence: The Talker's multi-codebook AR scheme plus a 200M-parameter causal ConvNet Code2Wav stage achieves a 234ms end-to-end first-packet latency at 1x concurrency and the lowest reported content-consistency error on SEED test-en (1.39) among all compared zero-shot TTS systems. confidence: high relevance: high - claim_id: qwen3omni_multimodal_no_degradation role: supports claim: Mixing unimodal and cross-modal training data from the earliest stage of pretraining allows a language model to add new input/output modalities without degrading its original text, vision, or audio-specific capabilities relative to matched unimodal baselines. source: §6, Table 16 evidence: A controlled comparison of parameter-matched text-only, vision-only, and Omni models trained on identical corpora, schedules, and compute shows the Omni model matches or exceeds the unimodal baselines on text, vision, and video benchmarks. confidence: high relevance: low - claim_id: qwen3omni_voice_cloning_language_unevenness role: complicates claim: Strong zero-shot voice-cloning performance in one or two conditioning languages does not guarantee comparable speaker-similarity performance uniformly across all supported languages. source: §5.2.2, Table 14 evidence: Against MiniMax-Speech and ElevenLabs Multilingual v2 on a 10-language test set, the system leads by a substantial margin on Chinese, English, and French but reports only competitive, non-leading speaker-similarity or content-consistency scores on several other languages. confidence: medium relevance: high - claim_id: qwen3omni_purpose_built_audio_encoder role: supports claim: A large-scale, purpose-built supervised audio encoder trained from scratch for a multimodal LLM's audio pathway can outperform reusing a general pretrained ASR encoder (e.g., Whisper) as the perceptual front-end for both speech understanding and downstream speech generation. source: §1, §2.2, Table 6, Table 7 evidence: Replacing the Whisper-based audio encoder from the predecessor system with AuT, trained from scratch on 20 million hours of supervised audio at a 12.5 Hz token rate, is cited as a key driver of gains across ASR, lyric-ASR, and voice-interaction benchmarks. confidence: medium relevance: medium limitations: - Speech generation quality is evaluated exclusively with automatic metrics (WER/CER, embedding-based speaker similarity); no human MOS or listening-test results are reported for the Talker's synthesized speech. - The reported 234ms first-packet latency is described as theoretical, measured under a specific deployment configuration, and degrades substantially under higher concurrency (up to 1172ms at 6-way concurrency). caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.17988' published_date: "2025-09-22" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: low evidence_role: - infrastructure current_role: minor method_family: - nonautoregressive_flow_matching claims: - claim_id: in_the_wild_pipelines_produce_usable_tts_corpora_for_low_resource_languages role: supports claim: In-the-wild recordings can be converted into usable TTS training corpora at scale for languages lacking studio-recorded resources, provided the processing pipeline applies dedicated noise, diarization, VAD, and perceptual-quality filtering. source: §3.1, §3.2, Table 1 evidence: An adapted Emilia-style pipeline (denoising, Pyannote diarization, Silero VAD, DNSMOS P.835 OVRL >= 3.0 filtering) applied to parliamentary recordings yields 900 hours of Finnish and 5090 hours of Swedish TTS-ready speech, where previously only 60 hours (Finnish) or zero hours (Swedish) of public data existed. confidence: high relevance: low - claim_id: cross_validating_two_asr_models_filters_noisy_transcripts role: supports claim: Cross-validating transcripts from two independently trained ASR models is a practical way to filter noisy automatic transcripts for corpus construction when a single ASR system is unreliable for the target language. source: §3.1 evidence: Because faster-whisper-large-v3 underperformed on Finnish, the pipeline retained only segments where Whisper and a Finnish-specific Wav2Vec2-large model's transcripts agreed within 5% WER, using Whisper's transcript as the final label. confidence: medium relevance: low - claim_id: explicit_alignment_improves_intelligibility_at_naturalness_cost role: complicates claim: Explicit duration/alignment modeling in non-autoregressive TTS improves intelligibility and reduces hallucination relative to implicit alignment learned via diffusion transformers, but this comes at some cost to perceived naturalness. source: §5, Table 2 evidence: Matcha-TTS (explicit MAS alignment) achieves substantially lower CER (2.55% Finnish, 4.66% Swedish) and far fewer hallucinated utterances than F5-TTS-Base (6.72% Finnish, 13.64% Swedish CER; up to 18/500 hallucinations), yet F5-TTS-Base scores higher (better) CMOS in both languages, showing CER and CMOS can rank systems oppositely. confidence: high relevance: low - claim_id: speaker_similarity_metric_ranking_inconsistent_across_languages role: complicates claim: The relative speaker-similarity performance of prompt-based zero-shot conditioning versus fixed speaker-embedding conditioning is not consistent across languages or training corpora, and results from one language should not be assumed to transfer to another. source: §5, Table 2 evidence: On the same two model architectures, fixed speaker-embedding conditioning (Matcha-TTS) outperforms prompt-based conditioning (F5-TTS-Base) on Finnish SIM (0.566 vs 0.538) but underperforms it on Swedish SIM (0.442 vs 0.53). confidence: high relevance: high limitations: - The corpus is drawn entirely from formal parliamentary proceedings, a narrow speech register, and subjective evaluation used only 20-22 raters per language, a modest panel size for CMOS/SMOS testing. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.18928' published_date: "2025-09-23" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - diffusion - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_diffusion_acoustic claims: - claim_id: ardm_dpo_extension_to_diffusion role: supports claim: Direct Preference Optimization can be extended beyond discrete-token generative models to continuous-token autoregressive diffusion models for speech, by marginalizing the DPO objective over intermediate diffusion states of the autoregressive sampling trajectory. source: §3, §4.2, Table 1 evidence: The derived ARDM-DPO objective reduces to a log-sigmoid of the difference in denoising losses between the fine-tuned and reference model on winning vs. losing trajectories; applied to a 0.4B DiTAR model it nearly doubles F0 variance (14.2 to 29.2) while WER improves (5.17 to 3.73) after 200 training steps. confidence: high relevance: low - claim_id: ardm_dpo_automatic_proxy_preference_data role: supports claim: Automatic proxy signals, rather than human-labeled preference pairs, can be used to construct DPO training data that improves a targeted, measurable aspect of speech generation quality. source: §4.3, Table 2 evidence: Task B builds 430k preference pairs by ranking 16 candidate outputs per prompt using a separately trained CTC model's negative log-likelihood as an intelligibility proxy; this reduces CER by 25% (8.37 to 6.32) on the Seed-TTS-Eval hard test set while preserving speaker similarity and naturalness in subjective tests. confidence: high relevance: high - claim_id: ardm_dpo_training_instability role: complicates claim: DPO fine-tuning of autoregressive diffusion speech models is not guaranteed to be stable and can require early stopping to prevent degradation of speaker similarity and naturalness. source: §4.2, Fig. 3, §5 evidence: Across the beta in {200, 400, 800} grid search for Task A, speaker similarity decreases monotonically throughout training regardless of beta, and the diffusion loss for both preferred and dispreferred samples increases during training instead of diverging as the objective predicts. confidence: high relevance: high - claim_id: ardm_dpo_beats_bestofk_and_raft role: refines claim: Preference-optimizing a generative TTS model via a single fine-tuning run can match or exceed the quality gains obtained by inference-time best-of-K sampling or iterative rejection-sampling fine-tuning, without their added inference-time or multi-round training cost. source: §4.2, Table 1 evidence: On Task A, ARDM-DPO (200 steps, beta=200) reaches F0V 29.2, higher than best-of-64 sampling (26.6, no training) and three iterations of RAFT rejection-sampling SFT (20.1), while requiring a single training run and no extra inference-time sampling. confidence: high relevance: low limitations: - All experiments fine-tune a single proprietary 0.4B DiTAR checkpoint pretrained on an internal 280,000-hour corpus, with no code or model release; the reported gains cannot be independently reproduced or verified on a public base model. - Training instability on Task A (monotonically decreasing speaker similarity across all tested beta values) is acknowledged as unresolved and requires manual early stopping rather than a principled stopping criterion. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.19231' published_date: "2025-09-23" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - VC - evaluation architecture: - diffusion - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - diffusion_voice_cloning claims: - claim_id: chiressd_style_disentanglement_suppresses_mispronunciation role: supports claim: Disentangling acoustic (pronunciation-linked) style from prosodic and speaker-identity style in a TTS-based reconstruction system allows mispronunciation patterns to be suppressed while speaker identity is preserved. source: §5.1, Table 1 evidence: ChiReSSD, which selectively fine-tunes acoustic/prosodic style encoders and the pitch extractor while reusing StyleTTS2's disentangled style representations, achieves higher speaker similarity (0.62) and lower pitch deviation (19.98%) than a one-shot baseline that transfers style holistically without domain adaptation (0.52 similarity, 22.58% F0 difference). confidence: medium relevance: high - claim_id: chiressd_asr_metrics_conflict_with_identity role: complicates claim: Optimizing a speech reconstruction system for automatic-speech-recognition-based intelligibility metrics can conflict with optimizing for speaker-identity preservation. source: §5.2, Table 2 evidence: A single-speaker adult-voice TTS baseline with no style transfer obtains the lowest WER/CER on STAR (0.34/0.11) precisely because it discards the child speaker's identity, while the identity-preserving ChiReSSD model scores worse on the same ASR-based metrics (WER 0.49, CER 0.29) despite being judged clinically more accurate. confidence: medium relevance: medium - claim_id: chiressd_automatic_pcc_approximates_clinical role: supports claim: An automatic phone-recognition-based estimate of consonant production accuracy can approximate expert clinical judgments of speech intelligibility well enough to reduce, though not eliminate, the need for manual phonetic transcription. source: §5.3 evidence: Automatic PCC estimates derived from a universal phone recognizer and Levenshtein distance correlate with a certified speech-language therapist's manual PCC annotations at Pearson correlation 0.63 across 21 sample pairs. confidence: medium relevance: low - claim_id: chiressd_generalizes_across_disorder_populations role: supports claim: A style-based TTS reconstruction approach adapted for one clinical population of disordered speech can generalize to a structurally different disorder and age group without population-specific retraining. source: §5.4, Table 3 evidence: ChiReSSD, trained only on pediatric SSD data, reduces CER to below 0.03 and WER to as low as 0.06 on the TORGO adult dysarthria dataset across severity levels, while keeping speaker similarity at 0.74-0.77. confidence: medium relevance: high limitations: - Clinical validation of the automatic PCC metric rests on only 21 annotated sample pairs from a single speech-language therapist, with no inter-rater reliability reported; the STAR evaluation set is small (234 utterances from six children); the guidance parameters (alpha, beta) were tuned rather than learned. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.19592' published_date: "2025-09-23" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: decoding_codebooks_of_a_multi_codebook_acoustic_frame role: supports claim: Decoding codebooks of a multi-codebook acoustic frame with explicit intra-frame dependencies (iteratively) yields a generated token distribution closer to the ground truth than decoding all codebooks in parallel under an independence assumption. source: §3.3.1, Fig. 2e evidence: Across all tested frame-stacking factors (1, 2, 4), every autoregressive- or MaskGIT-local-transformer configuration achieves lower Fréchet Distance than every parallel-sampled configuration, including the unstacked parallel baseline. confidence: high relevance: low - claim_id: offloading_intra_frame_codebook_decoding_to_a_small role: supports claim: Offloading intra-frame codebook decoding to a small auxiliary transformer lets a primary acoustic decoder predict multiple codec frames per generation step, substantially increasing throughput without retraining the underlying codec at a lower frame rate. source: §2.4, §3.3.2, Table 1, Fig. 2f evidence: At a frame-stacking factor of 2, the autoregressive local-transformer model reaches 2.1x throughput over the unstacked parallel baseline while improving Fréchet Distance and keeping WER, speaker similarity, and MOS within or better than baseline; the MaskGIT variant reaches 3.1x throughput at comparable quality. confidence: high relevance: high - claim_id: parallel_independent_codebook_prediction_degrades_disproportionately_not_just role: complicates claim: Parallel independent codebook prediction degrades disproportionately, not just proportionally, as more codebook information is packed into a single decoding step. source: §3.3.2 evidence: Applying parallel sampling to a 2x frame-stacked model (instead of routing through the local transformer) increases unseen-speaker Fréchet Distance by 67% relative to the unstacked parallel baseline and lowers MOS. confidence: high relevance: medium - claim_id: the_throughput_gains_of_iterative_masked_prediction_decoding role: complicates claim: The throughput gains of iterative masked-prediction decoding for acoustic codebooks come at a quality cost that grows sharply once the number of sampling steps is small relative to the number of tokens being resolved per step. source: §3.3.2, Fig. 2a evidence: At a stacking factor of 4, the MaskGIT local transformer with 3 sampling steps decoding 32 tokens per step (8 codebooks × 4 stacked frames) shows a significant MOS drop relative to baseline, while the autoregressive local transformer at the same stacking factor does not exhibit this drop. confidence: high relevance: low limitations: - 'Robustness to unseen speakers degrades substantially at higher frame-stacking factors: unseen-speaker speaker similarity falls from 0.765 at stacking factor 1 to 0.642 (AR LT) and 0.624 (MaskGIT LT) at stacking factor 4, which the authors themselves flag by recommending high stacking only "when not needing zero-shot functionality."' - All experiments build on a single base system (Koel-TTS) and a single codec (NanoCodec, FSQ-based, 8 codebooks at 21.5 fps); it is untested whether the same tradeoffs hold for RVQ-based codecs, different codebook counts, or other primary-decoder architectures. The MaskGIT local transformer's degradation at high stacking is attributed to using only 3 sampling steps, but the paper does not run the ablation that would confirm more steps recover quality, leaving the speed-quality Pareto frontier for MaskGIT only partially characterized. Training data is described only as "the same 18k hours of data as in the Koel-TTS paper," with no further specification of language, speaker count, or domain in this paper itself. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.19668' published_date: "2025-09-24" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: image_generation_cfg_techniques_fail_to_transfer_to_zero_shot_tts role: supports claim: CFG strategies effective in image generation (weight schedules, perpendicular re-weighting, zero-init) do not improve zero-shot TTS quality and can degrade it. source: §5, Figure 1, Figure 2 evidence: Weight schedules, perpendicular re-weighting, and zero-init from image generation all fail to improve F5-TTS, with high CFG weights early in generation particularly harmful to both SIM and WER. confidence: high relevance: high - claim_id: timestep_selective_cfg_improves_speaker_similarity_without_wer_cost role: supports claim: Applying standard CFG only in early timesteps and switching to speaker-emphasized CFG for later timesteps can improve speaker similarity in zero-shot TTS without substantially increasing WER. source: §4, §5, Figure 4 evidence: The def text strategy raises F5-TTS SIM from 0.675 to 0.682 on LibriSpeech (WER 0.020 to 0.022) and from 0.679 to 0.690 on Seed-TTS-eval English (WER unchanged at 0.018). confidence: high relevance: high - claim_id: separated_condition_cfg_effectiveness_depends_on_text_architecture_and_language role: refines claim: The effectiveness of separated-condition CFG in zero-shot TTS is dependent on the text representation architecture of the underlying model, with language differences producing divergent outcomes for the same CFG strategy. source: §5, §6, Figure 6 evidence: The def text strategy improves SIM for F5-TTS on English but shows no SIM improvement and increased WER on Mandarin, while CosyVoice 2's larger LLM-based text encoder shows no comparable language gap. confidence: medium relevance: high - claim_id: inference_time_cfg_tuning_insufficient_to_close_sota_gap role: complicates claim: Inference-time CFG tuning alone is insufficient to close the speaker similarity gap between open-weight and closed-source zero-shot TTS systems. source: §6, Table 1 evidence: Even with the best CFG variant, F5-TTS and CosyVoice 2 SIM scores (around 0.68-0.69) remain well below closed-source systems Minimax-Speech (SIM 0.738) and Seed-TTS (SIM 0.762) on English Seed-TTS-eval. confidence: high relevance: high limitations: - Only two base models are tested and the root cause of the Mandarin failure for F5-TTS is hypothesized but not verified experimentally; no subjective evaluation (MOS) is reported, only objective SIM/WER metrics. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.19883' published_date: "2025-09-24" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - singing architecture: - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - modular_hybrid_systems claims: - claim_id: comelsinger_prompt_leakage_language_independent role: supports claim: Prompt-based conditioning in masked generative or codec-language speech synthesis models causes measurable leakage of prosodic attributes from the acoustic prompt into the synthesized output, independent of the target language. source: §V.A, Table I evidence: Paired-prompt outputs show consistently lower pitch/energy/jitter differences than unpaired-prompt outputs from the same speaker on both LibriTTS (English) and AISHELL-3 (Mandarin) when synthesizing with MaskGCT. confidence: high relevance: medium - claim_id: comelsinger_contrastive_reduces_leakage role: supports claim: Explicit contrastive regularization between the acoustic prompt and an external control signal (e.g., melody/pitch) reduces attribute leakage and improves controllability in prompt-based zero-shot synthesis. source: §V.D, Table V evidence: Removing the coarse-to-fine (sequence + frame level) contrastive loss increases F0-RMSE from 0.042 to 0.08 and lowers SingMOS from 4.32 to 4.12 on the seen-singer test set, with sequence-level and frame-level components independently ablated to show complementary effects on speaker-identity and pitch-detail metrics respectively. confidence: high relevance: high - claim_id: comelsinger_inloop_transcription_supervision role: supports claim: Integrating auxiliary transcription-derived frame-level supervision directly into a synthesis model's training loop, rather than using it only as an offline data-cleaning step, improves fine-grained attribute alignment. source: §V.D, Table V evidence: Removing the in-loop SVT auxiliary loss produces the largest single-component degradation in the ablation, raising F0-RMSE from 0.042 to 0.194 and lowering SingMOS from 4.32 to 3.95. confidence: high relevance: low - claim_id: comelsinger_lora_matches_full_finetuning role: refines claim: Parameter-efficient fine-tuning can match or exceed full fine-tuning when adapting a large pretrained codec-based speech model to a lower-resource downstream domain, provided the low-rank capacity is placed appropriately. source: §V.D, Table VII evidence: LoRA fine-tuning of the S2A diffusion estimator (6.51% trainable parameters) achieves lower F0-RMSE (0.053) and higher speaker similarity (0.92) than fully fine-tuning the same backbone (100% trainable, F0-RMSE 0.099, SECS 0.859). confidence: medium relevance: high limitations: - The zero-shot evaluation set (OpenSinger) lacks native music-score annotations, so pitch/duration sequences are paired from a different corpus (M4Singer); evaluation is confined to Mandarin singing corpora; overall S2A/T2S model size is not reported; the subjective evaluation panel is modest (20 musically trained raters). caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.19928' published_date: "2025-09-24" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - evaluation architecture: [] relevance: medium evidence_role: - evaluation_caution current_role: active_evidence method_family: [] claims: - claim_id: token_edit_distance_correlates_better_with_prosody_perception_than_acoustic_proxies role: supports claim: Acoustic proxy metrics for prosodic variation correlate weakly with human perception, while distance measures computed over discretized self-supervised speech tokens correlate substantially better. source: §4.1, Table 1 evidence: Averaged Pearson correlation with human PMOS ratings is r-bar = 0.30 for log F0 RMSE and r-bar = 0.66 for MCD, versus r-bar = 0.77 for the proposed token-edit-distance metric (DS-WED), aggregated via Fisher's Z transformation across 1000 samples and 2000 ratings. confidence: high relevance: low - claim_id: ar_generation_more_prosodically_diverse_than_flow_matching_but_not_mgm role: supports claim: Autoregressive generation provides greater output diversity in prosody than non-autoregressive flow-matching models with implicit text-speech alignment, but this advantage does not extend to non-autoregressive masked generative modeling. source: §4.4-4.5, Table 2 evidence: On the DS-WED benchmark across LibriSpeech test-clean and Seed-TTS test-en, three AR systems (XTTS-v2, CosyVoice, CosyVoice 2) outperform three flow-matching NAR systems (E2 TTS, F5-TTS, ZipVoice), while the masked generative modeling system (MaskGCT) surpasses all AR systems on LibriSpeech and remains competitive on Seed-TTS despite training on the same Emilia corpus as the flow-matching systems. confidence: high relevance: medium - claim_id: duration_control_significant_but_insufficient_for_nar_prosody_diversity role: complicates claim: Explicit duration control during inference is a significant, but not sufficient, factor in restoring prosodic diversity to non-autoregressive TTS systems with implicit alignment. source: §4.5, Table 3 evidence: Applying duration perturbation (0.8-1.2x) to two flow-matching/MGM NAR systems increases DS-WED diversity by 13.8-28.5%, but the perturbed flow-matching system (F5-TTS) still lags behind AR and MGM systems evaluated without perturbation. confidence: medium relevance: low - claim_id: intelligibility_dpo_suppresses_prosody_diversity role: complicates claim: Preference optimization that targets one quality dimension (intelligibility) can measurably suppress output diversity along an unrelated axis (prosody) as a side effect. source: §4.5, Table 4 evidence: Applying DPO for intelligibility to CosyVoice 2 and MaskGCT reduces DS-WED prosody-diversity scores by 18.8% and 2.9-3.2% respectively across two test sets, with no diversity-specific reward term in the alignment objective. confidence: medium relevance: medium - claim_id: llm_judge_unreliable_for_finegrained_prosodic_variation role: contradicts claim: A general-purpose large audio language model with strong multimodal reasoning capability can serve as a reliable automatic judge of fine-grained prosodic variation. source: §4.5, Table 5 evidence: Gemini 2.5 Pro, prompted to rate relative prosodic difference within groups of five samples, achieves only a weak correlation with human PMOS ratings (r-bar = 0.27) and unstable, wide-confidence-interval correlations with objective acoustic and token-based metrics. confidence: high relevance: medium limitations: - DS-WED has only been validated on English speech; the benchmark covers only seven open-source systems and two evaluation corpora, and the DPO and duration-perturbation exploration each cover only two systems, limiting generalization of the paradigm-level conclusions. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.22718' published_date: "2025-09-24" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - singing architecture: - hybrid relevance: low evidence_role: - historical_context current_role: minor method_family: - modular_hybrid_systems claims: - claim_id: performsinger_visual_cues_substitute_duration role: supports claim: Synchronized visual cues from a speaker's or singer's mouth region can substitute for explicit phoneme-level duration annotations in duration-based speech and singing synthesis pipelines. source: §3.5, §5.1, Table 1 evidence: Replacing externally supplied phoneme durations with a duration predictor conditioned on lip-cue-fused content features allows the duration predictor to operate without duration inputs at inference, while achieving lower MCD and F0 Frame Error than a duration-free StyleSinger baseline with no visual input. confidence: medium relevance: medium - claim_id: performsinger_speech_fusion_needs_adaptation_for_singing role: complicates claim: Fusion mechanisms designed for one cross-modal alignment problem do not transfer directly to singing, because singing's rhythmic and prosodic complexity differs from speech. source: §5.2, Table 3 evidence: Adopting StyleDubber's Stepwise Monotonic Multi-head Attention (designed for movie-dubbing lip-speech alignment) for textual-visual fusion in this SVS setting produced worse MCD, speaker cosine similarity, and lip-sync error than the no-visual baseline, whereas the proposed VCFM improved on the same baseline across all metrics. confidence: medium relevance: medium - claim_id: performsinger_staged_training_beats_joint role: supports claim: Staged training that first establishes a strong acoustic model before introducing an auxiliary modality improves quality over jointly training all components from the start. source: §5.2, Table 3 evidence: The two-stage schedule (stage one trains phoneme/pitch encoders and decoder; stage two adds the frozen visual encoder and VCFM) outperforms single-stage training on every objective metric reported. confidence: medium relevance: low - claim_id: performsinger_dubbing_architectures_dont_generalize role: complicates claim: Visual dubbing architectures built for spoken dialogue do not generalize to singing without substantial adaptation, even when augmented with pitch information. source: §4.1, §5.1, Table 1 evidence: HPMDubbing, a lip-synchronized speech dubbing model, and its pitch-augmented variant both perform far below SVS-specific baselines on every metric, despite the pitch-augmented variant adding phoneme-level pitch as input. confidence: medium relevance: low limitations: - All results are reported on a single self-constructed dataset (9 singers, 69 songs, ~3 hours, Chinese only), evaluated against two adapted rather than natively-designed baselines; the visual encoder is frozen during stage-two training rather than fine-tuned jointly; model parameter counts and inference latency are not reported. caveats: - For zero-shot TTS, this paper provides low-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.20802' published_date: "2025-09-25" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: many_transformer_layers_in_autoregressive_llm_tts_backbones role: supports claim: Many transformer layers in autoregressive LLM-TTS backbones contribute little to synthesis quality and can be removed with minimal loss in naturalness and speaker similarity. source: §4.1, Table 1 evidence: Halving CosyVoice 2's depth to 12 layers (39.7% fewer parameters) increases Seed-TTS WER by only 0.68 and decreases NMOS by 0.13, while speaker similarity and UTMOS remain essentially unchanged. confidence: high relevance: high - claim_id: layer_importance_criteria_developed_for_pruning_text_only role: refines claim: Layer-importance criteria developed for pruning text-only LLMs do not transfer directly to speech generation backbones; intelligibility-grounded criteria are needed to identify prunable layers correctly. source: §2.1, §4.2, Table 2 evidence: Cosine-based layer importance (input/output latent similarity) diverges from WER-based importance in TTS backbones, and substituting it for the proposed WER-based criterion increases WER from 1.59 to 1.74 and CER from 0.54 to 0.61 on LibriTTS test-clean. confidence: high relevance: low - claim_id: knowledge_distillation_can_recover_most_of_the_performance role: supports claim: Knowledge distillation can recover most of the performance lost from aggressive layer pruning in speech-generation LLMs using only a small fraction of the original pretraining data. source: §3, §4.1, Table 1b evidence: Fine-tuning pruned variants required under 5% of the original pretraining data (25% of LibriTTS for CosyVoice 2, an upper-bounded 12.5% of LibriHeavy for LLaSA) yet speaker similarity and UTMOS changed by at most 0.045 and 0.04 respectively relative to the uncompressed backbones. confidence: high relevance: high - claim_id: robustness_to_layer_pruning_varies_substantially_across_llm role: complicates claim: Robustness to layer pruning varies substantially across LLM-TTS backbones depending on how redundant their transformer layers are, so a single pruning ratio does not generalize uniformly. source: §4.1 evidence: LLaSA showed a larger relative quality drop after 50% layer pruning (speaker similarity −0.045, UTMOS −0.04) than CosyVoice 2 at the same pruning ratio, attributed to LLaSA's WLI values being more uniformly high across layers, indicating less exploitable redundancy. confidence: high relevance: high limitations: - The evaluation covers only two backbones (CosyVoice 2 and LLaSA-1B) and English-only test sets (LibriTTS test-clean, Seed-TTS test-en), so it is untested whether the WER-based pruning criterion and dynamic distillation scheme generalize to other LLM-TTS architectures, multilingual settings, or streaming inference. The more aggressive pruning configurations (9-layer CosyVoice 2) trade a larger, unquantified increase in WER for additional speed and memory gains, and the paper does not characterize where this trade-off becomes unacceptable for deployment. Computing WLI itself requires running WER evaluation over a data subset for each candidate layer removal, adding an upfront cost to the pruning procedure that is not reported in wall-clock terms. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.22727' published_date: "2025-09-25" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - nonautoregressive_flow_matching claims: - claim_id: diamoetts_joint_training_induces_style_averaging role: supports claim: Joint training across multiple dialects induces style averaging, where dialect-specific prosodic identity is diluted unless the model has a mechanism to route dialect-specific information separately. source: §4.4.2, Table 3 evidence: Ablating the dialect-aware MoE module (removing it while keeping the IPA frontend) degrades both MOS (2.46 vs. 3.15 on Xi'an) and WER (45.08% vs. 33.00% on Xi'an) relative to the full model. confidence: high relevance: medium - claim_id: diamoetts_ipa_resolves_g2p_ambiguity_better_than_pinyin role: supports claim: A standardized phonetic representation (IPA) resolves grapheme-to-phoneme ambiguity better than orthography-based inputs (e.g., pinyin) when a single model must serve multiple dialects with divergent character-to-sound mappings. source: §4.4.2, Table 3 evidence: Replacing the IPA frontend with pinyin input (keeping the MoE module) collapses MOS to roughly 1.0-1.2 and raises WER to roughly 90-93% across four dialects. confidence: high relevance: medium - claim_id: diamoetts_peft_extends_to_new_lowresource_dialect role: supports claim: Parameter-efficient fine-tuning (LoRA plus lightweight adapters) on a frozen multidialect backbone can extend zero/few-shot speech synthesis to a new, extremely low-resource dialect using only a few hours of data. source: §3.3, §4.1, Table 2 evidence: Stage 3 adapts the frozen Stage-2 model to Peking Opera (Jingbai/Yunbai) and Nanjing dialect using only ~3 hours of data per target style, training only LoRA and Conditioning Adapters while the flow-matching backbone stays frozen. confidence: medium relevance: medium - claim_id: diamoetts_open_data_underperforms_proprietary_on_wer role: complicates claim: Open, modest-scale multidialect training data can substantially underperform proprietary, large-scale commercial systems on standard intelligibility metrics even when subjective naturalness scores are comparable. source: §4.4.1, Table 1, Table 2 evidence: Trained on ~1.1k total hours versus CosyVoice2's ~150k hours and Qwen-TTS's ~3,000k hours, DiaMoE-TTS's WER trails the baseline average by wide margins in several dialects even where its UTMOSv2 score is on par with or better than the commercial average. confidence: high relevance: medium limitations: - Word error rate is substantially worse than commercial baselines across nearly every dialect evaluated, and the paper does not report a controlled comparison isolating whether this gap stems from training data scale, IPA/G2P errors, or the MoE routing itself. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.22062' published_date: "2025-09-26" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - autoregressive_speech_token_lm claims: - claim_id: injecting_explicit_linguistic_structure_into_the_primary_codebook role: supports claim: Injecting explicit linguistic structure into the primary codebook of a neural speech codec reduces the downstream language model's learning burden and improves synthesis intelligibility. source: §4.3, Table 4 evidence: Removing the semantic distillation loss during codec-conditioned TTS training raises WER from 3.31% to 3.97% on SeedTTS-test, 9.74% to 11.83% on PGC-Hard, and 16.57% to 18.34% on PGC-Poly, with SIM and UTMOS also degrading. confidence: high relevance: low - claim_id: an_automatic_speech_recognition_model_can_serve_as role: supports claim: An automatic speech recognition model can serve as an effective semantic teacher for codec distillation, as an alternative to self-supervised speech representation models. source: §E.2, Table 5 evidence: S3Codec distills Whisper encoder embeddings (rather than HuBERT/SSL features) into the first RVQ level; a small model trained with S3Codec reaches 3.30% WER on SeedTTS-test vs. 4.21% for an otherwise identical model using undistilled DAC tokens. confidence: high relevance: low - claim_id: fully_autoregressive_tts_systems_that_omit_an_explicit role: complicates claim: Fully autoregressive TTS systems that omit an explicit continuous acoustic-feature conditioning stage (e.g., mel-spectrogram or speaker-similarity-vector guidance) tend to underperform hybrid AR+NAR or flow-matching systems on speaker similarity even when intelligibility is competitive. source: §4.2, Table 2-3 evidence: CaT-TTS reports SIM of 0.668-0.678 across test sets versus 0.71-0.80 for Seed-TTS, CosyVoice 2/3, and F5-TTS, despite comparable or better WER among AR-only baselines. confidence: high relevance: high - claim_id: test_time_parallel_decoding_with_learned_input_dependent role: refines claim: Test-time parallel decoding with learned, input-dependent aggregation weights can reduce autoregressive error accumulation at near-zero added latency, but the achievable robustness gain is bounded by how many parallel streams are used, trading GPU utilization against benefit. source: §4.3, §3.3, Figure 3-4 evidence: MAPI ablation across increasing parallel-stream counts shows WER improving and becoming more stable across 10 repeated inferences per sample, while the authors note GPU resource utilization rises correspondingly and stream count must be tuned per deployment scenario. confidence: high relevance: low limitations: - Training relies on an unreleased proprietary corpus (~200k hours, ~85% Chinese / ~15% English), and neither code nor a demo is available, which limits independent verification of the reported results. - The semantic-distillation ablation (removing the loss) and the MAPI ablation are both run on smaller sub-datasets and reduced-size "CaT-TTS-small" models rather than the full 0.4B system, so it is not established that the same magnitude of gains transfers to the full-scale model. Speaker similarity remains a clear weak point relative to hybrid and NAR baselines, which the authors attribute to the deliberate absence of continuous acoustic conditioning rather than treat as a target for improvement. The evaluation is also dominated by Chinese-language and Chinese-out-of-domain test sets (PGC-Hard, PGC-Poly, Seed-TTS test-zh/test-hard), with comparatively less English-language evidence. caveats: - For zero-shot TTS, this paper provides medium-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.22167' published_date: "2025-09-26" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - codec architecture: - VAE relevance: high evidence_role: - core_evidence - architecture_variant - control_evidence current_role: active_evidence method_family: - continuous_latent_vae claims: - claim_id: regularizing_a_continuous_vae_latent_space_toward_semantic_structure_derived role: supports claim: Regularizing a continuous VAE latent space toward semantic structure derived from self-supervised speech features mitigates the reconstruction-generation trade-off in latent-based non-autoregressive TTS. source: §4.1, Table 1 evidence: At a fixed 64-dimensional latent size, adding cosine-similarity alignment to WavLM features reduces WER from 2.65% (vanilla VAE) to 2.10% while raising speaker similarity from 0.59 to 0.64 on LibriSpeech-PC test-clean. confidence: high relevance: high - claim_id: semantic_alignment_regularization_of_the_generation_target_accelerates_training_convergence role: supports claim: Semantic alignment regularization of the generation target accelerates training convergence of latent diffusion/flow-matching TTS models. source: §4.1, Figure 3 evidence: Training-step comparisons show Semantic-VAE-based F5-TTS reaching lower WER and higher SIM than both the vanilla-VAE and mel-spectrogram baselines at the same number of training steps. confidence: high relevance: medium - claim_id: the_intelligibility_versus_speaker_similarity_trade_off_in_vae_latent role: refines claim: The intelligibility-versus-speaker-similarity trade-off in VAE latent representations is governed by which layer of a self-supervised model is used for semantic supervision, not eliminated outright by adding semantic alignment. source: §4.3, Table 3 evidence: Ablating SSL layer choice shows the final layer of WavLM/HuBERT substantially degrades SIM (0.50-0.58) despite comparable WER, while an intermediate layer (WavLM layer 23) gives the best joint WER/SIM balance. confidence: high relevance: low - claim_id: cosine_similarity_alignment_losses_to_self_supervised_speech_features_preserve role: supports claim: Cosine-similarity alignment losses to self-supervised speech features preserve useful semantic structure in a latent space more effectively than L1 or L2 distance losses to the same features. source: §4.3, Table 3 evidence: Negative cosine alignment achieves 2.10% WER / 0.64 SIM, compared to 3.12% WER / 0.47 SIM for L1 alignment and 4.37% WER / 0.48 SIM for L2 alignment under otherwise identical settings. confidence: high relevance: low - claim_id: adding_a_semantic_regularization_objective_to_a_vae_s_training role: complicates claim: Adding a semantic regularization objective to a VAE's training loss does not necessarily degrade the representation's reconstruction fidelity, contrary to the general expectation that additional regularization terms cost reconstruction quality. source: §4.2, Table 2 evidence: Semantic-VAE reconstruction metrics on LibriTTS test-other (PESQ 3.74, STOI 0.96, UTMOS 3.56) are nearly identical to an unregularized vanilla VAE of the same architecture (PESQ 3.75, STOI 0.97, UTMOS 3.57). confidence: high relevance: medium limitations: - The high-resource baseline numbers in Table 1 (CosyVoice, FireRedTTS, E2 TTS, F5-TTS at 100k-580k training hours) are quoted directly from their original papers rather than re-run under matched data and compute, so any comparison against Semantic-VAE's low-resource (0.6k-hour) results is illustrative context only, not a controlled comparison. The method is validated on English speech only, at a single latent configuration (64 dimensions, 40Hz), and on two NAR backbones (F5-TTS, E2 TTS); generalization to AR codec-token TTS systems, other languages, or other latent dimensionalities/frame rates is untested. The paper also does not report inference latency or discuss how the added SSL model changes deployment cost of the VAE training pipeline (inference itself is unaffected, since the SSL branch is training-only). caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.24650' published_date: "2025-09-29" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - TTS - VC architecture: - autoregressive-LM - diffusion relevance: high evidence_role: - core_evidence - architecture_variant - control_evidence current_role: active_evidence method_family: - hybrid_ar_diffusion_acoustic claims: - claim_id: a_differentiable_scalar_quantization_bottleneck_applied_to_hidden_states_rather role: supports claim: A differentiable scalar-quantization bottleneck applied to hidden states, rather than used as a discrete prediction target, can induce semantic/acoustic task separation inside a continuous autoregressive TTS model without an external speech tokenizer. source: §4.3, Table 6 evidence: Removing the FSQ bottleneck from an otherwise identical hierarchical architecture increases ZH-hard-case CER from 18.19% to 24.92%, while FSQ dimensionality shows a non-monotonic optimum around 128-256 dimensions rather than monotonic improvement with capacity. confidence: high relevance: medium - claim_id: explicitly_separating_acoustic_detail_recovery_into_a_dedicated_residual_module role: supports claim: Explicitly separating acoustic detail recovery into a dedicated residual module improves robustness on complex inputs beyond what a single semantic language model plus a diffusion decoder achieves. source: §4.4, Table 7 evidence: Removing the RALM (TSLM output feeding the diffusion decoder directly, architecturally close to DiTAR) degrades EN-WER from 2.98% to 4.34% and ZH-hard-case CER from 18.19% to 25.0%; removing the historical acoustic embedding from the RALM input degrades results further. confidence: high relevance: medium - claim_id: learning_rate_schedule_design_not_just_architecture_materially_affects_zero role: complicates claim: Learning-rate schedule design, not just architecture, materially affects zero-shot speaker similarity in large-scale continuous TTS training. source: §4.5, Table 8 evidence: A two-phase Warmup-Stable-Decay schedule's decay phase alone improves ZH-hard-case CER from 13.22% to 8.87% and SIM by 4.4 points over the stable-phase-only checkpoint on an otherwise identical model. confidence: high relevance: high - claim_id: classifier_free_guidance_strength_in_diffusion_based_tts_decoders_trades role: complicates claim: Classifier-free guidance strength in diffusion-based TTS decoders trades off intelligibility against speaker similarity non-monotonically, with both very low and very high guidance scales degrading both metrics simultaneously. source: §4.6, Table 9 evidence: CFG scale 1.0 (no guidance) yields EN-WER 16.32% and SIM 55.1%, while scale 5.0 yields EN-WER 12.78% and SIM 60.7%; the optimum at scale 2.0 achieves EN-WER 1.85% and SIM 72.9%, with degradation on both sides of the optimum. confidence: high relevance: high - claim_id: removing_dependency_on_a_pre_trained_discrete_speech_tokenizer_does role: refines claim: Removing dependency on a pre-trained discrete speech tokenizer does not require sacrificing zero-shot voice cloning quality relative to discrete-token-based open-source TTS systems. source: §4.2, Table 3 evidence: On SEED-TTS-EVAL, the fully continuous VoxCPM reports SIM of 72.9% (EN) and 77.2% (ZH), exceeding the discrete-token-based IndexTTS2 and CosyVoice2 baselines on the same benchmark. confidence: high relevance: high limitations: - Multilingual capability is limited to Chinese and English by construction; the paper explicitly reports uncertain generalization to other languages, and prosody/emotion control lacks any intuitive or precise user-facing conditioning mechanism. The causal audio VAE operates at 16kHz, which the authors acknowledge falls short of the 24kHz or 44.1kHz sampling rates typically expected for high-fidelity applications. Baseline comparisons draw on official implementations or numbers reported in prior papers rather than a uniformly controlled re-evaluation, so cross-system rankings on tables that mix reproduced and self-reported numbers should be read cautiously. The training corpus (1.8M hours) is internal and not released, which limits independent reproduction of the full-scale result even though code and weights for the trained model are public. caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. - id: '2509.25131' published_date: "2025-09-29" entry_date: '2026-07-26' year: 2025 venue: arXiv task: - SCA - TTS architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - acceleration_evidence - control_evidence current_role: active_evidence method_family: - hybrid_ar_semantic_nar_acoustic claims: - claim_id: chunking_text_into_aligned_segments_with_a_short_token_delay role: supports claim: Chunking text into aligned segments with a short token-delay before speech decoding reduces error accumulation in long-form autoregressive speech generation. source: §4.2, Table 6 evidence: Removing chunk-based decoding raises Long-TTS-Eval error rates above those of concurrent long-form TTS baselines, and with it enabled MGM-Omni-TTS-2B achieves EN-hard WER 26.26 versus 42.48-98.61 for CosyVoice2, MOSS-TTSD-v0.5, and Higgs-Audio-v2. confidence: high relevance: high - claim_id: multi_token_parallel_decoding_is_not_restricted_to_rvq_speech role: supports claim: Multi-token parallel decoding is not restricted to RVQ speech tokenizers and can be applied effectively to finite scalar quantization (FSQ) tokenizers. source: §4.2, Table 6 evidence: Increasing parallel decoding size on the CosyVoice2 FSQ tokenizer maintains TTS quality on Seed-TTS-Eval while cutting inference RTF by roughly 3x at parallel size 4. confidence: high relevance: medium - claim_id: increasing_the_parallel_decoding_size_trades_off_synthesis_error_rate role: complicates claim: Increasing the parallel decoding size trades off synthesis error rate against inference speed rather than improving both simultaneously. source: §4.2 evidence: Larger parallel sizes in the ablation slightly raise audio error rate even as they substantially accelerate inference, leading the authors to select a parallel size of 4 as a balance point. confidence: high relevance: low - claim_id: separating_multimodal_reasoning_from_speech_synthesis_into_distinct_model_components role: refines claim: Separating multimodal reasoning from speech synthesis into distinct model components can improve long-form audio understanding without sacrificing speech generation efficiency. source: §4.1.1, §4.1.3, Figure 5, Table 5b evidence: The dual-track brain-mouth design lets the MLLM handle needle-in-the-haystack audio inputs up to 4,500 seconds while the SpeechLM independently achieves the lowest RTF among compared long-form TTS systems. confidence: high relevance: medium limitations: - 'The long-form evaluation itself is partly self-authored: Long-TTS-Eval is introduced by this paper, and while its construction and normalized-text scoring procedure are documented, results on it cannot yet be cross-checked against independent replications. The comparison in Table 5b is limited to three baseline systems, and the qualitative long-speech examples in the appendix (a classical Chinese poem and a code-switched English-Chinese poem) are illustrative rather than a systematic error analysis. The paper does not report results on emotion or prosody control, nor does it evaluate robustness to reference audio recorded in noisy or far-field conditions. The 32B MLLM variant''s long-form and vision-speech results are mixed relative to the 7B variant (e.g., lower TextVQA-Speech and EN-hard performance context is not directly reported for 32B TTS), suggesting scaling benefits are not uniform across all sub-tasks.' caveats: - For zero-shot TTS, this paper provides high-relevance evidence; claims primarily concerning other concepts remain included but are marked low relevance. claim_clusters: - id: reference_conditioning_preserves_speaker_identity claim: Reference-conditioned models can preserve unseen-speaker identity without speaker-specific fine-tuning. status: strongly_supported confidence: high supporting_papers: - '2010.05646' - 2025.acl-demo.37 - 2025.acl-industry.42 - 2025.acl-long.1043 - 2025.ccl-1.77 - 2025.naacl-short.65 - '2209.03143' - '2301.02111' - '2303.03926' - '2304.09116' - '2305.07243' - '2308.16692' - '2312.01479' - '2401.07333' - '2406.04904' - '2406.05370' - '2406.07855' - '2407.05407' - '2411.09943' - '2411.19770' - '2412.04724' - '2412.10117' - '2412.15649' - '2412.16846' - '2502.05512' - '2503.01710' - '2503.11026' - '2504.02407' - '2505.07916' - '2505.17589' - '2506.04077' - '2506.18296' - '2507.02380' - '2507.04349' - '2507.07799' - '2507.08319' - '2507.15272' - '2507.20140' - '2508.02038' - '2508.04141' - '2508.04585' - '2508.07302' - '2508.17031' - '2509.07376' - '2509.12171' - '2509.12831' - '2509.13068' - '2509.15462' - '2509.15845' - '2509.19231' - interspeech-2025-0305 - interspeech-2025-0319 - interspeech-2025-0455 - interspeech-2025-0596 - interspeech-2025-0656 - interspeech-2025-0787 - interspeech-2025-1101 - interspeech-2025-1192 - interspeech-2025-1684 - interspeech-2025-1726 - interspeech-2025-1779 - interspeech-2025-2815 contradicting_papers: [] refining_papers: - 2025.acl-long.346 - 2025.ccl-1.77 - 2025.naacl-long.242 - 2025.naacl-short.65 - '2409.09098' - '2411.19770' - '2412.16846' - '2503.11026' - '2505.07916' - '2505.17589' - '2506.04077' - '2506.09874' - '2507.09282' - '2507.10985' - '2507.12197' - '2507.21138' - '2509.05863' - '2509.07376' - '2509.09631' - '2509.15845' - interspeech-2025-0305 - interspeech-2025-0656 - interspeech-2025-2595 - interspeech-2025-2787 caveats: - Reported gains depend strongly on the reference distribution, speaker encoder, and whether comparisons use matched data and evaluation protocols. last_reviewed: '2026-07-26' - id: prompt_quality_and_duration_bound_cloning claim: Reference quality, duration, and selection materially bound zero-shot cloning fidelity. status: strongly_supported confidence: high supporting_papers: - 2025.acl-long.1043 - '2301.02111' - '2304.09116' - '2406.00654' - '2406.04904' - '2409.00750' - '2409.03283' - '2411.19770' - '2504.02407' - '2505.07916' - '2505.17589' - '2508.09702' - '2509.12831' - '2509.18928' - iclr-2025-cuFzE8Jlvb - interspeech-2025-0596 - interspeech-2025-2787 contradicting_papers: [] refining_papers: - 2025.findings-naacl.279 - 2025.naacl-short.65 - '2411.19770' - '2508.16332' - '2509.19928' - iclr-2025-cuFzE8Jlvb caveats: - Studies use different prompt durations and corruption conditions, so a universal minimum reference length is not established. last_reviewed: '2026-07-26' - id: cross_lingual_cloning_faces_language_leakage claim: Cross-lingual voice cloning remains vulnerable to language, accent, and prosody leakage from the reference. status: contested confidence: medium supporting_papers: - 2025.acl-long.346 - 2025.acl-long.598 - 2025.acl-long.65 - 2025.acl-long.654 - 2025.acl-short.81 - 2025.coling-main.685 - 2025.findings-naacl.279 - 2025.naacl-srw.6 - '2209.03143' - '2301.02111' - '2303.03926' - '2305.07243' - '2306.12925' - '2308.16692' - '2310.00704' - '2312.01479' - '2402.01912' - '2402.08093' - '2402.13236' - '2403.16973' - '2406.02430' - '2406.04904' - '2407.05407' - '2407.08551' - '2409.03283' - '2412.15649' - '2503.11026' - '2505.07916' - '2505.17589' - '2506.21619' - '2507.07799' - '2507.15272' - '2507.21138' - '2508.07302' - '2508.09702' - '2509.05863' - '2509.11425' - '2509.14579' - '2509.15845' - '2509.17516' - '2509.19883' - interspeech-2025-0406 - interspeech-2025-0551 - interspeech-2025-1192 - interspeech-2025-1344 - interspeech-2025-raju25_interspeech contradicting_papers: - '2509.19928' refining_papers: - 2025.findings-acl.687 - 2025.naacl-demo.12 - '2407.05361' - '2507.02176' - '2507.16875' - '2509.05863' - '2509.12831' - '2509.15253' - '2509.17765' - '2509.17988' - '2509.19668' - interspeech-2025-0305 - interspeech-2025-0815 - interspeech-2025-1531 caveats: - Multilingual coverage and language-pair difficulty vary widely; one reported contradiction prevents treating leakage as universal. last_reviewed: '2026-07-26' - id: disentangling_content_speaker_and_prosody_improves_control claim: Separating linguistic content, speaker identity, and prosody improves controllability and speaker transfer. status: strongly_supported confidence: high supporting_papers: - 2025.acl-demo.37 - 2025.acl-long.346 - 2025.acl-long.790 - 2025.naacl-long.242 - '2209.03143' - '2308.16692' - '2402.08093' - '2403.03100' - '2406.02430' - '2407.05407' - '2407.08551' - '2411.09943' - '2412.04724' - '2412.10117' - '2503.01710' - '2505.07916' - '2506.21619' - '2507.01348' - '2507.02176' - '2507.15272' - '2508.02038' - '2508.04141' - '2508.04195' - '2508.07426' - '2508.19205' - '2509.13068' - '2509.14684' - '2509.15462' - '2509.18928' - '2509.19231' - '2509.19592' - '2509.19883' - '2509.22727' - interspeech-2025-0455 - interspeech-2025-0464 - interspeech-2025-1081 - interspeech-2025-1101 - interspeech-2025-1115 - interspeech-2025-1192 - interspeech-2025-1229 - interspeech-2025-1440 - interspeech-2025-1531 - interspeech-2025-1638 - interspeech-2025-1684 contradicting_papers: [] refining_papers: - 2025.acl-long.1043 - '2411.19770' - '2502.04128' - '2502.07243' - '2506.09874' - '2507.02176' - '2508.04996' - '2509.15253' - iclr-2025-hQvX9MBowC - interspeech-2025-0305 - interspeech-2025-0575 - interspeech-2025-0815 - interspeech-2025-1101 - interspeech-2025-1531 - interspeech-2025-2787 caveats: - Representation probes and downstream similarity scores do not always demonstrate causal disentanglement. last_reviewed: '2026-07-26' - id: semantic_codec_tokens_improve_intelligibility claim: Speech tokens with explicit semantic structure improve intelligibility in codec-based zero-shot TTS. status: strongly_supported confidence: high supporting_papers: - 2025.findings-acl.115 - '2209.03143' - '2305.09636' - '2308.16692' - '2402.08093' - '2402.13236' - '2407.05407' - '2409.00750' - '2409.03283' - '2412.10117' - '2412.15649' - '2501.06282' - '2503.01710' - '2505.17589' - '2508.04141' - '2508.04195' - '2508.19205' - '2509.11425' - '2509.13068' - '2509.15462' - '2509.22167' - '2509.24650' - interspeech-2025-1229 contradicting_papers: [] refining_papers: - '2502.07243' - '2507.21138' - '2509.15462' caveats: - Semantic supervision can remove acoustic detail, so intelligibility gains need joint speaker-fidelity evaluation. last_reviewed: '2026-07-26' - id: continuous_targets_support_high_fidelity_cloning claim: Continuous acoustic targets can support high-fidelity cloning without relying on discrete codec prediction. status: strongly_supported confidence: high supporting_papers: - 2025.acl-long.65 - '2301.02111' - '2312.15821' - '2406.05551' - '2407.08551' - '2502.11128' - '2502.18924' - '2503.11026' - '2507.15272' - '2508.07302' - '2508.19205' - '2509.14684' - '2509.14784' - iclr-2025-cuFzE8Jlvb - interspeech-2025-0455 - interspeech-2025-1779 contradicting_papers: [] refining_papers: - '2502.11128' - '2507.15272' - '2509.08379' - '2509.14579' - '2509.14684' - '2509.18928' - '2509.19883' - '2509.22062' - '2509.24650' - interspeech-2025-1236 caveats: - Continuous systems differ in latent construction and decoder cost; evidence does not establish one continuous target as generally best. last_reviewed: '2026-07-26' - id: hybrid_semantic_acoustic_generation_balances_quality claim: Separating semantic planning from acoustic reconstruction can balance intelligibility and speaker fidelity. status: strongly_supported confidence: high supporting_papers: - '2209.03143' - '2301.02111' - '2303.03926' - '2308.16692' - '2406.05551' - '2412.10117' - '2501.06282' - '2502.07243' - '2502.11128' - '2503.01710' - '2505.13000' - '2505.17589' - '2508.19205' - '2509.13068' - '2509.15969' - '2509.17021' - '2509.19928' - '2509.24650' - interspeech-2025-1115 contradicting_papers: [] refining_papers: - 2025.acl-long.1043 - '2509.22062' caveats: - Additional stages introduce interface errors and make gains difficult to attribute without matched ablations. last_reviewed: '2026-07-26' - id: data_scale_and_diversity_improve_generalization claim: Broader speaker and acoustic diversity in training data improves zero-shot generalization. status: strongly_supported confidence: high supporting_papers: - 2025.acl-long.654 - 2025.acl-short.81 - 2025.coling-main.685 - 2025.findings-naacl.279 - 2025.naacl-short.65 - '2301.02111' - '2303.03926' - '2305.07243' - '2310.00704' - '2312.01479' - '2312.15821' - '2402.01912' - '2402.08093' - '2403.03100' - '2406.02430' - '2406.04904' - '2407.05361' - '2409.03283' - '2411.19770' - '2502.18924' - '2504.02407' - '2505.17589' - '2507.09282' - '2508.07426' - '2509.14784' - '2509.15969' - '2509.17765' - interspeech-2025-0455 - interspeech-2025-1101 - interspeech-2025-1344 - interspeech-2025-1726 - interspeech-2025-raju25_interspeech contradicting_papers: [] refining_papers: - 2025.findings-naacl.279 - '2312.01479' - '2409.09098' - '2507.01348' - '2508.07426' - '2509.22727' - interspeech-2025-0406 - interspeech-2025-0656 - interspeech-2025-1115 caveats: - Many large-scale results use proprietary corpora, and scale is often confounded with architecture and filtering quality. last_reviewed: '2026-07-26' - id: low_resource_adaptation_can_preserve_voice_cloning claim: Parameter-efficient and low-resource adaptation can improve cloning with limited target-speaker data. status: strongly_supported confidence: high supporting_papers: - 2025.acl-industry.42 - 2025.acl-short.81 - 2025.findings-naacl.279 - '2409.03283' - '2505.07916' - '2507.15272' - '2507.16875' - '2508.07426' - '2509.22727' - interspeech-2025-1344 contradicting_papers: [] refining_papers: - 2025.acl-long.1043 - 2025.findings-naacl.279 - '2509.19883' caveats: - The boundary between zero-shot inference and few-shot adaptation differs across studies and should not be collapsed in comparisons. last_reviewed: '2026-07-26' - id: robustness_requires_in_the_wild_prompt_coverage claim: Robust voice cloning requires training and evaluation coverage for noisy and out-of-domain references. status: strongly_supported confidence: high supporting_papers: - 2025.acl-demo.37 - 2025.acl-long.1043 - 2025.acl-long.346 - '2301.02111' - '2312.15821' - '2402.01912' - '2406.02430' - '2406.04904' - '2406.05370' - '2409.03283' - '2411.19770' - '2412.10117' - '2504.02407' - '2505.17589' - '2508.09702' - '2509.12831' - '2509.14684' - '2509.15462' - interspeech-2025-0319 - interspeech-2025-0596 - interspeech-2025-0787 - interspeech-2025-1101 - interspeech-2025-2787 contradicting_papers: [] refining_papers: - '2411.19770' - '2506.04077' - '2509.08379' - '2509.14684' - '2509.15845' - interspeech-2025-0656 - interspeech-2025-2787 caveats: - Robustness tests are heterogeneous and often omit far-field, channel-mismatch, or adversarial prompt conditions. last_reviewed: '2026-07-26' - id: automatic_metrics_do_not_fully_capture_cloning_quality claim: Automatic intelligibility and speaker-similarity metrics do not fully capture perceived cloning quality. status: contested confidence: medium supporting_papers: - 2025.acl-industry.42 - '2312.15821' - '2406.02430' - '2406.05370' - '2406.18009' - '2503.01710' - '2503.14345' - '2504.20581' - '2505.07916' - '2505.17589' - '2506.09874' - '2506.18296' - '2507.14534' - '2507.16875' - '2507.20140' - '2507.21138' - '2508.19098' - '2509.08379' - '2509.17021' - interspeech-2025-0047 - interspeech-2025-1115 - interspeech-2025-1229 - interspeech-2025-1440 contradicting_papers: - '2509.19928' refining_papers: - 2025.acl-long.346 - 2025.icnlsp-1.34 - 2025.naacl-long.242 - 2025.naacl-srw.6 - '2206.04658' - '2409.09098' - '2412.16846' - '2502.11128' - '2505.07916' - '2507.12197' - '2507.16875' - '2508.07426' - '2509.05863' - '2509.09631' - '2509.19668' - '2509.22718' - '2509.24650' - interspeech-2025-1081 - interspeech-2025-1084 - interspeech-2025-1229 - interspeech-2025-1236 - interspeech-2025-1531 - interspeech-2025-1726 caveats: - Metric behavior depends on the ASR and speaker encoder; contradictory evidence shows that disagreement with listeners is not uniform. last_reviewed: '2026-07-26' - id: intelligibility_and_speaker_similarity_trade_off claim: Zero-shot TTS frequently trades intelligibility against speaker similarity rather than improving both monotonically. status: strongly_supported confidence: high supporting_papers: - 2025.acl-long.1498 - 2025.acl-long.313 - 2025.acl-long.598 - 2025.acl-long.65 - 2025.acl-long.654 - 2025.acl-long.790 - 2025.naacl-srw.6 - '2303.03926' - '2312.15821' - '2401.07333' - '2403.03100' - '2406.02430' - '2406.04904' - '2406.05551' - '2406.18009' - '2407.05361' - '2407.08551' - '2411.09943' - '2412.16846' - '2502.04128' - '2502.07243' - '2502.11128' - '2502.18924' - '2503.01710' - '2503.14345' - '2504.02407' - '2505.07916' - '2505.13000' - '2507.03887' - '2507.04349' - '2507.14988' - '2507.20140' - '2507.21138' - '2507.22746' - '2508.03543' - '2508.04141' - '2508.04195' - '2508.06262' - '2508.09702' - '2508.16332' - '2508.19098' - '2508.19205' - '2509.13068' - '2509.15462' - '2509.15969' - '2509.17021' - '2509.18928' - '2509.19231' - '2509.19592' - '2509.19668' - '2509.19883' - '2509.20802' - '2509.22167' - '2509.22727' - iclr-2025-cuFzE8Jlvb - interspeech-2025-0319 - interspeech-2025-0455 - interspeech-2025-0464 - interspeech-2025-0596 - interspeech-2025-0989 - interspeech-2025-1081 - interspeech-2025-1236 - interspeech-2025-1344 - interspeech-2025-1440 - interspeech-2025-1536 - interspeech-2025-1726 - interspeech-2025-1779 - interspeech-2025-2447 contradicting_papers: [] refining_papers: - 2025.icnlsp-1.34 - 2025.naacl-short.65 - 2025.naacl-srw.6 - '2407.05361' - '2409.09098' - '2502.04128' - '2502.11128' - '2505.07916' - '2507.02380' - '2507.09282' - '2507.15272' - '2507.16875' - '2508.16332' - '2509.05863' - '2509.07376' - '2509.08379' - '2509.13068' - '2509.14784' - '2509.15462' - '2509.15845' - '2509.17516' - '2509.19231' - '2509.19668' - '2509.19883' - '2509.22062' - '2509.24650' - iclr-2025-868masI331 - iclr-2025-cuFzE8Jlvb - iclr-2025-hQvX9MBowC - interspeech-2025-0989 - interspeech-2025-1081 - interspeech-2025-1192 - interspeech-2025-1229 - interspeech-2025-1236 - interspeech-2025-2595 - interspeech-2025-2787 caveats: - Apparent trade-offs can reflect metric choice, guidance settings, or unmatched baselines rather than an architectural law. last_reviewed: '2026-07-26' - id: inference_acceleration_preserves_usable_quality claim: Decoding acceleration can reduce zero-shot TTS latency while retaining usable intelligibility and speaker fidelity. status: strongly_supported confidence: high supporting_papers: - 2025.acl-demo.37 - 2025.acl-long.313 - 2025.naacl-srw.6 - '2402.08093' - '2406.02430' - '2412.04724' - '2502.18924' - '2507.14534' - '2508.19098' - '2509.08379' - '2509.09631' - '2509.15969' - '2509.17765' - '2509.25131' - interspeech-2025-1081 - interspeech-2025-1779 - interspeech-2025-2447 contradicting_papers: [] refining_papers: - '2507.21138' - '2509.09748' - '2509.12831' - '2509.15462' - '2509.25131' caveats: - Speed claims use different hardware, solvers, and latency definitions; aggressive acceleration can still degrade hard-case quality. last_reviewed: '2026-07-26' - id: explicit_style_and_prosody_control_extends_cloning claim: Explicit style, emotion, and prosody conditioning extends zero-shot cloning beyond timbre transfer. status: strongly_supported confidence: high supporting_papers: - 2025.acl-industry.42 - 2025.acl-long.346 - 2025.acl-long.790 - 2025.findings-acl.1226 - 2025.findings-acl.687 - 2025.naacl-long.242 - '2209.03143' - '2305.07243' - '2312.01479' - '2404.03204' - '2406.02430' - '2406.07855' - '2407.05407' - '2409.00750' - '2409.03283' - '2412.04724' - '2501.06282' - '2502.07243' - '2503.11026' - '2505.07916' - '2506.21619' - '2507.04349' - '2508.02038' - '2508.03543' - '2508.07426' - '2508.17031' - '2509.17516' - '2509.22718' - '2509.22727' - interspeech-2025-0464 - interspeech-2025-1101 - interspeech-2025-1115 - interspeech-2025-1192 - interspeech-2025-1638 - interspeech-2025-1684 contradicting_papers: [] refining_papers: - 2025.acl-long.346 - 2025.findings-acl.687 - '2502.07243' - '2503.11026' - '2505.17589' - '2507.20140' - '2507.21138' - '2508.16332' - '2509.14579' - '2509.15253' - interspeech-2025-0575 - interspeech-2025-1115 - interspeech-2025-1638 caveats: - Control fidelity and speaker preservation are rarely evaluated jointly across the same prompt and language conditions. last_reviewed: '2026-07-26' - id: long_form_generation_needs_alignment_and_memory_controls claim: Long-form zero-shot generation benefits from explicit alignment, chunking, or memory mechanisms that limit error accumulation. status: strongly_supported confidence: high supporting_papers: - 2025.acl-short.81 - 2025.findings-acl.1226 - '2301.02111' - '2303.03926' - '2304.09116' - '2312.15821' - '2409.03283' - '2411.09943' - '2502.07243' - '2503.14345' - '2506.18296' - '2508.04585' - '2508.17031' - '2508.19205' - '2509.11425' - '2509.17516' - '2509.25131' - interspeech-2025-0319 - interspeech-2025-0464 - interspeech-2025-1229 contradicting_papers: [] refining_papers: - '2507.20140' - '2509.25131' - iclr-2025-868masI331 - interspeech-2025-1726 caveats: - Long-form benchmarks are recent and partly system-specific, limiting independent comparison of failure rates. last_reviewed: '2026-07-26' - id: text_and_pronunciation_modeling_improve_intelligibility claim: Explicit text and pronunciation modeling improves intelligibility on difficult or multilingual inputs. status: strongly_supported confidence: high supporting_papers: - '1712.05884' - 2025.acl-long.313 - '2402.08093' - '2402.13236' - '2406.02430' - '2406.18009' - '2409.03283' - '2503.01710' - '2508.04195' - '2508.09702' - '2509.02020' - '2509.19231' - '2509.19668' - '2509.22718' - '2509.22727' - interspeech-2025-0596 - interspeech-2025-1081 - interspeech-2025-2031 - interspeech-2025-raju25_interspeech contradicting_papers: [] refining_papers: - 2025.acl-long.790 - 2025.findings-naacl.279 - '2505.07916' - '2507.16875' - '2507.20140' - '2508.16332' - '2509.14784' - '2509.19668' - iclr-2025-cuFzE8Jlvb - iclr-2025-hQvX9MBowC - interspeech-2025-0815 - interspeech-2025-2031 - interspeech-2025-2787 caveats: - Text-front-end gains may be language-specific and can trade flexibility against pronunciation consistency. last_reviewed: '2026-07-26' - id: speaker_encoder_choice_changes_reported_similarity claim: Speaker-encoder choice materially changes measured zero-shot similarity and system rankings. status: strongly_supported confidence: high supporting_papers: - 2025.naacl-short.65 - '2402.08093' - '2406.18009' - '2503.01710' - '2503.11026' - '2504.20581' - '2505.07916' - '2506.18296' - '2509.22167' - interspeech-2025-1192 - interspeech-2025-1440 - interspeech-2025-1684 contradicting_papers: [] refining_papers: - '2504.20581' - '2509.05863' - interspeech-2025-1229 caveats: - Scores from different embedding models are not directly comparable, and calibration against human identity judgments remains limited. last_reviewed: '2026-07-26' - id: synthetic_and_pseudo_labeled_data_expand_training claim: Synthetic and pseudo-labeled speech can expand zero-shot TTS training coverage. status: strongly_supported confidence: medium supporting_papers: - '2209.03143' - '2406.02430' - '2407.05407' - interspeech-2025-2595 contradicting_papers: [] refining_papers: - '2506.04077' - '2506.18296' - interspeech-2025-1536 caveats: - Evidence is limited and synthetic-data quality, diversity, and error reinforcement remain important confounds. last_reviewed: '2026-07-26' method_families: - id: autoregressive_speech_token_lm name: Autoregressive speech-token language models summary: Autoregressive language models predict discrete or semi-discrete speech representations conditioned on text and a reference voice. They offer flexible in-context cloning but inherit sequence-length, exposure-bias, and latency constraints. papers: - '2209.03143' - '2301.02111' - '2303.03926' - '2305.09636' - '2306.12925' - '2310.00704' - '2401.07333' - '2402.01912' - '2402.08093' - '2403.16973' - '2404.03204' - '2406.00654' - '2406.04904' - '2406.05370' - '2406.07855' - '2407.08551' - '2409.00750' - '2411.01156' - '2502.04128' - '2502.05512' - '2503.01710' - iclr-2025-868masI331 - iclr-2025-cuFzE8Jlvb - 2025.findings-naacl.279 - 2025.naacl-demo.12 - 2025.naacl-srw.6 - '2505.13000' - '2507.01348' - '2507.07799' - '2507.12197' - '2507.21138' - 2025.acl-long.1498 - 2025.acl-long.65 - 2025.acl-short.81 - 2025.findings-acl.115 - '2507.17527' - 2025.icnlsp-1.34 - '2504.10352' - '2508.04141' - '2508.06262' - interspeech-2025-0047 - interspeech-2025-0319 - interspeech-2025-0464 - interspeech-2025-0551 - interspeech-2025-0787 - interspeech-2025-0989 - interspeech-2025-1084 - interspeech-2025-1641 - interspeech-2025-1993 - interspeech-2025-2447 - interspeech-2025-2787 - '2508.15442' - '2509.02020' - '2509.05863' - '2506.04077' - '2509.11425' - '2509.13068' - '2412.16846' - '2509.15969' - '2509.17021' - '2509.17765' - '2509.19592' - '2509.20802' - '2509.22062' open_questions: - How far can decoding acceleration reduce latency without increasing long-form errors or weakening speaker identity? - id: hybrid_ar_semantic_nar_acoustic name: Autoregressive semantic planning with non-autoregressive acoustics summary: A semantic or codec-token language model performs autoregressive planning, while a flow-matching or related non-autoregressive stage restores acoustic detail. The split targets intelligibility and speaker fidelity with different objectives. papers: - '2407.05407' - '2409.03283' - '2412.10117' - '2412.15649' - '2501.06282' - '2502.07243' - '2503.14345' - '2505.07916' - '2505.17589' - '2507.02380' - 2025.acl-long.598 - '2508.02038' - '2508.14049' - '2508.04195' - '2508.04585' - '2508.07302' - '2508.16332' - '2508.19098' - '2502.11128' - '2506.21619' - '2509.04072' - '2509.15462' - '2509.15845' - '2509.17516' - '2509.25131' open_questions: - Which interface between semantic and acoustic stages best prevents information loss and cross-stage error propagation? - id: hybrid_ar_diffusion_acoustic name: Autoregressive planning with diffusion acoustics summary: An autoregressive module predicts linguistic or coarse acoustic structure and a diffusion decoder reconstructs detailed speech. These systems trade extra staging and sampling cost for acoustic flexibility. papers: - '2305.07243' - '2406.02430' - '2507.09282' - interspeech-2025-1115 - '2508.16790' - '2509.18928' - '2509.24650' open_questions: - Can diffusion refinement be reduced to consistently low step counts without eroding difficult-speaker similarity? - id: nonautoregressive_flow_matching name: Non-autoregressive flow-matching voice cloning summary: Flow-matching models generate mel, waveform, or latent acoustic targets without an autoregressive speech-token loop. Reference conditioning is typically delivered through prompting, masking, or learned speaker representations. papers: - '2312.15821' - '2406.18009' - '2412.04724' - '2504.02407' - '2506.13053' - '2507.03887' - '2507.04349' - '2506.09874' - '2507.14988' - '2507.16875' - 2025.acl-long.1043 - 2025.acl-long.313 - 2025.acl-long.790 - 2025.findings-acl.687 - '2507.20140' - '2503.11026' - '2508.03543' - '2508.04996' - interspeech-2025-0305 - interspeech-2025-0455 - interspeech-2025-1229 - interspeech-2025-1236 - interspeech-2025-1344 - interspeech-2025-1536 - interspeech-2025-1779 - interspeech-2025-2449 - interspeech-2025-raju25_interspeech - '2509.08379' - '2509.08696' - '2509.09631' - '2509.09748' - '2509.11084' - '2509.12171' - '2509.14579' - '2509.14684' - '2509.17988' - '2509.19668' - '2509.22727' open_questions: - How robust are alignment-free flow systems to noisy, very short, or cross-lingual references outside curated benchmarks? - id: diffusion_voice_cloning name: Diffusion-based voice cloning summary: Diffusion systems iteratively generate acoustic representations conditioned on text and a reference speaker. They provide expressive generation and flexible conditioning but face sampling-speed and guidance trade-offs. papers: - '2304.09116' - '2403.03100' - '2411.09943' - '2502.18924' - iclr-2025-hQvX9MBowC - 2025.naacl-long.242 - '2507.15272' - 2025.ccl-1.77 - interspeech-2025-1101 - '2411.19770' - '2509.19231' open_questions: - Which distillation and guidance schemes preserve speaker identity across domains at genuinely real-time latency? - id: encoder_decoder_voice_cloning name: Encoder-decoder voice cloning summary: Transformer encoder-decoder systems map text and speaker evidence to acoustic outputs using explicit attention or learned alignments. They remain important for adaptation, data efficiency, and controlled comparisons. papers: - '1712.05884' - '2409.09098' - '2507.08319' - 2025.acl-industry.42 - 2025.acl-long.346 - '2508.07426' - interspeech-2025-0596 - interspeech-2025-1192 - interspeech-2025-2031 - interspeech-2025-2189 - '2508.17031' open_questions: - When do explicit alignments remain preferable to alignment-free generative objectives for unseen speakers? - id: continuous_latent_vae name: Continuous-latent and VAE voice cloning summary: Continuous latent representations and VAEs compress speech without making discrete codec tokens the primary generation target. This can retain acoustic detail but creates representation and semantic-alignment trade-offs. papers: - '2312.01479' - 2025.acl-long.654 - interspeech-2025-0468 - interspeech-2025-0575 - interspeech-2025-0815 - interspeech-2025-1440 - interspeech-2025-1531 - '2509.07376' - '2509.22167' open_questions: - How should continuous latents balance semantic predictability, reconstruction fidelity, and speaker information? - id: gan_based_voice_synthesis name: GAN-based and adversarial voice synthesis summary: Adversarial generators and vocoder-integrated systems perform direct or tightly coupled acoustic synthesis. They provide low-latency baselines and representation evidence, though stability and coverage vary. papers: - '2010.05646' - '2206.04658' - '2308.16692' - 2025.naacl-short.65 - interspeech-2025-0406 - interspeech-2025-1081 - '2507.14534' - '2509.04685' - '2509.12831' open_questions: - Can adversarial objectives match newer generative models on diverse unseen speakers without sacrificing inference efficiency? - id: modular_hybrid_systems name: Modular hybrid voice-cloning systems summary: Hybrid systems combine multiple generators, encoders, adapters, or retrieval modules without a single dominant AR, flow, or diffusion pattern. Their modularity supports specialized controls but complicates attribution and deployment. papers: - '2406.05551' - 2025.acl-demo.37 - 2025.acl-long.1252 - 2025.findings-acl.1226 - '2507.22746' - interspeech-2025-0656 - interspeech-2025-1638 - interspeech-2025-1684 - interspeech-2025-1726 - '2508.19205' - '2509.14784' - '2509.19883' - '2509.22718' open_questions: - Which modules contribute independently to speaker generalization, and which mainly compensate for limitations elsewhere in the pipeline? reassessment_queue: [] open_questions: - Which reference-audio properties predict cloning success across languages, recording conditions, and speaker demographics? - Can evaluations separate perceived identity, acoustic similarity, intelligibility, and style transfer without relying on one speaker encoder? - Which representation and generation split best preserves speaker detail while remaining streamable and robust on long-form speech? - How much of recent zero-shot improvement comes from architecture rather than proprietary data scale and filtering? - What safeguards reliably prevent unauthorized cloning while preserving legitimate personalization and accessibility uses? trend_notes: - From 2023 through Q3 2025, the field shifted from encoder-decoder and GAN-centered cloning toward autoregressive speech-token models and hybrid autoregressive-plus-flow pipelines. - Flow matching became a major non-autoregressive acoustic generator in 2024–2025, both as a standalone cloning model and as the acoustic stage behind semantic token language models. - Recent systems increasingly target cross-lingual, expressive, streaming, and long-form cloning rather than treating short-form English speaker similarity as sufficient. - 'Evaluation remains fragmented: WER or CER and embedding similarity dominate automatic reporting, while prompt robustness and human identity judgments are less consistently standardized.' - Large proprietary multilingual corpora increasingly accompany frontier results, making data scale and curation difficult to separate from architectural gains.