concept: prosody-control last_updated: '2026-07-25' paper_count: 94 papers: - id: '1609.03499' published_date: "2016-09-12" entry_date: '2026-07-25' year: 2016 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control claims: - claim_id: direct_generation_of_raw_audio_waveforms_without_intermediate role: supports claim: Direct generation of raw audio waveforms, without intermediate vocoder parameters, produces substantially higher naturalness than parametric or concatenative synthesis pipelines as judged by human listeners. source: §3.2, Table 1 evidence: On the MOS naturalness test using proprietary Google speech databases, WaveNet (conditioned on linguistic features and log-F0) scored 4.21 on North American English and 4.08 on Mandarin Chinese, compared to 3.86 / 3.47 for HMM-driven concatenative and 3.67 / 3.79 for LSTM-RNN parametric baselines (Table 1). confidence: high relevance: low - claim_id: dilated_causal_convolutions_enable_autoregressive_audio_models_to role: supports claim: Dilated causal convolutions enable autoregressive audio models to achieve receptive fields orders of magnitude larger than standard causal convolutions with comparable computational cost. source: §2.1, Figure 3 evidence: 'The core component is a stack of dilated causal convolutions: dilation doubles with each layer (1, 2, 4, ..., 512), and the pattern is repeated across multiple blocks, yielding a receptive field that grows exponentially with depth rather than linearly.' confidence: high relevance: low - claim_id: a_single_autoregressive_model_conditioned_on_speaker_identity role: supports claim: A single autoregressive model conditioned on speaker identity can represent many voices with shared internal structure, and multi-speaker training improves per-speaker quality relative to single-speaker training. source: §3.1 evidence: In the multi-speaker experiment on VCTK (109 speakers, 44 hours), a single global-conditioning model captured all speakers' characteristics, with validation performance improving over single-speaker training, suggesting that speaker-level representations are shared and mutually reinforcing in the model's internal structure. confidence: high relevance: low - claim_id: receptive_field_size_is_a_binding_constraint_for role: complicates claim: 'Receptive field size is a binding constraint for prosodic naturalness: when the receptive field is insufficient to cover phrase-level F0 contours, prosody degrades even when segmental quality remains high.' source: §3.2 evidence: Adding the external F0 conditioning resolved this, showing that receptive field size is a binding constraint for prosodic naturalness independent of model capacity. confidence: high relevance: high - claim_id: autoregressive_raw_waveform_generation_achieves_high_naturalness_at role: complicates claim: Autoregressive raw-waveform generation achieves high naturalness at the cost of sequential sample-level inference, creating a fundamental speed-quality trade-off that constrains deployment in real-time applications. source: §4, §3.2 evidence: 'All timestep predictions are computed in parallel during training (ground-truth samples available at all positions), but inference is strictly sequential: one sample at a time, each fed back into the network to predict the next.' confidence: high relevance: low limitations: - Inference is strictly sequential at the sample level, requiring approximately one computation step per generated sample. At the reported generation rates (roughly 1.5× real-time compute), WaveNet is not suitable for real-time TTS deployment without hardware-specific optimisation or a parallel decoding approximation. - Evaluation is conducted on proprietary Google TTS databases, making direct replication by external researchers impossible. The MOS comparison is fair internally (same data, same test sentences for all systems) but cannot be directly compared to numbers from other published evaluations. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '1703.10135' published_date: "2017-03-29" entry_date: '2026-07-25' year: 2017 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: end_to_end_tts_models_trained_from_characters role: supports claim: End-to-end TTS models trained from characters with seq2seq attention can match or exceed production statistical parametric systems in subjective naturalness without hand-engineered linguistic features. source: §5.2, Table 2 evidence: The encoder thus produces robust sequence representations without requiring phoneme-level alignment or a hand-crafted linguistic frontend. confidence: high relevance: low - claim_id: training_stability_and_alignment_quality_in_character_level role: supports claim: Training stability and alignment quality in character-level seq2seq TTS improve substantially when the decoder emits multiple output frames per attention step rather than one. source: §3.3 evidence: 'A key design choice is the reduction factor r: the decoder predicts r non-overlapping mel frames per step rather than one, dividing decoding steps by r, accelerating training convergence, and encouraging the attention to advance more readily through the input sequence (r=2 in the main experiments).' confidence: high relevance: low - claim_id: cbhg_style_encoders_combining_multi_scale_convolution_highway role: supports claim: CBHG-style encoders combining multi-scale convolution, highway networks, and bidirectional recurrence yield more robust text representations than standard RNN encoders, reducing mispronunciation rates. source: §3.2, §5.1 evidence: The encoder centres on a novel CBHG module (1-D Convolution Bank, Highway network, and bidirectional GRU). confidence: high relevance: medium - claim_id: post_processing_networks_with_access_to_the_full role: supports claim: Post-processing networks with access to the full decoded sequence improve harmonic structure in predicted spectrograms compared to frame-level-only decoding. source: §3.4, §5.1 evidence: Removing the post-processing net eliminates fine harmonic structure in the output. confidence: high relevance: low limitations: - Evaluated on a single internal speaker in a controlled studio environment. No multi-speaker, out-of-domain, or noisy-data experiments are reported. The internal dataset is not released, making direct replication impossible. - Griffin-Lim waveform synthesis introduces audible artefacts that depress MOS scores and prevent fair comparison against systems using neural vocoders. The authors explicitly flag this as a known limitation and describe neural inversion as ongoing work. The reduction factor r is fixed at inference; the paper does not explore adaptive or learned stopping. The model still requires clean text-normalised input; robustness to raw text (numbers, abbreviations, punctuation) is handled by a rule-based preprocessing step rather than learned normalisation. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '1712.05884' published_date: "2017-12-16" entry_date: '2026-07-25' year: 2017 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: a_compact_low_level_acoustic_intermediate_representation_can role: supports claim: A compact low-level acoustic intermediate representation can bridge text encoding and neural vocoder synthesis without requiring hand-crafted linguistic features, enabling fully end-to-end neural TTS at human-quality levels. source: §2.1, Table 1 evidence: A key design insight is that the mel spectrogram serves as a compact, phase-invariant acoustic intermediate that is both easy to predict from text (smooth enough for MSE training) and easy to invert into high-quality audio (sufficient low-level detail for a small WaveNet). confidence: high relevance: low - claim_id: location_sensitive_attention_which_incorporates_cumulative_attention_weights role: supports claim: Location-sensitive attention, which incorporates cumulative attention weights as a conditioning signal, reduces failure modes such as repetition and omission compared to standard additive attention in autoregressive TTS decoders. source: §2.2 evidence: The first stage is a recurrent encoder-decoder network with location-sensitive attention that maps character sequences directly to 80-channel log-mel spectrogram frames. confidence: high relevance: low - claim_id: a_pre_net_information_bottleneck_in_the_autoregressive role: complicates claim: A pre-net information bottleneck in the autoregressive decoder is essential for stable attention alignment during training. source: §2.2 evidence: 'The decoder is autoregressive: at each step, the previous predicted frame passes through a two-layer pre-net bottleneck (256 units) before being combined with the attention context and fed through two unidirectional LSTM layers (1024 units).' confidence: high relevance: low - claim_id: training_the_vocoder_on_predicted_rather_than_ground role: supports claim: Training the vocoder on predicted rather than ground-truth intermediate features is necessary because predicted features are systematically oversmoothed; vocoders trained on clean features degrade when exposed to predicted inputs. source: §3.3.1, Table 2 evidence: Training WaveNet on ground-truth spectrograms and then synthesising from predicted (oversmoothed) spectrograms is the worst configuration (4.362), confirming that training and inference distributions must match. confidence: high relevance: low - claim_id: neural_vocoders_conditioned_on_compact_mel_spectrograms_can role: supports claim: Neural vocoders conditioned on compact mel spectrograms can operate with substantially smaller receptive fields than those conditioned on linguistic features, enabling significant architecture simplification without quality loss. source: §3.3.4, Table 4 evidence: This compares to 4.341 for WaveNet conditioned on linguistic features, 4.001 for Tacotron with Griffin-Lim, 4.166 for the concatenative production system, and 3.492 for parametric synthesis. confidence: high relevance: low limitations: - All experiments use a single proprietary speaker and an internal dataset. The paper provides no evidence of generalisation to multiple speakers, different languages, or recording conditions, and no training or evaluation data is publicly released, limiting independent reproducibility. - 'Prosody remains imperfect: 23 of 100 test sentences are rated as containing unnatural prosody (wrong emphasis, unnatural pitch), and occasional mispronunciations are identified as the main gap between the system and human speech. The paper''s authors explicitly identify prosody modeling as the primary remaining challenge.' caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '2006.04558' published_date: "2020-06-08" entry_date: '2026-07-25' year: 2020 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - explicit_acoustic_variance_control - voice_quality_and_paralinguistic_feature_control claims: - claim_id: explicit_variance_conditioning_on_pitch_energy_and_duration role: supports claim: Explicit variance conditioning on pitch, energy, and duration in non-autoregressive TTS reduces the information gap between text and speech, enabling quality matching or surpassing autoregressive models without autoregressive inference. source: §2.2, §3.2.1, Table 1 evidence: 'The variance adaptor contains three predictors operating sequentially on the phoneme hidden sequence: a duration predictor, a pitch predictor, and an energy predictor.' confidence: high relevance: high - claim_id: phoneme_durations_derived_from_forced_alignment_are_substantially role: supports claim: Phoneme durations derived from forced alignment are substantially more accurate than those extracted from autoregressive teacher attention maps, and this accuracy difference directly improves output voice quality. source: §3.2.2, Table 5 evidence: MFA-derived durations improve CMOS by 0.195 over teacher-extracted durations, confirming that alignment accuracy directly affects voice quality (Table 5). confidence: high relevance: medium - claim_id: modeling_pitch_in_the_frequency_domain_via_continuous role: supports claim: Modeling pitch in the frequency domain via continuous wavelet transform produces more natural pitch distributions in synthesized speech than direct time-domain F0 regression. source: §2.3, §3.2.3, Table 3 evidence: Pitch is modeled in the frequency domain via continuous wavelet transform (CWT), which decomposes the pitch contour into 10 wavelet components forming a pitch spectrogram — this regularises pitch prediction against the high temporal variability that makes direct F0 regression difficult. confidence: high relevance: high - claim_id: removing_teacher_student_distillation_from_non_autoregressive_tts role: supports claim: Removing teacher-student distillation from non-autoregressive TTS training can simultaneously simplify the pipeline, reduce training time, and improve output quality relative to the distillation approach. source: §1, §3.2.1, Table 2 evidence: FastSpeech 2 replaces the teacher-student pipeline with direct supervision on ground-truth mel-spectrograms and addresses the one-to-many mapping problem by explicitly providing variance information at training time. confidence: high relevance: low - claim_id: end_to_end_parallel_text_to_waveform_generation role: supports claim: End-to-end parallel text-to-waveform generation is technically feasible but more sensitive to variance conditioning than mel-spectrogram-based acoustic models, particularly in the pitch dimension. source: §2.4, §3.2.3, Table 6 evidence: Removing pitch from FastSpeech 2 costs 0.245 CMOS; removing it from FastSpeech 2s costs 1.130 CMOS, indicating that end-to-end waveform generation is especially reliant on explicit pitch conditioning. confidence: high relevance: high limitations: - All experiments use a single studio-quality English speaker (LJSpeech). Generalisation to multi-speaker, emotional, noisy, or cross-lingual conditions is untested. The paper does not include speaker similarity metrics or speaker-conditioned variants. - The variance adaptor relies on an external forced alignment tool (MFA) and pitch extraction (PyWorldVocoder) at training time; these are not learned end-to-end, which limits the system from being fully self-contained. The authors acknowledge this and note future work toward eliminating external alignment tools. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2104.00355' published_date: "2021-04-01" entry_date: '2026-07-25' year: 2021 venue: arXiv task: - TTS - VC architecture: - GAN - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - explicit_acoustic_variance_control - disentangled_prosody_content_timbre - signal_level_pitch_modification claims: - claim_id: ssl_content_representations_that_are_well_disentangled_from role: supports claim: SSL content representations that are well-disentangled from speaker identity also exhibit stronger voice conversion performance, while representations that entangle speaker information perform worse at conversion but better at pitch reconstruction. source: §4, Table 1, Table 2 evidence: VQ-VAE performs substantially worse (EER 9.65), confirming that its content codes entangle speaker identity. confidence: high relevance: high - claim_id: discrete_speech_units_learned_by_ssl_models_can role: supports claim: Discrete speech units learned by SSL models can form the basis of an ultra-low-bitrate speech codec that outperforms classical parametric codecs in subjective quality. source: §4, Figure 2 evidence: The low-bitrate codec experiment (MUSHRA on VCTK, 20 utterances from 5 unseen speakers) shows HuBERT with 50 units at 365 bps outperforms Opus at 9 kbps, Codec2 at 2.4 kbps, and LPCNet at 1.6 kbps in perceived quality. confidence: high relevance: low - claim_id: among_self_supervised_content_encoders_hubert_units_carry role: supports claim: Among self-supervised content encoders, HuBERT units carry less speaker and pitch information than VQ-VAE units, making them better suited for downstream controllable synthesis. source: §4, Table 2 evidence: VQ-VAE outperforms HuBERT and CPC on F0 reconstruction metrics (VDE 7.19 vs. 13.09 for HuBERT on LJSpeech), indicating that VQ-VAE retains more pitch information in its content codes, making it less suitable for independent prosody control. confidence: high relevance: high - claim_id: pitch_and_speaker_identity_can_be_independently_conditioned role: supports claim: Pitch and speaker identity can be independently conditioned in a GAN vocoder through separate discrete token streams, enabling controllable F0 manipulation without retraining. source: §3, §4 evidence: A separate F0 encoder applies a VQ-VAE trained on extracted pitch contours, producing a 20-token discrete pitch sequence at 12.5 Hz (65 bps). confidence: high relevance: high limitations: - The codec evaluation uses only 20 utterances from 5 VCTK speakers, all unseen during training but from the same corpus. Generalization to out-of-domain speech (conversational, noisy, or non-English) is untested. - The resynthesis MOS scores remain well below ground truth on both LJSpeech (3.66 vs. 4.33) and VCTK (3.41 vs. 4.08), indicating a quality gap the system does not close. Disentanglement is evaluated indirectly through proxy metrics (EER, VDE, FFE) rather than a direct information-theoretic measure. The speaker encoder requires speaker embeddings from training-set speakers for the lookup-table variant; the d-vector approach generalizes but relies on a separately trained verification model. No ablation isolates the contribution of the F0 conditioning stream to final MOS. The MUSHRA scores in Figure 2 are visual only, making exact numerical comparison to baselines difficult to reproduce from the paper text alone. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2304.09116' published_date: "2023-04-18" entry_date: '2026-07-25' year: 2023 venue: arXiv task: - TTS - VC - singing architecture: - diffusion - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - multimodal_visual_prosody_conditioning - speech_singing_shared_prosody claims: - claim_id: latent_diffusion_models_operating_on_continuous_codec_vectors role: supports claim: Latent diffusion models operating on continuous codec vectors avoid the word-skipping and repetition errors that arise from autoregressive generation over long discrete token sequences. source: §2.3, §5.3, Table 7 evidence: NaturalSpeech 2 sidesteps the discrete-token dilemma by using a custom RVQ codec that produces continuous latent vectors rather than discrete indices. confidence: high relevance: low - claim_id: speech_prompting_via_in_context_learning_during_training role: supports claim: Speech prompting via in-context learning during training enables zero-shot speaker adaptation without requiring speaker embeddings or multi-step speaker encoding pipelines. source: §3.3, §5.5 evidence: On LibriSpeech test-clean (zero-shot setting, all speakers unseen during training), NaturalSpeech 2 achieves a CMOS of 0.00 reference-anchored, matching ground truth at +0.04 CMOS — a result the authors interpret as human-level naturalness on this benchmark. confidence: high relevance: low - claim_id: prosody_adherence_in_zero_shot_tts_improves_monotonically role: supports claim: Prosody adherence in zero-shot TTS improves monotonically with the length of the reference speech prompt, at least up to 10 seconds. source: §5.5, Table 10 evidence: On LibriSpeech test-clean (zero-shot setting, all speakers unseen during training), NaturalSpeech 2 achieves a CMOS of 0.00 reference-anchored, matching ground truth at +0.04 CMOS — a result the authors interpret as human-level naturalness on this benchmark. confidence: high relevance: high - claim_id: non_autoregressive_tts_architectures_maintain_near_zero_error role: complicates claim: Non-autoregressive TTS architectures maintain near-zero error rates on adversarially difficult phoneme sequences where autoregressive models degrade significantly. source: §5.3, Table 7 evidence: On robustness, NaturalSpeech 2 achieves 0% error rate on 50 adversarially hard sentences (vs. 24% for Tacotron, 34% for Transformer-TTS, and known failures for VALL-E). confidence: high relevance: low - claim_id: a_system_trained_jointly_on_speech_and_singing role: supports claim: A system trained jointly on speech and singing data can synthesise singing in novel timbres using only a speech reference prompt, demonstrating cross-modal timbre transfer within a shared latent space. source: §5.6 evidence: A novel RVQ cross-entropy loss (L_ce-rvq) computed at each residual quantizer level is added as a regularisation term, contributing to sharper latent prediction. confidence: high relevance: medium limitations: - The direct comparison with VALL-E is based on VALL-E demo page samples rather than a controlled shared test set — the 16 compared utterances are cherry-picked by the VALL-E authors and may not be representative. This limits the strength of the head-to-head quality claim. - The model is described as still underfitting at 300K training steps, meaning reported results are likely below the system's ceiling performance. Inference requires 150 diffusion steps (ODE solver), and 1000 steps for singing, which is slow for real-time deployment. The paper cites consistency models as future work for acceleration. Training and evaluation are English-only, so multilingual generalisation is uncharacterised. The singing dataset is approximately 30 hours of web-crawled data with no formal provenance or quality validation beyond alignment filtering, which raises questions about singing style coverage. Code and model weights are not publicly released, limiting reproducibility. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2402.01912' published_date: "2024-02-02" entry_date: '2026-07-25' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: low evidence_role: - historical_context current_role: minor method_family: - structured_language_prosody_control claims: - claim_id: automatic_acoustic_labeling_can_substitute_for_human_annotations role: supports claim: Automatic acoustic labeling can substitute for human annotations in training large-scale instruction-conditioned speech language models without a loss in attribute control accuracy relative to human-labeled systems. source: §3.1, §3.2, §4.1 evidence: Objective attribute control is validated using the same automatic classifiers that labeled the training data. confidence: high relevance: medium - claim_id: including_a_small_proportion_of_high_fidelity_audio role: supports claim: Including a small proportion of high-fidelity audio (approximately 1%) in a predominantly noisy training corpus, combined with explicit recording-quality labels, enables a speech LM to generate professional-sounding speech on demand from text prompts alone. source: §3.1.2, §4.2, Table 1 evidence: 'High audio fidelity is achieved through a combination of: (1) using DAC rather than EnCodec, which the authors report provides subjectively and objectively superior audio quality, and (2) including LibriTTS-R in training, which provides approximately 500 hours (roughly 1%) of clean, professionally enhanced speech alongside the crowd-sourced data.' confidence: high relevance: low - claim_id: the_choice_of_neural_audio_codec_has_a role: supports claim: The choice of neural audio codec has a measurable effect on perceptual audio quality in autoregressive TTS; higher-fidelity codecs translate directly to higher MOS and objective quality scores. source: §3.3, §4.2, Table 1–2 evidence: 'Objective audio quality metrics (PESQ, STOI, SI-SDR) evaluated via TorchAudio-SQUIM on 20 samples with "excellent recording quality" descriptions show the proposed model substantially outperforms Audiobox and approaches ground-truth values: PESQ 3.84 vs. 3.46 (Audiobox), STOI 0.996 vs. 0.988, SI-SDR 26.53 dB vs. 21.84 dB (Table 1).' confidence: high relevance: low - claim_id: natural_language_conditioning_on_accent_can_be_achieved role: supports claim: Natural language conditioning on accent can be achieved in a single TTS model covering dozens of accents, though classifier accuracy reflects the noise and imbalance inherent in automatic accent labeling of crowd-sourced data. source: §3.1.1, §4.1 evidence: A large language model (Stable Beluga 2) then converts keyword tuples into natural language sentences, eg. "a woman with a deep voice speaking slowly and somewhat monotonously with a Hungarian accent in an echoey room with background noise." This approach is applied to the 45k-hour English Multilingual LibriSpeech corpus and the smaller 585-hour LibriTTS-R corpus (included for its higher audio fidelity from the Miipher speech-enhancement model). confidence: high relevance: low limitations: - The evaluation compares only against Audiobox. No standard TTS baselines (reference-based zero-shot systems, encoder-decoder models) are included, making it impossible to assess whether the MOS gains arise from the conditioning approach, the codec choice, or the training data mix. - The system is evaluated only on English audiobook speech; generalization to conversational, spontaneous, or non-English speech is stated as future work but untested. The accent accuracy of 68% indicates that discrete accent labels in crowd-sourced data are noisy, and the model's C50 control was found to be unreliable even after training. Model size and total compute are not reported. The automatic labeling pipeline requires training multiple specialized classifiers (accent, gender), each of which introduces its own noise floor. The demo website is the only verification source; no code or model weights are released. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '2402.08093' published_date: "2024-02-12" entry_date: '2026-07-25' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - implicit_scale_and_data_prosody_modeling - prosody_analysis_and_evaluation_method claims: - claim_id: scaling_autoregressive_codec_tts_to_500m_parameters_and role: supports claim: Scaling autoregressive codec TTS to 500M+ parameters and 10K+ hours of training data produces qualitatively different prosody rendering on linguistically complex inputs compared to smaller models trained on less data. source: §4.3, Figure 4, Table 5 evidence: The WavLM-based speechcodes match VQ-VAE on English (MUSHRA 74.7 vs 74.8, non-significant) but outperform it on Spanish despite Spanish comprising only 2% of training data (MUSHRA 74.7 vs 73.3, significant for some speakers), supporting the hypothesis that better representations matter more in lower-resource languages (Table 3). confidence: high relevance: high - claim_id: ssl_based_speech_representations_with_explicit_speaker_disentanglement role: supports claim: SSL-based speech representations with explicit speaker disentanglement outperform purely acoustic codec representations for zero-shot TTS, particularly in lower-resource languages. source: §4.1, Table 3 evidence: The WavLM-based speechcodes match VQ-VAE on English (MUSHRA 74.7 vs 74.8, non-significant) but outperform it on Spanish despite Spanish comprising only 2% of training data (MUSHRA 74.7 vs 73.3, significant for some speakers), supporting the hypothesis that better representations matter more in lower-resource languages (Table 3). confidence: high relevance: low - claim_id: a_streamable_convolutional_decoder_can_match_or_exceed role: supports claim: A streamable convolutional decoder can match or exceed a diffusion-based spectrogram decoder in subjective naturalness while reducing synthesis compute by approximately 3x and enabling low-latency streaming. source: §4.2, §4.5, Table 4 evidence: The speechcode decoder matches or improves over the diffusion decoder on 4 of 6 evaluated speakers, while offering a 3x compute reduction and enabling streaming (Table 4). confidence: high relevance: low - claim_id: applying_bpe_to_discrete_speech_tokens_reduces_autoregressive role: supports claim: Applying BPE to discrete speech tokens reduces autoregressive sequence length by approximately 40% without degrading downstream synthesis quality, enabling longer-context training. source: §2.2.3 evidence: A speech tokenizer converts waveforms into a compact discrete sequence; an autoregressive Transformer ("SpeechGPT") predicts those tokens from text and a reference speech clip; and a convolutional decoder renders the predicted tokens to audio. confidence: high relevance: low - claim_id: autoregressive_tts_trained_at_scale_generalises_to_a role: supports claim: Autoregressive TTS trained at scale generalises to a wide range of textual phenomena without any explicit prosody annotation or task-specific supervision. source: §4.3, §6 evidence: Baseline comparisons (Tortoise, Bark, YourTTS) use publicly available checkpoints not trained on comparable data volumes; the gains partly reflect data scale rather than architecture alone. confidence: high relevance: high limitations: - Model weights are not released, and evaluation uses proprietary test speakers. The MUSHRA baselines (YourTTS, Bark, TortoiseTTS) are not trained on comparable data or compute, making architecture-level conclusions difficult to separate from scale effects. - The speechcode decoder is tightly coupled to a specific frozen SpeechGPT checkpoint via hidden-state conditioning, preventing modular updates and complicating experimentation. The paper identifies this as a limitation requiring future work. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2403.03100' published_date: "2024-03-05" entry_date: '2026-07-25' year: 2024 venue: arXiv task: - TTS - VC architecture: - diffusion - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - disentangled_prosody_content_timbre claims: - claim_id: explicit_disentanglement_of_speech_attributes_in_the_codec role: supports claim: Explicit disentanglement of speech attributes in the codec representation reduces the complexity of zero-shot generation and improves speaker similarity, quality, and prosody simultaneously. source: §3, §4.2, Table 1, Table 2 evidence: Disentanglement is enforced through a combination of supervised auxiliary losses (F0 prediction on prosody codes, phoneme prediction on content codes, speaker classification on the timbre vector), gradient reversal layers to actively suppress cross-attribute information leakage, and detail dropout that randomly masks acoustic detail codes during training to prevent the codec from offloading content and prosody into the detail subspace. confidence: high relevance: high - claim_id: gradient_reversal_combined_with_attribute_specific_supervised_losses role: supports claim: Gradient reversal combined with attribute-specific supervised losses is an effective mechanism for suppressing cross-attribute information leakage in neural codec quantization. source: §3.2.2, Appendix B.4 evidence: Disentanglement is enforced through a combination of supervised auxiliary losses (F0 prediction on prosody codes, phoneme prediction on content codes, speaker classification on the timbre vector), gradient reversal layers to actively suppress cross-attribute information leakage, and detail dropout that randomly masks acoustic detail codes during training to prevent the codec from offloading content and prosody into the detail subspace. confidence: high relevance: low - claim_id: the_factorization_paradigm_for_codec_representations_is_architecture role: supports claim: The factorization paradigm for codec representations is architecture-agnostic and improves both autoregressive and non-autoregressive generators when applied. source: §4.3.2, Table 6 evidence: The paper also shows the factorisation paradigm generalises to autoregressive generation by substituting VALL-E for the diffusion modules while retaining FACodec, demonstrating the design is not tied to its diffusion formulation. confidence: high relevance: low - claim_id: discrete_masked_diffusion_over_disentangled_codec_tokens_is role: supports claim: Discrete masked diffusion over disentangled codec tokens is faster than autoregressive LM-based codec generation at comparable or better quality. source: Appendix A.5, Table 10 evidence: The factorized diffusion model is a discrete masked diffusion system (mask-and-predict, not continuous DDPM), operating over the discrete tokens from FACodec. confidence: high relevance: low - claim_id: performance_on_zero_shot_tts_scales_predictably_with role: supports claim: Performance on zero-shot TTS scales predictably with both training data volume and model size when the underlying speech representation captures disentangled attributes. source: §4.4, Tables 7, 8 evidence: FACodec also enables zero-shot voice conversion without task-specific training, achieving Sim-O of 0.86 and WER of 3.46% on VCTK, competitive with dedicated VC models (Table 14). confidence: high relevance: low limitations: - FACodec requires phoneme-level transcriptions for content supervision during training, constraining its applicability to languages and settings where reliable alignments are unavailable. The zero-shot TTS evaluation is English-only; multilingual generalisation is stated as future work but not demonstrated. - 'Additional limitations: the attribute factorization is incomplete (background sounds, energy, and other fine-grained characteristics are not captured, as noted in Appendix C); the acoustic detail subspace retains some content and prosody leakage without gradient reversal (verified qualitatively in Appendix B.4); and the prosody evaluation relies on MCD and emotion classifiers on the RAVDESS dataset, which assesses a narrow range of acted emotions rather than naturalistic prosodic variation.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2406.07855' published_date: "2024-06-12" entry_date: '2026-07-25' year: 2024 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - disentangled_prosody_content_timbre claims: - claim_id: phoneme_monotonic_alignment_in_decoder_only_autoregressive_tts role: supports claim: Phoneme monotonic alignment in decoder-only autoregressive TTS can close most of the robustness gap caused by unconstrained attention, achieving near-ground-truth WER without encoder-decoder architectural changes. source: §3.2, Table 1 evidence: Unlike encoder-decoder monotonic attention methods, this approach is compatible with the decoder-only Transformer architecture. confidence: high relevance: low - claim_id: downsampling_only_the_first_rvq_layer_of_a role: supports claim: Downsampling only the first RVQ layer of a neural codec at inference time reduces autoregressive steps and latency by more than half, with negligible impact on PESQ and STOI. source: §3.1, Table 5 evidence: 'Applying merging only to the first RVQ layer (rather than multiple layers) preserves audio quality: PESQ and STOI drop negligibly when only layer 1 is merged at 2x, while merging four or eight layers causes significant degradation.' confidence: high relevance: low - claim_id: robustness_improvements_that_route_additional_phoneme_tokens_through role: complicates claim: Robustness improvements that route additional phoneme tokens through the autoregressive stream (as in ELLA-V) improve WER but increase inference time, illustrating a robustness-efficiency trade-off in codec LM TTS. source: §5.3, Table 4 evidence: For inference efficiency (Table 4), VALL-E R (2x merging) generates 10 seconds of speech in 3.67 seconds (375 AR steps), compared to VALL-E's 10.27 seconds (750 AR steps) — a 64% reduction. confidence: high relevance: low - claim_id: explicit_phoneme_level_alignment_in_a_codec_lm role: supports claim: Explicit phoneme-level alignment in a codec LM enables independent control of prosody and timbre by substituting preset phoneme sequences at inference, enabling a form of voice conversion. source: §3.2.3, Table 3 evidence: 'Because VALL-E R explicitly tracks phoneme progress during inference, it also supports prosody control: replacing the self-predicted phoneme sequence with a preset aligned phoneme sequence from a reference utterance allows timbre and prosody to be controlled independently, enabling a form of voice conversion.' confidence: high relevance: high limitations: - All evaluations use LibriSpeech (clean English read speech). Robustness gains from monotonic alignment and codec-merging quality preservation have not been tested on noisy, expressive, or multilingual speech. - The model size is not explicitly reported, though the architecture (12-layer Transformer, 1024-dim hidden, 16 heads) matches the VALL-E reference scale. Code and model weights are not publicly released, limiting reproducibility. The prosody control evaluation uses MCD-DTW-SL, a proxy metric; perceptual validation of prosody cloning quality is absent. The merging rate of 2x is validated by reconstruction metrics but its downstream effect on naturalness under diverse speaker and content conditions is not fully explored. RALL-E (chain-of-thought prompting for robustness) is included only in the efficiency comparison, not in the WER robustness comparison, making head-to-head robustness assessment with concurrent work incomplete. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: 2025.coling-main.352 published_date: "2025-01-19" entry_date: '2026-07-25' year: 2025 venue: COLING task: - TTS architecture: - diffusion - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - stochastic_generative_prosody_model - implicit_scale_and_data_prosody_modeling claims: - claim_id: applying_diffusion_based_classifier_free_guidance_to_low role: supports claim: Applying diffusion-based classifier-free guidance to low-dimensional prosodic features (pitch, energy, duration) rather than the full Mel-spectrogram achieves faster inference than Mel-diffusion baselines while retaining prosodic diversity benefits over regression-based models. source: §3.2, Table 1 evidence: On synthesis speed (RTF), DiffStyleTTS (0.048) is faster than Grad-TTS (0.250) and Guided-TTS (0.479) because it runs diffusion only over low-dimensional prosodic features rather than full Mel-spectrograms. confidence: high relevance: high - claim_id: hierarchical_conditioning_combining_coarse_grained_implicit_style_tokens role: supports claim: Hierarchical conditioning combining coarse-grained implicit style tokens with fine-grained explicit prosodic diffusion outperforms both purely regression-based and discriminator-constrained diffusion baselines on naturalness and prosodic distribution accuracy. source: §3.2, Table 1 evidence: Coarse-grained implicit style conditions (ISC) are extracted from reference Mel-spectrograms via a GST module (10 tokens, 256-D token embeddings, 4-head attention). confidence: high relevance: high - claim_id: classifier_free_guidance_scale_controls_a_trade_off role: complicates claim: Classifier-free guidance scale controls a trade-off between prosodic diversity and naturalness quality in diffusion-based prosody prediction, with distortion artefacts emerging at high guiding scales. source: §3.3, Table 2 evidence: A dynamic thresholding improvement corrects the standard deviation of the guided noise output to prevent phoneme distortion (elongation/noise artefacts) that arises at high guiding scales (η ≥ 7.0). confidence: high relevance: high - claim_id: dynamic_thresholding_applied_to_cfg_noise_outputs_effectively role: supports claim: Dynamic thresholding applied to CFG noise outputs effectively mitigates phoneme distortion caused by excessive guiding scales in prosodic diffusion models. source: §2.2, §3.3 evidence: A dynamic thresholding improvement corrects the standard deviation of the guided noise output to prevent phoneme distortion (elongation/noise artefacts) that arises at high guiding scales (η ≥ 7.0). confidence: high relevance: high - claim_id: gst_based_implicit_style_conditioning_enables_meaningful_prosodic role: supports claim: GST-based implicit style conditioning enables meaningful prosodic transfer across speakers, outperforming both regression-based style transfer and discriminator-constrained diffusion in subjective preference evaluations. source: §3.4, Table 3 evidence: 'The conditional diffusion module implements classifier-free guidance (CFG) using two jointly trained denoisers sharing the same bidirectional dilated convolution architecture (12 residual layers, WaveNet-style): one conditioned on both text embeddings and implicit style conditions (Ψ_θ1), one conditioned on text embeddings only (Ψ_θ2).' confidence: high relevance: high limitations: - Speaker timbre and prosody are not disentangled — the model can transfer prosodic style but cannot fully separate it from speaker identity. The GST tokens show partial overlap in t-SNE visualizations, indicating shared prosodic modes and incomplete independence. The evaluation is conducted on a private Mandarin Chinese dataset (9 male speakers from similar genres), limiting generalization claims across languages, genders, or more varied speaking styles. The 200-step diffusion process at inference is still far slower than FastSpeech2, and no acceleration scheme (consistency models, flow-matching) is explored. Whether the CFG approach generalises to cross-lingual or highly emotional speech remains untested. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.coling-main.518 published_date: "2025-01-19" entry_date: '2026-07-25' year: 2025 venue: COLING task: - TTS architecture: - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - stochastic_generative_prosody_model claims: - claim_id: applying_conditional_flow_matching_to_a_self_supervised role: supports claim: Applying conditional flow matching to a self-supervised prosody latent space rather than to full acoustic features enables diverse prosody generation at inference without requiring a reference utterance. source: §2.3, §3.3 evidence: Rather than sampling a prosody vector from a VAE posterior (StyleTTS 2 approach), ProsodyFlow trains a conditional flow matching model to learn the distribution of prosody vectors s conditioned on PLBert text features h_bert. confidence: high relevance: high - claim_id: self_supervised_speech_representations_wavlm_provide_a_more role: supports claim: Self-supervised speech representations (WavLM) provide a more effective prosody conditioning signal for TTS than conventional pitch and energy predictors, as shown by ablation. source: §3.4, Table 3 evidence: 'Ablation study (Table 3, component removal): removing prosody flow matching (using a reference encoder instead) causes the largest drop (CMOS -0.27); removing WavLM causes -0.18 CMOS; replacing flow matching with diffusion causes only -0.04 CMOS (consistent quality, faster inference).' confidence: high relevance: high - claim_id: flow_matching_in_a_prosody_latent_space_achieves role: supports claim: Flow matching in a prosody latent space achieves comparable quality to diffusion-based prosody modeling with substantially lower computational cost. source: §3.4, Table 3 evidence: 'Ablation study (Table 3, component removal): removing prosody flow matching (using a reference encoder instead) causes the largest drop (CMOS -0.27); removing WavLM causes -0.18 CMOS; replacing flow matching with diffusion causes only -0.04 CMOS (consistent quality, faster inference).' confidence: high relevance: high - claim_id: a_small_number_of_flow_matching_function_evaluations role: supports claim: A small number of flow matching function evaluations (n=1) is sufficient to match or exceed legacy TTS baselines on MOS and WER, confirming the sample efficiency of flow matching for prosody. source: §3.3, Table 2 evidence: 'Ablation study (Table 2, varying NFE): even n=1 (MOS 3.92, WER 2.61%, RTF 0.0114) competes with baseline models, demonstrating flow matching efficiency.' confidence: high relevance: high limitations: - The model is validated only on single-speaker LJSpeech; extension to multi-speaker and zero-shot settings is explicitly acknowledged as future work. The model architecture size is not reported. WavLM parameters are frozen throughout training, which may limit adaptation to unusual prosody distributions. The comparison set does not include the most recent flow matching TTS systems (Voicebox, Matcha-TTS, E2 TTS) — only legacy baselines (FastSpeech 2, VITS, StyleTTS 2) and one diffusion model (DiffProsody). The absence of speaker diversity means the prosody variability shown may primarily reflect intra-speaker diversity of LJSpeech rather than generalizable prosodic modeling. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2502.04128' published_date: "2025-02-06" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - implicit_scale_and_data_prosody_modeling claims: - claim_id: single_stage_autoregressive_tts_trained_with_next_token role: supports claim: Single-stage autoregressive TTS trained with next-token prediction over discrete speech tokens is competitive with multi-stage AR+NAR pipelines on intelligibility and speaker similarity in continuation mode, though SIM-o gaps remain due to codec acoustic reconstruction limits. source: §3.2.4, Table 3 evidence: However, speaker similarity (SIM-o) for direct inference is notably weaker than these baselines, reflecting the acoustic reconstruction ceiling of a single-VQ codec versus multi-layer RVQ or mel-based vocoders. confidence: high relevance: low - claim_id: both_model_scale_and_training_data_volume_independently role: supports claim: Both model scale and training data volume independently improve TTS quality across naturalness, prosody, and text comprehension, consistent with scaling laws observed in text LLMs. source: §2.3, §3.2.2, Tables 2, 4 evidence: 'Scaling both model size and training data consistently improves all metrics: WER on test-en drops from 3.71% (1B-80k) to 2.97% (8B-250k), and emotion similarity on ESD rises from 0.753 to 0.778 (English) across model sizes (Table 4).' confidence: high relevance: high - claim_id: inference_time_compute_scaling_via_speech_understanding_verifiers role: complicates claim: Inference-time compute scaling via speech understanding verifiers can substantially improve speaker similarity and emotional expressiveness beyond what train-time scaling alone achieves, at the cost of additional inference compute. source: §2.4, §3.2.3, Figure 2, Table 2 evidence: Inference-time scaling uses off-the-shelf speech understanding models as verifiers. confidence: high relevance: low - claim_id: pure_process_reward_model_beam_search_for_tts role: supports claim: Pure process reward model beam search for TTS is prone to mode collapse that degrades content accuracy (WER), and a hybrid partial-PRM strategy is needed to preserve both speaker similarity and intelligibility. source: §3.2.3, Figure 2 evidence: 'Two search strategies are compared: Best-of-N (output reward model, ORM) and beam search with a process reward model (PRM) that evaluates speaker similarity at each 0.5-second step.' confidence: high relevance: low - claim_id: single_vq_codecs_can_achieve_intelligibility_and_naturalness role: complicates claim: Single-VQ codecs can achieve intelligibility and naturalness competitive with multi-layer RVQ codecs at the same token rate, but acoustic fidelity (speaker similarity) remains the limiting factor for single-VQ reconstruction. source: §3.1.2, Table 1 evidence: However, speaker similarity (SIM-o) for direct inference is notably weaker than these baselines, reflecting the acoustic reconstruction ceiling of a single-VQ codec versus multi-layer RVQ or mel-based vocoders. confidence: high relevance: low limitations: - 'The SIM-o gap between Llasa and RVQ-based baselines is intrinsic to the single-VQ design: acoustic reconstruction from a 65,536-entry single codebook at 50 Hz is weaker than 8-layer RVQ codecs, and this gap is only partially recovered by inference-time search. Systems requiring high timbre fidelity in a single inference pass would need a different codec design.' - Inference-time compute scaling requires running multiple candidates (beam search or Best-of-N) with auxiliary verifier models, which increases latency and compute cost substantially and makes the approach unsuitable for real-time or low-resource applications. The paper does not characterize latency or wall-clock overhead of the search strategies. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2502.18924' published_date: "2025-02-26" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - diffusion - VAE relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: providing_coarse_stochastic_phoneme_anchors_rather_than_fully role: supports claim: Providing coarse stochastic phoneme anchors rather than fully expanded forced alignments improves both naturalness and robustness simultaneously in latent diffusion TTS. source: §3.2, §4.3, Table 4, Table 7 evidence: 'The compact latent rate is intentional: the authors show that fewer tokens per second substantially improves the quality of the downstream diffusion model (§4.5, Table 6).' confidence: high relevance: low - claim_id: compact_continuous_latent_representations_at_very_low_token role: supports claim: Compact continuous latent representations at very low token rates enable higher zero-shot TTS quality than discrete codecs at higher bit rates when used as the target space for diffusion. source: §4.5, Table 5, Table 6 evidence: The WaveVAE encoder compresses 16 kHz speech into a highly compact continuous latent space at 25 vectors per second, achieved by a VAE with KL regularisation, spectrogram reconstruction loss, and adversarial discriminators (multi-period, multi-scale, and multi-resolution). confidence: high relevance: low - claim_id: piecewise_rectified_flow_distillation_reduces_inference_steps_from role: supports claim: Piecewise rectified flow distillation reduces inference steps from 25 to 8 with negligible degradation in speaker similarity and intelligibility. source: §3.2, §4.2, Table 1 evidence: 'For acceleration, MegaTTS 3 applies Piecewise Rectified Flow (PeRFlow) distillation: the ODE trajectory is divided into K time windows, and a student model is trained to span each window in fewer steps by mimicking the teacher''s endpoint within each window.' confidence: high relevance: low - claim_id: decoupled_text_and_speaker_guidance_scales_in_classifier role: supports claim: Decoupled text and speaker guidance scales in classifier-free guidance provide a continuous accent intensity control axis without requiring accent labels. source: §3.2, §4.4, Table 3 evidence: Multi-condition classifier-free guidance separates the text guidance scale (alpha-txt) and speaker guidance scale (alpha-spk). confidence: high relevance: low - claim_id: latent_diffusion_tts_systems_exhibit_strong_data_and role: supports claim: Latent diffusion TTS systems exhibit strong data and model scaling behaviour, with both speaker similarity and intelligibility improving consistently as training data grows from 2k to 600k hours and model size grows from 0.5B to 7B parameters. source: Appendix D, Table 8 evidence: Scaling experiments (Table 8) show that SIM-O improves from 0.52 at 2k training hours to 0.66 at 600k hours, and a 7B parameter model reaches 0.74 SIM-O and 1.90% WER, confirming data and model scalability. confidence: high relevance: low limitations: - The main results are reported on LibriSpeech test-clean, a read-speech corpus recorded in controlled conditions. The scaling and cross-domain results (Appendix D) use an internal test set of 400 samples, limiting external reproducibility for those claims. - Language coverage is restricted to English and Chinese despite the 600k-hour multilingual training corpus. The sparse alignment mechanism still depends on an external forced aligner (Montreal Forced Aligner) at training time, which requires a transcription pipeline and does not eliminate the dependency on alignment tools, merely relaxing it. The relationship between alignment anchor density and generation quality is explored only qualitatively; no principled analysis determines the optimal sparsity level. Code and checkpoints are not publicly available at time of writing. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: 2025.nodalida-1.32 published_date: "2025-03-02" entry_date: '2026-07-25' year: 2025 venue: NoDaLiDa/Baltic-HLT 2025 task: - TTS architecture: [] relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - language_and_domain_specific_prosody_modeling claims: - claim_id: explicit_phonological_annotation_in_tts_input_can_substitute role: supports claim: Explicit phonological annotation in TTS input can substitute for context-dependent disambiguation when morphological parsing is unavailable or unreliable. source: §3.2 evidence: The system achieves near-perfect expert recognition of palatalisation and quantity contrasts in Estonian homographs by requiring users to supply diacritics rather than relying on statistical disambiguation, compared to 50% chance-level performance in standard Estonian TTS systems. confidence: high relevance: low - claim_id: perception_based_accuracy_metrics_alone_are_insufficient_for role: complicates claim: Perception-based accuracy metrics alone are insufficient for evaluating isolated-word TTS quality. source: §3.1, §3.2 evidence: The evaluation uses expert phoneticians (n=8) on a controlled set of 64 homograph pairs with no naturalness MOS, no intelligibility test with naive listeners, and no comparison to natural speech recordings; the scope of claims about "very good" performance is therefore limited to pronunciation accuracy on a specialist-curated test. confidence: high relevance: low - claim_id: purpose_built_isolated_word_training_corpora_enable_statistical role: supports claim: Purpose-built isolated-word training corpora enable statistical parametric TTS systems to generate citation-form prosody that is inappropriate when systems are trained on continuous speech. source: §2.1, §4 evidence: Training on 31,215 professionally recorded isolated Estonian words (10h 36min) from CombiDic yields utterance prosody judged suitable for pronunciation training and dictionary applications. confidence: high relevance: high limitations: - The user interface requires manual diacritic insertion, which creates a significant usability barrier for non-linguists. The paper acknowledges this and plans automatic tagging with multiple homograph options as future work. More substantively, the evaluation relies entirely on expert listeners; it is unknown whether the synthesised contrasts are perceptible to the target users (language learners, speech therapy clients) who may have less phonological sensitivity. There is no naturalness evaluation alongside the accuracy test, so the system's overall speech quality relative to other Estonian TTS systems or to natural speech is not established. The choice of Merlin (a 2016-era toolkit) is noted as intentional for data efficiency and control, with more advanced TTS techniques planned for future systems. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2503.01710' published_date: "2025-03-03" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: disentangling_speech_tokens_into_linguistic_content_and_speaker role: supports claim: Disentangling speech tokens into linguistic content and speaker attributes within a single-stream codec enables a standard LLM to perform zero-shot TTS without a multi-stage pipeline. source: §3, §4.1 evidence: 'Speaker similarity in zero-shot TTS is Spark-TTS''s relative weakness: it scores 0.672 SIM on test-zh and 0.584 SIM on test-en, outperforming Llasa (0.684, 0.574) but falling behind multi-stage models such as MaskGCT (0.774, 0.714) and Seed-TTS (0.796, 0.762).' confidence: high relevance: low - claim_id: small_llm_backbones_can_achieve_competitive_zero_shot role: supports claim: Small LLM backbones can achieve competitive zero-shot TTS intelligibility when the codec reduces per-token modeling complexity through semantic alignment. source: §6.4, Table 4 evidence: 'The Spark-TTS language model fine-tunes Qwen2.5-0.5B in a decoder-only configuration, directly predicting the concatenated token sequence: for zero-shot TTS, the model conditions on text and global tokens extracted from a reference audio, then predicts the semantic token sequence.' confidence: high relevance: low - claim_id: single_stage_autoregressive_tts_consistently_trails_multi_stage role: supports claim: Single-stage autoregressive TTS consistently trails multi-stage or non-autoregressive methods on speaker similarity metrics, even when intelligibility is comparable. source: §6.4, Table 4, Limitation evidence: 'Speaker similarity in zero-shot TTS is Spark-TTS''s relative weakness: it scores 0.672 SIM on test-zh and 0.584 SIM on test-en, outperforming Llasa (0.684, 0.574) but falling behind multi-stage models such as MaskGCT (0.774, 0.714) and Seed-TTS (0.796, 0.762).' confidence: high relevance: low - claim_id: fsq_based_global_token_quantization_with_learnable_cross role: supports claim: FSQ-based global token quantization with learnable cross-attention queries produces better speaker attribute representation than group-VQ at equivalent token lengths. source: §6.2, Table 2 evidence: The global tokenizer processes the Mel spectrogram through an ECAPA-TDNN encoder followed by a cross-attention module with learnable queries, producing a fixed-length sequence of 32 tokens that represent time-invariant speaker attributes. confidence: high relevance: low - claim_id: attribute_controllable_tts_benefits_from_hierarchical_coarse_to role: supports claim: Attribute-controllable TTS benefits from hierarchical coarse-to-fine prediction within the LM inference loop rather than requiring separate conditioning modules. source: §4.1, §6.3 evidence: 'For attribute-controlled voice creation, a chain-of-thought (CoT) mechanism is employed: the model first predicts fine-grained attribute values (pitch in Hz rounded to integer, speaking rate in syllables-per-second) from coarse-grained labels (gender, 5-level pitch, 5-level speed), then generates global tokens, then semantic tokens.' confidence: high relevance: low limitations: - Speaker similarity in zero-shot cloning is meaningfully lower than multi-stage methods (SIM 0.672 vs. 0.774 for MaskGCT on test-zh). The paper attributes this to AR variability without explicit disentanglement constraints between semantic and global tokens, and no solution is evaluated in this work. - The VoxBox training data and BiCodec codec are trained on separate, relatively limited datasets (3k hours for BiCodec; 102.5k hours for the LM). The BiCodec training data is English-only (LibriSpeech + Emilia EN/CN), which may limit acoustic reconstruction quality for languages outside this distribution. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: 2025.findings-naacl.38 published_date: "2025-04-29" entry_date: '2026-07-25' year: 2025 venue: NAACL task: - TTS architecture: - transformer-enc-dec relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - emotion_expressive_prosody_control - structured_language_prosody_control claims: - claim_id: selecting_and_acoustically_emphasizing_a_small_subset_of role: supports claim: Selecting and acoustically emphasizing a small subset of emotion-salient words, rather than conditioning on a single holistic style embedding, improves emotion recognizability of synthesized dialogue speech. source: §5.1, Table 1 evidence: The proposed model improves Speech Emotion Accuracy by 12.98 points over an emotion-label-only baseline and 16.47 points over a dialogue-context-only baseline, with consistent MOS and ABX gains across all five emotion categories. confidence: high relevance: medium - claim_id: large_language_models_can_identify_text_spans_that role: supports claim: Large language models can identify text spans that carry emotional salience with moderate agreement to human judgment, without being trained on acoustic or prosodic data. source: §5.2, Table 2 evidence: GPT-3.5's word selections for emotion-bearing rationales overlap with a single human annotator's selections at a rate of 0.735, with comparable average word counts (2.5 vs. 3.2 words per utterance) across four non-neutral emotion categories. confidence: high relevance: high - claim_id: training_a_separate_modality_independent_representation_for_masked role: complicates claim: Training a separate, modality-independent representation for masked acoustic cues is necessary; forcing a single representation to jointly encode textual rationale and acoustic detail degrades performance. source: §5.3, Table 4 evidence: Removing the dedicated acoustic embedding and relying only on the phoneme-level binary rationale label drops SEA by 6.94 points relative to the full model; using full (unmasked) audio as the embedding target rather than selectively masked audio drops SEA by a further 9.59 points relative to the full model. confidence: high relevance: low - claim_id: llm_based_emotion_tag_prediction_accuracy_varies_substantially role: complicates claim: LLM-based emotion tag prediction accuracy varies substantially across emotion categories and is markedly worse for emotions underrepresented in the training distribution. source: §5.2, Appendix A, Figure 4 evidence: Stage 1 prompting holistic emotion tag accuracy is 84% for happiness and 74% for surprise but only 57% for anger and sadness, the two emotion categories that together make up under 7% of the labelled training utterances. confidence: high relevance: medium limitations: - 'The training and evaluation data are heavily emotion-imbalanced: only 20.4% of DailyTalk utterances carry a non-neutral emotion label, and within that subset happiness accounts for 85.8% while anger is only 2.55%. Results for the lowest-accuracy, lowest-frequency emotion categories (anger, sadness) should be treated with caution.' - The paper's own limitations section flags that the approach assumes GPT-3.5 can reliably correlate text-based word selection with the acoustic features a human speaker would naturally emphasize when reading the same text aloud, a capability the authors acknowledge is not independently verified beyond the single-annotator overlap study and the Llama-3 generalizability check. The evaluation uses only one dataset (DailyTalk) and one TTS backbone (FastSpeech2); whether the selective masking loss transfers to autoregressive or diffusion-based backbones, or to languages other than English, is untested. No code or demo availability is reported in the paper. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: 2025.findings-naacl.471 published_date: "2025-04-29" entry_date: '2026-07-25' year: 2025 venue: NAACL task: - evaluation architecture: [] relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - prosody_analysis_and_evaluation_method claims: - claim_id: prosodic_cues_in_natural_speech_carry_information_sufficient role: supports claim: Prosodic cues in natural speech carry information sufficient to partially guide spoken comprehension models above chance level. source: §4.1, Table 2 evidence: A WavLM-based SQA model trained and tested on 300 Hz low-pass filtered (prosodic condition) audio achieves FF1 18.49 on SLUE-SQA-5 test, substantially above the white-noise chance baseline of 6.03. confidence: high relevance: high - claim_id: ssl_based_spoken_language_models_do_not_effectively role: complicates claim: SSL-based spoken language models do not effectively leverage prosodic information when lexical cues are simultaneously available, even when lexical data is a small minority of training. source: §4.2, Figure 5 evidence: When only 10% of training data carries lexical information and 90% is prosodic, models rapidly shift toward lexical strategies, with lexical and natural evaluation losses converging near prosodic-only levels within training. confidence: high relevance: high - claim_id: the_use_of_tts_synthesised_speech_in_sqa role: complicates claim: The use of TTS-synthesised speech in SQA training data may not capture the prosodic characteristics of natural speech, limiting the study of prosody in comprehension models. source: §1, §3.1 evidence: The paper selects SLUE-SQA-5 specifically because prior SQA datasets relied on TTS synthesis, whose prosodic properties differ from natural speech; the prosodic condition experiments are only valid under the natural-speech assumption. confidence: high relevance: high - claim_id: prosodic_cues_in_sqa_are_at_least_partially role: refines claim: Prosodic cues in SQA are at least partially question-sensitive rather than globally highlighting salient passage regions. source: §4.1, Table 4 evidence: Random pairing of questions and contexts in the prosodic condition reduces FF1 from 18.49 to 9.77, showing the model's prosodic utilization depends on question-context alignment, not only on passage-level salience signals. confidence: high relevance: high limitations: - 'The prosodic and lexical conditions cannot achieve perfect information separation: the 300 Hz low-pass filter retains some residual lexical information (WER ~57% at 300 Hz cut-off), and flattening F0/intensity preserves rhythm. Results for the prosodic condition therefore reflect a lower bound on prosodic-only performance, and the lexical condition''s prosodic residue may slightly inflate cross-condition scores.' - The study uses extractive SQA where answers are timestamped spans in spoken passages. This frames prosody as a localization cue; its role in open-ended or inferential comprehension tasks is not addressed. The choice of deeper WavLM layers (stronger semantic encoding) likely underrepresents prosodic information relative to earlier layers, which may encode prosody more directly. The corpus (Spoken Wikipedia) is read-aloud speech, so findings may not generalize to conversational or spontaneous speaking styles where prosody is more variable and communicatively richer. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.naacl-long.242 published_date: "2025-04-29" entry_date: '2026-07-25' year: 2025 venue: NAACL task: - TTS architecture: - diffusion - GAN - VAE - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - stochastic_generative_prosody_model claims: - claim_id: compressing_prosodic_variation_into_a_fixed_length_discrete role: complicates claim: Compressing prosodic variation into a fixed-length discrete latent before diffusion modeling makes one-step distillation feasible with a small simulation budget, whereas bootstrapping-based distillation methods require significantly more samples or fail entirely on variable-length latents. source: §3.3, Appendix A.3, Table 6 evidence: This diffusion model is then distilled to one step via a simulation-based approach with perceptual loss (prosody decoder output as perceptual metric), requiring only 10k samples. confidence: high relevance: high - claim_id: codec_free_end_to_end_waveform_synthesis_achieves role: supports claim: Codec-free end-to-end waveform synthesis achieves competitive or superior human-rated naturalness and similarity compared to codec-dependent systems, while adversarial training with multimodal discriminators compensates for the quantisation reconstruction gap. source: §3.1, §3.4, §4.2, Table 1 evidence: 'Training uses mel reconstruction loss, a speaker embedding feature matching (SEFM) loss for timbre fidelity, plus two multimodal discriminators: one for the waveform decoder (conditioned on all decoder inputs) and one for the prosody decoder.' confidence: high relevance: low - claim_id: jointly_encoding_the_speech_prompt_and_input_text role: supports claim: Jointly encoding the speech prompt and input text into prompt-aligned embeddings alongside a global style vector provides larger naturalness and similarity gains than using either conditioning alone in zero-shot TTS. source: §4.3, Table 3 evidence: Combined with prompt-aligned text embeddings (PATE) — a joint encoding of the input text and prompt speech — to produce richer speaker conditioning. 2. confidence: high relevance: medium - claim_id: automatic_speaker_embedding_similarity_metrics_can_diverge_from role: supports claim: Automatic speaker embedding similarity metrics can diverge from human-rated similarity in zero-shot TTS when codec-free systems are compared to codec-dependent ones, suggesting that codec representations may be tuned to neural-network perceptions rather than human perception. source: §4.2, Table 1 evidence: 'On LibriSpeech test-clean with 3-second reference prompts (Table 1, 40 unseen speakers): - StyleTTS-ZS (LibriLight) achieves CMOS-N = 0.00 (best), outperforming NaturalSpeech 3 (CMOS-N = -0.28), FlashSpeech (CMOS-N = -0.42), and VALL-E (CMOS-N = -1.07). - WER: 0.79% — best across all compared models, better than ground truth (0.34% is GT itself). - UT-MOS: 4.16, RTF: 0.03 (vs. 0.30 for NaturalSpeech 3, 0.62 for VALL-E). - Speaker embedding SIM: 0.56 — competitive with most, below NaturalSpeech 3 (0.66), which the authors attribute to perceptual vs. neural-network-centric similarity measures.' confidence: high relevance: low limitations: - The model is English-only (LibriTTS and LibriLight are English audiobook corpora), limiting multilingual applicability. Speaker embedding SIM is somewhat lower than NaturalSpeech 3, suggesting the codec-free approach may still leave timbre fidelity on the table for neural-network-centric metrics. The GAN-based acoustic synthesizer does not benefit from iterative refinement; the authors acknowledge that replacing it with a distilled diffusion component could improve quality. The RTF advantage is significant but the prosody RVQ (fixed K=50) has limited capacity for very long or very short utterances. Distillation quality degrades compared to the full diffusion model at very small sample sizes (1k–5k), meaning the 10k minimum is a practical requirement. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: 2025.naacl-long.484 published_date: "2025-04-29" entry_date: '2026-07-25' year: 2025 venue: NAACL task: - SCA - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control claims: - claim_id: a_single_stream_control_token_representation_for_full role: supports claim: A single-stream control-token representation for full-duplex dialogue can match or exceed a doubled-channel alternating representation in both naturalness and behavioral adherence, while using fewer tokens per unit of audio duration. source: §7.1, §7.2, Table 4, Table 5 evidence: On the same Llama3.2-1B backbone and Behavior-SD training data, the streamlined-unit variant scores 4.09 naturalness MOS and 0.58 interruption adherence vs. 3.90 MOS and 1.21 interruption adherence for the alternating-unit variant. confidence: high relevance: high - claim_id: conditioning_a_speech_prompted_tts_synthesizer_on_a role: supports claim: Conditioning a speech-prompted TTS synthesizer on a speaker's very first utterance, in addition to their most recent utterance, materially improves long-range speaker identity consistency in independently synthesized multi-turn dialogue. source: §7.3, Table 6 evidence: WavLM-Base+ cosine similarity between the prompt and synthesized speech rises from 0.680 (no prompt) to 0.889 when conditioning on the first utterance alone, and combining first-utterance and previous-utterance prompts yields the best transition smoothness (0.872) without degrading global identity (0.885). confidence: high relevance: low - claim_id: pretraining_a_dialogue_generation_model_on_text_only role: complicates claim: Pretraining a dialogue generation model on text-only dialogue before fine-tuning on speech-unit sequences improves adherence to narrative content but is not necessary for behavioral adherence. source: §7.2, Table 5 evidence: Removing the text-dialogue pretraining stage drops narrative adherence (GPT-4o-rated) from 3.11 to 2.80, while behavioral adherence scores for filler words, backchannels, and interruptions remain comparable (0.10/0.87/0.64 vs. 0.15/0.87/0.58 with pretraining). confidence: high relevance: low - claim_id: cascaded_llm_then_tts_pipelines_for_spoken_dialogue role: complicates claim: Cascaded LLM-then-TTS pipelines for spoken dialogue generation can achieve strong narrative coherence and sound quality but struggle to maintain conversational meaningfulness because they lack any mechanism to track behavioral context (speaker turn identity, backchannel placement) across the full dialogue. source: §7.1, §6.2, Table 3 evidence: GPT-4o and Llama3-70B cascaded baselines achieve the highest narrative adherence scores (4.58 and 4.09) but their meaningfulness MOS (3.97 for both) is lower than the proposed model (4.04), attributed by the authors to speaker confusion and misplaced backchannels in cascaded outputs. confidence: high relevance: low limitations: - All quantitative evaluation (human MOS, behavioral adherence, speaker consistency) is conducted entirely on Behavior-SD's own synthetic test split, which is itself generated by the same LLM and TTS pipeline used for training data. No evaluation is reported on real, human-recorded spoken dialogue, so it remains untested whether behavioral adherence and naturalness gains transfer outside the paper's own synthetic data distribution. - The authors note occasional mispronunciation, limited control over complex emotions beyond the five discrete categories used for style captioning, and limited non-lexical vocalization beyond laughter. The word error rate on the Behavior-SD test split (3.55%, measured via Whisper-Large V3) is higher than the underlying CosyVoice TTS system's WER on LibriSpeech (2.89%), attributed to filler-word insertion and proper-name misrecognition rather than a fundamental synthesis quality gap. The dataset and model are limited to two-speaker dialogues; the authors flag multi-speaker extension as future work. Behavior-SD's narratives derive from SODA, which itself derives from a smaller set of social-commonsense scenarios, so diversity of conversational topics is bounded by that source. As with any LLM/TTS-synthesized dataset, the authors flag inherited biases from training data and the risk of misuse for voice impersonation or deepfake audio. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2507.06235' published_date: "2025-05-20" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: [] relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - signal_level_pitch_modification - voice_quality_and_paralinguistic_feature_control claims: - claim_id: fundamental_frequency_and_lower_formant_frequencies_are_primary role: supports claim: Fundamental frequency and lower formant frequencies are primary acoustic predictors of perceived cuteness in synthetic voices. source: §4.6.1 evidence: Phase 1 Spearman correlations show strong positive relationships between kawaii ratings and F0 (rs=0.89) and F1 (rs=0.74), while F2 (p=0.08) and F3 (p=0.25) are non-significant. confidence: high relevance: high - claim_id: acoustic_frequency_manipulations_that_amplify_cuteness_perceptions_in role: complicates claim: Acoustic frequency manipulations that amplify cuteness perceptions in generative TTS voices do not transfer to naturally recorded or professionally processed voices. source: §5.4, Table 6 evidence: The three-semitone F0/formant shift that improved kawaii for TTS voices (Phase 1) significantly reduced kawaiiness for game character voices in Phase 2 (H3 rejected, U-test p<0.001, r=0.07), attributed to processing artefacts and possible ceiling effects in professionally crafted voices. confidence: high relevance: low - claim_id: automated_vocoder_based_pitch_and_formant_shifting_matches role: complicates claim: Automated vocoder-based pitch and formant shifting matches manual audio editing for cuteness perception but diverges on secondary perceptual attributes. source: §4.6.2, Table 3, Table 4 evidence: Wilcoxon signed-rank tests found no significant differences in kawaii ratings across Cubase, Legacy-STRAIGHT, and WORLD for all five TTS voices, but significant differences emerged for humanlikeness, animal-likeness, trustworthiness, and excitedness in several voices. confidence: high relevance: high - claim_id: perceived_cuteness_of_synthetic_voices_is_more_strongly role: supports claim: Perceived cuteness of synthetic voices is more strongly predicted by social desirability attributes than by acoustic frequency measures in isolation. source: §5.4, Table 7 evidence: GEE analysis across game character voices found favorability (coef=0.51), humanlikeness (coef=0.19), familiarity (coef=0.14), and trustworthiness (coef=0.13) as significant predictors of kawaiiness, with effect sizes that exceed those of frequency correlations. confidence: high relevance: low limitations: - All four study phases recruited exclusively Japanese adults via Yahoo! Crowdsourcing Japan, with no participants under 18 and a participant pool skewed toward ages 35-54. Generalisability of the kawaii vocalics manipulation to non-Japanese listeners, younger audiences, or voices in languages other than Japanese is untested. - The paper lacks a validated measurement scale for kawaii, relying on a single Likert item from prior work. Perceptual ratings across a large number of voice clips per session introduce potential fatigue and order effects that could not be fully controlled despite randomisation. The study treats F0 and formants as independent variables but does not control for interactions between them or for speech content effects. The automated manipulation pipeline (WORLD, Legacy-STRAIGHT) introduces resynthesis artefacts that may confound perceptual ratings, as the authors acknowledge. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2507.01611' published_date: "2025-07-02" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - signal_level_pitch_modification claims: - claim_id: explicit_spectral_envelope_modeling_in_a_neural_vocoder role: supports claim: Explicit spectral envelope modeling in a neural vocoder enables reliable pitch modification beyond the training frequency range, where purely neural waveform predictors fail. source: §V.B, §V.D, Tables II, VI, Fig. 6 evidence: QHARMA-GAN maintains f0 RMSE of 0.08-0.11 Hz across pitch-scale factors from 0.5x to 2x on VCTK, while Vocos fails catastrophically (0.65-0.67 Hz); soprano singing voices from an unseen language are reproduced with full harmonic range, whereas HiFi-GAN and Vocos generate only low-frequency harmonics. confidence: high relevance: high - claim_id: incorporating_classical_signal_processing_constraints_into_a_neural role: supports claim: Incorporating classical signal-processing constraints into a neural vocoder reduces data requirements without sacrificing subjective quality. source: §V.D, Table VII evidence: QHARMA-GAN trained on 919 LJSpeech utterances achieves MOS 3.85 compared to HiFi-GAN's 3.53 under the same data constraint; the gap is attributed to HiFi-GAN's overfitting on insufficient data, while QHARMA-GAN's hybrid structure provides analytical priors that reduce the burden on the neural component. confidence: high relevance: low - claim_id: hybrid_vocoders_that_analytically_reconstruct_phase_can_achieve role: complicates claim: Hybrid vocoders that analytically reconstruct phase can achieve higher subjective naturalness than end-to-end neural vocoders despite lower PESQ scores, suggesting objective spectral distance metrics do not fully capture perceptual frequency smoothness. source: §V.B, Table III evidence: QHARMA-GAN scores MOS 4.21 on VCTK vs HiFi-GAN's 4.08, while PESQ is comparable (3.14 vs 3.14) and both trail QHM's PESQ of 3.45 despite QHM scoring the lowest subjective MOS (4.07); the divergence is attributed to QHARMA-GAN's ability to reduce frequency distortion that HiFi-GAN exhibits in some samples. confidence: high relevance: low - claim_id: v_uv_detection_errors_are_a_critical_bottleneck role: complicates claim: V/UV detection errors are a critical bottleneck for hybrid pitch-modification vocoders, particularly at extreme pitch-raising factors. source: §V.B, Table IV evidence: QHARMA-GAN outperforms WORLD in pitch lowering (MOS 3.01 vs 2.98 at rho=0.5x on VCTK) but underperforms WORLD in pitch raising (2.72 vs 2.82 at rho=2x), attributed to V/UV misclassification reducing harmonic component count in unvoiced segments during upward pitch shifts. confidence: high relevance: high limitations: - 'Accurate V/UV detection is an unsolved dependency: QHARMA-GAN''s pitch modification quality is bounded by the accuracy of the V/UV classifier, which the paper treats as a fixed external component. The authors identify this as "an urgent research topic for the future," but no solution is proposed.' - QHARMA-GAN requires an additional f0 predictor module when used as part of a TTS backend, unlike mel-spectrogram-to-waveform vocoders that take mel-spectrograms directly as input. This additional module adds pipeline complexity and introduces a potential error source in the f0 prediction step. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2507.03912' published_date: "2025-07-05" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: [] relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - linguistic_context_prosody_prediction - language_and_domain_specific_prosody_modeling - prosody_analysis_and_evaluation_method claims: - claim_id: combining_acoustic_foundation_model_features_with_phoneme_level role: supports claim: Combining acoustic foundation model features with phoneme-level linguistic BERT features improves automatic prosody label prediction over either modality alone. source: §5.4, Table 1 evidence: On the CSJ corpus, HuBERT-base + PnG BERT achieves 89.8% ACC accuracy versus 89.0% with HuBERT alone and 82.5% with PnG BERT alone; consistent gains appear on HL and BI labels. confidence: high relevance: high - claim_id: ssl_speech_features_substantially_outperform_hand_crafted_acoustic role: supports claim: SSL speech features substantially outperform hand-crafted acoustic features for automatic prosody annotation. source: §5.4, Table 1 evidence: HuBERT-base achieves 89.8% ACC accuracy versus 75.2% for melspectrogram and 62.9% for F0 alone on CSJ; the gap widens further when comparing macro F1 scores on rare prosodic classes. confidence: high relevance: high - claim_id: language_matched_pre_training_benefits_ssl_model_utility role: supports claim: Language-matched pre-training benefits SSL model utility for prosody prediction in a pitch-accent language. source: §5.6, Table 2 evidence: Japanese-trained HuBERT-base and wav2vec2.0-base outperform English and multilingual SSL variants across all four prosodic label types on CSJ, though margins are modest (under 1 percentage point on most labels). confidence: high relevance: high - claim_id: prosodic_boundary_and_pause_signals_cannot_be_reliably role: complicates claim: Prosodic boundary and pause signals cannot be reliably estimated from text alone; explicit acoustic evidence is required. source: §5.5 evidence: Without acoustic input, pause presence prediction fails to distinguish short pauses; boundary pitch movement symbols in ACC labels show higher error rates than when HuBERT features are included. confidence: high relevance: high limitations: - The paper does not evaluate whether the predicted prosodic labels improve downstream TTS naturalness or prosody controllability. All results are prosody annotation accuracy numbers on CSJ; the claimed benefit to TTS training remains unvalidated. - The system is evaluated only on Japanese, a pitch-accent language with specific prosodic structure. Applicability to other languages with different prosodic hierarchies (e.g., stress-timed languages with ToBI conventions) is not tested. The annotation model depends on accurate phoneme alignments from a forced aligner; errors in alignment propagate directly into label prediction. Finally, CSJ is a monologue corpus of academic lectures; generalization to conversational or spontaneous speech styles with different prosodic distributions is an open question. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2507.08012' published_date: "2025-07-05" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer claims: - claim_id: latent_controllable_features_absent_from_a_tts_model role: supports claim: Latent controllable features absent from a TTS model's training annotations can be discovered by applying PCA to embeddings of fixed-input generated samples and enrolling the identified dimensions as new description-prompt labels. source: §4.3, Tables 2–3 evidence: For T3 (no emotion labels), PCA of 1,000 Wav2Vec2 embeddings revealed emotional intensity as the primary variance axis; iterative re-labelling and fine-tuning progressively separated neutral from emotive utterances, improving neutral cluster assignment from 59.3% to 89.3% across rounds. confidence: high relevance: medium - claim_id: variance_based_feature_discovery_in_prompt_based_tts role: complicates claim: Variance-based feature discovery in prompt-based TTS is unreliable when the model's output distribution is highly constrained by existing conditioning labels. source: §4.4, Figure 6 evidence: T3-emotion, already trained with explicit emotion and intensity labels, generated highly consistent F0 contours under neutral-emotion prompts (Figure 6), leaving insufficient variance for meaningful feature discovery in the fixed-input analysis set. confidence: high relevance: medium - claim_id: embedding_based_clustering_of_tts_output_variance_can role: complicates claim: Embedding-based clustering of TTS output variance can surface acoustic artefacts of the training corpus (such as recording-condition variation) rather than prosodic features of interest. source: §4.4, Figure 8 evidence: For T3-emotion with a diverse analysis set, the principal component correlated strongly with GeMaps-v01b loudness features attributable to microphone distance differences in the Talromur-3 corpus, not to prosody. confidence: high relevance: high - claim_id: self_supervised_speech_representations_suitable_for_prosody_analysis role: complicates claim: Self-supervised speech representations suitable for prosody analysis are inherently entangled with speaker identity and linguistic content, requiring fixed-input generation sets to isolate prosodic variation. source: §3.2, Figure 1 evidence: Wav2Vec2 summary embeddings cluster by target text and speaker identity regardless of which network layer is used (Figure 1), motivating the fixed-input analysis design where text, speaker, and prompt are held constant. confidence: high relevance: high limitations: - No subjective listening tests are reported. All quality and controllability assessments use automatic metrics (ASR-based WER, speaker embedding cosine similarity, diversity score). Whether the discovered features correspond to perceptually meaningful and user-controllable dimensions is not established. - The method is evaluated on a single Icelandic speaker (Ingrid) for the fine-tuning stages, with no cross-speaker or cross-language generalisation experiments. The paper does not address why T3-emotion fails and T3 succeeds beyond noting the former's consistent output distribution; whether adding a more diverse analysis set would help, or whether an alternative embedding choice (not Wav2Vec2) would be less sensitive to recording conditions, remains open. The use of Icelandic as the sole test language, while noted as language-agnostic in principle, is unverified on any other language. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2507.04598' published_date: "2025-07-07" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - emotion_expressive_prosody_control claims: - claim_id: hierarchical_dependency_aware_emotion_prediction_yields_better_emotional role: supports claim: Hierarchical, dependency-aware emotion prediction yields better emotional naturalness and intelligibility than treating prosodic granularities independently. source: §5.1.1, Table 1 evidence: VA (Multi-Step) achieves MUSHRA naturalness 62.2 vs. 57.5 (single-step VA) with ground-truth ED, and WER 2.45 vs. 4.61 with predicted ED on LibriTTS-R. confidence: high relevance: high - claim_id: multi_level_emotion_intensity_control_when_implemented_at role: supports claim: Multi-level emotion intensity control, when implemented at the phoneme and word levels, enables fine-grained and quantifiable manipulation of prosodic features consistent with perceptual expectations. source: §5.2, Table 4, Figure 6 evidence: BWS controllability tests show the proposed model more consistently associates low intensity with least-expressive and high intensity with most-expressive ratings across all five emotions compared to MsEmoTTS, and prosodic trend analysis (duration, pitch mean/std, energy) confirms expected acoustic correlates at each level. confidence: high relevance: high - claim_id: sequential_multi_step_prediction_across_hierarchical_levels_introduces role: complicates claim: Sequential multi-step prediction across hierarchical levels introduces error accumulation that can degrade pitch and duration alignment even when overall naturalness improves. source: §5.1.2, Table 2 evidence: In the VA setting, multi-step prediction does not outperform single-step on pitch distortion or frame disturbance metrics, attributed to cascaded prediction error and increased sensitivity from joint training with ED difference loss. confidence: high relevance: high - claim_id: the_gap_between_ground_truth_and_text_predicted role: refines claim: The gap between ground-truth and text-predicted emotion distributions at the word level is smaller than at the phoneme level, suggesting that phoneme-level emotion is harder to infer from text. source: §5.1.3, Table 3 evidence: Table 3 shows mean absolute ED differences are lower for words than phonemes in the multi-step (Predicted) condition, and the authors interpret this as the model prioritizing cross-segment dependencies over precise ED matching. confidence: high relevance: medium limitations: - The system is evaluated exclusively on English data, and the authors identify extension to additional languages as future work. Both training datasets (LibriTTS-R, ESD) are relatively clean, and the method's robustness in noisy or spontaneous conditions is untested. The MsEmoTTS baseline is re-implemented within the FastSpeech2 framework rather than evaluated in its original form, which introduces a potential confound in the emotion editing comparison. The external module's claim of compatibility with arbitrary TTS systems is not verified against any system other than FastSpeech2. Model size and computational cost are not reported. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2507.04817' published_date: "2025-07-07" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - VC architecture: - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - disentangled_prosody_content_timbre - emotion_expressive_prosody_control claims: - claim_id: explicit_interpretable_prosodic_conditioning_in_voice_conversion_can role: supports claim: Explicit, interpretable prosodic conditioning in voice conversion can match the intelligibility and speaker similarity of systems using implicit or disentangled representations, at a fraction of the parameter cost. source: §5.1.1, Table 2 evidence: Fast-VGAN (~3.2M params) achieves WER of 0.000% and cosine similarity 0.648, competitive with ControlVC (~20M params, WER 0.089%, similarity 0.652) and superior to HiFi-VC (~14M params, WER 2.857%) on seen-speaker VCTK conversion. confidence: high relevance: high - claim_id: adapting_target_speaker_specific_prosodic_parameters_pitch_ambitus role: complicates claim: Adapting target-speaker-specific prosodic parameters (pitch ambitus, speech rate) during voice conversion does not reliably improve perceived speaker identity, and combining multiple prosodic adaptations simultaneously degrades both naturalness and similarity. source: §5.2.1, Table 4 evidence: When ambitus and speech rate dilation are applied jointly, subjective MOS naturalness drops from 3.63 to 3.13 and speaker similarity from 3.47 to 2.99, compared to baseline Fast-VGAN conversion without prosodic parameter adaptation. confidence: high relevance: high - claim_id: neutral_to_expressive_speech_resynthesis_can_be_achieved role: supports claim: Neutral-to-expressive speech resynthesis can be achieved by applying prosodic contours from expressive reference recordings to a model trained exclusively on neutral speech, without requiring expressive training data. source: §4.3, §5.1.3, Table 3 evidence: Fast-VGAN trained only on neutral Expresso speaker utterances achieves 0.867 ± 0.029 cosine speaker similarity on expressive resynthesis across confused, happy, and sad conditions, with WER (19.46%) comparable to real recordings (19.38%). confidence: high relevance: high - claim_id: extreme_prosodic_transformations_in_gan_based_voice_conversion role: complicates claim: Extreme prosodic transformations in GAN-based voice conversion degrade naturalness and speaker similarity, with performance falling off on both sides of the neutral setting in a bell-shaped pattern. source: §5.2.2, Figure 3 evidence: MOS scores for naturalness and speaker similarity peak at unmodified (neutral) settings and decline for both compression and expansion of duration (up to 3x/0.33x), ambitus (up to ±1 octave), and F0 transposition (up to ±1 octave), as measured in subjective listening tests with approximately 20 participants per condition. confidence: high relevance: high limitations: - 'Speaker generalisation is restricted to seen speakers: speaker identity is encoded as a learned lookup embedding, so the model cannot convert to unseen target speakers without retraining. The evaluation uses only 8 VCTK speakers with 2 utterances per speaker pair -- a narrow test set that may not reflect performance across the full VCTK diversity or cross-corpus speakers.' - No demo or code is reported, limiting reproducibility. The speaker similarity metric uses Resemblyzer cosine embeddings, which may not correlate well with human speaker identity judgements; the subjective similarity results already show some misalignment (Fast-VGAN subjective similarity 3.47 vs ControlVC 2.82, while objective cosine similarity is nearly tied at 0.648 vs 0.652). The expressive synthesis experiment uses only 4 Expresso speakers (2M, 2F) and 3 emotions, leaving generalisation to other speakers and emotional categories open. The MBExWN vocoder introduces its own quality ceiling, and the combined system has not been evaluated on noisy or telephony-quality input speech. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2507.09310' published_date: "2025-07-12" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - VC - TTS architecture: - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control claims: - claim_id: implicit_style_modeling_via_an_auxiliary_reconstruction_loss role: supports claim: Implicit style modeling via an auxiliary reconstruction loss can preserve speaking style characteristics in voice conversion without requiring explicit acoustic feature extraction. source: §3.3, §3.4, Table 2 evidence: A style reconstruction loss using a frozen Lombard/neutral classifier achieves SIIB scores comparable to explicit conditioning on f0, mgc0, and mgc1 across male and female target speakers, with no statistically significant differences in most conditions. confidence: high relevance: medium - claim_id: the_effectiveness_of_explicit_prosodic_features_for_style role: complicates claim: The effectiveness of explicit prosodic features for style-preserving voice conversion depends on speaker gender, limiting the generalizability of feature-based conditioning approaches. source: §3.4, Table 2 evidence: F0 conditioning improves SIIB for male target speakers but is detrimental for female target speakers at both SNR-1 and SNR-3; spectral tilt and energy (mgc0+mgc1) are beneficial across genders. confidence: high relevance: high - claim_id: speaker_identity_remains_perceptible_to_listeners_in_noisy role: supports claim: Speaker identity remains perceptible to listeners in noisy conditions, confirming that speaker similarity is a relevant evaluation dimension for intelligibility-enhancing voice conversion. source: §2.3, Figure 1, §3.4, Figure 6 evidence: Pilot study results show that listeners maintain consistent similarity rankings between speakers at SNR-1 and SNR-3 noise levels, and the implicit conditioning VC model preserves speaker similarity comparably to the CopyCat baseline. confidence: high relevance: low - claim_id: combining_implicit_and_explicit_style_conditioning_does_not role: complicates claim: Combining implicit and explicit style conditioning does not consistently improve intelligibility over either approach alone, and can degrade performance in some configurations. source: §3.4, Figure 5, Table 2 evidence: Fusion of Ls with individual explicit features (e.g., Ls+f0 for female targets) reduces SIIB compared to Ls alone; only Ls+mgc0+mgc1 reliably improves over Ls in most conditions, and gains are inconsistent across genders. confidence: high relevance: medium limitations: - The model does not modify duration, which is an observed acoustic correlate of the Lombard effect, potentially leaving an intelligibility benefit unexploited. The dataset is small (54 speakers, 100 short utterances each) and recorded under a constrained sentence task that produces somewhat robotic prosody, limiting the naturalness of synthesised samples. Only two target speakers are evaluated in the VC experiments, making it difficult to assess robustness across a wider voice population. The authors propose extending to larger conversational datasets, evaluating downstream Lombard TTS intelligibility using transcriptions and hearing-loss-targeted metrics, and exploring adversarial disentanglement to control Lombard intensity independently of speaker identity. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: 2025.acl-industry.42 published_date: "2025-07-27" entry_date: '2026-07-25' year: 2025 venue: ACL task: - TTS architecture: - GAN - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - linguistic_context_prosody_prediction - language_and_domain_specific_prosody_modeling claims: - claim_id: accurate_grapheme_to_phoneme_conversion_is_the_single role: supports claim: Accurate grapheme-to-phoneme conversion is the single most important preprocessing component for intelligibility in tonal-language TTS, contributing more to WER reduction than tokenisation or pause prediction combined. source: §6, Table 3 evidence: Ablation confirms the G2P module contributes most (removing it raises WER from 6.3% to 22.5%), followed by tokenization (WER 10.2%) and pause prediction (WER 6.5%, NMOS 3.8). confidence: high relevance: high - claim_id: proprietary_general_purpose_tts_systems_degrade_substantially_on role: supports claim: Proprietary general-purpose TTS systems degrade substantially on domain-specific vocabulary, while systems trained with targeted vertical-domain data show much smaller performance drops in specialised settings. source: §6, Table 2 evidence: 'On domain-specific tasks (finance, healthcare, education, law), the advantage is even clearer: proprietary systems degrade sharply (Google WER jumps to 14.5%), while the proposed system shows much smaller degradation (domain WER 6.5%).' confidence: high relevance: low - claim_id: low_resource_tts_systems_can_achieve_competitive_speaker role: supports claim: Low-resource TTS systems can achieve competitive speaker similarity in zero-shot voice cloning relative to voice conversion models when trained with a style embedding module, even without speaker-specific fine-tuning. source: §6, Table 4 evidence: 'TTS Model: - Three pretrained feature extractors (trained on AiShell, LibriSpeech, JVS, KsponSpeech, then fine-tuned on Thai) supply forced-alignment, pitch, and energy ground truth. - A style encoder extracts a latent style vector from the input waveform for zero-shot voice cloning. - A GAN-based waveform decoder reconstructs speech from phoneme embeddings, duration/pitch/energy features, and the style vector.' confidence: high relevance: medium - claim_id: contextual_prosody_modelling_via_a_tonal_phoneme_aware role: supports claim: Contextual prosody modelling via a tonal phoneme-aware BERT encoder improves naturalness scores in tonal-language TTS beyond what standard duration and pitch predictors achieve. source: §5 evidence: Loss is L1 (time) + mel spectrogram (frequency) + GAN perceptual. - A Phoneme-Tone BERT (12-layer, 768 hidden, trained on 1M Thai sentences with masked LM) provides contextual prosody representations for three predictors (duration, pitch, energy). confidence: high relevance: high limitations: - '- The approach has been validated only on Thai; generalization to other tonal or low-resource languages is claimed but not demonstrated. - Model size and full architecture details are not reported, limiting reproducibility. - The zero-shot comparison is against OpenVoice (a VC model) rather than other zero-shot TTS systems. - Data collection from TikTok and YouTube raises questions about license and long-term data availability.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.acl-long.1471 published_date: "2025-07-27" entry_date: '2026-07-25' year: 2025 venue: ACL task: - evaluation architecture: [] relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - linguistic_context_prosody_prediction - implicit_scale_and_data_prosody_modeling claims: - claim_id: prosodic_features_such_as_pitch_loudness_and_prominence role: supports claim: Prosodic features such as pitch, loudness, and prominence are redundant with an extended past linguistic context of 3–8 words, but are redundant with future context only over a short span of 1–2 words. source: §5, §7 evidence: 'For pitch, loudness, and prominence (both absolute and relative): - Past context: MI increases until ~3–8 words, then plateaus — consistent with the long-scale past redundancy hypothesis. - Future context: MI saturates after 1–2 words — consistent with the short-scale future redundancy hypothesis. - Past MI is higher than future MI at all context lengths above 1 word.' confidence: high relevance: high - claim_id: duration_and_pause_exhibit_the_opposite_asymmetry_to role: supports claim: 'Duration and pause exhibit the opposite asymmetry to other prosodic features: they carry little redundancy with past context beyond word identity, but show stronger redundancy with future context, likely due to sentence-boundary predictability.' source: §5, §6.4 evidence: 'For duration and pause: - Past context adds little beyond the word identity itself (MI does not increase with n > 0). - Future context saturates after ~4 words for pause, ~1 word for duration. - Duration and pause have higher MI with future context than with past context, opposite to other features.' confidence: high relevance: high - claim_id: fine_tuned_masked_language_models_provide_a_viable role: supports claim: Fine-tuned masked language models provide a viable framework for quantifying how much linguistic context is needed to predict prosodic features, with mutual information serving as an interpretable upper-bounded estimator. source: §3.1, §4.2 evidence: The study quantifies the mutual information MI(P_t; W_{n,m}) between a prosodic feature P_t at word t and a linguistic context window W_{n,m} of n past words and m future words (including the word itself), systematically varying n and m from 0 to 9. confidence: high relevance: high - claim_id: the_asymmetric_time_scale_of_prosody_context_redundancy role: supports claim: 'The asymmetric time scale of prosody-context redundancy implies that prosody serves at least two distinct communicative functions: reinforcing information from longer-range past context and signaling short-term upcoming structure.' source: §6.2, §6.3, §7 evidence: 'For pitch, loudness, and prominence (both absolute and relative): - Past context: MI increases until ~3–8 words, then plateaus — consistent with the long-scale past redundancy hypothesis. - Future context: MI saturates after 1–2 words — consistent with the short-scale future redundancy hypothesis. - Past MI is higher than future MI at all context lengths above 1 word.' confidence: high relevance: high limitations: - '- Dataset limited to English audiobook narration (LibriTTS), which may exhibit higher prosody-context redundancy than spontaneous speech. - Mutual information values are cross-entropy upper bounds; estimates may be loose for short-span BERT inputs. - Single model for all (n,m) combinations may be suboptimal for any specific combination. - Duration/pause results are noisy due to the highly skewed pause distribution (89.4% are zero seconds); a zero-inflated model is suggested for future work. - Open question: how do these time scales generalize across languages? (Companion work Wilcox et al. 2025 begins to address this.)' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.acl-long.346 published_date: "2025-07-27" entry_date: '2026-07-25' year: 2025 venue: ACL task: - TTS architecture: - transformer-enc-dec - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - disentangled_prosody_content_timbre - emotion_expressive_prosody_control claims: - claim_id: disentangling_speaker_timbre_and_speaking_style_into_separate role: supports claim: Disentangling speaker timbre and speaking style into separate codec representations is a necessary condition for simultaneous zero-shot speaker cloning and style control; without explicit decoupling, the two conditioning signals interfere and controllability collapses. source: §4.3, Table 4 evidence: 'The codec decoupling ablation (test set A) is striking: without decoupling, controllability collapses (pitch 0.492, speed 0.517, emotion 0.237), confirming that independently disentangling the speech prompt and style prompt is necessary to prevent cross-channel interference.' confidence: high relevance: medium - claim_id: natural_language_style_descriptions_have_an_inherent_many role: supports claim: Natural language style descriptions have an inherent many-to-many relationship with audio that cannot be resolved by timbre conditioning alone, requiring a probabilistic model of style variation such as a mixture density network. source: §3.3 evidence: On test set C (out-of-domain style descriptions), ControlSpeech degrades much less than baselines across all metrics, indicating substantially better generalization from the SMSD's mixture density mechanism. confidence: high relevance: medium - claim_id: zero_shot_speaker_cloning_capability_in_style_controllable role: supports claim: Zero-shot speaker cloning capability in style-controllable TTS can be achieved by building on a large-scale pre-trained disentangled codec without sacrificing audio quality relative to dedicated zero-shot TTS systems. source: §4.2, Table 2 evidence: ControlSpeech achieves the best speaker similarity (Spk-sv 0.89), WER (2.9%), and MOS-Q (3.91 ± 0.09), outperforming all baselines on audio quality while uniquely providing zero-shot timbre cloning (all style-only baselines lack this capability). confidence: high relevance: medium - claim_id: probabilistic_sampling_from_a_mixture_density_model_of role: supports claim: Probabilistic sampling from a mixture density model of style representations improves both style diversity and generalization to out-of-domain style descriptions compared to deterministic style encoding. source: §4.2, §4.3, Table 3 evidence: On test set C (out-of-domain style descriptions), ControlSpeech degrades much less than baselines across all metrics, indicating substantially better generalization from the SMSD's mixture density mechanism. confidence: high relevance: medium - claim_id: pitch_control_is_measurably_harder_to_preserve_when role: complicates claim: Pitch control is measurably harder to preserve when timbre and style are controlled simultaneously, suggesting that pitch conditioning interacts with speaker identity in ways that speed, volume, and emotion do not. source: §4.2, Table 1 evidence: On VccmDataset test set A (style controllability, using GT timbre as prompt), ControlSpeech achieves the best speed accuracy (0.829 vs. 0.786 for PromptStyle), volume accuracy (0.894 vs. 0.831 for Salle), and emotion accuracy (0.557 vs. 0.412 for InstructTTS). confidence: high relevance: high limitations: - 'The paper explicitly notes two limitations: (1) The training dataset is still limited in scale for style-controllable TTS; significantly larger datasets (tens of thousands of hours with style annotations) may be needed to achieve more advanced controllability. (2) The exploration of generative model architectures is narrow — only non-autoregressive parallel decoding is tried. Diffusion or flow-matching generators operating in the disentangled codec space might offer better quality or diversity.' - 'Beyond the paper''s self-assessment: pitch accuracy is the only metric where ControlSpeech underperforms style-only baselines on both in-domain and out-of-domain test sets. The authors attribute this to simultaneous timbre-style control, but the mechanism is unexplained and unresolved. The paper does not evaluate on standard TTS benchmarks (LibriSpeech, VCTK), relying entirely on VccmDataset, which makes external comparison difficult. The demo availability is not confirmed in the paper. Ethical risks from zero-shot voice cloning are acknowledged but only partially addressed (watermarking is proposed as future work).' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: 2025.acl-long.790 published_date: "2025-07-27" entry_date: '2026-07-25' year: 2025 venue: ACL task: - VC architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - emotion_expressive_prosody_control claims: - claim_id: self_consistency_training_enables_shortcut_flow_matching_to role: supports claim: Self-consistency training enables shortcut flow matching to match full-step quality in voice conversion with as few as two inference steps. source: §4.2, Table 1 evidence: R-VC at NFE=2 matches NFE=10 across all quality metrics (SECS 0.930 vs 0.931, UTMOS 4.1 vs 4.1, QMOS 4.03 vs 4.05, SMOS 4.11 vs 4.12) while reducing inference time by 2.83x; vanilla CFM degrades sharply below 10 steps. confidence: high relevance: low - claim_id: explicit_rhythm_modeling_via_a_target_conditioned_duration role: supports claim: Explicit rhythm modeling via a target-conditioned duration model substantially improves emotion style transfer in zero-shot VC. source: §4.3, §4.5, Table 2, Table 4 evidence: Removing the duration module from R-VC drops the emotion score from 0.59 to 0.425 on the ESD dataset, while also increasing WER slightly; baselines that preserve source rhythm score 0.395-0.489. confidence: high relevance: high - claim_id: fine_grained_duration_prediction_in_non_autoregressive_models role: complicates claim: Fine-grained duration prediction in non-autoregressive models introduces instability in voice conversion that coarser duration strategies do not fully resolve. source: §7, Table 4 evidence: R-VC's masked transformer duration model produces occasional over-extended pronunciations; sentence-level duration as a fallback yields worse WER (9.86 vs 6.95) and UTMOS (3.58 vs 3.85), offering no stability improvement in practice. confidence: high relevance: high - claim_id: data_perturbation_before_discrete_content_tokenisation_reduces_timbre role: supports claim: Data perturbation before discrete content tokenisation reduces timbre leakage more effectively than relying on the self-supervised representation alone. source: §4.5, Table 4 evidence: Removing pitch perturbation before HuBERT token extraction degrades WER from 3.51 to 7.28 and speaker similarity from 0.930 to 0.869, confirming that perturbation actively suppresses content-irrelevant speaker information. confidence: high relevance: low limitations: - 'The masked transformer duration model has a known instability: inaccurate predictions cause over-extended pronunciations. Sentence-level duration as an alternative proved worse in both stability and quality, leaving robust duration modeling as an unresolved challenge.' - The system is evaluated only on English (MLS, LibriSpeech, ESD) and English Seed-TTS subsets; generalisation to cross-lingual or multilingual VC is untested. Training data (20k hours) is smaller than top competitors such as CosyVoice-VC (171k hours), which makes speaker similarity comparisons somewhat favourable to R-VC but also means that high-similarity performance on out-of-distribution accents or recording conditions is unknown. The RTF of 0.12 using 2-step inference is faster than most flow-matching competitors but still 20% slower than non-diffusion methods (FACodec-VC RTF 0.10). caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.unlp-1.11 published_date: "2025-07-31" entry_date: '2026-07-25' year: 2025 venue: UNLP task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - linguistic_context_prosody_prediction - language_and_domain_specific_prosody_modeling claims: - claim_id: context_aware_sentence_level_neural_models_outperform_word role: supports claim: Context-aware sentence-level neural models outperform word-level neural baselines for lexical stress prediction in morphologically complex languages. source: §3.2.4, Table 2 evidence: ByT5 fine-tuned on sentence-level synthetic data achieves 87.7% word-level accuracy, compared to 73.2% for the word-level Ukrainian Accentor, by incorporating broader sentence context during inference. confidence: high relevance: high - claim_id: neural_models_do_not_surpass_dictionary_based_systems role: complicates claim: Neural models do not surpass dictionary-based systems on contextual stress disambiguation even when trained at the sentence level. source: §3.2.4, Table 2 evidence: ByT5 G2P achieves 58.1% ambiguous word accuracy vs. 64.3% for Ukrainian Word Stress (First); the dictionary system's POS-based disambiguation and guaranteed fallback remain advantageous for heteronym resolution. confidence: high relevance: high - claim_id: hybrid_pipelines_combining_neural_and_dictionary_based_stress role: supports claim: Hybrid pipelines combining neural and dictionary-based stress prediction outperform either method alone for low-resource languages. source: §3.2.4, Table 2 evidence: Combining ByT5 G2P with Ukrainian Word Stress (OnAmbiguity.Skip) achieves 92.5% word-level and 52.0% sentence-level accuracy, exceeding both standalone neural (87.7%, 35.3%) and dictionary-only (88.7%, 41.5%) results. confidence: high relevance: high - claim_id: carefully_designed_rule_based_g2p_systems_can_achieve role: supports claim: Carefully designed rule-based G2P systems can achieve very low word error rates for languages with relatively transparent orthographies, substantially outperforming naive mapping. source: §4.4, Table 4 evidence: The rule-based Ukrainian phonemizer reaches 1.23% WER on a constructed phonologically diverse dataset, vs. 48.75% WER for a naïve letter-to-phoneme baseline, with errors concentrated in abbreviations and complex consonant clusters. confidence: high relevance: low limitations: - The phonemizer does not handle abbreviations or numerical expressions, and the stress prediction model is trained entirely on automatically generated pseudo-labels with no manually verified training sentences. Both limitations constrain applicability to controlled text in standard Ukrainian orthography. - The stress model's contextual disambiguation is limited by sparse heteronym representation in the synthetic training corpus, which relies on an ASR-based annotation pipeline with inherent labelling errors. The evaluation benchmark contains only 1,026 sentences, making sentence-level accuracy estimates noisy. Neither system accounts for regional dialects or non-standard language varieties. The phonemizer operates at word level and cannot handle sentence-level phenomena such as cross-word assimilation. Full-text phonemization evaluation is deferred to future work. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.ccl-1.77 published_date: "2025-08-01" entry_date: '2026-07-25' year: 2025 venue: workshop task: - TTS architecture: - diffusion relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - stochastic_generative_prosody_model - emotion_expressive_prosody_control - multimodal_visual_prosody_conditioning claims: - claim_id: cross_modal_facial_features_can_provide_sufficient_speaker role: supports claim: Cross-modal facial features can provide sufficient speaker identity signal for zero-shot voice generation without reference audio. source: §3.2, Table 1, Table 2 evidence: The cross-modal biometric unseen speaker modelling module maps a face image to a speaker embedding via ResNet50 and a trainable visual network; Id.Acc of 66.87/68.99 on LRS2/GRID exceeds all baselines that require reference audio at inference. confidence: high relevance: low - claim_id: diffusion_based_denoising_in_a_multimodal_tts_dubbing role: supports claim: Diffusion-based denoising in a multimodal TTS/dubbing pipeline improves prosodic diversity relative to deterministic autoregressive or attention-based baselines. source: §4.2.4, Figure 4 evidence: Running HFSD-V2C 10 times per speaker produces diverse F0 contours capturing individual accent patterns; Neural Dubber and HPMDubbing produce fixed prosodic distributions. confidence: high relevance: high - claim_id: hierarchical_multimodal_prosody_modelling_video_audio_and_text role: supports claim: Hierarchical multimodal prosody modelling (video, audio, and text levels jointly) improves emotion accuracy over methods relying on fewer modalities. source: §4.2.1, §4.2.5, Table 1, Table 2 evidence: Emo.Acc reaches 65.63 on LRS2 and 66.32 on GRID, against 61.46 and 63.66 for HPMDubbing; the ablation (w/o UP) drops Emo.Acc to 22.08/27.64, confirming the prosody module's role. confidence: high relevance: high - claim_id: zero_shot_visual_voice_cloning_accuracy_remains_significantly role: complicates claim: Zero-shot visual voice cloning accuracy remains significantly below ground-truth speaker identity, indicating that cross-modal biometric embeddings do not fully replace reference audio. source: §4.2.1, Table 1 evidence: GT Id.Acc on LRS2 is 91.52 vs. HFSD-V2C's 66.87; GT MOS is 4.72 vs. 4.29 for HFSD-V2C, a gap of 0.43 that persists after hierarchical multimodal conditioning. confidence: high relevance: low limitations: - The evaluation is restricted to LRS2 and GRID, both of which contain constrained speaking styles (BBC broadcasts and phonetically structured lab speech), leaving generalisation to spontaneous conversational video undemonstrated. The subjective MOS is collected on only 20 clips rated by 20 evaluators, which is a limited sample for drawing robust conclusions. The model trains on both LRS2 and GRID but the zero-shot claim means unseen speakers at test time, not unseen datasets; the extent of genuine out-of-domain generalisation is not assessed. Code and demos are not released, limiting reproducibility. Finally, the method depends on visible, well-lit face images, which may not be robust in natural video production environments. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.sigdial-1.27 published_date: "2025-08-01" entry_date: '2026-07-25' year: 2025 venue: workshop task: - SCA - TTS architecture: - transformer-enc-dec relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control - language_and_domain_specific_prosody_modeling claims: - claim_id: integrating_emotion_conditioned_tts_into_a_spoken_dialogue role: supports claim: Integrating emotion-conditioned TTS into a spoken dialogue system produces measurably more appropriate emotional speech than a neutral TTS baseline. source: §3.5, Table 2 evidence: Speech emotion appropriateness scores were 4.10 (proposed) vs. 1.70 (baseline) with a very large effect size (Cohen's d = 3.07, p < 0.001) across 20 human-judged conversations. confidence: high relevance: medium - claim_id: sentiment_driven_emotional_speech_control_does_not_straightforwardly role: complicates claim: Sentiment-driven emotional speech control does not straightforwardly translate into significantly higher user engagement in short task-oriented interactions. source: §3.5, Table 2 evidence: Engagement scores and number of SDS turns showed large effect sizes favouring the proposed system (d = 0.824, d = 0.831) but failed to reach statistical significance (p = 0.09, p = 0.07) with n = 20 conversations; the authors attribute this partly to abrupt discrete emotion shifts that ignore conversational history. confidence: high relevance: low - claim_id: discrete_context_free_emotion_label_assignment_produces_step role: complicates claim: Discrete, context-free emotion label assignment produces step-change tonal shifts that may undermine conversational naturalness even when individual utterance emotion is appropriate. source: §3.5 evidence: Mean engagement remained around 3.0 (on a 5-point scale), and the authors explicitly note that abrupt emotion transitions without considering prior context may have suppressed engagement gains. confidence: high relevance: medium - claim_id: subjective_evaluation_of_emotional_spoken_dialogue_systems_requires role: supports claim: Subjective evaluation of emotional spoken dialogue systems requires task-specific metrics beyond standard TTS naturalness; generic MOS-type ratings do not capture emotion appropriateness or conversational engagement. source: §3.3 evidence: The paper proposes a seven-item questionnaire covering RAG quality, task achievement, speech emotion appropriateness, and engagement, arguing that existing evaluation frameworks lack appropriate metrics for social goals in emotional SDSs. confidence: high relevance: medium limitations: - The evaluation uses only 20 conversation samples (10 per system) conducted by the authors themselves, which is underpowered for the engagement and turn-count analyses and does not include independent users. Significance could not be established for engagement despite large effect sizes, and the results may not generalise beyond news summarisation. - 'The system uses discrete, context-free emotion tagging: sentiment is predicted independently for each LLM response without tracking emotional trajectory across turns. This produces abrupt shifts and likely suppresses engagement. Future work would benefit from emotion-aware dialogue state tracking and smoother emotion interpolation across conversational turns. Task achievement scores plateaued around 3.0 for both systems, suggesting that retrieval quality and response coherence are the binding constraints for this application, not TTS expressiveness.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2508.03543' published_date: "2025-08-05" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - emotion_expressive_prosody_control claims: - claim_id: training_free_inference_time_activation_steering_can_achieve role: supports claim: Training-free inference-time activation steering can achieve emotion control in flow-matching TTS models that is competitive with or superior to systems explicitly trained for emotion conditioning. source: §4.2, Table 1 evidence: EmoSteer-TTS is a plug-in inference-time method for adding emotion control to any DiT-based flow-matching TTS model. confidence: high relevance: medium - claim_id: pre_trained_flow_matching_tts_models_encode_emotion role: supports claim: Pre-trained flow-matching TTS models encode emotion as approximately linear directions in DiT activation space, enabling vector arithmetic over multiple emotion categories. source: §3, §4.6 evidence: EmoSteer-TTS is a plug-in inference-time method for adding emotion control to any DiT-based flow-matching TTS model. confidence: high relevance: medium - claim_id: emotion_bearing_information_in_tts_dit_activations_is role: supports claim: Emotion-bearing information in TTS DiT activations is concentrated in a subset of token positions identifiable via speech emotion recognition scores, and selective steering of these positions outperforms global activation modification. source: §3.3, §4.6 evidence: The top-k positions (where emotion is concentrated) are identified; only these positions are targeted for steering during inference. confidence: high relevance: medium - claim_id: activation_steering_for_emotion_control_generalises_across_model role: supports claim: Activation steering for emotion control generalises across model architectures and scales without retuning, producing consistent results on F5-TTS (22 layers), E2-TTS (8 layers), and CosyVoice2 (56 layers). source: §4.2, §4.5 evidence: EmoSteer-TTS is a plug-in inference-time method for adding emotion control to any DiT-based flow-matching TTS model. confidence: high relevance: medium - claim_id: training_free_emotion_manipulation_via_activation_steering_incurs role: supports claim: Training-free emotion manipulation via activation steering incurs a moderate speaker similarity cost relative to the unsteered baseline, indicating residual speaker-emotion entanglement in DiT representations. source: §4.2, Table 1 evidence: Table 3 compares EmoSteer-TTS against label-based (EmoSphere++, EmoDubber) and description-based (EmoVoice) emotion TTS systems, plus unsteered CosyVoice2 as a description-based baseline. confidence: high relevance: medium limitations: - '- The steering vectors are derived from difference-in-means, which assumes linearity of emotional representations in DiT activations — a plausible but unverified assumption. - Speaker similarity drops slightly in EmoSteer+F5 (S-SIM 0.59) vs unsteered F5 (0.66), suggesting some speaker-emotion coupling in the activations. - The method requires access to DiT intermediate activations, ruling out fully black-box deployment. - The topk selection method depends on emotion2vec quality; errors in emotion token detection propagate to steering performance. - Evaluated on 6 basic emotions only; nuanced or blended emotions outside the training distribution are not assessed. - Laughter, whispering, and other non-categorical paralinguistic phenomena are not covered.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2508.04585' published_date: "2025-08-06" entry_date: '2026-07-25' year: 2025 venue: ACM MM task: - TTS - SCA architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - linguistic_context_prosody_prediction - emotion_expressive_prosody_control - multimodal_visual_prosody_conditioning claims: - claim_id: matching_the_token_rates_of_speech_and_facial role: supports claim: Matching the token rates of speech and facial landmark codecs enables frame-level synchronisation between synthesised speech and talking-face animations without post-hoc alignment. source: §4.1.3, §4.2 evidence: LmkCodec operates at 25 Hz with a codebook of size 1000, matching the speech tokenizer's rate and enabling frame-level alignment. confidence: high relevance: low - claim_id: llm_based_joint_prediction_of_interleaved_speech_and role: supports claim: LLM-based joint prediction of interleaved speech and visual tokens in dialogue context outperforms cascaded speech-then-video generation on both emotional accuracy and lip synchronisation. source: §6.2, §6.3, Table 2, Table 3 evidence: 'Built on Qwen2.5-0.5B, EVSLM takes the serialized multimodal context and sequentially predicts: (1) the target emotion token, (2) interleaved facial expression and speech tokens via a bimodal hard alignment strategy.' confidence: high relevance: low - claim_id: including_visual_dialogue_history_talking_face_animations_of role: supports claim: Including visual dialogue history (talking-face animations of prior turns) improves emotion accuracy and speaker consistency in conversational speech synthesis relative to audio-text-only context. source: §6.4, Table 4 evidence: 'UniTalker frames the problem as Conversational Speech-Visual Synthesis (CSVS): given the multimodal history of a dialogue (text, speech, talking-face animations, speaker identity, and emotion labels for each prior turn) and the current turn''s speaker and text, predict the target utterance''s emotion, speech, and talking-face animations jointly.' confidence: high relevance: medium - claim_id: low_rate_discrete_tokenisation_of_facial_landmarks_1 role: supports claim: Low-rate discrete tokenisation of facial landmarks (1 token per frame) is more effective for LLM contextual modelling than higher-rate representations, even at a cost in geometric reconstruction fidelity. source: §6.1, §6.2, Table 1, Table 2 evidence: The LmkCodec's FSQ design with 1 token per frame also proves better than the 8-token-per-frame GRVQ variant, demonstrating that shorter sequences aid LLM contextual modeling even at some reconstruction cost. confidence: high relevance: low - claim_id: emotion_guided_conditioning_of_the_speech_renderer_including role: supports claim: Emotion-guided conditioning of the speech renderer, including predicted emotion labels as explicit conditioning, improves measured emotional expressiveness over systems that rely on implicit contextual inference alone. source: §6.4, Table 4 evidence: In speech emotion expressiveness (MOS_SE), the margin is +0.111 over Empatheia. confidence: high relevance: medium limitations: - The Talking-face Animations Renderer (EchoMimic) is a pre-trained third-party module that receives no additional fine-tuning in this pipeline. Its outputs are constrained by the quality ceiling and biases of the EchoMimic base model, limiting the paper's ability to attribute animation quality gains to UniTalker specifically vs. the renderer. - 'Rendering latency is notable: speech generation takes approximately 2 seconds and animation rendering takes approximately 5 seconds per 25 frames on an RTX 4080 with 32 GB RAM, making the system unsuitable for real-time interaction in its current form. The paper acknowledges this and lists streaming optimisation as future work.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2507.20091' published_date: "2025-08-07" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - SCA - TTS architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - disentangled_prosody_content_timbre - emotion_expressive_prosody_control - voice_quality_and_paralinguistic_feature_control claims: - claim_id: codec_token_speech_lms_trained_on_tens_of role: supports claim: Codec-token speech LMs trained on tens of thousands of hours of data do not develop prosodic understanding or generation as emergent capabilities through pre-training alone. source: §1, §4.2–4.4 evidence: A sentence-level [Global] token capturing aggregate prosodic extremity is optionally prepended to control expressive range during generation; the authors find it beneficial for generation tasks and detrimental for understanding tasks. confidence: high relevance: high - claim_id: replacing_codec_tokens_with_explicit_interpretable_word_level role: supports claim: Replacing codec tokens with explicit, interpretable word-level prosody annotations enables a pre-trained text LLM to acquire prosody processing capabilities — including contrastive focus, emotion recognition, and style transfer — without task-specific fine-tuning. source: §3.1, §4.2–4.4 evidence: The decoder adapts StyleTTS2's duration predictor and prosody predictor — replacing their dependence on StyleTTS2's diffusion-based style embedding with conditioning on the word-level values generated by the LLM — while leaving all other StyleTTS2 modules frozen at their pre-trained weights. confidence: high relevance: high - claim_id: disentangling_prosodic_and_linguistic_content_in_the_token role: supports claim: Disentangling prosodic and linguistic content in the token sequence trades voice quality fidelity for prosodic expressiveness and controllability. source: §3.1, §5 evidence: A sentence-level [Global] token capturing aggregate prosodic extremity is optionally prepended to control expressive range during generation; the authors find it beneficial for generation tasks and detrimental for understanding tasks. confidence: high relevance: high - claim_id: prosody_to_content_dependencies_detecting_emphasis_or_emotion role: complicates claim: Prosody-to-content dependencies (detecting emphasis or emotion from speech) are more difficult for codec-token LMs than content-to-prosody generation. source: §4.3, Table 4 evidence: On prosody understanding tasks, ProsodyLM produces a statistically significant 6.6% log-probability increase for emphasised words (emphasis detection on EmphAssess), while both codec-token baselines show near-zero signal. confidence: high relevance: high - claim_id: word_level_prosody_tokens_impose_only_marginal_degradation role: supports claim: Word-level prosody tokens impose only marginal degradation on text language modelling capability when interleaved with transcription tokens in a joint sequence. source: §4.5, Table 9 evidence: Text perplexity on held-out audiobooks increases by only 2 points relative to a text-only fine-tuned Llama (11.70 vs. 13.87), indicating that prosody tokens add minimal interference to language modelling. confidence: high relevance: high limitations: - ProsodyLM is trained and evaluated exclusively on audiobooks (Librilight). Audiobook prosody is stylised and read-speech in character; generalisation to spontaneous conversation, emotional dialogue, or cross-domain settings is untested and likely requires substantial data re-collection or domain adaptation. - The decoder is frozen at StyleTTS2 pre-trained weights, which constrains output quality to that system's capability ceiling and limits the range of expressible voice characteristics. The five-dimensional prosody token cannot encode voice quality changes (breathiness, creakiness, vocal effort), which are important for fine-grained expressiveness. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.06890' published_date: "2025-08-09" entry_date: '2026-07-25' year: 2025 venue: ASRU task: - VC architecture: - GAN - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - disentangled_prosody_content_timbre - emotion_expressive_prosody_control claims: - claim_id: frame_level_emotion_representations_improve_speaker_emotion_classification role: supports claim: Frame-level emotion representations improve speaker emotion classification accuracy and prosody transfer fidelity over utterance-level representations in voice conversion. source: §IV.C, Table I evidence: In subjective MOS tests with 25 participants, Maestro-EVC scores 4.06 naturalness, 4.11 emotion similarity, 4.02 speaker similarity, and 4.15 prosody similarity — the largest margin is in prosody similarity, where ZEST scores 2.86. confidence: high relevance: high - claim_id: adversarial_disentanglement_via_gradient_reversal_layers_reduces_phonetic role: supports claim: Adversarial disentanglement via gradient reversal layers reduces phonetic leakage into emotion embeddings, improving intelligibility under cross-linguistic-content conversion. source: §IV.C, Table I evidence: A gradient reversal layer (GRL) with a content classifier then suppresses residual phonetic information leaking into the emotion representation, enforcing disentanglement. confidence: high relevance: medium - claim_id: explicit_conditioning_on_extracted_prosodic_features_f0_and role: supports claim: Explicit conditioning on extracted prosodic features (F0 and energy) from an emotion reference transfers temporal dynamics more faithfully than implicit prediction from latent codes. source: §IV.A, Table I evidence: Explicit emotional prosody transfer (EEPT) conditions F0 and energy prediction directly on smoothed prosodic features extracted from the emotion reference using a Savitzky-Golay filter. confidence: high relevance: high - claim_id: training_time_prosody_augmentation_through_temporal_shifting_and role: supports claim: Training-time prosody augmentation through temporal shifting and warping improves robustness of prosody transfer under mismatched reference conditions without sacrificing naturalness. source: §IV.C, Table I evidence: To bridge the training-inference gap introduced by prosody mismatch, a prosody augmentation strategy randomly applies either temporal shifting or piecewise time warping to the smoothed features during training, teaching the FE predictor to generalize across misaligned prosody sequences. confidence: high relevance: high limitations: - Training and primary evaluation use only the English ESD corpus — 350 parallel utterances across 10 speakers and 5 emotion categories. This is a narrow domain; generalisation to spontaneous, noisy, or multilingual emotional speech is entirely untested. - The small, parallel ESD corpus makes it difficult to assess whether the disentanglement holds under more naturalistic or non-parallel conditions. The ablation study evaluates the seen scenario only; it is not clear whether the ablated variants degrade similarly on unseen speakers and emotions. Speaker classification accuracy (SCA) is reported as a zero-shot metric for the seen-speaker scenario but becomes undefined for unseen speakers, so that dimension of the zero-shot evaluation lacks a corresponding metric. Model size and inference speed are not reported, which matters for the real-time dubbing applications the paper motivates. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.07426' published_date: "2025-08-10" entry_date: '2026-07-25' year: 2025 venue: ASRU task: - TTS architecture: - transformer-enc-dec - GAN - VAE relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: accent_label_quality_exerts_a_larger_effect_on role: supports claim: Accent label quality exerts a larger effect on accent synthesis fidelity than raw data volume, and filtering noisy self-reported labels consistently outperforms using them directly. source: §V-C, Table VI evidence: This enables both filtering of noisy self-reported labels and discovery of accent-labeled subsets from entirely unlabeled data. confidence: high relevance: low - claim_id: speaker_accent_disentanglement_in_low_resource_settings_benefits role: supports claim: Speaker-accent disentanglement in low-resource settings benefits from voice conversion-based timbre augmentation, even when the augmented data introduces acoustic degradation. source: §V-D, Table VIII evidence: 'The second strategy applies kNN-VC (k-nearest-neighbours voice conversion) to diversify speaker timbre within each accent class: each training utterance is converted to a randomly sampled LibriTTS speaker while preserving accent characteristics, effectively multiplying the number of distinct speaker timbres available for low-resource accents.' confidence: high relevance: low - claim_id: automatic_accent_identification_metrics_and_human_perceptual_evaluations role: supports claim: Automatic accent identification metrics and human perceptual evaluations of accent plausibility frequently diverge, limiting the reliability of either metric alone as a benchmark. source: §V-B, §V-E evidence: The filtered system scores similarly to the unfiltered system in most accent evaluations, but shows clear improvements for Australian and Filipino accents. kNN-VC augmentation improves objective accent similarity but shows unclear or mixed effects in human evaluation. confidence: high relevance: low - claim_id: large_zero_shot_tts_systems_can_produce_competitive role: supports claim: Large zero-shot TTS systems can produce competitive accented speech through style copying without explicit accent conditioning, often matching or exceeding dedicated accent-controllable models on naturalness. source: §V-B, Table VII evidence: CosyVoice2, using zero-shot style copying, achieves DCF = 0.748 on average but is competitive or stronger on specific accents. confidence: high relevance: medium - claim_id: geolocation_based_accent_label_discovery_is_extensible_to role: supports claim: Geolocation-based accent label discovery is extensible to languages and accent varieties without existing labeled speech, but precision varies substantially across accent regions depending on broadcast training data coverage. source: §III-A, Table III evidence: 'Three data configurations are compared: unfiltered (raw CommonVoice self-reported labels), filtered (geolocation-based filtering of self-reported labels), and unlabeled (geolocation-based label discovery from scratch).' confidence: high relevance: low limitations: - The paper evaluates English accent synthesis only, despite XTTS-v2 supporting multiple languages. The claim that geolocation-based label discovery generalises to any accent in any language is theoretical; precision on Canada (13.3%), Scotland (9.4%), Germany (15.9%), Ireland (20.4%), and Malaysia (7.1%) is poor, limiting practical applicability to a subset of the 11 accents studied. - 'Human evaluation is expensive and was conducted for only 4 of the 11 or more accents included in training, leaving uncertain whether objective metric rankings carry over to human preference for the remaining accents. The kNN-VC augmentation results are inconsistent: human evaluators found no clear benefit for augmented versus non-augmented systems in Australian and Scottish evaluations, contradicting objective metric improvements. The WER gap between the proposed systems and CosyVoice2 (13.1% vs. 7.1%) is notable and unexplained in depth. The geolocation model was trained on radio broadcasts, which may introduce domain mismatch when applied to crowd-sourced microphone speech.' caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '2508.08399' published_date: "2025-08-11" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - codec architecture: - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - disentangled_prosody_content_timbre claims: - claim_id: fully_discrete_disentanglement_of_phonetic_prosodic_and_speaker role: complicates claim: Fully discrete disentanglement of phonetic, prosodic, and speaker information in a speech codec is achievable without phoneme labels or F0 supervision, at the cost of a small reconstruction quality degradation. source: §III, §IV.B, Table II evidence: Quantizing the prosody stream reduces EER for VC (more information bottleneck, cleaner disentanglement), but quantizing the speaker vector slightly increases EER relative to continuous speaker, posing a direct trade-off between LLM-compatibility and speaker identity fidelity. confidence: high relevance: high - claim_id: quantizing_speaker_vectors_into_discrete_codes_reduces_speaker role: complicates claim: Quantizing speaker vectors into discrete codes reduces speaker identity fidelity compared to continuous speaker representations, posing a fundamental trade-off between LLM compatibility and speaker retention. source: §IV.B, Table III evidence: Quantizing the prosody stream reduces EER for VC (more information bottleneck, cleaner disentanglement), but quantizing the speaker vector slightly increases EER relative to continuous speaker, posing a direct trade-off between LLM-compatibility and speaker identity fidelity. confidence: high relevance: low - claim_id: instance_normalization_of_ssl_residual_features_provides_a role: supports claim: Instance normalization of SSL residual features provides a label-free mechanism to separate time-invariant speaker statistics from time-variant prosodic content. source: §III.B evidence: 'Prosody vectors P: computed from the residual R = W − C after instance normalization (IN) removes the time-invariant speaker statistics.' confidence: high relevance: high - claim_id: fully_discrete_speech_codecs_can_match_conventional_voice role: supports claim: Fully discrete speech codecs can match conventional voice conversion methods on intelligibility and naturalness while enabling attribute manipulation through codebook-level operations. source: §IV.B, Table III evidence: On one-shot voice conversion, SKQ3+σ achieves WER 2.62%, EER 5.62%, and UTMOSv2 2.92 — broadly matching the baseline SKQ (2.25% / 6.88% / 3.03). confidence: high relevance: low limitations: - All experiments use LibriSpeech clean speech (16 kHz, studio conditions); performance on noisy, spontaneous, or out-of-domain speech is untested. The one-shot VC evaluation uses only two reference speakers (one male, one female), limiting statistical confidence in the speaker similarity results. - The model is not tested on any downstream application (TTS, ASR, speech LM), despite this being the stated motivation. Whether the disentangled discrete tokens actually improve over non-disentangled tokens on downstream tasks remains an open question — the paper acknowledges this as future work. The GRVQ codebook dimensionality analysis shows a clear trade-off between bitrate and speaker identity, but optimal bitrate allocation across the three streams is not systematically explored. Prosody quantization codebook interpretability beyond F0 correlation (e.g., energy, duration, speaking rate) is not investigated. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.09702' published_date: "2025-08-13" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS - evaluation architecture: [] relevance: low evidence_role: - historical_context current_role: minor method_family: - latent_reference_prosody_transfer claims: - claim_id: quality_mismatch_between_clean_training_corpora_and_real role: supports claim: Quality mismatch between clean training corpora and real-world inference prompts is a meaningful source of degradation in zero-shot speech generation systems. source: §1 evidence: 'For visual-prompt-based selection, retrieved M3PDB audio achieves comparable speaker similarity to Imaginary Voice face-to-speech synthesis (15.14% vs 16.18% SS) while substantially improving audio quality (UTMOSv2: 2.69 vs 2.10), highlighting the gap between generative face-to-speech quality and clean audio drawn from a curated database (Table 4).' confidence: high relevance: low - claim_id: cross_lingual_proxy_prompts_selected_by_linguistic_proximity role: supports claim: Cross-lingual proxy prompts selected by linguistic proximity can produce lower synthesis error than native-language prompts of lower audio quality. source: §3.2, Table 3 evidence: For unseen language synthesis (Italian via XTTS-v2), cross-lingual prompts selected by M3PDB yield CER of 2.78%, compared to 4.32% for native Italian prompts and 5.21% for French prompts selected at random — suggesting that cross-lingual proxy selection can outperform naive use of native-language audio (Table 3). confidence: high relevance: low - claim_id: multi_modal_annotation_that_fuses_visual_and_acoustic role: supports claim: Multi-modal annotation that fuses visual and acoustic signals yields substantially higher age and gender labelling accuracy than audio-only or text-only annotation strategies. source: §5.1, Table 2 evidence: 'Annotation uses a multi-modal multi-agent framework: a GPT-4o central controller with Retrieval-Augmented Generation (RAG) selects from a registry of specialised agent models (DeepFace for visual age/gender, Emotion2Vec for emotion, Whisper for language and content) and computes weighted aggregations over their outputs.' confidence: high relevance: low - claim_id: a_curated_high_quality_audio_database_outperforms_generative role: supports claim: A curated, high-quality audio database outperforms generative face-to-speech synthesis as a source of speaker prompts when audio quality is the binding constraint. source: §5.3, Table 4 evidence: 'For visual-prompt-based selection, retrieved M3PDB audio achieves comparable speaker similarity to Imaginary Voice face-to-speech synthesis (15.14% vs 16.18% SS) while substantially improving audio quality (UTMOSv2: 2.69 vs 2.10), highlighting the gap between generative face-to-speech quality and clean audio drawn from a curated database (Table 4).' confidence: high relevance: low - claim_id: cascaded_interruptible_similarity_retrieval_is_a_viable_approach role: supports claim: Cascaded, interruptible similarity retrieval is a viable approach to prompt selection under strict latency constraints without dedicated hardware. source: §4.2, §5.4, Table 5 evidence: 'For visual-prompt-based selection, retrieved M3PDB audio achieves comparable speaker similarity to Imaginary Voice face-to-speech synthesis (15.14% vs 16.18% SS) while substantially improving audio quality (UTMOSv2: 2.69 vs 2.10), highlighting the gap between generative face-to-speech quality and clean audio drawn from a curated database (Table 4).' confidence: high relevance: medium limitations: - No subjective listening tests are reported. All quality metrics (UTMOSv2, automatic speaker similarity, Emotion2Vec emotion similarity) are automatic predictors. The relationship between these metrics and perceptual quality in the prompt selection context is not validated. - The evaluation spans only two downstream tasks (TTS and speech translation) and a narrow set of conditions. The claim that M3PDB generalises to other speech generation tasks (interactive systems, voice conversion) is stated as a future direction but not evidenced. The database construction pipeline is described as time-consuming, which limits reproducibility for groups without significant compute. Coverage of 18 languages is broader than most existing resources but still leaves many of the world's languages out. The unseen-language proxy mechanism is validated only on Italian; its reliability for more distant language pairs is unknown. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '2508.09767' published_date: "2025-08-13" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control claims: - claim_id: parameter_efficient_fine_tuning_with_lora_can_restore role: supports claim: Parameter-efficient fine-tuning with LoRA can restore explicit pronunciation controllability to LLM-TTS systems that omit G2P front-ends, without degrading cross-lingual performance. source: §3.2, §5.1 evidence: Because the LoRA module is language-specific and optional, inference on other languages simply omits it, leaving cross-lingual performance unaffected in principle. confidence: high relevance: low - claim_id: bpe_based_multilingual_tts_models_systematically_underperform_on role: supports claim: BPE-based multilingual TTS models systematically underperform on phonologically complex languages where reading ambiguity is high and pitch accent is lexically contrastive. source: §1, §5.2 evidence: On the accent stress-test (Test Set 2), the baseline CosyVoice 2 achieves accent correctness of only 0.498 — near chance — even when kana transcription is supplied to resolve kanji ambiguity. confidence: high relevance: high - claim_id: phoneme_tag_injection_via_special_tokens_can_provide role: supports claim: Phoneme-tag injection via special tokens can provide localised pronunciation control without leaking into untagged text spans. source: §5.2 evidence: 'A follow-up probe using 50 katakana-only sentences confirms that the phoneme-tag mechanism does not bleed pronunciation control into spans outside the tags: accent correctness is 0.829 for UtterTune vs. 0.796 for the baseline, with no significant difference.' confidence: high relevance: low - claim_id: speaker_similarity_in_zero_shot_tts_is_robust role: supports claim: Speaker similarity in zero-shot TTS is robust to targeted LoRA fine-tuning on language-specific pronunciation data. source: §5.1, Table 1 evidence: Only the embeddings of the two new tokens and the LoRA matrices are updated during fine-tuning; the original embedding matrix remains frozen. confidence: high relevance: low limitations: - Evaluation uses two custom test sets authored or curated by the single researcher-author, with no publicly available benchmark comparison. The listening test crowd cohort (16 workers for MOS, 15 for accent correctness) is small for a task with high inter-rater variance. Results should be treated as promising but not yet independently validated. - The approach requires users to supply phonemic transcription and accent markup, which demands linguistic expertise most end users lack. The paper targets only standard Tokyo Japanese; dialectal variation and code-switching remain untested, as the author acknowledges. The LoRA module is trained on a single female-heavy corpus (JSUT) and 100-speaker JVS, which may limit generalisation to speaker types not well represented in that data. Applicability to other prosodically complex languages — Mandarin tones, Scandinavian pitch accent — is left for future work. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2508.11224' published_date: "2025-08-15" entry_date: '2026-07-25' year: 2025 venue: ASRU task: - evaluation - TTS architecture: [] relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control claims: - claim_id: ssl_models_using_frame_wise_masked_prediction_capture role: supports claim: SSL models using frame-wise masked prediction capture relative prosodic contours within an utterance rather than absolute acoustic magnitudes, making them insensitive to global intensity rescaling. source: §V-A, Fig. 1 evidence: SSL models trained with frame-wise masked prediction (HuBERT, ContentVec, data2vec) consistently respond more strongly to word-level prosodic emphasis than to utterance-level uniform scaling, for both pitch and intensity. confidence: high relevance: high - claim_id: models_pretrained_to_predict_discrete_targets_encode_phoneme role: supports claim: Models pretrained to predict discrete targets encode phoneme-like structure effectively at small cluster sizes, while models pretrained on continuous targets require larger cluster sizes to approach the same phonemic alignment. source: §V-A, Fig. 3 evidence: 'Regarding cluster size and SSL pretraining objective: models trained to predict discrete labels (HuBERT, ContentVec) capture phoneme-like structure even at 100 clusters and show monotonically increasing prosodic sensitivity with cluster size.' confidence: high relevance: low - claim_id: training_k_means_clustering_on_emotionally_expressive_speech role: supports claim: Training k-means clustering on emotionally expressive speech increases the prosodic sensitivity of resulting tokens for most SSL model and layer combinations. source: §V-B, Table I evidence: Training k-means on emotional speech (MEAD) significantly increases sensitivity to prosodic variation in most conditions (Table I, p < 0.05), at the cost of moderately increased sensitivity to real speaker variation. confidence: high relevance: high - claim_id: applying_a_temporal_moving_average_to_ssl_features role: complicates claim: Applying a temporal moving average to SSL features before k-means clustering provides an adjustable trade-off between prosodic sensitivity and speaker invariance, with intermediate window sizes improving both simultaneously. source: §V-C, Fig. 5 evidence: 'The study also evaluates two additional design choices: (1) whether training k-means on emotional speech (MEAD dataset) versus neutral read speech (LibriSpeech) changes prosodic sensitivity, and (2) whether applying a moving average to SSL features before clustering, at varying window sizes, affects the trade-off between prosodic sensitivity and speaker invariance.' confidence: high relevance: high - claim_id: differences_between_ssl_pretraining_objectives_in_their_token role: supports claim: Differences between SSL pretraining objectives in their token-level linguistic and prosodic encoding are concentrated in the final transformer layers, while intermediate layers exhibit largely similar behaviour across model families. source: §V-A, §V-B evidence: Models trained to predict continuous features (data2vec, emotion2vec) exhibit degraded prosodic sensitivity at small cluster sizes at the final transformer layers, but their intermediate layers (around layers 9–10) behave similarly to HuBERT and ContentVec. confidence: high relevance: high limitations: - The analysis measures sensitivity via TER — a proxy for how much token sequences change in response to acoustic manipulation — rather than directly probing what information is decodable from the tokens. Whether the observed sensitivity differences translate to actual gains in downstream prosody-related tasks (prosody-conditional TTS, emphasis transfer, emotion recognition from discrete tokens) remains untested. The evaluation uses a single corpus (TIMIT) of read speech by native English speakers, which may limit generalisability to spontaneous, conversational, or multilingual speech. Duration was excluded from prosody analysis because deduplication collapses durational information; this omission means the benchmark does not cover the full prosody space. The study does not test acoustic tokens (neural codec outputs), focusing exclusively on semantic tokens. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.11273' published_date: "2025-08-15" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - emotion_expressive_prosody_control claims: - claim_id: discretizing_self_supervised_speech_features_via_k_means role: supports claim: Discretizing self-supervised speech features via k-means produces more stable prosodic control signals than retaining continuous SSL representations in encoder-decoder TTS. source: §5.6, Table 1 evidence: 'A parallel two-component design: (1) a phoneme encoder using Montreal Forced Aligner for English and pyopenjtalk/ESPNet for Japanese produces phoneme sequences passed directly to the decoder; (2) a frozen DeBERTaV3 semantic encoder (language-adapted tokenizer only) produces multilingual semantic embeddings that condition the emotional and prosodic modules via cross-attention but are not directly decoded.' confidence: high relevance: high - claim_id: combining_continuous_spherical_emotion_vectors_with_discrete_ssl role: supports claim: Combining continuous spherical emotion vectors with discrete SSL prosody tokens improves both naturalness and intelligibility compared to spherical emotion vectors alone. source: §5.1, §5.3, Tables 1–2 evidence: 'One-hot Emotion ID embeddings (categorical: Angry, Sad, Happy, Surprised, etc.) are concatenated with the spherical AVD vector, providing both continuous nuance and discrete stylistic priors.' confidence: high relevance: high - claim_id: speaker_independent_prosody_conditioning_via_ssl_tokens_can role: supports claim: Speaker-independent prosody conditioning via SSL tokens can generalize across reference speakers with minimal degradation, enabling robust cross-speaker emotion transfer. source: §5.6, Table 1 evidence: Speaker-mismatched reference audio has minimal adverse impact, suggesting robust speaker-independent prosody modeling. confidence: high relevance: high - claim_id: semantic_text_encoders_contribute_to_emotional_and_prosodic role: supports claim: Semantic text encoders contribute to emotional and prosodic consistency in multilingual TTS even when they do not serve as direct decoder inputs. source: §5.6, Table 1 evidence: 'A parallel two-component design: (1) a phoneme encoder using Montreal Forced Aligner for English and pyopenjtalk/ESPNet for Japanese produces phoneme sequences passed directly to the decoder; (2) a frozen DeBERTaV3 semantic encoder (language-adapted tokenizer only) produces multilingual semantic embeddings that condition the emotional and prosodic modules via cross-attention but are not directly decoded.' confidence: high relevance: high limitations: - '- Evaluations are limited to single-speaker datasets in two languages, making it unclear whether EmoSSLSphere generalises to multi-speaker, low-resource, or unseen-language scenarios. - Subjective listener panels are small, and emotional authenticity is assessed primarily via AVD RMSE as a proxy rather than direct perceptual emotion ratings. - Speaker similarity (SPK-SIM) is not evaluated, making it hard to quantify speaker fidelity claims. - Cross-speaker emotion transfer is described but not formally evaluated; inference always uses same-speaker reference audio. - Separate per-language encoder instances do not scale to many-language settings without significant parameter overhead. - Integration with semi-supervised training (EmoSphere++) and extension to zero-shot speaker scenarios are listed as future work.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.12001' published_date: "2025-08-16" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - VAE - GAN - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - stochastic_generative_prosody_model - emotion_expressive_prosody_control - signal_level_pitch_modification claims: - claim_id: applying_mixture_of_experts_routing_to_the_duration role: supports claim: Applying Mixture-of-Experts routing to the duration predictor of a non-autoregressive TTS system produces phoneme duration distributions that more closely match ground-truth speaker-specific prosodic variation than deterministic or stochastic single-path predictors. source: §IV-C, Table III evidence: Distribution visualizations (Fig. 2 and 3) show FNH-TTS's duration histogram follows the ground-truth distribution most closely on both single-speaker (LJSpeech) and multi-speaker (VCTK) data, with visible speaker-specific variation across VCTK speakers that other systems fail to reproduce. confidence: high relevance: high - claim_id: improving_duration_prediction_diversity_in_nar_tts_without role: supports claim: Improving duration prediction diversity in NAR TTS without upgrading the vocoder degrades synthesis quality, because richer prosodic information exceeds the artifact-rejection capacity of standard GAN discriminators. source: §IV-A, Table I evidence: 'Ablation: MoE-DP alone with HiFi-GAN degrades MOS to 3.92 on LJSpeech, confirming that the vocoder upgrade is necessary to realize MoE-DP''s prosodic gains.' confidence: high relevance: high - claim_id: frequency_domain_and_multi_band_discriminators_combd_and role: supports claim: Frequency-domain and multi-band discriminators (CoMBD and SBD) are more effective than HiFi-GAN's multi-period discriminator at suppressing spectral artifacts when a more expressive duration predictor is used. source: §V-B, Table II evidence: 'Two adversarial discriminators are added: a Collaborative Multi-Band Discriminator (CoMBD) that applies multi-scale discrimination at multiple waveform resolutions for temporal coherence, and a Sub-Band Discriminator (SBD) that uses PQMF analysis to decompose into frequency sub-bands, each processed with multi-scale dilated convolutions.' confidence: high relevance: high - claim_id: word_error_rate_is_an_unreliable_proxy_for role: complicates claim: Word Error Rate is an unreliable proxy for prosody modeling quality in TTS, as simpler prosodic patterns can yield lower WER by reducing ASR system disruption. source: §IV-C, §III evidence: During training, a load-balancing auxiliary loss (L_aux, scaled by alpha) encourages uniform expert utilization so that different experts specialize in distinct prosodic patterns rather than collapsing. confidence: high relevance: high limitations: - '- The MoE-DP requires the discriminator upgrade to yield any net benefit; neither module alone suffices, which raises questions about robustness when either component is replaced or updated independently. - Evaluation is limited to English datasets; whether MoE-DP''s speaker-specific prosody gains transfer to more typologically diverse languages with different prosodic structures is untested. - The paper notes that current vocoders still struggle under highly diverse prosodic patterns (disharmony components persist even with CoMBD+SBD), pointing to an open challenge for future vocoder designs. - WER is shown to be an unreliable proxy for prosody quality, but the proposed duration-accuracy metric (MFA-based classification) requires ground-truth phoneme duration annotations, limiting its generalizability. - No code or demo links are provided, limiting reproducibility. - Inference RTF on CPU (0.046) is not compared directly against autoregressive or diffusion-based competitors that operate on the same hardware, making efficiency claims difficult to contextualize.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-0047 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - emotion_expressive_prosody_control claims: - claim_id: mixing_high_quality_audio_only_data_with_audio role: supports claim: Mixing high-quality audio-only data with audio-visual corpora via contrastive modality alignment substantially improves speech naturalness in face-driven TTS systems. source: §2.3, Table 1 evidence: Ablation confirms that mixing high-quality LibriTTS-R audio data contributes the largest MOS gain (+0.50), followed by style augmentation (+0.26 in MOS), with contrastive learning primarily benefiting face-voice association (SIM metric) rather than naturalness. confidence: high relevance: low - claim_id: neural_style_transfer_augmentation_of_training_face_images role: supports claim: Neural style transfer augmentation of training face images reduces the domain gap between photorealistic faces and artistic portraits, improving both naturalness and face-matching scores. source: §2.2, Table 1 evidence: 'Style augmentation: During training, input face images are randomly passed through CAST neural style transfer and augmented with gray-scaling and blurring (50% chance).' confidence: high relevance: medium - claim_id: in_context_prompting_with_a_user_selected_speech role: supports claim: In-context prompting with a user-selected speech sample enables consistent voice generation across multiple utterances from the same face, addressing the inherent one-to-many ambiguity of face-to-voice mapping. source: §2.4 evidence: For diverse but consistent voice generation, the model uses sampling-based decoding to produce multiple candidate voices from the same face, then accepts a user-selected sample as an in-context prompt to enforce voice consistency across subsequent utterances. confidence: high relevance: low - claim_id: contrastive_pre_training_of_face_and_audio_encoders role: supports claim: Contrastive pre-training of face and audio encoders primarily improves face-voice association rather than overall speech naturalness. source: §3.4.1, Table 1 evidence: Ablation confirms that mixing high-quality LibriTTS-R audio data contributes the largest MOS gain (+0.50), followed by style augmentation (+0.26 in MOS), with contrastive learning primarily benefiting face-voice association (SIM metric) rather than naturalness. confidence: high relevance: low - claim_id: objective_acoustic_metrics_speaking_rate_si_sdr_c50 role: supports claim: Objective acoustic metrics (speaking rate, SI-SDR, C50, pitch standard deviation) shift predictably with natural language descriptors, confirming that text-driven style control transfers to measurable acoustic properties. source: §3.4.3, Table 3 evidence: Controllability is verified using objective metrics (speaking rate, SI-SDR, C50, pitch std) that shift predictably with descriptive text changes. confidence: high relevance: high limitations: - '- Voice consistency score (VCS) for RV-TTS (3.96) is below the audio-driven YourTTS (4.42), indicating residual inconsistency in pure face-conditioned generation. - The artistic portrait test uses only 20 copyright-free images; broader and more controlled evaluation is needed. - Descriptive text label generation for LRS3/VoxCeleb2 relies on Data-Speech auto-annotation rather than human labels. - Face encoder is ResNet50; more powerful vision-language models may improve face-voice matching. - No multilingual evaluation.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: interspeech-2025-0143 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: [] relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: multimodal_fusion_of_data_driven_acoustic_and_word role: supports claim: Multimodal fusion of data-driven acoustic and word-level linguistic embeddings outperforms unimodal and knowledge-based features for multilingual sentence mode classification. source: §4, Table 2 evidence: 'The study evaluates three systems for sentence mode prediction (3-class: ''.'', ''!'', ''?'') on Italian, French, and German: System 1 (main comparison): An MLP classifier (100 hidden units, ReLU) applied to a variety of feature representations: - Acoustic: (a) knowledge-based (COMPARE 2016, 6373 features, openSMILE) and (b) data-driven (WavLM Large final-layer embeddings, mean-pooled to sentence level) - Linguistic: (a) phoneme-level (XPhoneBERT, multilingual phoneme BERT) and (b) word-level (XLM-RoBERTa, trained on 2.5TB of 100-language CommonCrawl) - Early fusion (EF): (a) data-driven acoustic + word-level linguistic, and (b) all four feature types combined Both intra-language (5-fold cross-validation) and inter-language (train on one, test on another) protocols are evaluated.' confidence: high relevance: low - claim_id: multilingual_ssl_representations_trained_on_large_corpora_can role: supports claim: Multilingual SSL representations trained on large corpora can transfer sentence-mode discriminative information across languages without language-specific training. source: §5 evidence: 'The study evaluates three systems for sentence mode prediction (3-class: ''.'', ''!'', ''?'') on Italian, French, and German: System 1 (main comparison): An MLP classifier (100 hidden units, ReLU) applied to a variety of feature representations: - Acoustic: (a) knowledge-based (COMPARE 2016, 6373 features, openSMILE) and (b) data-driven (WavLM Large final-layer embeddings, mean-pooled to sentence level) - Linguistic: (a) phoneme-level (XPhoneBERT, multilingual phoneme BERT) and (b) word-level (XLM-RoBERTa, trained on 2.5TB of 100-language CommonCrawl) - Early fusion (EF): (a) data-driven acoustic + word-level linguistic, and (b) all four feature types combined Both intra-language (5-fold cross-validation) and inter-language (train on one, test on another) protocols are evaluated.' confidence: high relevance: low - claim_id: state_of_the_art_asr_systems_are_unable role: supports claim: State-of-the-art ASR systems are unable to reliably detect exclamatory sentence mode from speech, producing recall rates below chance level for that class. source: §4, Table 3 evidence: 'Best intra-language UAR (System 1, early fusion of all features): - German: 74.22% (data-driven EF) / 73.40% (all EF) - French: 67.10% (data-driven EF) - Italian: 64.14% (data-driven EF) Key findings: - Linguistic FRS (word-level XLM-RoBERTa) consistently outperform acoustic FRS for interrogative sentence discrimination. - Acoustic FRS (knowledge-based) capture exclamatory sentences better. - Early fusion outperforms unimodal features in intra-language settings. - Cross-lingual transfer is feasible: models trained on German achieve UAR up to 63.62% on French (inter-language). - WHISPER (System 2) performs poorly on exclamatory sentences (recall well below chance for all languages), despite strong declarative/interrogative recall, suggesting current ASR systems cannot reliably generate prosodically informative punctuation. - Emotional speech (EMO-DB) causes significant degradation: most emotional samples are misclassified as exclamatory, reflecting the definitional overlap between exclamatory tone and emotional speech.' confidence: high relevance: low - claim_id: sentence_mode_prediction_performance_degrades_substantially_on_emotional role: supports claim: Sentence mode prediction performance degrades substantially on emotional speech due to definitional overlap between exclamatory sentence mode and emotional expressiveness. source: §5, Table 4 evidence: 'Best intra-language UAR (System 1, early fusion of all features): - German: 74.22% (data-driven EF) / 73.40% (all EF) - French: 67.10% (data-driven EF) - Italian: 64.14% (data-driven EF) Key findings: - Linguistic FRS (word-level XLM-RoBERTa) consistently outperform acoustic FRS for interrogative sentence discrimination. - Acoustic FRS (knowledge-based) capture exclamatory sentences better. - Early fusion outperforms unimodal features in intra-language settings. - Cross-lingual transfer is feasible: models trained on German achieve UAR up to 63.62% on French (inter-language). - WHISPER (System 2) performs poorly on exclamatory sentences (recall well below chance for all languages), despite strong declarative/interrogative recall, suggesting current ASR systems cannot reliably generate prosodically informative punctuation. - Emotional speech (EMO-DB) causes significant degradation: most emotional samples are misclassified as exclamatory, reflecting the definitional overlap between exclamatory tone and emotional speech.' confidence: high relevance: low limitations: - '- No end-to-end TTS experiment; the study measures sentence mode prediction accuracy, not synthesized prosody quality. - Labels derived from punctuation marks are a proxy for sentence mode; they may not reflect actual prosodic realization (especially for audiobooks where speakers may monotonize exclamatory passages). - Only three languages evaluated; coverage of typologically diverse languages is absent. - MLP classifier is simple; more powerful sequence models may perform better. - UAR is used rather than accuracy due to class imbalance, but class sizes differ substantially across languages.' caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: interspeech-2025-0253 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - autoregressive-LM - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - linguistic_context_prosody_prediction claims: - claim_id: maintaining_dynamically_updated_compressed_memory_across_sentences_improves role: supports claim: Maintaining dynamically updated compressed memory across sentences improves naturalness and coherence in paragraph-level TTS compared to methods that use fixed-window preceding sentences. source: §3.4.1, Table 1 evidence: 'On the internal Chinese Mandarin audiobook test set (Table 1): | System | MOS | CoMOS | SpeechBERT | CER | SPK-SIM (mean, var) | |---|---|---|---|---|---| | Proposed | 3.796 | 3.992 | 80.448 | 4.140% | 85.685 (0.019) | | MMCE-Qformer | 3.557 | 3.885 | 79.031 | 5.075% | 85.110 (0.021) | | CLAP-RAG | 3.489 | 3.717 | 78.892 | 6.234% | 84.920 (0.037) | | Baseline | 3.468 | 3.460 | 77.776 | 5.850% | 85.051 (0.035) | The proposed method achieves the best naturalness (MOS), coherence (CoMOS), content accuracy (CER), and speaker consistency (lowest SPK-SIM variance) while using only fixed 64-token context — the same cost as MMCE-Qformer but without requiring 5 full preceding sentences.' confidence: high relevance: low - claim_id: speech_context_representations_are_more_effective_than_text role: supports claim: Speech context representations are more effective than text context representations for guiding prosodic coherence in autoregressive LM-based TTS, due to the one-to-many relationship between text and speech. source: §3.4.2, Table 1 evidence: Ablation confirms that both speech memory (Mem-S) and text memory (Mem-T) contribute, with speech memory having the larger effect (likely because speech captures richer prosodic information than text). confidence: high relevance: high - claim_id: applying_bidirectional_attention_to_prefix_tokens_via_a role: supports claim: Applying bidirectional attention to prefix tokens via a prefix mask enhances in-context learning in decoder-only TTS LMs without compromising autoregressive generation consistency. source: §2.3, §3.4.2, Table 1 evidence: Prefix mask replaces the standard causal mask for the memory/text prefix tokens, applying bidirectional attention to them while maintaining unidirectional attention over generated speech tokens. confidence: high relevance: low - claim_id: excessively_long_variable_length_inference_prompts_increase_hallucination role: supports claim: Excessively long variable-length inference prompts increase hallucination and content errors in autoregressive TTS, while fixed-length context representations mitigate this instability. source: §3.4.1, Table 1 evidence: Inference requires only one preceding context plus fixed-length 64-token memory, compared to 5 full preceding sentences (MMCE-Qformer) or variable-length prompts up to ~900 tokens (CLAP-RAG). confidence: high relevance: low limitations: - '- Evaluation is monolingual (Chinese Mandarin only); generalizability to other languages is unconfirmed. - The gap to Ground Truth (MOS 4.406 vs 3.796) remains substantial; paragraph-level naturalness is still an open problem. - No code release; replication requires re-implementing CAM on a CosyVoice backbone. - The model is evaluated only on single-speaker audiobooks; how it handles multi-speaker paragraphs is unknown. - The fixed 32-embedding Perceiver Resampler bottleneck may lose fine-grained phonetic detail.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: interspeech-2025-0347 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - codec - singing architecture: - GAN - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - disentangled_prosody_content_timbre - speech_singing_shared_prosody claims: - claim_id: injecting_explicit_periodic_signals_into_a_neural_codec role: supports claim: Injecting explicit periodic signals into a neural codec decoder enables independent F0 control during waveform reconstruction, decoupling pitch from the discrete token stream. source: §4.2, §4.3, Table 1, Figure 2 evidence: Period variants achieve substantially lower F0-RMSE than Base (which embeds pitch implicitly in tokens) across all pitch shift conditions, and MOS improves from 2.37 to 3.28 at no shift. confidence: high relevance: high - claim_id: including_singing_voice_data_in_codec_training_improves role: supports claim: Including singing voice data in codec training improves F0 accuracy at high pitch ranges that speech-only corpora do not cover. source: §4.2, Figure 2, Figure 3 evidence: Period+GT reduces F0-RMSE relative to Period when log F0 is shifted upward by 6-12 semitones, corresponding to the extended high-pitch coverage of GTSinger vs. LibriTTS. confidence: high relevance: high - claim_id: gradient_reversal_based_pitch_disentanglement_in_neural_codecs role: complicates claim: Gradient reversal-based pitch disentanglement in neural codecs does not reliably improve perceptual quality and may degrade it. source: §4.3, Table 1 evidence: Period-GRL scores 2.44 MOS vs. Period at 3.28 MOS at no shift; Period-GRL+GT scores 3.33 vs. Period+GT at 3.55. The authors note the effect varies with training data domain, leaving the mechanism unclear. confidence: high relevance: high - claim_id: codec_training_on_data_with_a_wider_pitch role: complicates claim: Codec training on data with a wider pitch range can improve quality at high pitches but reduces quality at lower pitches that are underrepresented in the new data. source: §4.3, Table 1 evidence: At -6 semitone shift, Period+GT (3.29 MOS) scores lower than Period (3.42 MOS), attributed to the model allocating capacity to the wider pitch range of GTSinger at the cost of fidelity in the lower range. confidence: high relevance: high limitations: - All subjective evaluation is conducted on a proprietary Japanese children's song dataset recorded by two singers unseen during training. Results may not generalise to other singing styles, languages, or recording conditions. - 'The GRL disentanglement module did not improve naturalness in subjective evaluation and in some conditions worsened it; the authors acknowledge this interaction with training data domain requires clarification. The evaluation is a codec reconstruction task only: no downstream discrete-token-based singing synthesis system is presented, so the utility of PeriodCodec in a full SVS pipeline remains undemonstrated. Model size is not reported.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-0383 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - VC architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: discrete_unit_voice_conversion_can_be_extended_to role: supports claim: Discrete-unit voice conversion can be extended to control subjective perceptual attributes beyond speaker identity by adding a scalar conditioning signal to the synthesis model. source: §3, §5.3, Figure 6 evidence: A FastSpeech 2 model conditioned on HuBERT-based discrete units, ECAPA-TDNN speaker embeddings, and a target likability scalar successfully steered perceived likability for 3 of 4 speakers in pairwise preference tests. confidence: high relevance: low - claim_id: automatic_likability_predictors_based_on_tdnn_regression_on role: supports claim: Automatic likability predictors based on TDNN regression on crowd-sourced ratings can provide sufficient proxy labels to train large-scale perceptual attribute control systems. source: §4.2, Table 2 evidence: The predictor achieved LCC 0.46 and SRCC 0.49 with human ratings (p < 3e-17) and 74% binary classification accuracy; its outputs were used to automatically annotate the JVS and JTES corpora for VC training. confidence: high relevance: low - claim_id: strong_likability_control_and_speaker_identity_preservation_are role: complicates claim: Strong likability control and speaker identity preservation are in tension in discrete-unit VC systems, particularly at extreme target values. source: §5.2, §5.3, Figures 4–6 evidence: At target likability = 2 (outside the training range), CER increased substantially for female speakers and speaker m49 showed degraded speaker similarity and unexpected subjective likability drop; the inference-time scalar multiplier partially mitigates but does not eliminate this trade-off. confidence: high relevance: low - claim_id: voice_likability_control_demonstrates_effective_behaviour_for_majority role: complicates claim: Voice likability control demonstrates effective behaviour for majority speaker groups but can fail for individual speakers due to identity-likability interaction effects. source: §5.3, Figure 6 evidence: Three of four speakers showed significant preference differences between target -1 and 1; speaker m49 exhibited the opposite trend, attributed to failure to preserve speaker identity, indicating that per-speaker variation is a real limitation. confidence: high relevance: low - claim_id: perceived_voice_likability_is_a_multi_factorial_attribute role: refines claim: Perceived voice likability is a multi-factorial attribute requiring demographic-stratified modelling, not a single group-level score. source: §2, §4.2, Table 2 evidence: The predictor uses four separate listener-group outputs (by gender and age); per-group LCC ranges from 0.36 to 0.41 while the aggregate LCC is 0.46, confirming systematic variation across listener demographics. confidence: high relevance: low limitations: - The evaluation is limited to four speakers (two female, two male) and one language (Japanese), with training on Japanese speech corpora. Cross-lingual and multi-lingual generalisability is entirely untested. - 'The practical control range of the system is narrow: despite targeting values from -2 to 2, predicted likability shifts only from approximately -0.51 to -0.23, suggesting the model substantially operates within each speaker''s inherent likability range rather than achieving wide stylistic transfer. This is acknowledged by the authors but not resolved.' caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: interspeech-2025-0408 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - SCA - TTS architecture: - transformer-enc-dec - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control claims: - claim_id: combining_paralinguistic_intimacy_adaptation_with_linguistic_register_change role: supports claim: Combining paralinguistic intimacy adaptation with linguistic register change improves user satisfaction more than linguistic-only adaptation in multi-session dialogue. source: §5.2, Table 2 evidence: 'Paralinguistic adaptation (new contribution): A Tacotron 2-based TTS system, augmented with an intimacy embedding network replacing the speaker embedding, is trained on the SMOC Japanese spontaneous multimodal corpus.' confidence: high relevance: medium - claim_id: intimacy_conditioned_tts_trained_on_spontaneous_conversational_data role: supports claim: Intimacy-conditioned TTS trained on spontaneous conversational data can reproduce prosodic features (speech rate and F0) that distinguish high- and low-intimacy speech. source: §5.1 evidence: 'Objective evaluation confirms the TTS model captures intimacy: speech rate RMSE and log-F0 RMSE are lower when synthesized speech intimacy level matches the ground-truth label.' confidence: high relevance: high - claim_id: binary_intimacy_labels_are_sufficient_to_produce_measurable role: supports claim: Binary intimacy labels are sufficient to produce measurable prosodic differences in synthesised speech, suggesting that coarse-grained conditioning captures style variation in spontaneous dialogue. source: §5.1 evidence: Binary intimacy labels switch in sync with linguistic style changes across the three experiment days. confidence: high relevance: high - claim_id: paralinguistic_intimacy_may_be_less_effectively_conveyed_through role: supports claim: Paralinguistic intimacy may be less effectively conveyed through speech prosody alone than through linguistic register, as gains in friendliness ratings remain modest even when prosodic style shifts. source: §5.2 evidence: Binary intimacy labels switch in sync with linguistic style changes across the three experiment days. confidence: high relevance: high limitations: - Only binary (high/low) intimacy labels are used; a continuous intimacy scale might enable finer-grained adaptation. The experiment spans only 3 days; longer interactions may reveal larger effects. The CG agent's gesture and emotion were frozen, which may have suppressed multimodal intimacy cues. The study is Japanese-language only, limiting generalizability. Future work intends to incorporate multimodal behaviors (gestures, facial expressions) and longer-term experiments. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-0433 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - VC architecture: - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - disentangled_prosody_content_timbre claims: - claim_id: voice_conversion_architectures_designed_for_human_speech_require role: supports claim: Voice conversion architectures designed for human speech require non-trivial adaptation to generalise to non-human vocalizations with broad frequency ranges and transient-rich characteristics. source: §4.2.2, Table 1 evidence: Replacing the proposed preprocessing pipeline with a conventional speech-focused one degraded WER from 24.02% to 62.29% and MOS-S from 3.78 to 3.53, indicating that human-speech assumptions about frame resolution and frequency range materially impair non-human sound conversion. confidence: high relevance: low - claim_id: isolating_style_conditioning_to_the_prior_network_and role: supports claim: Isolating style conditioning to the prior network and normalizing flow, and excluding it from the posterior encoder and decoder, reduces style leakage and improves speaker similarity in CVAE-based voice conversion. source: §4.2.3, Table 1 evidence: Adding the style embedding to the audio encoder and decoder (w/ SEED ablation) reduced MOS-S from 3.78 to 3.61, attributed to style overlap between the reference encoder output and latent acoustic tokens. confidence: high relevance: medium - claim_id: kl_annealing_mitigates_posterior_collapse_in_vae_based role: supports claim: KL annealing mitigates posterior collapse in VAE-based voice conversion and improves linguistic content preservation, particularly for complex non-human vocalizations. source: §4.2.3, Table 1 evidence: Removing KL annealing increased CER from 15.48% to 28.89% and WER from 24.02% to 44.69%, while MOS scores changed minimally, indicating that linguistic clarity is the primary casualty of over-regularization in early training. confidence: high relevance: low - claim_id: existing_fundamental_frequency_f0_estimation_methods_are_not role: complicates claim: Existing fundamental frequency (F0) estimation methods are not reliable for sounds lacking a well-defined harmonic structure, constraining prosodic feature extraction in non-human voice conversion systems. source: §3.1 evidence: The authors tested frame-level F0 from non-human sounds but found existing estimators (Praat, CREPE, SPICE, PESTO) exhibited limitations due to absent harmonic structure; the system falls back to energy-only prosodic features, leaving robust F0 extraction as an open problem. confidence: high relevance: high limitations: - The dataset is entirely internal and the evaluation uses only 9 human raters on an unspecified number of test samples, limiting reproducibility and the statistical reliability of MOS scores. - No publicly available data or code is confirmed, making direct comparison and reproduction difficult. The evaluation benchmarks non-human VC against baselines that were not adapted for non-human sounds, which is the correct setup for the paper's argument but means absolute MOS values are not comparable to human-speech VC literature. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: interspeech-2025-0464 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - VC - codec architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - disentangled_prosody_content_timbre claims: - claim_id: explicit_mutual_information_minimisation_at_the_codec_embedding role: supports claim: Explicit mutual information minimisation at the codec-embedding level provides effective prosody-timbre disentanglement for voice conversion. source: §3.5, Table 3 evidence: Removing the MI loss (L_MI) from the full system leads to a notably higher normalised F0 distance in the prosody-from-source scenario (3.28 vs. 2.82), while quality and timbre metrics change only modestly, isolating prosody control as the primary benefit of the MI objective. confidence: high relevance: high - claim_id: in_context_learning_codec_lms_can_serve_as role: supports claim: In-context learning codec LMs can serve as controllable VC backbones when augmented with prosody-disentangling encoder modules. source: §3.4, §3.5, Table 2, Table 3 evidence: The proposed system builds on VALL-E X's ICL mechanism and outperforms VALL-E X in speaker similarity (ASV 0.91 vs. 0.84), intelligibility (WER 0.101 vs. 0.115), naturalness (MOS 4.36 vs. 4.19), and prosody alignment (F0 distance 2.70 vs. 3.10) in the prompt-based scenario. confidence: high relevance: high - claim_id: prosody_disentanglement_at_the_codec_level_introduces_a role: complicates claim: Prosody disentanglement at the codec level introduces a small trade-off in absolute codec reconstruction fidelity compared to the unmodified encoder. source: §3.3, Table 1 evidence: PACE's ASV score (0.662) and NISQA score (3.98) are lower than the baseline EnCodec encoder (0.681, 4.17), though the gap does not substantially affect system-level VC performance. confidence: high relevance: high - claim_id: prosody_from_source_and_prosody_from_prompt_are role: refines claim: Prosody-from-source and prosody-from-prompt are distinct capability axes in voice conversion; systems strong at one do not automatically handle the other. source: §3.5, Table 3 evidence: VALL-E X supports only prosody-from-prompt and is excluded from the source-prosody evaluation; TriAAN-VC and ProsoVC support only source-prosody and are excluded from the prompt-prosody evaluation. Only the proposed system is evaluated in both modes. confidence: high relevance: high limitations: - All evaluation is conducted on LibriTTS-clean-100 and test-clean, a relatively clean single-domain corpus with 247 speakers. Generalisation to noisy environments, expressive or emotional speech, or cross-lingual settings is not tested. - The 54-hour training dataset is modest for a codec language model approach; it is unclear whether the disentanglement quality degrades with longer or more expressive source utterances. No code or demo is reported, limiting reproducibility. The paper does not ablate the number of RVQ codebooks or the sensitivity of the MI-minimisation trade-off weight (lambda_MI), leaving the robustness of the disentanglement objective undercharacterised. Prosody is operationalised solely through f0 and UV binary flags; richer prosodic dimensions such as energy, speaking rate, and phrase-level structure are not captured. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-0723 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer claims: - claim_id: encoder_representations_in_trained_encoder_decoder_tts_models role: supports claim: Encoder representations in trained encoder-decoder TTS models encode prosodic and phonetic properties in a distributed, neuron-level format that is recoverable by linear classifiers. source: §3.1, §5.1 evidence: 'Acoustic Correlation Analysis: A logistic or linear regression classifier is trained to predict prosodic properties (pitch, duration, energy) or semantic tokens (HuBERT layer-6 k-means) from the encoder''s last-layer LSTM activations.' confidence: high relevance: high - claim_id: gradient_ascent_in_a_compressed_vae_latent_space role: supports claim: Gradient ascent in a compressed VAE latent space produces more controlled prosodic edits than direct activation manipulation in the ambient space, reducing off-manifold artifacts. source: §3.2.1, §5.2 evidence: 'Manifold Preserving CAE: To avoid pushing activations off the data manifold during gradient ascent, activations are first encoded into a beta-VAE latent space (dim=16).' confidence: high relevance: high - claim_id: anchoring_latent_edits_to_a_vq_vae_prototype role: supports claim: Anchoring latent edits to a VQ-VAE prototype codebook is necessary to preserve phoneme identity under large prosodic shifts at inference time. source: §3.2.2, §5.2 evidence: 'Prototype Loss: A VQ-VAE codebook anchors edits to the nearest discrete prototype, preventing large latent shifts from altering unintended speech content (phoneme identity during prosody edits, and vice versa).' confidence: high relevance: high - claim_id: post_hoc_activation_editing_can_correct_mispronunciations_in role: supports claim: Post-hoc activation editing can correct mispronunciations in grapheme-input TTS without a pronunciation dictionary, using a speech-only correction query as the sole supervision signal. source: §5.3, Table 1 evidence: For pronunciation correction, the target is to adjust semantic tokens (HuBERT k-means clusters) per grapheme to match a speech-only correction query. confidence: high relevance: low limitations: - '- Validated only on Tacotron 2; applicability to flow-matching or codec-based LM TTS is untested. - Feature entanglement remains: controlling duration slightly shifts pitch (acknowledged in the paper), suggesting that targeting specific neurons rather than full activation vectors is needed. - Mispronunciation correction requires a speech-only query word as supervision — not purely zero-shot. - LJSpeech is single-speaker; multi-speaker generalization is not evaluated.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-0754 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - emotion_expressive_prosody_control claims: - claim_id: expressive_emotional_prosody_can_suppress_or_distort_intended role: supports claim: Expressive emotional prosody can suppress or distort intended word-level emphasis without explicit attention-level intervention, degrading perceptual emphasis clarity particularly in high-arousal emotions. source: §1, §2.4 evidence: 'Key additions: Weakly supervised emphasis labeling: The EmphaClass recognizer (an SSL model fine-tuned for frame-level emphasis detection) is applied to the English portion of the Emotional Speech Database (ESD, 10 speakers, 5 emotions, ~1.2 hours/speaker) to generate pseudo-labels marking emphasized word boundaries.' confidence: high relevance: high - claim_id: variance_based_pitch_and_duration_features_derived_from role: supports claim: Variance-based pitch and duration features derived from emphasis pseudo-labels provide effective local prosody modulation signals for jointly supervised emphasis and emotion control in TTS. source: §2.3 evidence: 'Variance-based emphasis features: Rather than injecting emphasis as a binary switch, EME-TTS models pitch variance and duration variance — the deviation of emphasized regions from sentence-level averages.' confidence: high relevance: high - claim_id: an_attention_bias_mechanism_targeting_predefined_emphasis_positions role: supports claim: An attention-bias mechanism targeting predefined emphasis positions improves listener emphasis recognition accuracy across emotion categories compared to a system without such a mechanism. source: §3.2.1, Table 1 evidence: 'Each EPE block combines: - Multi-Head Attention for context - Conditional Cross Attention (CCA) that re-weights self-attention using the emotion embedding - An Emphasis Adapter (EA) that adds a masked bias to attention weights at the designated emphasis positions, scaled by a learned `strength` parameter (set to 0.2) - Conditional Layer Normalization integrating the emotion embedding During inference, emphasis positions are predicted by GPT-4 given the emotion label and input text, eliminating the need for manual annotation.' confidence: high relevance: high - claim_id: llm_predicted_emphasis_positions_can_substitute_for_manual role: supports claim: LLM-predicted emphasis positions can substitute for manual annotation at inference time in emphasis-controllable emotional TTS, enabling controllable emphasis without per-utterance human labeling. source: §1, §3.1 evidence: 'Each EPE block combines: - Multi-Head Attention for context - Conditional Cross Attention (CCA) that re-weights self-attention using the emotion embedding - An Emphasis Adapter (EA) that adds a masked bias to attention weights at the designated emphasis positions, scaled by a learned `strength` parameter (set to 0.2) - Conditional Layer Normalization integrating the emotion embedding During inference, emphasis positions are predicted by GPT-4 given the emotion label and input text, eliminating the need for manual annotation.' confidence: high relevance: high - claim_id: joint_emphasis_emotion_modeling_improves_subjective_emotion_recognition role: complicates claim: Joint emphasis-emotion modeling improves subjective emotion recognition accuracy for difficult emotions (angry, sad) while leaving high-arousal emotions (happy, surprise) less changed. source: §3.2.2, Table 3 evidence: 'The EPE especially helps for "surprise" emotion (0.64 vs. 0.55), where pitch rise at sentence end otherwise creates false emphasis percepts. - Emotion Accuracy (subjective): EME-TTS achieves mean 0.67 vs. 0.58 for EmoSpeech and 0.48 for CosyVoice2.' confidence: high relevance: high limitations: - '- Dataset is small (ESD, ~1.2 hours/speaker, 5 emotions) and single-language (English); broader emotional diversity and multilingual transfer are untested. - EPE `strength` is a fixed hyperparameter (0.2); adaptive or learned strength per emotion is not explored. - Inference-time reliance on GPT-4 for emphasis prediction adds latency and cost; a lightweight alternative would be needed for deployment. - Happy emotion shows limited improvement, suggesting the emotion-emphasis interaction is not uniform.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-0762 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - flow-matching relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: recursive_psychometric_function_estimation_can_establish_a_perceptually role: supports claim: Recursive psychometric function estimation can establish a perceptually calibrated mapping from a continuous TTS conditioning parameter to perceived accentedness, yielding at least 9 distinguishable perceptual levels for English-accented Swedish. source: §3.2, §4.2 evidence: '- The sigmoid psychometric function for English-accented Swedish achieves R² = 0.983, validating the PEA scale. - For MAP-SV: at least 9 perceptually distinguishable EA levels within [0,1]; for MAP-EN: only 3. - Aligned transcription+PEA conditions receive significantly more "best" votes (χ²(2, N=630) = 50.2, p < .05). - No single PEA setting dominates across all insertion types in PREF-VAR; both fully Swedish (sv-0.00) and intermediate English (en-0.67) receive notable best-vote counts.' confidence: high relevance: low - claim_id: misalignment_between_transcription_style_and_accentedness_level_is role: supports claim: Misalignment between transcription style and accentedness level is consistently dispreferred by listeners, regardless of which misaligned combination is presented. source: §3.3, §4.3, Table 1 evidence: Aligned conditions (sv-0, en-1) receive significantly more "best" votes than misaligned (sv-1, en-0), confirming that transcription style must match accentedness. 4. confidence: high relevance: medium - claim_id: listener_preference_for_accentedness_in_intrasentential_english_insertions role: supports claim: Listener preference for accentedness in intrasentential English insertions is insertion-type-dependent rather than uniform, requiring fine-grained PEA control beyond binary Swedish/English switching. source: §3.4, §4.4, Table 1 evidence: '- The sigmoid psychometric function for English-accented Swedish achieves R² = 0.983, validating the PEA scale. - For MAP-SV: at least 9 perceptually distinguishable EA levels within [0,1]; for MAP-EN: only 3. - Aligned transcription+PEA conditions receive significantly more "best" votes (χ²(2, N=630) = 50.2, p < .05). - No single PEA setting dominates across all insertion types in PREF-VAR; both fully Swedish (sv-0.00) and intermediate English (en-0.67) receive notable best-vote counts.' confidence: high relevance: low - claim_id: perceptual_sensitivity_to_accent_direction_is_asymmetric_listeners role: supports claim: 'Perceptual sensitivity to accent direction is asymmetric: listeners can distinguish more granular levels of English-accentedness in Swedish speech than of Swedish-accentedness in English speech.' source: §4.2 evidence: '- The sigmoid psychometric function for English-accented Swedish achieves R² = 0.983, validating the PEA scale. - For MAP-SV: at least 9 perceptually distinguishable EA levels within [0,1]; for MAP-EN: only 3. - Aligned transcription+PEA conditions receive significantly more "best" votes (χ²(2, N=630) = 50.2, p < .05). - No single PEA setting dominates across all insertion types in PREF-VAR; both fully Swedish (sv-0.00) and intermediate English (en-0.67) receive notable best-vote counts.' confidence: high relevance: low limitations: - '- Single speaker; generalization to other Swedish voices or to multilingual models with many speakers is unclear. - Listener panels are small (16–32 per experiment), and the paper acknowledges insufficient power for detailed analysis of insertion-type effects. - Only one language pair (Swedish/English); the methodology could generalize to other code-switching scenarios but was not tested. - The recursive psychometric fitting procedure requires significant respondent time per EA comparison set. - The matrix result is missing one sentence due to a technical failure.' caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: interspeech-2025-0815 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - VC architecture: - VAE - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - emotion_expressive_prosody_control claims: - claim_id: discrete_speech_unit_representations_reduce_source_speaker_leakage role: supports claim: Discrete speech unit representations reduce source speaker leakage in voice conversion but introduce pronunciation artefacts that degrade intelligibility compared to continuous feature counterparts. source: §4.2, Table 1 evidence: Content representations are derived from mHuBERT-147 discrete speech units rather than continuous features; quantisation suppresses speaker-dependent information that continuous representations would carry, and the multilingual nature of mHuBERT-147 enables zero-shot cross-lingual conversion. confidence: high relevance: low - claim_id: mix_style_layer_normalisation_mitigates_the_train_inference role: supports claim: Mix-style layer normalisation mitigates the train-inference mismatch caused by content-style dependence in style encoders, improving zero-shot generalisation on unseen speakers. source: §4.3, Table 2 evidence: 'Ablation results (Table 2) confirm that all four proposed components contribute: removing F0 cross-attention degrades SECS by 0.6pp and ECA by 12.5pp; removing Mix-LN causes the most severe WER degradation on LibriTTS (19.83%), consistent with the hypothesis that Mix-LN reduces the train-inference content-style mismatch; removing the similarity loss weakens unseen-speaker conversion; and replacing mHuBERT with MMS features lowers WER but substantially reduces emotion transfer.' confidence: high relevance: medium - claim_id: enriching_global_style_embeddings_with_explicit_pitch_and role: supports claim: Enriching global style embeddings with explicit pitch and energy features improves emotion transfer fidelity in expressive voice conversion beyond mel-spectrogram-only style encoding. source: §3.5, §4.3, Table 2 evidence: 'Ablation results (Table 2) confirm that all four proposed components contribute: removing F0 cross-attention degrades SECS by 0.6pp and ECA by 12.5pp; removing Mix-LN causes the most severe WER degradation on LibriTTS (19.83%), consistent with the hypothesis that Mix-LN reduces the train-inference content-style mismatch; removing the similarity loss weakens unseen-speaker conversion; and replacing mHuBERT with MMS features lowers WER but substantially reduces emotion transfer.' confidence: high relevance: high - claim_id: cross_attention_fusion_of_local_f0_contours_with role: supports claim: Cross-attention fusion of local F0 contours with content embeddings produces stronger prosodic alignment to the target than additive F0 injection in non-autoregressive voice conversion. source: §3.1, §4.3, Table 2 evidence: Local F0 contours are extracted at the same frame rate as the content embeddings (320 samples/frame) and fused with them via multi-head cross-attention, with content as query and F0 as key/value. confidence: high relevance: high - claim_id: zero_shot_cross_lingual_voice_conversion_is_achievable role: supports claim: Zero-shot cross-lingual voice conversion is achievable with a monolingual training corpus when content representations are extracted from a multilingual speech model, though intelligibility degrades for unseen source languages. source: §4.4, Table 4 evidence: Content representations are derived from mHuBERT-147 discrete speech units rather than continuous features; quantisation suppresses speaker-dependent information that continuous representations would carry, and the multilingual nature of mHuBERT-147 enables zero-shot cross-lingual conversion. confidence: high relevance: low limitations: - 'The proposed system incurs a substantial WER penalty relative to baselines: 7.98% vs. 5.01% (ESD) and 8.84% vs. 3.48% (LibriTTS) for the full model, with discrete units identified as the cause. This intelligibility regression is acknowledged but not resolved; future work is deferred.' - The evaluation uses a small subjective panel (15 listeners, 10–15 samples per model), limiting the statistical power of MOS comparisons. The cross-lingual results are restricted to English and German; how performance degrades for more distant language pairs is untested. The model is trained on English-only data, and German-to-English conversion shows a 30.84% WER, suggesting significant cross-lingual generalisation limits. The model size and computational cost are not reported, making it difficult to assess deployment feasibility. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-0902 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - VC architecture: - VAE - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - voice_quality_and_paralinguistic_feature_control claims: - claim_id: explicit_conditioning_on_acoustic_voice_quality_features_cpps role: supports claim: Explicit conditioning on acoustic voice quality features (CPPS, H1-H2, H1-A3, creakiness) enables controllable manipulation of phonation type in a VC system. source: §3.2, §4.1 evidence: The key addition is five voice quality encoders — one per feature (creakiness, CPPS, H1-H2, H1-A3, pitch) — each implemented as an affine layer with the same dimensionality as the WavLM representation. confidence: high relevance: medium - claim_id: breathy_voice_is_perceived_as_more_intimate_and role: supports claim: Breathy voice is perceived as more intimate and more invested than modal voice when paralinguistic content is held constant across speakers. source: §5.2, Table 1 evidence: 'Subjective evaluation (25 participants per condition, Prolific, £12/hr, ~7.35 min per test): - Breathy vs. modal voice: Breathy rated significantly more intimate (p=.01) and more invested (p<.01).' confidence: high relevance: medium - claim_id: creaky_voice_is_perceived_as_less_intimate_and role: supports claim: Creaky voice is perceived as less intimate and less positive than modal voice, consistent with prior findings using natural speech stimuli. source: §5.2, Table 1 evidence: 'No significant valence difference. - Creaky vs. modal voice: Creaky rated significantly less intimate (p=.04) and less positive (p<.01).' confidence: high relevance: low - claim_id: fine_tuning_an_existing_vc_backbone_with_lightweight role: supports claim: Fine-tuning an existing VC backbone with lightweight feature encoders is sufficient to achieve intuitive control over multiple acoustic correlates of voice quality without degrading the base model's conversion capability. source: §3.3, §4.1 evidence: The key addition is five voice quality encoders — one per feature (creakiness, CPPS, H1-H2, H1-A3, pitch) — each implemented as an affine layer with the same dimensionality as the WavLM representation. confidence: high relevance: medium limitations: - '- CPPS is measured over voiced segments rather than sustained vowels, introducing minor irregularities. - Pitch control generalizes imperfectly across genders due to corpus-level standardization combining both genders. - The system models acoustic correlates of voice quality rather than voice quality per se; phonetic validation by trained phoneticians would strengthen claims. - No MUSHRA or absolute MOS for speech naturalness — evaluation focused on paralinguistic perception rather than synthesis fidelity. - Integration into live dialogue systems for adaptive voice quality in conversation is proposed as future work.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: interspeech-2025-0948 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - VC architecture: - VAE - diffusion relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - stochastic_generative_prosody_model - emotion_expressive_prosody_control claims: - claim_id: natural_language_prompts_enable_more_flexible_and_subjectively role: supports claim: Natural language prompts enable more flexible and subjectively accurate emotion control in voice conversion than numeric intensity values or reference audio selection. source: §1, §3.4 evidence: An emotion descriptor uses a pre-trained RoBERTa model followed by a linear projection to convert a natural-language description into a coarse emotion embedding. confidence: high relevance: medium - claim_id: a_diffusion_based_mapping_from_text_embeddings_to role: supports claim: A diffusion-based mapping from text embeddings to speech emotion embeddings is sufficient to replace reference audio at inference time without significant quality loss. source: §2.1, §3.2, Table 1 evidence: This embedding is then refined by a prompt mapper — a score-based diffusion model implemented as stacked Transformer encoder layers — that is trained to predict the fine-grained emotion representation from the coarser text embedding, conditioned on reference embeddings extracted by the Emotion2Vec model (a large-scale SSL model for speech emotion). confidence: high relevance: medium - claim_id: joint_training_of_a_text_to_emotion_mapper role: supports claim: Joint training of a text-to-emotion mapper with reference emotion embeddings improves prosody naturalness over direct prediction from text alone. source: §3.3, Table 1 evidence: This embedding is then refined by a prompt mapper — a score-based diffusion model implemented as stacked Transformer encoder layers — that is trained to predict the fine-grained emotion representation from the coarser text embedding, conditioned on reference embeddings extracted by the Emotion2Vec model (a large-scale SSL model for speech emotion). confidence: high relevance: high - claim_id: preserving_speaker_identity_during_emotional_pitch_manipulation_requires role: supports claim: Preserving speaker identity during emotional pitch manipulation requires an explicit F0 constraint in the speaker encoder; adversarial training alone is insufficient. source: §2.3, §3.3, Table 1 evidence: Speaker identity is preserved by an augmented speaker verification model trained with an F0 regression constraint (log-F0 L2 loss), discouraging identity drift when pitch is manipulated. confidence: high relevance: high - claim_id: mixed_emotion_synthesis_remains_harder_to_control_than role: complicates claim: Mixed-emotion synthesis remains harder to control than single-category emotion intensity across both subjective and objective metrics. source: §3.4, Table 2, Table 3 evidence: 'Objective metrics follow the same pattern: MCD 4.70, CER 4.09%, and log-F0 RMSE 42.58, all best among the five systems tested (Table 1).' confidence: high relevance: medium limitations: - Training and evaluation are conducted entirely on TextrolSpeech, a single corpus with a limited speaker set. Generalisation to out-of-domain speakers, languages, or acoustic conditions is untested, and all reported numbers should be interpreted within that constraint. - The evaluation uses only 25 listeners for subjective MOS across 132 utterances — a borderline sample size that may limit statistical reliability. The mixed-emotion accuracy (61.3%) is notably lower than single-attribute control, and the system's handling of complex emotional blends (e.g., contempt with happiness) is not analysed in depth. The discrete HuBERT token approach for linguistic content may introduce quantisation artefacts not reported in the paper. Future real-time or streaming deployment, mentioned in the conclusion as a direction, is not addressed in the current architecture. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-1020 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: - VAE relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control claims: - claim_id: larger_codebook_sizes_are_the_primary_lever_for role: supports claim: Larger codebook sizes are the primary lever for improving discrete prosody representation quality, with embedding vector dimensionality having minimal independent effect. source: §5 evidence: Across 150+ experiments, FFE improves monotonically with codebook bin count (20 to 320), while embedding size had "surprisingly insignificant" effect with smaller sizes often outperforming larger ones. confidence: high relevance: high - claim_id: joint_discrete_encoding_of_f0_and_energy_is role: supports claim: Joint discrete encoding of F0 and Energy is achievable with minimal reconstruction trade-off relative to F0-only encoding, when codebook capacity is matched. source: §5, Table 5 evidence: The FI strategy reaches 0.49% FFE 20 % while simultaneously embedding Energy at 6.33 dB RMSE, outperforming the F0-only reference (1.60% FFE 20 %) from prior work. confidence: high relevance: high - claim_id: speaker_normalisation_of_f0_improves_energy_reconstruction_but role: complicates claim: Speaker normalisation of F0 improves energy reconstruction but degrades F0 reconstruction accuracy when unvoiced regions must be preserved. source: §5, Tables 5-6 evidence: FN+EN+VM achieves energy RMSE of 3.00 dB vs 6.33 dB for FI, but FFE 20 % rises to 3.30% vs 0.49% for FI, indicating a direct quality trade-off between unvoiced-region handling and energy embedding fidelity. confidence: high relevance: high - claim_id: downstream_utility_of_prosody_embeddings_in_tts_or role: complicates claim: Downstream utility of prosody embeddings in TTS or speech LM systems is unverified; embedding quality is measured only through reconstruction accuracy rather than impact on synthesis naturalness. source: §4, §6 evidence: Evaluation is limited to VDE and FFE reconstruction metrics on LibriTTS; no TTS or downstream speech generation experiments are reported. confidence: high relevance: high limitations: - No downstream TTS evaluation is provided, so the practical benefit of these embeddings for synthesis quality is undemonstrated. The evaluation is limited to English multispeaker speech (LibriTTS), and it is unclear how well the learned codebooks would transfer to other languages or acoustic conditions. The paper excludes audio files shorter than one second from training, which may bias the codebook away from short utterances common in conversational speech. Future work proposed by the authors involves prosody-conditioned text generation, which would serve as the first downstream validation of the released embeddings. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-1098 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer claims: - claim_id: bert_based_prosody_predictors_for_pitch_accent_languages role: supports claim: BERT-based prosody predictors for pitch-accent languages can be extended to multi-speaker settings through speaker style embedding injection without requiring accent label annotation. source: §2.1, §3.4, Table 2, Table 3 evidence: GST-BERT-TTS integrates averaged GST vectors into BERT token embeddings via a linear adaptor, reducing log f0 MSE from 0.773 (fo-BERT) to 0.266 and raising top-level accent correctness from 47.5% to 65% on Hi-Fi-CAPTAIN. confidence: high relevance: high - claim_id: speaker_style_embeddings_derived_from_seen_speaker_averages role: complicates claim: Speaker style embeddings derived from seen-speaker averages do not generalise well to unseen speakers in label-free prosody prediction. source: §4.1 evidence: The authors report that using the GST embedding of an unseen speaker during inference produced flat, monotonous prosody, attributing the failure to insufficient speaker diversity in the training corpus and limited coverage of GST's latent space. confidence: high relevance: high - claim_id: improvements_in_prosody_prediction_accuracy_and_accent_correctness role: complicates claim: Improvements in prosody prediction accuracy and accent correctness do not automatically translate into higher overall MOS in multi-speaker TTS. source: §3.4, Figure 3 evidence: GST-BERT-TTS outperforms fo-BERT on MSE and accent correctness, yet MOS scores remain lower than TTS+Accent; the authors attribute this partly to vocoder artefacts on a high-variability multi-speaker corpus and to the fact that MOS conflates naturalness and audio clarity with prosodic correctness. confidence: high relevance: high - claim_id: multi_task_prosody_prediction_covering_fundamental_frequency_energy role: supports claim: Multi-task prosody prediction covering fundamental frequency, energy, and duration jointly improves over pitch-only prediction when speaker conditioning is present. source: §3.4, Table 2 evidence: Adding energy and duration prediction to fo-BERT (full params.) had minimal impact on log f0 MSE (0.773 vs. 0.776), whereas GST-BERT (full params.) showed a synergistic improvement over GST-BERT (log f0 only), reducing log f0 MSE from 0.302 to 0.266. confidence: high relevance: high limitations: - Evaluation uses a proprietary internal Japanese corpus; the accent correctness listening test involves only two speakers. Generalisation of the approach to other pitch-accent languages, larger speaker sets, or publicly available benchmarks is untested. - GST embeddings fail to provide useful conditioning for genuinely unseen speakers, limiting the method's zero-shot applicability. Addressing this may require a larger and more diverse speaker pool or a different style representation strategy. The authors acknowledge a tension between BERT's strength in contextual linguistic modelling and its suitability for constrained, character-level prosodic parameter regression, and identify integration with large language models as a direction requiring future methodological work. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-1192 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: minor method_family: - structured_language_prosody_control claims: - claim_id: adversarial_disentanglement_via_a_gradient_reversal_layer_can role: supports claim: Adversarial disentanglement via a gradient reversal layer can separate voice impression information from speaker identity in a zero-shot TTS speaker encoder, enabling independent modulation of perceived voice characteristics. source: §2.2, §4.1, Figure 5 evidence: The control module applies GRL + 0.8 dropout to the speaker embedding to remove impression signal, then reintroduces it via an 11-dim impression vector; cosine similarity to the target speaker remains above the inter-speaker distribution at all tested modulation levels. confidence: high relevance: low - claim_id: llms_can_generate_low_dimensional_speech_style_parameter role: supports claim: LLMs can generate low-dimensional speech style parameter vectors from free-form natural language descriptions, providing a usable zero-shot interface for voice characteristic control. source: §2.3, §4.3, Table 4 evidence: ChatGPT-4o prompted with dimension definitions and pre-modulation values produces impression vectors preferred over unmodulated speech in 94.6% of "sleepy" trials and 74.6% of "urgent, attention-grabbing" trials (n=436, crowdsourced). confidence: high relevance: medium - claim_id: fine_grained_impression_control_in_tts_involves_a role: complicates claim: 'Fine-grained impression control in TTS involves a trade-off: stronger modulation produces more perceptually distinct impressions but degrades naturalness, particularly when the target impression is socially incongruent with the source speaker.' source: §4.2, Table 3 evidence: MOS naturalness at maximum modulation (±3) falls to 2.71-2.88 for the Powerful-Weak and Dark-Bright dimensions, compared to 3.61-3.72 at zero modulation; low-scoring samples correspond to combinations such as "feminine male" or "strongly dark/bright." confidence: high relevance: low - claim_id: automatic_annotation_of_high_dimensional_perceptual_voice_attributes role: complicates claim: Automatic annotation of high-dimensional perceptual voice attributes at training scale requires indirect labeling pipelines that introduce estimation error, limiting the precision of supervision. source: §3.2 evidence: Crowdsourced subjective ratings were collected for only 1,154 of 20,270 speakers; an HuBERT-based estimator extrapolated labels to the remaining data with an RMSE of 0.338 on held-out utterances. confidence: high relevance: low limitations: - All training and evaluation data is a proprietary in-house Japanese corpus; no public datasets are used, and the evaluation spans only two held-out speakers. The results are not directly reproducible, and generalization to other languages, speaking domains, or TTS architectures is untested. - Eleven impression dimensions exhibit substantial inter-correlation (Table 1 reports correlations up to 0.8, e.g., Thick-Thin vs. High-Low Pitched), which means independent perceptual dimensions are not fully captured by the vector design. The paper demonstrates stable simultaneous two-dimension modulation but does not test combinations across weakly correlated dimensions at extreme values. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: interspeech-2025-1334 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: multi_scale_neural_feature_extraction_that_captures_cross role: supports claim: Multi-scale neural feature extraction that captures cross-frequency coupling improves iEEG-to-speech intelligibility over single-band or handcrafted acoustic feature approaches. source: §2.1.1, §2.1.2, Table 1 evidence: MiSTR's wavelet encoder (DWT with Daubechies-4) combined with Phase-Amplitude Coupling features achieves Pearson correlation 0.91 and STOI 0.73, compared to 0.72 and 0.61 for a regression baseline relying on simpler acoustic features from the same dataset. confidence: high relevance: low - claim_id: transformer_based_sequence_decoders_outperform_rnn_and_cnn role: supports claim: Transformer-based sequence decoders outperform RNN and CNN architectures for Mel spectrogram reconstruction from neural signals. source: §2.2.2, §3.3, Table 1 evidence: MiSTR Transformer decoder achieves STOI 0.73 against 0.48 (bLSTM), 0.52 (CNN), 0.56 (3D-CNN), 0.59 (Seq2Seq), and 0.64 (encoder-decoder) on the same iEEG dataset under identical evaluation conditions. confidence: high relevance: low - claim_id: harmonic_phase_constrained_iterative_phase_reconstruction_reduces_spectral role: supports claim: Harmonic-phase-constrained iterative phase reconstruction reduces spectral artifacts in neural speech synthesis compared to unconstrained vocoding. source: §2.3, §4, Table 1, Figure 2 evidence: 'IHPR achieves HNR 12.7 dB, over 1.6 dB above the best baseline (encoder-decoder: 11.1 dB), and MiSTR''s spectrograms preserve harmonic structures in high-frequency regions where baselines exhibit blurring and artifact distortions.' confidence: high relevance: low - claim_id: automated_perceptual_quality_estimators_used_as_proxies_for role: complicates claim: Automated perceptual quality estimators used as proxies for human listening tests may not provide sufficient validation for clinical or neuroprosthetics applications of neural speech synthesis. source: §3.3, §4 evidence: MiSTR reports perceptual quality solely through MOSA-Net (MOSA-Net score 3.38), explicitly framing it as an alternative to "time-consuming listening tests." No human listener evaluation is conducted, leaving the perceptual validity of the improvements in intelligibility and naturalness unverified. confidence: high relevance: low limitations: - The evaluation is conducted on a single dataset of 10 participants with pharmacoresistant epilepsy implanted with depth electrodes (Dutch, iEEG). All perceptual quality comparisons rely on MOSA-Net rather than human listener tests. No cross-subject generalization to unseen individuals is assessed. - The dataset's clinical and linguistic specificity (Dutch epilepsy patients with implanted electrodes) limits how broadly the results generalize. iEEG-based speech synthesis requires invasive electrode placement, making large-scale data collection and multi-subject generalization substantially more difficult than standard TTS. The paper uses an F0 proxy derived from the neural signal rather than actual acoustic F0, which may limit the fidelity of the prosodic features. Future directions proposed by the authors include end-to-end decoding without intermediate representations and integration of diffusion-based waveform generation. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: interspeech-2025-1494 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - multimodal_visual_prosody_conditioning - prosody_analysis_and_evaluation_method claims: - claim_id: visual_features_extracted_from_video_carry_prosodic_information role: supports claim: Visual features extracted from video carry prosodic information that complements text-derived features for pitch and duration prediction in TTS. source: §3.2, Table 2; §3.3, Table 3 evidence: A feedforward network trained solely on Omnivore visual features achieves substantially lower pitch and energy MSE than a mean-predictor baseline on CMD2, and combining visual features with text reduces pitch MSE from 0.43 to 0.39 and duration MSE by 3% relative over text-only PED prediction. confidence: high relevance: high - claim_id: visual_context_conditioning_does_not_uniformly_improve_all role: complicates claim: Visual context conditioning does not uniformly improve all acoustic dimensions in prosody prediction. source: §3.3, Table 3; §3.4, Table 4 evidence: Adding visual features to the text-based PED predictor degrades energy MSE from 0.50 (text-only) to 0.59 (Text+VF-Omnivore), and energy improvements in the full TTS system are the smallest of the three prosody dimensions (5% relative vs. 33% for pitch and 49% for duration). confidence: high relevance: high - claim_id: objective_prosody_metrics_are_insufficient_to_verify_perceptual role: complicates claim: Objective prosody metrics are insufficient to verify perceptual quality gains in visually-conditioned TTS systems. source: §3.4, Table 5 evidence: VisualSpeech is evaluated entirely on MCD, Log F0 RMSE, pitch/energy/duration MSE, and STOI/PESQ; no human listening test or MOS study is reported, leaving the perceptual significance of the measured improvements unconfirmed. confidence: high relevance: high - claim_id: cross_attention_fusion_between_visual_and_text_encoder role: supports claim: Cross-attention fusion between visual and text encoder representations enables modality integration in sequence-to-sequence prosody prediction without requiring equal-length sequences. source: §2.3 evidence: The visual-text fusion module addresses the length mismatch between the phoneme sequence and the variable-length visual feature sequence by using text queries attending to visual key-value pairs, with output aligned to the phoneme level for per-phone PED prediction. confidence: high relevance: high limitations: - No human evaluation is conducted. All improvements are measured on objective metrics (MCD, Log F0 RMSE, prosody MSE), and it is unknown whether the observed gains translate to perceptibly more natural or contextually appropriate speech for listeners. - The CMD2 dataset is derived from movie audio which contains residual noise, music, and variable recording conditions despite preprocessing; the paper acknowledges this directly as a limitation of the current study. The approach is tested only on English speech from a movie domain, and generalisability to other languages, read speech, or clean studio recordings is untested. The visual features used (Omnivore, ResNet50) are general-purpose video and image recognition models, not speech-specific; whether domain-adapted visual encoders would yield larger prosody gains is an open question. The paper does not address how VisualSpeech would operate when video is unavailable at inference time, which limits its applicability to scenarios where visual input is always present. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-1726 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: - hybrid relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - language_and_domain_specific_prosody_modeling claims: - claim_id: fine_tuning_large_scale_tts_models_on_minimal role: supports claim: Fine-tuning large-scale TTS models on minimal scripted read-aloud data (order of minutes) produces personalised voices with perceived AAC suitability comparable to fine-tuning on hour-scale spontaneous recordings. source: §3.2, Table 1 evidence: FT-Read (12 min of read-aloud sentences, SECS 0.643) and FT-Spont-Short (1h spontaneous, SECS 0.706) show no significant difference in MUSHRA-rated listener preference (p=0.213) for a Hungarian stroke survivor case study. confidence: high relevance: low - claim_id: zero_shot_tts_adaptation_from_dysarthric_reference_audio role: supports claim: Zero-shot TTS adaptation from dysarthric reference audio achieves perceptual suitability for assistive communication comparable to adaptation from typical speech, enabling personalisation for users who lack pre-morbid recordings. source: §3.2 evidence: ZS-Atypical and ZS-Typical show no significant difference in MUSHRA-rated AAC suitability (p=0.683), despite a large objective speaker similarity gap (SECS 0.324 vs 0.635). confidence: high relevance: low - claim_id: embedding_based_objective_speaker_similarity_metrics_may_not role: refines claim: Embedding-based objective speaker similarity metrics may not reliably predict perceived suitability in assistive communication contexts, where other voice qualities matter more than timbre fidelity. source: §3.1, §3.2, Table 1 evidence: ZS-Atypical achieves the lowest SECS (0.324) of all evaluated systems yet is rated comparably to ZS-Typical (SECS 0.635) by human listeners in the MUSHRA-like AAC suitability test (p=0.683). confidence: high relevance: low - claim_id: using_dysarthric_speech_as_a_prosodic_reference_input role: supports claim: Using dysarthric speech as a prosodic reference input for a fine-tuned TTS system yields measurable alignment with the speaker's intended prosodic pattern while preserving intelligibility. source: §5.2 evidence: Dysarthric-prompted synthesis achieved 29.3% higher SSIM in scalogram comparison of prosodic peaks versus unprompted synthesis (0.0548 vs 0.0424, p=0.0013 by binomial test); CER increased only from 2.9% to 2.3% (actually improved) when switching to dysarthric-prompted synthesis. confidence: high relevance: high limitations: - The entire evaluation rests on a single research participant (one Hungarian stroke survivor). Generalisability across speakers, languages, and impairment types is untested, and the authors acknowledge this directly. - The interface requires typed text input, making it inaccessible for AAC users with motor impairments. The paper treats the interface as an evaluation platform rather than a functional device; eye-tracking compatibility is noted as future work. The prosody adaptation mechanism relies on the fact that for identical lexical content, prosodic patterns partially transfer through the audio prompt, which may not hold for diverse sentence types or severe dysarthria cases. The study also evaluates only XTTS-v2 and does not compare against other zero-shot or fine-tuning approaches that have emerged in parallel. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: interspeech-2025-1940 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - stochastic_generative_prosody_model claims: - claim_id: stochastic_prosody_predictors_achieve_naturalness_parity_with_human role: supports claim: Stochastic prosody predictors achieve naturalness parity with human recordings while enabling prosodic diversity that deterministic predictors cannot produce. source: §3.4, Figure 5, Figure 6 evidence: RF at 0.4 temperature shows no statistically significant difference from human naturalness ratings (Kruskal-Wallis, Dunn post-hoc); the deterministic baseline achieves highest naturalness but lowest diversity across 40 Prolific raters on ADEPT. confidence: high relevance: high - claim_id: for_explicit_prosody_modeling_cascaded_prediction_of_prosodic role: refines claim: For explicit prosody modeling, cascaded prediction of prosodic features outperforms joint prediction for duration but the ordering of pitch and energy within the cascade has negligible impact. source: §3.3, Table 1 evidence: Table 1 shows joint prediction yields 17% higher duration JS divergence (0.562) vs. cascading (0.481), while pitch-first and energy-first orderings differ by less than 0.005 across all prosodic parameters. confidence: high relevance: high - claim_id: naturalness_and_prosodic_diversity_are_inversely_correlated_constraining role: complicates claim: Naturalness and prosodic diversity are inversely correlated, constraining simultaneous optimization of both properties in TTS. source: §3.4, Figure 5, Figure 6 evidence: Subjective ratings show no system achieves both maximal naturalness and maximal diversity; even human speech exhibits this trade-off, with the human baseline scoring highest on diversity but not on naturalness. confidence: high relevance: high - claim_id: sampling_temperature_in_generative_prosody_predictors_enables_effective role: supports claim: Sampling temperature in generative prosody predictors enables effective runtime control over prosodic variability without retraining. source: §3.3, Figure 2, Figure 3 evidence: Pitch contour variance and duration variance increase monotonically with temperature for all stochastic methods (NF, CFM, RF), following a near-exponential relationship that allows a log-linear temperature axis for more intuitive user control. confidence: high relevance: high limitations: - Experiments are restricted to read speech (LibriTTS), with conversational speech explicitly deferred to future work. Prior work found significant benefits for stochastic duration prediction specifically in conversational scenarios, so the present findings may not generalize to that more demanding setting. The stochastic models do not yet capture the full diversity of human prosodic distributions, with synthetic KDE distributions consistently narrower and more unimodal than human reference distributions from RAVDESS; the authors hypothesize that larger predictor modules could close this gap. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-2032 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - signal_level_pitch_modification claims: - claim_id: explicit_prosody_conditioning_via_label_encoded_tokenization_enables role: supports claim: Explicit prosody conditioning via label-encoded tokenization enables word-level stress control in flow-based TTS without modifying the core model architecture. source: §4.1, Table 1 evidence: GlowEx TTS, which encodes stressed words as uppercase tokens in an expanded character lookup table within GlowTTS, achieves 88.76% true positive rate under supervised DNN stress detection and outperforms baseline GlowTTS in unsupervised K-Means classification (83.54% vs. 66.6%). confidence: high relevance: high - claim_id: duration_modification_is_a_more_reliable_cue_for role: supports claim: Duration modification is a more reliable cue for perceptible word-level stress exaggeration than energy modification in cascaded TTS-vocoder systems. source: §6.1, §6.2, Table 1 evidence: 'Duration-based WORLD vocoder modifications consistently achieve higher stress detection true positive rates and higher exaggeration quality MOS (Matcha+Duration: 97.59% TP, MOS 4.6) compared to energy-only modifications (Matcha+Energy: 88.76% TP, MOS 4.5) across all six TTS systems evaluated.' confidence: high relevance: high - claim_id: unsupervised_and_supervised_evaluation_methods_for_prosody_detection role: complicates claim: Unsupervised and supervised evaluation methods for prosody detection may disagree on the effectiveness of explicit stress conditioning approaches. source: §6.1, Table 1 evidence: GlowEx TTS outperforms baseline GlowTTS under unsupervised K-Means stress detection (83.54% vs. 66.6%) but shows nearly identical performance under supervised DNN classification (88.76% vs. 89.84%), suggesting the benefit of label conditioning is partially masked when classifiers are trained on the same prosody distribution. confidence: high relevance: high - claim_id: energy_modifications_in_post_hoc_vocoder_based_exaggeration role: complicates claim: Energy modifications in post-hoc vocoder-based exaggeration can reduce perceived naturalness even while boosting stress prominence, creating a trade-off for applications that require both. source: §6.2, Figure 4 evidence: Energy modifications consistently lower overall quality MOS relative to duration-only modifications across all cascaded TTS systems (e.g., Matcha overall MOS 4.5 with energy vs. 4.57 with duration), while duration modification preserves the better balance between exaggeration quality and naturalness. confidence: high relevance: high limitations: - The study is restricted to English with a small dataset (1000 sentences from Tatoeba, 900 training/100 validation) manually annotated by the authors, limiting statistical reliability and generalizability to other languages or speaking styles. All six TTS baselines in the cascaded strategy use pretrained models not fine-tuned to the Tatoeba domain. The forced alignment step introduces potential errors that are not independently evaluated. The perceptual study recruits 20 evaluators who are experts in speech and language processing, rather than actual L2 learners, which may not reflect the end-user experience. The authors identify suppression of non-target word prominence as future work. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-2159 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS - evaluation architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - linguistic_context_prosody_prediction claims: - claim_id: neural_tts_systems_trained_on_standard_read_speech role: complicates claim: Neural TTS systems trained on standard read-speech corpora do not reliably generate fine-grained prosodic distinctions required for syntactic disambiguation. source: §2.4, Table 3 evidence: Qualitative CQR assessment of five open-source systems (Parler-TTS, Coqui-XTTS, VoiceCraft, ToucanTTS, Matcha-TTS) found that none reliably produced distinct acoustic cues distinguishing comma vs. no-comma conditions; issues included voice instability, disfluencies, and failure to convey meaning differences between prosodic conditions. confidence: high relevance: high - claim_id: fine_tuning_a_tts_system_on_a_small role: supports claim: Fine-tuning a TTS system on a small targeted dataset with explicit prosodic contrasts can improve measurable prosodic distinctions for some sentence types. source: §3.4 evidence: 'Fine-tuning Matcha-TTS on 100 human-recorded sentences with controlled pause contrasts yielded statistically significant pause duration improvements for List-type sentences (unseen set first boundary: U = 0, p < 0.001, median 1.69 ms before vs. 107.81 ms after); improvement did not generalise to Direct Address types (p = 0.59).' confidence: high relevance: high - claim_id: targeted_fine_tuning_for_prosodic_control_does_not role: complicates claim: Targeted fine-tuning for prosodic control does not generalise uniformly across syntactically distinct prosodic boundary types. source: §3.4, §4 evidence: The same fine-tuning procedure that significantly improved List-type pause contrasts had no measurable effect on Direct Address types, suggesting the base model has systematic prosodic constraints (such as fixed stress placement on proper names) that a small dataset cannot override. confidence: high relevance: high - claim_id: prosodic_training_data_diversity_rather_than_model_architecture role: supports claim: Prosodic training data diversity, rather than model architecture alone, is a limiting factor in TTS prosody accuracy. source: §4, §5 evidence: The authors attribute systematic prosodic failures across all five evaluated systems to the absence of diverse prosodic patterns in standard training corpora (e.g., LJ Speech), and demonstrate that adding a 100-sentence targeted dataset produces measurable improvement in at least one prosodic boundary type. confidence: high relevance: high limitations: - The Study 1 evaluation is based on four stimuli per system assessed qualitatively by the authors alone, without external listeners or statistical analysis. Conclusions about which systems "fail" should be treated as suggestive observations rather than systematic evidence. - The fine-tuning experiment uses a single speaker for both training data collection and a small stimulus size (100 training sentences), with no multi-speaker generalisation tested. The acoustic analysis focuses solely on pause duration and does not account for F0, intensity, or lengthening cues that also mark prosodic boundaries. Matcha-TTS's continued failure on Direct Address types despite fine-tuning points to possible inductive biases in the base model's duration predictor that are not well understood. The authors propose extending the approach to combinations of pause, pitch, and lengthening in future work. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-2189 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - emotion_expressive_prosody_control claims: - claim_id: combining_acoustic_context_with_text_conditioning_in_a role: supports claim: Combining acoustic context with text conditioning in a masked prediction framework produces more accurate prosody prediction than style encoders or emotion representations adapted to the task. source: §4.1, Table 1 evidence: ProMode outperforms StyleTTS2*, Wav2Vec2-SER*, and Emotion2Vec* across all F0 and energy metrics at frame and phoneme level on GigaSpeech Test; since all models share the same decoder, performance differences isolate the encoder contribution. confidence: high relevance: high - claim_id: self_supervised_representations_trained_for_emotion_recognition_do role: complicates claim: Self-supervised representations trained for emotion recognition do not transfer effectively to prosody prediction, even after full fine-tuning on prosody-labelled data. source: §4.1, Table 1 evidence: Emotion2Vec* and Wav2Vec2-SER* show markedly lower F0 RPA and higher RMSE than StyleTTS2* and ProMode; Emotion2Vec* underperforms Wav2Vec2-SER* despite training on more labeled emotion data, suggesting the SER pre-training introduces a representation bias that hinders prosody regression. confidence: high relevance: high - claim_id: a_dual_decoder_architecture_with_an_auxiliary_acoustic role: supports claim: A dual-decoder architecture with an auxiliary acoustic-only loss prevents the prosody encoder from collapsing onto text context in masked prediction, where removing the loss causes catastrophic degradation. source: §4.3, Table 1 evidence: Ablation removing AOL drops F0 RPA from 43.9% to 25.8% (the largest degradation of any ablation), confirming that without AOL the decoder attends exclusively to text and the prosody encoder's contribution is suppressed. confidence: high relevance: high - claim_id: task_agnostic_prosody_models_that_predict_f0_independently role: supports claim: Task-agnostic prosody models that predict F0 independently can improve both naturalness and prosody preference in downstream TTS systems without requiring joint end-to-end training. source: §4.2, Table 2, §4.2.2 evidence: Replacing FluentSpeech's built-in pitch predictor with ProMode-predicted pitch improves UTMOS from 3.00 to 3.10, WER from 4.41% to 3.99%, and AutoPCP from 2.47 to 2.64; ABX prosody preference is statistically significant over all three baselines. confidence: high relevance: high limitations: - All experiments are conducted on English GigaSpeech (read and spontaneous speech); the authors identify multilingual extension as future work but provide no cross-lingual evidence. ProMode requires ground-truth phoneme durations from a forced aligner during evaluation, which creates a dependency on accurate alignment at inference time. The downstream integration only tests replacement of the pitch predictor within FluentSpeech; whether the benefit holds for other TTS backbones or for energy-conditioned synthesis is unevaluated. The model size is not reported, making compute cost comparisons to baselines unclear. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: interspeech-2025-2684 published_date: "2025-08-17" entry_date: '2026-07-25' year: 2025 venue: Interspeech task: - VC architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - disentangled_prosody_content_timbre - emotion_expressive_prosody_control claims: - claim_id: explicit_disentanglement_of_content_and_prosody_into_separate role: supports claim: Explicit disentanglement of content and prosody into separate discrete token spaces via different SSL representations enables independent control of speaking style in zero-shot voice conversion. source: §2.1, §3.2, Table 2 evidence: Using HuBERT (content) and ContentVec with SimVQ bottleneck (prosody) as separate extractors, with a prosody mask transformer for reference-guided style transfer, yields F0 Corr of 0.941 for prosody preservation and 0.822 for prosody conversion on ESD/VCTK, while ablations show that using a single mel-based prosody input significantly degrades performance. confidence: high relevance: high - claim_id: flow_matching_with_in_context_learning_achieves_competitive role: supports claim: Flow matching with in-context learning achieves competitive zero-shot voice conversion quality at substantially fewer parameters than larger autoregressive systems. source: §3.2, Table 1 evidence: Discl-VC (131M params) achieves MOS 4.31, UTMOS 4.079, and SECS 0.929 on VCTK zero-shot VC, surpassing Vevo (922M params) on naturalness and matching speaker similarity, demonstrating that compact flow matching transformers are competitive with large-scale AR systems for VC. confidence: high relevance: low - claim_id: prosody_transfer_from_a_reference_speaker_introduces_a role: complicates claim: Prosody transfer from a reference speaker introduces a trade-off with target speaker identity preservation. source: §3.2, Table 2 evidence: In the prosody conversion task, Discl-VC achieves lower SECS (0.847) than Vevo (0.892) despite better UTMOS and WER, suggesting that explicitly replacing source prosody with reference tokens disrupts timbre modeling and degrades speaker similarity relative to a system where prosody and identity are more entangled. confidence: high relevance: high - claim_id: simvq_freezing_codebook_vectors_and_learning_a_linear role: supports claim: SimVQ (freezing codebook vectors and learning a linear projection) reduces codebook collapse in VQ-based speech discretisation. source: §3.3, Table 3 evidence: Ablation replacing SimVQ with standard VQ causes degradation across all metrics (UTMOS 4.042 vs 4.079, WER 2.182% vs 1.946%, F0 Corr 0.970 vs 0.973, SECS 0.928 vs 0.929), attributed to codebook collapse during training. confidence: high relevance: low - claim_id: ssl_based_prosody_extraction_for_vc_can_retain role: complicates claim: SSL-based prosody extraction for VC can retain speaker-correlated information when the same-speaker assumption holds during training, causing train-inference mismatch. source: §3.3, Table 3 evidence: Ablation without ContentVec (using mel spectrogram first 20 dimensions as prosody input) shows the largest performance drop (UTMOS 4.034, WER 3.147%), attributed to prosody tokens containing residual speaker information because source and target audio come from the same speaker during training, creating a mismatch with cross-speaker inference. confidence: high relevance: high limitations: - Evaluation is limited to English and uses only two baselines (Vevo and FAcodec) on a relatively small test set (450 pairs for zero-shot VC). The prosody conversion task's lower speaker similarity suggests that the prosody mask transformer introduces identity leakage when reference prosody comes from a different speaker. The paper does not report results on multilingual speakers or noisy conditions, leaving generalisation untested. The two-stage training procedure adds complexity and requires the stage-1 encoder to be frozen before the prosody mask transformer can be trained. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.13028' published_date: "2025-08-18" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control - language_and_domain_specific_prosody_modeling claims: - claim_id: feedback_loss_from_a_sarcasm_classifier_can_be role: supports claim: Feedback loss from a sarcasm classifier can be integrated into TTS training to bias synthesized speech toward detector-recognizable sarcastic prosody. source: §2.2, §4.2, Table 2 evidence: 'Training follows three stages: (1) pre-training on LibriTTS for a read-speech baseline (800k iterations); (2) fine-tuning on 6.17 hours of conversational speech extracted from sitcom episodes (Friends, The Big Bang Theory) processed with Emilia-Pipe, to adapt prosody diversity; and (3) fine-tuning on the MUStARD++ sarcasm dataset (601 sarcastic, 601 non-sarcastic utterances) with the feedback loss active.' confidence: high relevance: high - claim_id: bi_modal_sarcasm_detection_that_combines_acoustic_features role: supports claim: Bi-modal sarcasm detection that combines acoustic features and text substantially outperforms audio-only detection, suggesting that sarcasm in speech is often semantically encoded and not recoverable from prosody alone. source: §4.1, Table 1 evidence: To this standard transformer encoder-decoder architecture, the authors add a bi-modal sarcasm detector that processes both speech (mel spectrogram through spectral convolution, temporal recurrence, and multi-head self-attention) and text (BERT embeddings). confidence: high relevance: high - claim_id: two_stage_fine_tuning_from_neutral_read_speech role: supports claim: Two-stage fine-tuning — from neutral read speech to conversational speech and then to target style — provides a viable data strategy for low-resource expressive speech synthesis. source: §2.3, §3.1 evidence: 'Training follows three stages: (1) pre-training on LibriTTS for a read-speech baseline (800k iterations); (2) fine-tuning on 6.17 hours of conversational speech extracted from sitcom episodes (Friends, The Big Bang Theory) processed with Emilia-Pipe, to adapt prosody diversity; and (3) fine-tuning on the MUStARD++ sarcasm dataset (601 sarcastic, 601 non-sarcastic utterances) with the feedback loss active.' confidence: high relevance: medium - claim_id: subjective_sarcasm_perception_in_listening_tests_is_difficult role: complicates claim: Subjective sarcasm perception in listening tests is difficult to isolate from the sarcastic content of the text, creating ambiguity in whether listeners respond to prosody or semantics. source: §5, §4.3 evidence: The subjective evaluation uses 13 listeners only, does not report an absolute MOS value or confidence intervals, and all stimuli are drawn from a sarcasm-labelled dataset, which may prime listeners toward attributing sarcasm regardless of prosody. confidence: high relevance: high limitations: - 'The evaluation design has a critical circularity: the same detector architecture trained on the same data distribution is used both as the training feedback signal and as the primary objective metric. Improvements in detection score on synthesized outputs are therefore expected by construction and cannot be treated as independent evidence of sarcasm-aware synthesis.' - The paper does not include an ablation that isolates the contribution of the feedback loss from the two-stage fine-tuning; it is therefore unclear which component drives the reported gains. The subjective study is small (13 listeners) and does not report MOS confidence intervals, making statistical significance uncertain. All evaluation stimuli are drawn from a sarcasm corpus, which may prime listener responses. The system has not been compared against more expressive modern TTS models (e.g., flow-matching or diffusion-based systems with style conditioning). Finally, the system is English-only and trained on North American sitcom speech, limiting generalizability to other languages, speakers, or sarcasm conventions. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.16332' published_date: "2025-08-22" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS - VC - singing architecture: - autoregressive-LM - flow-matching - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - explicit_acoustic_variance_control - emotion_expressive_prosody_control - signal_level_pitch_modification - speech_singing_shared_prosody claims: - claim_id: joint_pre_training_on_speech_and_singing_voice role: supports claim: Joint pre-training on speech and singing voice data improves generation quality in both domains, with singing data contributing to expressive speech prosody richness and speech data compensating for the limited size of singing corpora. source: §V-A, Table I evidence: On singing voice, Vevo2 substantially outperforms zero-shot TTS baselines in N-CMOS (gap > 1.5) and SS-CMOS (gap > 0.9), validating the benefit of joint training. confidence: high relevance: high - claim_id: chromagram_based_prosody_tokenization_provides_a_notation_free role: supports claim: Chromagram-based prosody tokenization provides a notation-free, octave-invariant representation that bridges the F0 distribution gap between speech and singing, enabling unified prosody control without expert MIDI annotations. source: §III-A evidence: 'Two custom VQ-VAE tokenizers are introduced: - Prosody tokenizer (6.25 Hz, 56.25 bps, codebook 512): encodes chromagram features, which are octave-free (bridging the F0 distribution gap between speech and singing) and notation-free (extractable without MIDI annotation).' confidence: high relevance: high - claim_id: autoregressive_models_initialized_from_llm_checkpoints_can_achieve role: supports claim: Autoregressive models initialized from LLM checkpoints can achieve competitive zero-shot TTS quality at low codec frame rates (12.5 Hz) when paired with strong post-training, partially compensating for the sequence granularity disadvantage relative to higher frame-rate systems. source: §V-A, Table VII evidence: On the SeedTTS benchmark (regular speech), post-trained Vevo2 achieves WER 3.64 (en) / 2.94 (zh) and SIM-o 0.693 / 0.754, competitive with CosyVoice 2 and MaskGCT despite using a lower frame-rate (12.5 Hz vs 25–50 Hz) content-style tokenizer. confidence: high relevance: low - claim_id: single_objective_preference_alignment_in_multi_capability_speech role: supports claim: Single-objective preference alignment in multi-capability speech models degrades complementary objectives; joint multi-objective optimization is necessary to avoid quality regressions on non-targeted capabilities. source: §V-F, Figure 5 evidence: Single-objective optimization degrades the complementary objective (intelligibility-only training drops melody accuracy from 65% to 50%); joint optimization achieves both gains. confidence: high relevance: low - claim_id: inference_time_pitch_shift_applied_to_prosody_tokens role: complicates claim: Inference-time pitch shift applied to prosody tokens improves speaker similarity in voice and singing conversion tasks, at the cost of a minor intelligibility degradation due to train-inference distribution mismatch. source: §V-E, Table VI evidence: 'Inference-time controllability: The system flexibly recombines text, prosody source, style reference, and timbre reference to perform TTS, VC, SVS, SVC, speech/singing editing, duration control (97%+ accuracy via chromagram length scaling), and pitch region control (via F0 shift before prosody token extraction).' confidence: high relevance: high limitations: - '- Training-inference mismatch when applying pitch shift: content-style tokens from pitch-shifted audio are out-of-distribution, causing slight WER degradation. The paper flags future work to include pitch shift augmentation during training. - Singing voice data used (7K hours source-separated from in-the-wild songs) may have variable quality. - The 12.5 Hz frame rate of the content-style tokenizer slightly limits VC intelligibility compared to Vevo-FM (50 Hz), as shown in Table VIII. - Melody-MOS for Vevo2 in SVC is slightly lower than the FM-only Vevo2-FM, suggesting a quality-style tradeoff when adding text input. - Generalization to tonal languages in singing is not specifically evaluated.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.17031' published_date: "2025-08-23" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec - GAN relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - multimodal_visual_prosody_conditioning claims: - claim_id: cross_modal_attention_between_phoneme_and_audio_representations role: supports claim: Cross-modal attention between phoneme and audio representations is more effective than global speaker embeddings for preserving localised prosodic and acoustic context in speech insertion tasks. source: §4.3, Table 3 evidence: Removing cross-modal attention and replacing it with a global speaker encoder worsens MCD on dev-clean, confirming that fine-grained contextual style transfer outperforms speaker-level global conditioning. confidence: high relevance: high - claim_id: non_autoregressive_tts_architectures_with_duration_prediction_can role: supports claim: Non-autoregressive TTS architectures with duration prediction can infer variable-length insertion segments without explicit phoneme-level alignment at inference time. source: §3.1, §3.2 evidence: MFA-derived phoneme alignments provide duration supervision during training; at inference, durations are predicted by the variance adaptor, enabling variable-length output. confidence: high relevance: high - claim_id: combining_local_and_global_adversarial_losses_with_a role: supports claim: Combining local and global adversarial losses with a triplet-based style matching loss reduces perceptual artefacts in synthesised speech segments that L1 reconstruction loss alone cannot eliminate. source: §3.3, §4.3 evidence: Phase two adds local and global LSGAN adversarial losses with feature matching, plus a triplet-based style matching loss. confidence: high relevance: medium - claim_id: speech_insertion_quality_degrades_more_gracefully_with_insertion role: supports claim: Speech insertion quality degrades more gracefully with insertion length when the model conditions directly on local audio context rather than full-utterance speaker embeddings. source: §4.2, Table 1 evidence: Quality degrades gracefully with insertion length — long insertions score 3.97 MOS, versus 2.80 for MetaStyleSpeech and 4.47 for ground truth. confidence: high relevance: low limitations: - Code and demos are not publicly released with the preprint, and model size is not reported, making independent evaluation and comparison difficult. - 'Evaluation is limited to LibriTTS dev-clean and dev-other; both are clean read-speech corpora, so generalisation to noisy or spontaneous speech is untested. The user study comprises only 15 utterances rated by 6 annotators — a small sample that limits statistical power. MCD is used as the primary objective metric, but it is an indirect measure of naturalness quality and is known to correlate imperfectly with perceptual preference. The comparison set is narrow: SpeechPainter is referenced in the related work but not included in the quantitative evaluation because it uses a fixed-duration formulation, leaving the question of how well RephraseTTS would perform relative to more recent neural audio inpainting approaches unanswered. Duration quality of the inserted segment relative to ground truth is not measured directly.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2508.17494' published_date: "2025-08-24" entry_date: '2026-07-25' year: 2025 venue: workshop task: - TTS architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - structured_language_prosody_control claims: - claim_id: cascaded_task_decomposition_separating_structural_tag_prediction_from role: supports claim: Cascaded task decomposition — separating structural tag prediction from numerical parameter regression — substantially outperforms joint LLM generation for SSML-based prosody control. source: §4.4, §5.4, Table 4, Table 5 evidence: Few-shot prompting improves numerical accuracy but can cause unexpected structural collapses (Llama 3's prosody tagging nearly disappears in few-shot mode). confidence: high relevance: high - claim_id: prompt_only_llms_zero_shot_and_few_shot role: supports claim: Prompt-only LLMs (zero-shot and few-shot) systematically under-generate prosodic markup tags relative to gold annotations, and this failure persists across architectures and scales. source: §5.3, Figure 3 evidence: The LLM benchmarking (§5.3, Table 3) finds that all zero-shot and few-shot prompted models consistently under-generate break and prosody tags relative to the gold standard. confidence: high relevance: high - claim_id: prosody_enhancement_via_ssml_yields_substantial_perceptual_gains role: supports claim: Prosody enhancement via SSML yields substantial perceptual gains over neutral commercial TTS voices, even when the underlying synthesiser is not retrained. source: §5.1 evidence: Perceptual evaluation (18 listeners, 30 one-minute pairs each) shows MOS rising from 3.20 (Azure Henri baseline) to 3.87 with SSML enhancement (p < 0.005), a 20% improvement. 15 of 18 listeners preferred the enhanced version in over half of comparisons; 7 preferred it in more than 75% of comparisons (§5.1). confidence: high relevance: high - claim_id: french_tts_prosody_normalised_relative_to_a_synthetic role: supports claim: French TTS prosody normalised relative to a synthetic baseline captures linguistically meaningful patterns — phrase-final pitch rises, deliberate pacing — without requiring manual annotation. source: §3, Appendix A evidence: Each syntagm is annotated with four prosodic features — median pitch (expressed as a semitone offset converted to percentage), volume (LUFS-derived gain), speaking rate (words per second delta), and inter-syntagm break duration — normalised relative to a Microsoft Azure Henri baseline voice to produce relative delta values suitable for SSML encoding. confidence: high relevance: high limitations: - The entire pipeline is calibrated and evaluated against a single commercial TTS voice (Azure Henri, fr-FR). SSML tag semantics — the acoustic realisation of percentage pitch and rate adjustments — are implementation-dependent and voice-dependent. Transfer to any other engine or voice requires voice-specific recalibration, limiting the method's out-of-the-box generalisability. - The dataset is 14 hours of proprietary French podcasts; generalisation to other French domains, other speech styles (spontaneous, informal, unpunctuated text), or other languages is unvalidated. The paper's pipeline assumes that punctuation and syntactic cues reliably predict prosodic boundaries — an assumption that breaks down for social media text or transcribed spontaneous speech. Fine-tuning each Qwen 2.5-7B stage requires approximately 15 GB of GPU memory at 4-bit quantisation, which constrains deployment in low-resource settings. The perceptual test involved 18 listeners, a sample size sufficient for statistical significance but small for robust effect-size estimation across listener backgrounds. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2509.00675' published_date: "2025-08-31" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - linguistic_context_prosody_prediction - implicit_scale_and_data_prosody_modeling claims: - claim_id: speaker_specific_phrasing_behaviour_is_a_substantive_source role: complicates claim: Speaker-specific phrasing behaviour is a substantive source of variance in RP insertion that generic multi-speaker models fail to capture, and modeling it explicitly improves both objective and subjective phrasing quality. source: §5.1.2, Table 3, Table 5 evidence: 'For unseen speakers at inference, the paper proposes a lightweight embedding adapter: a two-linear-layer network trained to map PSVM embeddings from the seen-speaker space to the adapted embedding space used by phrasing models with trainable layers, enabling few-shot adaptation with as few as 5–10 reference utterances per unseen speaker without any fine-tuning of the phrasing model itself.' confidence: high relevance: high - claim_id: phoneme_level_language_models_outperform_subword_level_models role: supports claim: Phoneme-level language models outperform subword-level models on phrase break prediction, even at smaller model sizes, because phoneme representations carry acoustic information more directly relevant to pause insertion than subword tokens. source: §5.1.3, Table 4 evidence: 'On the PLM side, the paper experiments with both subword-level models (BERT, XLNet, RoBERTa, ALBERT, DeBERTaV3 in BASE and LARGE variants) and phoneme-level models: Mixed-Phoneme BERT (MP BERT), which introduces sup-phoneme auxiliary tokens aligned with phoneme tokens, and Phoneme-Level BERT (PL BERT), which uses a phoneme-to-grapheme pre-training objective.' confidence: high relevance: high - claim_id: scaling_subword_plms_from_base_to_large_yields role: supports claim: Scaling subword PLMs from BASE to LARGE yields diminishing returns for phrasing tasks, suggesting a representational ceiling specific to this task modality. source: §5.1.3, Table 4 evidence: Scaling subword PLMs from BASE to LARGE yields only modest F0.5 gains (~0.01–0.009) despite tripling parameter count, suggesting that subword representations approach a ceiling for this task (Table 4, §5.1.3). confidence: high relevance: high - claim_id: pre_trained_speaker_verification_embeddings_capture_prosodic_and role: supports claim: Pre-trained speaker verification embeddings capture prosodic and fluency-related characteristics that transfer to phrasing models via few-shot adaptation without fine-tuning, enabling reasonable generalization to unseen speakers. source: §5.2.2, Table 7 evidence: 'For unseen speakers at inference, the paper proposes a lightweight embedding adapter: a two-linear-layer network trained to map PSVM embeddings from the seen-speaker space to the adapted embedding space used by phrasing models with trainable layers, enabling few-shot adaptation with as few as 5–10 reference utterances per unseen speaker without any fine-tuning of the phrasing model itself.' confidence: high relevance: high - claim_id: f0_5_score_and_naturalness_mos_can_diverge role: supports claim: F0.5 score and naturalness MOS can diverge for phrasing models using different PLMs, indicating that objective phrasing accuracy does not fully predict perceived speech naturalness. source: §5.2.3, Table 8 evidence: Notably, MP BERT-based proposed models achieve competitive or higher MOS than BERT-BASE counterparts for unseen speakers despite lower F0.5 scores, suggesting MP BERT's pause placement better aligns with human naturalness preferences even when it underfits the aggregate distribution (Table 8). confidence: high relevance: high limitations: - Training data is exclusively from LibriTTS-R audiobook readings. The resulting phrasing models are likely miscalibrated for spontaneous speech, conversational TTS, or out-of-domain styles; generalization is explicitly flagged by the authors as an open problem. - 'Additional limitations: the paper cannot disentangle the contributions of phoneme vs. subword information within phoneme-level PLMs (since their pre-training also includes grapheme-level objectives), leaving the mechanism of improvement partially unclear. The embedding adapter for unseen speakers assumes that the mapping from PSVM embeddings to trained embeddings is approximately injective and learnable with a small network — this assumption may not hold for speakers whose acoustic characteristics fall outside the training distribution.' caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: 2025.clicit-1.27 published_date: "2025-09-01" entry_date: '2026-07-25' year: 2025 venue: workshop task: - TTS - evaluation architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - structured_language_prosody_control - language_and_domain_specific_prosody_modeling claims: - claim_id: orthographic_pre_processing_of_input_text_accent_marking role: supports claim: Orthographic pre-processing of input text (accent marking, grapheme substitution, elision resolution) can reduce segmental substitution errors in LLM-based TTS applied to low-resource or historical languages without any model retraining. source: §3.2, §5.2 evidence: Replacing rare digraphs and adapting graphemes (〈c〉 to 〈k〉, 〈ae〉 to 〈ai〉, 〈qu〉 to 〈kw〉) reduced segmental substitutions by approximately one third; no retraining of GPT-4o-mini-tts was performed. confidence: high relevance: low - claim_id: natural_language_prompt_instructions_can_steer_general_purpose role: supports claim: Natural language prompt instructions can steer general-purpose LLM-based TTS toward prosodically-correct speech in low-resource languages without fine-tuning. source: §3.3, §3.4 evidence: A concise system prompt specifying slow pace, explicit stress, and syllable articulation, combined with orthographically pre-processed input, produced at least one expert-validated recording for all 216 Latin verse lines. confidence: high relevance: medium - claim_id: llm_based_tts_systems_trained_on_modern_multilingual role: complicates claim: LLM-based TTS systems trained on modern multilingual data produce systematic cross-lingual phonological interference when applied to historically or orthographically distinct languages. source: §5.1 evidence: GPT-4o-mini-tts imposed Italian, English, or Spanish prosodic templates on Latin text depending on vocabulary similarity; the phenomenon confirms that Latin-specific data is essentially absent from the model's training corpus. confidence: high relevance: low - claim_id: prompt_based_prosody_control_is_more_effective_for role: complicates claim: Prompt-based prosody control is more effective for stress placement than for enforcing absolute phonemic length distinctions. source: §5.6, §6 evidence: The workflow approximated Latin vowel quantity through slower pacing on ictic syllables but could not enforce a fixed heavy-to-light duration ratio, the standard for quantitative metre in phonetic work. confidence: high relevance: high limitations: - The corpus depends on GPT-4o-mini-tts, a proprietary model with inaccessible weights. Should the API access policy change, exact reproduction of the dataset becomes impossible. - 'The corpus is small (216 lines, approximately 24 minutes), falling well short of the material typically required for full end-to-end TTS fine-tuning. Stylistic range is narrow: both texts belong to the Augustan literary period and formal register; colloquial Latin, post-Classical spelling, and late-antique forms are untested. A single synthetic voice is used throughout, precluding multi-speaker or gender-diverse analysis. Only the classical reconstructed pronunciation is represented; ecclesiastical pronunciation is excluded. Expert validation, while rigorous, introduces selection bias from a single institution''s phonological tradition. Quantity is approximated through pacing rather than explicit duration control; no fixed heavy-to-light ratio is enforced, leaving a gap relative to the phonetic gold standard.' caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.01391' published_date: "2025-09-01" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - transformer-enc-dec relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: ssl_derived_discrete_tokens_can_substitute_phoneme_sequences role: supports claim: SSL-derived discrete tokens can substitute phoneme sequences as input representations in a TTS front-end without a substantial naturalness penalty in automatic evaluation. source: §V.B, Table II evidence: Third, the predicted token sequences are fed into FastSpeech 2 as input representations in place of phonemes, and FastSpeech 2 predicts mel-spectrograms which are converted to waveforms by a vocoder. confidence: high relevance: low - claim_id: g2p_free_tts_pipelines_that_learn_text_to role: supports claim: G2P-free TTS pipelines that learn text-to-token mappings from paired speech data avoid the language-specific resource burden of phoneme dictionaries and morphological analysers. source: §I, §III evidence: The unit error rate (UER) between predicted and oracle pseudo-language labels is 7.47%, while CER for synthesized speech is 21.28% (proposed), 20.63% (oracle), and 18.24% (baseline). confidence: high relevance: low - claim_id: in_an_ssl_token_based_tts_pipeline_the role: supports claim: In an SSL-token-based TTS pipeline, the spectral predictor contributes more to naturalness differences than the text-to-token mapping stage. source: §V.B, Table II evidence: 'Two control conditions bracket the proposed method: a conventional G2P baseline using OpenJTalk, Mecab, and Marine with full phoneme, duration, and accent labels; and an oracle that feeds SSL-derived labels from the ground-truth audio directly to FastSpeech 2, bypassing the text-based predictor entirely.' confidence: high relevance: low - claim_id: ssl_based_speech_representations_preserve_sufficient_acoustic_quality role: supports claim: SSL-based speech representations preserve sufficient acoustic quality through a discretise-then-synthesise pipeline to remain competitive with G2P-derived representations on codec-style quality metrics. source: §V.C, Table II evidence: WARP-Q acoustic quality for the proposed method (2.63) is comparable to the oracle (2.64) and above the baseline (2.47), suggesting the SSL-based pipeline does not introduce significant acoustic degradation relative to the G2P approach. confidence: high relevance: medium limitations: - All evaluations use automatic metrics only (UTMOS, CER, WARP-Q, SDR) on 100 utterances from a single speaker subset of JVS. No subjective MOS or preference tests are reported; the conclusions about naturalness parity are therefore tentative. - The system is demonstrated exclusively on Japanese and does not yet extend to multilingual settings. The T5 tokenizer is language-specific (tohoku-BERTv3), which the authors acknowledge as a barrier to multilingual scalability. Future directions involve BPE tokenizers (mT5, ByT5) to reduce this dependency. The oracle's counter-intuitive lower UTMOS than the proposed system is left unexplained and may indicate an interaction between SSL token sequence statistics and FastSpeech 2's duration predictor. Individual contribution analysis of duration, pitch, and accent inputs to FastSpeech 2 is identified as missing. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '2506.21619' published_date: "2025-09-03" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - GAN - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: influential method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - disentangled_prosody_content_timbre - emotion_expressive_prosody_control - structured_language_prosody_control claims: - claim_id: tying_the_duration_target_embedding_to_the_semantic role: supports claim: Tying the duration-target embedding to the semantic positional embedding table enables near-perfect token-count precision in autoregressive TTS without sequence-level supervision. source: § Proposed Method — Duration Control, Table 4 evidence: For duration control (Table 4), token number error rate is below 0.02% at 1× scaling and below 0.07% even at 0.75× compression on SeedTTS test-zh — near-perfect precision for an autoregressive model. confidence: high relevance: high - claim_id: gradient_reversal_layer_training_effectively_disentangles_emotional_prosody role: supports claim: Gradient Reversal Layer training effectively disentangles emotional prosody from speaker timbre in zero-shot TTS, enabling independent control of expression and identity from separate reference signals. source: § Proposed Method — Emotional Control, Table 2 evidence: A Gradient Reversal Layer (GRL) during training forces `e` to be invariant to speaker timbre, while a frozen speaker perceiver conditioner extracts `c` (timbre). confidence: high relevance: high - claim_id: gpt_latent_fusion_between_the_autoregressive_semantic_module role: supports claim: GPT-latent fusion between the autoregressive semantic module and the flow-matching acoustic module improves phonetic clarity under high emotional expressiveness, as measured by WER on the emotional test set. source: § Semantic-to-Mel Module, Table 2 evidence: 'IndexTTS2 is a three-module cascade: a Text-to-Semantic (T2S) autoregressive transformer, a Semantic-to-Mel (S2M) flow-matching model, and a BigVGANv2 vocoder.' confidence: high relevance: low - claim_id: a_three_stage_curriculum_that_separates_base_capability role: supports claim: A three-stage curriculum that separates base capability training from emotion-conditioner fine-tuning is essential for emotional expressiveness; removing it causes EMOS to collapse from 4.22 to 2.82. source: § Experiment Results — Emotional Performance, Table 2 evidence: 'Training uses a three-stage curriculum: (1) full dataset without emotion conditioning to establish base capability; (2) 135-hour emotional subset with emotion conditioner trainable and GRL active; (3) full dataset fine-tune with all conditioners frozen for robustness.' confidence: high relevance: medium - claim_id: knowledge_distillation_from_a_large_reasoning_llm_into role: supports claim: Knowledge distillation from a large reasoning LLM into a small fine-tuned model provides a usable soft emotion routing interface for TTS that outperforms instruction-tuned baselines on naturalness and prosody. source: § Text-to-Emotion, Table 3/5 evidence: Natural language emotion control (T2E, Table 3/5) outperforms CosyVoice2 on all four MOS dimensions (SMOS 3.875 vs. 2.973). confidence: high relevance: high limitations: - The emotional training data (135 hours across 361 speakers) is modest and partly commercial, raising questions about reproducibility and demographic coverage. The GRL-based disentanglement works for the 7 basic emotions defined, but the emotion space is discrete and limited — continuous or compositional emotion representations are not addressed. The paper notes WER degrades slightly on AISHELL-1 relative to IndexTTS, suggesting the emotional training may slightly hurt clean-speech intelligibility. Duration control is evaluated only at up to 1.25× scaling; extreme compression or expansion is untested. The code release is stated as planned but not confirmed at submission. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2506.23367' published_date: "2025-09-03" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - flow-matching relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control claims: - claim_id: targeted_duration_manipulation_of_specific_phonemic_contrasts_is role: supports claim: Targeted duration manipulation of specific phonemic contrasts is more effective for L2 speech comprehension than undifferentiated global speech rate reduction. source: §3.3.1, §3.3.2, Tables 2, 4 evidence: Clarity mode (1.6x stretch on tense vowels only, 0.75x elsewhere) achieved 15.15% total WER vs. 19.82% for global 1.2x stretch and 24.30% for base in the double-word experiment; tense-vowel WER improved from 60.23% (base) to 29.48% with targeted stretch vs. 37.57% with full stretch in the single-word experiment. confidence: high relevance: high - claim_id: asr_based_word_error_rate_can_serve_as role: contradicts claim: ASR-based word error rate can serve as a reliable proxy for L2 listener intelligibility in TTS evaluation. source: §3.3.3, Table 7 evidence: Whisper ASR showed no benefit from clarity mode (17.68% WER vs. 17.10% for base) while L2 listeners achieved their lowest WER with clarity mode (15.15%); ASR's tense/lax substitution pattern (71.42% minimal-pair substitutions on base TTS) differed fundamentally from L2 human error distributions; overall ASR WER varied by less than 3% across conditions that produced large differences in human WER. confidence: high relevance: low - claim_id: global_speech_rate_reduction_is_an_effective_clarity role: complicates claim: Global speech rate reduction is an effective clarity strategy for L2 listener experience. source: §3.3.1, §3.3.2, Tables 2, 3, 5, 6 evidence: Full-stretch TTS scored significantly lower on naturalness (nMOS ~4.2 vs. ~7.0-7.9 for other conditions), prosody, encouragement, and respect in both single- and double-word experiments; L2 participants rated both "too fast" (base) and "too slow" (stretch) as less respectful and encouraging. confidence: high relevance: high - claim_id: l2_listeners_subjective_perceived_intelligibility_judgments_reflect_their role: complicates claim: L2 listeners' subjective perceived intelligibility judgments reflect their actual comprehension accuracy for TTS systems. source: §3.3.1, §3.3.2, §5 evidence: Despite clarity mode producing lower objective WER than emphasis in both experiments, L2 participants consistently rated emphasis (stretching all target words) as most intelligible subjectively; participants believed that overall phrase slowing was most intelligible even when it was not, suggesting they were unaware of the duration mechanism improving their comprehension. confidence: high relevance: low limitations: - 'The study is restricted to French-L1 English-L2 speakers at relatively high proficiency (mode: level 5/5), covering only three English tense/lax vowel pairs. Generalizability to other L1 backgrounds, lower proficiency levels, and vowel distinctions beyond duration contrast is untested.' - The 1.6x duration stretch factor was calibrated empirically and may not be optimal across different base speech rates, speaker styles, or emotional expressivity contexts. Inter-individual differences in duration cue weighting were noted in the authors' prior work, suggesting the mechanism may be ineffective for a subset of L2 listeners. The clarity mode addresses duration cues only; formant-based spectral cues and consonantal clarity mechanisms remain unexplored. The word-marking interface (exclamation points surrounding target words) requires either manual annotation or a large language model to identify difficult words, introducing a practical dependency for real-world deployment. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2509.03940' published_date: "2025-09-04" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - SCA - evaluation architecture: [] relevance: low evidence_role: - historical_context current_role: minor method_family: - prosody_analysis_and_evaluation_method claims: - claim_id: acoustic_quality_paralinguistic_appropriateness_is_the_most_consistently role: supports claim: Acoustic quality (paralinguistic appropriateness) is the most consistently underperformed capability across current spoken dialogue models, even for proprietary systems, lagging well behind text generation and reasoning dimensions. source: '§Experiments: LLM-based Result, Table 3' evidence: 'Across the LLM-based dimensions, Contextual Coherence is the strongest capability for all models (GPT-4o: 4.48), while Acoustic (Paralinguistic) Appropriateness is universally the weakest (GPT-4o: 3.82, best open-source Step-Audio: 3.43).' confidence: high relevance: medium - claim_id: model_parameter_count_does_not_reliably_predict_speech role: supports claim: 'Model parameter count does not reliably predict speech synthesis quality in spoken dialogue systems: smaller, better-trained models can match or exceed much larger counterparts on naturalness metrics.' source: '§Experiments: Metric-based Results, Table 2' evidence: Step-Audio at 132B achieves the highest BertScore F1 (84.16) but scores poorly on speech naturalness (UTMOS 2.42), a clear trade-off between semantic accuracy and synthesis quality. confidence: high relevance: low - claim_id: llm_based_evaluation_augmented_with_discrete_acoustic_features role: supports claim: LLM-based evaluation augmented with discrete acoustic features achieves strong alignment with human judgment on multi-dimensional spoken role-playing tasks. source: '§Evaluation Framework, §Experiments: Subjective experimental results' evidence: LLM-based evaluation adds an acoustically-aware judge that transcribes the model's spoken response with Whisper, extracts per-sentence emotion labels via Emotion2Vec, and augments the text input with pitch, energy, and speaking rate bins before judging. confidence: high relevance: low - claim_id: there_is_an_optimal_context_window_length_for role: supports claim: There is an optimal context window length for spoken role-playing models beyond which additional conversational history degrades rather than improves performance. source: '§Experiments: Ablation study on context length, Table 4' evidence: 'An ablation over context window length (4–10 turns) shows a non-monotonic trend: a context of 6 turns achieves the best overall score, suggesting diminishing returns from additional history.' confidence: high relevance: low limitations: - The benchmark is sourced exclusively from movie dialogue, which is scripted and stylistically distinct from spontaneous conversational speech. Generalisation of the evaluation findings to natural speech interaction scenarios is not established. - Character quality validation covers only 20 characters rated by 5 annotators — the 93% satisfactory/acceptable rate is encouraging but the sample is small. The human evaluation for judge validation similarly covers only 20 dialogue instances. The benchmark is currently restricted to two-speaker conversations, limiting applicability to multi-party or group dialogue settings. Future work noted by the authors includes expanding the dataset and fine-tuning models on the role-playing task, but neither training data nor fine-tuning baselines are provided in this paper. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '2509.04072' published_date: "2025-09-04" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control claims: - claim_id: narrative_aware_segmentation_of_audiobook_data_into_character role: supports claim: Narrative-aware segmentation of audiobook data into character quotation and narration subsets yields training material with measurably higher emotional diversity than standard sentence-level audiobook splits. source: §3.2, Table 5 evidence: Training experiments fine-tune SparkTTS (autoregressive, Qwen2.5-based) and F5-TTS (flow-matching, DiT-based) on LibriQuote subsets, and also train SparkTTS from scratch on full narration+quotation data. confidence: high relevance: low - claim_id: flow_matching_tts_models_show_larger_expressivity_gains role: supports claim: Flow-matching TTS models show larger expressivity gains from fine-tuning on targeted expressive speech data than autoregressive models with equivalent training setups. source: §4.3, Table 2 evidence: Fine-tuning SparkTTS improves WER substantially (2.0–2.1 on LibriSpeech-PC vs. 3.06 baseline) but does not meaningfully improve ContextMOS (2.89–2.97 vs. 2.94 baseline), suggesting the autoregressive training objective is less responsive to expressive data than the flow-matching objective. confidence: high relevance: medium - claim_id: conditioning_tts_synthesis_on_surrounding_narrative_context_rather role: complicates claim: Conditioning TTS synthesis on surrounding narrative context rather than only the target utterance text improves contextual appropriateness of synthesized speech at the cost of modest intelligibility degradation. source: §4.3, Table 2 evidence: A contextual conditioning variant replaces SparkTTS's text input with the surrounding narrative paragraph as additional context. confidence: high relevance: low - claim_id: current_open_source_tts_systems_are_substantially_less role: supports claim: Current open-source TTS systems are substantially less expressive than human audiobook narrators on contextual benchmarks, even when naturalness scores (MOS) are comparable. source: §5.2, Table 4 evidence: Fine-tuning SparkTTS improves WER substantially (2.0–2.1 on LibriSpeech-PC vs. 3.06 baseline) but does not meaningfully improve ContextMOS (2.89–2.97 vs. 2.94 baseline), suggesting the autoregressive training objective is less responsive to expressive data than the flow-matching objective. confidence: high relevance: medium - claim_id: llm_extracted_speech_delivery_pseudo_labels_verbs_and role: supports claim: LLM-extracted speech-delivery pseudo-labels (verbs and adverbs) from narrative prose are reliable enough to serve as training signals for expressive TTS, achieving high precision when confidence-filtered. source: §3.3, Figure 2 evidence: A filtered high-expressivity subset (Q_f, 379 hours) is built by retaining quotations where Phi-4 (with self-reported confidence scoring) extracts a speech adverb or an explicitly expressive speech verb from the narrative context — producing pseudo-labels such as "he whispered softly" that characterise intended delivery. confidence: high relevance: medium limitations: - The LibriQuotetest ground-truth is recorded by amateur LibriVox volunteers, who themselves show insufficient expressivity for many quotations. ContextMOS scores for ground-truth (3.55 average) are only marginally above the best TTS systems, raising questions about whether the benchmark captures professional audiobook narration standards or amateur reading behaviour. - The training set is not WER-filtered, meaning a small proportion of transcription errors may remain. The gender distribution of LibriVox speakers is unknown, introducing potential bias. Experiments use only English fiction; cross-lingual and non-fiction applicability is untested. The contextual conditioning results are encouraging but use only one-paragraph context windows — longer narrative context might yield larger gains. LibriQuote's utility for neural audio codec training (noted as future work) remains to be demonstrated. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.06074' published_date: "2025-09-07" entry_date: '2026-07-25' year: 2025 venue: EMNLP task: - TTS architecture: - transformer-enc-dec relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - linguistic_context_prosody_prediction - multimodal_visual_prosody_conditioning claims: - claim_id: word_level_semantic_and_prosodic_context_modeling_in role: supports claim: Word-level semantic and prosodic context modeling in dialogue history improves prosody expressiveness in conversational TTS over utterance-level context encoders. source: §3.5, Table 1; §3.6, Table 2 evidence: MFCIG-CSS achieves N-DMOS 3.980 and P-DMOS 3.899 on DailyTalk, outperforming seven baselines that operate at the utterance level by 0.122 and 0.104 respectively; ablation removing both graph modules causes the largest performance collapse across all metrics. confidence: high relevance: high - claim_id: graph_neural_networks_with_sequential_cross_turn_aggregation role: supports claim: Graph neural networks with sequential cross-turn aggregation can encode complementary semantic and prosodic interaction patterns from multimodal dialogue history for prosody-aware TTS. source: §3.6, Table 2 evidence: 'SIG and PIG are independently ablated: removing SIG reduces N-DMOS by 0.147 and P-DMOS by 0.106; removing PIG produces comparable drops. Both modules contribute distinctly and their combined use provides the strongest performance.' confidence: high relevance: high - claim_id: objective_energy_error_and_subjective_prosody_quality_metrics role: complicates claim: Objective energy error and subjective prosody quality metrics can rank conversational TTS systems differently. source: §3.5, Table 1 evidence: MFCIG-CSS achieves best N-DMOS (3.980) and P-DMOS (3.899) on DailyTalk but ranks second on MAE-E (0.314 vs. 0.310 for MSRGCN-CSS), indicating that signal-level energy accuracy does not fully predict human prosody quality judgments. confidence: high relevance: high - claim_id: prosody_modeling_gains_demonstrated_with_acoustic_feature_based role: complicates claim: Prosody modeling gains demonstrated with acoustic-feature-based TTS backbones may not transfer to codec-token-based or flow-based architectures. source: §5 Limitations evidence: MFCIG-CSS is validated only on a FastSpeech 2 backbone; the authors explicitly identify extension to VITS-based architectures and discrete token-based speech encoders as future work, acknowledging that the current validation scope limits generalizability claims. confidence: high relevance: high limitations: - MFCIG-CSS is evaluated on a single English dialogue dataset (DailyTalk, approximately 20 hours) with a FastSpeech 2 backbone, leaving generalization to other languages, longer conversations, noisy conditions, and modern autoregressive or codec-based TTS systems untested. The interaction graphs operate on frame-averaged acoustic features from Wav2Vec 2.0 and do not yet model fine-grained intra-word acoustic cues such as emotion, emphasis, or pauses. Extension to VITS-based architectures and discrete token-based speech encoders is the primary open direction identified by the authors. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2509.07038' published_date: "2025-09-08" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - singing architecture: - diffusion relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control - speech_singing_shared_prosody claims: - claim_id: explicit_conditioning_on_a_signal_derived_attribute_provides role: supports claim: Explicit conditioning on a signal-derived attribute provides substantially more controllability than implicit predictors trained on the same attribute as an auxiliary loss. source: §V.B, Table III evidence: Adding an energy predictor to the baseline reduces energy MAE from 0.33 to 0.30, while replacing the predictor with explicit phoneme-level energy input achieves 0.14 - a 57.6% reduction over baseline vs. 9% for the predictor. confidence: high relevance: low - claim_id: phoneme_level_aggregation_of_frame_level_features_enables role: supports claim: Phoneme-level aggregation of frame-level features enables user-friendly control in singing synthesis at acceptable precision cost. source: §III.B, §V.A, Table I, Table II evidence: Phoneme-level energy (L values, matching lyric and note sequence length) achieves energy MAE of 0.14 vs. 0.03 for frame-level (T values, roughly 1,000 per utterance), with higher MOS (3.78 vs. 3.57), demonstrating that coarser granularity trades minor precision loss for substantially reduced control burden. confidence: high relevance: medium - claim_id: ground_truth_signal_features_can_substitute_for_manually role: supports claim: Ground-truth signal features can substitute for manually annotated expressive attributes as conditioning signals in singing voice synthesis. source: §I, §III.B evidence: Frame-level energy is extracted directly from ground-truth mel-spectrograms without human annotation, yet conditioning on it enables effective dynamics control, reducing annotation costs relative to technique-labelled datasets used in prior controllable SVS work. confidence: high relevance: medium - claim_id: perceptual_quality_and_controllability_precision_do_not_align role: complicates claim: Perceptual quality and controllability precision do not align monotonically across conditioning granularities in SVS. source: §V.A, Table I, Table II evidence: The frame-level model achieves the best energy MAE (0.03) but lower MOS (3.57 ± 0.18) than the phoneme-level model (MOS 3.78 ± 0.19, energy MAE 0.14), suggesting that very tight energy matching may over-constrain generation in ways that reduce perceived naturalness. confidence: high relevance: low limitations: - MOS evaluation involved only 10 listeners, which the authors acknowledge constrains statistical reliability and generalizability. Results should be treated as directional rather than definitive. - The study uses only the Chinese subset of GTSinger and two speakers, leaving multilingual and speaker-diversity generalisability untested. The baseline DDPM architecture is explicitly noted to fall below state-of-the-art SVS performance, so the absolute quality figures do not reflect what the conditioning mechanism would achieve on a competitive backbone. Expressive attributes beyond dynamics (timbre, vibrato, advanced singing techniques) are not addressed. Phoneme-level energy may not capture fine-grained intra-phoneme temporal variation present in skilled vocal performance. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2505.17093' published_date: "2025-09-19" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: [] relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - structured_language_prosody_control claims: - claim_id: an_llm_based_intermediate_conversion_layer_can_reduce role: supports claim: An LLM-based intermediate conversion layer can reduce the expertise burden on users of instruction-conditioned TTS without requiring model retraining. source: §4.3, Table 2 evidence: P2VA-C converts persona descriptions to structured prompts via GPT-4o-mini zero-shot prompting, improving human MOS from 3.09 to 3.42 and reducing WER from 22% to 17% relative to an unguided baseline on 1,000 LJSpeech-transcript persona pairs. confidence: high relevance: medium - claim_id: natural_language_voice_attribute_prompts_generated_by_llms role: complicates claim: Natural language voice attribute prompts generated by LLMs should not be assumed to be demographically neutral. source: §5.1, Table 3 evidence: When personas lack explicit gender cues, LLM-generated voice attributes assign male gender 64% of the time (vs. 10% in source descriptions), and 90% of accent assignments default to North American or British variants. confidence: high relevance: low - claim_id: the_trade_off_between_structured_and_free_form role: refines claim: The trade-off between structured and free-form style conditioning in instruction-conditioned TTS extends to intermediate conversion layers, not only the TTS interface itself. source: §4.3, Table 2 evidence: P2VA-C (structured, predefined attributes) achieves lower WER and higher human MOS than P2VA-O (free-form), while P2VA-O preserves expressivity for attributes outside the predefined vocabulary. confidence: high relevance: medium - claim_id: fairness_in_voice_generation_systems_requires_addressing_bias role: complicates claim: Fairness in voice generation systems requires addressing bias in LLM-mediated style inference, not only in training data or TTS model design. source: §5.1, §5.2, Tables 4-6 evidence: Stereotyped gender-tone and gender-pitch co-occurrences (e.g., male assigned "Cognitive and Analytical" tone 41%, female assigned "Warm and Supportive" 44% and high pitch 62%) emerge from the LLM conversion layer itself, independent of the downstream TTS model's behavior. confidence: high relevance: medium limitations: - Evaluation is confined to English speech using a single-speaker dataset (LJSpeech), and the bias analysis covers only the five predefined attributes in P2VA-C. The scope of demographic bias in open-ended P2VA-O outputs is not characterized. - The user study does not report participant demographics, task instructions, or statistical significance, limiting the generalizability of the 90% alignment claim. The framework's dependence on Parler-TTS means results may not transfer to TTS systems with different conditioning interfaces. Bias mitigation methods are identified as future work; this paper provides quantification but no remediation. The attribute preset (P2VA-C) was derived from Parler-TTS training data labels, coupling the framework's controllability vocabulary to one model's training choices. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.15626' published_date: "2025-09-19" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - VAE relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer - disentangled_prosody_content_timbre claims: - claim_id: impression_leakage_in_controllable_tts_arises_structurally_when role: supports claim: Impression leakage in controllable TTS arises structurally when a single reference utterance is used for both speaker identity and style conditioning during training. source: §2, §5.2, Table 2 evidence: VIC-base achieves ∆V = 0.22 (significantly different from zero), confirming that the reference audio biases synthesized VI toward the reference's inherent voice impression even when GRL and high-rate dropout are applied. confidence: high relevance: medium - claim_id: decoupled_training_using_distinct_utterances_for_speaker_and role: supports claim: Decoupled training using distinct utterances for speaker and style conditioning reduces style leakage without changing the underlying TTS architecture. source: §4.1, §5.2, Table 2 evidence: VIC-dis uses a separate utterance from the same speaker for reference conditioning during training, reducing ∆V from 0.22 to 0.14 with a statistically significant improvement in RVI-MSE (0.61 to 0.51). confidence: high relevance: medium - claim_id: eliminating_the_speaker_reference_and_conditioning_solely_on role: supports claim: Eliminating the speaker reference and conditioning solely on a style vector achieves the strongest leakage reduction, at a moderate cost in speaker fidelity. source: §4.2, §5.2, Table 2 evidence: VIC-srf reduces ∆V to 0.05 (not significantly different from zero), while SECS drops to 0.72, which remains above the cross-speaker bound of 0.63 but below the same-speaker bound of 0.81. confidence: high relevance: medium - claim_id: llm_based_tts_systems_conditioned_on_natural_language role: complicates claim: LLM-based TTS systems conditioned on natural language prompts are inadequate for fine-grained numerical voice impression control. source: §5.1, §5.2, Table 2, Table 3 evidence: Qwen3-TTS (zero-shot) achieves VI-MSE of 0.82 vs. 0.39 for the VITS-based baseline; fine-tuning worsens controllability further (VI-MSE 0.87), and text-VI entanglement is confirmed empirically by punctuation biasing predicted impressions. confidence: high relevance: low - claim_id: perceptual_voice_impression_annotation_can_achieve_inter_annotator role: supports claim: Perceptual voice impression annotation can achieve inter-annotator agreement comparable to other subjective speech tasks, supporting the validity of VI corpora. source: §3, Table 1 evidence: Krippendorff's alpha averages 0.470 across 10 VI dimensions, exceeding reported agreement for speech emotion recognition (0.442) and singing voice preference (0.153). confidence: high relevance: low limitations: - The LibriTTS-VI corpus contains only 130 manually annotated utterances from 130 speakers; the remaining LibriTTS-R annotations are estimated by the VIE trained with data augmentation. Several VI dimensions fall below the reliable inter-annotator agreement threshold (alpha below 0.667), particularly J) Cold-Warm (0.197) and D) Calm-Restless (0.295), limiting annotation reliability for these dimensions. - Evaluation is confined to the LibriTTS-R audiobook domain, which is relatively clean and controlled. Generalisation of both the VIE and the VIC systems to spontaneous or noisy speech is untested. The subjective MOS evaluation uses only two speakers and the authors note potential interface bias due to rating-scale anchoring. The comparison with Qwen3-TTS relies on a single external system evaluated in a zero-shot configuration for which the model was not explicitly designed. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.17143' published_date: "2025-09-21" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - VC architecture: - autoregressive-LM relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - stochastic_generative_prosody_model - disentangled_prosody_content_timbre claims: - claim_id: in_zero_shot_voice_conversion_temporally_coarser_syllabic role: supports claim: In zero-shot voice conversion, temporally coarser syllabic representations reduce pitch leakage from linguistic features more effectively than standard frame-aligned SSL features, enabling cleaner prosody control at the cost of intelligibility. source: §2.1, §4, Table 2 evidence: MaskVCT-Spk using SylBoost syllabic tokens achieves the lowest FPC (0.167) among all tested systems, indicating near-complete pitch independence from the source, while MaskVCT-All with continuous features retains more pitch correlation (FPC 0.417). confidence: high relevance: high - claim_id: multiple_classifier_free_guidance_weights_applied_to_distinct role: supports claim: Multiple classifier-free guidance weights applied to distinct conditioning factors in a single masked generative model enable user-configurable inference-time trade-offs between intelligibility, pitch fidelity, and speaker similarity. source: §2.5, §3.4, Table 2 evidence: MaskVCT defines triple CFG weights (w_all, w_spk, w_ling) over speaker, pitch, and linguistic conditions within one trained model; sweeping these weights continuously interpolates between MaskVCT-All (WER 4.68%, S-SIM 0.865) and MaskVCT-Spk (WER 6.47%, S-SIM 0.895). confidence: high relevance: high - claim_id: syllabic_speech_representations_that_suppress_pitch_leakage_in role: complicates claim: Syllabic speech representations that suppress pitch leakage in voice conversion also degrade content intelligibility through syllable misreadings caused by coarse temporal quantisation. source: §4, §5, Table 2 evidence: MaskVCT-Spk achieves the highest speaker similarity (S-SIM 0.895) but the highest WER (6.47%) among systems tested, substantially above FACodec (3.55%) and FreeVC (3.96%); the conclusion section attributes misreadings to K-means syllable mapping errors in SylBoost. confidence: high relevance: high - claim_id: masked_non_autoregressive_codec_models_can_match_or role: supports claim: Masked non-autoregressive codec models can match or exceed autoregressive and diffusion-based baselines on speaker similarity and quality in zero-shot VC while operating with fewer discrete tokens per utterance. source: §3.4, §4, Table 2 evidence: MaskVCT-Spk (2048 tokens) achieves higher S-SIM (0.895) and SS-MOS (3.69) than MaskGCT-S2A (8192 tokens, S-SIM 0.863, SS-MOS 3.02) and competitive UTMOS (3.17 vs. 3.24). confidence: high relevance: low limitations: - Syllabic tokens introduce misreadings that WER alone cannot fully diagnose; the authors acknowledge the K-means quantiser cannot recover from incorrect syllable boundary assignments, and propose future work to address this with a trainable VQ module. - The model is English-only. Accent conversion experiments are limited to L2-ARCTIC and test only two conversion directions; how well the syllabic pitch-stripping generalises to tonal languages (where pitch is phonemic) is unexplored. The 511-pair test set is relatively small for statistical confidence, especially given the reported confidence intervals overlap for several key comparisons. No code or trained checkpoint is publicly released, limiting reproducibility. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2509.17516' published_date: "2025-09-22" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - latent_reference_prosody_transfer claims: - claim_id: decoupling_the_training_pairing_between_the_speech_prompt role: supports claim: Decoupling the training pairing between the speech prompt and the synthesis target (rather than using identical prompt/target audio) improves text-prosody alignment in style-controllable TTS, at the cost of an inflated speaker-similarity score under the coupled setup. source: §3.3, Table 2 evidence: Switching from non-decoupled to decoupled prompt/target training raises S-MOS from 3.45±0.09 to 3.82-3.93±0.06-0.07 while reducing speaker-similarity (SS) from 0.87 (interpreted as over-similar timbre/prosody) to 0.69-0.8. confidence: high relevance: high - claim_id: providing_an_autoregressive_speech_language_model_with_explicit role: supports claim: Providing an autoregressive speech language model with explicit surrounding-sentence text context improves narrative coherence in long-form speech generation beyond what sentence-isolated synthesis achieves. source: §2.1, §3.3, Table 1 evidence: Adding pre/post-context text sequences raises Chapter M-MOS from 3.88±0.07 (CosyVoice2 baseline, sentence-level synthesis) to 4.13±0.09 (Infer-ctx), and a qualitative example shows the model generating contextually appropriate laughter for a sentence with no explicit emotional markers when given the preceding sentence's context. confidence: high relevance: low - claim_id: the_strength_of_the_prompt_target_decoupling_threshold role: complicates claim: The strength of the prompt/target decoupling threshold used during training trades off speaker-identity stability against naturalness, rather than improving both simultaneously. source: §3.3, Table 2 evidence: Sweeping the voiceprint-similarity clustering threshold shows a lower threshold reduces speaker-similarity (with occasional timbre discontinuities) but yields slightly higher S-MOS, while a higher threshold approaches the non-decoupled model's inflated similarity at the cost of lower S-MOS. confidence: high relevance: medium - claim_id: self_distillation_from_a_pretrained_emotional_tts_model role: supports claim: Self-distillation from a pretrained emotional TTS model, filtered by objective quality metrics, can mitigate the scarcity of high-intensity emotional training data and improve controllable emotional expressiveness. source: §2.2, §3.3, Table 3, Table 4 evidence: On the CV3-Eval text-unrelated test set, the model trained with 500 hours of emotional data plus 5,300 hours of self-distilled augmentation achieves larger high- vs. low-intensity F1 discriminability than a CosyVoice2-instruct baseline (e.g. angry ΔF1 0.31 vs. 0.07) and higher mixed-emotion S-MOS (3.87±0.09 vs. 3.35±0.07). confidence: high relevance: low limitations: - All training data (1M-hour audiobook corpus, 100K-hour context-aware corpus, 500-hour emotional corpus, 5K-hour augmentation set) and all three test sets (Test-NAR, Test-DIA, Test-CHAP) are proprietary and internally constructed; no code, model weights, or evaluation data are released, which limits independent verification of the reported gains. - The evaluation is confined to Chinese-language audiobook content with a single subjective rater pool (50 native Chinese speakers); generalization to other languages or narration styles is untested. The decoupling-threshold ablation reports trends across tested values without identifying a clearly optimal setting, and the paper notes the risk of persona instability at high thresholds is hypothesized rather than directly measured across the full range. The authors flag future work on scaling chapter-level context data and applying reinforcement learning for further quality improvement. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.18531' published_date: "2025-09-23" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: [] claims: - claim_id: reinforcement_learning_with_automatically_verifiable_rewards_can_optimize role: complicates claim: Reinforcement learning with automatically verifiable rewards can optimize the measured objective while degrading unmeasured perceptual qualities such as prosodic naturalness. source: §4.3, Table 1, Figure 2 evidence: GRPO trained on a harmonic-mean reward of CER and token likelihood reduced CER to the lowest value among all tested systems (2.20%) but produced the lowest human-preference ELO (753.7), with log-F0 pitch distributions showing collapse toward monotone speech relative to the baseline. confidence: high relevance: high - claim_id: adding_an_auxiliary_reward_term_intended_to_fix role: complicates claim: Adding an auxiliary reward term intended to fix a known RL side effect can destabilize training instead of correcting the targeted attribute. source: §4.4 evidence: Extending the GRPO reward with a speaker-similarity utility increased similarity scores but inflated CER to 42.63% and produced degenerate non-terminating outputs (missing end-of-sequence tokens), consistent with reward hacking rather than genuine prosody improvement. confidence: high relevance: low - claim_id: small_scale_iterative_human_preference_optimization_can_recover role: supports claim: Small-scale iterative human preference optimization can recover perceptual speech qualities lost to metric-driven RL training without requiring a large static preference dataset. source: §4.5, Table 1 evidence: Using approximately 200 human preference pairs per round with a moving DPO reference, three iterative rounds raised ELO from 753.7 (GRPO) to a peak of 1190.1 at round 2, exceeding three commercial TTS baselines in human preference, while CER recovered to 3.30-3.60% by rounds 2-3. confidence: high relevance: low - claim_id: the_benefit_of_successive_rounds_of_iterative_preference role: refines claim: The benefit of successive rounds of iterative preference optimization is not monotonic and can saturate or reverse as the policy converges toward its own reference. source: §4.5 evidence: ELO peaked at round 2 (1190.1) and declined at round 3 (1064.2); the authors attribute this to a narrowing policy-reference gap reducing the informativeness of newly collected preference pairs in later rounds. confidence: high relevance: low limitations: - Beyond the evaluation scope, the paper does not report an ablation over the DPO sharpness parameter (beta) or over preference-pair budget per round, so it is unclear how sensitive the round-2 peak is to these choices. The mechanism behind the round-3 ELO decline is offered as a hypothesis (narrowing policy-reference gap) rather than directly verified. The GRPO speaker-similarity failure is also reported as a single run rather than across multiple reward-weighting configurations, leaving open whether a different weighting could avoid the observed instability. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.19231' published_date: "2025-09-23" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS - VC - evaluation architecture: - diffusion - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - disentangled_prosody_content_timbre - language_and_domain_specific_prosody_modeling claims: - claim_id: disentangling_acoustic_pronunciation_linked_style_from_prosodic_and role: supports claim: Disentangling acoustic (pronunciation-linked) style from prosodic and speaker-identity style in a TTS-based reconstruction system allows mispronunciation patterns to be suppressed while speaker identity is preserved. source: §5.1, Table 1 evidence: ChiReSSD, which selectively fine-tunes acoustic/prosodic style encoders and the pitch extractor while reusing StyleTTS2's disentangled style representations, achieves higher speaker similarity (0.62) and lower pitch deviation (19.98%) than a one-shot baseline that transfers style holistically from an unseen reference without domain adaptation (0.52 similarity, 22.58% F0 difference). confidence: high relevance: high - claim_id: optimizing_a_speech_reconstruction_system_for_automatic_speech role: complicates claim: Optimizing a speech reconstruction system for automatic-speech-recognition-based intelligibility metrics can conflict with optimizing for speaker-identity preservation. source: §5.2, Table 2 evidence: A single-speaker adult-voice TTS baseline with no style transfer obtains the lowest WER/CER on STAR (0.34/0.11) precisely because it discards the child speaker's identity, while the identity-preserving ChiReSSD model scores worse on the same ASR-based metrics (WER 0.49, CER 0.29) despite being judged clinically more accurate. confidence: high relevance: low - claim_id: an_automatic_phone_recognition_based_estimate_of_consonant role: supports claim: An automatic phone-recognition-based estimate of consonant production accuracy can approximate expert clinical judgments of speech intelligibility well enough to reduce, though not eliminate, the need for manual phonetic transcription. source: §5.3 evidence: Automatic PCC estimates derived from a universal phone recognizer and Levenshtein distance correlate with a certified speech-language therapist's manual PCC annotations at Pearson ρ = 0.63 across 21 original/reconstructed sample pairs. confidence: high relevance: low - claim_id: a_style_based_tts_reconstruction_approach_adapted_for role: supports claim: A style-based TTS reconstruction approach adapted for one clinical population of disordered speech can generalize to a structurally different disorder and age group without population-specific retraining. source: §5.4, Table 3 evidence: ChiReSSD, trained only on pediatric SSD data, reduces CER to below 0.03 and WER to as low as 0.06 on the TORGO adult dysarthria dataset across mild-to-severe severity levels, while keeping speaker similarity at 0.74-0.77, versus 0.43-0.55 for a non-personalized TTS baseline. confidence: high relevance: medium limitations: - Clinical validation of the automatic PCC metric rests on only 21 annotated sample pairs from a single speech-language therapist, with no inter-rater reliability reported; the ρ = 0.63 correlation should be treated as a preliminary estimate, not a validated clinical tool. - The STAR evaluation set itself is small (234 utterances from six children), and reconstruction quality is partly assessed with ASR models (whisperX) that are known to be poorly calibrated for child and disordered speech, which the paper acknowledges as a likely source of residual error inflation on STAR relative to LibriTTS. Generalization beyond the two evaluated accents/populations (Central Scottish child SSD, Canadian English adult dysarthria) is untested, and the guidance parameters (α, β) were tuned rather than learned, leaving open whether they transfer to other disorder types or severities without manual retuning. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.19883' published_date: "2025-09-24" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - singing architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - latent_reference_prosody_transfer - disentangled_prosody_content_timbre - speech_singing_shared_prosody claims: - claim_id: prompt_based_conditioning_in_masked_generative_or_codec role: supports claim: Prompt-based conditioning in masked generative or codec-language speech synthesis models causes measurable leakage of prosodic attributes from the acoustic prompt into the synthesized output, independent of the target language. source: §V.A, Table I evidence: Paired-prompt outputs show consistently lower pitch/energy/jitter differences than unpaired-prompt outputs from the same speaker on both LibriTTS (English) and AISHELL-3 (Mandarin) when synthesizing with MaskGCT. confidence: high relevance: high - claim_id: explicit_contrastive_regularization_between_the_acoustic_prompt_and role: supports claim: Explicit contrastive regularization between the acoustic prompt and an external control signal (e.g., melody/pitch) reduces attribute leakage and improves controllability in prompt-based zero-shot synthesis. source: §V.D, Table V evidence: Removing the coarse-to-fine (sequence + frame level) contrastive loss increases F0-RMSE from 0.042 to 0.08 and lowers SingMOS from 4.32 to 4.12 on the seen-singer test set, with sequence-level and frame-level components independently ablated to show complementary effects on speaker-identity and pitch-detail metrics respectively. confidence: high relevance: high - claim_id: integrating_auxiliary_transcription_derived_frame_level_supervision_directly role: supports claim: Integrating auxiliary transcription-derived frame-level supervision directly into a synthesis model's training loop, rather than using it only as an offline data-cleaning step, improves fine-grained attribute alignment. source: §V.D, Table V evidence: Removing the in-loop SVT auxiliary loss produces the largest single-component degradation in the ablation, raising F0-RMSE from 0.042 to 0.194 and lowering SingMOS from 4.32 to 3.95. confidence: high relevance: low - claim_id: parameter_efficient_fine_tuning_can_match_or_exceed role: refines claim: Parameter-efficient fine-tuning can match or exceed full fine-tuning when adapting a large pretrained codec-based speech model to a lower-resource downstream domain, provided the low-rank capacity is placed appropriately. source: §V.D, Table VII evidence: LoRA fine-tuning of the S2A diffusion estimator (6.51% trainable parameters) achieves lower F0-RMSE (0.053) and higher SECS (0.92) than fully fine-tuning the same backbone (100% trainable, F0-RMSE 0.099, SECS 0.859) when adapting MaskGCT to singing voice synthesis. confidence: high relevance: low limitations: - The zero-shot evaluation set (OpenSinger) has no native music-score annotations, so the authors pair OpenSinger audio with M4Singer's pitch/duration sequences to construct test inputs. This means the "unseen singer" evaluation is not evaluating on musically native score-audio pairs, which could understate or overstate melody-control difficulty relative to a genuinely paired unseen-singer benchmark. - Beyond that, evaluation is confined to Mandarin singing corpora (M4Singer, Opencpop, OpenSinger); no cross-lingual or multilingual SVS results are reported, despite the underlying MaskGCT backbone being multilingual-capable. Overall S2A/T2S model size is not reported (only the small SVT module's dimensions are given), limiting reproducibility of compute requirements. The subjective evaluation panel is modest (20 musically trained participants), typical for SVS papers but smaller than typical large-scale TTS MOS studies. Finally, the sequence-level contrastive objective depends on having multiple same-singer utterances available per training batch, which the curated corpora used here provide but which may not hold for less structured or lower-resource singing data. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2509.22718' published_date: "2025-09-24" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - singing architecture: - hybrid relevance: high evidence_role: - core_evidence - control_evidence current_role: active_evidence method_family: - explicit_acoustic_variance_control - multimodal_visual_prosody_conditioning - speech_singing_shared_prosody claims: - claim_id: synchronized_visual_cues_from_a_speaker_s_or role: supports claim: Synchronized visual cues from a speaker's or singer's mouth region can substitute for explicit phoneme-level duration annotations in duration-based speech and singing synthesis pipelines. source: §3.5, §5.1, Table 1 evidence: Replacing externally supplied phoneme durations with a duration predictor conditioned on lip-cue-fused content features (via VCFM) allows the duration predictor and Length Regulator to operate without duration inputs at inference, while achieving lower MCD and FFE than a duration-free StyleSinger baseline that has no visual input. confidence: high relevance: high - claim_id: fusion_mechanisms_designed_for_one_cross_modal_alignment role: complicates claim: Fusion mechanisms designed for one cross-modal alignment problem do not transfer directly to singing, because singing's rhythmic and prosodic complexity differs from speech. source: §5.2, Table 3 evidence: Adopting StyleDubber's Stepwise Monotonic Multi-head Attention (originally designed for movie-dubbing lip-speech alignment) for textual-visual fusion in this SVS setting produced worse MCD, COS, and LSE-D than the no-visual baseline, whereas the proposed VCFM improved on the same baseline across all metrics. confidence: high relevance: high - claim_id: staged_training_that_first_establishes_a_strong_acoustic role: supports claim: Staged training that first establishes a strong acoustic model before introducing an auxiliary modality improves quality over jointly training all components from the start. source: §5.2, Table 3 evidence: The two-stage schedule (stage one trains phoneme/pitch encoders and decoder; stage two adds the frozen visual encoder and VCFM) outperforms single-stage training on every objective metric (MCD 3.1125 vs. 3.1307, FFE 0.3921 vs. 0.4022, COS 0.9206 vs. 0.9195, LSE-C 1.4270 vs. 1.3888, LSE-D 10.2782 vs. 10.3146). confidence: high relevance: low - claim_id: visual_dubbing_architectures_built_for_spoken_dialogue_do role: complicates claim: Visual dubbing architectures built for spoken dialogue do not generalize to singing without substantial adaptation, even when augmented with pitch information. source: §4.1, §5.1, Table 1 evidence: HPMDubbing, a lip-synchronized speech dubbing model, and its pitch-augmented variant HPMDubbing-P both perform far below SVS-specific baselines on every metric (e.g., MOS-Q of 1.45–1.46 vs. 3.54–3.71 for singing-specific systems), despite HPMDubbing-P adding phoneme-level pitch as input. confidence: high relevance: high limitations: - All results are reported on a single self-constructed dataset (9 singers, 69 songs, ~3 hours, Chinese language only), evaluated against two baselines that were adapted from other tasks or settings rather than SVS systems natively designed for duration-free, multimodal input. There is no evaluation on an external or multi-lingual benchmark, so the generality of the reported gains beyond this dataset is untested. - The visual encoder is frozen during stage-two training rather than fine-tuned jointly with the rest of the pipeline, which may limit how well lip-cue features adapt to the singing domain specifically (the encoder was pretrained for general audio-visual speech tasks, not singing). The paper also does not report model parameter counts or inference latency, so the practical cost of adding a video branch relative to text-only SVS systems is not established. Manual TextGrid annotation required 40-45 human-hours per hour of audio, indicating that scaling VisualSinger-style datasets to more singers or languages will be labor-intensive. caveats: - Prosodic control is central to the paper's speech-generation contribution. - id: '2509.24570' published_date: "2025-09-29" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS - VC - evaluation architecture: - autoregressive-LM relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control - structured_language_prosody_control - implicit_scale_and_data_prosody_modeling claims: - claim_id: automated_pipelines_combining_expressive_tts_voice_conversion_and role: supports claim: Automated pipelines combining expressive TTS, voice conversion, and LLM-based instruction generation can produce large-scale paired speech style editing data without manual recording or annotation, provided a multi-criterion filtering step is applied. source: §2.1, §2.2, Fig. 3 evidence: The three-stage pipeline (EmoCapTTS synthesis + Chatterbox voice conversion + Qwen3-8B instruction generation) yields 382 hours and ~100,000 pairs from EARS and Expresso source material, filtered to WER < 10, style similarity > 0.5, and speaker similarity > 0.5 confidence: high relevance: medium - claim_id: fine_grained_diverse_natural_language_instructions_improve_both role: supports claim: Fine-grained, diverse natural-language instructions improve both in-domain accuracy and cross-domain generalization of instruction-guided speech style editing models relative to coarse, templated instruction sets. source: §4.2, Table 2 evidence: LlasaEdit trained on ISSE outperforms the same architecture trained on ESD across WER, style similarity, speaker similarity, and UTMOS in-domain (8.06 vs. 10.07 WER; 0.68 vs. 0.64 style-sim), and the ISSE-trained model's cross-domain performance on ESD exceeds the ESD-trained model's in-domain performance on several metrics confidence: high relevance: medium - claim_id: instruction_guided_style_editing_models_trained_on_narrow role: complicates claim: Instruction-guided style editing models trained on narrow, templated-instruction datasets fail catastrophically when evaluated on more diverse, fine-grained instruction distributions. source: §4.2, Table 2 evidence: The ESD-trained LlasaEdit model, when evaluated on the ISSE test set, produces a WER of 68.17, compared to 10.07 on its own in-domain ESD test set, indicating the model does not generalize beyond the coarse single-attribute instructions it was trained on confidence: high relevance: medium - claim_id: isolating_style_variation_from_speaker_identity_in_synthetically role: complicates claim: Isolating style variation from speaker identity in synthetically generated paired training data requires an explicit voice-conversion correction step, since expressive TTS models conditioned on style descriptions alone conflate style and timbre changes. source: §2.1 evidence: EmoCapTTS-generated stylized speech differs from the anchor speech in timbre because the model lacks explicit speaker control; a separate voice conversion stage (Chatterbox) is needed to re-align target timbre to the anchor speaker before the pair can be used to define a style-only edit confidence: high relevance: medium limitations: - The dataset and benchmark are limited to English, which the authors explicitly flag as constraining applicability to multilingual editing scenarios. The generated portion of ISSE (292 of 382 hours) is itself the product of a TTS+VC synthesis pipeline rather than real recordings, so any systematic biases or artifacts introduced by EmoCapTTS or Chatterbox could propagate into models trained on it; the quality-filtering thresholds (WER < 10, similarity > 0.5) are relatively loose and their effect on downstream editing fidelity is not separately ablated. The benchmark comparison is against a single alternative dataset (ESD) and a single model architecture (LlasaEdit); no comparison is made against other instruction-guided editing systems such as InstructSpeech, and no ablation isolates the individual contribution of instruction diversity versus raw data scale. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. - id: '2509.24650' published_date: "2025-09-29" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS - VC architecture: - autoregressive-LM - diffusion relevance: low evidence_role: - historical_context current_role: minor method_family: [] claims: - claim_id: a_differentiable_scalar_quantization_bottleneck_applied_to_hidden role: supports claim: A differentiable scalar-quantization bottleneck applied to hidden states, rather than used as a discrete prediction target, can induce semantic/acoustic task separation inside a continuous autoregressive TTS model without an external speech tokenizer. source: §4.3, Table 6 evidence: Removing the FSQ bottleneck from an otherwise identical hierarchical architecture increases ZH-hard-case CER from 18.19% to 24.92%, while FSQ dimensionality shows a non-monotonic optimum around 128-256 dimensions rather than monotonic improvement with capacity. confidence: high relevance: low - claim_id: explicitly_separating_acoustic_detail_recovery_into_a_dedicated role: supports claim: Explicitly separating acoustic detail recovery into a dedicated residual module improves robustness on complex inputs beyond what a single semantic language model plus a diffusion decoder achieves. source: §4.4, Table 7 evidence: Removing the RALM (TSLM output feeding the diffusion decoder directly, architecturally close to DiTAR) degrades EN-WER from 2.98% to 4.34% and ZH-hard-case CER from 18.19% to 25.0%; removing the historical acoustic embedding from the RALM input degrades results further. confidence: high relevance: low - claim_id: learning_rate_schedule_design_not_just_architecture_materially role: complicates claim: Learning-rate schedule design, not just architecture, materially affects zero-shot speaker similarity in large-scale continuous TTS training. source: §4.5, Table 8 evidence: A two-phase Warmup-Stable-Decay schedule's decay phase alone improves ZH-hard-case CER from 13.22% to 8.87% and SIM by 4.4 points over the stable-phase-only checkpoint on an otherwise identical model. confidence: high relevance: low - claim_id: classifier_free_guidance_strength_in_diffusion_based_tts role: complicates claim: Classifier-free guidance strength in diffusion-based TTS decoders trades off intelligibility against speaker similarity non-monotonically, with both very low and very high guidance scales degrading both metrics simultaneously. source: §4.6, Table 9 evidence: CFG scale 1.0 (no guidance) yields EN-WER 16.32% and SIM 55.1%, while scale 5.0 yields EN-WER 12.78% and SIM 60.7%; the optimum at scale 2.0 achieves EN-WER 1.85% and SIM 72.9%, with degradation on both sides of the optimum. confidence: high relevance: low - claim_id: removing_dependency_on_a_pre_trained_discrete_speech role: refines claim: Removing dependency on a pre-trained discrete speech tokenizer does not require sacrificing zero-shot voice cloning quality relative to discrete-token-based open-source TTS systems. source: §4.2, Table 3 evidence: On SEED-TTS-EVAL, the fully continuous VoxCPM reports SIM of 72.9% (EN) and 77.2% (ZH), exceeding the discrete-token-based IndexTTS2 and CosyVoice2 baselines on the same benchmark. confidence: high relevance: low limitations: - Multilingual capability is limited to Chinese and English by construction; the paper explicitly reports uncertain generalization to other languages, and prosody/emotion control lacks any intuitive or precise user-facing conditioning mechanism. The causal audio VAE operates at 16kHz, which the authors acknowledge falls short of the 24kHz or 44.1kHz sampling rates typically expected for high-fidelity applications. Baseline comparisons draw on official implementations or numbers reported in prior papers rather than a uniformly controlled re-evaluation, so cross-system rankings on tables that mix reproduced and self-reported numbers should be read cautiously. The training corpus (1.8M hours) is internal and not released, which limits independent reproduction of the full-scale result even though code and weights for the trained model are public. caveats: - The connection to prosodic control is contextual rather than a direct conditioning contribution. - id: '2509.26514' published_date: "2025-09-30" entry_date: '2026-07-25' year: 2025 venue: arXiv task: - TTS architecture: - autoregressive-LM - flow-matching - GAN relevance: medium evidence_role: - architecture_variant - control_evidence current_role: active_evidence method_family: - emotion_expressive_prosody_control - structured_language_prosody_control claims: - claim_id: decoupling_instruction_interpretation_from_speech_rendering_via_an role: supports claim: Decoupling instruction interpretation from speech rendering, via an explicit intermediate textual feature representation, can match or exceed instruction-annotated controllable TTS systems without requiring any manually labeled instruction-speech data. source: §3.2, Table 1 evidence: BatonVoice-1.7B reaches 57.6% emotion accuracy using 0 hours of manual instruction annotation, exceeding CosyVoice (43.8%, 556 hours) and CosyVoice2 (37.8%, 1,500 hours) on the same Seed-TTS-derived English emotion benchmark. confidence: high relevance: medium - claim_id: when_conditioning_signals_are_represented_as_explicit_text role: supports claim: When conditioning signals are represented as explicit text rather than learned embeddings, a downstream generation model can gain synthesis quality from a stronger upstream language model without any retraining of the generation model itself. source: §3.5, Figure 3b evidence: Holding BatonTTS fixed and only changing the external "conductor" LLM that produces the vocal plan, emotion accuracy rises monotonically from 29.8% (Qwen3-1.7B) through 39.8% (Qwen3-80B) and 47.8% (Qwen3-Max) to 57.6% (Gemini-2.5-Pro). confidence: high relevance: low - claim_id: automatic_emotion_classification_accuracy_and_human_judged_naturalness role: complicates claim: Automatic emotion-classification accuracy and human-judged naturalness can diverge, so strong performance on an LLM-judged style-control metric does not guarantee a favorable human preference outcome against commercial systems. source: §3.3, Table 2 evidence: On a free-form instruction-following test set judged by trained human annotators, BatonVoice wins only 56% of comparisons against CosyVoice and just 30% against Minimax-2.5-HD, with annotators specifically citing weaker fluency and naturalness. confidence: high relevance: medium - claim_id: numerical_structured_representations_of_prosodic_control_targets_transfer role: refines claim: Numerical, structured representations of prosodic control targets transfer more precisely to a conditioned TTS decoder than free-text qualitative descriptions of the same target style. source: §B.2, Table 5 evidence: On a RAVDESS reconstruction task, the structured numerical vocal-plan format achieves 1.54 Mel-Cepstral Distortion versus 2.62 for an equivalent caption-based qualitative description, and ablating any single numerical feature (pitch, energy, or spectral centroid) increases MCD. confidence: high relevance: high limitations: - The paper's headline emotion-control results across both English and Chinese benchmarks rely entirely on an LLM (Gemini-2.5-Pro) as an automatic judge rather than human raters; the one benchmark that does use human evaluation shows BatonVoice losing to the top closed-source baseline on naturalness and fluency (30% win rate against Minimax-2.5-HD), a gap not visible in the automated emotion-accuracy metric. - The vocal-plan representation is limited to pitch, energy, and spectral centroid; the authors note that finer-grained paralinguistic features such as emphatic stress and non-verbal vocalizations are not captured and are left to future work. Best reported results depend on using a large, capable, and likely costly external LLM (Gemini 2.5 Pro) as the conductor at inference time; using the in-house model of the same size as the orchestra as its own conductor (Qwen3-1.7B, 29.8% accuracy) performs far worse, so the practical quality of the system is bottlenecked by conductor access and cost, not by BatonTTS alone. Cross-lingual generalization is demonstrated on one unseen language (Chinese) with instructions and text machine-translated by the same LLM family used for judging, which leaves open how the approach performs on languages more typologically distant from English or evaluated independently of the conductor/judge model. caveats: - Prosodic control is an evaluated capability but not the paper's sole contribution. claim_clusters: - id: explicit_pitch_energy_duration_enable_controllability claim: Explicit prediction or conditioning of pitch, energy, and duration enables interpretable prosodic control and reduces variation left unresolved by text alone. status: strongly_supported confidence: high supporting_papers: - '2006.04558' - '2104.00355' - 2025.coling-main.352 - 2025.coling-main.518 - '2507.01611' - 2025.acl-industry.42 - 2025.acl-long.1471 - 2025.acl-long.790 - '2508.06890' - '2508.12001' - interspeech-2025-0047 - interspeech-2025-0347 - interspeech-2025-0754 - interspeech-2025-0948 - interspeech-2025-1098 - interspeech-2025-1494 - '2508.16332' - '2508.17031' - '2509.00675' - '2506.23367' - '2509.17143' - '2509.19883' contradicting_papers: [] refining_papers: - 2025.acl-long.346 - 2025.acl-long.790 caveats: - Evidence spans different acoustic backbones and attribute subsets; improvements in one variable do not guarantee joint control. last_reviewed: '2026-07-25' - id: stochastic_models_preserve_one_to_many_prosody claim: Stochastic prosody models preserve one-to-many variation better than deterministic regression while maintaining competitive synthesis quality. status: strongly_supported confidence: high supporting_papers: - 2025.coling-main.352 - 2025.coling-main.518 - 2025.acl-long.346 - 2025.ccl-1.77 - interspeech-2025-1940 contradicting_papers: [] refining_papers: - 2025.coling-main.352 caveats: - Reported gains use different latent spaces, inference budgets, and diversity metrics. last_reviewed: '2026-07-25' - id: prosodic_diversity_trades_off_with_naturalness claim: Increasing prosodic diversity or transformation strength can reduce naturalness, intelligibility, or speaker similarity beyond a task-dependent operating range. status: emerging confidence: medium supporting_papers: - 2025.coling-main.352 contradicting_papers: [] refining_papers: - 2025.coling-main.352 - '2508.11224' - interspeech-2025-2684 caveats: - The location of the operating point depends on representation, sampling scale, and evaluation protocol. last_reviewed: '2026-07-25' - id: disentanglement_enables_independent_prosody_transfer claim: Separating prosody from linguistic content and speaker timbre enables more independent transfer and manipulation of speaking style. status: strongly_supported confidence: high supporting_papers: - '2406.07855' - 2025.acl-long.346 - '2508.08399' - interspeech-2025-2684 contradicting_papers: [] refining_papers: [] caveats: - Several systems still retain speaker-correlated information or pay an intelligibility cost. last_reviewed: '2026-07-25' - id: reference_prompts_transfer_prosody_but_leak_attributes claim: Reference prompts transfer prosodic style effectively, but can leak speaker identity, content, pitch, or recording conditions into the output. status: strongly_supported confidence: high supporting_papers: - 2025.naacl-long.242 - '2508.11273' - interspeech-2025-0948 - interspeech-2025-1726 - '2508.17494' - 2025.clicit-1.27 - '2509.15626' - '2509.17516' - '2509.19883' contradicting_papers: [] refining_papers: - interspeech-2025-2684 - 2025.clicit-1.27 - '2509.17516' caveats: - Prompt duration, pairing policy, and same-speaker training assumptions materially affect both transfer and leakage. last_reviewed: '2026-07-25' - id: linguistic_context_improves_prosody_prediction claim: Linguistic and conversational context improves prediction of phrasing, prominence, stress, pitch, and duration over sentence-isolated or word-level conditioning. status: strongly_supported confidence: high supporting_papers: - '2507.03912' - 2025.acl-long.1471 - 2025.unlp-1.11 - '2507.20091' - '2508.11224' - interspeech-2025-0253 - interspeech-2025-0408 - interspeech-2025-0464 - interspeech-2025-2189 - '2509.06074' - '2509.17143' contradicting_papers: [] refining_papers: - '2507.08012' - interspeech-2025-1494 caveats: - Different attributes depend on asymmetric past and future context windows, and longer context is not uniformly beneficial. last_reviewed: '2026-07-25' - id: duration_and_alignment_are_central_to_rhythm_control claim: Duration, pause, and alignment modeling are central to intelligible rhythm, stress, phrasing, and cross-modal synchronization. status: strongly_supported confidence: high supporting_papers: - '2006.04558' - '2406.07855' - 2025.coling-main.352 - 2025.naacl-long.484 - 2025.acl-industry.42 - 2025.acl-long.1471 - 2025.acl-long.790 - '2508.12001' - interspeech-2025-0754 - interspeech-2025-0815 - interspeech-2025-1098 - interspeech-2025-1494 - interspeech-2025-1726 - interspeech-2025-2032 - '2508.17031' - '2509.00675' - '2506.21619' - '2506.23367' - '2509.17516' - '2509.22718' contradicting_papers: [] refining_papers: - '2507.03912' - '2507.04598' - 2025.acl-long.790 - interspeech-2025-1940 - '2509.00675' - '2509.22718' caveats: - Fine-grained duration control can become unstable, and alignment supervision is costly or language-dependent. last_reviewed: '2026-07-25' - id: fine_grained_local_control_is_harder_than_global_style claim: Word- and phoneme-level prosodic control is less reliable than utterance-level style control because local timing and linguistic interactions must be modeled explicitly. status: strongly_supported confidence: high supporting_papers: - '2406.07855' - 2025.coling-main.352 - '2507.03912' - '2507.04598' - 2025.unlp-1.11 - '2507.20091' - interspeech-2025-0754 - interspeech-2025-2032 - '2508.17031' - '2509.06074' - '2509.07038' - '2509.22718' contradicting_papers: [] refining_papers: - '2507.04598' - 2025.acl-long.790 - interspeech-2025-2159 caveats: - Local control results are concentrated in emphasis, emotion, and duration tasks rather than a shared comprehensive benchmark. last_reviewed: '2026-07-25' - id: targeted_data_and_finetuning_improve_prosodic_contrasts claim: Targeted expressive data or fine-tuning improves prosodic contrasts that broad read-speech training does not reliably produce. status: emerging confidence: medium supporting_papers: - interspeech-2025-2159 contradicting_papers: [] refining_papers: [] caveats: - Improvements often fail to generalize uniformly across contrast types, speakers, or domains. last_reviewed: '2026-07-25' - id: ssl_representations_are_useful_but_entangled_for_prosody claim: Self-supervised speech representations provide strong prosodic signals, but remain entangled with speaker identity, content, or task-specific information. status: strongly_supported confidence: high supporting_papers: - '2104.00355' - 2025.coling-main.518 - '2507.03912' - '2508.08399' - '2508.11224' - '2508.11273' - interspeech-2025-2684 - '2509.01391' - '2509.17143' contradicting_papers: [] refining_papers: - 2025.findings-naacl.471 - '2507.08012' - '2508.11224' - interspeech-2025-2189 - interspeech-2025-2684 caveats: - Which SSL layer or objective is best depends on whether the downstream task values reconstruction, transfer, or factor separation. last_reviewed: '2026-07-25' - id: objective_prosody_metrics_do_not_fully_predict_perception claim: Objective prosody and alignment metrics do not consistently predict perceived naturalness, expressiveness, or control success. status: emerging confidence: medium supporting_papers: - '2509.00675' contradicting_papers: [] refining_papers: - interspeech-2025-1494 - '2509.06074' caveats: - Disagreement varies by task and metric; objective measures remain useful for diagnosis even when unsuitable as sole endpoints. last_reviewed: '2026-07-25' - id: multimodal_cues_add_prosodic_information claim: Visual and other multimodal cues add information for prosody, emotion, timing, or duration beyond text-only conditioning. status: strongly_supported confidence: high supporting_papers: - 2025.ccl-1.77 - '2508.04585' - interspeech-2025-1494 - '2508.17031' - '2509.06074' - '2509.22718' contradicting_papers: [] refining_papers: - interspeech-2025-1494 - '2509.22718' caveats: - Benefits are dimension-specific and cross-modal fusion designs do not transfer uniformly between speech, dubbing, and singing. last_reviewed: '2026-07-25' - id: hierarchical_emotion_modeling_improves_expressive_prosody claim: Hierarchical or localized emotion modeling improves expressive prosody compared with a single global emotion representation. status: strongly_supported confidence: high supporting_papers: - 2025.findings-naacl.38 - '2507.04598' - 2025.ccl-1.77 - '2507.20091' - interspeech-2025-0754 - '2509.06074' contradicting_papers: [] refining_papers: - '2507.04598' caveats: - Local emotion targets are harder to infer from text and can accumulate prediction errors across levels. last_reviewed: '2026-07-25' - id: structured_prosody_plans_outperform_free_form_generation claim: Structured tags, SSML, or numerical vocal plans provide more precise prosodic control than unconstrained free-form generation. status: emerging confidence: medium supporting_papers: - '2508.17494' contradicting_papers: [] refining_papers: - '2505.17093' - '2509.26514' caveats: - Structured plans increase precision but require an interpretation layer and may constrain unanticipated attributes. last_reviewed: '2026-07-25' - id: language_specific_structure_matters_for_prosody claim: Tonal, stress-marked, low-resource, and historical languages require language-specific representations or preprocessing for reliable prosody. status: strongly_supported confidence: high supporting_papers: - '2507.03912' - 2025.acl-industry.42 - 2025.unlp-1.11 - '2508.13028' - 2025.clicit-1.27 contradicting_papers: [] refining_papers: - 2025.sigdial-1.27 - 2025.clicit-1.27 caveats: - Evidence is distributed across languages and tasks with few shared evaluation sets. last_reviewed: '2026-07-25' - id: speech_and_singing_benefit_from_shared_prosody_learning claim: Joint speech–singing learning and shared pitch or rhythm representations can improve expressive generation across both domains. status: strongly_supported confidence: high supporting_papers: - '2304.09116' - interspeech-2025-0347 - '2508.16332' - '2509.22718' contradicting_papers: [] refining_papers: - '2509.22718' caveats: - Cross-domain gains coexist with distribution mismatch in pitch range, rhythm, and annotation conventions. last_reviewed: '2026-07-25' - id: scale_and_data_improve_implicit_prosody_modeling claim: Model scale and broad training data improve implicit prosody rendering even without explicit prosodic annotation. status: strongly_supported confidence: high supporting_papers: - '2402.08093' - '2502.04128' - 2025.acl-long.1471 contradicting_papers: [] refining_papers: - 2025.coling-main.352 caveats: - Scale is confounded with data quality, architecture, and evaluation difficulty. last_reviewed: '2026-07-25' method_families: - id: explicit_acoustic_variance_control name: Explicit acoustic variance control summary: Pitch, energy, duration, rate, and related interpretable variables are predicted or supplied explicitly to control prosodic realization. papers: - '1609.03499' - '2006.04558' - '2104.00355' - 2025.coling-main.352 - 2025.coling-main.518 - 2025.naacl-long.484 - '2507.06235' - '2507.01611' - '2507.03912' - '2507.04598' - '2507.04817' - 2025.acl-industry.42 - 2025.acl-long.1471 - 2025.acl-long.346 - 2025.acl-long.790 - '2508.06890' - '2508.08399' - '2508.09767' - '2508.12001' - interspeech-2025-0047 - interspeech-2025-0347 - interspeech-2025-0408 - interspeech-2025-0433 - interspeech-2025-0754 - interspeech-2025-0815 - interspeech-2025-0948 - interspeech-2025-1020 - interspeech-2025-1098 - interspeech-2025-1494 - interspeech-2025-1940 - interspeech-2025-2032 - interspeech-2025-2189 - '2508.16332' - '2508.17031' - '2508.17494' - '2509.00675' - '2506.21619' - '2506.23367' - '2509.06074' - '2509.17143' - '2509.19883' - '2509.22718' open_questions: - How should explicit variables be normalized across speakers and languages without suppressing expressive variation? - id: latent_reference_prosody_transfer name: Latent and reference-based prosody transfer summary: Reference audio or learned latent/style representations transfer global or local prosodic characteristics to a target utterance. papers: - '2304.09116' - 2025.coling-main.518 - 2025.findings-naacl.38 - 2025.naacl-long.242 - '2507.08012' - '2507.04817' - 2025.acl-industry.42 - '2508.03543' - '2508.06890' - '2508.09702' - '2508.11273' - interspeech-2025-0464 - interspeech-2025-0723 - interspeech-2025-0948 - interspeech-2025-1098 - interspeech-2025-1726 - interspeech-2025-2684 - '2508.17494' - 2025.clicit-1.27 - '2506.21619' - '2509.15626' - '2509.17516' - '2509.19883' open_questions: - How can reference conditioning transfer prosody without leaking speaker identity, content, or recording conditions? - id: stochastic_generative_prosody_model name: Stochastic generative prosody models summary: Diffusion, flow matching, or other stochastic predictors model one-to-many prosodic variation rather than regressing to a single contour. papers: - 2025.coling-main.352 - 2025.coling-main.518 - 2025.naacl-long.242 - 2025.ccl-1.77 - '2508.12001' - interspeech-2025-0948 - interspeech-2025-1940 - '2509.17143' open_questions: - Where is the practical frontier between useful diversity, inference cost, and loss of naturalness? - id: disentangled_prosody_content_timbre name: Disentangled prosody, content, and timbre summary: Architectures separate prosody from linguistic content and speaker timbre so attributes can be transferred or manipulated independently. papers: - '2104.00355' - '2403.03100' - '2406.07855' - '2507.04817' - 2025.acl-long.346 - '2507.20091' - '2508.06890' - '2508.08399' - interspeech-2025-0347 - interspeech-2025-0433 - interspeech-2025-0464 - interspeech-2025-2684 - '2506.21619' - '2509.15626' - '2509.17143' - '2509.19231' - '2509.19883' open_questions: - Which objectives remove attribute leakage without sacrificing intelligibility or speaker similarity? - id: linguistic_context_prosody_prediction name: Linguistic and dialogue-context prosody prediction summary: Textual, conversational, and speaker context is encoded to predict phrasing, prominence, stress, pauses, and conversational prosody. papers: - '2507.03912' - 2025.acl-industry.42 - 2025.acl-long.1471 - 2025.unlp-1.11 - '2508.04585' - interspeech-2025-0253 - interspeech-2025-2159 - '2509.00675' - '2509.06074' open_questions: - How much past and future context is useful for each prosodic attribute, and how should irrelevant context be suppressed? - id: emotion_expressive_prosody_control name: Emotion and expressive prosody control summary: Emotion labels, intensities, learned directions, or expressive references condition prosodic realization at utterance and sub-utterance levels. papers: - 2025.findings-naacl.38 - '2507.04598' - '2507.04817' - '2507.09310' - 2025.acl-long.346 - 2025.acl-long.790 - 2025.ccl-1.77 - 2025.sigdial-1.27 - '2508.03543' - '2508.04585' - '2507.20091' - '2508.06890' - '2508.11224' - '2508.11273' - '2508.12001' - interspeech-2025-0047 - interspeech-2025-0754 - interspeech-2025-0815 - interspeech-2025-0948 - interspeech-2025-2189 - interspeech-2025-2684 - '2508.13028' - '2508.16332' - '2506.21619' - '2509.04072' - '2509.07038' - '2509.24570' - '2509.26514' open_questions: - Can emotion control remain perceptually coherent when intensity varies locally within an utterance? - id: structured_language_prosody_control name: Structured and language-mediated prosody control summary: Natural-language requests are mapped directly or through SSML, tags, or numerical plans into controllable prosodic targets. papers: - '2402.01912' - 2025.findings-naacl.38 - interspeech-2025-1192 - '2508.17494' - 2025.clicit-1.27 - '2506.21619' - '2505.17093' - '2509.24570' - '2509.26514' open_questions: - Which intermediate representation best combines user accessibility with precise timing and magnitude control? - id: multimodal_visual_prosody_conditioning name: Multimodal and visual prosody conditioning summary: Visual, textual, and acoustic modalities are fused to infer timing, emotion, pitch, duration, or dubbing alignment. papers: - '2304.09116' - 2025.ccl-1.77 - '2508.04585' - interspeech-2025-1494 - '2508.17031' - '2509.06074' - '2509.22718' open_questions: - Which visual signals genuinely add prosodic information beyond correlated text and speaker cues? - id: signal_level_pitch_modification name: Signal-level pitch and contour modification summary: Vocoder or signal-level mechanisms directly modify pitch and related contours, often beyond the range represented in acoustic-model training. papers: - '2104.00355' - '2507.06235' - '2507.01611' - '2508.12001' - interspeech-2025-2032 - '2508.16332' open_questions: - How far can pitch and contour manipulation move from the training distribution before voicing and identity failures dominate? - id: speech_singing_shared_prosody name: Shared speech–singing prosody representations summary: Shared or bridged representations transfer pitch, rhythm, timbre, and expressive information between speech and singing synthesis. papers: - '2304.09116' - interspeech-2025-0347 - '2508.16332' - '2509.07038' - '2509.19883' - '2509.22718' open_questions: - What representation preserves speech naturalness and singing accuracy without requiring symbolic music annotation? - id: implicit_scale_and_data_prosody_modeling name: Implicit scale- and data-driven prosody modeling summary: Large models and broad corpora acquire prosodic rendering without explicit acoustic-variable supervision. papers: - '2402.08093' - 2025.coling-main.352 - '2502.04128' - 2025.acl-long.1471 - '2509.00675' - '2509.24570' open_questions: - How much improvement comes from model capacity, data volume, data diversity, or codec and decoder choices? - id: language_and_domain_specific_prosody_modeling name: Language- and domain-specific prosody modeling summary: Language structure, speaking domain, and atypical speech require targeted representations, preprocessing, or adaptation for prosodic fidelity. papers: - 2025.nodalida-1.32 - '2507.03912' - 2025.acl-industry.42 - 2025.unlp-1.11 - 2025.sigdial-1.27 - interspeech-2025-1726 - '2508.13028' - 2025.clicit-1.27 - '2509.19231' open_questions: - Which prosodic mechanisms transfer across languages and domains, and which require targeted supervision? - id: prosody_analysis_and_evaluation_method name: Prosody analysis and evaluation methods summary: Diagnostic and evaluation methods identify prosodic information, predict labels, or expose gaps between measured and perceived behavior. papers: - '2402.08093' - 2025.findings-naacl.471 - '2507.03912' - interspeech-2025-1494 - '2509.03940' open_questions: - Can a shared analysis protocol connect acoustic contours, communicative function, and perceptual quality? - id: voice_quality_and_paralinguistic_feature_control name: Voice-quality and paralinguistic feature control summary: Interpretable voice-quality and paralinguistic acoustic features control phonation and perceived speaker characteristics. papers: - '2006.04558' - '2507.06235' - '2507.20091' - interspeech-2025-0902 open_questions: - How can voice-quality controls be changed independently of pitch, identity, and linguistic prosody? reassessment_queue: - id: prosodic_diversity_trades_off_with_naturalness type: claim_status reason: The trade-off appears repeatedly, but studies use incompatible diversity and naturalness measures. trigger: A controlled multi-model study traces a common diversity–naturalness frontier under matched data and evaluation. due: 2026-10 current_assessment: emerging watch_for: - Matched sampling-temperature studies - Joint subjective diversity and naturalness evaluation - id: objective_prosody_metrics_do_not_fully_predict_perception type: benchmark_validity reason: Metric disagreement is recurrent and directly affects cluster interpretation. trigger: A validated benchmark reports attribute-level correlations between objective contours, automatic judges, and human perception. due: 2026-10 current_assessment: emerging watch_for: - Cross-language perceptual validation - Metric behavior under local versus global control - id: explicit_acoustic_variance_control type: method_family reason: The family is broad and may need separation into predictor-based, externally specified, and post-hoc modification branches. trigger: At least three additional papers establish a stable subgroup with distinct conditioning placement. due: 2026-10 current_assessment: active_evidence watch_for: - Consistent separation between prediction, editing, and vocoder-stage control - id: ssl_representations_are_useful_but_entangled_for_prosody type: claim_status reason: Different SSL encoders and layers trade prosodic richness against speaker and content leakage. trigger: A layer-controlled comparison evaluates reconstruction, transfer, and leakage on the same corpus. due: 2026-10 current_assessment: strongly_supported watch_for: - Layer-wise SSL prosody benchmarks - Speaker-controlled leakage tests - id: structured_prosody_plans_outperform_free_form_generation type: claim_status reason: Evidence favors structured plans for precision but remains limited to a small set of interfaces and tasks. trigger: A matched renderer comparison evaluates free-form, categorical, SSML, and numerical controls with human raters. due: 2026-10 current_assessment: emerging watch_for: - Matched-interface studies - Local timing and magnitude evaluation - id: reference_prompts_transfer_prosody_but_leak_attributes type: claim_status reason: Prompt leakage depends on training pairing, tokenizer, speaker assumptions, and evaluation design. trigger: Independent work isolates these factors under matched voice conversion and TTS settings. due: 2026-10 current_assessment: strongly_supported watch_for: - Prompt/target decoupling ablations - Cross-speaker and cross-language leakage audits - id: emotion_expressive_prosody_control type: method_family reason: The family spans categorical emotion, localized intensity, activation steering, dialogue affect, and reference-based expression. trigger: At least three additional papers reinforce a stable mechanistic subdivision rather than a task-only distinction. due: 2026-10 current_assessment: active_evidence watch_for: - Separable localized-control, latent-steering, and reference-transfer branches open_questions: - How should prosodic control be represented so local timing, global style, and speaker identity remain independently editable? - Which objective and automatic metrics reliably predict human judgments of prosodic naturalness, diversity, and instruction adherence? - Where is the practical operating frontier between prosodic diversity and naturalness for stochastic generators? - How much linguistic, dialogue, and acoustic context is needed for each prosodic attribute? - Can language-independent prosody models retain tonal, stress, and phonemic-length distinctions without language-specific supervision? - Which SSL representations and layers preserve useful prosody while minimizing speaker and content leakage? trend_notes: - Prosody control moves from explicit pitch, energy, and duration predictors toward stochastic latent generators and flow matching by 2024–2025. - The 2025 literature increasingly targets word- and phoneme-level control rather than utterance-level style alone. - Reference prompting remains common, but 2025 work increasingly treats prompt leakage and speaker–prosody entanglement as first-class failure modes. - Language models appear both as prosody predictors and as interfaces that translate free-form requests into structured tags, SSML, or numerical plans. - Evaluation broadens in 2025 from contour reconstruction toward perceptual diversity, contextual appropriateness, attribute leakage, and human–metric disagreement.