EMNLP · 2025 · Conference
Chenlin Liu et al. · → Paper · Demo: ? · Code: ✓
GOAT is the first post-training framework to apply GFlowNets to LM-based TTS, reformulating autoregressive speech generation as trajectory flow optimization to reduce hallucinations by more than 50% on challenging test cases without additional inference cost.
Problem
Autoregressive language model-based TTS systems (the VALL-E family, CosyVoice, and related approaches) are prone to hallucination: generated speech that deviates from the input text through mispronunciations, word omissions, unnatural repetitions, or mid-generation collapse. Two dominant mitigation directions both carry unacceptable costs: training-time scaling demands large corpora and compute, while test-time scaling (e.g. beam search) introduces inference latency that is incompatible with real-time applications. No lightweight post-training method had been demonstrated to address the root cause of hallucination — high model uncertainty at specific decoding steps — in a general, resource-efficient way.
Method
GOAT (GFlOwNet-guided distribution AlignmenT) is a post-training framework built on top of CosyVoice 2 as the backbone. The approach proceeds in three stages.
Hallucination analysis. The authors first establish that hallucination in LM-based TTS is strongly correlated with model uncertainty (Pearson r = 0.636, Spearman r = 0.649, p < 1e-4 on 200 samples from SeedTTS-Eval test-hard). Uncertainty is measured via token-level entropy aggregated at character and utterance levels, adapting uncertainty-based hallucination detection from NLP to the speech modality.
GFlowNet objective. GOAT treats the autoregressive generation of speech tokens as a trajectory flow optimization problem. The forward sampling policy P_GFN is trained to produce token sequences with probability proportional to a reward function. Because LM-based TTS generation is deterministic conditioned on its prefix (each state has exactly one parent), the backward policy collapses to unity (P_B ≡ 1), simplifying the SubTrajectory Balance (SubTB) objective to depend only on the forward policy and the reward.
Internal reward and reward hacking suppression. Rather than relying on an external reward model or labeled data, GOAT uses an intrinsic reward: the cumulative log-probability of the generated token sequence under the backbone LM, sharpened by an inverse temperature T (0 < T < 1). A lower T produces a sharper reward distribution that encourages deterministic high-quality sequences. Two stability mechanisms suppress reward hacking (premature sequence truncation): (1) reward temperature decay, which linearly decreases T from 1.0 to a minimum of 0.825 during training, and (2) learning rate optimization combining warm-up and cosine annealing with a maximum learning rate of 1e-5.
Training details. GOAT post-trains the CosyVoice 2 language model component using LoRA adapters on only 1,000 text-speech pairs per language (Chinese from WenetSpeech4TTS, English from LibriTTS train-clean-100). Training runs for approximately 15 rounds (~3,500 global steps, ~7 hours) on 4 NVIDIA H100 GPUs with ~70 GB memory per GPU. Inference uses standard Repetition Aware Sampling (RAS) from CosyVoice 2 with top-p=0.8, top-k=25, introducing no latency beyond the LoRA overhead.
Key Results
All results are from Table 1, evaluated on SeedTTS-Eval. The key comparisons are against the unmodified CosyVoice 2 baseline using random multinomial sampling (RMS):
- On test-hard (challenging subset): CER drops from 13.72% to 6.28–6.61% (RAS/RMS), a >50% reduction.
- On test-zh: CER drops from 4.61% (baseline RMS) to 0.85–0.94% (GOAT models), matching or surpassing human CER of 1.31%.
- On test-en: WER drops from 7.10% (baseline RMS) to 1.96–2.43% (GOAT models), approaching human WER of 2.74%.
- UTMOS improves consistently across all test sets (e.g., test-en: 3.880 → 4.216 for RAS).
- Speaker similarity (SS) remains stable; small drops (0.84 → 0.80–0.83) are within perceptually negligible range when SS > 0.7.
Cross-lingual generalization holds: models trained on one language and evaluated on the other still outperform the baseline. Mixed-language training on 500 samples per language achieves performance comparable to monolingual training.
Ablation (Table 2): Enhanced SubTB reduces test-hard CER from 13.72% to 6.61%; degrading to plain Trajectory Balance yields 11.72–11.84%, demonstrating that fine-grained subtrajectory optimization is essential.
Uncertainty reduction (Table 3): All GOAT models reduce utterance-level uncertainty ratio (UUR) by 42–66% relative to baseline, with the matched-language 2500-step model achieving UUR = 0.39–0.42.
Inference latency: RTF on test-zh (2,000 samples) shows no statistically significant increase over baseline; real-time generation is maintained on a V100 GPU.
Novelty Assessment
The primary novelty is the application of GFlowNets to LM-based speech synthesis — the first such use case reported. The reframing of autoregressive TTS as trajectory flow optimization, and the derivation of a specialized SubTB objective that exploits the deterministic-prefix property of speech LMs, is methodologically original. The intrinsic reward design (sharpened LM probability with temperature decay) avoids external supervision, making the approach genuinely lightweight.
The contributions are a mix: the theoretical adaptation of GFlowNets to TTS is architecturally novel, the hallucination-uncertainty correlation analysis is an empirical contribution, and the training recipe (LoRA, small dataset, H100 × 4) is engineering-oriented. The overall contribution is incremental relative to broader RLHF/fine-tuning literature but represents a notable first application of GFlowNets in speech. The results are strong but are only demonstrated on one backbone (CosyVoice 2), limiting generality claims.
Field Significance
Tip
High — GOAT introduces a principled post-training alignment strategy for autoregressive TTS that is the first application of GFlowNets to the speech domain. It demonstrates that intrinsic reward-based distribution alignment can cut hallucination rates by over 50% on challenging test cases using fewer than 1,000 training samples per language, providing a lightweight alternative to beam search and large-scale retraining. The hallucination-uncertainty correlation analysis also provides the field with a concrete diagnostic framework for understanding autoregressive TTS failure modes.
Claims
- Token-level entropy is a statistically significant predictor of hallucination in autoregressive LM-based TTS, enabling unsupervised detection of likely error regions before synthesis is complete. (§2.1, §2.2)
- Reformulating autoregressive speech token generation as trajectory flow optimization via GFlowNets enables reward-proportional sampling that reduces hallucination without modifying the inference procedure. (§3.1, §3.2)
- An intrinsic reward derived from the backbone LM’s own probability distribution is sufficient to drive distribution alignment in post-training, without requiring human annotations or an external reward model. (§3.2.2)
- Fine-grained subtrajectory optimization is essential for hallucination suppression in long and complex utterances; degrading to trajectory-level balance significantly reduces the error reduction. (§4.4.5, Table 2)
- Post-training alignment for hallucination suppression in one language generalizes to unseen languages, with cross-lingual models retaining meaningful gains over the untuned baseline. (§4.4.4, Table 1)
Limitations and Open Questions
- Evaluated on a single backbone (CosyVoice 2); generalization to other LM-based TTS architectures (VALL-E, BASE TTS, LLaSA) is asserted but not demonstrated.
- The uncertainty-hallucination correlation explains only part of hallucination causes (correlation 0.636–0.649); prosody errors show no consistent uncertainty pattern and are not effectively addressed.
- The internal reward assumes the backbone LM assigns higher probability to higher-quality sequences — an assumption the authors acknowledge is a hypothesis.
- Training still requires H100 GPU access (~70 GB VRAM per GPU), which is non-trivial despite the small data requirement.
- Reward temperature is a sensitive hyperparameter; the sweet spot (0.825) was found empirically and may differ for other backbones or domains.
- No subjective listening evaluation (MOS/MUSHRA) is reported; quality improvements are measured entirely through ASR-based error rates and the automatic UTMOS predictor.
Wiki Connections
This paper advances autoregressive-codec-tts by addressing one of its core failure modes — hallucination in autoregressive speech token decoding — using a novel post-training alignment strategy. It is closely related to rlhf-speech, as GOAT is an intrinsic reinforcement learning method that aligns model outputs with a reward without human annotation. The entropy-based hallucination detection methodology connects to evaluation-metrics and subjective-evaluation. The zero-shot cross-lingual evaluation setup links to zero-shot-tts and multilingual-tts.