arXiv · 2026 · Preprint
Yong Ren et al. · → Paper · Demo: ? · Code: ?
Proposes Mean Continuation Log-Probability (MCLP), an interpretable metric that repurposes a pretrained large audio language model’s in-context learning to quantify speaking-style consistency, then uses MCLP as a GRPO reinforcement-learning reward (gated by a content-fidelity constraint) to align role-play TTS systems with scene and character instructions, validated on a newly constructed 1,435-hour drama-derived role-play dataset.
Problem
Large Audio Language Models (LALMs) have extended TTS into interactive role-play scenarios, where synthesized speech must faithfully match a character profile and scene description and remain stylistically coherent across multi-turn dialogue, not just produce isolated expressive utterances. Existing controllable/instruction-based TTS systems model style as a static, per-utterance property, which breaks down in role-play settings where style must evolve with conversational dynamics and interaction history. A deeper bottleneck blocks progress: there is no objective metric for quantifying speaking style itself, since style is entangled with emotion, prosody, and paralinguistic cues that resist a clean, interpretable definition. Without such a metric, reinforcement learning approaches for TTS style alignment fall back on discrete emotion-classification labels as a reward proxy, which cannot capture the continuous, context-dependent, and often non-emotional stylistic attributes role-play speech actually requires.
Method
The paper’s central idea is Mean Continuation Log-Probability (MCLP): given a pretrained LALM trained autoregressively on large-scale transcribed speech (interleaved text-and-audio TA4 tokens, following Step-Audio-2), a candidate utterance’s style is scored by constructing a context history that concatenates the transcript, the candidate audio, and the transcript again, then computing the mean log-likelihood the model assigns to the ground-truth audio tokens conditioned on this history. If the candidate audio is stylistically consistent with the ground truth, the model (which has implicitly learned a continuous latent space of speaking styles from its pretraining data) should assign the ground-truth continuation higher probability; if the candidate diverges stylistically, the ground-truth continuation becomes less predictable. The paper motivates this information-theoretically: since the transcript is held identical between candidate and ground truth, variation in mutual information between the two audio sequences is dominated by style rather than content, so under a well-calibrated pretrained density estimator, higher MCLP corresponds to lower conditional entropy and thus higher style-relevant mutual information with the reference.
Building on MCLP, the paper fine-tunes Step-Audio-2-mini-Base (a 7B decoder-only LALM) for role-play TTS (RP-TTS) in two stages. Supervised fine-tuning trains the model to generate the target turn’s interleaved TA4 sequence conditioned on the full role-play context: a scene description, character profiles for all speakers, and the dialogue history of prior turns (text instructions plus, when available, prior audio). Reinforcement learning then applies Group Relative Policy Optimization (GRPO) on the final turn of each dialogue, using a composite, gated reward designed to avoid the two failure modes a naive single-objective reward would produce: a style reward (the MCLP score, shifted into a positive range) and a content penalty (Character Error Rate from a cascaded Token2Wav-plus-ASR pipeline transcribing the generated audio). Rather than simply summing these, the reward is gated: if CER exceeds a tolerance threshold, the total reward is zeroed regardless of style score, forcing the model to first satisfy a minimum intelligibility bar before any stylistic reward can be earned, effectively creating a curriculum where content fidelity is a prerequisite rather than a co-equal, freely tradeable objective.
To support both training and evaluation, the paper constructs a large-scale role-play TTS dataset from the WenetSpeech YouTube-drama subset: 17,253 candidate drama videos are filtered, source-separated (Demucs) for clean vocals, speaker-diarized (pyannote), and hierarchically annotated, using DeepSeek-R1 to infer character profiles from aggregated per-speaker dialogue history, and Qwen-VL-7B to generate scene/atmosphere descriptions from video frames at silence-triggered scene boundaries. The final filtered corpus totals roughly 311k scenes and 1,435 hours, averaging 7.3 utterances and 2.33 distinct speakers per scene; a held-out, video-level-split test set of 900 scenes is stratified by dialogue length (2-10 turns) to prevent data leakage, and a further-filtered 16,186-scene high-quality subset (restricted to short 2-6-turn dialogues, non-trivial final-turn text length, and explicitly non-neutral emotional style) supplies the GRPO training data.
Key Results
Against three strong LALM baselines (closed-source GPT-Audio, open-source MiMo-Audio-7B-Instruct, and the instruction-tuned Step-Audio-2-mini), the proposed method achieves the best performance on every metric in both evaluation settings (with and without prior-turn audio history available). With audio history, it reaches CER 1.130% and Pinyin WER 0.511% (versus 3.276%/2.031% for the next-best baseline, Step-Audio-2-mini), the best MCLP score (-4.636, versus -4.753 to -4.849 for baselines), and a subjective MOS of 3.646±0.060, approaching the ground-truth naturalness ceiling of 4.411±0.052 and far exceeding all baselines (1.707-2.471). A “cold start” setting without audio history exposes a specific baseline failure mode: GPT-Audio’s CER catastrophically degrades from 11.974% to 44.679% when deprived of acoustic history, while the proposed method’s CER stays low in both settings (1.130% vs. 1.625%), and its MCLP score is even slightly higher with audio history than without (a +0.051 gain), showing it successfully exploits historical acoustic cues that baselines cannot leverage consistently. A human-metric consistency analysis (32 expert listeners, pairwise ΔMCLP-vs-ΔMOS win-rate binning) confirms MCLP tracks human perception of style: win rate is near chance (0.5) for small MCLP differences but exceeds 0.8 once the MCLP gap surpasses 0.1, a clear, interpretable correlation. Ablations isolate each component’s contribution: SFT alone already improves style-instruction following over the base model; adding RL yields further gains in both content fidelity (CER improves by 2.2-4.7 percentage points) and style (MCLP improves further). Critically, removing either reward component in isolation demonstrates genuine reward hacking: style-only optimization (no CER penalty) achieves the single best theoretical MCLP score (-4.590) but collapses CER above 50%, producing fixed, repetitive, linguistically meaningless acoustic patterns; content-only optimization (no MCLP reward) achieves the lowest CER (0.783%) but its MCLP score (-4.752) regresses below even the SFT-only baseline, with the model losing prosodic variation and over-articulating purely to satisfy ASR accuracy.
Novelty Assessment
MCLP is a genuinely novel evaluation instrument: rather than training a separate style classifier or relying on discrete emotion labels (the standard approach in prior RL-for-TTS-style-alignment work), it repurposes the intrinsic in-context-learning likelihood modeling already present in a pretrained LALM as a zero-additional-training, continuous, dense, and (per the information-theoretic argument) interpretable style-consistency signal. This is a meaningfully different design point from prior RL-for-TTS reward functions, which the paper explicitly identifies as limited to intelligibility/timbre metrics (CER, speaker similarity) or discrete emotion classification, neither of which captures the continuous, context-dependent nature of role-play style. The gated hybrid reward formulation is a considered response to a real, empirically demonstrated failure mode (reward hacking under either single-objective extreme), not a default design choice, and the paper’s ablation directly proves both hacking directions occur and that the gating resolves them. The RP-TTS dataset, built from real dramatic dialogue rather than synthetic role profiles (the dominant approach in prior Speech Role-Playing Agent benchmarks like SpeechRole and VoxRole), is a substantive, reusable artifact addressing the authenticity and scale limitations the paper identifies in prior role-play speech resources.
Field Significance
high — this paper resolves a genuine, previously identified bottleneck (the absence of an interpretable, trainable style-consistency signal for expressive TTS) by repurposing an LALM's own in-context likelihood modeling into a validated metric, then demonstrates via direct ablation that this signal, combined with an explicit content-fidelity gate, avoids the reward-hacking failure modes that either single-objective alternative (style-only or content-only) produces, a level of methodological rigor (both hacking directions independently demonstrated, not merely asserted) that is uncommon in RL-for-TTS work.
Claims
- supports: The in-context learning capability of a pretrained large audio language model can be repurposed into an interpretable, reference-based metric for speaking-style consistency, by measuring the model’s own likelihood of generating ground-truth audio when conditioned on a candidate utterance as stylistic context.
Evidence: MCLP scores correlate strongly with human MOS judgments in a pairwise win-rate analysis: when the MCLP difference between two utterances exceeds 0.1, the higher-MCLP utterance is preferred by human listeners in over 80% of comparisons, rising well above the chance-level 50% baseline as the MCLP gap increases. (§6.3, Figure 5)
- supports: Using a continuous, model-derived style-consistency signal as an RL reward for expressive TTS produces more effective stylistic alignment than relying on discrete emotion-classification-based reward proxies used in prior RL-for-TTS work.
Evidence: RL training with the MCLP-based reward improves style-consistency scores substantially over the SFT-only checkpoint (MCLP -4.725→-4.636 with audio history) while simultaneously reducing CER, and the resulting system achieves the highest MOS (3.646) among all evaluated systems, approaching ground-truth naturalness (4.411). (§6.2-6.3, Tables 2-3)
- contradicts: Optimizing a style-consistency reward alone, without an explicit content-fidelity constraint, causes severe reward hacking in RL-trained expressive TTS systems, producing outputs that maximize the style metric while catastrophically failing basic intelligibility, contrary to the assumption that a well-designed style reward alone is sufficient for expressive alignment.
Evidence: An ablation removing the CER-based content reward (style-only optimization) achieves the single highest theoretical MCLP score of any configuration (-4.590) but causes CER to skyrocket above 50%, with the model learning fixed, repetitive, linguistically meaningless acoustic patterns rather than genuine expressive speech. (§6.3, Table 3)
- complicates: Optimizing an expressive TTS system for content fidelity alone, without any explicit stylistic reward, produces a system that actively regresses in stylistic quality relative to a supervised-fine-tuning-only baseline, rather than merely failing to improve it further.
Evidence: An ablation removing the MCLP-based style reward (content-only optimization) achieves the lowest CER of any configuration (0.783%) but its MCLP score (-4.752) is worse than the SFT-only baseline (-4.725), with the model losing prosodic variation and over-articulating purely to maximize ASR accuracy. (§6.3, Table 3)
- supports: LALMs given only static role-play instructions, without access to prior turns’ audio, show measurably weaker and less consistent stylistic and content adherence than when given access to the conversation’s acoustic history, and RL training can substantially close this gap.
Evidence: Baseline LALMs (GPT-Audio in particular) suffer catastrophic intelligibility degradation without audio history (CER rising from 11.974% to 44.679%), while the proposed RL-trained method maintains low CER in both settings (1.130% vs. 1.625%) and achieves a higher MCLP score with audio history than without (+0.051), showing it successfully leverages historical acoustic cues that baseline systems fail to exploit consistently. (§6.2, Table 2)
Limitations and Open Questions
The subjective evaluation supporting the paper's headline MOS claims is comparatively small-scale: 32 professional annotators rated a curated set of only 31 samples (10 without audio history, 20 with history stratified across dialogue lengths), and after gold-standard quality-control filtering only 30 valid responses remained; the strength of the reported MOS gap (3.646 vs. baselines around 1.7-2.5) should be read against this limited sample size.
MCLP and the RP-TTS dataset are validated entirely on Mandarin Chinese drama data (WenetSpeech-derived, evaluated via CER and Pinyin WER), and the backbone LALM tested is limited to Step-Audio-2-mini-Base (7B); generalization of MCLP’s information-theoretic validity and the reward-gating strategy to other languages or LALM architectures is not empirically demonstrated. The gated reward’s tolerance threshold τ and CER penalty coefficient λ are set via a single empirically chosen configuration rather than a sensitivity analysis, leaving the reward formulation’s robustness to these hyperparameter choices unexamined.
Wiki Connections
- Instruction-Conditioned TTS — conditions expressive speech generation on structured natural-language scene descriptions and character profiles that must be adhered to consistently across multi-turn role-play dialogue.
- Emotion Synthesis — targets stylistically and emotionally expressive role-play speech generation, explicitly filtering RL training data to exclude neutral, non-expressive samples.
- RLHF Speech — applies GRPO reinforcement learning with a gated, composite reward (MCLP style score plus CER content penalty) to align an LALM-based TTS system with role-play style instructions, directly demonstrating reward-hacking failure modes under either single-objective ablation.
- Subjective Evaluation — validates the MCLP metric against genuine human MOS judgments from 32 professional annotators, and separately reports MOS comparisons between the proposed system and baselines.
- Evaluation Metrics — introduces Mean Continuation Log-Probability (MCLP), a novel interpretable metric quantifying speaking-style consistency by repurposing a pretrained LALM’s in-context learning capability.
- MiMo-Audio — used as a direct baseline LALM in the main comparison table, noted for strong context-aware speech continuation capabilities among the evaluated systems.
- RRPO — cited as closely related work on robust reward policy optimization for LLM-based emotional TTS, part of the broader RL-for-TTS-alignment landscape this paper’s MCLP reward extends.
- VoxRole — cited as a comparable Speech Role-Playing benchmark, part of the motivating context distinguishing this paper’s stylistic-consistency focus from prior semantic-role-alignment-focused agents.
- SpeechRole — cited as a comparable large-scale dataset and benchmark for evaluating speech role-playing agents, part of the prior-work landscape this paper’s dataset construction is positioned against.
- Speech-DRAME — cited as related work on human-aligned benchmark frameworks specifically for speech role-play evaluation.
- CosyVoice 2 — cited as an example of LLM-based controllable TTS achieving strong zero-shot timbre cloning, part of the background motivating the paper’s focus on the comparatively underexplored style-control problem.
- OV-InstructTTS — cited as related instruction-based TTS work from an overlapping author group, part of the open-ended natural-language style-description approaches this paper builds on.
- IndexTTS2 — cited as an example of emotionally expressive, duration-controlled TTS within the broader landscape of expressive speech generation systems.