arXiv · 2026 · Preprint

Jaehoon Kang et al. (Sungkyunkwan University) · → Paper · Demo: ? · Code: ?

GLASS trains one GRPO-optimized LoRA adapter per acoustic style direction (fast, slow, high pitch, low pitch) on a frozen zero-shot AR TTS backbone, using automatic post-generation rewards instead of style labels, and shows that the independently trained adapters can be swapped, interpolated, and summed at inference while the speaker prompt stays fixed.

Problem

In prompt-based zero-shot TTS, the speaker prompt fixes both voice identity and prosody: changing the speaking rate or pitch of a cloned voice usually requires another recording of the same speaker in the target style, and prompt selection gives example-based control only, with no continuous dial and no way to combine axes. Label- or description-conditioned controllable TTS (for example Parler-TTS natural-language guidance or the authors’ earlier fine-grained style control in prompt-based TTS) needs annotated data for every new attribute, and signal-processing post-hoc edits (time-stretching, pitch-shifting) damage naturalness and speaker similarity. Prior RL work on TTS (Seed-TTS, F5R-TTS, multi-reward GRPO, DMOSpeech 2) optimizes the whole generator for quality, not modular style controls.

Method

The backbone is CosyVoice 2 (CosyVoice2-0.5B), treated as an autoregressive speech-token policy conditioned on text and a speaker prompt. The whole backbone is frozen: LoRA (rank 16, scaling 32, dropout 0.05) is attached only to the q/v attention projections of the Qwen2 autoregressive token model, giving 1.08M trainable parameters (0.22% of the 495M-parameter AR backbone). Speech embeddings, the token decoder, the flow-matching acoustic model and the vocoder are untouched (§4.1, §Appendix A).

Each style direction k gets its own adapter trained with GRPO (DeepSeekMath, DeepSeek-R1). For each text and speaker prompt, G = 8 completions are sampled; rewards are normalized within the group into advantages, and the objective combines a PPO-clipped policy-gradient term with a token-level KL-style penalty against the frozen reference, implemented by disabling the LoRA layers (§3.2). The reward is an equal-weight (η = 0.5) mix of a WER anchor, R_WER = 1 − tanh(γ·WER) computed from Whisper-large-v3 transcripts (Whisper), following No Verifiable Reward for Prosody, and a style term: the generated speech-token length (speaking rate) or utterance-level voiced-frame mean F0 from pyworld (pitch), min-max normalized within the group and sign-flipped for the opposite direction (§3.3, §Appendix A). Training uses prompts from 50 LibriTTS-R train-clean-100 speakers (25 male, 25 female), one speaker held fixed per group, for 500 to 750 update steps per adapter (§Appendix A).

At inference the adapters are combined by linear LoRA arithmetic, ΔW = Σ w_k ΔW_k, giving same-axis interpolation (α·fast + (1 − α)·slow) and cross-axis composition (for example 0.5·fast + 0.5·high). The paper adopts w = 0.5 per adapter as the default composition point because full-strength sums overshoot (§3.4, §Appendix C).

Key Results

All evaluation is zero-shot transfer to unseen Common Voice speakers on Seed-TTS-eval test_en (N = 1,088), out of domain relative to the LibriTTS-R training prompts. Baselines are unmodified CosyVoice2-0.5B and DSP transforms of its output (librosa time-stretch at rates 1.5 and 0.6; ±4 semitone pitch shift) (§4.1, §Appendix A).

Single-axis control (Table 1). Target shifts are comparable to DSP: Fast LoRA reaches 5.59 SPS (DSP 5.48, baseline 3.65), Slow LoRA 2.30 SPS (DSP 2.19, so DSP slows more), High-pitch LoRA 156.1/241.0 Hz male/female mean F0 (DSP 150.9/239.9), Low-pitch LoRA 108.9/164.6 Hz (DSP 98.0/155.4, so DSP lowers pitch further). Off-axis statistics stay close to the baseline (for example Fast LoRA F0 120.4/191.0 vs. 120.4/192.2). The quality gap to DSP is large on speaker similarity and naturalness: LoRA SpkSim is 0.609 to 0.650 against 0.158 to 0.500 for DSP, and UTMOSv2 is 3.05 to 3.37 against 1.45 to 1.57. Human ratings (15 raters, 25 utterances per transformed system, five ratings per utterance) give LoRA S-MOS 4.12 to 4.72 and N-MOS 4.24 to 4.84 against DSP 1.40 to 3.08 and 2.00 to 2.76 (§4.2, §Appendix A, Table 1).

WER is the exception. Every adapter raises WER above the 2.81 baseline (3.01 to 3.49), and DSP is lower than the matching LoRA on two of four directions: slow-down 2.67 vs. 3.18 and pitch-up 2.59 vs. 3.01. LoRA is lower on pitch-down (3.11 vs. 3.28), and speed-up is essentially tied (3.49 vs. 3.50). Relative to the unmodified model, SpkSim drops by up to 0.046 (High-pitch LoRA) and UTMOSv2 moves by at most 0.23 (Slow LoRA) (Table 1).

Interpolation (Table 2, Figure 2a,b). On a gender-balanced 200-utterance subset, blending opposite adapters over α ∈ {0, 0.25, 0.5, 0.75, 1} moves SPS from 2.30 to 5.52 and male/female F0 from 107/166 Hz to 155/239 Hz. Per-utterance Spearman correlation between α and the target statistic averages 0.954 (speed), 0.874 (male pitch) and 0.923 (female pitch). WER is U-shaped, with the lowest value at α = 0.5 on both axes (2.16% speed, 2.14% pitch) and the highest at the slow endpoint (4.50%) (§4.3, §Appendix B, Table 2).

Composition (Tables 3 and 4, Figure 2c). At w = 0.5 per adapter, all four speed-pitch combinations move toward the intended quadrant and each axis retains 80% to 121% of its single-axis shift. At full strength (w = 1.0 each), pitch overshoots badly when paired with high (male F0 318.9 Hz, female 385.5 Hz for fast ⊕ high), and pairings with low collapse the voicing ratio to 0.32 to 0.37 (baseline 0.65), which the authors read as creaky or aperiodic phonation (§4.4, §Appendix C). Quality metrics (WER, SpkSim, UTMOS) for the composed adapters are not reported.

Novelty Assessment

The ingredients are all established: GRPO with automatic rewards for TTS (prior GRPO TTS work cited in §2.2), a WER reward form taken from earlier work, LoRA on a frozen backbone, and LoRA or task-vector arithmetic, which is well established in text-to-image diffusion. The contribution is putting these together for speech: each prosodic direction is defined only by a scalar reward computed after generation, without style labels or reference examples, and the resulting adapters are shown to compose. Training a separate adapter per signed direction, rather than one conditioned controller, is what makes the arithmetic possible. This is engineering integration plus a useful empirical demonstration, not a new architecture.

The comparison set is the paper’s weakest point. The only baselines are the unmodified model and DSP post-processing, which is a weak competitor for speaker similarity and naturalness. There is no comparison with prompt-based style control, label- or description-conditioned controllable TTS, or a single fine-tuned model conditioned on a rate or pitch value. There is also no ablation of the WER anchor weight η, the KL coefficient, or a plain supervised fine-tuning alternative. The two axes studied (rate and pitch) are also the two attributes for which simple automatic rewards already exist.

Field Significance

Moderate. The paper demonstrates that reward-defined, label-free style directions can be learned as small LoRA adapters on a frozen AR TTS model and recombined by weight arithmetic with roughly additive effects at half strength. This gives a concrete template for modular prosody control in prompt-based zero-shot TTS. The evidence is limited to two easily measured acoustic axes and DSP baselines, so it is an early demonstration rather than a validated alternative to conditioned controllable TTS.

Claims

  • supports: Group-relative policy optimization with automatic, non-differentiable acoustic rewards can teach a frozen zero-shot TTS model targeted prosodic shifts without any style-labelled training data.

    Evidence: LoRA adapters trained with token-length or mean-F0 rewards plus a WER anchor shift speaking rate from 3.65 to 5.59 or 2.30 SPS and mean F0 by up to +35.7 Hz (male) on unseen Seed-TTS-eval speakers, matching or approaching DSP-level target shifts. (§3.3, §4.2, Table 1)

  • supports: Model-side prosody manipulation preserves speaker identity and naturalness far better than signal-processing transforms of the synthesized waveform at comparable target shifts.

    Evidence: At similar SPS/F0 targets, LoRA SpkSim is 0.609 to 0.650 vs. 0.158 to 0.500 for DSP, UTMOSv2 is 3.05 to 3.37 vs. 1.45 to 1.57, and human S-MOS/N-MOS are 4.12 to 4.72 / 4.24 to 4.84 vs. 1.40 to 3.08 / 2.00 to 2.76. (§4.2, Table 1)

  • refines: Low-rank weight updates learned for separate prosodic directions combine approximately additively in weight space, but only when scaled down; summing them at full strength overshoots or degrades phonation.

    Evidence: At w = 0.5 each, speed and pitch adapters retain 80% to 121% of their single-axis effects; at w = 1.0 each, fast ⊕ high reaches 318.9 Hz male mean F0 (about 5.6× the single-axis shift) and pairings with low drop voicing ratio to 0.32 to 0.37 from 0.65. (§Appendix C, Tables 3 and 4)

  • complicates: Reward-driven style steering of zero-shot TTS carries an intelligibility cost even when an ASR-based reward anchor is included.

    Evidence: All four single-direction adapters raise WER above the 2.81 baseline (3.01 to 3.49), DSP gives lower WER than the matching adapter for slow-down and pitch-up, and interpolation endpoints reach 4.50% WER (slow) while the α = 0.5 blend reaches 2.16%. (§4.2, §4.3, Tables 1 and 2)

Limitations and Open Questions

Warning

The only comparisons are DSP post-processing and the unmodified backbone, and the unmodified backbone was not included in the human listening test. The claim that adapters preserve naturalness and speaker similarity relative to the original model therefore rests on automatic metrics (UTMOSv2, WavLM SpkSim), and no conditioned or prompt-based controllable TTS baseline is tested.

  • Only two acoustic axes are studied, speaking rate and pitch, and both have simple automatic proxies. The authors note that emotion, effort, accent or conversational style would need reliable automatic rewards or human preference feedback (§Limitations).
  • Rewards and most evaluations rely on proxies (token length, pyworld mean F0, Whisper WER, automatic similarity and naturalness), which may not capture perceptual style quality (§Limitations). Speech-token length as a rate reward also does not separate faster articulation from shorter pauses or dropped content.
  • The human test is small: 25 utterances per system, five ratings each.
  • WER, SpkSim and naturalness are not reported for multi-axis compositions, and the w = 0.5 operating point is chosen by inspection; the paper leaves robust adapter-weight selection to future work (§5, §Appendix C).
  • There are no ablations of η, β, LoRA rank, or a supervised alternative, and results are shown for a single backbone (CosyVoice2-0.5B) and English only.
  • Code and demo availability are not stated.

Wiki Connections

  • RLHF Speech — Applies GRPO with WER and acoustic-statistic rewards to train modular LoRA style adapters rather than to improve the generator as a whole.
  • Prosody Control — Provides explicit, continuously interpolable control of speaking rate and mean pitch, plus two-axis composition, without style labels.
  • Zero-Shot TTS — Adapters trained on LibriTTS-R speaker prompts transfer to unseen Seed-TTS-eval speakers while the prompt voice is kept fixed.
  • Autoregressive Codec TTS — Steers a frozen discrete-speech-token AR backbone (the CosyVoice 2 Qwen2 token model) through LoRA updates to its attention projections.
  • Subjective Evaluation — Uses 15 human raters for S-MOS and N-MOS on the LoRA and DSP-transformed systems; the unmodified baseline is not rated.
  • CosyVoice 2 — Frozen backbone and unmodified baseline; all adapters attach to its autoregressive token model.
  • Seed-TTS — Source of the Seed-TTS-eval test_en benchmark and SpkSim protocol, and an earlier example of WER-reward RL for TTS.
  • DeepSeekMath — Origin of the GRPO objective used to train each adapter.
  • No Verifiable Reward for Prosody — GLASS adopts its tanh-shaped WER reward as the intelligibility anchor.
  • Multi-Reward GRPO for TTS LLMs — Prior GRPO-based TTS work with prosody-related rewards that optimizes the full policy rather than modular adapters.
  • F5R-TTS — Prior GRPO application to TTS cited as the generator-level counterpart to GLASS’s adapter-level control.
  • Fine-grained style control in prompt-based TTS — The same group’s earlier style-control work, cited as an annotation- or reference-based alternative.
  • Spark-TTS — Source of the syllables-per-second speaking-rate metric used in all evaluations.