arXiv · 2026 · Preprint
Junyang Chen et al. (Nankai University) · → Paper · Demo: ✓ · Code: ?
Adds a GRPO stage with editing-specific rewards (ASR content accuracy, DTW-aligned mel-cepstral distortion on unedited regions, speaker similarity) on top of supervised speech-editing adaptation of CosyVoice2, trained on target-speech-free prompts built from ordinary speech-text pairs, and shows that the resulting language model improves both speech editing and zero-shot TTS, including in languages absent from the English-only post-training data.
Problem
End-to-end speech editors adapted from zero-shot TTS models by supervised fine-tuning, such as the predecessor CosyEdit, learn from paired original and target recordings that must be constructed by splicing. These constructed targets carry boundary ambiguity and acoustic mismatch that the model learns to reproduce, and the token-level reconstruction loss does not distinguish edited from unedited regions, nor semantic correctness from acoustic preservation. The authors argue this produces a preservation-accuracy trade-off: on RealEdit, supervised adaptation of CosyVoice2 improves acoustic preservation over the zero-shot TTS backbone but raises WER from 4.14 to 5.83 (§4.3, Table 2). Reinforcement learning had been applied to TTS with WER and speaker-similarity rewards, and to a cascaded editor (ECPA), but not with rewards targeting editing preferences in an end-to-end editor.
Method
CosyEdit2 keeps the CosyVoice2 stack (CosyVoice 2): BPE text tokenizers, the supervised semantic speech tokenizer with finite scalar quantization, an autoregressive text-speech language model built on Qwen2.5-0.5B, a conditional flow-matching model, and a vocoder (§3.1, Appendix B). The language model input is reformulated for editing as [start, original text, target text, original speech tokens, turn-of-speech], after which the model generates target speech tokens. The flow-matching module uses CosyEdit’s GOT-CFM formulation, conditioning on the complete original speech tokens and mel spectrogram so that silence, background noise, and other unedited acoustic context are available as a global guide (Appendix B.4, C.2). The CosyVoice2 HiFT-GAN vocoder is replaced with a BigVGAN initialized from the closest public checkpoint (bigvgan_v2_22khz_80band_256x, 88.20% of generator parameters reused), reconfigured to 24 kHz with hop size 480, and trained for 460k steps on 625 hours of LibriTTS, LibriTTS-R, and in-the-wild YODAS2 speech to handle noisy and music-laden recordings (§3.2, Appendix C.3).

Training has two stages. Stage 1 follows CosyEdit: the language model (8 epochs) and flow module (9 epochs) are fine-tuned separately from the CosyVoice2 checkpoint on the 250-hour GigaEdit-S editing set, and BigVGAN is trained independently (§3.2, Appendix C). Stage 2 applies Group Relative Policy Optimization (GRPO) to the language model only, with flow and BigVGAN frozen and used to decode rollouts to waveforms (§3.3). Its prompts need no target recordings: 3,000 GigaSpeech-XL utterances are paired with target texts produced by rule-based perturbations (insertion and substitution via RoBERTa masked language modeling, deletion, word swap, and multi-edit combinations) using nlpaug, and WhisperX forced alignment marks the unedited regions (§4.1, Appendix D).
Each of G = 4 rollouts is scored by three rewards. A content reward exp(-k_w · WER^α) uses the ASR transcript against the target text; an acoustic preservation reward applies exp(-k_m · max(MCD - δ, 0)) to DTW-aligned mel-cepstral distortion over unedited regions; and a speaker reward is the raw cosine similarity of speaker embeddings between original and generated speech (§3.3, Appendix D.5). The content and preservation rewards are combined multiplicatively, so content acts as a gate and preservation modulates within it (γ = 0.5), and the result is summed with the speaker reward using weights scheduled from (0.9, 0.1) for the first 290 steps to (0.8, 0.2) for the last 90. The authors introduce the MCD term because WER and speaker-similarity rewards alone can be satisfied by unnatural word-by-word prosody (reward hacking). Training runs 380 steps on two H800 GPUs with KL coefficient 0.001 against the Stage 1 reference policy.
At inference, speech editing uses the GRPO language model with the Stage 1 flow and BigVGAN. Zero-shot TTS swaps in only the GRPO language model and keeps the original CosyVoice2 flow and HiFT-GAN, so differences from CosyVoice2 reflect the language-model policy alone (§4.1).
Key Results
Speech editing, Ming-Freeform-Audio-Edit English (Table 6, basic | full). CosyEdit2 has the lowest WER on substitution (1.43 | 1.52 vs. SSR-Speech 1.90 | 1.95) and the lowest DNSMOS mean absolute error to the original (MAE-DNSMOS, a consistency measure the authors add because high absolute DNSMOS can reflect implicit denoising) on all three edit types (insertion 0.107 | 0.108 vs. SSR-Speech 0.139 | 0.128; deletion 0.131 | 0.131 vs. 0.132 | 0.134; substitution 0.137 | 0.132 vs. 0.146 | 0.140). SSR-Speech, a cascaded editor that uses explicit alignment, remains ahead on several cells: insertion WER basic (1.75 vs. 1.90) and insertion SS (0.94 vs. 0.93), and deletion WER (5.22 | 5.29 vs. 5.52 | 5.83) and SS (0.91 vs. 0.90). Substitution SS ties at 0.89 | 0.90. Against the predecessor CosyEdit, WER drops on every edit type (e.g. insertion 2.83 to 1.90 basic) and SS rises on every edit type (basic split: insertion 0.86 to 0.93, deletion 0.83 to 0.90, substitution 0.83 to 0.89). VoiceCraft-X, LEMAS-Edit, Ming-UniAudio, and ECPA (numbers copied from its paper since the model is unreleased) trail on WER and SS throughout.
Chinese subset (Table 7, read from the table image). CosyEdit2 has the best WER and SS on all edit types against VoiceCraft-X, LEMAS-Edit, and Ming-UniAudio (e.g. substitution WER 1.26 | 1.39, SS 0.89 | 0.89) and the lowest MAE-DNSMOS on insertion and substitution; on deletion LEMAS-Edit has the lowest MAE-DNSMOS (0.159 | 0.173 vs. 0.166 | 0.177). SSR-Speech and CosyEdit are not evaluated here.
RealEdit (Table 8). CosyEdit2 has the lowest WER of all systems (4.31 vs. CosyEdit 4.50, SSR-Speech 5.05; ground truth 6.06) and the lowest unedited-region MCD among systems with MCD reported (3.93 vs. CosyEdit 4.94, Ming-UniAudio 5.36; cascaded systems have no MCD). SSR-Speech has higher speaker similarity (98.31 vs. 97.91) and a lower MOSNet MAE to ground truth (0.14 vs. 0.25, with VoiceCraft at 0.24).
Ablation on RealEdit (Table 2). Stage 1 supervised fine-tuning improves MCD over the CosyVoice2 zero-shot baseline (6.68 to 5.82) but worsens WER (4.14 to 5.83). GRPO on the language model recovers WER to 4.71 and further lowers MCD to 5.50. Adding the adapted flow brings MCD to 4.07 and MAE-DNSMOS from 0.210 to 0.134; BigVGAN in the full system gives the final 4.31 WER, 97.91 SS, 3.93 MCD, 0.131 MAE-DNSMOS. CosyVoice2 still has the lowest WER (4.14), which the authors attribute to cleaner studio-like output that is easier for ASR.
Zero-shot TTS, CV3-Eval (Tables 3 to 5; VAD trimming applied to all prompts). CosyEdit2 has the lowest error rate in every column: multilingual zh/en/ja/ko CER or WER of 3.52/4.87/6.16/5.14 vs. CosyVoice2 3.77/5.45/7.76/6.89; hard-zh CER 8.06 vs. 15.70 and hard-en WER 5.93 vs. 8.11; and all ten cross-lingual pairs (e.g. to-zh from en 7.16 vs. 13.15). On the hard subsets SS is highest (75.86 and 67.32 vs. CosyVoice2 74.96 and 67.22) and DNSMOS is 3.85 vs. 3.84 on hard-zh and tied at 3.97 on hard-en. The Stage 1 model without GRPO is better than CosyVoice2 on hard-zh CER (13.17 vs. 15.70) but worse on hard-en WER (11.99 vs. 8.11), and trails the GRPO model on both, so most of the gain arises in the GRPO stage (Table 4). SS is not reported for the multilingual or cross-lingual subsets.
Seed-TTS-eval (Table 9). CER 1.16 on test-zh and WER 1.95 on test-en, both lowest (CosyVoice2 1.36 and 3.10). SS is highest on test-zh by 0.1 (75.2 vs. 75.1), but on test-en CosyVoice2 (65.7) is above CosyEdit2 (64.3).
Subjective evaluation (Tables 12 to 14; 10 student raters, native Chinese speakers with CET-6 English, 30 samples per edit type per language). English editing: CosyEdit2 has the highest IMOS on all edit types and the highest or tied-highest SMOS (insertion ties SSR-Speech at 4.637), but SSR-Speech is slightly ahead on preservation MOS for insertion (4.247 vs. 4.213) and deletion (4.407 vs. 4.403). Chinese editing: highest IMOS, SMOS, and PMOS on all edit types. Zero-shot TTS: highest IMOS and SMOS in English (4.730/4.418 vs. CosyVoice2 4.658/4.342) and Chinese (4.640/4.512 vs. 4.535/4.432).
Other. BigVGAN beats HiFT-GAN on every copy-synthesis metric on VoiceBank-DEMAND clean and noisy (e.g. noisy MCD 1.630 vs. 1.988; Table 10). In identity editing (target text equals original), CosyEdit2 reaches SS 99.08 and MCD 3.07 vs. CosyVoice2 96.92 and 6.24, close to the HiFT-GAN copy-synthesis bound of 99.02 and 3.03 (Table 11).
Novelty Assessment
The architecture is almost entirely inherited: the CosyVoice2 backbone, the editing input format and GOT-CFM come from CosyEdit, and the vocoder swap is a pretrained BigVGAN retrained for a new mel configuration. Compared with CosyEdit, what is new is (1) a second post-training stage of GRPO on the language model; (2) the editing-oriented reward, specifically the gated multiplicative combination of an ASR content reward with a thresholded DTW-MCD reward restricted to unedited regions, which targets preservation directly rather than through token reconstruction; (3) the target-speech-free prompt construction that turns any transcribed corpus into editing RL data, with only 3,000 utterances used; and (4) the finding that this editing-oriented RL transfers to zero-shot TTS, including the hard and cross-lingual CV3-Eval subsets and languages not seen in post-training. The contribution is primarily a training recipe. The cross-task transfer result is the most distinctive element, but the paper has no control that applies standard TTS-style GRPO (WER and speaker rewards on plain TTS prompts) to CosyVoice2, so it does not show that the editing formulation, rather than GRPO with an ASR reward in general, causes the TTS gains.
Field Significance
Moderate. The paper provides direct evidence that reinforcement learning with editing-specific rewards can break the preservation-accuracy trade-off of supervised editing adaptation, and that the same policy update improves the parent model’s zero-shot TTS intelligibility, most strongly on hard and cross-lingual prompts. It also demonstrates a cheap route to editing RL data (no target recordings) and an acoustic-preservation reward designed to counter the prosody degradation that WER-only rewards can induce.
Claims
- supports: Reinforcement learning with rewards computed on decoded speech can overcome the trade-off between content accuracy and acoustic preservation that supervised fine-tuning on spliced editing pairs produces in end-to-end speech editors.
Evidence: On RealEdit, supervised adaptation of CosyVoice2 lowers MCD from 6.68 to 5.82 but raises WER from 4.14 to 5.83; adding editing-oriented GRPO reduces WER to 4.71 while lowering MCD further to 5.50 and raising SS from 97.05 to 97.23. (§4.3, Table 2)
- supports: Preference optimization of a speech-token language model on speech-editing prompts can transfer to zero-shot TTS, improving intelligibility on difficult and cross-lingual prompts, including languages absent from the post-training data.
Evidence: With only the language model replaced and CosyVoice2’s flow and vocoder kept, CER/WER drops from 15.70 to 8.06 on CV3-Eval hard-zh and from 8.11 to 5.93 on hard-en, falls in all ten cross-lingual pairs, and improves in Japanese and Korean, despite English-only GRPO prompts; the Stage 1 model alone degrades hard-en WER to 11.99. (§4.4, Tables 3-5)
- refines: For speech editing, rewards on intelligibility and speaker similarity are insufficient on their own; preservation of unedited regions needs an explicit reward because WER-oriented optimization can trade naturalness for recognizability.
Evidence: The authors add a thresholded DTW-aligned MCD reward over unedited regions, multiplicatively gated by the WER reward, citing reward hacking toward word-by-word prosody; this paper reports no ablation that removes the MCD term, so the hacking claim rests on the authors’ training observations. (§3.3, Appendix D.5)
- complicates: Absolute perceptual quality predictors and ASR error rates can favour regenerated studio-clean speech over faithful edits, so editing evaluation needs consistency measures against the original recording.
Evidence: The CosyVoice2 zero-shot baseline has the lowest WER (4.14) and highest DNSMOS (3.28) on RealEdit yet the worst MCD (6.68) and MAE-DNSMOS (0.275); CosyEdit2 has a lower WER than the ground-truth recordings (4.31 vs. 6.06), and the authors attribute its residual errors to preserved background noise rather than editing errors. (§4.3, Table 2; §4.2)
- complicates: End-to-end editors without explicit alignment still trail alignment-based cascaded editors on deletion, where localizing removed content is the main difficulty.
Evidence: On Ming-Freeform-Audio-Edit English deletion, SSR-Speech has lower WER (5.22 | 5.29 vs. 5.52 | 5.83) and higher SS (0.91 vs. 0.90) than CosyEdit2, while CosyEdit2 leads only on MAE-DNSMOS. (§4.2, Table 6)
Limitations and Open Questions
- The zero-shot TTS gains are attributed to the editing formulation, but there is no baseline that applies conventional TTS GRPO (WER and speaker rewards on ordinary TTS prompts) to the same CosyVoice2 checkpoint. Without it, the improvement could come largely from GRPO with an ASR reward rather than from editing-oriented training. The comparison is also against CosyVoice2 rather than a newer RL-post-trained CosyVoice release.
- Reward formulation and hyperparameters were tuned by iterative human listening during training (§Limitations). No ablation isolates the MCD reward, the speaker reward, or the weight schedule.
- Language coverage is bounded by CosyVoice2 (Chinese, English, Japanese, Korean), and all editing post-training is English (GigaEdit-S, GigaSpeech-XL). The cross-lingual gains are inherited-capability preservation plus transfer, not multilingual training (§Limitations, §4.5).
- Editing covers only content (insertion, deletion, substitution); emotion, pitch, and speaking-style edits are not explored (§Limitations).
- The CV3-Eval comparison applies VAD trimming to prompts for all systems because CosyEdit2 otherwise inherits leading or trailing silence and noise as style cues, which points to a behavioural mismatch with standard zero-shot TTS usage. SS is not reported for the CV3-Eval multilingual and cross-lingual subsets.
- Subjective tests use 10 raters who are all native Chinese speakers, including for English samples, with 30 samples per condition. ECPA results are copied from its paper rather than rerun.
- Code release is not stated; only a demo page is provided.
Wiki Connections
- Flow Matching — the GOT-CFM flow-matching decoder, fine-tuned in Stage 1 and frozen during GRPO, accounts for the largest single MCD reduction in the ablation (5.50 to 4.07).
- GAN Vocoder — replaces CosyVoice2’s HiFT-GAN with a BigVGAN retrained on clean and in-the-wild speech for a 24 kHz CosyVoice2 mel configuration, with copy-synthesis comparisons on clean and noisy VoiceBank-DEMAND.
- Zero-Shot TTS — evaluates the GRPO-updated language model for zero-shot voice cloning on CV3-Eval and Seed-TTS-eval, reporting lower error rates than the CosyVoice2 backbone across monolingual, hard, and cross-lingual subsets.
- RLHF Speech — applies GRPO to the speech-token language model with a gated combination of ASR content, unedited-region MCD, and speaker-similarity rewards on target-speech-free editing prompts.
- Subjective Evaluation — reports human IMOS, SMOS, and a preservation MOS (PMOS) for editing in English and Chinese, plus IMOS and SMOS for zero-shot TTS.
- CosyEdit — direct predecessor by the same group; CosyEdit2 reuses its editing input format, GOT-CFM, and GigaEdit-S supervised stage, and adds the GRPO stage, the BigVGAN vocoder, and zero-shot TTS evaluation, outperforming it on editing WER, SS, and MCD.
- CosyVoice 2 — the backbone whose language model, tokenizer, and flow are adapted, and the same-backbone baseline for both editing and zero-shot TTS.
- VoiceCraft — source of the RealEdit benchmark used for the ablation and an editing baseline.
- VoiceCraft-X — multilingual editing and zero-shot TTS baseline that trails CosyEdit2 on all reported editing and TTS tables.
- Ming-UniAudio — large end-to-end speech LM editor and source of the Ming-Freeform-Audio-Edit benchmark, used as a baseline throughout.
- LEMAS — multilingual editing baseline (LEMAS-Edit) in the extended English and Chinese editing comparisons.
- CosyVoice 3 — source of the CV3-Eval zero-shot TTS benchmark used for the main TTS results.
- Multi-Reward GRPO — cited as prior multi-reward GRPO for TTS language models, which CosyEdit2 extends with an editing-specific preservation reward.