arXiv · 2026 · Preprint

Junyi Wang et al. (Tsinghua University / Huawei Technologies Co., Ltd) · → Paper · Demo: ✓ · Code: ?

ClariCodec turns a neural speech codec’s finite scalar quantiser into a stochastic policy so that the encoder can be fine-tuned with GRPO against a non-differentiable ASR word-error-rate reward, cutting LibriSpeech WER at 300 bps by roughly 23% relative while leaving predicted quality essentially unchanged.

Problem

Satellite and underwater links can offer only a few hundred bits per second, so speech coding there must prioritise intelligibility over waveform fidelity. Neural speech codecs are trained with reconstruction objectives (mel L1, adversarial and feature-matching losses) that reward acoustic similarity. The authors argue that at around 300 bps these objectives spend scarce bits on perceptual detail rather than on the linguistic content needed for intelligibility. Intelligibility itself is usually measured by WER, which is discrete and non-differentiable and therefore cannot be optimised directly with gradient-based reconstruction training.

Method

ClariCodec works in the log-mel domain (10 ms hop, 16 kHz audio). A ConvNeXt V2 encoder downsamples by 8x in time through three 2x blocks, each pairing a learnable convolutional branch with an average-pooling shortcut, producing a 12.5 Hz latent. A residual finite scalar quantiser (R-FSQ) with two layers, each using levels [8, 8, 8, 8] (12 bits per layer), gives 24 bits per frame and therefore 300 bps. Quantisation uses improved FSQ (iFSQ), which replaces the tanh bound with a sigmoid to improve codebook utilisation. A mirrored ConvNeXt V2 decoder reconstructs the mel spectrogram, and a Vocos vocoder trained from scratch together with the codec produces the waveform. The full system has 301M parameters.

The key design choice is stochastic quantisation (the authors’ “stochastic FSQ”): rather than rounding each latent dimension to the nearest grid level, negative squared distances to the grid points are treated as logits and a level is sampled with Gumbel-Softmax at temperature τ. The encoder plus quantiser then defines a policy over discrete codec tokens.

Training has two stages. Stage 1 trains the whole codec end to end for 500k steps on 50k hours of Libriheavy, using a mel L1 reconstruction loss, a hinge GAN loss over multi-period, multi-resolution and multi-scale discriminators following the Vocos recipe, and a feature-matching loss. Stage 2 (100k steps) freezes the quantiser, decoder and vocoder, so the mapping from tokens to waveform stays fixed, and fine-tunes only the encoder with GRPO. For each input, a group of 16 token sequences is sampled from the stochastic quantiser and decoded. A pretrained 1.1B-parameter FastConformer TDT-CTC ASR model transcribes both the reconstruction and the original audio, and the reward is the negative WER between the two transcripts. Advantages are normalised within each group. The authors say KL regularisation against a reference policy does not suit the unidirectional encoder, so they anchor the policy with a mel reconstruction loss instead (λ_RL = 10, λ_mel = 1).

Figure 1: Overview of the two-stage training framework of ClariCodec. In Stage 1, the full codec is trained end-to-end using a combination of L 1 mel reconstruction loss, adversarial loss, and feature matching loss to ensure high-fidelity speech reconstruction. In Stage 2, all modules excpet the encoder are frozen, and the encoder is fine-tuned using an RL objective where the reward signal is derived from a pretrained ASR model, explicitly optimizing for speech intelligibility. An L 1 mel reconstruction loss is used for preventing perceptual degradation during RL optimization.

Key Results

All results are on LibriSpeech test-clean and test-other, and every baseline uses its official checkpoint (Table 1). Evaluation WER uses a different ASR model (NeMo Conformer-Transducer XL) from the one that supplies the reward, and SIM uses a WavLM-based speaker verifier.

  • Effect of RL. Stage 2 lowers test-clean WER from 4.64% to 3.55% (23.5% relative) and test-other WER from 13.3% to 10.4% (21.8% relative). UTMOS rises slightly (4.12 to 4.16 on test-clean, 3.67 to 3.73 on test-other). STOI (0.87 / 0.84) and SIM (0.50 / 0.51) do not change, and test-clean PESQ falls marginally from 1.88 to 1.87.
  • Against baselines on intelligibility. At 300 bps, the lowest bitrate in the table, ClariCodec’s test-clean WER of 3.55% beats StableCodec-400 (4.88%), StableCodec-700 (3.91%), SoCodec at 466 bps (5.59%), WavTokenizer at 480 bps (7.38%), EnCodec at 750 bps (16.1%) and SemantiCodec at 312.5 bps (22.7%). It does not beat the two ASR-supervised semantic codecs: SAC at 525 bps reaches 2.00% and FlexiCodec at 640 bps reaches 2.57%. On test-other the gap is wider (SAC 4.15%, FlexiCodec 4.69%, ClariCodec 10.4%, with SoCodec close at 10.6%).
  • Against baselines on acoustic quality. ClariCodec trails the higher-bitrate systems on PESQ (1.87 vs. 2.20 FlexiCodec, 2.16 SAC, 1.92 StableCodec) and SIM (0.50 vs. 0.78 SAC, 0.71 FlexiCodec, 0.58 StableCodec-700). Its UTMOS of 4.16 is below StableCodec (4.31) and SAC (4.27) and level with FlexiCodec (4.15).
  • Stage 2 loss ablation (Table 2, test-clean). RL alone gives 3.54% WER but lowers PESQ to 1.83. Adding the mel anchor recovers PESQ to 1.87 with 3.55% WER. STOI, UTMOS and SIM barely move across the three configurations.

The comparison is not matched on bitrate, since every baseline runs at a higher bitrate than ClariCodec, and training data size (7.2k to 105k hours) and model size (15M to 950M) differ widely. EnCodec is run with only its first RVQ level to reach 750 bps, which handicaps it heavily.

Novelty Assessment

The authors describe this as the first application of RL to neural speech codec training. The genuinely new piece is the policy formulation: Gumbel-Softmax sampling over FSQ grid levels turns a deterministic quantiser into a sampling distribution, so GRPO with a sequence-level WER reward can update the encoder. The backbone is assembled from known parts (ConvNeXt V2 blocks, iFSQ, a Vocos vocoder and discriminators, GRPO from DeepSeekMath), so the architecture itself is incremental. The contribution is mainly a training-procedure change, and freezing the decoder while optimising only the encoder is a simple way to keep acoustic quality stable. The Stage 1 codec already reaches 4.64% WER at 300 bps without RL, so much of the headline competitiveness comes from the base design rather than from the RL stage.

Field Significance

Moderate. The paper shows that a codec’s quantiser can be reframed as a policy and tuned against a non-differentiable task metric with GRPO, giving a consistent intelligibility gain at a fixed bitrate without retraining the decoder. This gives the field a concrete recipe for task-reward fine-tuning of codec encoders. However, the evidence is limited to one English benchmark, objective metrics and a single reward type, and ASR-supervised semantic codecs at somewhat higher bitrates still reach lower WER.

Claims

  • supports: Reinforcement learning with an ASR-derived WER reward can improve the intelligibility of a neural speech codec at a fixed bitrate, beyond what reconstruction-loss training achieves.

    Evidence: GRPO fine-tuning of the encoder lowers LibriSpeech WER from 4.64% to 3.55% on test-clean and from 13.3% to 10.4% on test-other at 300 bps, with STOI and SIM unchanged and UTMOS slightly higher. (§4.1, Table 1)

  • supports: Making codec quantisation stochastic lets policy-gradient methods optimise non-differentiable objectives through a discrete bottleneck, while the token-to-waveform decoder stays frozen.

    Evidence: Gumbel-Softmax sampling over FSQ grid levels, using negative squared distances as logits, makes the encoder a policy; only the encoder is updated with group-normalised GRPO advantages while the quantiser, decoder and vocoder stay frozen. (§2.1.2, §2.3)

  • complicates: Optimising codecs purely for intelligibility trades away some acoustic fidelity under extreme compression, and a reconstruction anchor reduces but does not remove the loss.

    Evidence: RL loss alone lowers PESQ from 1.88 to 1.83; adding a mel reconstruction loss brings it back to 1.87 with almost the same WER (3.55% vs. 3.54%), still below the Stage 1 level. (§4.2, Table 2)

  • complicates: At very low bitrates, intelligibility-oriented training narrows but does not close the gap to codecs that use explicit ASR or semantic supervision at moderately higher bitrates, especially on harder speech.

    Evidence: At 300 bps ClariCodec reaches 3.55% / 10.4% WER on test-clean / test-other, against 2.00% / 4.15% for SAC at 525 bps and 2.57% / 4.69% for FlexiCodec at 640 bps; it also trails both on PESQ and SIM. (§4.1, Table 1)

Limitations and Open Questions

  • Objective metrics only. No human listening test is reported, so the claim that quality is preserved rests on UTMOS, PESQ, STOI and SIM. Intelligibility is measured only by ASR WER, not by human transcription.
  • Reward and evaluation from the same model family. Both the reward ASR (FastConformer TDT-CTC) and the evaluation ASR (Conformer-Transducer) are NVIDIA NeMo English models. The gains may partly reflect ASR-specific optimisation rather than human-perceived intelligibility; no other ASR or human test checks this.
  • Narrow evaluation scope. Results cover English read speech only (Libriheavy training, LibriSpeech evaluation). The satellite and underwater use case motivates the work, but channel errors, packet loss and noisy input are never tested.
  • Not streaming. The authors acknowledge the architecture is non-causal and leave a low-latency streaming version to future work (§5).
  • Downstream use untested. Effects on codec-based TTS or speech LMs are not evaluated, even though the authors name these as future directions (§5).
  • Single reward. Only a WER reward is used. Adding acoustic-quality rewards is left open, and the small PESQ and SIM values point to speaker fidelity as the main weakness at this bitrate.
  • No variance reporting. No confidence intervals or multiple seeds are given, and some ablation differences (PESQ 1.83 vs. 1.87) are small.

Wiki Connections

  • Neural Audio Codec — Proposes a 300 bps mel-domain residual-FSQ speech codec and adds a task-reward fine-tuning stage on top of standard reconstruction and adversarial codec training.
  • RLHF Speech — Applies GRPO with an AI-feedback reward (negative ASR WER) to the encoder of a speech-generating codec, treating stochastic quantisation as the policy.
  • DeepSeekMath — Supplies GRPO, the group-relative policy optimisation algorithm used for the Stage 2 encoder fine-tuning.
  • Finite Scalar Quantization — Provides the FSQ quantiser that ClariCodec stacks residually and turns into a stochastic policy.
  • Vocos — Provides the vocoder, trained from scratch with the codec, and the discriminator recipe used in Stage 1.
  • SAC — Baseline ASR-supervised dual-stream codec at 525 bps that beats ClariCodec on WER, PESQ and SIM, at 1.75x its bitrate.
  • FlexiCodec — Baseline low-frame-rate codec at 640 bps that beats ClariCodec on WER, PESQ and SIM at more than twice the bitrate.
  • StableCodec — Baseline at 400 and 700 bps that ClariCodec outperforms on test-clean WER while trailing it on PESQ, UTMOS and SIM.
  • WavTokenizer — Baseline single-codebook codec at 480 bps that ClariCodec beats on both WER and quality metrics.
  • EnCodec — Baseline run with a single RVQ level at 750 bps, which performs far worse than ClariCodec at that truncated setting.