arXiv · 2026 · Preprint

Denis Pavlov et al. (nineninesix.ai) · → Paper · Demo: ✓ · Code: ✓

GEPARD shows that a codec-token autoregressive TTS model can be served on stock vLLM with continuous batching, by keeping the backbone a plain full-attention transformer and moving voice cloning, text augmentation and classifier-free guidance out of the decode loop (into prefill or, via DPO, into the weights).

Problem

Voice agents need a low time-to-first-audio and cheap scaling across many concurrent streams. Most autoregressive codec TTS systems add components inside the decode loop: depth transformers over residual codebooks, cross-attention to the audio, or two-pass classifier-free guidance (CFG) at every frame. These components break the continuous batching and paged KV cache of standard LLM serving engines such as vLLM. A second problem is specific to decoder-only TTS: on 1-2 word inputs the model often fails to stop (“runaway”) or skips words, and full-benchmark WER hides this failure because public test sets contain almost no ultra-short prompts.

Method

GEPARD is a decoder-only transformer with no text output head. The backbone is derived from Qwen3.5 (14 blocks, hidden size 1024, 8 heads, ≈500M parameters). The authors removed the LinearBlock (linear-attention) layers so every block is full attention. Only the text embeddings and tokenizer are inherited from Qwen; the rest of the backbone is trained from scratch. The full model, with the audio interface and voice-cloning module, has 555.7M parameters. The input sequence is [speaker prefix (K=8) | text | audio].

Audio is tokenized with NVIDIA NanoCodec (22.05 kHz, 21.5 frames/s, 1.89 kbps), which uses GroupFSQ instead of residual vector quantization. Each of its 8 packed tokens is unfolded by mixed-radix decomposition into 4 FSQ channels, giving 32 channels per frame with alphabets of 8, 7, 6 and 6. The model predicts all 32 channels in parallel at each step with 32 small linear heads (216 logits in total), plus a separate binary stop head. The authors argue that GroupFSQ channels are nearly conditionally independent, so a depth transformer is unnecessary and one AR step per frame suffices. They did not run an ablation with and without a depth head (§3.2). Input frame embeddings are built from 32 lookup tables, a two-layer GELU MLP, an affine-free LayerNorm and a fixed scale matched to the text-embedding standard deviation. Section 3.1 documents the gradient-starvation and scale-drift problems that motivated this design.

The training loss is the sum of 32 cross-entropy terms plus a weighted BCE stop loss (weight 2, positive-class weight 25). Zero-shot voice cloning uses a 2-block Q-Former that compresses reference codec codes into 8 prefix tokens. These tokens are computed once at prefill. A learnable null prefix with 15% CFG dropout provides the unconditional path. Two regularizers train the compressor: a hinge-variance diversity loss and a supervised contrastive (SupCon) loss over speaker labels. The compressor is frozen after pretraining.

Pretraining uses ≈68.8k hours (27.6M clips, 19 sources). English makes up ≈85% of the mix, and smaller Spanish, Portuguese, Dutch and Kyrgyz portions are included. The run used 4 RTX 6000 GPUs and reached ≈7.09 of 8 planned epochs. Fine-tuning has two stages, both with LoRA on the backbone. The first is SFT with text repetition: short texts (under 13 tokens) are repeated in the prefill up to a 16-token budget, and only the final copy triggers audio. The second is length-normalized DPO (β = 2). Chosen trajectories for hard voices are generated offline with two-pass text CFG, so the DPO stage distills the CFG effect into single-pass weights. Pairs are ranked by an ASR (Whisper) reward with a two-sided length penalty, and the trajectory log-likelihood includes the stop head’s Bernoulli terms. Serving is single pass on stock vLLM, with the speaker prefix passed as prompt embeddings.

Key Results

Serving speed (Table 5, end-to-end with codec, SSE streaming, one 96 GB RTX PRO 6000). A single stream reaches TTFB 0.046 s and RTF 0.067 (≈15× faster than real time). Aggregate throughput rises to 116.1× at 32 streams, 168.3× at 128 and 203.9× at 256. Per-stream p50 RTF is 0.71 at 128 streams and 1.214 at 256, so real-time interactive serving holds up to about 128 streams per GPU. A throughput-oriented configuration reaches ≈224× at 256 streams, at the cost of higher TTFB. Increasing the codec decode batch gave no gain: the pipeline is limited by the LLM, not the codec (§6.2). Streams shorter than 2 s and network overhead are excluded (Appendix A.7).

Quality on Seed-TTS-eval EN (Table 6, 1088 paired prompts, reference cloning, GEPARD single-pass without CFG). The authors ran all seven systems themselves on identical inputs.

  • WER is 0.036, sixth of seven. VoxCPM2 has the lowest WER (0.015), and only Chatterbox is worse (0.063).
  • Speaker similarity (WavLM) is 0.585, the lowest in the cohort; the other systems score 0.789-0.867.
  • NISQA-MOS is 4.25, the highest in the cohort (next: Chatterbox 4.19, Fish-S2 and Qwen3-TTS 4.18).
  • GEPARD also has the highest NISQA NOI (4.16 vs. OmniVoice 4.14), COL (4.16 vs. Fish-S2 and Qwen3-TTS 4.14) and DIS (4.51 vs. Chatterbox 4.46).
  • UTMOS is 2.64, fourth of seven. Qwen3-TTS has the highest UTMOS (2.87).

Against non-cloning commercial and large-scale systems (Table 8), GEPARD trails ElevenLabs, Grok-TTS, Cartesia and Kokoro on WER and NISQA-MOS. It beats VibeVoice on WER (0.036 vs. 0.109) and NISQA-MOS (4.25 vs. 4.17). No human listening test is reported.

Short-phrase stability (§5). On a stress test of 8 short words × 4 voices × 50 runs, the baseline SFT model fails 91.5% of the time. Frame-level probes rule out three explanations: stop-head calibration (p_stop is effectively a delta function, with no frames between 0.1 and 0.5), a bad first frame, and speaker coverage. The remaining cause is that the 8-token speaker prefix outweighs 1-2 text tokens. Inference-time text CFG (w = 2.6) supports this: success on a hard held-out voice rises from 7.5% to 48.8%, and the overall average from 28.9% to 51.6% (Table 3). In the text-budget calibration, the failure rate falls from 18.8% to 5.0% between 12 and 13 text tokens (Figure 4). On single-word inputs × 5 difficult voices in single-pass mode, DPO raises the clean rate from 26.8% (baseline SFT) to 52.9%, 63.5% and 67.1% across rounds 1-3. Runaway plus timeout falls from ≈55% to ≈5%, and mean WER from 5.76 to 1.56. The round-3 clean rate exceeds the 64.8% that improved SFT reaches with CFG (§5.5). The released model is the round-2 checkpoint, because round 3 drifts on out-of-distribution prompts. The stress-test numbers are reported for round 3, not for the released model.

Novelty Assessment

The contribution is mainly engineering integration and a well-documented failure analysis. The parts are existing work: a Qwen-derived backbone, an off-the-shelf FSQ codec, a Q-Former speaker prefix, SupCon, text repetition (credited to VoiceStar) and length-normalized DPO. Parallel FSQ-channel prediction without a local transformer also has a precedent the authors cite (the decoder-less mode of Magpie-TTS). The new elements are system-level. The design explicitly targets unmodified vLLM serving. It distills two-pass text CFG into single-pass weights by using CFG rollouts as DPO positives. It also diagnoses short-input runaway as prefix dominance and supports that with probes. The serving measurements are concrete and reproducible in spirit.

The model-design evidence is weak. Two key decisions have no quantitative ablation: removing linear attention and dropping the depth head. Quality is measured with automatic metrics only. The stress-test cohorts are small, internal and not released as a benchmark. The report presents itself as v0.0.1 and does not claim state-of-the-art quality.

Field Significance

Moderate — GEPARD shows that a frame-parallel FSQ codec LM can run on an unmodified LLM serving engine with ≈200× aggregate real-time throughput on one GPU, if every non-standard mechanism is moved to prefill or into the weights. Its main reusable contribution is a diagnosis and mitigation of short-input runaway in decoder-only TTS, a failure that standard benchmarks such as Seed-TTS-eval do not measure.

Claims

  • supports: Codec channels from grouped finite scalar quantization can be predicted in parallel within one autoregressive step, removing the need for a depth transformer and keeping the TTS decoder compatible with standard LLM serving engines.

    Evidence: GEPARD predicts 32 unfolded NanoCodec FSQ channels per frame with independent heads, serves on stock vLLM, and reaches RTF 0.067 for one stream and 203.9× aggregate real time at 256 streams. A depth-head ablation was not run. (§2.3, §3.2, §6.2, Table 5)

  • complicates: Standard zero-shot TTS benchmarks under-test ultra-short inputs, so decoder-only TTS can look stable on public WER while failing on one- or two-word utterances.

    Evidence: Seed-TTS-eval has a minimum of 3 words (median 13 tokens), and GEPARD scores WER 0.036 there. On single-word stress tests the baseline SFT model fails 91.5% of the time, with median WER 2.0 from repeated words. (§5.1, §6.3, Figure 9)

  • refines: In decoder-only TTS with a speaker prefix, runaway generation on very short texts comes from weak text conditioning relative to the prefix, not from stop-predictor calibration, and boosting text weight is the effective lever.

    Evidence: Stop probabilities are binary, with no frames between 0.1 and 0.5, so threshold or temperature fixes cannot help. Onset temperature changes nothing (51% vs. 49% failure). Text CFG raises success on a hard voice from 7.5% to 48.8%. Text repetition drops failure from 18.8% to 5.0% between 12 and 13 text tokens. (§5.2.1, §5.2.2, §5.3, Table 3, §5.4.1, Figure 4)

  • supports: Preference optimization with guided-decoding rollouts as positive examples can distill the benefit of two-pass classifier-free guidance into a single-pass TTS model.

    Evidence: Length-normalized DPO, with CFG-generated chosen trajectories and stop-head Bernoulli terms in the log-likelihood, raises the single-pass clean rate on single-word inputs from 26.8% to 67.1% over three rounds. This exceeds the 64.8% that improved SFT reaches with CFG. (§5.4.2, §5.5)

  • complicates: A compact quantized speaker prefix trained only on same-speaker reference and target pairs gives clean signal quality but weak speaker similarity on unseen voices.

    Evidence: On Seed-TTS-eval with reference cloning, GEPARD has the highest NISQA-MOS in its cohort (4.25) but the lowest WavLM similarity (0.585 vs. 0.789-0.867 for the six baselines). The authors attribute this to representation leakage in the frozen Q-Former compressor. (§4.2, §6.3, Table 6, §7.1)

Limitations and Open Questions

Warning

Voice cloning is a proof of concept. Speaker similarity on Seed-TTS-eval is 0.585, the lowest of seven open-source cloning systems. Quality is measured only with automatic metrics, and there is no human listening test.

  • Two core design choices have no quantitative ablation: full attention instead of the hybrid Qwen3.5 layers (decided from informal listening), and parallel channel prediction without a depth head (§2.2, §3.2, §7.1).
  • Short-phrase instability is reduced but not solved. Section 7.1 reports a 55% failure rate for the hardest female voice. The authors suggest explicit length conditioning may be needed.
  • The stress-test results come from internal voice sets and prompts and are reported for DPO round 3. The released checkpoint is round 2, chosen because round 3 drifts on casual, URL and emotional prompts.
  • Pretraining is ≈85% English, and all quantitative evaluation is in English. Section 7.1 says the audio interface degrades outside English. Cross-lingual timbre transfer without accent leakage is described only qualitatively.
  • The paper mentions MOS changes after fine-tuning and DPO (+0.22 to +0.29, up to 4.94 on long texts) without saying how MOS was obtained, so these values are not recorded here.
  • Speed figures come from localhost runs with simultaneous request bursts. Streams under 2 s are excluded, and an earlier RTX 5090 run (RTF ≈ 0.040) was uncontrolled.

Wiki Connections

  • Autoregressive Codec TTS — GEPARD is a decoder-only codec LM that predicts all 32 FSQ channels of a frame in one step instead of using sequential or depth-transformer codebook decoding.
  • Streaming TTS — The system streams audio chunk by chunk on vLLM and reports TTFB, RTF and aggregate throughput from 1 to 256 concurrent streams.
  • Zero-Shot TTS — A Q-Former prefix from reference audio provides zero-shot cloning, evaluated on Seed-TTS-eval, where speaker similarity is the system’s main weakness.
  • RLHF Speech — Length-normalized DPO with an ASR reward and CFG-generated positives distills guidance into single-pass weights and reduces short-input runaway.
  • Seed-TTS — Seed-TTS-eval (English, 1088 prompts) is the public benchmark for all quality comparisons.
  • VALL-E — Cited as the sequential codebook-by-codebook design whose longer sequences GEPARD avoids by predicting whole FSQ frames.
  • VALL-E 2 — Cited as a depth-transformer-style design that GEPARD avoids because it conflicts with stock vLLM serving.
  • Finite Scalar Quantization — The FSQ scheme behind NanoCodec underpins the paper’s argument that codec channels are independent enough for parallel prediction.
  • VoxCPM2 — Baseline on Seed-TTS-eval with the lowest WER (0.015) and highest speaker similarity (0.867) in the cohort.
  • Fish Audio S2 — Baseline on Seed-TTS-eval with WER 0.016 and similarity 0.789, and NISQA-MOS 4.18 below GEPARD’s 4.25.
  • OmniVoice — Baseline on Seed-TTS-eval with WER 0.016 and similarity 0.848.
  • Qwen3-TTS — Baseline on Seed-TTS-eval with the highest UTMOS in the cohort (2.87); GEPARD’s backbone is also derived from the Qwen3.5 family.