arXiv · 2025 · Preprint
Xu et al. (Qwen Team) · → Paper · Demo: ? · Code: ✓
Qwen2.5-Omni is an end-to-end omni-model that simultaneously perceives text, images, audio, and video while generating both text and streaming natural speech via a Thinker-Talker architecture, where a dual-track autoregressive Talker directly consumes the Thinker LLM’s hidden representations to produce audio tokens decoded by a sliding-window flow-matching DiT vocoder.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
Qwen2.5-Omni is cited by TTS and SCA papers primarily as a reference point for end-to-end speech generation from a general-purpose multimodal backbone. The Thinker-Talker architecture separates semantic reasoning (Thinker, a transformer decoder initialized from Qwen2.5) from speech production (Talker, a dual-track autoregressive model), allowing the system to generate text and speech simultaneously without interference between the two output streams. The Talker uses an internal codec (qwen-tts-tokenizer) and a sliding-window DiT with flow matching for streaming waveform generation, removing the need for word-level text-speech alignment during training. On the SEED-TTS evaluation benchmark, the RL-optimised Qwen2.5-Omni-7B achieves WER of 1.42%/2.33%/6.54% on test-zh/test-en/test-hard, competitive with dedicated zero-shot TTS systems such as MaskGCT and CosyVoice 2. The paper is a reference for the emerging paradigm of unified omni-models that subsume TTS as a capability rather than treating it as an isolated task.
Wiki Connections
Related concepts: autoregressive-codec-tts · streaming-tts · spoken-language-model · neural-codec
In-corpus papers cited by this work: 2406.02430 (Seed-TTS) · 2412.10117 (CosyVoice 2)
Cited by: 2504.18425 (Kimi-Audio, Qwen2.5-Omni as reference omni-model architecture)
2507.16632 — Step-Audio 2 Technical Report