arXiv · 2024 · Preprint
Qwen Team et al. (Alibaba) · → Paper · Demo: ? · Code: ✓
Qwen2.5 introduces a series of open-weight dense LLMs (0.5B to 72B) and MoE variants, pre-trained on 18 trillion tokens with multi-stage reinforcement learning post-training, achieving competitive performance with models five times larger.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
Qwen2.5 provides a family of decoder-only transformer LLMs used by TTS and SCA systems as the language backbone for text understanding, instruction following, and token generation. The model family spans 0.5B to 72B parameters, enabling deployment across resource-constrained edge devices and high-capacity server inference. TTS and SCA systems adopt Qwen2.5 primarily as the language model component responsible for processing natural language instructions and generating intermediate text or semantic token sequences before the acoustic stage. The post-training pipeline, which uses supervised fine-tuning on over one million examples combined with offline DPO and online GRPO, produces models with strong instruction-following and long-context capabilities that speech generation pipelines depend on for controllable and coherent output.