arXiv · 2025 · Preprint
KimiTeam et al. (Moonshot AI) · → Paper · Demo: ? · Code: ✓
Kimi-Audio is an open-source audio foundation model that unifies speech recognition, audio understanding, generation, and real-time speech conversation in a single LLM-based architecture trained on 13 million hours of audio.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
Kimi-Audio introduces a hybrid audio representation strategy combining 12.5 Hz discrete semantic tokens (derived from a supervised ASR-based tokenizer) with continuous Whisper acoustic features, feeding both into an LLM initialized from Qwen2.5-7B. The architecture uses parallel text and audio generation heads sharing lower transformer layers, enabling simultaneous text and audio token prediction. A chunk-wise streaming detokenizer based on flow matching with a look-ahead mechanism converts semantic tokens to mel-spectrograms at low latency, before a BigVGAN vocoder produces final waveforms. TTS and SCA researchers reference this model as a design point for universal audio-language architectures that jointly support understanding and generation tasks, including zero-shot TTS (Kimi-TTS subsystem) and voice conversion (Kimi-VC subsystem) built on the same backbone.
Wiki Connections
Related concepts: spoken-language-model · speech-to-speech · neural-codec · flow-matching
In-corpus papers this paper builds on: 2410.00037 (Moshi) · 2503.20215 (Qwen2.5-Omni) · 2407.21783 (LLaMA 3) · 2301.02111 (VALL-E) · 2206.04658 (BigVGAN) · 2206.08317 (Paraformer) · 2310.13289 (SALMONN) · 2310.00704 (UniAudio) · 2409.06666 (LLaMA-Omni) · 2411.00774 (Freeze-Omni) · 2412.02612 (GLM-4-Voice) · 2408.16725 (Mini-Omni) · 2410.17799 (OmniFlatten) · 2502.04128 (Llasa) · 2407.10759 (Qwen2-Audio) · 2311.07919 (Qwen-Audio)