arXiv · 2025 · Preprint
Keren, Kozhevnikov, Meng et al. (Meta FAIR) · → Paper · Demo: ? · Code: ✓
Omnilingual ASR introduces the first large-scale ASR system designed for extensibility, covering 1,600+ languages through a 7B-parameter wav2vec 2.0 encoder paired with an LLM-inspired decoder, with zero-shot transcription capability for unseen languages using only a handful of in-context audio-text examples.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
Omnilingual ASR provides a suite of open-source self-supervised speech encoders (300M, 1B, 3B, and 7B parameters) pre-trained on 4.3M hours of speech across 1,600+ languages, offering TTS and multilingual speech systems a massively multilingual feature extraction backbone for downstream tasks. Its encoder-decoder architecture separates a wav2vec 2.0 SSL encoder from an LLM-inspired autoregressive Transformer decoder, enabling CTC-based and attention-based ASR variants that can serve as intelligibility evaluators or feature providers in multilingual TTS pipelines. A zero-shot extension trains the decoder to condition on N in-context speech-text pairs, allowing transcription of languages absent from supervised training without any fine-tuning — a paradigm that lowers the data barrier for low-resource language coverage. The system also releases the Omnilingual ASR Corpus, a new multilingual dataset spanning 348 languages with an average of 10 hours per language, providing a training and evaluation resource for speech generation research in under-served languages.
Wiki Connections
- self-supervised-speech — Omnilingual ASR scales wav2vec 2.0 SSL pre-training to 7B parameters across 1,600+ languages, demonstrating that SSL speech encoder capacity continues to improve cross-lingual representation quality beyond the 2B-parameter frontier established by prior work.
- multilingual-tts — The open-source SSL encoders and the Omnilingual ASR Corpus provide evaluation and feature-extraction resources for multilingual TTS systems targeting low-resource and long-tail languages.
- 1412.6980 (Adam optimizer) — Omnilingual ASR uses Adam for all SSL and ASR fine-tuning stages across its model family.
- 2306.12925 (AudioPaLM) — Omnilingual ASR and AudioPaLM share the paradigm of combining a speech encoder with an LLM-style Transformer decoder for sequence generation, with Omnilingual ASR applying it to massively multilingual ASR.
- 2007.10310 (CoVoST 2) — Omnilingual ASR evaluates speech-to-text translation performance on CoVoST 2, benchmarking cross-lingual generalization of its LLM-ASR models.