arXiv · 2025 · Preprint
Gemma Team et al. (Google DeepMind) · → Paper · Demo: ? · Code: ?
Gemma 3 is a family of open, multimodal decoder-only language models (1B to 27B parameters) that introduces vision understanding, 128K-token long context via a 5:1 local-to-global attention interleaving scheme, and expanded multilingual coverage, trained end-to-end with knowledge distillation from a larger teacher model.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
Gemma 3 provides a family of open, lightweight decoder-only language models that can serve as a text backbone in speech generation systems where compute budget or deployment footprint constrains model choice. The models range from 1B to 27B parameters, enabling integration at multiple scales, with instruction-tuned variants that support strong instruction-following across more than 35 languages. The 128K-token context window and multilingual tokenizer (262k vocabulary, balanced for non-English scripts) make Gemma 3 a practical backbone for multilingual TTS or spoken conversational agent pipelines that require long-context language understanding. The quantized checkpoints (Int4, SFP8) further lower the barrier to on-device or low-latency deployment alongside a neural audio codec or streaming vocoder.
Wiki Connections
- multilingual-tts — Gemma 3’s expanded multilingual pre-training data and balanced tokenizer provide a foundation for language models used in multilingual TTS systems.
- spoken-language-model — Gemma 3’s instruction-tuned variants with long-context and multimodal capabilities can serve as the language backbone in spoken conversational agent architectures.
- 2407.21783 (LLaMA 3) — Gemma 3 and LLaMA 3 are contemporaneous open foundation LM families occupying the same role as interchangeable text backbones for TTS and SCA systems.