arXiv · 2023 · Preprint
Mentzer et al. (Google Research) · → Paper · Demo: ? · Code: ✓
FSQ replaces vector quantization in VQ-VAEs with per-channel scalar rounding over a fixed low-dimensional bounded grid, achieving near-complete codebook utilization without commitment losses, codebook reseeding, or entropy penalties.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
Finite Scalar Quantization (FSQ) introduces a quantization scheme where each dimension of a low-dimensional latent vector is independently bounded and rounded to a fixed set of integer values, forming an implicit codebook as the Cartesian product of per-channel value sets. This eliminates the codebook collapse problem endemic to standard vector quantization, where large codebooks suffer from poor utilization even with auxiliary losses. Neural audio codec systems and speech tokenizers can use FSQ as an alternative quantization backbone to residual vector quantization (RVQ), since its guaranteed high codebook utilization simplifies training and enables larger effective codebooks without auxiliary stabilization machinery. TTS and codec architectures draw on FSQ when designing discrete speech token representations, particularly where stable and expressive codebooks are needed for autoregressive language model training over speech tokens.
Wiki Connections
- neural-codec — FSQ provides an alternative quantization scheme to VQ/RVQ used in audio codecs
- autoregressive-codec-tts — TTS systems built on discrete speech tokens benefit from FSQ’s stable codebook training