arXiv · 2022 · Preprint
Sang-gil Lee et al. (NVIDIA) · → Paper · Demo: ✓ · Code: ✓
BigVGAN introduces periodic Snake activations with anti-aliased representations into a scaled-up GAN vocoder (112M parameters), achieving strong zero-shot generalisation to unseen speakers, languages, singing voices, and instrumental audio without fine-tuning.
Problem
GAN vocoders such as HiFi-GAN produce high-fidelity speech for seen speakers recorded in clean environments, but degrade substantially when presented with out-of-distribution mel spectrograms: unseen recording environments, novel languages, non-speech vocalizations, or musical audio. Existing universal vocoder work using autoregressive or flow-based models is slower and architecturally constrained, while larger GAN models had not previously been explored because the failure modes of large-scale GAN training for audio were not well understood. The community lacked a single pretrained vocoder capable of generalising across diverse audio types without per-target fine-tuning.
Method
BigVGAN builds on the HiFi-GAN generator backbone and introduces two complementary architectural changes. First, it replaces Leaky ReLU activations with the Snake function (f_alpha(x) = x + sin^2(alpha*x)/alpha), which provides a learnable periodic inductive bias with per-channel frequency parameters. The monotonicity of sin^2(x) keeps optimisation tractable. Second, it wraps every Snake activation in an anti-aliased nonlinearity: the signal is upsampled 2x, passed through Snake, then downsampled 2x using Kaiser-windowed sinc low-pass filters. This suppresses aliasing artifacts from high-frequency components that cannot be represented in the discrete-time output. The resulting module is called the anti-aliased multi-periodicity composition (AMP).
On the discriminator side, the multi-scale discriminator (MSD) from HiFi-GAN is replaced with a multi-resolution discriminator (MRD), which operates on linear spectrograms at multiple STFT resolutions and sharpens spectral structure. The training objective otherwise follows HiFi-GAN: least-square adversarial loss, feature matching loss, and a mel-spectrogram regression loss.
The paper trains two model scales: BigVGAN-base (14M parameters, comparable to HiFi-GAN V1 in size) and BigVGAN (112M parameters). Scaling from 14M to 112M requires addressing specific training instabilities: the anti-aliased nonlinearity amplifies gradient norms from the multi-period discriminator, causing early collapse. Gradient clipping at a global norm of 10^3 and halving the learning rate to 1e-4 stabilise training. Batch size is doubled from 16 to 32. All models are trained for 1M steps on LibriTTS train-full (all subsets including noisy train-other-500) at 24 kHz with a 100-band log-mel spectrogram covering [0, 12] kHz.
Key Results
On in-distribution LibriTTS evaluation, BigVGAN (112M) achieves MOS 4.11 and SMOS 4.26 versus HiFi-GAN V1’s MOS 4.08 and SMOS 4.15 (Table 2). The SMOS gap (+0.11 over HiFi-GAN) is more diagnostic than MOS, since SMOS requires side-by-side comparison with ground truth and captures fine-grained speaker fidelity. BigVGAN-base with 14M parameters already outperforms HiFi-GAN on SMOS (+0.05), demonstrating that the architectural improvements matter independently of scale.
For out-of-distribution speech across unseen languages and recording environments (Table 3), BigVGAN outperforms HiFi-GAN by larger margins under noisy conditions (simulated: 4.26 vs. 4.13 SMOS; real-world Korean: 4.38 vs. 4.21 SMOS), with BigVGAN-base also reaching significance. On musical audio from MUSDB18-HQ (Table 4), BigVGAN achieves an average SMOS of 4.26 vs. 4.08 for HiFi-GAN, with the largest gains on singing voice (4.37 vs. 4.26) and other instruments (4.25 vs. 3.92). Linguistic accuracy on CER across six languages (Table 11) confirms BigVGAN’s fidelity advantage: BigVGAN-base (14M) outperforms a 112M HiFi-GAN on four of six languages.
Synthesis speed at 24 kHz is 44.72x real-time for the 112M model and 70.18x for BigVGAN-base, both comfortably above real-time despite the anti-aliasing overhead (Table 1).
Ablation confirms both components are necessary: removing the low-pass filter degrades OOD performance, and removing both filter and Snake activation (reverting to vanilla HiFi-GAN with MRD) is worse still, with all differences statistically significant (p < 0.01, Wilcoxon signed-rank).
Novelty Assessment
The paper’s core novelty is two-fold: the introduction of Snake activations as a periodic inductive bias for waveform generation, and the pairing with anti-aliased upsampling/downsampling inspired by StyleGAN3. Neither technique is invented here (Snake activations originate in Liu et al. 2020 for time-series regression; anti-aliased representations follow Karras et al. 2021 for image synthesis), but their combination and adaptation to the specific requirements of neural vocoding is non-trivial and carefully validated. The failure-mode analysis for large-scale GAN training on audio is a practical contribution with no prior counterpart in this domain.
The scale (112M parameters for a GAN vocoder) and training data scope (LibriTTS full, including noisy splits) are both unprecedented at time of publication. The claim of universal vocoding, including musical audio, goes beyond prior universal-speech claims and is backed by a rigorous multi-condition evaluation. The choice to use the full LibriTTS training set including noisy subsets, and to test on MUSDB18-HQ music, broadens the empirical scope meaningfully.
The discriminator replacement (MSD to MRD) is validated but minor; the key contribution is clearly the generator architecture.
Field Significance
Tip
High, architectural-novelty and empirical-benchmark combined. BigVGAN establishes that periodic inductive bias and anti-aliasing are necessary ingredients for robust GAN vocoders at scale, providing a strong empirical baseline across in-distribution and out-of-distribution conditions. Its release as an open-source 112M checkpoint makes it a practical reference implementation for downstream TTS and VC pipelines that require universal vocoders.
Claims
- Periodic activation functions with learnable frequency parameters improve GAN vocoder robustness to out-of-distribution audio, including unseen recording environments, languages, and non-speech signals. (§3.2, §4.5, §4.6)
- Anti-aliasing via low-pass filtering around nonlinear activations reduces high-frequency artifacts in neural vocoders and is complementary to periodic inductive bias. (§3.3, §4.7, Table 4)
- GAN vocoders benefit significantly from scaling model capacity when paired with appropriate architectural improvements and training stabilisation techniques, whereas naive scaling without those changes leads to early collapse. (§3.4, §4.7)
- Similarity MOS (SMOS), which requires direct comparison with ground truth, is a more sensitive evaluation metric than standard MOS for distinguishing speaker fidelity under diverse and out-of-distribution conditions. (§4.3)
- Training on diverse, multi-condition speech data (including noisy recordings) is a necessary complement to architectural improvements for achieving universal vocoding performance. (§4.7, Table 5)
Limitations and Open Questions
Warning
BigVGAN is trained and evaluated on 24 kHz audio only. Generalisation to 44.1 kHz or 48 kHz audio (common in music and broadcast) is not tested and the model architecture is not designed for it.
The 112M model synthesises at 44.72x real-time, which is slower than HiFi-GAN (93.75x) and BigVGAN-base (70.18x). For latency-sensitive applications, the architectural overhead of filtered Snake activations is a concrete trade-off. The paper acknowledges that spectral normalisation, anti-aliased upsampling layers, and larger discriminators all degraded quality; the practical lessons are reported informally but without a principled explanation of why regularisation harms audio GANs when it helps image GANs. MOS evaluations are conducted with native English-speaking Mechanical Turk workers even for non-English and OOD conditions, which may introduce evaluator bias. The model is trained exclusively on speech despite being evaluated on music and instruments, and the extrapolation limits of this approach for broader audio generation are not characterised.
Wiki Connections
Core methods: gan-vocoder, zero-shot-tts
Evaluation tools used: evaluation-metrics, subjective-evaluation
Multi-lingual generalisation setting: multilingual-tts
Cited by: 2504.18425 (Kimi-Audio, BigVGAN used as vocoder in Kimi-TTS decoding pipeline)
2306.00814 — Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis 2409.05377 — BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec