arXiv · 2025 · Preprint
Du et al. · → Paper · Demo: ? · Code: ?
FreeGAN demonstrates that GAN-free mel-spectrogram vocoders can match adversarially-trained systems in subjective quality by combining an amplitude prior from the pseudo-inverse mel filter, a SNAKE-ConvNeXt v2 backbone, and a frequency-weighted anti-wrapping phase loss — while training nearly 15x faster than BigVGAN.
Problem
GAN-based mel-spectrogram vocoders (HiFi-GAN, BigVGAN) dominate the field, but adversarial training carries significant costs: discriminators can contain five times as many parameters as the generator, training is slow, and storing discriminator weights for fine-tuning adds substantial storage overhead. Non-GAN alternatives either impose high model complexity (flow-based vocoders), require autoregressive decoding, or are too slow for real-time CPU inference (diffusion-based vocoders). The field lacked a practical demonstration that adversarial training could be dropped from the mel-spectrogram vocoder pipeline without a meaningful quality regression.
A secondary question the paper addresses is whether the conclusion generalises beyond the proposed system: which structural properties of a vocoder determine whether it can dispense with GAN?
Method
FreeGAN adopts an amplitude-phase serial prediction framework. Rather than feeding the mel-spectrogram directly into the model, it first computes an amplitude prior by applying the Mel pseudo-inverse filter (Moore-Penrose pseudo-inverse of the mel filterbank matrix) to the mel-spectrogram and taking the absolute value with a small floor constant. This prior bootstraps amplitude prediction and reduces the burden on the network, which the authors argue also improves downstream phase prediction accuracy.
The backbone is a novel SNAKE-ConvNeXt v2 block: a ConvNeXt v2 block with the GELU activation replaced by the SNAKE activation function (a learnable periodic activation, f(x) = x + (1/α)sin²(αx)), which improves modelling of periodic waveform components. One block handles amplitude prediction from the prior; four blocks with parallel estimation handle phase prediction, reflecting the greater difficulty of accurate phase modelling.
The training objective is GAN-free. A frequency-weighted anti-wrapping (FWAW) phase loss extends the prior anti-wrapping phase loss by exponentially upweighting higher-frequency bins, counteracting the observation that high-frequency phase prediction degrades most severely when GAN supervision is removed. This is combined with amplitude loss, reconstructed STFT loss, and mel-spectrogram loss. Waveform reconstruction uses inverse STFT (iSTFT). Generator parameters total 13.4M; without discriminators, storage and training costs are substantially reduced.
Key Results
On the VCTK test set (multi-speaker, 8 unseen speakers, 2937 utterances), FreeGAN achieves MOS 4.12 ± 0.06, indistinguishable from BigVGAN (4.14 ± 0.06, p=0.61) and APNet2 (4.13 ± 0.06, p=0.76). Objective metrics tell a more striking story: FreeGAN leads all baselines on SNR (7.73 dB), MCD (0.65 dB), F0-RMSE (15.19 cents), and V/UV error (2.63%), though its UTMOS (3.92) falls slightly below BigVGAN (3.97) and APNet2 (4.00). Computationally, FreeGAN ties Vocos at 2.70 GFLOPS — the lowest in the comparison — while training in 191 seconds per epoch versus 1937 for BigVGAN. Eliminating discriminators also reduces storage: the parameter count is 13.4M + 0 for FreeGAN versus 13.0M + 70.7M for HiFi-GAN.
The paper’s second experiment tests whether the improvement strategies generalise to other vocoders. For explicit-phase-prediction systems (APNet2, FreeV), removing GAN and applying the proposed strategies produces no statistically significant quality drop (ABX preference p=1.00 and p=0.177, respectively). For systems without explicit phase prediction (HiFi-GAN, iSTFTNet, Vocos), removing GAN causes a noticeable quality drop that the strategies only partially recover — GAN remains necessary for those architectures.
Note
UTMOS and MOS rankings do not always agree across baselines; BigVGAN has higher UTMOS than FreeGAN yet comparable MOS, suggesting automatic predictors and subjective listeners weight different quality dimensions differently on this corpus.
Novelty Assessment
The architecture is genuinely novel in its combination of components, but each element draws on existing work: the pseudo-inverse amplitude prior is adapted from FreeV (Interspeech 2024), SNAKE activation from prior neural audio work, and anti-wrapping phase loss from the authors’ earlier APNet paper. The FWAW phase loss — extending anti-wrapping to weight high-frequency bins more heavily — is the most original technical contribution.
The paper’s broader intellectual contribution is the controlled comparison: by systematically removing GAN from five vocoder architectures and testing with and without the proposed strategies, it produces the clearest published evidence that the necessity of GAN in mel-spectrogram vocoders is contingent on whether the system performs explicit phase prediction, not a universal requirement. This diagnostic framing is more useful to the field than yet another hybrid vocoder ablation.
The evaluation is limited to VCTK at 16 kHz — multi-speaker English only, without naturalness tests on noisy or out-of-domain conditions. BigVGAN is acknowledged to be a general-purpose system evaluated somewhat unfairly on a constrained dataset, so the efficiency comparison overstates FreeGAN’s advantage in zero-shot settings.
Field Significance
Moderate — FreeGAN provides clear empirical evidence that the GAN discriminator, long treated as an essential ingredient of high-quality mel-spectrogram vocoders, is dispensable for architectures that explicitly predict the phase spectrum. This clarifies a design principle the field has not previously articulated explicitly: it is the phase modelling capability, not adversarial training per se, that sustains quality. The result is primarily relevant to practitioners building production vocoders where training cost and storage matter, and to researchers exploring the boundary between GAN and non-GAN vocoder families.
Claims
- Adversarial training is not necessary for high-quality mel-spectrogram vocoders that explicitly predict the phase spectrum; frequency-weighted phase supervision can substitute for discriminator-provided phase signal. (§IV.A, Table I)
- The necessity of GAN in neural vocoders is contingent on the presence of explicit phase prediction: removing adversarial training without explicit phase modelling results in a quality gap that targeted improvements do not fully close. (§IV.B, Table III)
- Eliminating GAN discriminators from vocoder training substantially reduces wall-clock training time and storage requirements with no measurable subjective quality cost for explicit-phase architectures. (§IV.A, Table I)
- Automatic MOS predictors (UTMOS) and subjective MOS scores can rank vocoder systems differently, making UTMOS an unreliable sole proxy for naturalness evaluation. (§IV.A, Table I)
Limitations and Open Questions
Warning
All experiments use VCTK at 16 kHz with a constrained multi-speaker English setup. Generalisation to 24 kHz or 44.1 kHz targets, non-English speakers, noisy or out-of-domain conditions, and downstream TTS pipelines (as opposed to ground-truth mel reconstruction) is untested and explicitly deferred to future work.
The comparison to BigVGAN is acknowledged to be partially unfair: BigVGAN is trained and optimised for general-purpose, large-scale audio generation, so the efficiency figures reflect FreeGAN’s advantage in a more constrained setting. Whether FreeGAN’s approach scales to the quality levels BigVGAN achieves in zero-shot conditions remains open.
The FWAW phase loss design relies on a heuristic exponential weighting scheme across frequency bins with a hyperparameter (ρ = 2.5); how sensitive results are to this choice and whether a learned weighting would improve further is not explored.
Code availability is not reported, which limits reproducibility.
Wiki Connections
- gan-vocoder
- evaluation-metrics
- subjective-evaluation
- CosyVoice — cited as a downstream consumer of mel-spectrogram vocoders