arXiv · 2026 · Preprint

Ünal Ege Gaznepoğlu et al. (International Audio Laboratories Erlangen / Fraunhofer IIS / Trinity College Dublin) · → Paper · Demo: ✓ · Code: ?

A controlled diagnosis of why the Vocos time-frequency vocoder trails time-domain GAN vocoders: magnitude prediction is easy for its 1D-convolutional ConvNeXt backbone, while phase is the bottleneck, and switching the backbone to 2D convolutions sharply improves phase-difference prediction.

Problem

Vocos predicts STFT magnitude and phase and reconstructs audio with an inverse STFT. This avoids the costly cascaded upsampling of time-domain vocoders, but its quality still trails them, with artefacts resembling phasiness (the loss of clarity known from classical phase-vocoder methods). Prior work disagrees on the cause and the remedy. Proposed causes include the CNN being unsuited to phase recovery, which is argued to be autoregressive. Proposed remedies include ground-truth magnitude and phase losses, pseudo-inverse mel filters, dropping the iSTFT (WaveNeXt) and 2D convolutions with a harmonic prior (WaveHax). The authors also note that none of these studies compared Vocos to time-domain vocoders on equal training terms.

Method

The study has two strands.

Vocoding comparison. All vocoders take 80-band log mel spectrograms restricted to 0 to 8 kHz at 22.05 kHz sampling. The authors chose a bandlimited input because it should accentuate the harmonic-structure weakness of time-frequency vocoders. Vocos (13.5M params) is a Conv1D input layer, a stack of ConvNeXt blocks and a Head that maps outputs to log-magnitude and phase, followed by an inverse STFT with a Hann window. The authors find that Vocos’s default magnitude clamp of 100 is too low for up to 41% of training samples (at -1 dBFS amplitude scaling), and raise it to 400 (Figure 2). The time-domain reference is BigVGANv2: the official 112M checkpoint matching the input representation (trained on more data), a 112M model retrained on LibriTTS, and a 14.4M BigVGANv2-base sized to match Vocos. The “modified” Vocos and both retrained BigVGANv2 models share one recipe. It uses a multi-period discriminator, the EnCodec MS-STFT discriminator and the MS-SB-CQT discriminator. The generator loss is 15 L_mel + L_adv + 2 L_fm, following DAC, with AdamW for 2M steps (Table 1). Training uses LibriTTS train-clean-100, train-clean-360 and train-other-500 (about 585 h, 2,456 speakers).

Isolation and phase-difference probes. Two oracle variants separate the STFT components. Vocos-Mag predicts only log-magnitude and uses ground-truth phase. Vocos-Phase predicts only phase and uses ground-truth magnitude. To test whether the backbone can model the time-frequency structure that phase reconstruction relies on, the authors adapt it to predict baseband and frequency phase differences (BPD and FPD) from a 513-bin ground-truth log-magnitude spectrogram. The training objective is a wrapping-aware L1 loss (L_wa), and the predicted differences are turned into phase by the least-squares reconstruction of Masuyama et al. They compare the original Conv1D backbone (15.0M) with a Conv2D variant that has 6 ConvNeXt blocks and (32, 64) channels (37.1K params), plus the causal 2D-convolution benchmark model of Masuyama et al. (247K).

Objective metrics are the multi-resolution log-mel loss L_mel, SCOREQ with the ground-truth reference, voiced/unvoiced F1, periodicity RMSE and log-spectral convergence (LSC, in dB). The subjective test is MUSHRA with 16 listeners on 14 of the 20 out-of-distribution German utterances (D2). Its lower anchor is a pseudo-inverted mel spectrogram with PGHI phase.

Key Results

The time-domain advantage survives controlled training. The retrained BigVGANv2 (112M) beats Vocos (modified) on every Table 2 metric on both test sets (for example L_mel 0.846 vs. 1.308 on D1, LSC -18.06 vs. -12.39 dB). The 14.4M BigVGANv2-base, at a parameter count close to Vocos’s 13.5M, also beats Vocos (modified) on every metric and test set (L_mel 1.172 vs. 1.308 on D1, 1.126 vs. 1.309 on D2) (§5.1, Table 2).

The shared training recipe gives mixed objective results for Vocos. Compared with Vocos trained with its official recipe, Vocos (modified) improves L_mel (1.389 to 1.308 on D1, 1.463 to 1.309 on D2) but worsens SCOREQ (0.161 to 0.188 on D1, 0.213 to 0.258 on D2). Pitch metrics and LSC barely change (§5.1, Table 2). In MUSHRA, by contrast, Vocos (modified) is rated significantly above Vocos (official), and the official BigVGANv2 checkpoint is rated significantly above both (Figure 4). Of the objective metrics, only L_mel agrees with the listening test. The authors suggest that the limited bandwidth of SCOREQ’s wav2vec 2.0 representation makes it less sensitive to these differences. Exact MUSHRA scores are plotted, not tabulated.

Phase, not magnitude, limits Vocos. Vocos-Mag (ground-truth phase) has the lowest L_mel in Table 2 on both test sets (0.495 on D1, 0.552 on D2). It also has the best V/UV F1, periodicity RMSE and LSC on both sets, and the best SCOREQ on D1 (0.073). On D2 its SCOREQ (0.147) trails the official BigVGANv2 checkpoint (0.130). Vocos-Phase (ground-truth magnitude) is worst on every metric and test set, for example LSC -10.39 dB on D1 against -12.39 dB for jointly predicting both components (§5.2, Table 2). So giving Vocos the true magnitude and asking it only for phase is worse than letting it predict both. The authors read this as evidence that freedom to predict an “inconsistent” magnitude compensates for phase errors.

1D convolutions bottleneck phase-difference prediction. With ground-truth log-magnitude input, the Conv1D Vocos backbone reaches L_wa 0.646 and LSC -20.24 dB. The Conv2D variant reaches L_wa 0.112 and LSC -29.56 dB with about 400 times fewer parameters (37.1K vs. 15.0M) at slightly higher compute (3.5B vs. 3.0B FLOPs/s). It is also better than the Masuyama et al. benchmark on both metrics (0.238, -26.12 dB), which needs 21.8B FLOPs/s (§5.3, Table 3).

The comparisons are fairly controlled. The retrained BigVGANv2 models and Vocos (modified) share data, losses, discriminators and training length. The main exception is that the MUSHRA test uses the official BigVGANv2 checkpoint, which was trained on a larger dataset, rather than the retrained one.

Novelty Assessment

This is an analysis paper, not a new vocoder. The oracle-magnitude and oracle-phase ablation is a simple, informative probe. It shows that joint prediction beats phase-only prediction, which argues against losses that pull predicted STFT components toward the ground truth, although the paper does not train such losses itself. The phase-difference experiment links the Vocos weakness to the 1D-convolution inductive bias rather than to the absence of autoregression. The Conv2D backbone used there is a probe trained on ground-truth magnitudes; it is not integrated into a vocoder. The controlled BigVGANv2 comparison and the clamping-threshold fix are useful hygiene. No new architecture is proposed, and the paper leaves open how to get 2D time-frequency inductive bias into a mel-conditioned vocoder without relying on harmonic priors. The arXiv comment states that the paper was accepted at IWAENC 2026.

Field Significance

Moderate — The paper provides controlled evidence that the quality gap between iSTFT-based and time-domain GAN vocoders persists when training recipes and parameter counts are matched, and it locates the gap in phase modelling by 1D-convolutional backbones. It also cautions against regularizing predicted magnitudes toward the ground truth. The result narrows the design question for time-frequency vocoders to inductive bias over the time-frequency plane.

Claims

  • supports: Time-domain GAN vocoders keep a quality advantage over iSTFT-based time-frequency vocoders even when training losses, discriminators, data and parameter count are matched.

    Evidence: Under one shared recipe on LibriTTS, both a 112M BigVGANv2 and a 14.4M BigVGANv2-base beat a 13.5M Vocos on L_mel, SCOREQ, V/UV F1, periodicity RMSE and LSC on in-domain and out-of-distribution test sets. In MUSHRA, the official BigVGANv2 checkpoint is rated significantly above both Vocos variants. (§5.1, Table 2, Figure 4)

  • refines: In spectrogram-domain neural vocoders, phase prediction rather than magnitude prediction is the main source of the quality deficit.

    Evidence: Vocos given ground-truth phase and predicting only log-magnitude is best on most Table 2 metrics, while Vocos given ground-truth magnitude and predicting only phase is worst on every metric and test set. (§5.2, Table 2)

  • complicates: Constraining a spectrogram-domain vocoder to predict ground-truth-consistent magnitudes can hurt quality, because the freedom to predict inconsistent magnitudes compensates for phase errors.

    Evidence: Predicting phase alone with ground-truth magnitude (LSC -10.39 dB on LibriTTS test-clean) is worse than jointly predicting magnitude and phase (-12.39 dB). The authors infer from this that STFT-coefficient regularization losses may be counterproductive, but they do not train such losses. (§5.2, Table 2)

  • supports: Convolutions over both time and frequency capture the structure needed for STFT phase reconstruction far better than 1D convolutions that treat frequency bins as channels.

    Evidence: Predicting phase differences from ground-truth log-magnitude, a Conv2D ConvNeXt backbone with 37.1K params reaches wrapping-aware loss 0.112 and LSC -29.56 dB. The 15.0M Conv1D Vocos backbone reaches 0.646 and -20.24 dB, and a 247K causal 2D-convolution benchmark reaches 0.238 and -26.12 dB. (§5.3, Table 3)

  • complicates: Learned reference-based quality predictors can miss vocoder quality differences that listeners detect reliably.

    Evidence: SCOREQ rates Vocos trained with its official recipe better than the modified Vocos (0.213 vs. 0.258 on German D2), but MUSHRA listeners rate the modified Vocos significantly higher. Only the L_mel loss agrees with the listening test, and the authors attribute SCOREQ’s insensitivity to the limited bandwidth of its wav2vec 2.0 features. (§5.1, Table 2, Figure 4)

Limitations and Open Questions

  • The paper diagnoses the problem but does not build an improved vocoder. The Conv2D backbone is tested only on phase-difference prediction from ground-truth log-magnitudes, and the authors note that narrow 2D receptive fields may fail to capture harmonics spread across frequency.
  • The MUSHRA test is small: 16 listeners, 14 proprietary German utterances from two speakers. It includes the official BigVGANv2 checkpoint (trained on more data) rather than the matched retrained model, and the oracle variants are not tested at all.
  • All experiments use one input configuration (80-band mel, 0 to 8 kHz, 22.05 kHz). The authors chose it to accentuate the time-frequency vocoder weakness, so the size of the gap may differ for full-band inputs.
  • The claim that ground-truth STFT regularization losses may be counterproductive is an inference from the oracle ablation. The paper does not train Vocos with those losses.
  • No vocoder evaluation inside a TTS pipeline: all inputs are ground-truth mel spectrograms (copy synthesis).

Wiki Connections

  • GAN Vocoder — Gives a controlled comparison of the iSTFT-based Vocos and time-domain BigVGANv2 GAN vocoders under matched losses and discriminators, and locates Vocos’s deficit in phase modelling.
  • Subjective Evaluation — Runs a MUSHRA test whose ranking agrees with the L_mel loss but not with the learned SCOREQ predictor.
  • Vocos — The system under analysis; the paper retrains it, fixes its magnitude-clamping threshold, and ablates its magnitude and phase heads.
  • BigVGAN — The time-domain reference vocoder, used as the official checkpoint and retrained at 112M and 14.4M under the shared recipe.
  • Is GAN Necessary for Mel-Spectrogram-based Neural Vocoder? — Cited as a work adding ground-truth magnitude and phase loss terms, an approach the oracle ablation suggests may be counterproductive.
  • Learning Neural Vocoder from Range-Null Space Decomposition — Cited for pseudo-inverse magnitude recovery and STFT regularization, which the paper’s Vocos-Mag and Vocos-Phase results call into question.
  • EnCodec — Supplies the multi-scale STFT discriminator used in the shared vocoder training recipe.
  • LibriTTS — Provides the training data and the in-domain test set (test-clean).