arXiv · 2025 · Preprint
Gu, Zhang, Wang, Li, Wu & Juvela (Spellbrush, Aalto University, and The Chinese University of Hong Kong, Shenzhen) · → Paper · Demo: ✓ · Code: ✓
A signal-processing-grounded fix for the aliasing artifacts introduced by nonlinear activations and upsampling layers in neural vocoders and codecs, replacing the unconstrained activation with an oversampled antiderivative-anti-aliased activation and the ConvTranspose upsampling layer with a resampling scheme, yielding Pupu-Vocoder and Pupu-Codec, which outperform existing systems on singing voice, music, and general audio while matching them on speech.
Problem
Neural vocoders and codecs reconstruct waveforms from acoustic or latent representations, and their fidelity directly bounds the quality of any downstream generative system built on top of them. While current models synthesize perceptually natural speech, they struggle with high-fidelity music and singing voice, because unconstrained nonlinear activation functions generate harmonics beyond the Nyquist frequency that “fold back” into audible aliasing artifacts, and the widely-used ConvTranspose upsampling layer mirrors low-frequency content into the newly-created high-frequency band, producing both “mirrored” aliasing and a periodic “tonal artifact.” Digital signal processing has long had anti-aliasing techniques for exactly these problems, but their integration into neural vocoder and codec architectures has remained largely unexplored, leaving synthesis fidelity in high-harmonic-content domains (singing, music) as a persistent weak point.
Method
The paper targets the two sources of aliasing directly. For activations, it applies antiderivative anti-aliasing (ADAA): rather than applying the nonlinearity to the discrete signal directly, ADAA reconstructs a continuous-time version of the signal via linear interpolation, applies the activation in continuous time (where no sampling-rate constraint exists to cause aliasing), and recovers the discrete output through a closed-form expression involving the activation’s first-order antiderivative. The paper selects SnakeBeta (used in BigVGAN and DAC) as the base activation specifically because its antiderivative eliminates the denominator term that would otherwise require a numerically-unstable threshold-based fallback, yielding a bounded, numerically stable “ADAA SnakeBeta,” combined with a 2x oversampling factor for further aliasing suppression. For upsampling, the paper replaces ConvTranspose (whose zero-interlacing operation is mathematically equivalent to frequency-domain spectral replication, causing both mirrored aliasing and periodic tonal artifacts) with a resampling scheme: zero-interlacing followed by a longer, more carefully-designed low-pass filter (Kaiser-windowed sinc, filter size 16 versus the size-6 filters used in prior work), with a noise-like, high-pass-filtered deterministic prior injected to fill the resulting empty high-frequency band and stabilize training. These two anti-aliased modules (“AF Conv Blocks”) replace the corresponding blocks in BigVGAN- and DAC-derived architectures. Pupu-Codec consists of a convolutional encoder, a DAC-style residual vector quantizer (86Hz frame rate, up to 8kbps), a decoder built from AF Conv Blocks, and four discriminator types (multi-period, multi-scale, multi-band, and multi-scale sub-band CQT); replacing the waveform-input encoder and RVQ module with a mel-spectrogram input yields Pupu-Vocoder. Both models are trained with a DAC-style adversarial and reconstruction loss combination (multi-scale mel loss, codebook/commitment losses for the codec, adversarial and feature-matching losses across all discriminators) on a large multi-domain dataset spanning speech, singing voice, music, and general audio, each split into an academic evaluation set (public benchmark corpora) and an industrial evaluation set (proprietary, high-quality production data).
Key Results
On a dedicated synthetic test-signal benchmark (sine, sawtooth, triangle waves), the proposed anti-aliased activation and upsampling modules achieve the lowest average aliasing-to-harmonic ratio among all compared activation and upsampling alternatives, including plain SnakeBeta at various oversampling factors, ELU, ConvTranspose, and linear/nearest interpolation. On analysis-synthesis evaluation across speech, singing voice, music, and general audio (against HiFi-GAN, BigVGAN, EnCodec, DAC, BigCodec, and the aliasing-free Vocos baselines), Pupu-Vocoder and Pupu-Codec achieve MUSHRA scores that are consistently comparable to or better than baselines on speech and consistently outperform them on singing voice, music, and audio, while yielding comparable results to strong same-parameter-scale baselines on speech-specific objective metrics (MOS-Pred, F0-RMSE). The smaller Pupu-Codec variant outperforms the similarly-scaled EnCodec baseline and achieves performance comparable to substantially larger models across domains, indicating the anti-aliased modules improve parameter efficiency, not just peak quality. In a dynamic-bitrate encoding evaluation on industrial-setting singing voice, Pupu-Codec-large outperforms DAC and BigCodec subjectively at high and medium bitrates (8/5.33/2.67 kbps) and is comparable to them at the lowest bitrate tested (1.78 kbps), where the information bottleneck itself limits high-frequency reconstruction regardless of anti-aliasing quality. An ablation study on singing voice confirms each component’s contribution: removing oversampling, reverting to a plain activation, or removing the deterministic high-frequency noise prior each measurably degrades both objective and subjective (C-MOS) quality, with the deterministic prior’s removal causing the largest subjective quality drop.
Novelty Assessment
The underlying anti-aliasing techniques (oversampling, antiderivative anti-aliasing, resampling in place of transposed convolution) are each individually established in prior digital signal processing and adjacent neural-vocoder literature; the paper’s contribution is the specific, carefully-engineered combination and integration of these techniques into a working, trainable neural vocoder/codec architecture, including the numerically-stabilized ADAA SnakeBeta derivation, the longer Kaiser-windowed low-pass filter design, and the deterministic high-frequency noise prior needed to keep training stable once the aliased (but information-carrying) high-frequency content is removed. The domain-agnostic framing and the newly constructed synthetic test-signal benchmark for isolating and measuring aliasing (independent of any downstream generation task) is a genuine, reusable evaluation-methodology contribution alongside the architectural one.
Field Significance
high — This is a rigorous, signal-processing-grounded treatment of a well-documented but under-addressed problem (aliasing) in neural vocoders and codecs, validated with a dedicated synthetic benchmark, comprehensive ablations, and both objective and subjective (MUSHRA, C-MOS) evaluation across four audio domains, accepted to a peer-reviewed journal (IEEE TASLP).
Because the anti-aliased activation and upsampling modules are architecture-agnostic drop-in replacements rather than a new end-to-end system, they offer a concrete, adoptable fix that other neural vocoder and codec designs (including those underlying TTS and SVS pipelines) can incorporate directly, particularly for domains with dense harmonic content like singing voice and music where the paper shows the largest gains; the comparable-rather-than-superior results on plain speech indicate the technique’s benefit scales with a signal’s harmonic complexity rather than being universal.
Claims
- supports: Replacing an unconstrained nonlinear activation function with an oversampled, antiderivative-anti-aliased version measurably reduces aliasing artifacts in neural audio synthesis, and this reduction translates into improved perceptual audio quality, particularly for high-harmonic-content audio like singing voice and music.
Evidence: On the synthetic test-signal benchmark, ADAA SnakeBeta combined with 2x oversampling achieves the lowest average aliasing-to-harmonic ratio (-45.95 dB) among all tested activation variants, and an ablation on singing voice shows removing oversampling or reverting to a plain activation both measurably degrade C-MOS quality (-0.38 and up to -1.45 respectively). (§V.C, Table I, Table V)
- supports: Replacing the ConvTranspose upsampling layer with a resampling scheme (zero-interlacing plus a longer low-pass filter) combined with a deterministic high-frequency noise prior removes both mirrored aliasing and tonal artifacts more effectively than linear or nearest-neighbor interpolation alternatives, and the deterministic prior is necessary to prevent a training-stability trade-off from this removal.
Evidence: The proposed upsampling module achieves the lowest average aliasing-to-harmonic ratio (-53.93 dB) among all tested upsampling alternatives including ConvTranspose and linear/nearest interpolation, while an ablation removing the deterministic high-frequency prior causes the largest C-MOS quality drop (-1.49) of any single ablated component tested. (§V.C, Table I, Table V)
- complicates: Anti-aliasing improvements to neural vocoder and codec architectures provide domain-dependent benefit, yielding clear quality gains on harmonically dense signals like singing voice, music, and general audio, but only comparable (not superior) performance on speech, where existing baseline architectures are already comparatively well-optimized.
Evidence: Across the analysis-synthesis evaluation, Pupu-Vocoder and Pupu-Codec consistently outperform baselines on singing voice, music, and audio domains, while achieving performance that the authors describe as merely comparable to baselines on speech, both objectively and subjectively. (§V.C.2, Table II, Table III)
- complicates: Anti-aliased neural audio synthesis modules incur a non-trivial computational cost trade-off, particularly on CPU, that must be weighed against their quality gains for latency-sensitive deployment scenarios.
Evidence: Pupu-Codec-large has a higher CPU real-time factor than BigCodec (10.2 vs. 6.8) due to unoptimized DDSP oversampling and filtering kernels, though it remains faster than BigCodec on GPU (0.0157 vs. 0.0255) due to having fewer parameters; the authors note the approach targets offline production scenarios that can tolerate the added compute. (§V.C.2)
Limitations and Open Questions
The authors identify three open issues in their own future-work discussion: Pupu-Codec’s performance at extremely low bitrates still needs improvement, which matters directly for LLM-based acoustic models that depend on compact token sequences; overall music synthesis quality remains limited relative to speech and singing, motivating exploration of architectures better suited to polyphonic harmonic structure; and the oversampling strategy itself introduces meaningful computational overhead (particularly on CPU) that the authors plan to address via higher-order ADAA techniques that could eliminate the need for oversampling altogether. The paper’s baseline comparisons for the general-audio and music domains extend beyond this wiki’s core TTS/VC/SCA scope, so those specific numeric results should be read as supporting evidence for the general vocoder/codec technique rather than as TTS-domain claims.
Wiki Connections
- GAN Vocoders — introduces anti-aliased activation and upsampling modules as drop-in replacements for the standard nonlinear activation and ConvTranspose layers used in GAN-based vocoder architectures like BigVGAN and HiFi-GAN.
- Neural Codec — applies the same anti-aliased modules to a DAC-derived RVQ neural codec architecture (Pupu-Codec), improving reconstruction fidelity particularly for singing voice and music.
- Subjective Evaluation — validates results with real human-rater MUSHRA and Comparative MOS listening tests across speech, singing voice, music, and audio domains, alongside a dedicated synthetic aliasing-measurement benchmark.
- BigVGAN — Pupu-Vocoder’s decoder architecture is directly derived from BigVGAN, with its activation and upsampling modules replaced by the proposed anti-aliased versions; BigVGAN also serves as a primary comparison baseline.
- HiFi-GAN — used as a neural vocoder baseline across all analysis-synthesis evaluations.
- Vocos — used as the aliasing-free time-frequency-domain referential baseline system, shown to struggle with high-frequency phase reconstruction relative to the proposed time-domain approach.
- BigCodec — used as a neural codec baseline across all analysis-synthesis and dynamic-bitrate evaluations.
- EnCodec — used as a neural codec baseline across all analysis-synthesis and dynamic-bitrate evaluations, showing the most severe quality degradation at low bitrates among compared codecs.