arXiv · 2026 · Preprint
Huanchen Cai et al. (KTH Royal Institute of Technology) · → Paper · Demo: ? · Code: ?
Brings voice range profiling (voice mapping) from clinical voice science to TTS evaluation. It plots cycle-level crest factor, spectrum balance, and smoothed cepstral peak prominence (CPPs) over the f0 × SPL plane, then compares six LJSpeech-trained systems against the original speaker.
Problem
Standard objective TTS metrics (MCD, ESTOI, PESQ, V/UV error) are computed per frame or per utterance and then averaged. They do not show how voice quality changes with pitch and loudness inside a single speaker, so they miss vocal effort, the coupling of phonation mode with voice quality, and long-form instability. MOS has its own problems: range-equalizing bias, relative rather than absolute scores, and unstable values across studies. The paper notes that reported MOS for Tacotron 2 ranges from 3.70 to 4.53 across papers. The authors want an objective, signal-processing measure of whether a TTS voice reproduces the speaker’s dynamic range and the way voice quality changes across phonation conditions.
Method
The method is an evaluation protocol. No synthesis model is proposed. The authors take 100 consecutive LJSpeech sentences from the LJ050 block (about 27 minutes of audio) and resynthesize them with public pre-trained LJSpeech checkpoints from Coqui TTS and ESPnet. The systems are Merlin (DNN statistical parametric), Tacotron 2, Transformer TTS, FastSpeech 2, Glow-TTS, and VITS. To isolate vocoder effects, Tacotron 2 is held fixed as the acoustic model and paired with Multiband-MelGAN and with UnivNet. The test texts come from the training corpus, so the setup deliberately measures model behaviour under controlled conditions rather than generalization (§2.1).
FonaDyn 3.1.0 segments the output into phonatory cycles and extracts f0, SPL, crest factor (peak-to-RMS ratio per cycle), spectrum balance (power above 2 kHz relative to power below 1.5 kHz, in dB), and CPPs (Awan et al. method, 512 cepstral bins, temporal and quefrency smoothing). Every cycle becomes a triplet (f0, SPL, metric). The f0–SPL plane is divided into cells of 1 semitone × 1 dB, and each cell holds the mean metric value. A voice map is the set of occupied cells. Voice range is the number of occupied cells, and difference maps compare each system with the original speaker over the cells they share. TTS corpora are not SPL-calibrated, so the authors assume the outputs are “self-calibrated”: the systems do not condition on volume and they preserve the speaker’s intensity patterns (§2.3, §4).
Key Results
- Voice range. Cumulative voice-map area levels off after about 20 sentences. The original speaker covers the widest range, VITS comes closest, the transformer-based systems fall in the middle, and Glow-TTS covers a limited range. Mismatched model–vocoder pairs (“degraded” synthesis) lose much of their coverage because noise breaks f0 tracking (§3.2, Figure 3). The paper reports this result only as a plot. No numerical range values are given.
- CPPs. Merlin reaches 12.24 dB, against 7.73 dB for the original speaker. The authors read this excess periodicity as the source of its robotic sound. The neural systems fall between 6.68 dB (FastSpeech 2) and 7.96 dB (Glow-TTS), and VITS (7.75 dB) is within 0.02 dB of the original speaker. Degraded synthesis reaches 9.73 dB with a large spread (§3.3, Table 4 in the paper).
- Spectrum balance. Glow-TTS (−14.89 dB) is the only system above the original speaker (−15.47 dB). Its difference map puts the gain in the softer SPL regions, which the authors interpret as better soft phonation. Merlin (−19.75 dB) and degraded synthesis (−31.32 dB) lose high-frequency energy (§3.3–3.4, Table 3, Figure 6).
- Crest factor. All working systems are within +0.10 of the original speaker (2.53). Degraded synthesis falls to 2.18, so a low crest factor mainly separates failed synthesis from working systems (§3.3, Table 2).
- Vocoders. With Tacotron 2 as the acoustic model, UnivNet stays closer to the original speaker than Multiband-MelGAN on both spectrum balance (−15.92 dB vs −16.80 dB) and CPPs (7.92 dB vs 7.15 dB). Its CPPs advantage is concentrated above about 65 dB SPL (§3.5, Tables 5–6, Figure 7).
None of these results is validated against listeners. The paper runs no listening test or MOS comparison, so links between metric values and perceived quality (“robotic”, “dull”, “natural”) are the authors’ interpretation, sometimes supported by citations to the voice literature.
Novelty Assessment
FonaDyn voice maps are the authors’ own earlier tooling from clinical voice research, and CPPs, crest factor, and spectrum balance are established voice-quality measures. The new element is applying voice mapping to synthetic speech and using difference maps against the original speaker as a system-level TTS diagnostic. That is a reasonable transfer of a method from another field. The empirical side is thin: one female speaker, six systems from 2016–2021 with no current zero-shot or LM-based TTS, in-training texts only, no perceptual validation, and only descriptive statistics (means and 95% confidence intervals with no significance tests). The suggested CPPs thresholds (7–8 dB natural, above 10 dB robotic) are drawn from one speaker, the Merlin outlier, and one degraded case. They are hypotheses, not calibrated cutoffs.
Field Significance
Low. The paper offers a visualization-based, objective diagnostic for how TTS systems cover a speaker’s pitch and loudness range and how voice quality varies across it, a dimension that utterance-averaged metrics miss. As evidence, it is an exploratory pilot study. Without perceptual validation or coverage of current systems, it shows that the tool can separate systems, not that it predicts perceived quality.
Claims
- supports: Utterance-averaged objective metrics hide how a synthetic voice covers a speaker’s pitch and loudness range. Mapping cycle-level voice-quality measures over the f0–SPL plane exposes differences in dynamic range between TTS systems.
Evidence: Voice-map coverage separates the original LJSpeech speaker (widest), VITS (closest to it), mid-range transformer-based systems, and Glow-TTS (narrow range), while mean crest factor differs by at most 0.10 across these same systems. (§3.2, Figure 3; §3.3, Table 2)
- supports: Excess periodicity is a measurable acoustic signature of older statistical-parametric synthesis compared with neural end-to-end TTS.
Evidence: Merlin’s mean CPPs is 12.24 dB against 7.73 dB for the original speaker. Neural systems trained on the same data (Tacotron 2, Transformer TTS, FastSpeech 2, Glow-TTS, VITS) fall between 6.68 and 7.96 dB, and VITS is within 0.02 dB of the speaker. (§3.3, Table 4; §3.4, Figures 4–5)
- refines: Neural vocoders differ in how faithfully they preserve periodicity and spectral tilt, and the differences depend on vocal effort rather than being uniform.
Evidence: With Tacotron 2 held fixed, UnivNet gives higher CPPs than Multiband-MelGAN mainly above about 65 dB SPL, while its spectrum-balance advantage becomes inconsistent below 70 dB SPL. (§3.5, Tables 5–6, Figure 7)
- complicates: Comparing synthetic and natural voices with voice-range methods is limited because TTS corpora carry no sound-level calibration.
Evidence: LJSpeech and most TTS datasets lack SPL calibration, so the authors have to assume system outputs are “self-calibrated”. The intensities they map are not physical SPL values, and level affects the voice-quality metrics. (§2.3; §4)
Limitations and Open Questions
Warning
The paper reports no listening test or other perceptual validation. Claims that particular CPPs ranges sound “natural” or “robotic”, or that higher spectrum balance means better soft phonation, are the authors’ interpretations of acoustic measures and have not been tested against listener judgments of these systems.
- All results come from one female speaker (LJSpeech) and texts drawn from the models’ own training data. Generalization to unseen text, multi-speaker or multilingual models, and zero-shot systems is untested. The authors acknowledge that comparing against the original voice does not suit models trained on mixed corpora (§4).
- The systems evaluated date from 2016–2021. Current flow-matching, diffusion, and codec-LM TTS systems are mentioned but not evaluated (§1.3).
- Statistics are limited to means, standard deviations, and 95% confidence intervals, with no significance testing. Voice-range differences are shown only as plots (§3.2–3.3).
- The SPL “self-calibration” assumption rests on observation and is not measured (§2.3, §4).
- Proposed extensions include jitter, shimmer, and HNR mapped over f0–SPL, application to voice conversion assessment, and voice maps computed during training (§4).
Wiki Connections
- Evaluation Metrics — Proposes voice mapping with cycle-level CPPs, crest factor, and spectrum balance as an objective, system-level TTS diagnostic alongside frame-level metrics such as MCD and PESQ.
- GAN Vocoder — Compares Multiband-MelGAN and UnivNet behind a fixed Tacotron 2 acoustic model, finding UnivNet closer to the original speaker in periodicity and spectral balance.
- Transformer Encoder-Decoder TTS — Evaluates Tacotron 2, Transformer TTS, and FastSpeech 2, which fall in the middle of the voice-range distribution with CPPs close to the original speaker.
- Tacotron 2 — One of the six systems evaluated and the fixed acoustic model for the vocoder comparison. Its MOS spread across papers (3.70 to 4.53) is cited as evidence that MOS is unstable.
- FastSpeech 2 — One of the six systems evaluated. It has the lowest CPPs of the neural systems (6.68 dB) and is also cited as an example of MOS variability across studies.
- Tacotron — Cited as the start of end-to-end neural TTS in the paper’s historical overview of the systems it profiles.
- A Survey on Neural Speech Synthesis — The review the authors point readers to for TTS model coverage beyond the six systems they evaluate.
- F5-TTS — Listed among recent systems the study deliberately does not evaluate, which marks how far its coverage reaches.