arXiv · 2026 · Preprint
Parth Khadse et al. (TCS Research) · → Paper · Demo: ? · Code: ?
Trains matched and deliberately mismatched text-speech direction pairs (forward, reverse, and reverse-text/forward-speech) into Tacotron-2 and VITS-TTS to test whether end-to-end TTS training benefits from the anatomical constraints that shape natural human articulation.
Problem
End-to-end TTS systems learn a mapping from text to acoustic speech patterns from paired training data, and the field generally assumes the model implicitly absorbs the structural regularities of natural speech production, including the constraint that human articulators can only move smoothly between certain configurations. Prior work on visible-speech symbols and word difficulty has shown that these anatomical constraints make some sound sequences objectively harder for humans to articulate. What has not been tested is whether an end-to-end TTS system, which has no articulators and no anatomical constraints of its own, actually relies on or benefits from this structure when it learns to synthesize speech. If e2e-TTS training is purely a statistical sequence-to-sequence mapping, it should train just as readily on an “unnatural” pairing, such as reversed text mapped to reversed speech, as on the conventional forward pairing.
Method
The authors build three families of systems from the same pre-trained WaveGlow vocoder and seq2seq training pipeline, varying only the direction of the text and speech used during training: (a) e2e-TTS, the conventional forward-text-to-forward-speech baseline; (b) r-e2e-TTS, trained on reverse text mapped to reverse speech (reverse mel-spectrograms, later reversed back for analysis); and (c) rtfs-e2e-TTS, trained on reverse text mapped to the corresponding forward (i.e., mismatched) speech. Reverse text and reverse speech are both synthetically constructed by reversing the string or waveform, not collected from humans speaking backward.
Two architectures are used to test whether any effect is specific to autoregressive modeling: Tacotron-2, a two-stage recurrent seq2seq model with location-sensitive attention feeding a pre-trained WaveGlow flow-based vocoder, and VITS-TTS, a single-stage non-autoregressive model that integrates a VAE-style prior/posterior encoder, Monotonic Alignment Search (MAS) for text-speech alignment, and an adversarial (GAN-style) waveform decoder. All models are trained under identical hyperparameters (batch size 16, learning rate 2.5e-4, 50,000 iterations) on the LIMMITS’25 single-speaker English corpus. A separate ablation replaces character tokenization with byte-pair encoding (BPE) plus a length-regulator operator that expands each subword token to match its duration, to isolate the effect of tokenization granularity from the effect of text-speech direction.
Evaluation uses two channels: (1) objective transcription accuracy (WER, CER) via Whisper small.en and WhisperX small on 1701-1715 held-out LibriSpeech-960 sentences not seen during training, and (2) subjective ratings from five human listeners, comprising 5-point MOS scores for naturalness and intelligibility on 16 samples, plus a forced-choice paired preference test on 5 sentence pairs per listener (25 total judgments).
Key Results
On Tacotron-2, r-e2e-TTS outperforms the conventional e2e-TTS baseline by a wide margin: WER drops from 17.3% to 10.8% with Whisper (a 37.6% relative reduction), and the reverse-text/forward-speech rtfs-e2e-TTS variant does even better, reaching 9.0% WER. Naturalness and intelligibility MOS both favor r-e2e-TTS (3.75 vs. 3.40, and 3.58 vs. 3.38 respectively), and in a paired preference test, listeners chose r-e2e-TTS audio over e2e-TTS audio in 23 of 25 judgments (92%).
The same direction of advantage reproduces on VITS-TTS (11.8% vs. 12.4% WER), though the margin is much smaller than on Tacotron-2, indicating the effect is not confined to one architecture but its magnitude is architecture-dependent. The rtfs-e2e-TTS condition, however, fails outright on VITS-TTS: MAS enforces a strict monotonic left-to-right correspondence between text and audio, so training on mismatched (reverse-text, forward-speech) pairs produces no usable alignment and WER balloons to over 100%. BPE tokenization has a substantially larger effect on WER than text-speech direction does: switching from character to BPE tokens drops WER to 9.6% for the forward baseline (compared to 17.3% for the character-based baseline), and the residual reverse-vs-forward gap under BPE shrinks to 0.2 percentage points.
Novelty Assessment
The contribution is a training-recipe and experimental-design novelty rather than an architectural one: Tacotron-2 and VITS-TTS are used unmodified, and the manipulation is entirely in how the training pairs are constructed. The value of the paper lies in the counter-intuitive empirical result (unnatural text-speech pairings train at least as well as, and by several measures better than, natural pairings) replicated across two structurally different TTS architectures and two tokenization schemes. The evidence is real but the scope is narrow: a single speaker, a single language, a single dataset, and a small human-evaluation panel. The authors are candid that they do not know why reverse-trained systems transcribe and score better, leaving the mechanism unexplained.
Field Significance
Moderate — this paper provides a clean, cross-architecture negative result against the assumption that human articulatory naturalness materially benefits end-to-end TTS training: the systems tested behave as data-driven sequence mappers whose training success does not depend on the text-speech pairing resembling natural human articulation. The finding is consistent across an autoregressive and a non-autoregressive architecture, which strengthens it beyond a single-model curiosity, but the small evaluation panel, single-speaker/single-language setup, and unexplained mechanism keep the result at the level of a well-designed probing study rather than a result with immediate practical application.
Claims
- supports: End-to-end TTS training is a data-driven text-to-acoustic sequence mapping whose success does not depend on the text-speech pairing reflecting natural human articulatory structure.
Evidence: Systems trained on reverse text/reverse speech and on mismatched reverse-text/forward-speech pairs both converge and achieve equal or lower WER than the conventional forward-forward baseline on Tacotron-2 (10.8% and 9.0% vs. 17.3% WER, Whisper small.en). (§3, Table 2, Table 10)
- refines: The benefit of an “unnatural” text-speech training pairing over the conventional pairing is architecture-dependent in magnitude and can fail entirely under alignment mechanisms that impose strict monotonicity.
Evidence: The reverse-pairing WER advantage is large on Tacotron-2 (17.3% to 10.8%) but small on VITS-TTS (12.4% to 11.8%); the mismatched reverse-text/forward-speech condition trains successfully with Tacotron-2’s soft attention but fails outright with VITS-TTS’s Monotonic Alignment Search, which cannot align reversed text against forward speech (WER over 100%). (§3.2, §3.3, Table 9, Table 11)
- complicates: Text tokenization granularity can be a larger determinant of TTS transcription accuracy than the text-speech training direction.
Evidence: Switching from character to BPE tokenization lowers WER from 17.3% to 9.6% on the forward baseline, a far larger change than the 6.5-point gain from using reverse pairing with character tokens; under BPE, the reverse-vs-forward WER gap shrinks to just 0.2 percentage points. (§3.1, Table 8)
- complicates: Large gaps in automatic ASR-based intelligibility proxies do not necessarily correspond to equally large gaps in subjective naturalness ratings, and small-panel listening tests may need a forced-choice preference format to reliably detect the difference.
Evidence: A 37.6% relative WER reduction between conditions corresponds to a comparatively modest MOS gap on a 5-point scale (3.75 vs. 3.40 naturalness, 3.58 vs. 3.38 intelligibility), while a forced-choice paired preference test with the same five listeners cleanly separates the two systems (23/25 judgments). (§3, Table 2, Table 4, Table 5)
Limitations and Open Questions
Warning
The paper’s central manipulation, “reverse speech,” is a synthetically time-reversed waveform, not audio of a human actually articulating in reverse. The authors’ own follow-up experiment (Appendix C) finds that human subjects cannot naturally produce genuine reverse speech at all, and that recordings of their attempts sound unintelligible even when played back in reverse. This means the study’s negative result shows that e2e-TTS training tolerates an artificial signal manipulation, not that it is indifferent to genuine human articulatory constraints, which is a narrower and less direct test of the paper’s stated hypothesis than the framing suggests.
The quantitative study covers a single speaker, a single language (English), and a single training corpus (LIMMITS’25, ~40 hours), so it is unclear whether the results hold with multi-speaker or multilingual data. The human evaluation is limited to five listeners rating a small number of samples (8-16 for MOS, 5 pairs each for preference), well below typical listening-test sample sizes. All experiments use read speech; the authors explicitly flag that they do not know whether the same pattern would hold for spontaneous speech, which carries different pause and disfluency structure. Finally, the paper does not offer a mechanistic explanation for why reverse-trained systems are transcribed more accurately and rated more natural than forward-trained ones, leaving this as an open question for future work.
Wiki Connections
- Evaluation Metrics — uses WER/CER from two ASR systems (Whisper, WhisperX) as an intelligibility proxy alongside MOS, and finds that the two evaluation channels can diverge in effect size even when they point in the same direction.
- Subjective Evaluation — runs 5-point MOS ratings for naturalness and intelligibility plus a forced-choice paired preference test with five human listeners to corroborate its ASR-based findings.
- GAN Vocoder — trains VITS-TTS, whose non-autoregressive decoder is trained with an adversarial discriminator, as a second architecture to test whether the reverse-pairing effect generalizes beyond Tacotron-2’s autoregressive design.