arXiv · 2025 · Preprint
Abdul Rehman et al. (Bournemouth University) · → Paper · Demo: ✗ · Code: ✓
Introduces the Bournemouth Forced Aligner (BFA), a CTC-based multilingual phoneme aligner that predicts both phoneme onset and offset boundaries up to 240x faster than Montreal Forced Aligner (MFA), while explicitly modelling inter-phoneme silence gaps.
Problem
Forced alignment (assigning precise time boundaries to phonemes given a known transcript) underpins TTS training pipelines that need phoneme durations, as well as audio-visual synchronization and prosody research. The dominant tool, Montreal Forced Aligner (MFA), relies on HMM-GMM machinery that is slow to run at scale and language-specific in its dictionaries. Neural alternatives built on ASR backbones (Charsiu, WhisperX) are faster but have been shown in prior comparative studies to underperform MFA on pure alignment accuracy. Existing aligners, including MFA, also universally assume phonemes are acoustically contiguous, discarding the short silences and prosodic gaps that occur between them.
Method
BFA combines three components: a Contextless Universal Phoneme Encoder (CUPE, adapted from the authors’ own prior work) for framewise acoustic feature extraction, an espeak-ng-based multilingual phonemization module that maps IPA symbols to a data-driven phoneme inventory, and a CTC-based dynamic-programming decoder for temporal alignment. The encoder is trained with a multi-task objective using separate prediction heads for a 67-class phoneme set and a 17-class coarse phoneme-group set (plus CTC blank tokens), enabling alignment at both fine- and coarse-grained levels. Three model variants are trained on different corpora to probe language specificity: an English-optimized model (LibriSpeech), a European multilingual model (Multilingual LibriSpeech across 7 non-English languages), and a universal model (Multilingual Spoken Words Corpus across 35 non-English languages).
The alignment decoder formulates a CTC state path that alternates blank and phoneme states and applies Viterbi-style dynamic programming over log-probabilities. On top of standard CTC decoding, BFA adds four algorithmic strategies: logarithmic probability boosting (factor β = 5.0) for expected phonemes to counter cross-lingual underrepresentation; a probability floor (ε = 10⁻⁸) to prevent target phonemes from being eliminated entirely; hierarchical decoding, which splits an utterance into independent sub-problems at detected silence regions and Viterbi-decodes each region separately; and a post-processing completeness guarantee that inserts any missing target phoneme at its most probable frame. Unlike conventional aligners that predict only phoneme onsets, BFA predicts both onset and offset boundaries per phoneme, so the gap between one phoneme’s offset and the next phoneme’s onset becomes an explicit, adjustable-tolerance inter-phoneme silence estimate.
Key Results
On TIMIT, BFA (English-optimized) reaches 71.4/84.6/87.9% recall at 20/40/60ms tolerance versus MFA’s 71.9/81.2/82.8%, i.e., comparable recall at tight tolerance and higher recall as tolerance relaxes; on Buckeye, BFA reaches 63.7/73.2/76.3% recall versus MFA’s 58.1/70.8/74.8% at the same tolerances (Table 2). This comes at a precision cost: because BFA predicts roughly twice as many boundaries (onset and offset) as MFA’s onset-only annotations, its 20ms onset precision is markedly lower (55.6% vs. 81.2% on TIMIT; 48.6% vs. 58.5% on Buckeye), though precision converges closer to MFA at 60ms tolerance (95.3% vs. 98.9% on TIMIT).
The multilingual variants trained without any English data (BFA_eu on 7 European languages, BFA_world on 35 languages) achieve recall within 1-11 points of the English-optimized BFA_en when evaluated on English test sets, with BFA_eu nearly matching BFA_en (71.0 vs. 71.4% recall at 20ms on TIMIT) while BFA_world trails more substantially (60.9% recall at 20ms on TIMIT).
The headline efficiency result is processing speed: BFA aligns a 10-second clip in roughly 0.2-0.25s, compared to roughly 45-60 minutes for a 535-second Buckeye clip and 1 minute for a 3.1-second TIMIT clip under MFA, yielding a measured 45-242x total-time speedup depending on corpus, with BFA running at 0.05-0.1x real-time factor versus MFA’s 52-194x (Table 1). Dual boundary prediction reveals that 30-40% of phonemes in both TIMIT and Buckeye have a measurable preceding inter-phoneme gap (Table 3), a structural detail invisible to onset-only aligners. An ablation on the universal model shows that probability boosting is necessary to prevent recall collapse (35.4% to 71.2% at 20ms with vs. without boosting on TIMIT) while having negligible effect on the English-only model, and that completeness enforcement gives a small additional recall gain on the lower-baseline universal model (Table 4).
Novelty Assessment
The contribution is primarily engineering integration and algorithmic refinement rather than new architecture: the acoustic encoder (CUPE) is adapted from the authors’ own prior publication, and CTC-based Viterbi decoding for alignment is a well-established technique. The genuinely new elements are the decoding-side heuristics assembled here (probability boosting, floor enforcement, hierarchical silence-aware decoding, completeness guarantee) and the explicit dual onset/offset boundary formulation, which is a real departure from the onset-only convention used by MFA, Charsiu, and WhisperX. The speed advantage (45-242x) is large and well-documented against a properly benchmarked baseline. The cross-lingual generalization claim, however, is evaluated only on English test sets; the paper itself acknowledges that “evaluation on additional languages would further validate this approach,” so the multilingual claim rests on training-corpus composition rather than on multilingual test accuracy.
Field Significance
Moderate — this paper provides a fast, open, multilingual alternative to MFA-style forced alignment with explicit modeling of inter-phoneme silence, a structural detail existing aligners discard by construction. Its contribution is an engineering and evaluation one: it demonstrates that a CTC-based neural aligner can match or exceed MFA’s recall at relaxed tolerances while running orders of magnitude faster, which is directly useful for TTS pipelines that need phoneme-level duration labels at scale or in interactive settings.
Claims
- supports: Neural CTC-based forced aligners can achieve substantial inference speedups over classical HMM-GMM aligners while retaining competitive recall at practical tolerance levels.
Evidence: The proposed aligner processes a 10-second clip in ~0.2-0.25s versus roughly 45-60 minutes for a comparable Buckeye clip under MFA, a measured 45-242x total-time speedup, while matching or exceeding MFA’s recall at 40ms and 60ms tolerance on both TIMIT and Buckeye. (§3, Table 1, Table 2)
- complicates: Aligners that predict both phoneme onset and offset boundaries trade precision for recall relative to onset-only baselines, because doubling the predicted boundary count increases the chance of spurious matches against onset-only ground truth.
Evidence: At 20ms tolerance on TIMIT, the proposed aligner’s onset precision drops to 55.6% versus MFA’s 81.2%, even though its recall is comparable, because the model predicts roughly double the boundary count that onset-only annotations penalize as insertions. (§4 “End Boundary”, Table 2)
- supports: Multilingual phoneme representations trained without target-language data can generalize to unseen-language alignment tasks with only modest performance degradation.
Evidence: Aligner variants trained exclusively on non-English multilingual corpora (7-language and 35-language configurations) achieve recall within 1-11 percentage points of an English-only trained variant when evaluated on English test sets, with the 7-language variant nearly matching it (71.0% vs. 71.4% recall at 20ms on TIMIT). (§4 “Cross-lingual Performance”, Table 2)
- refines: Natural speech contains measurable silent gaps between phonemes that conventional onset-only forced alignment discards by assuming acoustic contiguity.
Evidence: Explicit dual boundary prediction shows 30-40% of phonemes on both TIMIT and Buckeye have a measurable inter-phoneme gap, with median gap durations of 39-41ms on TIMIT and 39-40ms on Buckeye. (§4 “End Boundary”, Table 3)
Limitations and Open Questions
Warning
Cross-lingual generalization is validated only on English test sets. The multilingual and universal variants are trained on non-English corpora but are never evaluated on non-English alignment accuracy in this paper, so the claim that the universal phoneme representation “generalizes across languages” rests on training-data composition, not on measured performance in those languages.
The dual boundary formulation roughly doubles predicted boundary count relative to onset-only ground truth annotations, which mechanically depresses precision scores at tight tolerances and complicates direct precision comparison with onset-only aligners. Evaluation is restricted to two English-language benchmarks (TIMIT, a clean read-speech corpus, and Buckeye, spontaneous conversational speech); performance on tonal languages and other typologically distant languages is left to future work, as the authors themselves note.
Wiki Connections
- Multilingual TTS — trains and evaluates three aligner variants across English-only, 7-language European, and 35-language universal phoneme inventories, testing how well multilingual training generalizes to a fixed evaluation language.
- Evaluation Metrics — benchmarks phoneme-level recall, precision, and boundary-distance accuracy against Montreal Forced Aligner across two standard alignment corpora, extending typical alignment evaluation with explicit inter-phoneme gap statistics.
- Streaming TTS — frames its 45-242x speedup over MFA as enabling forced alignment in interactive and streaming speech pipelines that were previously constrained by slow, non-real-time aligners.