arXiv · 2026 · Preprint
Tajamul Ashraf et al. (KAUST / University of Kashmir / Gaash Lab, National Institute of Technology Srinagar) · → Paper · Demo: ✓ · Code: ✓
Introduces Bolbosh, the first dedicated open-source neural TTS system for Kashmiri, built by supervised cross-lingual adaptation of a flow-matching TTS backbone with an expanded grapheme vocabulary that explicitly preserves Perso-Arabic diacritics.
Problem
Kashmiri is spoken by roughly seven million people but has no dedicated neural TTS system. Existing multilingual TTS frameworks that unofficially cover Indic languages, such as IndicParler, transfer poorly to Kashmiri in zero-shot: the paper reports a MOS of only 1.86 for this baseline, with frequent vowel mispronunciations and prosodic distortion. The authors attribute this to two compounding factors: inadequate modeling of Perso-Arabic diacritics, which encode subtle but intelligibility-critical vowel distinctions, and phonotactic mismatch between Kashmiri and the higher-resource languages these multilingual models were trained on. A further obstacle is data: Kashmiri text-speech resources are scarce, fragmented across scripts (Perso-Arabic, Devanagari, Roman), and dialectally variable, which complicates alignment and generalization for any model trained from scratch.
Method
Bolbosh fine-tunes a pretrained multi-speaker English Matcha-TTS checkpoint using Optimal Transport Conditional Flow Matching (OT-CFM), rather than training a Kashmiri model from scratch. The architecture follows Matcha-TTS: a Transformer text encoder maps normalized grapheme sequences to contextualized representations; a duration predictor performs explicit, deterministic length regulation (rather than relying on autoregressive attention for alignment); pitch and energy predictors operate at the grapheme level and are expanded to frame level via the predicted durations to condition the decoder; and an OT-CFM decoder learns a continuous velocity field transporting a Gaussian prior to the target mel-spectrogram distribution, internally using Monotonic Alignment Search rather than an external aligner. A frozen, pretrained HiFi-GAN vocoder reconstructs the waveform. The training objective sums an L1 mel-reconstruction loss with MSE losses on duration, pitch, and energy predictions.
Two low-resource adaptation choices are central to the approach. First, cross-lingual initialization: starting from an English multi-speaker checkpoint rather than random weights provides a transferable acoustic prior and accelerates Monotonic Alignment Search convergence on the small Kashmiri corpus, but requires expanding the model’s grapheme vocabulary from the English character set to 272 symbols to cover Kashmiri characters and diacritics. Second, multi-speaker regularization: training combines a small, studio-quality corpus (RASA, 36.28 hours) with a larger but noisier, spontaneous multi-speaker corpus (IndicVoices-R Kashmiri split, 43.61 hours), each utterance carrying a learned speaker embedding; at inference the model is conditioned only on RASA speaker embeddings to preserve studio-level output quality while still benefiting from the phonetic and prosodic diversity IndicVoices-R contributes during training.
Because the two training corpora differ substantially in recording conditions, the authors apply a three-stage acoustic enhancement pipeline to the noisier IndicVoices-R data before training: (i) dereverberation and denoising via the Resemble-Enhance framework (a UNet-based denoiser with latent conditional-flow-matching refinement), (ii) dynamic silence trimming of segments below 40 dB peak amplitude, and (iii) loudness normalization to -23.0 LUFS (ITU-R BS.1770-4) followed by resampling to 22.05 kHz. On the text side, the pipeline performs Unicode canonicalization, number expansion, and character filtering while explicitly preserving pronunciation-critical diacritics; the system omits grapheme-to-phoneme conversion entirely, relying on the expanded grapheme vocabulary and end-to-end learning to map graphemes directly to acoustics, since standardized Kashmiri G2P resources do not exist.
Key Results
On objective metrics, Bolbosh reaches an MCD of 3.73 versus 4.73 for the IndicParler multilingual baseline, and a relative WER (rWER, computed against a proxy Kashmiri ASR system and normalized to that system’s own error rate on ground-truth audio) of 4.14% versus 46.75% for IndicParler when diacritics are preserved in the input. Removing diacritics degrades both systems, but far more severely for IndicParler (rWER rises to 100.32%) than for Bolbosh (13.23%), which the authors interpret as evidence that explicit diacritic modeling is what stabilizes the grapheme-to-acoustic mapping. On subjective evaluation, a 32-listener native-speaker MOS study (5-point scale) gives Bolbosh 3.634 ± 0.061 against IndicParler’s 1.864 ± 0.065, with ground-truth human recordings scoring 4.614 ± 0.059. A separate proxy-ASR benchmarking step (Table 2) is used only to select the ASR configuration for rWER computation, not to evaluate Bolbosh itself: the diacritics-removed IndicConformer RNN-T variant achieves the lowest WER (41.20%) among the ASR systems tested, and is adopted as the rWER reference model. All comparisons are against a single multilingual baseline (IndicParler); no comparison is made against other TTS architectures (e.g., VITS, Grad-TTS, FastSpeech 2) discussed in the related work.
Novelty Assessment
The architectural contribution is limited: Bolbosh reuses the Matcha-TTS OT-CFM architecture and HiFi-GAN vocoder without structural modification. The genuine contribution is a low-resource, cross-lingual adaptation recipe combined with the first dedicated Kashmiri speech corpus curation and training pipeline: cross-lingual checkpoint initialization, grapheme vocabulary expansion for a diacritic-rich Perso-Arabic script, a corpus-heterogeneity enhancement pipeline, and multi-speaker regularization that anchors inference to studio-quality speaker embeddings while training on a broader, noisier mixture. This is best characterized as an engineering-integration and dataset-contribution paper rather than an architectural one; its significance is establishing the first working, open-source TTS system and benchmark for a specific under-resourced language, and in doing so providing a case study for diacritic-sensitive low-resource adaptation more generally.
Field Significance
moderate — Bolbosh contributes a fully open-source TTS system, curated 79.9-hour training corpus, and evaluation benchmark for a specific under-resourced language (Kashmiri) that previously had no dedicated neural TTS system at all, and demonstrates that supervised cross-lingual flow-matching adaptation with explicit diacritic modeling can close most of the gap between unintelligible zero-shot transfer and near-human MOS. Its contribution is scoped to a single language and a single baseline comparison, so its significance is primarily as a reproducible low-resource TTS recipe and public resource rather than a new modeling technique.
Claims
- supports: Zero-shot multilingual TTS transfer can fail severely for languages whose script relies on diacritics the source model was never trained to represent, even when the target language shares a language family with better-resourced training languages.
Evidence: The IndicParler multilingual baseline, evaluated on Kashmiri without dedicated adaptation, scores a MOS of 1.864 ± 0.065 and an rWER of 46.75% (rising to 100.32% when diacritics are stripped from the input), against 3.634 ± 0.061 MOS and 4.14% rWER for the adapted system. (§5.4, §5.5, Table 3, Table 4)
- supports: Initializing a flow-matching TTS model from a pretrained checkpoint in a higher-resource language, combined with vocabulary expansion to cover the target language’s script, can enable stable alignment and competitive synthesis quality under limited paired target-language data.
Evidence: Fine-tuning a pretrained English multi-speaker Matcha-TTS (OT-CFM) checkpoint on 79.9 hours of Kashmiri speech, with the grapheme vocabulary expanded to 272 symbols to cover Kashmiri diacritics, yields an MCD of 3.73 and MOS of 3.634, described as establishing a new benchmark for the language. (§4.3, §5.4, §5.5)
- complicates: Combining heterogeneous training corpora of differing recording quality for low-resource TTS requires an explicit acoustic-domain-matching pipeline; naively mixing clean studio recordings with noisy spontaneous recordings risks destabilizing Monotonic-Alignment-Search-based alignment.
Evidence: The IndicVoices-R Kashmiri split (93.25% spontaneous recordings) exhibits noise, reverberation, and amplitude variation that the authors state can destabilize alignment; they apply a three-stage enhancement pipeline (dereverberation, silence trimming, LUFS loudness normalization) before mixing it with the studio-quality RASA corpus for training. (§3.1, §3.1.1)
- complicates: Proxy ASR-based intelligibility metrics for extremely low-resource languages can themselves be unreliable, limiting how much weight word error rate results can carry relative to subjective evaluation.
Evidence: All Kashmiri ASR systems benchmarked as WER proxies show high absolute error rates (41.20%-94.34% depending on model and diacritic condition), which the authors cite as justification for treating WER as a supplementary metric rather than a primary one, relying on MOS and MCD as the main evaluation criteria instead. (§5.2, §5.3, Table 2)
Limitations and Open Questions
The evaluation compares Bolbosh against a single multilingual baseline (IndicParler); no comparison is reported against other established low-resource TTS architectures discussed in the paper's own related work (e.g., VITS, Grad-TTS, FastSpeech 2), so the magnitude of improvement attributable specifically to the flow-matching architecture versus the adaptation recipe as a whole cannot be isolated from this paper alone.
The evaluation is limited to a single language and a single training/test split drawn from two corpora, one of which (RASA) supplies both training and the entire validation/test sets, so generalization to unseen Kashmiri dialects or speakers outside these corpora is untested. Model size is not reported. The proxy ASR system used for WER has itself has high absolute error on Kashmiri, and the paper does not report a listener demographic breakdown beyond native-speaker status and count (32 listeners). The authors note future work directions of multi-dialect modeling, enhanced prosody control, and extending the approach to other under-resourced languages, implying prosody control was not yet explicitly modeled beyond frame-level pitch and energy prediction.
Wiki Connections
- Flow Matching — applies OT-CFM within the Matcha-TTS framework as the core generative mechanism, without architectural modification, to a new low-resource adaptation setting.
- Zero-Shot TTS — demonstrates and quantifies a zero-shot multilingual transfer failure case (MOS 1.86) that motivates supervised cross-lingual adaptation instead.
- Multilingual TTS — extends multilingual TTS coverage to Kashmiri, a language unofficially or poorly supported by existing multilingual Indic TTS systems, via cross-lingual checkpoint adaptation rather than joint multilingual training.
- Speaker Adaptation — uses learned per-utterance speaker embeddings across two heterogeneous corpora during training while conditioning inference exclusively on a fixed subset of speaker embeddings for quality control.
- Subjective Evaluation — reports a native-speaker MOS listening study with 95% confidence intervals as the primary quality measure, alongside objective MCD and proxy-ASR rWER metrics.