arXiv · 2025 · Preprint
Ranjbar Kalahroodi et al. (University of Tehran) · → Paper · Demo: ✗ · Code: ?
Introduces ParsVoice, a 1,804-hour multi-speaker Persian TTS corpus built by an automated pipeline that turns raw audiobooks into sentence-aligned, quality-filtered training data, and validates it by fine-tuning a zero-shot multilingual TTS model on the resulting speech.
Problem
Persian speech resources for TTS lag far behind English and other high-resource languages. The paper surveys existing Persian corpora and finds them small and largely single-speaker: DeepMine-Multi-TTS offers 120 hours across 67 speakers, ArmanTTS provides 9 hours from a single speaker, and ManaTTS offers 86 hours from one speaker, while the largest available Persian resource, DeepMine+, is commercially restricted. Multilingual efforts such as Common Voice, Multilingual LibriSpeech, and VoxPopuli remain skewed toward European languages, and Common Voice’s Persian portion is described as insufficient quality for TTS. This scarcity of scale, speaker diversity, and open accessibility constrains multi-speaker Persian TTS development in particular, since TTS (unlike ASR) requires clean, precisely aligned audio-text pairs rather than merely large volumes of noisy data.
Method
The paper’s contribution is an automated corpus-construction pipeline applied to public-domain audiobooks from IranSeda, plus the resulting ParsVoice corpus. The pipeline has four stages. First, intelligent audio segmentation combines WebRTC voice activity detection, transcription via the Google Speech-to-Text API, and a BERT-based sentence-completion classifier (a ParsBERT model fine-tuned on synthetically truncated Persian sentences, reaching F1 97.4) that iteratively extends segment boundaries in 0.1-second increments until a complete sentence is captured. Second, a boundary optimization algorithm removes residual silence or artifacts at segment edges through a two-stage binary-search-then-linear-search strategy that re-transcribes trimmed audio until the transcription starts to diverge from the original. Third, a text-audio quality assessment framework scores each segment on Persian-specific text criteria (character validity, length, repetition, phonetic coverage) and standard audio criteria (SNR, dynamic range, spectral features, clipping, silence, background-music presence via inaSpeechSegmenter), each normalized to [0,1] and combined into weighted composite scores used to filter the corpus. Fourth, speaker identification assigns consistent global speaker labels using ECAPA-TDNN embeddings in a two-stage local-diarization-then-global-clustering pipeline (UMAP dimensionality reduction, consensus cluster-count estimation, agglomerative and spectral clustering), needed because roughly 40% of audiobook metadata lacked narrator names.
Running this pipeline over 2,000 of 3,807 available audiobooks yields 3,526 hours of clean speech from 5,158,344 initial segments, filtered down to a 1,804-hour high-quality subset (audio quality ≥ 0.8, text quality ≥ 0.5) spanning 470+ speakers, with a separately fine-tuned ParsBERT punctuation-restoration model (F1 91.33%) applied to reconstruct missing punctuation in the final text.
To validate the corpus for TTS training, the authors fine-tune XTTS, an existing zero-shot multilingual TTS model that operates directly on raw text without phoneme input, on the high-quality ParsVoice subset. They extend the GPT vocabulary with 2,500 Persian BPE tokens and fine-tune with batch size 16 for 170,000 steps. No new TTS architecture, conditioning mechanism, or training objective is proposed; the model itself is XTTS, used unmodified apart from vocabulary extension and Persian fine-tuning.
Key Results
On 90 synthesized samples from unseen Persian speakers, judged by 40 human raters, the fine-tuned XTTS model achieves a naturalness MOS of 3.6/5 and a Speaker Similarity MOS of 4.0/5 (Table 3). Objective metrics give WER 22.57% and CER 12.78% (scored with Google STT), and an ECAPA-TDNN cosine speaker-similarity score of 80%. The paper compares these numbers to FastSpeech2 End-to-End (MOS 3.72, SMOS 4.02) and FastSpeech2 Cascaded (MOS 3.34, SMOS 3.81), both trained on the separate DeepMine dataset by prior work, and characterizes its own result as “competitive.” At the dataset level, the boundary optimization algorithm removes 442.73 hours (11.2%) of unwanted silence and artifacts from the pre-filtered corpus, and automatically assigned global speaker IDs match known narrator labels with 97.0% consistency on the subset where narrator metadata exists. Table 2 reports 1,804 hours across 470+ speakers for ParsVoice against 86 hours/1 speaker for ManaTTS and 120 hours/67 speakers for DeepMine-Multi-TTS, a roughly 10x scale increase over the largest prior open Persian TTS corpus by hours and by speaker count.
Novelty Assessment
The contribution is a dataset and its construction pipeline, not a TTS architecture. XTTS is used unmodified except for Persian vocabulary extension and fine-tuning, so the reported MOS/SMOS numbers validate the corpus’s usability rather than demonstrate a new modeling technique. Within the pipeline, individual components (WebRTC VAD, Google STT, ECAPA-TDNN speaker embeddings, inaSpeechSegmenter) are existing off-the-shelf tools; the genuinely new pieces are the BERT-based sentence-completion classifier used to enforce segment linguistic integrity, and the binary-search boundary optimization algorithm for precise audio-text trimming, both engineering additions rather than novel model architectures. The corpus’s scale (1,804 high-quality hours, 470+ speakers) is a real and substantial advance over prior open Persian TTS resources, which is the paper’s central and most defensible claim.
Field Significance
Moderate — this paper closes a specific, well-documented resource gap for Persian TTS by releasing a corpus an order of magnitude larger than prior open alternatives, together with a reusable pipeline for turning audiobook archives into TTS-ready data for other low-resource languages. Its significance is bounded to a single-language resource and validation pipeline rather than a new modeling or evaluation technique that generalizes beyond corpus construction.
Claims
- supports: Automated pipelines combining voice-activity detection, ASR-based re-transcription, and a linguistic completeness classifier can convert long-form audiobook recordings into TTS-ready segments at large scale without manual sentence-boundary annotation.
Evidence: The three-phase segmentation pipeline (WebRTC VAD, Google STT transcription, ParsBERT-based sentence-completion classifier with iterative 0.1-second boundary extension) processed 2,000 audiobooks into 5,158,344 initial segments, yielding 3,526 hours of clean speech before quality filtering. (§3.2, §4.1, Table 1)
- supports: Fine-tuning an existing zero-shot multilingual TTS model on a newly constructed low-resource-language corpus can produce naturalness and speaker-similarity quality comparable to native single-language TTS systems trained on smaller proprietary datasets.
Evidence: XTTS fine-tuned on the 1,804-hour ParsVoice subset achieves MOS 3.6/5 and SMOS 4.0/5 on unseen Persian speakers, compared to MOS 3.72/SMOS 4.02 and MOS 3.34/SMOS 3.81 for two FastSpeech2 systems trained on the separate DeepMine corpus. (§5, Table 3)
- complicates: Automatic ASR-based intelligibility metrics can understate the quality of synthesized speech in low-resource languages when the scoring ASR system itself is undertrained on synthetic voices in that language.
Evidence: The fine-tuned system scores WER 22.57% and CER 12.78% via Google STT, yet human raters scored text accuracy at 4.0/5; the authors attribute the gap to Google STT’s limited exposure to synthetic Persian speech. (§5)
- complicates: Speaker identity labels derived from unsupervised diarization and clustering over large multi-narrator audiobook corpora do not perfectly recover ground-truth narrator identity, so speaker-conditioned training data built this way carries residual label noise.
Evidence: The two-stage ECAPA-TDNN diarization-and-clustering pipeline achieves 97.0% consistency with known narrator labels only on the subset of audiobooks that had usable narrator metadata (roughly 60% of the corpus), with the remainder unverifiable against ground truth. (§3.5, §4.1)
Limitations and Open Questions
Warning
The paper’s headline “competitive performance” claim (Table 3) compares its fine-tuned XTTS system against FastSpeech2 systems evaluated in prior work on a different dataset (DeepMine), with different raters and evaluation protocols. This is not a controlled head-to-head comparison, and the reported MOS/SMOS differences should not be read as a direct system ranking.
The corpus’s speaker metadata is incomplete: about 40% of audiobooks lack narrator names, and the true number of underlying speakers is estimated via clustering rather than known ground truth for a large share of the data. Gender balance (33% female, 67% male) is derived only from the subset of narrator names that are available, so it may not represent the full corpus. The automatic quality-filtering thresholds (audio ≥ 0.8, text ≥ 0.5) are empirically weighted heuristics validated only through the downstream TTS fine-tuning result on this single corpus, not through a systematic ablation of threshold choices. The paper states the dataset is publicly released but does not report whether the pipeline’s own code is released alongside it.
Wiki Connections
- Zero-Shot TTS — validates the corpus by fine-tuning a zero-shot multilingual TTS model and evaluating it on unseen Persian speakers, testing whether the corpus supports zero-shot speaker generalization rather than only fixed-speaker synthesis.
- Speaker Adaptation — frames the fine-tuned XTTS result as opening “rapid speaker adaptation” possibilities for Persian, since the corpus’s 470+ speaker diversity is intended to support multi-speaker and adaptable TTS training.
- Subjective Evaluation — validates corpus quality through a 40-rater MOS/SMOS listening study on synthesized held-out-speaker samples rather than relying on automatic metrics alone.
- Evaluation Metrics — builds Persian-specific composite audio- and text-quality scoring frameworks used to filter the corpus, combining signal-level and linguistic criteria into weighted thresholds rather than adopting a single off-the-shelf metric.
- XTTS — used unmodified as the TTS backbone, fine-tuned on ParsVoice with an extended Persian vocabulary to validate the corpus’s downstream training utility.
- ManaTTS — cited as the largest prior open single-speaker Persian TTS corpus (86 hours), against which ParsVoice’s 1,804-hour, 470+-speaker scale is directly compared.