Interspeech · 2025 · Conference
Toyin et al. (Mohamed Bin Zayed University of Artificial Intelligence) · → Paper · Demo: ? · Code: ?
ArVoice is the largest open-source multi-speaker Modern Standard Arabic speech corpus for TTS, providing 83.52 hours across 11 voices (7 human, 4 synthetic) with diacritized transcriptions, and demonstrating through baseline experiments that diacritization significantly improves Arabic TTS intelligibility and quality.
Problem
Arabic TTS research is constrained by a scarcity of clean, multi-speaker, diacritized speech corpora in Modern Standard Arabic (MSA). Existing open-source Arabic datasets were designed primarily for ASR, drawing from noisy broadcast sources that introduce variable recording quality and overlapping speech unsuitable for synthesis. The two existing TTS-focused resources (ASC and ClArTTS) are both single-speaker, limiting multi-speaker model development. A further complication is the absence of diacritics in most Arabic text resources: diacritics encode vowel information that is critical for unambiguous pronunciation, and their omission leads to underspecified inputs and unintelligible synthesis. Text-based diacritization tools trained on Classical Arabic have been shown to produce high error rates on MSA transcripts, making this a persistent practical obstacle.
Method
ArVoice is constructed in four parts. The first two parts consist of newly commissioned professional recordings. Part 1 covers approximately 1,800 fully diacritized MSA sentences drawn from the Al Jazeera split of the Tashkeela corpus, recorded by four professional voice talents (two male, two female) from Egypt, Jordan, and Morocco, with each speaker recording a non-overlapping text subset. Part 2 uses 900 non-diacritized sentences from the Khaleej news corpus, recorded by two additional speakers (one male from Palestine, one female from Egypt) who vocalized the text according to their natural pronunciation. Part 3 is a modified version of the existing Arabic Speech Corpus (ASC): GPT-4 was used to reverse non-standard transcript normalizations introduced for concatenative synthesis (such as spelled-out nunation), followed by manual correction and validation against three ASR models. The correction reduced WER on the ASC test set by approximately 40 percentage points (from ~45% to ~7% CER). Part 4 consists of parallel synthetic speech generated by Google’s Standard and WaveNet commercial TTS models for the same texts used in Parts 1 and 3, covering four synthetic voices intended for data augmentation and parallel voice conversion.
Three open-source TTS models are fine-tuned on the human speech portions: ArTST-tts (a transformer-based Arabic speech model using x-vector speaker embeddings and a HiFi-GAN vocoder), VITS (VAE with normalizing flows and adversarial training, end-to-end), and Fish-Speech (a dual autoregressive architecture using grouped finite scalar vector quantization and an LLM for linguistic feature extraction). Two voice conversion models are also trained: AAS-VC (non-autoregressive sequence-to-sequence, parallel) on synthetic Part 4 data, and KNN-VC (nearest-neighbor frame replacement, non-parallel) on human Parts 1 and 3.
Key Results
Pairwise preference tests with native Arabic speakers show that VITS trained with diacritics is significantly preferred over VITS trained without (statistically significant, 95% CI). For ArTST-tts, the preference for the non-diacritized variant is not statistically significant. Automatic intelligibility evaluation using WER confirms diacritics help: VITS with diacritics reaches 20.67% WER versus 35.69% without (Table 4). Fish-Speech fails on Arabic entirely after fine-tuning, producing WER above 100%, and is excluded from further analysis.
A second preference test comparing the best VITS variant (with diacritics) against the best ArTST variant (without diacritics) reveals a strong and statistically significant preference for VITS, despite ArTST being purpose-built for Arabic. Multi-speaker VITS is also strongly preferred over single-speaker VITS trained on the same target speaker’s 2.69 hours, demonstrating a benefit from pooling data across speakers.
Synthetic data augmentation (Part 4 combined with human speech) reduces WER by 8.2 percentage points absolute compared to human-speech-only VITS training, and subjective preference tests favor the augmented model (44% preference vs 30% for baseline).
For voice conversion, AAS-VC achieves speaker similarity of 0.72 and false acceptance rate of 0.95 at a 0.5 threshold; KNN-VC reaches 0.69 SS and 0.81 FAR.
Novelty Assessment
The contribution is primarily the dataset itself. The construction methodology is careful: text from multiple registers is sourced and pre-processed, professional recording standards are followed, demographic diversity across speaker origin is considered, and transcript correction for the ASC subset is validated quantitatively with ASR models. The use of GPT-4 to reverse concatenative-synthesis normalizations in ASC transcripts is a pragmatic engineering step that substantially improves transcript accuracy.
The baseline experiments are not methodologically novel; all five systems are existing open-source models. Their value is demonstrating dataset utility and surfacing practically important findings: diacritics matter for Arabic TTS, synthetic augmentation helps, and Arabic-specialized models do not automatically outperform general-purpose ones given sufficient fine-tuning data.
Field Significance
Moderate — ArVoice addresses a genuine bottleneck in Arabic TTS research by providing the largest open-source multi-speaker MSA corpus designed for synthesis, with diacritized transcriptions and parallel synthetic speech for augmentation. Its empirical baselines establish concrete reference points for diacritics-dependent quality and cross-system comparisons that future Arabic TTS work can build on.
Claims
-
supports: Diacritization of Arabic text input significantly improves TTS intelligibility and subjective quality in Modern Standard Arabic synthesis.
Evidence: VITS trained with diacritics achieved 20.67% WER versus 35.69% without, and was the only model with a statistically significant preference for the diacritized variant in pairwise preference tests with native speakers. (§4.1, Table 4)
-
supports: Multi-speaker training improves synthesis quality for individual target speakers relative to speaker-specific training on equivalent data volume.
Evidence: Multi-speaker VITS was strongly preferred over single-speaker VITS trained on the same speaker’s 2.69 hours; both models were evaluated on the same speaker in a pairwise preference test. (§4.1, Figure 2b)
-
supports: Synthetic speech data augmentation can improve TTS intelligibility even when the augmenting speech is of variable quality.
Evidence: Training VITS on all ArVoice parts including synthetic speech (Google Standard and WaveNet TTS) reduced WER by 8.2% absolute versus human-speech-only training, with pairwise preference also favoring the augmented model (44% vs 30%). (§4.1)
-
complicates: Multilingual LLM-based TTS systems do not reliably transfer to Arabic without adequate language-specific training data or adaptation, and can produce unintelligible output despite fine-tuning.
Evidence: Fish-Speech, a dual-autoregressive LLM-based multilingual system, achieved WER above 100% on Arabic after fine-tuning on ArVoice human speech, and was excluded from subjective evaluation due to confirmed low intelligibility. (§4.1, Table 4)
Limitations and Open Questions
Warning
The professionally recorded portions of ArVoice (Parts 1 and 2) are not freely downloadable; access requires signing a formal Data Usage Agreement. Only the ASC-derived and synthetic portions are available under CC BY 4.0. This limits reproducibility for experiments requiring the full human-voice data.
Human speech totals approximately 10 hours across 7 speakers, which is sufficient for fine-tuning but constraining for zero-shot multi-speaker TTS or large-scale model training. Coverage is restricted to Modern Standard Arabic; dialectal varieties are not represented, leaving adaptation to Egyptian, Gulf, or Moroccan Arabic as an open problem. Each preference test uses 25 samples evaluated by at least 10 raters per pair, which is adequate for detecting large differences but may lack power for subtle distinctions. The evaluation of VC systems relies on automated speaker verification similarity scores rather than human perceptual evaluation of voice identity.
Wiki Connections
- Multilingual TTS — ArVoice expands open-source speech synthesis resources to Modern Standard Arabic, directly supporting non-English TTS research.
- Subjective Evaluation — Pairwise preference tests with native Arabic speakers recruited via Prolific are used to compare diacritized versus non-diacritized TTS and to rank competing systems.
- Evaluation Metrics — WER computed using Arabic ASR models (ArTST, Whisper, Nvidia CTC-large) serves as an automated intelligibility proxy for TTS output quality.
- Voice Conversion — AAS-VC (parallel) and KNN-VC (non-parallel) are trained on ArVoice to demonstrate the dataset’s utility for Arabic VC research.