ArabicNLP 2025 · 2025 · Workshop
Majd Hawasly et al. · → Paper · Demo: ? · Code: ?
Introduces an end-to-end multi-task grapheme-to-phoneme model for Egyptian Arabic that jointly restores diacritics and substitutes dialect-specific “special sounds” before mapping to phonemes, and releases the first test set combining diacritization and verbatim pronunciation for Egyptian tweets.
Problem
Grapheme-to-phoneme (G2P) conversion is a required front-end step for both TTS and pronunciation assessment, but nearly all prior G2P work targets standard, well-resourced language varieties. Dialectal Arabic is the primary spoken register for most Arabic speakers, yet it lacks standard orthography: written dialectal text typically omits diacritics (short vowels) and, crucially, some spoken sounds have no direct Arabic-alphabet mapping or are pronounced differently from their written form depending on the dialect (e.g., a letter realized as one consonant in Egyptian and a different one in Levantine). Prior Arabic G2P and diacritization work was dialect-specific, rule-based, or focused on Modern Standard Arabic, leaving no generic method that jointly handles vowel restoration and dialect-specific sound substitution for a dialect without standard orthography.
Method
DialG2P treats dialectal G2P as three chained subtasks over an undiacritized dialectal input: (i) restore the diacritical marks (short vowels), (ii) substitute characters that only occur in the spoken form of the dialect with dialect-specific “special sound” characters added to the vocabulary, and (iii) deterministically map the resulting character sequence to phonemes via a 1-to-1 character-to-phoneme table.
Input text is preprocessed following the convention of prior Arabic diacritization work (Mubarak et al., 2019a; 2019b): a repeated sentence-start token and a repeated sentence-end token are added, and a sliding window of size seven extracts overlapping seven-word/token lines, which are then tokenized to individual letters with a special word-separator symbol. The core model is an attention-based sequence-to-sequence Transformer encoder-decoder (Vaswani et al., 2017) with six layers, 512 hidden units, and eight self-attention heads per layer. It is multi-task trained to jointly predict the diacritic mark for each letter and, based on context, to substitute certain letters with the special-sound characters. Because the sliding window presents each word up to seven times with different surrounding context, a majority-voting step over the overlapping predictions determines the final per-letter output for every word before the deterministic character-to-phoneme mapping is applied.
Training uses 10,000 manually diacritized Egyptian Arabic tweets sampled from the QADI corpus (Abdelali et al., 2021), split 80/10/10 into train/validation/test. A native-speaker linguist fully diacritized the text and transcribed the verbatim (dialectal) pronunciation; a hand-crafted substitution rule set, derived from the frequency statistics of special-sound usage in the annotated data, supplies the special-sound training labels. The transformer is trained for 300,000 steps with batch size 512 using the LazyAdam optimizer. To characterize dialectal variation in special-sound usage, the authors also collected and annotated 10,000 Levantine Arabic tweets from QADI in the same way, but this Levantine data is used only for the special-sound frequency analysis (e.g., a letter pronounced as one sound 98.5% of the time in Egyptian tweets versus 2.46% of the time in Levantine tweets), not for training or evaluating the DialG2P model itself.
The paper benchmarks DialG2P against: a diacritization-only Transformer with the identical architecture and data split; that same Transformer combined with a rule-based default special-sound (defSS) substitution; GPT-4 prompted zero-shot and 10-shot to diacritize the input (with and without the defSS fallback, since GPT-4 alone handles special sounds poorly); and a “Transformer cascade” that chains the diacritization Transformer with the same special-sound rule set used to generate DialG2P’s training labels.
Key Results
On a held-out test set of 1,000 Egyptian Arabic tweets, DialG2P reaches a word error rate (WER) of 5.15%, phoneme error rate (PER) of 1.71%, diacritic error rate (DER) of 1.67%, and character error rate (CER) of 0.05% — the last of these the lowest CER of any system tested. These results place DialG2P essentially on par with the two-stage Transformer cascade (WER 5.11%, PER 1.70%, DER 1.62%, CER 0.09%) despite DialG2P being a single unified model rather than two chained components, and clearly ahead of the diacritization Transformer augmented only with the default special-sound rule (Transformer+defSS: WER 6.32%, PER 2.02%, DER 1.62%, CER 0.41%). DialG2P does show a small regression in diacritic error rate relative to the single-task diacritization Transformer (1.67% vs. 1.62%), which the authors attribute to reduced task-specific capacity from multi-task training.
GPT-4 prompting lags far behind on this task even with the special-sound fallback: zero-shot GPT-4 reaches WER 47.57% (40.71% with defSS), and 10-shot in-context prompting improves this only to WER 33.66% (25.14% with defSS) — roughly 5-8x worse WER than DialG2P or the specialized baselines. The authors note that GPT-4 alone does not restore special sounds well at all, and results are reported only with the manual defSS fallback applied on top of its diacritization output.
Novelty Assessment
The underlying architecture is an off-the-shelf attention-based Transformer encoder-decoder (Vaswani et al., 2017); there is no new architectural component. The genuine contribution is the task formulation: combining diacritic restoration and dialect-specific special-sound substitution into a single multi-task model for a dialect that has no standard orthography, evaluated against both a matched cascaded pipeline and a general-purpose LLM. This is best characterized as an engineering integration of an existing sequence model into a new, previously unaddressed dialectal G2P setting, paired with a genuine evaluation-resource contribution: the authors release the first test set pairing full diacritization with verbatim dialectal pronunciation for Egyptian Arabic tweets. The empirical scope is narrow — a single dialect, a single annotator, and a workshop-length paper — so the result should be read as a first demonstration rather than a broadly validated method.
Field Significance
Moderate — this paper contributes a useful, previously unavailable evaluation resource for dialectal Arabic G2P and provides a concrete data point that a joint multi-task formulation can match a cascaded pipeline without added complexity, while also showing that a strong general-purpose LLM (GPT-4) is not yet competitive with task-specific supervised models on this structured phonemic-normalization task. Its scope is limited to a single Egyptian Arabic dataset and a single annotator, so its contribution is incremental evidence for the dialectal-G2P sub-area rather than a new architectural direction.
Claims
- supports: An end-to-end multi-task model that jointly predicts diacritics and dialect-specific sound substitutions can match the accuracy of a cascaded pipeline built from the same components, without requiring two separately trained stages.
Evidence: DialG2P (WER 5.15%, PER 1.71%, CER 0.05%) performs on par with the Transformer cascade baseline (WER 5.11%, PER 1.70%, CER 0.09%) on the same held-out Egyptian Arabic test set, achieving the lowest CER of any system tested. (§3.7, Table 1)
- complicates: Multi-task training for text-normalization front-ends can trade a small amount of accuracy on one subtask for the convenience of a single unified model.
Evidence: DialG2P shows a slight regression in diacritic error rate relative to the single-task diacritization Transformer (1.67% vs. 1.62% DER), attributed to reduced task-specific model capacity under multi-task training. (§3.7, Table 1)
- contradicts: A general-purpose large language model prompted with in-context examples can serve as a competent substitute for a task-specific supervised model on structured phonemic/orthographic normalization tasks.
Evidence: GPT-4 with 10-shot in-context prompting and a manual default-special-sound fallback still reaches WER 25.14%, roughly five times worse than the dedicated DialG2P model’s 5.15% WER on the same test set; without the fallback, GPT-4’s zero-shot WER is 47.57%. (§3.7, Table 1)
- complicates: Grapheme-to-phoneme systems for dialects lacking standard orthography must model dialect-specific sound substitutions as a distinct error source from diacritic restoration, since substitution patterns vary sharply across dialects.
Evidence: The rate at which a given letter is pronounced with its dialect-shifted “special sound” differs sharply by dialect in the authors’ annotated data (98.5% in Egyptian tweets versus 2.46% in Levantine tweets for one such substitution), showing the mapping cannot be assumed to generalize across dialects without dialect-specific modeling. (§2, Figure 1)
Limitations and Open Questions
Training and gold-label data rely on a single native-speaker annotator per dialect and a hand-crafted rule set for generating special-sound substitution labels, and all empirical results are limited to Egyptian Arabic; annotation reliability and cross-dialect generalization are untested.
The authors state their method is designed to be generic enough to apply to other dialects or languages with similar orthography-pronunciation gaps, and state an intention to extend the work to other Arabic dialects, but no experiments beyond Egyptian Arabic are reported in this paper. As a short workshop paper, the evaluation is also limited to a single 1,000-tweet test set and does not report significance testing or human evaluation of the resulting phoneme sequences.
Wiki Connections
- Evaluation Metrics — benchmarks a dedicated multi-task model, a matched cascaded pipeline, and a general-purpose LLM (GPT-4) using WER, PER, DER, and CER, and contributes a new test set for evaluating dialectal G2P specifically.