EMNLP · 2025 · Conference
Wong et al. (Apple) · → Paper · Demo: ✗ · Code: ✓
Replaces the rule-based text normalization (TN) front-end of a production TTS system with few-shot LLM prompting, cutting word error rate across eight languages while releasing a new multilingual TN benchmark.
Problem
Text normalization, the step that converts written forms (numbers, dates, currencies, abbreviations, symbols) into their canonical spoken equivalents, has historically relied on weighted finite-state transducers (WFSTs) built from hand-crafted rules. These rule-based systems can be highly accurate but require substantial manual engineering and ongoing maintenance, and they scale poorly to new languages, especially morphologically rich or low-resource ones such as Arabic and Polish. Prior LLM-based TN work (Zhang et al., 2024) showed that GPT models with few-shot prompting and self-consistency reasoning can beat production WFST systems, but that work was English-only. PolyNorm asks whether a single prompting framework can generalize across typologically diverse languages while also solving a second, underexplored problem: the lack of a shared, high-quality multilingual TN evaluation resource.
Method
PolyNorm reframes TN as a few-shot, in-context-learning (ICL) task for a frozen, general-purpose LLM rather than a rule-authoring or model-training problem. No fine-tuning is performed; the same GPT-4o and GPT-4o-mini models are queried through a structured prompt with three components: (1) an instruction prompt that defines the TN task and category-specific rules (e.g., how to spell out acronyms, how to verbalize punctuation in URLs), (2) a set of in-context learning examples that demonstrate correct normalization per category, and (3) the target unnormalized input. The instruction prompt is held in standardized English across all eight target languages (American English, German, French, Mexican Spanish, Italian, Lithuanian, Japanese, and Mandarin Chinese), with only the ICL examples localized to each language’s linguistic and stylistic conventions; Japanese additionally receives a supplementary prompt to guide katakana output for homograph disambiguation.
The ICL examples themselves (80-100 per language, excluded from the evaluation benchmark) are drawn from machine-translated and expert-verified Kestrel data for English and from DeepSeek-R1-generated synthetic examples for the other languages, all reviewed by internal language experts for stylistic consistency. The authors also describe a hillclimbing iteration loop: language experts review LLM outputs on development sets, flag systematic error categories (e.g., inconsistent date or currency formatting), and the team revises or adds targeted ICL examples to correct them, repeating across iterations (the paper reports results for iteration 2 and iteration 3 in the appendix).
To support evaluation, the authors build PolyNorm-Benchmark, a new multilingual dataset spanning 27 TN categories (cardinal, date, currency, address, ISBN, sports score, chemical formula, and others; Table 1) built on and expanded from the Kestrel taxonomy. Each language contributes 540 examples (20 per category), generated initially by DeepSeek-R1 and then edited and verified by internal language experts for linguistic precision and naturalness, with orthographic and formatting conventions (e.g., French comma decimal separators, katakana for Japanese) matched to each locale. The benchmark and its construction pipeline are released publicly.
Key Results
Across all eight languages, both GPT-4o-mini and GPT-4o reduce WER (CER for Chinese and Japanese) relative to Apple’s production rule-based baseline, with GPT-4o showing the larger gains (Table 2). GPT-4o’s overall WER ranges from 4.17% (German) to 7.88% (Japanese), compared to the baseline’s 9.72% (French) to 17.49% (Japanese). BLEU improves in tandem, e.g., American English rises from 70.84% (baseline) to 89.85% (GPT-4o). The paper highlights qualitative cases where the rule-based baseline systematically fails but PolyNorm succeeds: domain-specific tokens such as URLs and usernames, which evolve too quickly for static rules, and dash disambiguation, where the correct spoken form depends on context (sports scores read a dash as “to”, phone numbers read digits individually), which PolyNorm infers directly without an explicit rule for each case. Because the production baseline is proprietary, the authors cannot release its code or provide external access to it, so head-to-head comparisons rest on the numbers reported in the paper rather than an independently reproducible baseline.
Novelty Assessment
The core technical idea, few-shot LLM prompting for TN, is not new; it extends Zhang et al. (2024)‘s English-only GPT-based TN work. PolyNorm’s contribution is primarily engineering integration and dataset construction: a language-agnostic prompt template that holds instructions constant while localizing only the ICL examples, applied and validated across eight typologically diverse languages, plus the corresponding hillclimbing workflow for iteratively improving ICL coverage. The most durable artifact is PolyNorm-Benchmark itself, an expert-verified, 27-category multilingual TN evaluation set, which is a genuine dataset-contribution since no comparable shared multilingual TN benchmark previously existed at this scope. No new model architecture or training procedure is introduced; the underlying LLMs are used off-the-shelf and frozen throughout.
Field Significance
Moderate — This paper demonstrates that a single, language-agnostic few-shot prompting framework can extend LLM-based text normalization from English to a typologically diverse set of eight languages while consistently beating a production rule-based system on WER. Its most durable contribution to the field is the released PolyNorm-Benchmark, which provides a shared, expert-verified multilingual evaluation resource for a TTS front-end task that previously lacked one; the underlying method itself does not introduce new architecture or training methodology.
Claims
- supports: Few-shot prompting of a general-purpose LLM can replace rule-based text normalization front-ends for text-to-speech across typologically diverse languages, reducing word error rate without task-specific model training.
Evidence: GPT-4o achieves overall WER between 4.17% and 7.88% across all eight evaluated languages, below the production rule-based baseline’s 9.72%-17.49% range, using the same frozen model and only localized in-context examples. (§4, Table 2)
- supports: LLM-based text normalization can resolve context-dependent token ambiguities that are costly to encode as explicit rules in a WFST-based system.
Evidence: PolyNorm correctly verbalizes a dash as “to” in sports scores (e.g., “3-2” to “three to two”) but reads phone-number digits individually, inferring the distinction from context rather than a hand-authored rule set, and similarly segments unpredictable domain-specific strings such as URLs into speakable components. (§4)
- supports: LLM-generated synthetic data combined with expert linguistic review can produce a scalable, standardized multilingual evaluation resource for a speech-processing task that previously lacked shared benchmarks.
Evidence: PolyNorm-Benchmark is constructed by generating unnormalized/normalized pairs with DeepSeek-R1 across 27 categories for each of 8 languages, then editing and verifying each of the 540 examples per language with internal language experts to ensure orthographic and phonetic accuracy. (§2)
- complicates: The reliability of few-shot LLM-based text normalization depends heavily on the quality and category coverage of curated in-context examples, and residual errors persist for underrepresented or morphologically complex patterns.
Evidence: GPT-4o’s per-language WER remains uneven across iterations of ICL refinement, e.g., Lithuanian’s overall WER falls from 12.22% to 6.99% between iteration 2 and iteration 3 while some categories such as legal references stay above 10% WER in multiple languages, indicating that category-specific ICL gaps are not uniformly closed by iteration. (§3.2, Tables 5-6)
Limitations and Open Questions
The production rule-based baseline is proprietary and cannot be released, so the paper's head-to-head WER/BLEU comparisons cannot be independently reproduced against that exact system; readers can only verify PolyNorm's own results against the released benchmark, not the baseline's.
The authors also note that system performance is highly dependent on the quality and representativeness of the curated ICL examples, and that categories covered in those examples may not capture the full range of normalization phenomena across languages and varieties; normalization errors can propagate into downstream tasks such as translation or sentiment analysis. The paper flags diacritization restoration (e.g., Arabic, Hebrew) and integration of suprasegmental features such as pitch accent and tone (relevant for Japanese katakana normalization) as open problems not addressed by the current system.
Wiki Connections
- Multilingual TTS — addresses the text-normalization front-end for multilingual TTS pipelines, evaluating a single prompting framework across eight typologically diverse languages rather than a shared acoustic model.
- Evaluation Metrics — introduces PolyNorm-Benchmark, a new expert-verified, 27-category multilingual dataset for measuring text normalization quality with WER, CER, and BLEU.