NAACL · 2025 · Conference
Kuan and Lee (National Taiwan University) · → Paper · Demo: ? · Code: ?
Instruction-guided TTS models systematically amplify societal gender stereotypes when interpreting occupation-related style prompts, and prompt-based mitigation strategies are insufficient and can even reverse bias direction.
Problem
Instruction-guided TTS systems accept natural language style prompts to shape the character of synthesized speech. When prompts reference occupations (“Act like a nurse”), the model must infer vocal characteristics associated with the role. This creates a channel through which societal gender stereotypes encoded in training data can surface as measurable bias in the generated audio. Prior work had advanced the capability of instruction-guided synthesis but had not examined whether these models reflect or amplify occupational gender associations, nor whether simple inference-time interventions could correct such biases.
Method
The study uses four Parler-TTS variants as subjects: Large v1 (2.2B parameters, 45,000 hours of training audio), Mini v1 (880M parameters, same data), Mini v0.1 (880M parameters, 10,500 hours), and Mini Expresso (Mini v0.1 fine-tuned on the Expresso dataset). These are the only publicly available open-source models that accept a free-form text style prompt without requiring a reference audio clip.
For each of 109 occupations drawn from Zhao et al. (2018) and GPT-4o-generated lists, five style prompt templates are paired with ten neutral content prompts, yielding 500 generated samples per occupation per model. Gender classification is performed using a wav2vec 2.0-based gender recognition model. Emotion and speaking rate are also measured but show no significant occupation-driven effects.
Bias is quantified by comparing an occupation-prompted group against three control groups (empty string, generic “Act like a person” prompts, and neutral unrelated sentences) using chi-square tests and standardized residuals. A positive residual for a gender indicates overrepresentation relative to the model’s inherent baseline. Three prompt-based mitigation methods are then evaluated: appending “paying attention to diversity” (Method 1), “recognizing that this occupation can be male or female” (Method 2), or an adapted ethical intervention phrase from Bansal et al. (2022) (Method 3).
Key Results
Gender bias follows societal stereotypes across all four models. Strongly male-coded occupations (fisherman, electrician, plumber, carpenter, mechanic) yield consistently high positive residuals toward male voices, with standardized residuals reaching 26.6 for fisherman in the Large v1 model. Strongly female-coded occupations (nurse, nanny, midwife, receptionist) yield correspondingly high residuals toward female voices, reaching 27.2 for midwife in Mini v0.1. The magnitude varies by model: Large v1 tends to produce stronger male bias for traditionally masculine roles, while Mini v0.1 shows stronger female bias for conventionally feminine ones. Mini Expresso generally exhibits milder bias overall.
None of the three prompt-based mitigation methods prove reliably effective across models and occupations. Methods 2 and 3 frequently reverse the bias direction for male-associated roles, shifting overrepresentation from male to female rather than achieving neutrality. Some occupations see bias increase after mitigation. This inconsistency holds across all four baseline models, ruling out a model-size explanation.
Novelty Assessment
The contribution is primarily an evaluation methodology applied to a new problem domain. The authors develop a systematic pipeline for measuring occupational gender bias in instruction-guided TTS, covering 109 occupations, multiple prompt templates, and statistical bias quantification via chi-square tests and standardized residuals. The finding that these models exhibit and potentially amplify societal stereotypes is not surprising given analogous results in text-to-image generation, but constitutes the first documented investigation in the TTS domain. The analysis of mitigation strategies adds practical value by showing that simple inference-time prompting is insufficient. The study’s scope is limited by the availability of a single open-source model family (Parler-TTS); the methodology is presented as extensible to future models.
Field Significance
Moderate — This paper establishes a baseline understanding of occupational gender bias in instruction-guided TTS and provides a reusable evaluation framework applicable to future models. Its primary contribution is empirical: it demonstrates that biases in training data propagate into vocally expressed gender associations when models are given ambiguous style instructions. The finding that prompt-based mitigation can introduce compensatory bias in the opposite direction adds a cautionary data point for practitioners deploying instruction-conditioned TTS in public-facing applications.
Claims
-
supports: Instruction-conditioned TTS models reflect and amplify occupational gender stereotypes present in training data when interpreting ambiguous style prompts.
Evidence: Across four Parler-TTS variants and 109 occupations, chi-square analysis shows statistically significant overrepresentation of male voices for male-coded occupations and female voices for female-coded occupations, with standardized residuals up to 27.2 for individual occupation-model pairs. (§4.3, Table 1)
-
complicates: Prompt-based fairness interventions can introduce compensatory bias rather than achieving gender neutrality in instruction-conditioned TTS.
Evidence: Methods 2 and 3 (adding explicit fairness language to style prompts) frequently reverse the bias direction for traditionally masculine occupations across all four models, shifting overrepresentation from male to female rather than reducing bias magnitude. (§4.4, Tables 2–4)
-
complicates: Gender bias magnitude in instruction-guided TTS varies substantially across model variants trained on the same data distribution, limiting the generalizability of any single model’s bias profile.
Evidence: Large v1 and Mini v1 share training data but display different bias patterns by occupation; Mini v0.1 and Mini Expresso (fine-tuned variant) likewise differ, suggesting model scale and fine-tuning interact with bias in ways not fully explained by data alone. (§4.3)
-
supports: Evaluating gender bias in TTS requires controlling for the model’s inherent gendered tendency in baseline generation, separate from the effect of specific style prompts.
Evidence: Even with an empty style prompt or neutral non-occupational prompts, Parler-TTS models generate voices skewed toward a particular gender, necessitating three distinct control groups to isolate the effect of occupation-related prompting. (§3.3, §4.2)
Limitations and Open Questions
Warning
The analysis is restricted to Parler-TTS, the only publicly available open-source model family that accepts free-form style prompts without requiring a reference audio clip at the time of publication. All quantitative findings are specific to this model family and may not generalise.
The study uses a binary gender classification framework (male/female) constrained by current automatic gender recognition models. Occupations are drawn from a fixed list of 109 that, while broad, cannot represent the full diversity of professions. Emotion and speaking rate are measured but no occupation-related effects are found for either attribute, leaving open the question of whether other speech attributes (accent, perceived age, confidence) also carry occupational bias. No training-time debiasing approaches are evaluated.
Wiki Connections
- Instruction-Conditioned TTS — this paper identifies a systematic failure mode of instruction-conditioned TTS: occupation-related prompts elicit gendered vocal characteristics that mirror societal stereotypes.
- Evaluation Metrics — the paper introduces a chi-square residual framework for quantifying occupational gender bias in TTS output that extends beyond standard audio quality metrics.
- Subjective Evaluation — the study relies on automatic gender classification rather than human listening tests, highlighting the role of automated evaluators in large-scale TTS bias measurement.