arXiv · 2025 · Preprint
Yihao Wu et al. (Nanyang Technological University, Soul AI Lab) · → Paper · Demo: ✗ · Code: ✓
Introduces the first systematic evaluation of paralinguistic bias (age, gender, accent) in end-to-end spoken dialogue LLMs across decision-making and recommendation tasks, in both single-turn and multi-turn conversations.
Problem
Text-only LLMs have been extensively studied for stereotype and demographic bias, but spoken dialogue models (SDMs) that consume and produce audio directly expose paralinguistic cues, such as accent, gender, and perceived age, that text-only systems never see. Prior speech fairness work is narrow: ASR and TTS bias studies measure task-specific performance disparities, and existing fairness corpora (Fair-Speech, EARS, Spoken StereoSet, Sonos Bias, MuTOX) are recognition- or toxicity-oriented rather than built for evaluating decision-making or recommendation behavior. Text-based stereotype benchmarks (WinoBias, StereoSet, CrowS-Pairs) encode paralinguistic attributes only as text descriptions, which does not transfer to spoken input. No prior resource lets researchers hold linguistic content fixed while varying only paralinguistic attributes and observe how a spoken dialogue LLM’s decisions or recommendations shift, especially over multiple conversational turns with corrective feedback.
Method
The authors construct FairDialogue, a controlled dataset built via a two-stage pipeline. First, GPT-4o generates balanced, paralinguistically-neutral text utterances (roughly 2-3 sentences, about 10 seconds of speech) for two task categories: decision-making (interview assessments, task assignments, award distributions) and recommendation (career guidance, course selection, entertainment suggestions). Prompts explicitly exclude demographic or emotional cues from the text itself, leaving only content-relevant background (education, experience, skills). Second, the balanced text is synthesized into speech with controlled paralinguistic variation: Index-TTS2 (a VQ-VAE-based, prompt-guided TTS system with a BigVGAN vocoder) is used to control gender and age, while ElevenLabs’ multilingual v2 model is used to control accent (US, UK, India, Australia, African), holding all other factors constant. The resulting corpus totals roughly 1,700 minutes of audio across about 7,200 samples.
Four spoken dialogue LLMs are evaluated zero-shot, with no fine-tuning by the authors: two open-source models (Qwen2.5-Omni, GLM-4-Voice) and two closed-source APIs (GPT-4o Audio, Gemini-2.5-Flash). Inference uses beam width 1 with sampling disabled for reproducibility. All audio outputs are transcribed with Whisper for consistent downstream analysis. Bias is quantified with two metric families adapted from the text-fairness literature to the audio setting: Group Unfairness Score (GUS), which measures the maximum disparity in positive-decision probability between paralinguistic attribute groups for decision-making tasks, and similarity-based normalized statistics rate and variance (SNSR/SNSV), computed from a pairwise ranking-agreement metric (PRAG*@K) over top-K recommendation lists, for recommendation tasks. A separate multi-turn protocol isolates samples where all models initially gave identical negative decisions in single-turn settings (showing no apparent bias), then applies four rounds of negated, corrective feedback and measures the revision success rate (RST) and average number of rounds (ANR) needed for each attribute group to flip its decision.
Key Results
In single-turn decision-making, closed-source models show markedly lower bias than open-source ones: Gemini-2.5 reports GUS in the 0.12-0.14 range across award, interview, and assignment tasks, versus 0.17-0.20 for Qwen2.5-Omni and 0.19-0.21 for GLM-4-Voice (Table 2). Within open-source models, gender and age drive most of the disparity, while accent-related bias stays low across all four models (average GUS below 0.15). The pattern reverses for recommendation tasks: GLM-4-Voice (SNSR 0.66-0.68) and GPT-4o Audio (0.63-0.72) show larger cross-group disparities than Qwen2.5-Omni (0.50-0.58) and Gemini-2.5 (0.46-0.52), and accent-driven bias is substantially larger than in decision tasks. Task type interacts with which model is most biased: GLM-4-Voice is worst on entertainment recommendations, Qwen2.5-Omni is worst on occupation recommendations (Table 2).
The multi-turn experiment (Table 3) finds that decisions initially uniform and negative across all models and attribute groups become attribute-dependent under repeated corrective feedback. Elder Male speakers achieve the highest revision success rate in both Qwen2.5-Omni (88%) and GLM-4-Voice (95%), while Young Female speakers have the lowest (69% and 84%, respectively). The two open-source models diverge in which attribute drives the effect: Qwen2.5-Omni shows a pronounced age effect (Elder Males need more rounds) with minimal gender difference, while GLM-4-Voice shows a larger gender effect with an overall lower average-rounds-to-revise. This demonstrates that biases invisible in single-turn evaluation can surface once a model is pushed through multiple conversational turns.
Novelty Assessment
The contribution is a dataset and an evaluation methodology, not a new model or architecture. The paper adapts two existing fairness metrics from the text/recommendation literature (group unfairness, PRAG*-based similarity range/variance) to the audio domain rather than proposing new ones, and the TTS systems used to build FairDialogue (Index-TTS2, ElevenLabs) are off-the-shelf tools for controlled data generation, not novel components. What is new is the systematic pairing of controlled paralinguistic variation with decision-making and recommendation tasks specifically, and the multi-turn corrective-feedback protocol that surfaces biases invisible in single-turn evaluation. The evaluation covers only four models and two task families, so claims about which model class (open- vs. closed-source) is “more biased” should be read as evidence from this specific benchmark rather than a general characterization.
Field Significance
Moderate — the paper fills a genuine gap: prior speech fairness work targeted recognition or toxicity, not decision-making or recommendation behavior in end-to-end spoken dialogue systems. It provides a reusable, publicly released dataset and evaluation code for a task class that was previously unmeasured, and its multi-turn protocol demonstrates that single-turn bias evaluation can miss disparities that only appear under repeated interaction.
Claims
- supports: Closed-source spoken dialogue systems can exhibit substantially lower paralinguistic decision bias than open-source alternatives evaluated under the same protocol.
Evidence: Gemini-2.5-Flash achieves Group Unfairness Scores of 0.12-0.14 across award, interview, and assignment decision tasks, compared to 0.17-0.20 for Qwen2.5-Omni and 0.19-0.21 for GLM-4-Voice on the same FairDialogue samples. (§3.3.1, Table 2)
- complicates: A model’s relative fairness ranking on one conversational task type does not transfer to another task type.
Evidence: GPT-4o Audio and GLM-4-Voice, which are not the most biased models on decision-making, show the largest cross-group disparities (SNSR 0.63-0.72 and 0.66-0.68) on recommendation tasks, while Gemini-2.5 and Qwen2.5-Omni, weaker on recommendation-related accent bias in decisions, perform comparatively better here. (§3.3.1, Table 2)
- complicates: Single-turn bias evaluation can understate a spoken dialogue system’s paralinguistic bias relative to multi-turn interaction.
Evidence: On samples where all four models initially gave identical negative decisions in single-turn evaluation (no apparent bias), applying four rounds of corrective feedback revealed attribute-dependent revision success rates: Elder Male speakers reached 88-95% revision success versus 69-91% for Young Female speakers, with the effect attributed to age in Qwen2.5-Omni and to gender in GLM-4-Voice. (§3.3.2, Table 3)
- supports: Paralinguistic attributes carried only through voice (not through text content) are sufficient to bias spoken dialogue LLM outputs in real-world-relevant decision and recommendation tasks.
Evidence: FairDialogue holds text content demographically neutral while varying only synthesized speech attributes (age, gender, accent), and all four evaluated models still show measurable disparities in decision acceptance rates and recommendation rankings across these attribute groups. (§2, §3.3.1)
Limitations and Open Questions
Warning
The evaluation covers only four spoken dialogue systems (two open-source, two closed-source) and paralinguistic variation is synthetic: age and gender are controlled via Index-TTS2 prompt-audio guidance and accent via ElevenLabs voice selection, rather than drawn from real speakers with those attributes. Findings about which model class is more biased may not generalize beyond this specific dataset and model set.
The multi-turn protocol is restricted to decision-making tasks and only tests a negate-and-reassert corrective feedback pattern over four rounds; it does not explore other conversational strategies (e.g., explanation requests, disagreement) that might reveal different bias patterns. All outputs are transcribed via Whisper before analysis, which introduces an additional ASR-dependent step between the model’s spoken output and the reported metric. The paper does not propose or evaluate any bias mitigation technique, leaving open how the identified disparities could be reduced.
Wiki Connections
- Spoken Language Model — evaluates paralinguistic bias directly in the audio-in, audio-out spoken dialogue LLMs (Qwen2.5-Omni, GLM-4-Voice, GPT-4o Audio, Gemini-2.5-Flash) that this concept covers.
- Speech-to-Speech — the decision-making and recommendation tasks are conducted entirely through spoken input and spoken output, without a text-only fallback path in the evaluation.
- Evaluation Metrics — adapts group unfairness and ranking-similarity metrics from the text/recommendation fairness literature into audio-specific fairness measures (GUS, SNSR, SNSV) for spoken dialogue systems.
- IndexTTS2 — used as one of two TTS systems to synthesize the FairDialogue dataset, chosen specifically for its prompt-audio guidance over gender and age attributes.