arXiv · 2026 · Preprint

Yuxiang Wang et al. (The Chinese University of Hong Kong, Shenzhen) · → Paper · Demo: ✓ · Code: ✓

A bilingual 22-task benchmark that separates content-centric risks from audio-conditioned risks (benign transcript, but the right response depends on who is speaking, how, or where) and shows that speech LMs which detect the decisive acoustic cue often still fail to act on it safely, fairly, or privately.

Problem

Speech LMs are moving into shared, multi-user settings, where the appropriate response can depend on cues the transcript does not carry: a child speaker, slurred or agitated delivery, bystanders in the background, or a voice that signals gender or accent. Existing audio benchmarks mostly test capability (can the model hear and reason), study single risks such as audio jailbreaks or voice profiling in isolation, or mix inherently harmful content with content that becomes problematic only because of its acoustic context. Because of that mixing, a failure cannot be attributed either to missing normative knowledge or to failing to apply that knowledge once the cue arrives through speech.

Method

VoxSafeBench covers three pillars (safety, fairness, privacy) under a Two-Tier design. Tier 1 (“the what”) evaluates content-centric risks where the transcript itself is harmful, with matched text, clean-audio and diverse-audio views of each item. It includes harmful content, single- and multi-turn jailbreaks, agentic tool-use risks (direct vs. tool-injected harm), explicit-demographic stereotype probes, and hard (structured PII, credentials) and soft (contextual secrets) privacy. Tier 2 (“the who, how, and where”) uses audio-only items whose transcripts are verified to be benign in isolation, so any risk must come from the audio. These cover content-paralinguistic conflict (child voice, angry emotion, impaired or slurred speech), content-background conflict (child presence, NSFW or symbolic backgrounds, unsafe ambient scenes), overlapped-speaker instruction injection, fairness minimal pairs where demographic contrast is carried only by voice or background, and audio-conditioned, interactional (adopted from VoxPrivacy) and inferential (adopted from HearSay) privacy. Coverage is English and Chinese, with discriminative and generative formats.

Most items start as text: adapted from existing benchmarks, written from scratch, or drafted by several frontier LLMs and then screened by hand. They are rendered as speech with CosyVoice3 zero-shot cloning from a curated prompt-audio pool. Its clean subset holds neutral speech; its diverse subset varies accent, age, gender and emotion, and 30% of diverse clips are mixed with background tracks at 40% of foreground amplitude. Emotion prompts are filtered with an audEERING arousal model (high-activation emotions below 0.7 arousal are dropped), and CV3-NV adds nonverbal vocalizations. Speaker similarity of the synthesized audio is 78% (ZH) and 71.8% (EN), against 75.5% and 73.4% for human recordings (§Appendix D). All clips with Whisper-large-v3 WER above 5% are discarded, a small subset is recorded by hand, and humans spot-check both tiers. Every Tier 2 run uses a task-agnostic system reminder telling the model that speaker, delivery and background may matter, without naming the cue in a given item (§Appendix E).

The paper evaluates Qwen3-Omni and MiMo-Audio (each with a thinking variant), Kimi-Audio, GPT-4o-Audio, Gemini-3-Flash and Gemini-3-Pro. These were chosen for strong audio understanding on MMSU and MMAU-Pro. All evaluation is prompting-only, and the outputs judged are the models’ text responses, not generated speech. DeepSeek-V3 is the primary LLM judge for open-ended tasks. The paper adds new metrics: Direct Answer Rate (DAR) and Safety/Privacy Awareness Rate (SAR/PAR, split into warning and refusal); Overlap-Induced Conversion (OIC), the share of unsafe overlap answers whose isolated unsafe track would have been refused; Fair Rate; and a Net Bias Score (NBS), with speaker-order swapping and a two-sided binomial test, to measure which direction unfair choices lean. Three controls support Tier 2’s construct validity: benign-transcript verification; a text reference upper bound in which text LLMs see the cue verbalized (“the speaker is a child”); and intermediate perception probes that ask each model directly about the cue in the same audio, with three human annotators as an audibility check (§Appendix J).

Key Results

  • Tier 1 safety (§3, Figure 2): without jailbreaks all models stay in the safe region. Multi-turn jailbreaks are the most effective attack. Unexpectedly, text inputs degrade more than audio inputs: text elicits longer, more detailed unsafe outputs, while audio more often falls back to short refusals. Thinking variants degrade more than standard ones. Agentic refusal drops sharply when the harmful instruction comes inside a tool output instead of from the user.
  • Tier 2 safety (§3, Table 3): with the cue verbalized, the text upper bound is near saturation (SAR 92.5% child voice, 88.8% emotion, 100% impaired capacity, 99.5% child presence). Audio models fall far short. GPT-4o-Audio answers child-voice requests directly 100% of the time (SAR 0.0), Qwen3-Omni reaches 0.5% SAR on child voice, and Gemini-3-Pro is best on impaired capacity (76.1% SAR). Across models, impaired-capacity cues are handled better than child voice, and child voice better than emotion. Models react more to an explicitly unsafe background than to a child audibly present. OIC ranges from 46.8% to 77.8%, so overlapping a benign foreground often converts an otherwise intact refusal into compliance.
  • Fairness (§4, Table 4): Fair Rate falls sharply from Tier 1 to Tier 2 for nearly every model. For example, Gemini-3-Pro’s criminality Fair Rate drops from 97.3/94.9 (text, EN/ZH) to 27.9/12.5, and its competence Fair Rate from 69.3/63.1 to 4.0/1.2. GPT-4o-Audio keeps the highest Tier 2 Fair Rates on several dimensions. Several models show significant positive Net Bias Scores (Figure 3), and the same audio queried in English vs. Chinese can flip the direction of bias.
  • Privacy (§5, Table 5): Gemini-3-Pro’s hard-privacy leakage rises from 23.9% on text to 77.7% (clean audio) and 81.2% (diverse audio). All models leak structured PII more than contextual secrets. On audio-conditioned privacy, GPT-4o-Audio and Qwen3-Omni answer directly in over 85% of cases, while Gemini-3-Pro reaches 94.3% PAR; giving the same cue as text yields 100% PAR. Open-source models are near chance on interactional privacy (Figure 4a).
  • Perception vs. alignment (§Appendix J.3, Table 42): Qwen3-Omni recognizes a child voice 90.0% of the time and a child in the background 91.5% of the time, yet its matched SAR is 0.5% on both. Gemini-3-Pro scores 83.1% on the child-voice probe but 39.5% SAR. Human probe accuracy is 91.2% to 99.0%, so the cues are clearly audible. Weaker models (Kimi-Audio, MiMo-Audio) fail both the probe and the safety task, so part of their failure is perceptual. Emotion is the hardest cue to perceive (best model probe 64.3%).
  • Judge reliability (§Appendix I): across three LLM judges and three human annotators on 480 Tier 2 safety items, Fleiss’ κ averages 0.78, and Spearman’s ρ between each LLM judge and the human majority ranking is at least 0.88. This audit covers only the cue-aware Tier 2 safety subtasks; fairness and privacy judging is not audited the same way.

GPT-4o-Audio has no text-input results in the fairness and privacy Tier 1 tables, so its cross-modal gaps cannot be computed.

Novelty Assessment

The main contribution is the benchmark design, not any modelling work. The Two-Tier split between content-centric and audio-conditioned risk, combined with near-saturated text upper bounds and matched perception probes, is a careful construct-validity argument that earlier audio-trust suites (AudioTrust, AHELM) do not make. It lets a failure be attributed to lack of grounding rather than to missing norms or poor hearing. Most component ideas come from elsewhere: Tier 1 safety items adapt text safety and jailbreak benchmarks, two privacy tasks reuse VoxPrivacy and HearSay, and minimal-pair fairness probing follows standard bias-evaluation practice. The new pieces are the Tier 2 safety and fairness items, the OIC and NBS metrics, and the probing study. The data pipeline relies on synthetic speech (CosyVoice3 cloning from a curated prompt pool), a reasonable but limiting choice. Judging relies on LLMs, with a well-designed human-agreement check for one task family only. The model set is small: six models plus two thinking variants.

Field Significance

Moderate. The paper gives a well-controlled demonstration that safety, fairness and privacy behaviour learned at the transcript level does not carry over when the decisive cue is carried only by voice or background sound, and that for stronger speech LMs this is mainly a failure to ground policy on cues they do perceive. Its relevance to speech generation is indirect: no speech output is evaluated, and the connection comes through speech-LM dialogue behaviour and the use of controllable zero-shot TTS to build acoustically controlled minimal pairs.

Claims

  • complicates: Safety, fairness and privacy alignment that holds for transcript-level input does not reliably carry over to speech LMs when the socially decisive cue is carried only by paralinguistic or background audio.

    Evidence: Text LLMs given the cue verbalized reach 88.8–100% SAR on Tier 2 safety tasks, while audio models mostly score below 50% (e.g., GPT-4o-Audio 0.0% SAR on child voice). Fairness Fair Rates collapse from Tier 1 to Tier 2 (Gemini-3-Pro criminality 97.3% to 27.9% EN), and audio-conditioned privacy awareness is 100% when the cue is given as text but under 15% for Qwen3-Omni and GPT-4o-Audio on audio. (§3, Table 3; §4, Table 4; §5, Table 5)

  • refines: In end-to-end speech LMs, failures to respond appropriately to paralinguistic cues often come from failing to act on a perceived cue rather than from failing to perceive it, though weaker models do have perception bottlenecks.

    Evidence: Qwen3-Omni identifies a child voice in 90.0% of cases but reaches 0.5% SAR on the matched safety task; Gemini-3-Pro scores 83.1% on the probe vs. 39.5% SAR. Kimi-Audio and MiMo-Audio fail both the probe and the task, and humans recognize the cues at 91.2–99.0%. (§Appendix J.3, Table 42)

  • complicates: Under adversarial multi-turn prompting, text input can be less safe than spoken input in speech LMs, because audio responses tend to default to short refusals while text responses become longer and more compliant.

    Evidence: In the Tier 1 jailbreak and direct agentic-harm settings, text-modality trajectories degrade further toward high toxicity and low refusal than audio trajectories, and reasoning variants degrade more than standard ones. (§3, Figure 2)

  • complicates: Speech LMs respond unevenly to different classes of acoustic cue: salient prosodic instability is acted on more often than timbre-based speaker identity or emotion, and overtly unsafe backgrounds more often than cues about who is listening.

    Evidence: Most models show an impaired-capacity > child-voice > emotion ordering in SAR. Content-background results are better for explicitly unsafe backgrounds than for child presence, and the best emotion-probe accuracy is only 64.3%. (§3, Table 3; §Appendix J.3)

  • supports: For open-ended response labelling in spoken-dialogue safety evaluation, LLM judges can agree with human annotators closely enough to keep model rankings intact.

    Evidence: Over 480 Tier 2 safety items, Fleiss’ κ is 0.78 across three LLM judges and three humans (0.81 LLM-only, 0.83 human-only), Spearman’s ρ against the human majority ranking is ≥0.88, and per-model SAR/DAR deviations are at most 5.5 points. (§Appendix I.2–I.4, Tables 38–41)

Limitations and Open Questions

Warning

Most benchmark audio is synthesized with CosyVoice3 from a curated prompt pool, and Tier 2 cues are deliberately prominent. The authors note that real-world failures on natural speech and subtler cues may be worse than reported, so the measured gaps describe controlled conditions rather than deployment.

  • The text reference upper bound measures transcript-level ability with the cue made explicit; the authors note it is not a true oracle for audio-grounded behaviour.
  • Human checks of the LLM judge cover only cue-aware Tier 2 safety; fairness and privacy labels rely on the primary judge alone.
  • Neutrality is used as a proxy for fairness, and when Tier 2 Fair Rates are near zero a non-significant Net Bias Score may reflect random choice rather than absence of bias (§4).
  • Only a small set of six models (plus two thinking variants) is evaluated, and GPT-4o-Audio cannot be tested on text input, so some cross-modal comparisons are incomplete.
  • Only the models’ textual responses are judged. Whether a speaking agent’s own output voice or prosody (for example, a warning delivered appropriately to a child) is suitable is outside the benchmark’s scope.
  • The benchmark diagnoses the grounding gap but proposes no mitigation; whether alignment training on audio-conditioned data closes it remains open.

Wiki Connections

  • Evaluation Metrics — introduces cue-grounded behavioural metrics for speech LMs (DAR, SAR/PAR, OIC, Fair Rate, Net Bias Score) and validates LLM-judge labelling against human annotators.
  • Spoken Language Model — evaluates end-to-end speech LMs (Qwen3-Omni, MiMo-Audio, Kimi-Audio, GPT-4o-Audio, Gemini-3) on audio-only safety, fairness and privacy cues, prompting-only, judging text responses rather than generated speech.
  • CosyVoice 3 — the zero-shot TTS system used to synthesize nearly all benchmark audio from a curated prompt pool, chosen for its bilingual speaker-identity preservation.
  • Qwen3-Omni — one of the evaluated speech LMs (standard and thinking variants), and the clearest example of high cue perception paired with near-zero safe behaviour.
  • MiMo-Audio — evaluated speech LM (standard and thinking variants), whose low probe accuracy shows a genuine perception bottleneck.
  • Kimi-Audio — evaluated open-source speech LM, showing strong modality and language imbalances in fairness and weak cue perception.
  • Whisper — Whisper-large-v3 serves as the intelligibility filter, discarding synthesized clips with WER above 5%.
  • VoiceBench — a capability-oriented voice-assistant benchmark that this work contrasts with its social-alignment focus.
  • Resurfacing Paralinguistic Awareness — cited as diagnostic evidence that paralinguistic signals are suppressed inside audio LMs, consistent with the lexical-dominance account this benchmark supports.