arXiv · 2025 · Preprint
Woszczyk et al. (Imperial College London) · → Paper · Demo: ? · Code: ?
ClaritySpeech proposes an end-to-end pipeline that chains ASR, text-level dementia obfuscation, and zero-shot TTS to conceal cognitive impairment markers in speech while preserving speaker identity without any model fine-tuning.
Problem
Individuals with dementia face two compounding problems: their altered speech patterns — hesitations, lexical errors, incoherent phrasing, irregular prosody — create identifiable acoustic and linguistic markers that expose a sensitive health condition, and existing ASR systems trained on fluent speech fail to transcribe disordered speech accurately. Prior work on speech privacy addressed acoustic features through anonymisation, noise injection, or voice conversion but did not target the cognitive linguistic content that is also diagnostic of dementia. Meanwhile, speaker adaptation methods that could re-synthesise a person’s voice required substantial audio recordings, creating a data-collection burden and introducing further privacy risks if models are trained on external servers.
Method
ClaritySpeech is a three-stage pipeline. First, Whisper large v3 transcribes the dementia-affected audio. Second, a text obfuscation module (DiDOTS, trained via knowledge distillation from a large language model) rewrites the transcript to remove dementia-specific linguistic patterns — incoherent phrasing, excessive pausing, repetition — while preserving semantic intent. Third, a zero-shot TTS system re-synthesises the corrected text conditioned on a short reference segment (approximately 6 seconds) from the original speaker, transferring voice timbre without any model fine-tuning.
The framework evaluates three TTS backends: XTTSv2 (a GPT-based autoregressive language model with HiFiGAN decoding), StyleTTS2 (diffusion-based style transfer with adversarial training), and Hierspeech++ (hierarchical VAE with decoupled F0 and duration control). The paper explicitly exploits the “normalising nature” of zero-shot TTS models: because these models are trained on large, diverse corpora of fluent speech, they naturally smooth out atypical acoustic characteristics such as irregular pitch, long silences, and energy fluctuations when attempting to clone a voice, incidentally filtering dementia markers.
Evaluation is multi-dimensional: dementia leakage is measured as drop in adversarial F1 detection across three modalities (audio, text, fusion) under both static (obfuscation-unaware) and adaptive (obfuscation-aware) adversary conditions; utility is quantified by WER, UTMOS, and ECAPA-TDNN speaker similarity.
Key Results
Across both test sets (ADReSS and ADReSSo), all three TTS backends reduce dementia detection F1, with mean drops of 16% and 10% respectively. XTTSv2 achieves the best privacy-utility balance on ADReSS: WER drops from 0.73 to 0.08, UTMOS improves from 1.65 to 2.15, and speaker similarity reaches 0.50 (Table 1). StyleTTS2 achieves higher audio quality (UTMOS 2.86) and lower WER (0.04) but substantially worse speaker similarity (0.36), indicating that it normalises too aggressively to preserve voice identity. Hierspeech++ maintains competitive speaker similarity (0.52) but carries the highest WER (0.21) and only moderate privacy gain.
Ablation on ADReSS reveals complementary roles for each component: removing text obfuscation raises text-modality F1 by 10% (0.59 to 0.68); removing zero-shot TTS collapses speaker similarity to 0.12 while improving naturalness; removing ASR causes a modest 3% privacy drop across adversaries. The full pipeline achieves a mean real-time factor of 6.07 on CPU, driven by ASR (RTF 3.01) and TTS (RTF 1.71) components.
Novelty Assessment
The contribution is engineering integration. Whisper, DiDOTS, and all three zero-shot TTS backends are existing systems; the paper assembles them into a unified pipeline without architectural modification. The genuine novelty lies in the problem framing: treating zero-shot TTS as a normalising transform that incidentally filters cognitive disorder markers is a creative insight that extends beyond standard speech anonymisation. The multi-dimensional evaluation framework, covering privacy, speaker identity, intelligibility, and audio quality under both static and adaptive adversarial conditions, is more thorough than typical obfuscation papers. The datasets are small and limited to English cookie-theft description tasks, which restricts the generalisability claims substantially.
Field Significance
Moderate — ClaritySpeech demonstrates that off-the-shelf zero-shot TTS can serve as a privacy-preserving normalisation mechanism for a specialised atypical speech domain, extending TTS utility beyond standard fluency and naturalness applications. The paper’s contribution is primarily in problem framing and empirical pipeline design; the underlying speech generation systems are unchanged. It provides useful evidence that the “normalising” side-effect of zero-shot TTS can be a deliberate design choice rather than a limitation.
Claims
-
supports: The normalising tendency of zero-shot TTS models can be deliberately exploited to filter atypical speech characteristics while preserving general voice timbre.
Evidence: ClaritySpeech applies XTTSv2 to dementia-affected speech without fine-tuning, reducing adversarial dementia detection F1 by 16% on ADReSS (total mean 0.70 to 0.59) while maintaining speaker similarity of 0.50, as the model smooths prosodic irregularities learned from diverse fluent training data. (§2.2, §4, Table 1)
-
complicates: Speaker identity preservation and speech naturalness trade off against each other when zero-shot TTS is applied to atypical speech.
Evidence: StyleTTS2 achieves UTMOS 2.86 and WER 0.04 on ADReSS but speaker similarity of only 0.36, while XTTSv2 achieves UTMOS 2.15 and speaker similarity 0.50, showing that models that normalise most aggressively sacrifice voice fidelity. (§4, Table 1)
-
supports: Text-level content correction and acoustic re-synthesis provide complementary privacy protection that neither provides alone in disordered speech obfuscation.
Evidence: Ablation on ADReSS shows that removing text obfuscation raises text-modality F1 by 10% (0.59 to 0.68), while removing zero-shot TTS collapses speaker similarity to 0.12 and degrades privacy protection; the full pipeline requires both components. (§4, Table 2)
-
complicates: Pipeline-based speech obfuscation systems that chain ASR, text processing, and TTS carry latency that precludes real-time deployment on standard hardware.
Evidence: ClaritySpeech achieves a mean real-time factor of 6.07 on CPU for 3-6 second audio samples; ASR alone contributes RTF 3.01 and TTS contributes RTF 1.71, making live speech obfuscation impractical without GPU acceleration. (§4, Table 4)
Limitations and Open Questions
Warning
Evaluation uses small, low-diversity datasets (ADReSS: 690 samples, ADReSSo: 947 samples) from a single task (cookie-theft description). Generalisability to other dementia types, languages, or conversational speech is untested.
The adaptive adversary setting, in which the detection model trains on both original and obfuscated samples, remains difficult to defeat — mean F1 drops are modest (roughly 5 percentage points) under adaptive conditions. No comparison with speaker-dependent fine-tuned TTS models is provided, leaving open whether fine-tuning would offer a better privacy-utility trade-off despite the privacy risks. The system is English-only. Latency (RTF 6.07) rules out real-time applications without hardware acceleration. The effect on the speaker’s perceived identity from the listener’s perspective is measured only automatically via ECAPA-TDNN embeddings, not through subjective listening tests.
Wiki Connections
- Zero-Shot TTS — the framework uses zero-shot TTS as its re-synthesis backbone, leveraging its training-free speaker cloning capability and normalising behaviour to filter atypical speech characteristics.
- Speaker Adaptation — the pipeline adapts voice identity at inference time using a short reference audio clip, avoiding any per-speaker model training.
- Voice Conversion — ClaritySpeech converts dementia-affected speech to normalised output with the same speaker timbre, addressing a specialised atypical-to-typical voice transformation that overlaps conceptually with VC.
- Evaluation Metrics — the paper employs UTMOS, WER, and ECAPA-TDNN cosine speaker similarity together with adversarial F1 classification to evaluate a multi-dimensional privacy-utility trade-off, illustrating how automatic metrics can proxy subjective assessments in privacy-critical applications.