arXiv · 2026 · Preprint
Yuxin Li et al. (Nanyang Technological University) · → Paper · Demo: ? · Code: ?
Introduces a disentangled, depression-conditioned flow-matching TTS system that synthesizes speech pairing depressive acoustic patterns with non-depressive text, producing a counterfactual augmentation dataset that improves the robustness of downstream depression-detection classifiers.
Problem
Widely used depression-detection datasets such as DAIC-WOZ exhibit a strong statistical coupling between linguistic sentiment and diagnostic labels: the paper’s own sentiment analysis of DAIC-WOZ finds negative sentiment substantially more frequent in depressed than in healthy speech, confirmed by a chi-square test of independence. Models trained on such data are encouraged to rely on this sentiment-diagnosis shortcut rather than depression-related acoustic markers, which fails in real-world “Camouflaged Depression” scenarios where individuals maintain socially positive or neutral language despite underlying depressive states. Existing data-augmentation strategies for this problem either only perturb samples within a class (leaving the sentiment-diagnosis correlation untouched) or use generic voice-cloning TTS that reproduces diverse prosody without targeting the specific acoustic-semantic mismatch the detection models need to see.
Method
DepFlow is a three-stage, depression-conditioned TTS framework. First, a Depression Acoustic Encoder (DAE) takes frame-level WavLM-Large features, pools them into an utterance-level embedding, and passes it through a post-encoder to produce a 32-dimensional depression embedding. Three heads are attached to this embedding: an ordinal regression head predicting PHQ-8 severity thresholds, and two adversarial (gradient-reversal) heads that suppress speaker identity and residual linguistic content (using HuBERT-derived pseudo-phoneme labels as the content-adversarial target), so the embedding retains severity information while discarding speaker and content cues.
Second, the DepFlow TTS model adapts the Matcha-TTS architecture (text encoder, duration predictor, and a U-Net conditional flow-matching decoder) to accept the DAE-derived depression embedding as an additional conditioning signal, injected via FiLM: a lightweight MLP produces per-block scale-shift parameters that apply channel-wise affine modulation to intermediate decoder activations. The authors motivate FiLM over concatenation (too weak to influence deep decoder layers) or cross-attention (added computational overhead) as a way to apply uniform, global conditioning influence across the full decoding hierarchy. The backbone is pretrained on CSTR VCTK without depression conditioning, then fine-tuned on DAIC-WOZ with the depression condition and FiLM generator added, training on duration loss, a Gaussian prior loss, and the flow-matching velocity-regression loss. A pretrained HiFi-GAN vocoder converts generated mel-spectrograms to waveform.
Third, a prototype-based severity mapping mechanism converts a continuous PHQ-8 target score into a smooth depression conditioning embedding: subject-level DAE embeddings are averaged and normalized into five severity-bin prototypes, and at inference a target severity score is mapped to an interpolation weight between the two adjacent prototypes using spherical linear interpolation (SLERP), which the authors argue preserves angular geometry and avoids linear-interpolation artifacts.
Key Results
The DAE embedding shows high speaker-verification EER (0.355) and low linguistic-content recoverability (R² 0.21, CKA 0.014 against HuBERT features) while retaining ROC-AUC 0.693 for depression classification via linear probing, indicating the embedding suppresses speaker and content information while preserving diagnostic signal. In the severity-controlled synthesis protocol (five severity variants per DAIC-WOZ test utterance, 4,700 total samples), the projected embedding trajectory shows a Concordance Index of 0.744 and Spearman correlation of 0.598 against intended severity, and several clinically recognized acoustic correlates (silence-speech ratio, shimmer, HNR, formant centralization) shift monotonically with intended severity. Synthesized speech WER (13.93%) closely matches natural DAIC-WOZ speech (14.06%), and speaker similarity (SIM-o) remains stable across severity levels, indicating severity manipulation does not compromise intelligibility or speaker identity.
For the downstream task, augmenting three depression-detection architectures (DepAudioNet, an ECAPA-TDNN-based speaker-disentangling model, and a WavLM-based hierarchical CTC model) with the DepFlow-generated Camouflage Depression-oriented Augmentation (CDoA) dataset improves macro-F1 by 9%, 12%, and 5% respectively over no augmentation, and consistently outperforms conventional augmentation (FrAUG, SpecAugment, Mixup) and a CosyVoice-2-based voice-cloned TTS baseline that generates similarly camouflage-oriented data without a learned depression-aware acoustic manifold.
Novelty Assessment
The TTS architecture itself, Matcha-TTS with a conditional flow-matching decoder and a HiFi-GAN vocoder, is not novel; the contribution is the combination of an adversarially disentangled depression embedding, FiLM-based global conditioning, and SLERP-based prototype interpolation to give the TTS system a controllable, clinically grounded severity axis, plus the specific data-generation strategy (pairing depressive acoustics with non-depressive text) targeting a documented dataset bias. The comparison against a CosyVoice-2-based voice-cloning baseline that performs the same camouflage-style augmentation task without disentangled depression conditioning is the key ablation-like evidence that the disentanglement step, not merely synthetic data volume or diversity, drives the downstream gains. The severity-controllability evaluation (Concordance Index, Spearman correlation, intra-speaker acoustic correlations) is relatively thorough for a paper whose primary contribution is a downstream augmentation pipeline rather than the TTS architecture. The paper is explicit that its downstream results are correlational rather than causal.
Field Significance
Moderate, the paper provides a concrete demonstration that adversarially disentangled, continuously controllable TTS conditioning can generate targeted counterfactual training data that measurably improves robustness against a specific, documented shortcut-learning failure mode in a clinical speech task. Its direct relevance to the TTS field is narrower than papers advancing generation quality or architecture: the TTS component here is a means to a downstream robustness end, evaluated on a single clinical dataset (DAIC-WOZ) and paired with a single pretraining corpus (VCTK).
Claims
- supports: FiLM-based global conditioning enables continuous, monotonic control over a paralinguistic acoustic attribute in flow-matching TTS without degrading linguistic intelligibility or speaker identity.
Evidence: Severity-controlled synthesis shows strong ordinal consistency between intended severity and the projected embedding trajectory (Concordance Index 0.744, Spearman ρ 0.598), while synthesized-speech WER (13.93%) closely matches natural recordings (14.06%) and speaker similarity remains stable across severity levels. (§5.2.1, §5.2.3, Table 5)
- supports: Adversarial gradient-reversal training can suppress speaker and linguistic information from an acoustic embedding while retaining task-relevant discriminative signal.
Evidence: The depression acoustic embedding achieves high speaker-verification EER (0.355) and low content-recovery R² (0.21) and CKA (0.014) after adversarial training, while still reaching ROC-AUC 0.693 for depression classification via linear probing. (§5.1, Table 3)
- supports: Disentangled, condition-controllable TTS synthesis can generate counterfactual acoustic-semantic combinations that improve the robustness of downstream classifiers trained on datasets with spurious label-feature correlations.
Evidence: Augmenting three depression-detection architectures with a dataset pairing depressive acoustics with non-depressive text improves macro-F1 by 9%, 12%, and 5% respectively, outperforming conventional augmentation (Mixup, SpecAugment, FrAUG). (§5.3.1, Table 4)
- complicates: Generic voice-cloning TTS, without an explicit disentangled conditioning mechanism for the target attribute, is insufficient for generating counterfactual augmentation data that transfers into downstream robustness gains.
Evidence: A CosyVoice-2-based voice-cloned TTS baseline producing similarly camouflage-oriented augmentation data, but without a learned depression-aware acoustic manifold, underperforms the disentangled approach across all three downstream detection architectures. (§5.3.1, Table 4)
Limitations and Open Questions
Warning
The learned severity axis has not been validated against clinician ratings or across broader populations and datasets; all severity structure is derived from PHQ-8 self-report labels within a single dataset (DAIC-WOZ), and the downstream robustness gains are correlational, not causally established.
- The paper’s own ethical discussion flags that synthesizing speech with depressive acoustic characteristics introduces risks of misuse, cultural-stereotype reinforcement, and psychological harm, and argues for governance, informed consent, and provenance/watermarking safeguards.
- Evaluation is restricted to English speakers (VCTK pretraining, DAIC-WOZ fine-tuning); the authors flag multilingual and demographically diverse extensions as future work.
- Downstream evaluation covers three depression-detection architectures on one dataset; generalization to other clinical speech corpora or other shortcut-learning scenarios is untested.
Wiki Connections
- Flow Matching — adapts a Matcha-TTS-style conditional flow-matching decoder, adding FiLM-based conditioning on a learned depression embedding rather than proposing a new flow-matching objective.
- Disentanglement — trains the Depression Acoustic Encoder with adversarial gradient-reversal heads to explicitly suppress speaker and linguistic content from the depression embedding, with EER, R², and CKA measurements as disentanglement evidence.
- Flow Matching for Generative Modeling — the paper’s decoder training follows the conditional flow matching formulation from this paper, as implemented in Matcha-TTS.
- HiFi-GAN — used as the pretrained vocoder that converts DepFlow’s generated mel-spectrograms to waveform.
- CosyVoice 2 — the paper’s downstream detection experiments include a CosyVoice-2-based voice-cloned TTS baseline for generating camouflage-oriented augmentation data, which DepFlow outperforms.
- Seed-TTS — the paper’s WER evaluation protocol (Whisper-Large-v3 transcription) follows the protocol established in this paper.