arXiv · 2023 · Preprint
Tu Anh Nguyen et al. (Meta AI) · → Paper · Demo: ? · Code: ✓
Introduces EXPRESSO, a 47-hour expressive speech dataset covering 26 spontaneous styles (read and improvised dialogue), paired with a resynthesis benchmark that evaluates how well discrete self-supervised units encode and transfer expressive content and speaker identity.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
EXPRESSO addresses a gap in the training resources available for textless or discreteunit-based speech synthesis: most existing datasets consist of read speech, which lacks the spontaneity, hesitations, non-verbals, and paralinguistic variety found in natural conversation. The dataset provides 47 hours of professionally recorded North American English speech from 4 speakers, spanning 7 read styles and 25 improvised dialogue styles, recorded at 48 kHz. The accompanying benchmark defines automatic metrics for content preservation (WER), pitch preservation (F0 Frame Error), and expressive style preservation (classifier accuracy), enabling systematic comparison of different self-supervised discrete encoders (HuBERT variants and EnCodec) across in-domain and out-of-domain conditions. By exposing tradeoffs between bitrate, phonetic quality, and disentanglement from speaker and style, the benchmark provides a structured evaluation framework for discrete speech representation methods used in expressive TTS and voice conversion research.
Wiki Connections
- self-supervised-speech — the benchmark evaluates HuBERT-based discrete units as the primary encoder family and measures how well SSL-derived representations disentangle content, speaker, and expressive style.
- evaluation-metrics — introduces three task-specific automatic metrics (WER for content, FFE for pitch, classifier accuracy for expressivity) tailored to the discrete expressive resynthesis setting.
- emotion-synthesis — EXPRESSO’s 26 expressive styles and improvised dialogue section provide training and evaluation material for systems that must capture and transfer spontaneous emotional and paralinguistic speech characteristics.
- neural-codec — the benchmark compares HuBERT units against EnCodec (RVQ-1 and RVQ-8) as discrete representations, revealing tradeoffs between compression quality and controllability for voice and style transfer.
- 2209.03143 (AudioLM) — EXPRESSO targets the same discrete-unit resynthesis paradigm that AudioLM builds on, extending evaluation to expressive and spontaneous speech conditions.
- 2210.13438 (EnCodec) — the benchmark uses EnCodec as a high-bitrate baseline tokenizer, contrasting its compression-oriented representations with the more phonetically disentangled HuBERT units.
- 2104.00355 (Speech resynthesis from discrete disentangled SSL representations) — EXPRESSO directly extends the discrete resynthesis task introduced in this prior work by adding expressive style as an additional dimension to preserve and evaluate.