arXiv · 2026 · Preprint

Haoyu Wang et al. (Tianjin University) · → Paper · Demo: ? · Code: ?

ExpPro is a training-free, two-stage procedure for choosing which reference clip to feed a zero-shot LM-based TTS model, filtering emotional prompt candidates by their own acoustic and textual properties and by how well a specific TTS model reproduces them, then picking the candidate whose transcript best matches the target text.

Problem

Zero-shot LM-based TTS systems inherit timbre, emotion, and quality from a few seconds of prompt speech, so the choice of prompt strongly shapes the output. The two common selection practices, random choice from a speaker’s emotional recordings or retrieval by prompt-text/target-text similarity, were designed for general synthesis. For emotional synthesis they often yield prompts with weak emotional intensity or unstable speaker cues, and text similarity is a poor proxy because the link between sentence content and the desired emotion is frequently weak. The paper asks whether emotion intensity and speaker consistency can be improved purely through better prompt selection, without retraining the TTS model.

Method

ExpPro (Expressive Prompting) operates on a pool of candidate prompt recordings per speaker and emotion and has a static stage, run once before synthesis, and a dynamic stage, run per input sentence.

Static stage, prompt speech quality (§II.B.1 to §II.B.2). Each candidate is first characterised by the mean and variance of its pitch contour, motivated by the observation that sad and comfort speech cluster at low pitch mean and variance while happy and surprised speech sit higher (Fig. 2). K-Means groups candidates into 10 clusters, and the m clusters with the most emotion-typical pitch statistics (stronger or weaker depending on the emotion class) are kept. The survivors are then scored by a sum of DNSMOS overall perceptual quality and a text-emotion coherence score: ChatGPT is given a fixed prompt (Table I) asking it to rate, to four decimal places, how well the candidate’s transcript matches its emotion label. The top n% by the combined score are retained.

Static stage, prompt model performance (§II.B.3). Because the same prompt behaves differently in different TTS models, each remaining candidate is used to synthesise 20 neutral descriptive sentences with the target model. The outputs are scored by CER (Paraformer-zh ASR), speaker similarity to the prompt (Resemblyzer and WavLM-large embeddings), and emotion similarity (ES, cosine similarity of emotion2vec-large representations), combined via a weighted sum (weights not reported), and the top k candidates are kept.

Dynamic stage (§II.C). At inference time, a sentence-embedding model (stsb-distilroberta-base) compares the input text with the transcripts of the k static-stage survivors, and the most semantically similar one becomes the prompt. Final settings are m = 3, n = 15, k = 5.

No model is trained. The method is evaluated as a wrapper around two pretrained LM-based zero-shot TTS systems, CosyVoice and GPT-SoVITS; model size and tokenizer details are not reported.

Key Results

Experiments use HE2D, a highly expressive Chinese emotional dataset from the authors’ prior work (two male, two female speakers, five emotions, four emotions per speaker, 200 utterances each), plus four Chinese speakers from ESD. Subjective tests use 20 native listeners rating Emotion MOS (which the paper defines as covering both quality and emotional expression) and Strength Perception (SP, a 0 to 1 rating of emotional strength), with 95% confidence intervals. Baselines are random selection and text-similarity retrieval with all-MiniLM-L6-v2.

  • Main comparison (Table II). On HE2D, ExpPro gives the highest MOS and SP for every emotion with both models. With CosyVoice, Happy MOS rises from 4.13 (random) and 4.33 (MiniLM) to 4.45 and SP from 0.767 to 0.811; Angry SP rises from 0.733 to 0.867. With GPT-SoVITS, Happy MOS goes from 4.02 (random) to 4.39. On ESD the gains are much smaller (CosyVoice Happy MOS 3.83 to 3.87, with intervals of about ±0.12 to ±0.14), and the authors note that improvements grow with the expressiveness of the prompt pool.
  • Model dependence (Table III). Ranking the same happy-emotion candidates from one HE2D speaker gives different top-3 and bottom-2 prompts for CosyVoice and GPT-SoVITS. Within CosyVoice, the top prompt reaches CER 1.55%, Resemblyzer similarity 93.66, and ES 98.37, against 2.63%, 89.65, and 43.36 for the bottom prompt, a large spread from prompt choice alone.
  • Selection range (Table IV). Tightening the pitch-cluster selection (smaller m) and the quality/coherence cut (smaller n) generally increases ES and SP, e.g. ES 89.83 to 95.59 as n falls from 25 to 10 at m = 3.
  • Ablations (Tables V and VI). Inverting the prompt-speech-quality stage lowers SP for all five emotions (e.g. Happy 0.782 to 0.633), but the objective metrics are mixed. Inverting the model-performance stage lowers MOS only by 0.01 to 0.05, inside the reported confidence intervals.

Comparisons are fair in the sense that all strategies select from the same candidate pool and use the same TTS models, but no statistical significance tests are reported, and many MOS differences on ESD and in Table VI fall within overlapping 95% intervals.

Novelty Assessment

The contribution is engineering integration: pitch clustering, DNSMOS, an LLM-as-judge text-emotion score, ASR/speaker/emotion embedding metrics, and sentence-embedding retrieval are all established tools, combined here into a prompt-curation pipeline. The most useful idea is the explicit model-in-the-loop stage, which treats prompt quality as a property of the prompt and the target model jointly rather than of the recording alone. The selection is a filtering heuristic with hand-chosen thresholds and undisclosed weights rather than a learned or principled criterion, and the evaluation is limited to Chinese speech, two TTS systems, and a small, partly in-house dataset.

Field Significance

Moderate — The paper provides direct evidence that, for LM-based zero-shot TTS, emotional intensity and speaker consistency vary substantially with the choice of reference prompt and that a prompt’s usefulness depends on the specific model consuming it. It offers a practical, training-free recipe for emotional prompt curation, but its evidence base is narrow and several component-level gains are small.

Claims

  • supports: In prompt-conditioned zero-shot TTS, the choice of reference prompt alone can produce large differences in intelligibility, speaker similarity, and emotional fidelity of the output, with the model held fixed.

    Evidence: With CosyVoice and the same speaker and emotion, the top-ranked prompt yields CER 1.55%, Resemblyzer similarity 93.66, and emotion similarity 98.37, versus 2.63%, 89.65, and 43.36 for the bottom-ranked prompt. (§IV.B, Table III)

  • supports: The usefulness of a given reference prompt is model-dependent, so prompt quality for zero-shot TTS cannot be judged from the recording alone.

    Evidence: Ranking identical happy-emotion candidates by weighted CER, speaker similarity, and emotion similarity produces different top-3 and bottom-2 prompts for CosyVoice and GPT-SoVITS. (§IV.B, Table III)

  • supports: Curating reference prompts for acoustic emotional salience and transcript-emotion coherence increases perceived emotional intensity in zero-shot emotional TTS without any model training.

    Evidence: On HE2D, ExpPro selection raises CosyVoice SP over random selection for all five emotions (e.g. Angry 0.733 to 0.867), and inverting the prompt-speech-quality stage lowers SP for every emotion (e.g. Happy 0.782 to 0.633). (§IV.A, Table II; §IV.D, Table V)

  • complicates: Gains from better prompt selection shrink when the available prompt pool is only mildly expressive.

    Evidence: On ESD the MOS gains over random selection are 0.02 to 0.18 across both models, often with overlapping 95% confidence intervals, much smaller than on the more expressive HE2D, which the authors attribute to weaker emotional expressiveness in the prompt speech. (§IV.A, Table II)

Limitations and Open Questions

Warning

Evaluation covers only Chinese speech from eight speakers across two datasets, one of them (HE2D) the authors’ own dataset described in prior work, with two TTS systems. No significance tests are reported, and many ESD and ablation differences fall within the reported confidence intervals.

  • The model-performance stage contributes little in the subjective ablation: inverting it lowers MOS by only 0.01 to 0.05 (Table VI), so its value rests mainly on the objective spreads in Table III.
  • The prompt-speech-quality ablation is mixed on objective metrics; the inverted selection matches or exceeds ExpPro on some speaker-similarity and ES entries (e.g. Sad ES 97.47 vs 96.31; Surprised Resemblyzer 89.25 vs 87.41) (Table V).
  • The weights of the combined scores, the direction of pitch-cluster selection per emotion, and the exact ChatGPT version are not fully specified, which limits reproducibility. The reliability of the ChatGPT coherence score is backed only by informal manual inspection.
  • The static stage requires a sizable per-speaker, per-emotion candidate pool (200 utterances per emotion in HE2D) and many synthesis runs per candidate, which may not be available in typical zero-shot use with a single short reference clip.
  • Emotion MOS mixes quality and emotional expression into one score, so it does not separate naturalness from expressiveness.
  • Whether the approach transfers to non-LM zero-shot TTS (e.g. flow-matching systems) or other languages is not tested, despite the claim of applicability to any prompt-based TTS model.

Wiki Connections

  • Zero-Shot TTS — Shows that reference-prompt selection is a major, training-free lever on zero-shot TTS output quality and speaker consistency.
  • Emotion Synthesis — Targets emotion intensity in prompt-driven emotional synthesis by curating emotionally salient prompts rather than adding explicit emotion conditioning.
  • Autoregressive Codec TTS — Evaluates prompt sensitivity in two LM-based token TTS systems and finds that the best prompts differ between them.
  • Subjective Evaluation — Uses a 20-listener Emotion MOS and a Strength Perception test with 95% confidence intervals as its primary evidence.
  • CosyVoice — Primary TTS model wrapped by ExpPro; the authors attribute its stronger speaker and emotion similarity over GPT-SoVITS to its ASR-supervised tokenizer and x-vector input.
  • emotion2vec — Provides the emotion representations used both to rank candidates and to measure emotion similarity.
  • Paraformer — ASR model used to compute CER for candidate ranking and evaluation.
  • VALL-E — Cited as a representative LM-based zero-shot TTS system whose in-context prompting motivates the work.
  • Seed-TTS — Cited as a representative LM-based zero-shot TTS system whose output depends on prompt speech.