IWSLT 2026 · 2026 · Workshop
Yu He et al. (Huawei Translation Service Center / Northwestern Polytechnical University) · → Paper · Demo: ✗ · Code: ✗
A training-free shared-task pipeline for Chinese and French cross-lingual voice cloning. It slices each long reference recording into overlapping 10-second prompts, runs the frozen Qwen3-TTS-12Hz-1.7B-Base once per prompt, and keeps the candidate whose ECAPA-TDNN embedding is closest to the target speaker.
Problem
The IWSLT 2026 Cross-Lingual Voice Cloning Track supplies 12 reference recordings of about five minutes each. Systems must speak given Chinese, French or Arabic sentences in those voices, and the reference audio is in a different language from the target. Prompt-based voice cloning models expect a short prompt, so a five-minute recording must be cut down before use. The authors argue that one short crop may miss parts of a speaker’s timbre and prosody. They also cite weak speaker similarity with short or noisy references as a known failure of current cloning systems. The team entered the Chinese and French tasks only.
Method
The system has no training stage. It has three steps.
- Backbone. The pretrained Qwen3-TTS-12Hz-1.7B-Base is used frozen (Qwen3-TTS). It is a 1.7B-parameter multi-codebook language model over tokens from the 16-layer Qwen3-TTS-Tokenizer-12Hz. It clones a voice from an audio prompt with no adaptation. The authors chose it because it covers both target languages and needs no extra adaptation work (§2).
- Reference slicing (§3.1). Each roughly five-minute recording is cut with a 10-second window and a 5-second step. This gives around 60 overlapping segments per speaker, all of which are kept. Audio is loaded at its original sampling rate with no resampling.
- Batch generation (§3.2). Every segment acts as a separate speaker prompt for every sentence in the official
chinese.txtandfrench.txtfiles. Each sentence therefore gets one candidate per segment, around 60 per speaker. - Best-of-N selection (§3.3). For each speaker, three 10-second segments are cropped at random from the original recording as the “baseline timbre samples”. ECAPA-TDNN (ECAPA-TDNN) embeddings are extracted from these crops and from every candidate. The candidate with the highest cosine similarity is kept for each sentence. §3.3 also names these crops as the reference for evaluating all cloned audio.
The paper reports no decoding settings (sampling temperature, seeds) and no compute cost for generating about 60 candidates per sentence.
Key Results
All evaluation is automatic and covers the blind data of each track (§4). The paper reports results for each of the 12 speakers but gives no averages. Rows are labelled with ACL 2023 paper IDs (for example 2023.acl-long.193), which the paper uses as speaker identifiers without further description. Content accuracy uses Qwen3-ASR-0.6B transcripts, scored as CER for Chinese and WER for French. Speaker similarity is ECAPA-TDNN cosine similarity.
| Track | Error rate range (%) | Cos Sim range | Speakers with Cos Sim below 0.5 |
|---|---|---|---|
| Chinese (Table 1) | CER 1.39 (.810) to 2.25 (.290) | 0.4236 (.14) to 0.7052 (.193) | 2 of 12 (.14, .290) |
| French (Table 2) | WER 4.40 (.3) to 8.52 (.289) | 0.4420 (.290) to 0.7173 (.23) | 2 of 12 (.290, .809) |
- Intelligibility. Chinese CER stays at or below 2.25% for all speakers. French WER is at or below 5.87% for every speaker except
2023.acl-long.289(8.52%). The paper offers no ground-truth or reference-system error rate as a point of comparison. - Speaker similarity. Similarity varies widely between speakers in both tracks. The French speaker
2023.acl-long.290scores 0.4420, the lowest French value. The authors attribute weak cases to cross-lingual prosodic mismatch between reference and target language (§4.2), but they do not test this. - Text versus table. §4.2 names
2023.acl-long.290(0.4430) as the lowest Chinese similarity. Table 1 lists2023.acl-long.14at 0.4236, which is lower and matches the range that §4.2 itself states (0.4236 to 0.7052). Both values were checked against the PDF.
There are no baselines. The paper does not compare against Qwen3-TTS with a single prompt or without selection, any other cloning system, or the track’s official ranking or organizer scores. It runs no ablation of window length, step, or candidate count.
Novelty Assessment
The contribution is engineering integration. Prompt-based cloning with a pretrained multilingual TTS model, sliding-window cropping of reference audio, and best-of-N re-ranking with a speaker verification model are all established techniques. Here they are combined for a shared task. The pipeline is simple and reproducible in principle. However, the paper shows no evidence that slicing or selection helps, because no condition without them is reported. The selection criterion and the reported similarity metric come from the same scorer, so the main similarity numbers partly measure the selection step itself.
Field Significance
Low — This is a shared-task system description. It reports single-language automatic results per track for an inference-time wrapper around a frozen pretrained model. Its main value is as a data point: even after choosing the best of about 60 candidates by speaker embedding, cross-lingual similarity with a pretrained codec language model still varies widely between speakers.
Claims
- complicates: Speaker-similarity scores lose meaning as evidence when the same speaker-verification model is used to select among generated candidates and then to report similarity.
Evidence: Candidates are chosen by maximum ECAPA-TDNN cosine similarity to three crops of the target recording. The reported Cos Sim uses the same ECAPA-TDNN cosine measure, with no unselected or single-prompt condition for comparison. (§3.3, §4.2, Tables 1–2)
- complicates: In cross-lingual voice cloning, speaker similarity varies widely between target speakers even with candidate re-ranking.
Evidence: After best-of-N selection over around 60 prompt segments per speaker, Cos Sim spans 0.4236–0.7052 for Chinese and 0.4420–0.7173 for French, with 2 of 12 speakers below 0.5 in each track. The authors suggest cross-lingual prosodic mismatch as the cause but do not test it. (§4.2, Tables 1–2)
- supports: A pretrained multilingual prompt-based TTS model can perform cross-lingual voice cloning without training and still keep content errors low.
Evidence: The frozen Qwen3-TTS-12Hz-1.7B-Base gives Chinese CER of 1.39–2.25% and French WER of 4.40–8.52% across 12 speakers, measured with Qwen3-ASR-0.6B transcripts. No reference or baseline error rate is reported. (§4.1, Tables 1–2)
Limitations and Open Questions
Warning
The reported speaker-similarity numbers are optimistic. They come from the same ECAPA-TDNN cosine measure that selects the output from about 60 candidates per sentence, and the reference crops come from the same recordings that supply the prompts. No baseline, ablation, human listening test, or official track score is reported, so the effect of slicing and selection is not measured.
- Evaluation is automatic only (ASR-based CER/WER and one speaker-embedding model). There is no naturalness rating, MOS, or human speaker-similarity judgment.
- Each track covers one target language. The Arabic task was not attempted.
- The paper does not state the language of the reference recordings, beyond saying that it differs from the target language.
- Generating one candidate per segment multiplies inference cost by about 60. The paper does not report this cost and does not test whether fewer segments would give similar results.
- Whether the target speakers are unseen by the pretrained model cannot be established, because Qwen3-TTS’s training data is not described here.
Wiki Connections
- Autoregressive Codec TTS — The system uses a frozen multi-codebook codec language model (Qwen3-TTS-12Hz-1.7B-Base) and steers it at inference time through prompt slicing and best-of-N selection.
- Qwen3-TTS — The frozen backbone for all generation in both tracks, used with its native prompt-based voice cloning.
- ECAPA-TDNN — Provides the speaker embeddings used both to select the best candidate and to report speaker similarity.
- CosyVoice 3 — Cited as one of several large pretrained cloning models; not compared against.
- IndexTTS2 — Cited as an alternative autoregressive zero-shot TTS model; not compared against.