arXiv · 2026 · Preprint
Wataru Nakata et al. (The University of Tokyo / National Institute of Advanced Industrial Science and Technology) · → Paper · Demo: ? · Code: ✓
Builds a re-runnable pipeline that turns public podcast audio into two-channel, speaker-separated dialogue, and uses it to release DuplexChat: about 415k hours (282,634 h English, 132,723 h Japanese) of full-duplex dialogue for training spoken dialogue language models.
Problem
Full-duplex spoken dialogue language models (SDLMs, e.g. dGSLM and Moshi) need dialogue in which each speaker sits on a separate audio track. Only then do turn-taking, backchannels and overlaps stay observable in training data. The public resources with separate channels are telephone corpora (CallHome at about 0.3k h, Fisher at about 2k h). They need paired participants and do not scale. Web-scale corpora such as GigaSpeech, YODAS, the Spotify Podcast Dataset, J-CHAT and Emilia are monaural, so per-speaker timing is lost once the voices are mixed (Table I). The paper targets the missing piece: an open, scalable way to get speaker-separated dialogue from web audio.
Method
DuplexChat-Pipe runs in four stages (§II, Fig. 1). No new model is trained; every stage uses an existing tool.
- Feed collection and language filtering (§II-A). The crawl starts from the public PodcastIndex database, which the authors’ earlier J-CHAT work also used. It keeps RSS feeds whose language tag matches the target locale and removes duplicate feed URLs.
- Audio retrieval and cleaning (§II-B). Feeds whose category tags match a curated list of music keywords are dropped. Episode audio is downloaded and resampled to 16 kHz. Episodes longer than three hours are removed, since in this crawl they are mostly music programs and long streams.
- Dialogue segmentation (§II-C). An open-source diarization model (pyannote speaker-diarization-community-1) labels each episode. Audio is split at silences of 5 s or more. The pipeline keeps only spans where exactly two speakers are active and drops spans shorter than 10 s. It also drops spans where one speaker holds more than 80% of the diarized speech, treating them as monologues. Spans over 10 minutes are cut into shorter pieces to fit the separator’s memory. Episodes with fewer than four valid clips are discarded.
- Speech separation and restoration (§II-D). Each mixed two-speaker clip goes through DialogueSidon, the authors’ diffusion-based model that jointly separates and restores (denoises, dereverberates) in-the-wild dialogue mixtures. The two estimated tracks are stored as left and right stereo channels.
The resulting corpus covers 13,041 English and 8,971 Japanese feeds, 664,209 and 401,494 episodes, and about 15.3M and 7.3M dialogues, with a mean dialogue length of 66.5 s and 65.2 s (Table II). The Japanese crawl covers every PodcastIndex feed available in April 2026. The English crawl covers only about 2% of available feeds because of compute limits (§III). For copyright reasons, the release contains only audio URLs and segment durations plus code to rebuild the audio; it also offers an opt-out process (§Ethics Statement).
Key Results
The paper trains no dialogue model. Its evaluation is a reference-free audio-quality comparison against Fisher and a turn-taking analysis of the corpus itself. All scores are automatic; there is no human listening test.
Audio and separation quality (§III-A, Table III; 600 random samples per corpus). Both DuplexChat subsets score above Fisher on predicted cleanliness:
| Corpus | DNSMOS | SQ-STOI | SQ-PESQ | ITC | ITD |
|---|---|---|---|---|---|
| DuplexChat-En | 2.92 | 0.96 | 3.23 | 0.383 | 0.88 |
| DuplexChat-Ja | 3.11 | 0.98 | 3.30 | 0.32 | 0.867 |
| Fisher | 2.72 | 0.91 | 2.33 | 0.383 | 0.84 |
SQ-STOI and SQ-PESQ are STOI and PESQ values predicted by the SQUIM model, not computed against a clean reference. ITC (intra-track consistency) is the cosine similarity of frame-wise ECAPA speaker embeddings within one separated track. ITD (inter-track distinctiveness) is one minus the cosine similarity between the two tracks’ speaker embeddings. Higher is better for both. DuplexChat-En ties Fisher on ITC (0.383) and is slightly higher on ITD (0.88 vs 0.84). DuplexChat-Ja has the lowest ITC of the three (0.32), but its ITD (0.867) is above Fisher’s. The authors attribute the weaker Japanese separation to DialogueSidon’s limited Japanese training data. The comparison is not like for like: Fisher’s channels are recorded telephone audio, while DuplexChat’s are restored model outputs. Restoration directly targets the quality that DNSMOS and SQUIM predict.
Turn-taking dynamics (§III-B, Table IV; about 700 English and 900 Japanese dialogues). The analysis follows the turn-taking procedure of Full-Duplex-Bench, with Silero VAD per channel, 0.5 s merging into talk spurts, and Parakeet ASR word counts. Japanese dialogue shows more turn exchanges (10.5 vs 6.2 per minute), shorter turns (4.0 vs 7.9 s), more backchannels (5.6 vs 3.1 per minute) and more simultaneous speech (21% vs 10%) than English. Overlapping transitions are similar (50% vs 48%). The authors note that this direction matches earlier cross-language turn-taking analyses. No matching statistics are reported for Fisher or another reference corpus, so the numbers show that the corpus contains overlaps and backchannels, but not that it matches a known human baseline.
Novelty Assessment
The contribution is data scale plus engineering integration. Every pipeline stage uses an existing component: PodcastIndex crawling and diarization-based two-speaker segmentation follow J-CHAT, pyannote diarization is off the shelf, and the separator is the authors’ separately published DialogueSidon. What is new is chaining separation and restoration onto web-scale podcast dialogue to produce two-channel output, and releasing the pipeline so others can rerun it. By the paper’s Table I, the result (about 415k h) is the largest dialogue resource it compares against, and the only one that pairs separated channels with a released pipeline. As a 4-page demo-track preprint, it gives no evidence of the property that would matter most: whether training on these reconstructed channels improves a full-duplex model.
Field Significance
Low: the corpus is large (about 415k h, roughly 200 times Fisher) and comes with a rerunnable pipeline, but the release is URLs plus rebuild code rather than audio, validation is automatic and limited to small samples, and no dialogue model is trained on it. The paper shows that separation and restoration can turn monaural web dialogue into two-channel data. It does not yet show that this data trains better dialogue models than recorded telephone corpora do.
Claims
- supports: Running speech separation and restoration on diarized monaural web dialogue can produce speaker-separated, two-channel dialogue data at a scale far beyond recorded telephone corpora.
Evidence: Diarization-guided two-speaker segmentation followed by DialogueSidon separation of podcast audio yields 282,634 h of English and 132,723 h of Japanese stereo dialogue, against about 2k h for Fisher and 0.3k h for CallHome, the recorded-channel resources in Table I. (§II-C, §II-D, Table I, Table II)
- complicates: Separation quality of reconstructed dialogue tracks depends on the separator’s coverage of the target language, so speaker-separated web corpora are not uniformly reliable across languages.
Evidence: Japanese tracks reach lower intra-track speaker consistency (ITC 0.32) than English tracks and Fisher (both 0.383), which the authors attribute to DialogueSidon’s limited Japanese training data; inter-track distinctiveness for Japanese (0.867) still sits above Fisher (0.84). (§III-A, Table III)
- complicates: Reference-free quality and speaker-embedding scores on separated, restored tracks are an indirect proxy for full-duplex training value, and can favor restored web audio over recorded channels.
Evidence: DuplexChat beats Fisher on DNSMOS and on SQUIM-predicted STOI and PESQ, but no spoken dialogue model is trained on the corpus; downstream effects are left to future work. (§III-A, Table III, §IV)
- supports: Turn-taking statistics in conversational speech vary substantially by language, while the share of turn transitions that involve overlap can stay similar.
Evidence: Japanese podcast dialogue has more turn exchanges (10.5 vs 6.2 per minute), more backchannels (5.6 vs 3.1 per minute) and more simultaneous speech (21% vs 10%) than English, but a similar share of overlapping transitions (50% vs 48%). (§III-B, Table IV)
Limitations and Open Questions
Warning
No spoken dialogue model is trained or evaluated on DuplexChat. Its value for full-duplex modeling rests on automatic audio-quality scores and turn-taking statistics alone, and the authors leave downstream training to future work.
- The separated channels are model estimates, not recordings. Separation errors such as leakage of one speaker into the other track, or a speaker swap within a clip, could distort the overlap and backchannel statistics that full-duplex models learn from. The paper does not measure how often such errors occur.
- Evaluation uses small random samples (600 clips for quality; about 700 English and 900 Japanese dialogues for turn-taking), with no human listening test and no turn-taking baseline from a recorded corpus.
- The English portion covers only about 2% of available PodcastIndex feeds, so the English statistics may not reflect the full source. The Japanese portion is close to the ceiling for this source.
- The release contains audio URLs and segment metadata, not audio. Users must re-download episodes and rerun diffusion-based separation, so the corpus will drift as links disappear, and rebuilding it needs substantial compute.
- Music and advertisement filtering relies on feed-level category tags and diarization. The paper does not report how much non-dialogue material remains.
Wiki Connections
- Spoken Language Model — A training corpus built for full-duplex spoken dialogue language models; the paper trains no model on it, so its value for speech LMs is not yet demonstrated.
- J-CHAT — The same authors’ earlier Japanese podcast and YouTube dialogue corpus; DuplexChat reuses its PodcastIndex crawl source and two-speaker segmentation rules, and adds speaker-separated channels and English coverage.
- Moshi — Cited as an example full-duplex dialogue model that needs separate-channel training data, and among the prior studies that use Fisher, which motivates Fisher as the comparison corpus.
- Full-Duplex-Bench — Its turn-taking analysis is adapted, with thresholds tuned to separated channels, to measure turn exchanges, backchannels and overlaps in DuplexChat.
- PersonaPlex — Cited among the prior full-duplex dialogue studies that use Fisher, part of the motivation for comparing against Fisher.
- Emilia — Compared in Table I as a large in-the-wild corpus with a released pipeline but no dialogue structure or separated channels.
- GigaSpeech — Compared in Table I as a web-scale but monaural, mixed-dialogue corpus.