ACL 2026 (Findings) · 2026 · Conference
Yangzhuo Li et al. (Xiamen University / Jilin University / Zhejiang University) · → Paper · Demo: ✗ · Code: ✗
Turns a serial-reasoning spoken dialogue model into an interleaved “Thinking-while-Talking” model whose think and talk block lengths are derived from a playback-covers-compute constraint, then uses GRPO with four LLM-judged block-level rewards to keep the fragmented reasoning logically coherent, cutting time-to-first-audio on VoiceBench from 20.35 s to 3.65 s while staying within 0.4 points of the serial-reasoning model’s overall score.
Problem
Adding explicit chain-of-thought (CoT) to end-to-end spoken dialogue models improves reasoning, but the “Thinking-then-Talking” pattern makes the user wait for the full reasoning trace before any audio is played: the serial-reasoning Step-Audio-2-mini-Think model has a VoiceBench time-to-first-audio (TTFA) of 20.35 s. Interleaved “Thinking-while-Talking” approaches such as SHANKS and STITCH alternate think and talk chunks, but the authors argue they cut chunks at fixed token counts, which (1) leaves the relative lengths of thinking and speaking unconstrained, so a long thought can outrun the preceding audio and starve the client audio buffer; (2) incurs a context-fragmentation cost from frequent mode switching; and (3) breaks reasoning steps mid-sentence, so chunks are not semantically complete units (“block atomicity”).
Method
Dual-Reasoner is initialized from Step-Audio-2-mini-Think (Step-Audio 2) and fine-tuned to emit an interleaved sequence of <think> blocks and talk blocks, where talk blocks interleave text and audio tokens at a fixed 1:4 ratio (padded with <tts-pad> when the text runs out). Model size is not reported.
Streaming Masking Mechanism. Block lengths are set by a continuity inequality: the playback duration of the previous talk block plus its decoding latency must cover the latency of generating the next think block, the next talk block, and its first audio frame (§3.2.1, Eq. 1). Plugging in measured rates (generation about 45 tokens/s after warm-up, 0.04 s of audio per audio token, audio tokens about 80% of a talk block) gives the rule N_think < 0.5 · N_talk (App. C, Eqs. 6-8, Table 8). Within that region, the authors sweep (N_think, N_talk) ∈ {(30, 65), (60, 125), (90, 185)} and adopt (60, 125), roughly three to five sentences of thinking per block (§3.2.1, §4.3). Note the training labels for SFT are still built by fixed-length segmentation (exactly 60 think tokens and 125 talk tokens per block) (App. G.3.1); the atomicity restoration happens in the RL stage.
Atomic-Consistency Restoration (ACR). A GRPO stage (group size 4, KL β = 0.01) optimizes a weighted sum of four rewards scored by GPT-4o and GPT-4o-audio (§3.2.2, §3.2.3, Table 9):
- Semantic Completeness (R_comp): punctuation heuristics plus an LLM semantic-closure check so each block is a closed unit.
- Internal-External Consistency (R_const): an LLM “Logical Alignment Auditor” penalizes talk content that is not grounded in the preceding think history.
- Sparse Logic Flow (R_flow): audits randomly sampled non-adjacent block pairs for long-range contradictions.
- Multimodal Outcome (R_out): rewards answer correctness and the naturalness and contextual appropriateness of the generated audio.
Weights are (w_comp, w_const, w_flow, w_out) = (0.2, 0.2, 0.2, 0.4); group-normalized advantages feed a PPO-clipped objective. In the GRPO stage only the LLM is updated; both SFT stages train the full model (Table 9).

Dual-Think-30k and curriculum. The training set has 31,567 samples (17,670 Slow Thinking, 13,897 Fast Thinking) drawn from OpenOrca, Competition Math (levels 4-5), APPS (interview/competition tiers), MedMCQA, and VoiceAssistant-400K from Mini-Omni (§3.3, App. B.1, Table 6). Qwen-Plus routes samples by difficulty; Qwen-Max verbalizes LaTeX and code into speakable text; a Qwen3-235B thinking model (Qwen3) writes CoT traces using an R-P-T-S (Role, Problem, Task, Symbolic) template that fixes the response’s affective tone before symbolic reasoning. Speech is synthesized with IndexTTS2 using SeedTTS-eval reference voices, and samples with Whisper-large-v3 WER above 5% are dropped (§3.3.2, §3.3.3). Note that the text in §3.3.3 calls VoiceAssistant-400K the Fast Thinking source while Table 6 lists all 10,063 of its samples as Slow; the PDF shows both as printed. Training runs SFT-1 (15k simple / 2k complex), SFT-2 (8k complex / 2k simple), then GRPO on 3k samples, with a 15x loss weight on the mode-switching control tokens in the SFT stages (§4.1, App. G.3).
Key Results
All latency numbers are for Dual-Reasoner and the two Step-Audio 2 variants only; no other baseline has TTFA or RTF reported. RTF and TTFA on MMAR were measured on a single A100 (Table 2). The paper does not state how RTF is computed.
VoiceBench (VoiceBench) (Table 1):
| Model | Overall | Alpaca | CoEval | SD-QA | MMSU | OB-QA | IFEval | AdvB |
|---|---|---|---|---|---|---|---|---|
| Step-Audio-2-Mini-Base | 67.24 | 4.02 | 3.42 | 58.81 | 57.63 | 71.42 | 45.35 | 98.26 |
| Step-Audio-2-mini-Think | 73.80 | 4.52 | 3.93 | 63.84 | 61.73 | 76.92 | 34.10 | 98.65 |
| Dual-Reasoner | 73.41 | 4.50 | 3.85 | 62.97 | 60.33 | 75.19 | 40.62 | 98.65 |
| Qwen2.5-Omni-7B | 74.12 | 4.49 | 3.93 | 55.71 | 61.32 | 81.10 | 52.87 | 99.42 |
| Kimi-Audio | 76.91 | 4.46 | 3.97 | 63.12 | 62.17 | 83.52 | 61.10 | 100 |
Dual-Reasoner gains 6.17 overall points over the Base model but sits 0.39 below the serial Think model, and is lower than the Think model on every sub-task except IFEval (higher) and AdvBench (tied). On IFEval it recovers part of the Think model’s drop (34.10 to 40.62) but stays below the Base model (45.35). Two baselines score higher overall: Kimi-Audio (76.91) and Qwen2.5-Omni (74.12). Lower-scoring baselines are MiniCPM-o (71.23), VITA-1.5 (64.53), GLM-4-Voice (56.48), Qwen2-Audio (55.80) and Step-Audio (50.84).
MMAR (Table 2): average accuracy 53.92% vs. 49.70% (Base) and 54.50% (Think). Against the Think model, Dual-Reasoner is lower on Sound (47.24 vs. 48.48), Speech (56.72 vs. 60.54), Sound-Music (18.18 vs. 27.27), Sound-Speech (52.38 vs. 55.50) and S-M-S (57.00 vs. 58.33), higher on Music (33.50 vs. 32.52), and tied on Music-Speech (51.22). Against the Base model it is lower on Music (33.50 vs. 36.89) and tied on Sound-Music and Music-Speech. Qwen2.5-Omni scores higher (56.74); Baichuan-Omni-1.5 (40.74), Audio-Reasoner (36.8), SALMONN (32.8), Audio-CoT (31.31), Qwen2-Audio (30.4) and Mellow (30.0) score lower. Note that Table 7 in the appendix reprints some of the Base and Dual-Reasoner MMAR cells with different values (e.g., Base S-M-S 66.67 vs. 52.67 in Table 2; Dual-Reasoner S-M-S 60.00 vs. 57.00), with identical averages; the page uses Table 2.
Latency (Tables 2, 3, 4):
| Benchmark | Metric | Base | Think | Dual-Reasoner |
|---|---|---|---|---|
| VoiceBench overall | TTFA (s) | 6.28 | 20.35 | 3.65 |
| VoiceBench overall | RTF | 2.39 | 7.04 | 1.05 |
| MMAR | TTFA (s) | 6.42 | 20.80 | 3.74 |
| MMAR | RTF | 2.39 | 7.04 | 1.06 |
| GSM8K | TTFA (s) | 6.51 | 22.15 | 3.82 |
| GSM8K | RTF | 2.30 | 7.96 | 1.08 |
Dual-Reasoner has lower TTFA and RTF than both baselines on every benchmark, including the non-reasoning Base model. Per VoiceBench sub-task, its TTFA ranges 3.41-3.89 s and its RTF ranges 0.82-1.35 (above 1 on Alpaca, SD-QA, OB-QA and IFEval) (Table 3). The §4.2 text describes the MMAR RTF gain as a “2.25x speedup over the Think variant”; 2.25x matches the ratio to the Base model (2.39 / 1.06), while the ratio to the Think model is about 6.6x (7.04 / 1.06).
GSM8K (Table 4): 78.69% accuracy vs. 79.21% (Think) and 48.35% (Base).
Ablations (Table 5, VoiceBench): relative to the full model (73.41 overall), removing the Flow reward gives 65.28 (-11.1%), the Consistency reward 66.95 (-8.80%), the Completeness reward 67.76 (-7.69%, IFEval 30.10), keeping only the Multimodal Outcome reward 68.05 (-7.30%), and removing ACR entirely 62.53 (-14.8%), which is below the Base model’s 67.24. Alternative weightings score 63.74-71.20. On MMAR, the shortest block setting (30, 65) degrades results and (90, 185) gives a small gain over (60, 125); raising the control-token loss weight to 30x drops the SFT model’s MMAR average to 42.39 (§4.3, Table 7). Table 7’s think60-talk125-rl row averages 51.88 while the Dual-Reasoner row averages 53.92; the paper does not explain how the two configurations differ.
No human listening test, speech-quality, or ASR-based intelligibility metric is reported for the generated audio.
Novelty Assessment
Interleaving reasoning with speech output is not new here; the paper explicitly positions itself against SHANKS and STITCH. The new pieces are (1) deriving the think/talk length ratio from a playback-latency inequality instead of choosing it ad hoc, and (2) the ACR reward set, which uses block-level, LLM-judged process rewards (completeness, think-to-talk grounding, long-range consistency) rather than only an outcome reward. Both are incremental recipe contributions on top of an existing speech LM backbone; the architecture itself is unchanged apart from the output format. The block length rule is a derivation from measured throughput of this specific model and hardware, not a general result. Dual-Think-30k contributes a mixed fast/slow speech-CoT training set, but the paper gives no release URL. Comparisons are fair with respect to the two Step-Audio 2 variants (same backbone); other baselines are reported without latency figures.
Field Significance
Moderate — The paper provides a concrete data point that interleaved reasoning can bring a CoT spoken dialogue model’s time-to-first-audio down from about 20 s to under 4 s while keeping most of the serial-reasoning model’s accuracy, and an ablation showing that block-level process rewards matter more than an outcome-only reward for interleaved streams. The gains are measured only against its own backbone variants, rely on proprietary LLM judges for training rewards, and leave the generated speech itself unevaluated.
Claims
- supports: Interleaving short reasoning blocks with speech output can reduce the first-audio latency of reasoning-augmented spoken dialogue models by several times while retaining most of the accuracy of full serial reasoning.
Evidence: Starting from the same backbone, VoiceBench TTFA falls from 20.35 s (serial Think model) to 3.65 s and RTF from 7.04 to 1.05, while the overall score moves from 73.80 to 73.41; GSM8K accuracy is 78.69% vs. 79.21%. (§4.2, Tables 1, 3, 4)
- complicates: Interleaved reasoning in spoken dialogue models still trades away some accuracy relative to serial reasoning, and can underperform a non-reasoning model on instruction following.
Evidence: Dual-Reasoner is below the serial Think model on five of seven VoiceBench sub-tasks and five of seven MMAR categories, and its IFEval score (40.62) remains below the non-reasoning Base model (45.35). (§4.2, Tables 1, 2)
- supports: When chain-of-thought is fragmented into interleaved blocks, process-level rewards that check block completeness, speech-to-reasoning grounding, and long-range consistency improve spoken dialogue performance more than an outcome-only reward.
Evidence: With all four rewards, VoiceBench overall is 73.41; with only the multimodal outcome reward it is 68.05, and without the long-range logic-flow reward 65.28. (§4.3, Table 5)
- complicates: Supervised fine-tuning on fixed-length interleaved think/talk segments can degrade a spoken dialogue model below its non-reasoning starting point unless followed by targeted reinforcement learning.
Evidence: The model without the ACR RL stage scores 62.53 overall on VoiceBench, below Step-Audio-2-Mini-Base at 67.24. (§4.3, Table 5, Table 1)
- refines: For streaming reasoning-then-speech generation, the admissible reasoning length per block is bounded by the playback duration of the preceding speech block, which ties block-length design to the model’s token throughput and audio frame rate.
Evidence: With about 45 tokens/s generation, 0.04 s per audio token and audio tokens making up about 80% of a talk block, the continuity condition reduces to N_think < 0.5 · N_talk. (§Appendix C, Eqs. 6-8, Table 8)
Limitations and Open Questions
The generated speech is never directly evaluated
Training includes an audio-naturalness reward, but no MOS, listening test, UTMOS, or ASR-based WER of the spoken output is reported, so whether interleaving affects speech quality, prosody, or think/talk boundary artifacts is unknown. All reported scores are VoiceBench, MMAR and GSM8K task metrics plus latency.
- Training rewards depend on proprietary GPT-4o and GPT-4o-audio judges, which the authors acknowledge raises cost and limits reproducibility (§Limitations).
- Dual-Think-30k is built from curated text corpora rendered with TTS, so robustness to noise, disfluencies and spontaneous speech is untested (§Limitations).
- The block length rule is derived from one model’s measured generation speed on specific hardware; it would need re-deriving for other backbones or deployment settings.
- RTF stays near or above 1 on several VoiceBench sub-tasks (up to 1.35), and its exact definition is not given, so how much playback headroom the system has in practice is unclear.
- Latency is compared only against the two Step-Audio 2 variants; no other interleaved or streaming reasoning system (e.g., SHANKS, STITCH) is evaluated head to head.
- Some reported numbers do not line up: MMAR cells differ between Table 2 and Table 7, the think60-talk125-rl ablation row does not match the final model’s MMAR average, and the source of VoiceAssistant-400K samples is described as Fast Thinking in text but Slow in Table 6.
- No code, model weights, or dataset release URL is given.
Wiki Connections
- Spoken Language Model — Fine-tunes a speech LM that consumes spoken queries so it reasons in interleaved blocks while answering.
- Speech-to-Speech — An end-to-end spoken dialogue system that generates speech responses directly, evaluated on spoken QA and reasoning benchmarks.
- Streaming TTS — Derives think/talk block lengths from a playback-covers-compute constraint and reports TTFA and RTF of the streamed speech output.
- RLHF Speech — Applies GRPO with four LLM-judged process and outcome rewards to a speech LM and ablates each reward.
- Step-Audio 2 — Provides the Step-Audio-2-mini-Think initialization and both latency and accuracy baselines (Base and Think variants).
- SHANKS — Prior interleaved thinking approach for spoken language models that the paper argues relies on fixed-length truncation.
- VoiceBench — Main benchmark for interaction quality and per-sub-task latency.
- Kimi-Audio — Baseline that scores higher than Dual-Reasoner on VoiceBench overall (76.91 vs. 73.41).
- Qwen2.5-Omni — Baseline that scores higher than Dual-Reasoner on both VoiceBench (74.12) and MMAR (56.74).
- Mini-Omni — Source of the VoiceAssistant-400K conversational data included in Dual-Think-30k.
- WavAlign — Same-group post-training work for spoken dialogue models cited for the training setup.
- Step-Audio-R1 — Serial “thinking-then-talking” RL reasoning model cited as the latency-bottlenecked alternative.