arXiv · 2026 · Preprint
Xuan Du et al. (Huawei Technologies) · → Paper · Demo: ✗ · Code: ✗
InterRS trains a speech LM to alternate short hidden reasoning segments with short spoken answer segments, with the reasoning length held to a target ratio by a GRPO reward. On spoken math and logic benchmarks it gets close to the accuracy of a think-first model while starting to answer immediately.
Problem
Chain-of-thought reasoning makes speech LMs more accurate on math and logic, but a model that thinks before it speaks has an unbounded first-token latency, which breaks real-time conversation. Fast-CoT or direct-answer models respond at once but lose accuracy. Earlier interleaved “thinking-while-speaking” methods, such as STITCH and Mini-Omni-Reasoner, cut latency, but the authors identify two weaknesses. The alignment between thinking and speaking content is rigid. And training has no explicit control over the thinking-to-answer length ratio, so a long reasoning block can produce an audible pause or an overlong reply.
Method
A response is a sequence of pairs (T1, A1), …, (Tn, An). Each thinking segment Ti is hidden background inference. Each answer segment Ai is a short spoken summary of the preceding reasoning, and it is streamed to a TTS engine for playback. The design rests on the assumption that generating the next thinking segment takes less time than playing back the current answer segment, so the reasoning is hidden behind speech that is already playing. The model switches state with two single tokens, <|thinking|> and <|answer|>, instead of paired <think>...</think> delimiters, which shortens sequences.
Data pipeline (§3.2). Questions from K&K logic puzzles and MetaMath are rewritten without symbols in a conversational style, then synthesised to audio with CosyVoice 2. On the answer side, the pipeline checks each reasoning trace for correctness, repairs flawed traces, and condenses the result into an oral-style summary. The summary is split into “one-breath” semantic units. A matching thinking segment is then built for each unit under a fixed thinking:answer length ratio of 4:1, and a linguistic refinement step smooths transitions between segments. The result is about 12,000 training samples.
Training (§3.3-3.4). The backbone is Qwen2.5-Omni-3B. Only the thinker module is fine-tuned. Interleaved SFT runs for 3 epochs at a learning rate of 4e-6 and teaches the format. GRPO follows for 2 epochs (group size 16, learning rate 1e-6), with no KL penalty. The reward is a weighted sum of three terms:
- TA-Balance reward. Any format violation scores zero. Otherwise each thinking segment gets a quadratic penalty on the distance between its word count and a target length, and the segment scores are averaged.
- Accuracy reward. A binary match between the content after the last
<|answer|>token and the ground truth. - Linguistic Quality reward. The answer segments are concatenated, and the reward is the group-normalised log-likelihood of that text under a frozen reference model, kept only where it is above the group mean. It applies only to samples with a correct final answer, so fluency cannot be traded for correctness. The idea is adapted from GRPO-CARE.
The talker or TTS stage is used as is and never trained or evaluated. Every training signal acts on the text produced by the thinker.
Key Results
Table 1 compares the systems on four benchmarks. Spoken-MQA has spoken input, and KK-audio and Meta-audio are the in-domain test sets. The paper does not say whether SATA-Bench questions were converted to audio. InterRS averages 49.33. Thinking SFT+RL (think first, no instant response) averages 51.10 and leads on SMQA (76.74 vs 74.43) and Meta-audio (61.50 vs 59.20). InterRS therefore keeps about 96.5% of the think-first model’s average accuracy. Against the instant-response Fast CoT SFT+RL baseline (36.92 average), InterRS gains 12.41 points, which is the headline “+13%”. Fast CoT SFT+RL still has the best SATA-Bench score (25.20, against 21.10 for InterRS and 24.00 for Thinking SFT+RL). Over Fast CoT SFT+RL, the largest InterRS gain is on SMQA (74.43 vs 53.19), followed by Meta-audio (59.20 vs 40.70) and KK-audio (42.57 vs 28.57); on KK-audio InterRS is also the best system overall, narrowly ahead of the think-first model (42.14). Adding RL on top of interleaved SFT raises the average from 47.19 to 49.33. The only external interleaved baseline, Mini-Omni-Reasoner, is reported on SMQA alone (69.01 vs 74.43), and the paper does not say whether it was rerun under the same conditions.
The ablations train and test on math data only. With the TA-Balance reward, total thinking length falls to 104.88, against 128.64 for unconstrained interleaving and 123.61 for a segment-count reward. Accuracy stays about the same (68.65 vs 68.63 and 69.02, with think-first at 69.40). The per-segment interquartile range shrinks to 11 tokens, against 12 (unconstrained interleaving), 26 (segment reward), and 42 (think-first) (Table 2, Figure 4). The data refinement step raises judged fluency from 1.65 to 1.74. Adding the Linguistic Quality reward raises it to 1.83 and accuracy to 69.29 (Table 3).
Every metric is computed on text. Instant response is a yes/no label, not a measured latency. Fluency is a DeepSeek-V3 judgement of the concatenated answer text on a 0-2 scale. No synthesised speech is evaluated: there are no MOS or human listening tests, no WER on the output speech, and no measurement of first-packet latency or of the pauses the method is meant to prevent.
Novelty Assessment
Interleaving reasoning with spoken output is not new. STITCH, Mini-Omni-Reasoner, and Mind-Paced Speaking already explore it. The new parts here are the training controls. One is a ratio-controlled data construction pipeline. The others are two GRPO reward terms: a quadratic segment-length penalty, and a likelihood-based fluency reward applied only to correct samples. The length reward is a simple soft constraint, and the fluency reward adapts an existing consistency reward from multimodal reasoning RL. The contribution is therefore an incremental training recipe on an off-the-shelf omni model, not a structural change to the speech LM. The comparison with Mini-Omni-Reasoner covers one benchmark and does not state the conditions. The other baselines are all variants of the same backbone, which keeps those comparisons controlled.
Field Significance
Moderate. This paper is a controlled data point showing that, on a single 3B omni model, interleaved reasoning trained with explicit length control can recover most of the accuracy of a think-first model while answering immediately. The evidence covers reasoning accuracy and LLM-judged text fluency only. The claim that latency is hidden in speech generation is a design assumption and is never measured on generated audio.
Claims
- supports: Interleaving short hidden reasoning segments with spoken answer segments lets speech LMs recover most of the reasoning accuracy of think-before-speaking models while keeping immediate response onset.
Evidence: InterRS averages 49.33 across SATA-Bench, KK-audio, SMQA, and Meta-audio, against 51.10 for a think-first SFT+RL model with no instant response and 36.92 for an instant-response Fast CoT SFT+RL model on the same Qwen2.5-Omni-3B backbone. (§4.3.2, Table 1)
- refines: Explicit length-shaping rewards during RL control the size of individual reasoning segments in interleaved speech LMs with little loss of accuracy, which unconstrained interleaving does not do.
Evidence: A quadratic per-segment length penalty cuts total thinking length to 104.88, against 128.64 unconstrained and 123.61 with a segment-count reward, and narrows the per-segment IQR to 11 tokens, with accuracy at 68.65 against 68.63 and 69.02. (§5.1, Table 2, Figure 4)
- supports: Fragmenting a spoken answer into interleaved segments harms its coherence unless training explicitly targets the fluency of the concatenated response.
Evidence: Without refined data or the reference-model likelihood reward, the judged fluency of the concatenated answer text is 1.65 out of 2. Data refinement raises it to 1.74, and the Linguistic Quality reward raises it to 1.83, while accuracy rises from 68.65 to 69.29. (§5.2, Table 3, Figure 3)
- complicates: Interleaved reasoning gains are uneven across task types, and fast-CoT training can still match or beat interleaving on some multi-answer reasoning benchmarks.
Evidence: On SATA-Bench, Fast CoT SFT+RL scores 25.20 and think-first SFT+RL scores 24.00, both above InterRS at 21.10, even though InterRS is far ahead on KK-audio (42.57 vs 28.57). (§4.3, Table 1)
- complicates: Latency and fluency benefits claimed for thinking-while-speaking speech LMs are often assessed only on text, leaving the quality and timing of the generated speech untested.
Evidence: Instant response is a yes/no label, fluency is a DeepSeek-V3 score on the concatenated answer text, and no synthesised audio, measured latency, or human rating is reported. (§4.2.2, §4.2.3)
Limitations and Open Questions
The main limitation is that speech output is never evaluated. The answer segments are passed to Qwen2.5-Omni’s speech generation, which is used unchanged. The core premise is that thinking segment Ti+1 finishes before answer segment Ai has played back, but the paper never measures it. It reports no generation speed, no time to first audio, and no gaps in the played speech. The 4:1 thinking:answer ratio is said to suit “standard on-device inference speeds”, but no device or throughput figure is given.
- Fluency is judged only by an LLM (DeepSeek-V3) on text. No human raters are used, and the judge is not validated against human judgement.
- Training data comes from two domains (K&K logic and MetaMath math), and the ablations use math only. Whether the method generalises to open-domain spoken dialogue is untested.
- The only external interleaved baseline (Mini-Omni-Reasoner) is reported on a single benchmark, with no detail on how its number was obtained.
- The authors note that evaluation uses static turn-taking. How interleaved reasoning should be cut short or adapted when the user interrupts is left to future work (§Limitations).
- Code and demos are not released.
Wiki Connections
- Spoken Language Model — Adapts the thinker of an omni speech LM to take spoken math and logic questions and produce interleaved reasoning and answer streams for real-time spoken dialogue.
- RLHF Speech — Uses GRPO with format and length, accuracy, and reference-likelihood fluency rewards to shape reasoning in a speech LM. The rewards act on text output, not on speech.
- Qwen2.5-Omni — Provides the 3B backbone. Only the thinker module is fine-tuned, and the speech generation stage is used unchanged.
- Mini-Omni-Reasoner — The external interleaved thinking-in-speaking baseline, which InterRS beats on SMQA (74.43 vs 69.01) in the one benchmark where both are reported.
- Mind-Paced Speaking — A related dual-brain approach to real-time reasoning in speech LMs, discussed as prior work but not compared experimentally.
- CosyVoice 2 — Used as a tool to turn rewritten text questions into spoken inputs for training and in-domain evaluation.
- Step-Audio-R1 — Cited for reasoning collapse under RL in audio reasoning models, the failure that the SFT warm-up and length reward aim to prevent.
- Audio-CoT — Cited as earlier chain-of-thought reasoning in audio language models, the think-first approach that this work makes interleaved.