arXiv · 2025 · Preprint
Donghang Wu et al. (StepFun) · → Paper · Demo: ? · Code: ?
Introduces a dual-LLM “Formulation Brain / Articulation Brain” architecture that lets a spoken language model reason and speak concurrently, letting an ongoing chain-of-thought pace a separate response-generation model instead of forcing a single LLM to interleave or complete its thinking before speaking.
Problem
Chain-of-thought (CoT) reasoning improves spoken language model (SLM) response quality but is normally generated in full before the model starts speaking, which is too slow for real-time dialogue. Existing “think-while-speaking” alternatives instead force a single LLM to interleave think tokens and response tokens within one autoregressive stream. The paper argues that this interleaving repeatedly switches the model between thinking mode and response mode, disrupting semantic coherence and, per the authors, degrading reasoning quality relative to pre-computing the full CoT first.
Method
MPS splits reasoning and speech generation across two separate LLMs rather than one. The Formulation Brain receives the user’s speech (and optional text) input and continuously generates chain-of-thought tokens, split into segments. The Articulation Brain receives the same user input plus each newly produced think segment, concatenated with the historical think and response segments, and generates the next response segment conditioned on this growing prefix. The response segments are passed to a streaming TTS model for real-time speech synthesis, so articulation begins before the Formulation Brain has finished its full reasoning chain. Both brains share the same underlying LLM parameters (initialized from Step-Audio 2) rather than being separately trained models.
To let the Articulation Brain generate coherent responses from partial reasoning, the paper introduces a think-incomplete supervised fine-tuning (SFT) procedure: training examples truncate the ground-truth CoT to a randomly chosen prefix length, wrap it in <think>/</think> tags, and pair it with the full ground-truth response as a standard next-token-prediction target. Because this only changes the training data, not the model’s input-output interface, it requires no architectural retraining of the Step-Audio 2 backbone. At inference, CoT and response segments use a fixed token budget (80 think tokens, 100 response tokens, following the ta4 format of 20 text tokens plus 80 speech tokens per response segment). Two inference-time configurations are evaluated: Think-First (MPS-thkfirst), which waits for the first think segment before speaking, and Speak-First (MPS-spkfirst), which begins articulating a response segment immediately in parallel with the first think segment, yielding near-zero added latency.
Key Results
On the Spoken-MQA mathematical reasoning benchmark, MPS-thkfirst reaches 93.9% average accuracy, matching or slightly exceeding the full-CoT Think-Before-Speaking baseline (MPS-tbs, 93.0%) while requiring only 80 extra tokens of latency before the first response token, versus 762 for MPS-tbs (Table 2). MPS-spkfirst reaches 92.8% accuracy with zero added latency tokens. Both MPS variants clearly outperform the only other think-while-speaking baseline evaluated, Mini-Omni-Reasoner, which reaches 68.6% accuracy on the same benchmark despite generating only 8 extra tokens (Table 2). On URO-Bench speech-conversation tasks, MPS-thkfirst outperforms MPS-tbs on nearly all subtasks and on average (89.1 vs. 87.8 Basic.Avg, 80.5 vs. 79.0 Pro.Avg, English split, Table 3), and MPS-spkfirst remains close to or above MPS-tbs on several subtasks despite its zero-latency configuration.
Baseline numbers for Mini-Omni-Reasoner and other SLMs on Spoken-MQA are taken from a separate cited paper rather than reproduced by the authors, and URO-Bench baselines are taken from the Step-Audio 2 technical report; only the Step-Audio 2 numbers were independently reproduced by this paper’s authors (§4.1, Table 1, Table 3 captions). All MPS variants and the MPS-wo/thk and MPS-tbs comparisons share the same Step-Audio 2 backbone and training data, which makes the internal comparisons (MPS vs. MPS-tbs vs. MPS-wo/thk) controlled, but the cross-paper comparison to Mini-Omni-Reasoner is not.
Novelty Assessment
The contribution is architectural: splitting reasoning and articulation into two coordinating LLM instances, with the reasoning stream pacing (rather than being interleaved with) the response stream, is a genuinely different mechanism from the single-model interleaving used by prior think-while-speaking methods. The think-incomplete SFT recipe is comparatively simple (truncated CoT prefixes as training targets) but is the specific technique that makes the architecture trainable without retraining the base LLM’s input-output format. The core empirical claim, that MPS matches full-CoT quality while cutting latency by roughly an order of magnitude, is well supported by the paper’s own controlled ablations (MPS-tbs vs. MPS-thkfirst vs. MPS-wo/thk, same backbone and data). The comparison against Mini-Omni-Reasoner, the main competing think-while-speaking method, is less rigorous: its numbers are copied from another paper rather than reproduced under matched conditions, so the reported accuracy gap should be read as suggestive rather than a fully controlled comparison.
Field Significance
Tip
High — this paper demonstrates that decoupling reasoning generation from response generation into two coordinating LLMs can preserve full chain-of-thought reasoning quality while reducing response latency to near zero, addressing a concrete and recently identified bottleneck (think-while-speaking) for real-time spoken dialogue systems.
The dual-brain decomposition offers an architectural alternative to single-model token interleaving for combining CoT reasoning with low-latency speech generation, and the paper shows this alternative can be trained via a lightweight SFT recipe on top of an existing SLM backbone rather than requiring a new pretraining regime. Its practical relevance is reinforced by the fact that it underlies a released production system (Step-Audio R1.1), though the paper itself does not report deployment-scale evaluation beyond the two benchmarks discussed above.
Claims
- supports: Decoupling reasoning generation from response generation into two separate but coordinating language models can match the response quality of full chain-of-thought pre-computation while eliminating most of its added latency.
Evidence: MPS-thkfirst reaches 93.9% average accuracy on Spoken-MQA versus 93.0% for the full pre-computed CoT baseline (MPS-tbs), while needing only 80 extra tokens before the first response token compared to 762 for MPS-tbs. (§4.3.1, Table 1, Table 2)
- supports: Pacing a response-generation model with segments of an external, ongoing reasoning stream produces more coherent and accurate responses than interleaving reasoning and response tokens within a single autoregressive model.
Evidence: Both MPS-thkfirst and MPS-spkfirst outperform Mini-Omni-Reasoner, a single-model interleaved think-while-speaking method, on Spoken-MQA (93.9% and 92.8% vs. 68.6% average accuracy). (§4.3.1, Table 2)
- complicates: Beginning a spoken response before any reasoning context is available measurably harms accuracy on tasks that depend on precise multi-step computation, even when reasoning catches up shortly afterward.
Evidence: MPS-spkfirst, which speaks its first response segment before any think segment exists, drops to 77.3% on the Arithmetic.Avg subset of Spoken-MQA versus 86.4% for MPS-thkfirst, while the gap nearly disappears on the Reasoning subtasks (94.9% vs. 94.9%). (§4.3.1, Table 1)
- refines: A model can be adapted to generate responses from incomplete reasoning traces through supervised fine-tuning on truncated chain-of-thought prefixes alone, without changing the model’s input-output token format or architecture.
Evidence: Think-incomplete SFT trains the Articulation Brain on randomly truncated CoT prefixes paired with full ground-truth responses, reusing the base LLM’s standard next-token-prediction objective; the resulting model still exceeds the full-CoT baseline on URO-Bench (89.1 vs. 87.8 Basic.Avg, English). (§3.3, §4.3.2, Table 3)
Limitations and Open Questions
The paper reports that the segmentation strategy is sensitive to design choices in ways that are not fully explained: applying the same variable-length, per-CoT-step segmentation used during think-incomplete SFT at inference time introduces “uncontrollable latency” rather than improving results, and using a fixed-token-count segmentation during the SFT phase (mirroring the inference-time strategy) also fails to improve performance (§3.3). Only the fixed-length inference-time segmentation (80 think tokens / 100 response tokens) is reported as effective, and the paper does not explore why the alternatives underperform. The comparison against the main competing think-while-speaking baseline, Mini-Omni-Reasoner, relies on numbers reported in another paper rather than a reproduction under identical conditions, and results are evaluated on a Step-Audio 2 backbone whose parameter count is not disclosed in this paper, limiting independent assessment of how the approach scales to other backbones or model sizes.
Wiki Connections
- Spoken Language Model — proposes a dual-LLM architecture in which both the reasoning and response-generating models consume the user’s external speech input directly, rather than only the model’s own generated audio.
- Streaming TTS — pipes each generated response segment to a streaming TTS model so that speech synthesis begins before the full response, or the full reasoning chain, is complete.
- Speech-to-Speech — evaluates the full architecture end-to-end as a speech-in, speech-out spoken dialogue system on URO-Bench’s speech conversation tasks.
- Mini-Omni-Reasoner — used as the primary think-while-speaking baseline; MPS reports higher Spoken-MQA accuracy than this single-model interleaved-token approach.
- Step-Audio 2 — supplies the LLM backbone, parameter initialization, and ta4 audio-text output format that both the Formulation Brain and Articulation Brain build on; also reproduced as the direct-response and full-CoT baseline.