SIGDIAL 2026 · 2026 · Workshop
Akiko Masaki-Kato et al. (NTT, Inc.) · → Paper · Demo: ? · Code: ?
Runs the conversational agent and the reasoning agent of a cascade spoken dialogue system concurrently, coordinated through a shared, versioned belief state, so that planning starts on partial intent instead of after slot filling and the multi-second planning silence of sequential dual-agent designs largely disappears.
Problem
Task-oriented spoken assistants must answer within roughly one second, yet LLM planning with retrieval and tool use takes several seconds per turn. Dual-agent designs split the work into a fast conversational agent and a slow reasoning agent (the Talker-Reasoner architecture and OpenAI’s Chat-Supervisor pattern). In practice they run sequentially: the conversational agent collects the request, calls the reasoner, and waits. The user hears a long silence exactly when planning happens. End-to-end full-duplex models such as Moshi reach sub-second latency by merging speech and language modelling into one network, but that gives up the separate, tool-capable general-purpose LLM on the reasoning side.
Method
Ping-Ponder is a cascade spoken dialogue system with three components. Ping is the fast conversational agent. It runs on the OpenAI Realtime API, which supplies streaming ASR, dialogue management, turn-taking and TTS. Ponder is the deliberative reasoning agent. It runs on a GPT-4.1-family model (gpt-4.1 in the interactive system, gpt-4.1-nano in the replay study) with external knowledge services wired in as tool calls. The shared belief state B links the two agents. No component is trained; the contribution is the coordination protocol.
B holds three field groups: intent slots as (key, value, confidence) triples, a planning status (idle, planning, partial, ready), and a plan draft with a queue of stable fragments that Ping may verbalise. B is an in-process key-value store with per-field version counters and change notifications, so it acts as a publish/subscribe channel rather than a per-turn snapshot as in classical dialogue state tracking. Write authority is partitioned: Ping alone writes the intent slots, and Ponder alone writes the planning status and plan draft. Single-writer groups make last-writer-wins safe without a consensus protocol.
Ping consumes partial ASR hypotheses, updates the slots, and at each decision point emits an acknowledgement (a filler while Ponder works), a clarification question (for low-confidence slots), or a final response once the status is ready. Ping never waits for Ponder. Ponder starts planning as soon as a domain-specified minimal slot subset is posted. It retrieves, drafts a plan, and writes intermediate results (partial) back early. If Ping overwrites a slot that a draft depends on, Ponder detects the version mismatch and restarts. Speculative prefetch dispatches the Ponder call against the anticipated next turn in the background, so its cost is absorbed off-turn. Ponder outputs a structured {slots, proposals} JSON update (500 tokens, temperature 0.3), which is injected into Ping’s system prompt for the next turn (§3, Table A5, Table A6).
Key Results
The evaluation has two parts. A paired single-operator case study compares Ping-Ponder with the blocking Chat-Supervisor baseline in live voice sessions. A replay study then feeds the logged user turns of 120 Japanese in-house dialogues and 106 single-domain SpokenWOZ English dialogues into four conditions: Baseline (Chat-Supervisor), No-prefetch, Ping-Ponder and Single-agent. All four share STT, TTS and the LLM family.
- Interactive case study (Table 1): planning-phase latency (heavy-model call start to output availability) falls from 7.11 s to 0.49 s in English and from 7.31 s to 0.96 s in Japanese. Intent-resolution latency to the first user-visible token rises slightly (0.42 to 0.45 s EN, 0.70 to 0.75 s JA). This rests on one operator and one session per language.
- TTFR in replay (Table 2): time to first response, measured from end of user turn to first agent chunk, is at parity with Baseline (1526 vs. 1576 ms JA; 891 vs. 886 ms EN). No-prefetch, which dispatches the same Ponder call after the user turn, costs +1227 ms (JA) and +661 ms (EN) relative to Ping-Ponder. This contrast is the clean single-factor ablation of speculative prefetch.
- Planning latency on fired turns (Table 3): 950 vs. 1376 ms in JA (-31%) and 703 vs. 1029 ms in EN (-32%). The EN Ping-Ponder figure rests on only 25 fired turns, since the planner fires in 13.2% of SpokenWOZ dialogues versus 75.8% of the Japanese ones. The paper states that this comparison confounds scheduling with output format and token budget (Baseline uses a 220-token natural-language supervisor).
- Turn-level latency (Table 4): on turns where planning fired, Ping-Ponder is 1135 ms faster than Baseline in JA (-35%) and 1166 ms faster in EN (-50%). On non-fired turns the gap is 105 ms (-5%) and 42 ms (-3%), so the gain concentrates on planning turns.
- Response quality (Tables A1, A2): a reference-based
gpt-4.1judge on 0-3 scales shows Ping-Ponder above Baseline in JA (appropriateness 2.73 vs. 2.52, faithfulness 2.12 vs. 1.97). In EN the three structured conditions cluster (Ping-Ponder 2.40 / 1.75 vs. Baseline 2.21 / 1.60). Single-agent has a slightly higher fluency score than Ping-Ponder in both languages (2.99 vs. 2.96 JA, 3.00 vs. 2.96 EN). No confidence intervals are reported for the quality scores, and the authors frame this as no evidence of degradation, not as a quality gain. - Judge validation (Tables A3, A4): one author re-scored 120 turns blind to condition. Human-judge agreement on faithfulness is moderate (Spearman ρ 0.75 EN; 0.34 JA on only 16 turns) and low on Japanese appropriateness (ρ 0.24), where the judge under-penalises mid-dialogue re-greetings. The
gpt-4.1and Claude judges agree with each other more than with the human. These ratings cover response text, not the synthesized speech.
In EN, the Single-agent control, which has no supervisor path, has a higher TTFR (999 ms) than both Baseline and Ping-Ponder (Table 2).
Novelty Assessment
The contribution is engineering integration. The authors state that separating dialogue management from planning, shared-state (blackboard) coordination, and incremental processing are not new (§2). What the paper adds is the concrete coordination discipline for two concurrent LLM agents: partitioned single-writer fields, per-field version counters with restart on stale drafts, and speculative prefetch of the reasoning call. All speech and language components are closed commercial APIs. The No-prefetch ablation cleanly isolates prefetch as the source of TTFR parity. The headline 93% / 87% figures come from a single-operator case study, and the replay comparison against Baseline confounds coordination with output format and token budget. The evaluation does not measure task success, groundedness, or human-perceived conversational quality of the spoken interaction.
Field Significance
Low — The paper gives controlled evidence that, in cascade spoken dialogue systems, reasoning latency is largely a scheduling problem: running a reasoning agent concurrently on partial intent, with speculative prefetch, removes most of the planning-turn silence without slowing the first response. The evidence is limited to two task-oriented corpora, closed API components, a single-operator live study, and LLM-judged text quality, so it serves as a design data point rather than a validated user-facing result.
Claims
- supports: In cascade spoken dialogue systems, starting deliberative reasoning on partial user intent rather than after the request is complete shortens the silence on turns that need planning.
Evidence: Against a blocking Chat-Supervisor baseline, concurrent planning through a shared belief state cuts turn-level latency on planning turns by 1135 ms (JA, -35%) and 1166 ms (EN, -50%), while non-planning turns change by only 3-5%. (§5.2.3, Table 4)
- supports: Speculatively prefetching a slow reasoning call removes most of the first-response latency cost that the call otherwise adds to a spoken dialogue system.
Evidence: With the Ponder call dispatched after the user turn (No-prefetch), TTFR rises to 2803 ms (JA) and 1547 ms (EN); with prefetch it is 1526 ms and 891 ms, at parity with the blocking baseline (1576 ms and 886 ms) and the chat-only Single-agent control (1542 ms and 999 ms). (§5.2.1, Table 2)
- complicates: Reference-based LLM judges can miss conversational-flow failures in spoken dialogue responses, so they need human validation before supporting quality claims.
Evidence: On a blind 120-turn re-annotation, human agreement with the
gpt-4.1judge on Japanese appropriateness is only ρ = 0.24, the judge does not penalise mid-dialogue re-greetings, and two LLM judges from different vendors agree with each other (faithfulness ρ = 0.82) more than with the human. (§Appendix A.1, Table A3) - complicates: Latency gains from restructuring a dialogue system’s coordination depend heavily on how often the task needs planning, so average-turn latency can hide or dilute them.
Evidence: The planner fires in 75.8% of Japanese dialogues but only 13.2% of SpokenWOZ dialogues, and the all-turn TTFR averages show parity even though planning turns are much faster. (§5.2.2, §5.2.3, Table 3, Table 4)
Limitations and Open Questions
Warning
No human listener evaluates the spoken interaction. The 93% / 87% headline comes from a single operator’s paired session, the replay study feeds logged user turns without multi-turn feedback, and response quality is judged on text by an LLM, validated only by one author on 120 turns.
- All components are closed commercial APIs (OpenAI Realtime API, GPT-4.1 family), and the beta Realtime interface used was later retired. The authors argue that open components (Pipecat, LiveKit Agents, Moshi) could replace them but have not tested this.
- Absolute latencies depend on the API’s TTS first-chunk latency for the voices used, so the numbers are characteristic rather than universal.
- Evaluation covers only task-oriented domains where planning follows slot elicitation. Open-domain chat and tasks with little reasoning are untested.
- The belief state can propagate systematic ASR errors that are re-asserted with growing confidence. The authors suggest ASR confidence scores and a disagreement policy as future work.
- The Japanese corpus is not yet released. The authors release replay outputs, judge prompts and per-turn scores, not the system code.
- Task success and groundedness are not measured, so the parity of Single-agent on judge scores cannot show what the planner adds.
Wiki Connections
- Speech-to-Speech — Ping-Ponder is a cascade spoken dialogue system (streaming ASR, LLM agents, TTS) that restructures agent scheduling to cut perceived latency on planning turns.
- Moshi — the paper contrasts its modular dual-agent design with Moshi’s full-duplex end-to-end approach to sub-second latency and names Moshi as a possible open replacement for the fast agent.