Abstract
Streaming TTS produces speech incrementally to reduce response delay and support interactive systems. Causal components, compact tokens, parallel decoding, and duplex training make low latency practical, while chunking, lookahead, reasoning, turn-taking, and inconsistent measurement remain central constraints.
Current State
Incremental acoustic or speech-token generation can substantially reduce first-packet and response latency. The evidence is strong. Examples include BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data, LLaMA-Omni: Seamless Speech Interaction with Large Language Models. Reported latency depends on hardware, buffering policy, network assumptions, and whether upstream processing is included.
Chunk size and read–write policy create a direct trade-off among latency, alignment, and acoustic quality. This is emerging within the reviewed graph; representative evidence includes LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis, LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM. Optimal chunks are architecture-, language-, and metric-dependent.
Causal constraints and limited lookahead reduce delay but can degrade quality unless the model is adapted explicitly. The evidence is strong. Examples include TS3-Codec: Transformer-Based Simple Streaming Single Codec, RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding. Pitch, phase, and boundary artifacts may respond differently to the same lookahead budget.
Teacher distillation can recover quality lost when codecs, acoustic decoders, or vocoders are made causal. The evidence is strong. Examples include TS3-Codec: Transformer-Based Simple Streaming Single Codec, A Distilled Low-Latency Neural Vocoder with Explicit Amplitude and Phase Prediction. The benefit may depend on access to a strong non-causal teacher and matched training data.
Interleaving or jointly generating text and speech tokens enables streaming output while retaining semantic guidance. The evidence is strong. Examples include Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming, LLaMA-Omni: Seamless Speech Interaction with Large Language Models. Interleaved decoding can inherit exposure errors from both text and acoustic token streams.
Parallel, multi-token, and speculative decoding reduce serial speech-token generation cost. The evidence is strong. Examples include Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming, Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction. Speedups can shrink when verification, codec decoding, or small-batch deployment is included.
Q4 shifts attention from component latency to concurrent work and moving bottlenecks. Systems can reason during listening, as in Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems, but premature reasoning can harm coherence. X-Talk and related modular studies show that once one component is accelerated, ASR, retrieval, acoustic decoding, vocoding, or orchestration can become dominant.
Method Landscape
- Autoregressive streaming generation. 32 memberships demonstrate a shared practical pattern, not matched superiority.
- Hybrid and modular streaming systems. 22 memberships demonstrate a shared practical pattern, not matched superiority.
- Flow-matching streaming decoders. 10 memberships demonstrate a shared practical pattern, not matched superiority.
- GAN-based causal streaming vocoders. 8 memberships demonstrate a shared practical pattern, not matched superiority.
- VAE-based compressed streaming latents. 2 memberships demonstrate a shared practical pattern, not matched superiority.
Key Trade-offs
- Low-frame-rate and compact speech tokenizers shorten sequences and improve streaming efficiency. Aggressive token compression can discard speaker, prosodic, or transient detail.
- Shared models can support both streaming and offline speech generation through masks, chunk schedules, or mode conditioning. Joint training may underperform separately optimized models at either extreme.
- Full-duplex speech systems require generation to incorporate an incoming audio stream while speaking. Many systems demonstrate interruption handling rather than unrestricted simultaneous dialogue.
- Natural streaming dialogue requires explicit modeling of turn boundaries, interruptions, barge-in, and response timing. Turn-taking benchmarks often use constrained speakers, prompts, or synthetic overlap.
- Component speed versus end-to-end responsiveness. A faster tokenizer, LM, or vocoder matters only when the full pipeline and interaction policy are measured together.
Open Questions
- Which latency definition and measurement boundary should be standard for streaming speech generation?
- How small can chunks and lookahead become before intelligibility, prosody, or speaker identity fails?
- Can native speech systems match text-mediated reasoning while preserving paralinguistic information?
- What training data and objectives support genuine simultaneous listening, speaking, interruption, and recovery?
- How should token rate, codebook structure, and parallel decoding be co-designed for interactive generation?
Go Deeper
Read Streaming Text-to-Speech: In Depth for detailed evidence, method relationships, disagreements, implications, and a representative reading path.
Scope
This synthesis covers 75 papers through Q4 2025. All 19 clusters are strongly supported within the encoded graph. Evidence spans multiple architectures, datasets, tasks, and evaluation protocols; shared lineages and unmatched speed or quality comparisons retain narrower interpretations.
This page is generated from wiki/_claims/streaming-tts.yaml (digest date: 2026-09-26). For complete structured claims and paper-level provenance, use the source YAML.