arXiv · 2024 · Preprint
Shao et al. (DeepSeek-AI) · → Paper · Demo: ? · Code: ✓
DeepSeekMath 7B is a domain-specialist language model for mathematical reasoning that introduces Group Relative Policy Optimization (GRPO), a memory-efficient variant of PPO that eliminates the need for a separate critic model by estimating advantages from group-relative reward normalization.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
DeepSeekMath’s primary contribution to the broader LLM community is GRPO, a reinforcement learning algorithm that reduces the computational overhead of PPO-style training by replacing the learned value function with group-relative reward statistics computed across a batch of sampled outputs from the same prompt. This makes RLHF-style fine-tuning feasible at smaller resource budgets without sacrificing policy improvement quality. The DeepSeek-AI model family, of which DeepSeekMath is an early member, provides open-weight LLM backbones that spoken conversational agent and instruction-conditioned TTS systems can use as language model components. GRPO itself has been adopted as a reward-optimization technique in speech generation training pipelines that involve reward signals from human or automatic preference judgements, extending the algorithm’s applicability beyond its original mathematical reasoning domain.
Wiki Connections
- 2312.11805 (Gemini) — cited as a frontier closed-source LLM baseline for mathematical reasoning benchmarks
- 2309.16609 (Qwen) — cited as a comparable open-source LLM in the reasoning benchmark comparison
- 1711.05101 (AdamW) — optimizer used for pretraining and fine-tuning
- 2303.08774 (GPT-4) — cited as the performance ceiling for mathematical reasoning tasks
- 2307.09288 (LLaMA 2) — cited as an open-weight baseline in the benchmark comparison