arXiv · 2024 · Preprint
Yao et al. (OpenBMB) · → Paper · Demo: ? · Code: ✓
MiniCPM-V is a series of efficient multimodal large language models (2B–8B parameters) designed to run on end-side devices such as mobile phones, achieving competitive vision-language performance through adaptive high-resolution visual encoding, token compression, and RLAIF-based alignment.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Context in Speech Generation below for why it is relevant.
Context in Speech Generation
MiniCPM-V is a vision-language model series optimised for deployment on resource-constrained end-side devices, including mobile phones and personal computers. Its architecture combines a SigLIP visual encoder with a perceiver resampler compression layer and a compact LLM backbone, enabling multimodal understanding within tight parameter and memory budgets. The series supports high-resolution image perception (up to 1.8 million pixels), strong OCR capabilities, and 30+ languages. Spoken conversational agent and multimodal speech systems can reference MiniCPM-V as a lightweight vision backbone for on-device multimodal understanding tasks that require integrating visual context alongside speech.
Wiki Connections
- 2303.08774 (GPT-4 Technical Report) — MiniCPM-V benchmarks its performance against GPT-4V-1106 as the reference frontier model on OpenCompass evaluations.
- 2302.13971 (LLaMA: Open and Efficient Foundation Language Models) — MiniCPM-V builds on the LLaMA model family as one of its LLM backbone options.
- 2307.09288 (Llama 2) — MiniCPM-Llama3-V 2.5, the strongest model in the series, uses the Llama 3 lineage as its language model backbone.