Qwen2.5-VL
A Flagship Vision-Language Model for Documents, Grounding, Long Video and GUI Agents
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, Junyang Lin
Read the Paper on arXivPaper Overview
Qwen2.5-VL (Qwen Team, Alibaba Group — technical report, February 2025) is the flagship of the Qwen vision-language series, released open-weight in 3B, 7B and 72B sizes. Its framing is memorable: today's large vision-language models are like the middle of a sandwich cookie — competent across many tasks, exceptional at none. Qwen2.5-VL argues the missing base layer is fine-grained perception: reading a dense invoice, pointing at the third bolt from the left, localizing an event to the second in an hour-long video.
The architecture is the familiar three-part stack — a vision encoder, a cross-modal merger, and the Qwen2.5 LLM — but each piece is re-engineered for native inputs. The encoder is a Vision Transformer trained from scratch with window attention in all but four layers, so cost scales linearly with the number of patches instead of quadratically. Images are never squashed to a fixed grid; they enter at their real resolution, and object coordinates are expressed in absolute pixels rather than normalized to . For video, MRoPE is aligned to absolute time, so a clip sampled at 1 FPS and the same clip at 2 FPS produce the same temporal position IDs — the model learns the pace of events, not the frame count.
Those choices are backed by a data effort that scales the pre-training corpus from Qwen2-VL's 1.2 trillion tokens to roughly 4.1 trillion, spanning interleaved image-text, OCR, document omni-parsing (everything as HTML with bounding boxes), grounding, long video, and GUI-agent trajectories. Post-training adds SFT over ~2M filtered pairs, chain-of-thought rejection sampling, and DPO — all with the ViT frozen.
The result: Qwen2.5-VL-72B matches GPT-4o and Claude 3.5 Sonnet, leading on document and diagram understanding (e.g. 96.4 DocVQA, 70.2 MMMU), while the 7B and 3B models outperform same-size competitors. Grounding turns into agency — 43.6 on ScreenSpot Pro, far past its own predecessor's 1.6 — letting the model drive real computers and phones without task-specific fine-tuning.
Chapter Roadmap
Click any topic to jump in
The Fine-Grained Perception Gap
LVLMs are competent everywhere and exceptional nowhere — the 'sandwich cookie' problem. Qwen2.5-VL bets on fine-grained perception as the missing base layer.
Native-Resolution ViT
A from-scratch ViT with window attention in all but four layers, 2D-RoPE, RMSNorm and SwiGLU — native resolution at linear cost.
MRoPE Aligned to Absolute Time
Temporal position IDs are tied to real seconds, not frame counts, so one model reads video consistently across any FPS.
MLP Vision-Language Merger
Four spatially-adjacent patches are grouped and projected by a 2-layer MLP — a 4x token cut before the LLM ever sees them.
Pre-Training — 4.1T Tokens
Three staged phases over ~4.1T tokens: ViT warm-up, joint multimodal training, then long-context video / document / agent data.
Post-Training — SFT + DPO
Two million filtered instruction pairs, CoT rejection sampling and DPO — all with the ViT frozen.
Benchmarks and agentic use
Results
The 72B flagship matches GPT-4o and Claude 3.5 Sonnet and leads on documents; the 7B and 3B beat same-size competitors.
Visual Agent
Grounding turns into agency: 43.6 on ScreenSpot Pro and real computer / phone control without task-specific fine-tuning.
Qwen2.5-VL opens with a diagnosis rather than a benchmark. The current generation of large vision-language models, the authors write, resembles the middle layer of a sandwich cookie — competent across a broad spread of tasks but falling short of exceptional performance on any of them. The interesting question is which layer is missing.
Their answer is the bottom one: fine-grained perception. A model can describe a photo fluently yet fail to read the total on an invoice, count the bottles on a shelf, point at a specific UI button, or say when in a two-hour video an event happened. These are not exotic capabilities — they are the substrate that real applications (document processing, robotics, GUI agents) are built on, and they are exactly where generalist LVLMs are weakest. So Qwen2.5-VL treats fine-grained perception as the foundation to establish first, with multi-modal reasoning (inherited from the strong Qwen2.5 LLM) as the top layer resting on it.
The paper is candid about why perception has lagged: the field's bottlenecks are computational complexity, limited context, poor fine-grained visual perception, and inconsistent performance across sequence lengths. Each of the report's four technical contributions targets one of these directly — window attention for compute, dynamic-FPS sampling and long-context training for length, absolute-time MRoPE for temporal precision, and a 4.1T-token data effort for perception breadth. Read that way, the rest of the paper is a systematic attack on the reasons the cookie's base layer was thin.
Crucially, none of this is bought with task-specific heads. Qwen2.5-VL aims for strong generalization without fine-tuning per task — the same weights detect objects, parse documents, localize video events and operate a phone. The bet is that if perception is good enough and native enough, generality follows.
Key Points
The 'sandwich cookie' framing: LVLMs are competent across tasks but exceptional at none — the missing base layer is fine-grained perception
Four named bottlenecks the paper targets: computational complexity, limited contextual understanding, poor fine-grained visual perception, and inconsistent performance across sequence length
Four matching contributions: window attention (compute), dynamic-FPS sampling (temporal length), absolute-time MRoPE (temporal precision), and 4.1T tokens of curated data (perception breadth)
Released open-weight in three sizes — 3B, 7B, 72B — spanning edge AI to high-performance computing
Strong generalization without task-specific fine-tuning: the same weights do detection, document parsing, video grounding and GUI control
Preserves the core language competencies of the underlying Qwen2.5 LLM — multimodal ability is added, not traded for