ZGCM-1
A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search
ZGCM Team, Zhongguancun Academy, Zhongguancun Institute of Artificial Intelligence
8
Concepts
Paper Overview
ZGCM-1 (ZGCM Team — Zhongguancun Academy & Zhongguancun Institute of Artificial Intelligence, 2026) is a fully open, 7.39B-parameter dense foundation model trained from scratch for mathematical reasoning and agentic search, and it is built to make a point about who gets to do frontier research. Two barriers keep the academic community out of this space: a scale barrier — deep reasoning is assumed to require hundreds of billions of parameters — and an opacity barrier — the models that do exist are released as opaque weights, with their filtering recipes, mid-training curricula, long-context schedules, and agent data kept private.
ZGCM-1 attacks both. Its core premise is that a compact model cannot passively memorize the open web, but it can overcome that parametric ceiling by coupling deliberate internal thinking with active external tool use — thinking harder and looking things up instead of trying to store everything. To make that trainable on academic compute across a 256K context, the team builds an end-to-end high-efficiency recipe: a hybrid gated sliding-window / global attention backbone, a stable FP8 Muon training stack, a progressive context curriculum with MDP-style supervision, and execution-grounded think/no-think alignment.
The efficiency numbers are the headline. Hybrid attention cuts the KV cache 6.4x and lifts throughput 3.94x at 256K; the FP8+Muon+TWEO co-design multiplies out to a ~4.2x pre-training time-to-loss speedup over a BF16/AdamW baseline. The capability numbers say the thesis holds: ZGCM-1-7B ranks first on average across 14 reasoning benchmarks at the 7B-8B scale (97.1% MATH-500, 75.0% AIME 2026) and stays competitive with frontier models such as Qwen3-235B-A22B and GLM-5.1 on agentic search.
And it is fully open: weights from every stage, intermediate checkpoints, training code, per-stage data and recipes, and W&B logs — plus eight distilled empirical findings and an honest account of an AI-native, agent-swarm development workflow.
Chapter Roadmap
Click any topic to jump in
The Scale & Opacity Barriers
Frontier reasoning is treated as the preserve of hundred-billion-parameter models with closed recipes. ZGCM-1's thesis: a compact model can compensate for parametric limits by coupling deliberate thinking with active tool use.
Hybrid Attention Architecture
Interleave gated sliding-window attention with global attention at a 5:1 ratio — 27 local layers, 5 global — for a 6.4x smaller KV cache and 3.94x throughput at 256K.
FP8 + Muon + TWEO Co-Design
A stable FP8 training stack: the Muon optimizer, hybrid FP8 precision, and TWEO outlier suppression multiply into a ~4.2x pre-training time-to-loss speedup over BF16/AdamW.
Curriculum & MDP Mid-Training
Progressively scale context across 600B tokens (16K to 64K to 256K) while reformulating agent interaction traces into Markov Decision Process state-action transitions for dense, step-level supervision.
Post-Training: Think/No-Think + RL
Quality-pruned SFT that interleaves explicit reasoning with direct answers in one weight set, followed by mixed GRPO reinforcement learning on math, code, and general tasks.
Agentic Search & System Agency
Deliberate thinking coupled with search, visit, and shell tools lets a 7B model contend with frontier systems on open-web deep research and stripped-binary function recovery.
Outcomes and lessons
Results
First on average across 14 reasoning benchmarks at the 7B-8B scale: 97.1% MATH-500, 75.0% AIME 2026, and 63.1% WebWalkerQA — competitive with models orders of magnitude larger.
Eight Findings & AI-Native R&D
Eight distilled empirical findings plus a candid audit of an agent-swarm development workflow whose autonomy is high for operations but low for architecture and algorithm design.
Concept by concept
8 sections, in reading order.
Unlock the full breakdown
All 8 concepts — figures, key points, formulas and interactive demos — are part of PixelBank Premium.
- 01The Scale & Opacity Barriers
- 02Hybrid Attention Architecture
- 03FP8 + Muon + TWEO Co-Design
- 04Curriculum & MDP Mid-Training
- 05Post-Training: Think/No-Think + Mixed RL
- 06Agentic Search & System Agency
- 07Results
- 08Eight Findings & AI-Native R&D
Keep reading
Related Papers
Go deeper