ZGCM-1
A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search
ZGCM Team, Zhongguancun Academy, Zhongguancun Institute of Artificial Intelligence
Read the Paper on arXivPaper Overview
ZGCM-1 (ZGCM Team — Zhongguancun Academy & Zhongguancun Institute of Artificial Intelligence, 2026) is a fully open, 7.39B-parameter dense foundation model trained from scratch for mathematical reasoning and agentic search, and it is built to make a point about who gets to do frontier research. Two barriers keep the academic community out of this space: a scale barrier — deep reasoning is assumed to require hundreds of billions of parameters — and an opacity barrier — the models that do exist are released as opaque weights, with their filtering recipes, mid-training curricula, long-context schedules, and agent data kept private.
ZGCM-1 attacks both. Its core premise is that a compact model cannot passively memorize the open web, but it can overcome that parametric ceiling by coupling deliberate internal thinking with active external tool use — thinking harder and looking things up instead of trying to store everything. To make that trainable on academic compute across a 256K context, the team builds an end-to-end high-efficiency recipe: a hybrid gated sliding-window / global attention backbone, a stable FP8 Muon training stack, a progressive context curriculum with MDP-style supervision, and execution-grounded think/no-think alignment.
The efficiency numbers are the headline. Hybrid attention cuts the KV cache 6.4x and lifts throughput 3.94x at 256K; the FP8+Muon+TWEO co-design multiplies out to a ~4.2x pre-training time-to-loss speedup over a BF16/AdamW baseline. The capability numbers say the thesis holds: ZGCM-1-7B ranks first on average across 14 reasoning benchmarks at the 7B-8B scale (97.1% MATH-500, 75.0% AIME 2026) and stays competitive with frontier models such as Qwen3-235B-A22B and GLM-5.1 on agentic search.
And it is fully open: weights from every stage, intermediate checkpoints, training code, per-stage data and recipes, and W&B logs — plus eight distilled empirical findings and an honest account of an AI-native, agent-swarm development workflow.
Chapter Roadmap
Click any topic to jump in
The Scale & Opacity Barriers
Frontier reasoning is treated as the preserve of hundred-billion-parameter models with closed recipes. ZGCM-1's thesis: a compact model can compensate for parametric limits by coupling deliberate thinking with active tool use.
Hybrid Attention Architecture
Interleave gated sliding-window attention with global attention at a 5:1 ratio — 27 local layers, 5 global — for a 6.4x smaller KV cache and 3.94x throughput at 256K.
FP8 + Muon + TWEO Co-Design
A stable FP8 training stack: the Muon optimizer, hybrid FP8 precision, and TWEO outlier suppression multiply into a ~4.2x pre-training time-to-loss speedup over BF16/AdamW.
Curriculum & MDP Mid-Training
Progressively scale context across 600B tokens (16K to 64K to 256K) while reformulating agent interaction traces into Markov Decision Process state-action transitions for dense, step-level supervision.
Post-Training: Think/No-Think + RL
Quality-pruned SFT that interleaves explicit reasoning with direct answers in one weight set, followed by mixed GRPO reinforcement learning on math, code, and general tasks.
Agentic Search & System Agency
Deliberate thinking coupled with search, visit, and shell tools lets a 7B model contend with frontier systems on open-web deep research and stripped-binary function recovery.
Outcomes and lessons
Results
First on average across 14 reasoning benchmarks at the 7B-8B scale: 97.1% MATH-500, 75.0% AIME 2026, and 63.1% WebWalkerQA — competitive with models orders of magnitude larger.
Eight Findings & AI-Native R&D
Eight distilled empirical findings plus a candid audit of an agent-swarm development workflow whose autonomy is high for operations but low for architecture and algorithm design.
The paper opens with two obstacles that, together, wall the broader community off from frontier reasoning. The scale barrier is the widely held belief that deep agentic search and hard mathematical reasoning are the exclusive preserve of models with hundreds of billions of parameters — models only a handful of labs can afford to train. The opacity barrier is subtler and, for research, worse: most competitive models are released strictly as open-weight rather than fully open-source. You get a checkpoint, but not the upstream filtering recipes, the mid-training curricula, the long-context schedules, or the multi-turn agent data that produced it. You can run the model; you cannot study how it was made.
ZGCM-1's answer to the scale barrier is a thesis about capacity. A 7B model simply does not have the parameters to memorize the open web the way a 235B model can. But the paper argues that raw memorization is the wrong target. A compact model can overcome its parametric ceiling by coupling deliberate internal thinking with active external tool use — spending inference-time computation on reasoning, and offloading factual recall to search and code execution rather than storing it in weights. Capacity that would otherwise go to memorization is redirected to process.
The answer to the opacity barrier is simply to open everything. The team releases model weights from the pre-training, mid-training, and post-training stages, intermediate checkpoints, training code and configurations, per-stage data and data recipes, W&B logs, and evaluation harnesses. The stated goal is that a lab with academic-scale compute can not only use ZGCM-1 but reproduce and extend the recipe that built it.
Everything downstream in the paper serves this thesis. The architecture and system co-design exist to make 256K-context training tractable on a modest cluster. The curriculum and agentic mid-training exist to teach the model how to think and act rather than what to remember. And the evaluation is designed to test the central claim directly: can a fully open 7B model, given tools and time to think, contend with the giants?
Key Points
Scale barrier: frontier reasoning and deep agentic search are widely seen as requiring hundreds of billions of parameters, excluding most of the academic community on compute grounds alone
Opacity barrier: competitive models ship as open-weight, not fully open-source — the filtering recipes, mid-training curricula, long-context schedules, and agent data stay private
Core thesis: a compact model cannot passively memorize the open web, but can overcome parametric capacity limits by coupling deliberate internal thinking with active external tool use
The model is a 7.39B dense decoder-only Transformer (GQA, RMSNorm, SwiGLU, RoPE) with a maximum context of 256K tokens
Fully open: weights from pre-training, mid-training, and post-training stages, intermediate checkpoints, training code, per-stage data and recipes, and W&B logs are all released
The whole recipe is engineered for academic compute budgets — every architecture and system choice is a tractability decision, not just a quality one