EUPE

Edge devices need vision encoders that are small ($<100$M params), fast ($<60$ ms on a phone CPU), and universal — strong at image classification, dense prediction (segmentation, depth, correspondence), and vision-language modeling, all from a single backbone. Domain experts achieve only one or two of these: PEcore and SigLIP2 win at classification and VLM but lose dense prediction; DINOv3 wins dense prediction but lacks language alignment.

The obvious fix is agglomerative distillation — distill from several expert teachers into one student — but prior work like RADIO and DUNE shows this falls apart at edge sizes. RADIOv2.5-B sits multiple points behind the best per-task expert on dense prediction and VLM. The reason is capacity: an 86 M-parameter student does not have enough room to absorb the conflicting feature distributions of 3+ specialized teachers directly.

EUPE (Meta, arXiv:2603.22387) introduces a deceptively simple recipe: scale up before scaling down. Stage 1 distills three foundation teachers — PEcore-G (1.9B, image-text), PElang-G (1.7B, language-aligned vision), and DINOv3-H+ (840M, dense prediction) — into a single 1.9 B-parameter proxy that has enough capacity to unify their representations. Stage 2 then distills that single unified proxy into the efficient student at fixed 256×256 resolution for 390 k iterations. Stage 3 finishes with a short multi-resolution finetune over a {256, 384, 512} pyramid for 100 k iterations.

The payoff: EUPE-ViT-B (86 M params, 47 GFLOPs, 55 ms on iPhone 15 Pro CPU at 256×256) matches or beats every same-size domain expert on its home task. ImageNet-zero-shot 79.7% beats PEcore-B (78.4) and SigLIP2-B (78.2). ImageNet-KNN 84.1% beats SigLIP2-B (83.2) and DINOv3-B (83.0). ADE20k mIoU 52.4 beats DINOv3-B (51.8). SPair correspondence 51.3 matches DINOv3-B (51.3). Versus the agglomerative baseline RADIOv2.5-B, EUPE-ViT-B wins by +15 points on SPair, +3.4 on ADE20k, +3.4 on TextVQA, +1.5 on GQA. The full family ships at six points on the size frontier: ViT-T (6 M), ViT-S (21 M), ViT-B (86 M), and ConvNext-T (29 M), -S (50 M), -B (89 M).

The ablation tells the real story. Without the proxy — i.e. distilling all three teachers directly into the 86 M student (Stage 2 only) — SPair collapses to 35.1 and ADE20k to 41.9. Add Stage 1 and you reach 41.0 / 43.3. Add Stage 3 multi-res on top and you finally hit 51.3 / 52.4. The proxy is not optional cleanup; it's load-bearing.