Dream-RSI

Dream-RSI (Zheng, Wu, Zhang et al. — Google, Google DeepMind, University of Maryland & University of Virginia, September 2026) attacks a bottleneck that appears once AI agents start improving themselves. In recursive self-improvement (RSI), an agent runs a discovery loop — propose a solution, evaluate it, incorporate feedback, refine — and modern discovery problems push that loop across thousands of proposal–evaluation cycles. At that scale the deciding factor is no longer the coding agent but the exploration strategy: which branches to pursue, which candidates to refine, how much to run in parallel, when to stop.

Improving that strategy is caught in a dilemma. Hand-designed strategies stay fixed and cannot learn from accumulated experience. Optimizing them online is brutally expensive: unlike scoring one candidate, scoring an exploration policy requires watching it shape a whole discovery run over many cycles — delayed, costly feedback — and the space of policies is vast, so many must be tried.

The paper's key insight is that the feedback you need has already been paid for. A completed discovery run records a structured tree of past decisions and their realized code-execution outcomes. Organized as a tree, that history becomes a replay simulator: an alternative policy can traverse different branches, in different orders, with different parallel groupings, and be scored by simply revealing outcomes already stored — no agent, no evaluator rerun. By analogy to model-based RL and World Models, Dream-RSI lets the agent “dream” over these worlds to refine its exploration policy off-policy, then redeploys the improved policy online to gather new experience and grow the simulator pool — closing an RSI loop at the meta-exploration layer.

Across algorithm engineering, mathematical optimization, and GPU kernel engineering, Dream-RSI matches or beats strong baselines while cutting discovery cost dramatically — up to 162x fewer discovery-agent calls than SimpleTES on the Lasso task, and 2.43x fewer generations on a VGG16 kernel at comparable performance.