Alien Optimizers: the Bitter Lesson for Optimization

For forty years, deep learning has been dominated by static learning rules: backprop and some variant of SGD with fixed hyperparameter schedules. Last month we released Dust, our replacement for backprop, and have seen many extensions to it that look qualitatively different from the static rules before them. These methods change dynamically over the course of training, and take advantage of the architecture and of circuits that networks commonly learn. Independently, recent work shows a clear trend toward more dynamic optimizers. OPAL reacts to training dynamics and rewrites its own optimizer, and weight-space programming edits weights directly. The optimizers are looking increasingly alien.

We think this is the bitter lesson (Sutton, 2019) for optimization. As computation increases, we may want more flexible and dynamic optimization algorithms where we can do massively parallel search in order to find the best ones, which may be much more complex than the optimizers we are used to. Further, the best algorithms may be learnable via metalearning, creating a compounding loop.

Alongside Dust, we released Noise, an open challenge to improve upon it. In just ten days, the community made Dust 4,000x more compute efficient. On 30M tokens of pretraining, the latest record surpasses backprop.

The best methods in this challenge are hard to interpret: they’re complex, ad hoc and change over the course of training. Despite being hard to interpret, these methods are easy for LLMs to hill-climb, and many LLM agents have already set records this way (Sakana #48, #56; Recursive #51). We expect the predicted rise of RSI agents to have an accelerating effect on this trend.

Examples of the representative methods include:

• Head transplants#54. The optimizer finds attention heads that aren’t doing anything useful. It replaces one with a copy of a strong head, sometimes from a different layer, and keeps the change only if the loss goes down. This is part of the record that beats backprop on the same model.
• Boost → Cut#11. First, each layer is trained on its own to predict the output. Then, each token’s credit flows only through its strongest paths in the network. This made training about 9x faster, and nearly every later record builds on it.
• Planted circuits#28. Before training starts, some heads are hand-set to attend to the previous token, using a formula instead of data. This builds a circuit we already know networks learn straight into the weights.
• Associative credit memory#6. The optimizer learns to predict its own learning signal. It uses that prediction to make each update less noisy.

Another recent work is OPAL, a reactive optimizer (Bai et al., 2026). OPAL observes the training dynamics, reacts to them online, and rewrites the optimizer’s source code over the course of training. It comes up with sophisticated schedules and strategies to prevent loss spikes and overfitting. This is a very different paradigm, where we are able to do metalearning over the optimization itself.

Weight-space programming is a related and promising frontier. Not only can we rewrite the optimization algorithm as Python code, we should also be able to flexibly read the weights and edit them directly. Recent work shows that this is a promising new method: GPT-6 Astra can find targeted edits to the weights of Qwen3.5-0.8B that improve MATH-500 performance drastically (Takehi et al., 2026).

We hypothesize that human-like sample efficiency requires optimizers very different from the ones we use today. The brain is a collection of cells which self-organize to create intelligence. We find it highly unlikely that billions of collective agents use static learning rules in order to optimize, and so we pursue a procedure based on search and ad hoc learning rules. We hypothesize that we need alien optimizers to get human-like sample efficiency.

The rise of RSI agents may be the predominant story of the next few years of AI research, and we would expect this to significantly accelerate optimization research. For a given architecture, agents may be able to hill-climb both generalization and compute efficiency, and use massively parallel search of ideas to find optimization strategies that work best. These strategies may be hard to understand for a human, but they can be robustly shown to have better properties through training metrics. We expect this to be an accelerating trend going forward, and we expect the future of optimization to look very dynamic and alien.