Jędrzej Maczan @ 🇭🇰 DAI 2026

So I know you've all been waiting for my take on weight-space programming, and honestly, I don't know whether to be excited, annoyed, or jealous. Or all three at once. The gist: researchers at MIT CSAIL (led by Omar Khattab, the man behind some seriously cool ideas) ran a few really simple experiments where they let Astra look at a small model's weights and update them, and it fixed them by hand. No training, no gradients. The result: the small model suddenly got much better, not just at math, but also at instruction following, and even at writing its own game-playing network. It also figured out which game a network had been trained on, just by looking at its weights. Excited: this is such a cool idea. Instead of training a model for weeks, you just edit its weights directly, like gene editing for neural networks. It's the dream we've always had: knowing how to edit specific weights. Annoyed: most of the magic here leans on existing interpretability tools which work only for small models, on being able to read things off the first or last layer, and on the fact that tasks like these already exist in the training data. So how much of this is just good PR for an old idea? Jealous: because I've tried things in this direction, and they worked a lot worse. A LOT worse. Bottom line: it's complicated.