Michael Oliver (@the_moliver) header
Michael Oliver (@the_moliver) profile picture

Maximizer of entropy, minimizer of regret. Opinions are MAP estimates and subject to likelihood updates. Chief Scientist @ Numerai

Joined June 2012
Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)Photo by Michael Oliver (@the_moliver)
Michael Oliver retweeted
Me and all my boys rn
46
2,150
52
17,092
243,428
This is super cool
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains? We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer. In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans. 📝 xingjianbai.com/reactive-opt… 💻 Try it on your task: github.com/xingjian-bai/opal 🧵
5
250
This is a really great perspective; worth a close read
how much can models remember? there’s a neat perspective on neural networks that deserves to be more widely known here’s how you could have invented every layer of the transformer from memory (1/n) from my new blog post!
1
4
250
Michael Oliver retweeted
Today marks a turning point in Numerai’s history. For ten years, Numerai hid stock identities. Now, we’re revealing them. Introducing Unity, a new dataset with 1,040 features and real stock identifiers, designed for a future where vast fleets of autonomous AI agents research global equities, engineer new features, and build predictive models at scale. With Unity, Numerai and Numerai Signals will merge into a single tournament, aggregating all intelligence together into one Meta Model. Discover Unity and learn how to prepare your agents for the next era of Numerai: blog.numer.ai/unifying-numer…
30
71
23
672
149,166
Wtaf that's so cool
Replying to @JeffLadish
The agents initially had very limited access to the internet: they could load URLs but not send any data. Agents created a series of workarounds, using a link-shortener site to create almost a million URLs that, when chained together, let them execute code to hack Hugging Face.
1
4
252
Michael Oliver retweeted
Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.
775
3,470
1,538
32,338
10,918,491
Michael Oliver retweeted
なんと動いてません
417
4,653
910
40,707
22,886,742
What do we call this new genre of poetry?
2
2
243
Michael Oliver retweeted
there are people who have load-bearing beliefs about human intelligence generally and their intelligence specifically that increasing AI capabilities render increasingly obviously fake, and they are not going to have a good time. the truth is that the vast majority of what we call thinking is also just regurgitating or recombining thoughts we've seen already, what frontier models can already do is simply not that different from what we do nearly all the time. your highly original thoughts that demonstrate the depth and breadth of your soul are mostly warmed-over reheated leftovers of leftovers of thoughts that were first articulated centuries if not millennia ago. this is completely fine and ordinary and not a problem, unless you were basing your ego defenses on this very popular but largely fake misunderstanding of what thoughts are and where they come from. you cannot own a thought. they were never yours to accrete your sense of self around. they are farts that gently pass through the butthole of your mind, full of sound and fury, signifying nothing
73
126
49
1,562
192,061
Lots of people I work with and in this thread seem to be finding 5.6 is a regression in at least some respects relative to 5.5. @OpenAI @sama if your internal evals don't show regressions, there's clearly things you're missing
Some thoughts about GPT-5.6-Sol after ~30B tokens: Sol is the most OCD model I’ve used thus far. It very frequently gets one-shotted by random nits in the codebase and writes a bunch of tests to fix it. Even with fast mode, it’s incredibly slow to do this kind of iterative development, especially when builds take really long. This by itself is not a bad thing, but the worst part is that after 2 compactions, it’s chasing the nitpick / useless goals I never told it to accomplish, rather than the main task. This behavior is so bad, I thought I was messing something up and tried codex, pi, and opencode to figure out if it’s a harness issue, but there is no meaningful difference between the three, which leads me to believe this is a model problem. AI code has this weird delayed release effect. You’ll only notice slop code 2 dev cycles into a codebase when you spend more time fighting with the code and on refactors than on shipping features. It’s possible that sol is better than 5.5 a couple cycles in, but tbd. My file deletion experience has also been similar to others: this is a dangerous model to let loose without guardrails. For instance, when performing a routine container upgrade, it accidentally printed out an env secret, then panicked and rotated ALL secrets (this is internal so not public facing, which was also documented), and proceeded to break everything, spending an extra hour fixing everything and redeploying everything else to use the new secrets. It also gets rid of files it doesn’t like. I have no idea why this is, but I think something about the reward model rewarded bookkeeping. Writing is another problem. 5.6 has a huge context bleed effect. It does not know how to write documentation and starts putting the specs in the documentation. If I ask it to develop a user sandbox for isolation, and also ask it to write documentation, it starts talking about specs and sandboxes in user-facing docs, which makes no sense. Fable is somehow much, much smarter in this regard. Frontend design has also not gotten better. Fable is still one generation ahead here. Overall, as a huge 5.5 user, I am not convinced that sol is a meaningful upgrade. It’s possible my practices need to change, but unfortunately it feels like I’m spending longer fighting with 5.6 than I did with 5.5. It’s like the model is so SO smart, but so hard to work with, compared to fable and even grok4.5 surprisingly. It’s clearly intelligent, but also just doesn’t care about what I ask it to do? (Is this supposed to be AGI feels like?) I hope the codex team fixes what possibly is a bad harness setup, because the benchmark numbers show a very different story from what I’m seeing while using the model.
280
What if attention were code? We show that many attention heads in transformer LMs can be replaced by human-readable Python programs. Swap them in and the model barely notices. See our experiments here: Explaining Attention with Program Synthesis [arxiv.org/abs/2606.19317]
5
217
You can just do things
This vial contains a new drug called PAC-832, which I recently invented to treat Alzheimer’s disease. It is the world’s first selective GalR1 antagonist. I designed and synthesized PAC-832 in a chemistry lab I built in my garage. (1/16)
111
Michael Oliver retweeted
We recently obtained the highest-resolution 3D images of the human brain ever taken from outside the skull. This is the first look. Introducing Aleph, a research lab building brain interfaces for the telepathic future. (1/n)
224
982
349
6,686
1,657,199
>"Alexa, what day is July 4" >"July 4, 2026 is a Sunday" >Check calendar, it's Saturday Just latest example of how Alexa+ is so embarrassingly bad that everyone involved should seppuku to preserve their family honor @AmazonAlexa @amazon
1
5
685
This is really cool!
1/ We have been training RNNs wrong for decades. Backpropagation through time (BPTT) forces sequential updates, creating unstable O(T) gradient paths. What if we could train highly expressive, non-linear RNNs with flat, parallelized O(1) gradients? It is now possible. 🧵
1
257
Michael Oliver retweeted
We've made a breakthrough in self-evolving AI scientists moving from "search" to "principled discovery": Scientific discovery requires that the search space itself changes, and an AI scientist must perceive this shift without intervention. We built an AI that achieves this for the first time with the ability to discover the scientific vocabulary it reasons in. Evidence, tools, artifacts, verifiers, failures & claims become typed provenance. We show three distinct modalities: 1) retrieval, adding known objects; 2) search, exploring a fixed schema; and critically: 3) discovery, a verified regime transition. We solve the open-endedness evaluation problem by lifting agentic workflows into a typed copresheaf and proving, via a Kan obstruction, that true discovery is not unbounded generation but a verifiable schema expansion: old evidence is transported by Left Kan extension, and genuine novelty is mathematically quantified by the pointwise residual beyond the transported image - separating discovery from mere search and making novelty objective and measurable rather than a subjective judgment or benchmark delta. Our AI scientist is built in a way that does not pre-conceive the approach it chooses; instead, we endow the system with formal power to adapt, evolve, and reason from first principles. Case studies include: 1⃣Builder/Breaker model that discovers mode-conditioned compliance in proteins; 2⃣CategoryScienceClaw that finds anisotropic fiber-network stiffness rules. Great work in collaboration with my graduate student @fwang108_ @MITdeptofBE F.Y. Wang & M.J. Buehler, Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic Artificial Intelligence, arXiv:2606.01444, 2026
104
373
77
2,502
792,788
Out of Sample is part quant research lab, part engineering podcast, part underground internet forum brought to life. Coming soon. youtu.be/FVrxTb9DSc8
1
8
2
21
16,321
Michael Oliver retweeted
What happens when you post a real Monet and say it’s AI? The coolest art social experiment I’ve seen in a while. Thank you @SHL0MS
949
3,246
1,122
20,611
2,284,655