In our Nature paper, we introduce the first superhuman Stratego AI, which we built using general techniques that we developed for RL & test-time compute under imperfect information. 1/N
This project was a multi-year collaboration with @EugeneVinitsky, @HengyuanH, @nanomena0, @zicokolter, and @gabrfarina, over which time we found numerous tricks that allowed us to succeed despite being limited to academic levels of compute. 8/N
For many details, including a purely GPU-based RL loop and data-filtering scheme that increased our sample-efficiency, compute-efficiency, and asymptotic performance, see the paper or codebase linked below. N/N
github.com/AtaraxosAI/strate…
nature.com/articles/s41586-0…
Samuel Sokota retweeted
1/ Ten months ago, I was ecstatic that AI could win IMO gold.
Today, that excitement feels quaint: an internal @OpenAI model has refuted Erdos’s unit distance conjecture—a research result that one could recommend “acceptance without any hesitation” to the Annals of Mathematics.
Today, we share a breakthrough on the planar unit distance problem, a famous open question first posed by Paul Erdős in 1946.
For nearly 80 years, mathematicians believed the best possible solutions looked roughly like square grids.
An OpenAI model has now disproved that belief, discovering an entirely new family of constructions that performs better.
This marks the first time AI has autonomously solved a prominent open problem central to a field of mathematics.
Samuel Sokota retweeted
New paper: "Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs".
Our system (BLF) matches human superforecasters on ForecastBench, and beats all the top methods (GPT-5, Cassi, Grok 4.20, and Foresight-32B).
🧵
Samuel Sokota retweeted
Hi everyone it's me aaron i made a youtube video about the game theory behind competitive pokemon. this has been ~2.5 months in the making and i am so happy to finally be able to release it.
ALT Thumbnail for my video. There is a pokeball in the middle of the picture, with various icons pointing with curved arrows towards it-- a stylized chess knight, hands representing rock paper scissors, a poker chip, a die, and a big block with a question mark on it.
Samuel Sokota retweeted
Are we done with new RL algorithms? Turns out we might have been optimizing the wrong objective.
Introducing MaxRL, a framework to bring maximum likelihood optimization to RL settings.
Paper + code + project website: zanette-labs.github.io/MaxRL…
🧵 1/n
The temporal difference error (TDE) is a fundamental object in RL because weighted sums of TDEs can be used as feedback for both value function and policy updates.
I was recently surprised to realize that there is a natural control variate for the TDE. 1/N
The existence of these control variates has been known at least as far back as 2018, when they were proposed by @M33pinator & @RichardSSutton.
arxiv.org/abs/1807.01830
Yet, despite their seeming fundamentality, they don’t seem to have had much uptake in deep RL. 6/N
