NYC
Joined September 2017
- Tweets771
- Following631
- Followers2.4K
- Likes4.1K
I see, some of them are (unfortunately) paywalled. It is annoying for sure; I just go to the next one.
Stop scrolling on X and read something wonderful today readsomethingwonderful.com/ 🙂
Gotcha. I would be curious to understand how many additional rewards and how much domain randomization is needed to make regularized self-play RL work for a planner in the wild.
The regularizer should nudge the behavior towards conventions that are implicit in good human driving and can be hard to specify. For that reason, it should make the training easier.
Simulator is emerge-lab.github.io/PufferD… built on the pufferlib infrastructure puffer.ai/
Thanks for the tag @BrantonDeMoss!
@___Harald___ We have a series of papers on this topic. Most recently arxiv.org/abs/2606.19370. All code is open-source
Daphne Cornelisse retweeted
Making it possible to do things that were previously impossible is infinitely more exciting than speeding up things that were already possible.
Working on biology is humbling. It gives me a similar feeling to looking up at the stars at night and imagining how tiny the Earth is in relation to the rest of the universe.
Daphne Cornelisse retweeted
Using AI to write for you - start to finish - is really just you robbing yourself of one of life’s greatest joys: to think through a problem and figure out what you think about it, and in the process becoming passionate enough to take on the new challenge of solving it.
Daphne Cornelisse retweeted
Octi/Nvidia + Abhishek's lab on a generational run. dex manip will be solved with sim.
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn?
That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done.
Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves.
Paper: arxiv.org/abs/2610.13354
Website: alpha-nist.github.io
@elliotarledge The network encoder is the bottleneck. This project was done with PufferDrive 2.0. It runs at ~300-400k SPS on a 4090 or 5090. IIRC, using one of the standard PufferLib MLPs gets you 1-1.5 M SPS, but the entity encoder is more performant.
Starting from PufferLib 5.0 + spending a little time optimizing the regularizer implementation should get you to ~800k or higher SPS, but the network is still a bottleneck (overview below).
Daphne Cornelisse retweeted
In our Nature paper, we introduce the first superhuman Stratego AI, which we built using general techniques that we developed for RL & test-time compute under imperfect information. 1/N
Depends entirely on your interests and goals, of course!
One potentially interesting direction in Craftax could be to iteratively remove feedback and see whether tabula-rasa RL/Pre-trained LLMs can still solve it. E.g., can Astra still beat the final boss without any of the intermediate achievements? Another direction is that the models are typically large and expensive to use. Could you compress their knowledge about the world in some way? Third, I think we can still improve on-policy RL algorithms further.
Dwarf Fortress is an interesting suggestion, though it seems challenging to evaluate models given the open-ended nature of the game.