SO MUCH TO RED TEAM so little time

env + scorer isn't enough without baseline runs. otherwise nobody knows if their number is any good

New jailbreak technique dropped: European Union laws

We turned @OpenAI's math problems into open-source RL environments on @huggingface! Early and rough (the verifier only accepts the exact original formalization, so an equivalent proof can still score 0), but this is what it looks like when a research release becomes executable training infrastructure for everyone. This is an exciting direction imo: turning open research into open executable environments that anyone can build on to train better open models! huggingface.co/datasets/Fine…

I make this same post like every 2-3 years, because there's always new data further supporting it. Georges Harik (@gharik) is one of the most prescient people in AI, and definitely the least well known and underappreciated one. In 1999 he joined Google as one of the first employees (like #7 or something) because he thought he could use his AI knowledge to build a better ad system. He and Salar Kamangar built Adwords (Salar later bought YouTube and Georges later was the force behind Google wanting to buy Andy rubin's company to build Android) Georges left Google so he could focus on AI research, which he self funded. He did so with someone he worked with closely at Google, our friend Noam Shazeer (they built the AdSense targeting engine). When Google finally decided to work hard on AI around the time of the deepmind acquisition, they tried to get Georges and Noam to come back. Georges wanted to stay independent, and our friend Noam went on to do great things obviously. This was my good fortune because it led to Georges and I hanging out a lot, because he basically needed a new lunch buddy. Georges at the time was working on language models and despite karpathy saying nobody saw LLMs being so big even in 2017, Georges was telling me they were on the cusp of changing everything in 2013. My wife and I literally changed some of our parenting priorities that year (less tigery) because georges convinced me the world was going to change so much. Later on he and I got involved in a company making CV photo filters called Lambda. They needed to make money to survive so they started building ML boxes which Georges bought a bunch of. But his significant other was mad there were so many loud and hot boxes in the house, she started worrying about fire risk, so Georges told the lambda guys they should build a cloud because in the future, the power density of the systems was going to be too big for even most enterprises to handle. This was like 2017. That's how lambda cloud began, and the premise Georges had was that AI was going to change things so much, demand growth would outpace supply growth, and the value of compute was eventually going to rise and that lambda was effectively going to be a REIT... so we should own as many GPUs as possible. It took the world almost a decade to finally figure out what Georges saw then (see attached tweet). About 5 years ago when pretraining was still the only thing people cared about, Georges started telling me inference was going to be super impt. He was working on research investigating post training as a way to improve reasoning. He found some other guys working on the same idea that had a paper known as the STAR paper, and they worked together to prove it even further with another paper called quietstar. (They all later cofounded the neolab humans&). Georges got some friends to build deepinfra.com to be one of the first inference clouds because he saw it all early. He and the humans& team also smartly used all their money to buy compute knowing it would go up in price and also get harder to obtain. It's kind of amazing to me how he has gotten everything right so far ahead of time. It's also amazing to me how VCs don't really get or believe a lot of the things he says, they often tell me I am wrong whenever I parrot Georges' current thoughts... and then years later they all put out charts and papers reaffirming those same things as if they knew it all along. The founders of Google recognize Georges' brilliance and still talk to him a lot. (Besides being smart, he is a lot of fun to hang out with.) I'm just wondering when the rest of the world figures it out