Personal update: after four years as Data/Tech Lead at @hashed_official, I'm joining @realAsteromorph as a Research Engineer, working on AI for Science. Hashed was the best team I've been part of. Working at the intersection of business and tech, those four years genuinely widened my perspective and my capabilities. @simonkim_nft , @Hashed_HongPro , @0xryankim @baekkyoumkim, thank you for trusting me with so much. 🙏 And if a founder ever asks me what Hashed is like from the inside, my answer is simple: I've never seen a team care this much about its people and its founders. Asteromorph is one of the youngest and most talented teams I've come across, taking on how AI can accelerate scientific discovery. I'm looking forward to the days ahead. I won't be at the ICML main conference, but with the whole field gathering in Seoul this month, I'll be around main workshops, the side events and meetups. If you're in town, I'd love to meet and talk AI for Science.🚀

there’s a lot of research on rsi and using ai to improve ai, but doing it well is really hard. getting a model to decent performance is pretty doable now, but reaching the top still isn’t, even on kaggle. keep experimenting 😎 @kaggle @GoogleColab

ai right now is 2021 defi. slop and real innovation mixed together, rounds and valuations getting bigger, the bar keeps moving. and like crypto, plenty of it pretends to work and doesn't. until you've used it and it actually ships, doubt the benchmarks and the experts. still, ai can go much further. fun to be doing the research and engineering that makes it the future, not just a bubble.

gpt6 astra isn't just good at code stuff anymore, design got a serious upgrade too. might dust off my f1 side project again

benchmark saturation (or benchmaxxing) is happening faster than before, but new benchmarks are appearing just as fast and i don't think they're being validated enough. the people who can make hard, important problems and good data are only going to matter more. a lot of people now try the models themselves instead of just comparing benchmark scores. that's what crossing the chasm feels like.

I think I'm pretty good at building PoC-level prototypes.😎 New @RSNA @kaggle competition

still very early in the competition, and this will probably be short-lived, but it's a moment I've never experienced before. @kaggle 1/900

not a bad start @kaggle

told you😏

<Lessons Learned from @kaggle Neurogolf 2026> - Neurogolf 2026 is based on the ARC-AGI 1 benchmark. The goal is to minimize memory usage and parameter count. I spent about a month using it as a testbed for loop engineering in AI4Science, automating as much as possible. It was my first return to Kaggle in 4 years since becoming a Kaggle Notebooks Grandmaster. I now feel just as drawn to competitions with real skin in the game as I do to sharing educational content and EDA. - The Kaggle CLI was one of the highlights. It makes submissions and competition data collection easy to automate. Datasets, notebooks, leaderboard data, and other assets can all be downloaded and managed through one interface. This made it a strong base for long-running loops. I used 2x Claude Code agents and 2x Codex agents. Claude was more useful during early exploration, while Codex became my main tool once I had a clearer direction. - My biggest takeaway was that prompt-based loop engineering struggles to escape local optima. Without outside knowledge, LLMs tend to keep refining the same ideas. The biggest jumps still came from human input. One key insight was that memory had a much larger effect on the score than parameter count. Once I told the loop to reduce memory, sometimes to zero, the score improved significantly. - Evaluation was just as important as generation. The benchmark used fixed tasks that humans could understand, so I could build an internal evaluation gate. That became the basis of the loop. - Long-running loops are still hard to maintain. I tried to make the process as autonomous as possible, but I still had to step in between once and ten times a day. Once the loop found a useful technique, it could keep climbing for more than twelve hours without intervention. - New model releases also had a visible effect. Scores rose when Fable appeared and again when GPT-5.6 was released. Personally, new versions also caused practical issues. CPU/memory usage became unstable, sub-agents conflicted, and goals that had worked before sometimes broke. - Kaggle competitions reward problem framing and insight more than raw implementation effort. Paying attention to the competition also matters. I was around the top 100 until three days before the deadline. Then someone shared a high-scoring public solution, and I fell outside the top 200 within two days. Public code had also helped me reach the top 100. That is part of the game. Regardless of rank or medal, a month-long competition is a great way to sharpen my thinking about evaluation and loop-based agent systems. For anyone interested in Kaggle loop engineering, I recommend @NVIDIAAI repository built by some of the strongest Kagglers. I am also continuing to automate my competition workflow and refine the loops behind it. :)

I should probably start a substack soon. thanks to @icmlconf, I’ve rediscovered how much I enjoy diving deep into papers and reviewing techs from frontier labs.

sudo pmset -a disablesleep 1

Agent benchmark evaluation will become an increasingly important task. And when thinking about "what makes a good benchmark," the 3-axes discussed in @SnorkelAI's recent presentation are a really useful framework: - Environment Complexity - Autonomous Horizon - Output Complexity I’ve also been wondering whether there should be a "time-related axis", covering things like contamination and how well a problem remains relevant over time, and when we might be able to detect those signals. @fredsala's talk and a few words of advice were helpful in thinking through this, and gave me more to think about :)

GPT-5.6 SOL FABLE 5 GROK4.5 SPARK1.1 GLM5.2 HY3 .... TOKEN IS ALL I NEED

I’ve been coming to COEX for over 10 years, and I’ve never seen it this crowded before.

We are already using AI on problems that are too complex, or too large, for even a small group of people to fully understand and verify. This changes what evaluation means. As benchmarks include more unseen tasks, false positives and false negatives become harder to avoid. A model may pass for the wrong reason, or fail for a reason the rubric does not capture. So the important question is not just whether the model got the answer right. It is whether the rubric is the right way to judge the answer at all. And when the model is wrong, we need to ask what kind of wrong it is. Why did it fail? How did it fail? What does the failure reveal? This is why expertise does not go to zero. As AI gets better, the need for people who can interpret, validate, and reason about its outputs becomes more important, not less.

ICML week, but still building at night a readable, searchable index of scientific llm benchmarks. > countless benchmarks and good scores, but still so few meaningful discoveries. why?

What does it mean for an AI model to be “good at science”? I started collecting Scientific LLM benchmarks in one place - across math, physics, chemistry, materials science, biology, and scientific agents. github.com/subinium/Awesome-… Suggestions are welcome.

i luv vanilla codex 600+ subagents🤖🤖🤖 @OpenAI