Shaurya Rohatgi (@shauryr) profile picture
sf, bay area
Joined April 2012
Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)Photo by Shaurya Rohatgi (@shauryr)
If models can write their own RL training tasks, can they review them as well as a human expert? I spent the weekend at the @ValsAI office building a benchmark to find out: Task Reviewer Bench (TRB). Placed 3rd in the Build Your Own Eval track as a solo hacker 🧵
9
6
3
131
10,891
our secrets are out!
Looking for a reference manual for building a frontier LLM? We wrote one 👇 The K2 Horizon technical report is now on arXiv: 134 pages on how we built six models, from architecture to inference. It may be the most detailed LLM training reference you can read today. • Architecture: we bring two new attention designs, MoVA and softplus attention gating, to strengthen reasoning. With them, our 36B model nearly matches a 32B dense one on one-seventh of the active parameters. • Pre-training: we trained four models, from 3.7B to 36B, on the same 22T tokens. Their normalized loss curves collapse onto nearly the same line, so a recipe that works on a small model carries over when scaling up. • Data: 60+ pages on the part most reports skip. How we generated ~10T synthetic tokens, and why we chose the data mix on predicted scores at scale, which reversed the small-run ranking. • Post-training: we document our full production-scale run for the 375B flagship. • Inference: Uno gives an autoregressive model diffusion speed with one small adapter. Up to 2.2× faster decoding, lossless by construction, no draft model. This is the last piece of the openness puzzle. K2 Horizon already shipped with 380 checkpoints, 13 TB of pre-training data, 58 training runs and 17 code repositories. The report explains the decisions behind them. More details: arxiv.org/abs/2611.21003
3
27
1,367
also got some anthropic swag
If models can write their own RL training tasks, can they review them as well as a human expert? I spent the weekend at the @ValsAI office building a benchmark to find out: Task Reviewer Bench (TRB). Placed 3rd in the Build Your Own Eval track as a solo hacker 🧵
1
19
3,316
Second finding: models miss what human reviewers look for. Most real bugs are tests that demand something the instruction never asked for. Models hunt instead for ways to cheat the tests. 12 of 52 bugs were missed by every model.
1
3
341
Shaurya Rohatgi retweeted
I'm excited to launch Benchmark Almanac! It tracks 3,322 benchmarks from 1,746 leaderboards, 522 launch posts, 346 system cards, and 1,233 open-source GitHub repos. To my knowledge, it's the most comprehensive repo out there. Website: benchmarkalmanac.com
31
28
8
233
28,918
Congrats to @thinkymachines for the new model. But calling 594B total, 17B active model a small model is wild work.
7
1
89
8,000
Shaurya Rohatgi retweeted
bruh RL data procurement pipelines are so cancerous, can someone just make tinder for RL envs already
12
4
218
10,296
Also closer to how much money is spent on buying data from vendors
Much closer to how these models feel to use
1
1
7
554
This is basically my workflow these days lmao
> Multiple, unrelated actors are independently adopting the same Hermes + DeepSeek combination oh no no no no
1
323
Proud to have been a core part of this release. We built a local search engine to augment data for pre and post-training. Agentic semantic packing was used to create length-controlled long context data. We also create RL environments in-house; we are releasing the skills that we used to create them. We used autoresearch-style looping to create and make tasks harder iteratively. A lot of useful artifacts in this release! I will do a deep dive of each artifact soon.
The most detailed open LLM data release yet: K2 Horizon's pre-training data and post-training data-generation pipelines are now open! 13 trillion pre-training tokens across 5 datasets and 24 subsets. Over 10 trillion tokens of synthetic data spanning mathematics, reasoning, coding, planning, and problem-solving. Beyond the scale, four highlights for model builders (details in the threads): - Scaling synthetic data without losing diversity. Our generation methods produce diverse training examples at scale, nearly matching natural web text in measured diversity. - More verification isn't always better. Strict verification can throw away useful training signal. - Small-scale wins can mislead. Our chosen data mix actually lost in smaller experiments. - Beyond teacher imitation. Our post-training data generation methods produce training trajectories the teacher couldn't generate on its own. Open pre-training datasets, post-training data-generation pipelines, and processing tools for researchers to inspect, reuse, and build on. Pre-training data: huggingface.co/collections/I… Post-training data-generation code & tools: github.com/ifm-ai/DATA_NAVIG…
8
6
1
48
2,261
Shaurya Rohatgi retweeted
We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here! Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify
New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…
38
119
26
1,067
138,508
Shaurya Rohatgi retweeted
ngl RSI startups training okish small/medium LLMs isn’t that impressive, the amount of literature, open source code and data out there for LLM training is huge now. Still hard no doubt, just low signal to me that your thing can do something interesting. show me your autoresearch thing has done something vaguely novel for a less saturated task
10
2
102
7,257
We're open-sourcing RL360, the RL infrastructure behind K2 Horizon's six domain experts: 8,192 concurrent sandboxes, 150 steps in 7 days, ~1.3M rollouts. Long-horizon agentic RL runs thousands of agents in real sandboxes on tasks that take minutes. Some of those sandboxes crash mid-task, and a crash arrives as the same low score as a wrong answer. Without infrastructure that can keep up, GPUs are underutilized and the model gets penalized for system crashes rather than its own mistakes. RL360 is built for this. We share how it works, what broke at scale, and what we learned in the thread. Code: github.com/ifm-ai/RL360 Run: wandb.ai/mbzuai-llm/rl360-pu…
11
48
13
320
50,405
Shaurya Rohatgi retweeted
The @SnorkelAI team did an audit of these findings using our internal QC platform. Tl/dr: We agree with many of the findings, but ~55% of the fixes from @preferencemodel had their own issues with reward hacking or over-prescriptiveness. Additionally: ~40% of the issues found were already tracked in the upstream repo - including 21 tasks with outstanding fix PRs! Building datasets like this - and maintaining ambitious open benchmarks around them - is hard. We believe it requires open, collaborative review with community maintainers and relevant domain experts to get right. @terminalbench was one of the first community efforts to build an open, continuous benchmark, and drive a vision of how OSS contributes to the rapidly advancing frontier of AI. We hope that the community will invest in working collaboratively with benchmark maintainers and task authors to keep this vision alive. --- Some takeaways + details of our preliminary audit: (1) We confirm 100% of the reward hacking / verification gap findings using our QC system. However, we find that 17/44 (39%) of the corrected tasks are just as or more reward hackable. Additionally, we don’t yet see evidence that these verification gaps meaningfully affected benchmark rollouts within CIs, though our analysis here is still ongoing. (2) Our preliminary review raises questions about the underspecification findings for 22/41 (54%) of the flagged tasks, including 7 explicitly marked by task authors as intentional design choices. These cases warrant expert peer review and adjudication: balancing over/under-specification is a critical and tricky design decision - much of human intelligence concerns dealing correctly with reasonable and realistic underspecification - and we believe these corrections made to the tasks are likely too over-specified, reducing realism and difficulty relative to spec. (3) Around 40% of the issues found were already tracked in the upstream repo (including 21 tasks with outstanding fix PRs)! OSS projects require community efforts to correct identified open issues, and ideally audits like these are done collaboratively with task authors and maintainers directly, in the traditional community spirit that has kept OSS alive for decades.
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated. Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
25
61
12
266
42,677
Yes! it is disappointing.
Learning that "RSI" benchmarks are just MLE/post-train style tasks repackaged with a new label was disappointing. We could just take the ~20 ML-adjacent tasks from TB 4 and re-release them as RSI-bench and everyone would clap.
2
7
1,124
K2 looking strong 💪
3
4
1
19
2,292