
Pinned Tweet
Locus sets SOTA on automated post-training capabilities and produces a model that's in production facing millions of users, completely end-to-end!
Here's a glimpse to the endless possibilities we're materializing with our research on automated scientific discovery:
The models are improving the models.
Locus, our automated AI research system, is SOTA on PostTrainBench and post-trains Qwen3 base models that surpass the human post-trained Qwen3 model.
Today, LLMs post-trained end-to-end by Locus are in production to millions. 🧵👇
PostTrainBench evaluates agents' ability to post-train models on various domains given 10 H100 hours.
We extend PostTrainBench via PostTrainBench+, which has a greatly expanded compute budget that provides clearer signal on automated post-training capabilities. We find that thousands of H100 hours help distinguish methods' performance post-training Qwen3 1.7B-Base models, and that Locus scales best. In this setting, modes trained by Locus collectively surpass the perforamce of the offical human post-trained Qwen3 1.7B model.
In a test of generalization, we ran Locus on all live Kaggle competitions with prize money and public leaderboards. After 16 days, Locus achieved the 4th highest average rank among all participants.
1
18
1,361
Justin retweeted
by far the most important result on AI for research this year!
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
Justin retweeted
I recently joined @intology because it’s clear that autonomous discovery systems can now accelerate progress on hard research problems, and I’m excited to scale Locus to solve more complex tasks, both in the sciences and beyond!
Justin retweeted
After 4+ years building models at @AmazonScience (Alexa + AWS AI), I joined @intology a few months ago!
We’re automating AI R&D. Here are our results showing Locus building scientific AI models that beat SOTA expert solutions in molecular design
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
We went from SOTA on autonomous post-training to frontier performance on autonomous molecular design (eg antibody design) in just a couple of months!
What's exciting about these results is that Locus achieved them via improvements in domain-agnostic auto-research capabilities:
- Diverse exploration
- Better Generalization to avoid overfitting
- Resilience to noise
- High sample-efficiency to learn from data-scarce tasks
This marks a crucial milestone towards building auto-research systems that can closing the loop for scientific challenges that have even larger search spaces and longer, more expensive, and noisier evaluation cycles.
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
Justin retweeted
🚨New work: A SOTA prompt optimization framework + a novel unsupervised measure of prompt performance.
Written rubrics and constitutions are crucially important for AI task improvement, safety following, pluralism and adaptation.
But some of the things we might want a rubric to do are difficult to put into words; how would you even describe “comic voice”? I’m human, albeit not that funny, and I'm totally not sure what that means.
What kinds of written constructs can you expect your judge to do well on and how good can you expect it to be?
Justin retweeted
So excited for this research! I've always been annoyed at the supremacy of verified rewards, and now we can finally show it.
In this work, we develop ways of measuring: how much of a reward signal is verifiable, how much can we put into words, and how much is shared but tacit, or impossible to articulate?
Justin retweeted
Introducing GameBoyWorlds, a new challenge testbed for self-improvement in embodied video games:
Paper: arxiv.org/abs/2609.32093
Website: dhananjayashok.github.io/Gam…
Thank you @jackclarkSF for covering our work with Locus!
"AI startup Intology, whose goal “is to automate R&D”, has released a new version of Locus, software it has developed to turn LLMs into capable researchers. The new version of Locus is able to get a score of 44.7% on PostTrainBench, a benchmark which sees how well AI systems can take an open weight model and improve its performance above its baseline."
"Posts like this highlight how we are under-eliciting today’s AI systems for their ability to automate AI R&D - especially striking is how the company can jump the performance of Opus 5 by 10 absolute percentage points"
Thanks to @jackclarkSF for covering @intology's work on Locus!
"Posts like this highlight how we are under-eliciting today’s AI systems for their ability to automate AI R&D - especially striking is how the company can jump the performance of Opus 5 by 10 absolute percentage points"
Full blog: importai.substack.com/p/impo…
Thanks to @jackclarkSF for covering @intology's work on Locus!
"Posts like this highlight how we are under-eliciting today’s AI systems for their ability to automate AI R&D - especially striking is how the company can jump the performance of Opus 5 by 10 absolute percentage points"
Full blog: importai.substack.com/p/impo…
Justin retweeted
Super cool results on PostTrainBench!
They also introduce a very interesting setting of PTB where the agent hours are extended by a large margin (4500 GPU hrs!)
The models are improving the models.
Locus, our automated AI research system, is SOTA on PostTrainBench and post-trains Qwen3 base models that surpass the human post-trained Qwen3 model.
Today, LLMs post-trained end-to-end by Locus are in production to millions. 🧵👇
PostTrainBench evaluates agents' ability to post-train models on various domains given 10 H100 hours.
We extend PostTrainBench via PostTrainBench+, which has a greatly expanded compute budget that provides clearer signal on automated post-training capabilities. We find that thousands of H100 hours help distinguish methods' performance post-training Qwen3 1.7B-Base models, and that Locus scales best. In this setting, modes trained by Locus collectively surpass the perforamce of the offical human post-trained Qwen3 1.7B model.
In a test of generalization, we ran Locus on all live Kaggle competitions with prize money and public leaderboards. After 16 days, Locus achieved the 4th highest average rank among all participants.
Justin retweeted
i’m increasingly convinced that the best agent evals will come from mining real agent failure traces. my view is that every failed trace contains a potential eval but not in its raw form. raw traces are messy, long and too specific. the research problem is to distill them into clean reproducible tests. the pipeline i’m interested in is (which i'm currently working on):
failure trace → failure attribution → earliest divergence point → minimal reproducible state → targeted eval → regression suite
this turns trace data from passive observability into an active improvement loop. like can we extract the exact decision point where the agent should have behaved differently? and can we convert that into an eval that catches the same failure class in the future? i guess this matters because most agent failures are trajectory-level failures and not just output-level failures.
personally i think this is much more realistic than relying only on hand-written benchmarks (imo they should look more like failure memory systems). hand-written evals encode what we think agents will fail on. traces encode what agents actually failed on. also once you have the mechanism, you can mutate the trace into variants. that is basically fuzzing for agents.
True progress is built on rigorous evals
Contrary to hype, coding agents make little scientific progress when
* tasked with long-horizon research
* controlled for data contamination
* allowed no human interventions
NanoGPT-Bench paves an exciting path towards scientific agents
Can coding agents do research?
We release NanoGPT-Bench, an internal eval we’ve used to test agents on an AI R&D problem with months of human progress
Codex, Claude Code, Autoresearch recover only 9.3% of human progress, mostly tuning hyperparams & ignoring algorithmic research
NanoGPT-Bench is built on the NanoGPT Speedrun, a popular LLM pretraining competition to minimize the training time of a GPT-2 style model. Existing human submissions constitute nearly 2 years of work. To control for dependencies and contamination in frontier models, we standardize evaluation to a 5-month window of world records. Evaluation is fully autonomous and end-to-end, with no human intervention or internet access. 🧵
1
11
1,669
I've joined @intology!
I'm excited to push the boundaries of AI-accelerated scientific discovery with an incredibly driven and talented team. Looking forward to dive deep into research on AI-driven automation and creativity!
Justin retweeted
New NanoGPT Speedrun WR at 105.9s (-1.0s) from @soren_dunn_ , with a triton kernel to fuse the logit softcap and multi-token prediction cross entropy calc. Interestingly, Soren mentioned that their autonomous system Locus at Intology discovered and implemented the improvement. github.com/KellerJordan/modd…
Can’t recommend working with Alex enough! It’s hard to think of a positive trait that doesn’t apply to Alex, he’s creative, caring, passionate, and super sharp. You shouldn’t miss out on this opportunitiy if ur planning on doing a PhD
✨ Very overdue update:
I'll be starting as an Assistant Professor in CS at University of Minnesota, Twin Cities, Fall 2026. I will be recruiting PhD students!!
Please help me spread the word! [Thread] 1/n
1
7
1,109
Justin retweeted
No better time to learn about that #AI thing everyone's talking about...
📢 I'm recruiting PhD students in Computer Science or Information Science @Cornell_Bowers!
If you're interested, apply to either department (yes, either program!) and list me as a potential advisor!




















