Maria Trofimova retweeted
🚀 SWE-rebench V2.1 RL is out: 13,337 free RL environments for coding agents (20 languages, Harbor format)!
🔧 Fixed tasks, envs, and graders → fast, reliable rollouts.
Doing RL post-training or research? DM or reply. We can share data and RL envs early for free, so we can make them better for open source!
🙏 Please RT or share with anyone who might need free RL envs!
Maria Trofimova retweeted
LMs can play chess, but they can't really explain their moves well (chess engines can't at all). Our new BinProp scaffold tackles this, and nearly triples LMs’ score on our chess-explanation benchmark. It works on other games too!
Try it on your own games on our site-🧵👇
Maria Trofimova retweeted
1/8 LLM agents can get stuck repeating unsuccessful actions or lose track of the task. Natural language feedback can help, but how do we train a model to give useful advice?
We introduce Caddie: a critic trained with RL from the agent’s final task outcome. A 4B critic is trained alongside a frozen Qwen3-4B agent, but its advice also improves larger models that were never used during training, including Qwen3.8-27B and Kimi K3.
Maria Trofimova retweeted
📟 Meet SWE-rebench-V2: the largest open, multilingual, executable dataset for training code agents!
We at Nebius AI R&D are releasing the biggest open dataset of RL environments for training coding agents. We built an automated pipeline to extract real-world tasks at scale, and now we are sharing everything with the community. This release is designed for large-scale RL training.
What’s inside:
> 32,000+ executable tasks — every task is based on a real-world issue and comes with a pre-built Docker env.
> 20 programming languages — moving beyond Python-only datasets (including less-represented ones like Lua, Clojure, etc.).
> 120,000+ extra tasks derived from real pull requests.
> High quality — tasks are filtered and labeled using an LLM ensemble. They are also enriched with metadata and tested interfaces to ensure solvability.
We are also dropping a technical report with all the details on our extraction pipeline and model evaluations.
📄 Paper and dataset: huggingface.co/papers/2602.2…
👾 Discord (we are online there for any feedback/issues): discord.gg/wXYmWpMu
We are open to research collaborations — feel free to reach out!
🔁 If you find this useful, please help us spread the word by sharing
Maria Trofimova retweeted
🎄 We release 67,074 Qwen3-Coder OpenHands trajectories on SWE-rebench + 2 RFT checkpoints!
RFT with this data on SWE-bench Verified:
Qwen3-30B-Instr: 26% → 50% P@1
Qwen3-235B-Instr: 46% → 62% P@1
agent: @OpenHandsDev
model: @Alibaba_Qwen
📷👇
Maria Trofimova retweeted
An extended writeup of our earlier research blogpost on training critics for SWE agents has been accepted to ICML! Some details below ⬇️
Maria Trofimova retweeted
One of our research interests in Nebius is agentic software engineering. Because of that, we have reviewed LOTS of agent evals on software engineering tasks, and there were issues about these evals that made us unhappy.
Today we are making a step towards fixing some of them ⬇️
Maria Trofimova retweeted
This stack of laminated π cards represents 4 years of my favorite #PiDay tradition: wandering through Moscow technical universities, giving them away freely while saying 'Happy Pi Day!' and reciting 3.14159265358979323846264 from heart. Watching faces light up when fellow math enthusiasts realized the date was priceless. 4000 cards distributed, 1000 still waiting at home. Love you #pi!
Maria Trofimova retweeted
As a follow up to our work on applying search to software engineering agents, today we are releasing datasets of problem instances and agent trajectories. This is the training data we previously used to achieve 40.6% on SWE-bench Verified using open-weight models only!
🧵⬇️
Maria Trofimova retweeted
Can open-weight models match frontier LLM performance on SWE-bench? They can if you equip them with search! We've been studying how guided search can improve SWE agents, and built an SWE-agent-based system that scores 40.6% on SWE-Bench Verified using only open-weight models.
🧵
Maria Trofimova retweeted
Discover how Nebius is advancing AI-driven software engineering with search and learning. Our autonomous agent achieves a 40.6% resolution rate on SWE-bench Verified, using only open-weight models.
Learn more from our research article: eu1.hubs.ly/H0dM3Rr0














