
Rating systems and paired comparison experimentation enjoyer @arena
Washington
Joined March 2016
- Tweets4.6K
- Following3.3K
- Followers1.6K
- Likes24.6K
Gestalt = BRAMPTON
teapaw.org/newsystems_/status/190…
Breaking🚨: There is a new model "Gestalt" that is beating everything, including the human golden designs 👀
Readers added context they thought people might want to know
This announcement comes from the creator of the referenced benchmark at Contra Labs. "Gestalt (stealth)" is an anonymous model included in their own leaderboard, with no independent verification or public details available.
contralabs.com/frontierdesign
contralabs.com/research/front…
171
one of my favorite posts in twitter, twita!
This Week in the Arena: We launched the Agent Arena Index, where Gemini 4 Neon landed #2, and GPT-6.2 Astra and Sol landed #6 and #11!
- Gemini 4 Neon (High) by @GoogleDeepMind debuted at #2 with 1336 points on the new Agent Arena Index! More info on this launch in thread.
- GPT-6.2 Astra (Max) by @OpenAI entered at #6 with 1291 points, while GPT-6.2 Sol (Max) entered at #11 with 1241. Sol joined the cost frontier at a $0.90 median cost per task.
- Qwen4-Max (xHigh) by @Alibaba_Qwen debuted at #14 overall, and as the #1 Chinese model.
- MAI-Image-2.7 by @MicrosoftAI left preview and landed #3 in Multi-Image Edit, and joined the cost frontier at $0.035 per image.
- Amazon-Web-Search-Agent with AgentCore Web Search by @amazon took #1 in Search Arena.
- Muse Ember (Max) by @AIatMeta scored 1608 pts in Code Arena: WebDev and landed on the Pareto frontier at about $0.03 median cost per task, up from the previous best of 1438 pts at a similar price.
Watch this week’s recap!
youtube.com/live/qQUc_r5f45Y
7
357
Today, we’re excited to introduce the Agent Arena Index, our new headline measure of overall agent performance, alongside six new focused indexes: Agentic Work, Coding, Chat, Task Outcomes, Tool Use, and Steering.
Together with the Alignment Index, these indexes provide a comprehensive view of how agents perform across real-world tasks, from completing complex workflows to using tools and adapting to user feedback.
Arena measures agentic performance from millions of real-world tasks by our global community. With the launch of the new indexes, we are updating our scores to provide a more stable and interpretable way to understand how agents perform in the real world.
Claude Fable 5.2 (Max) leads the Agent Arena Index by 134 points with a score of 1463. Top 5 labs on the Agent Arena Index:
- #1 @AnthropicAI: With Claude Fable 5.2 (Max) at 1463 pts
- #2 @GoogleDeepMind: With Gemini 4 Carbon (High) at 1307 pts
- #3 @OpenAI: With GPT-6 Astra (Max) at 1274 pts
- #4 @AIatMeta: With Muse Wonder Full (Max) at 1200 pts
- #5 @Alibaba_Qwen: With Qwen4-Max (xHigh) at 1191 pts
The Agent Arena Index combines four key signals of real-world agent performance:
- Task Completion: explicit confirmation of task success from users
- Praise vs. Complaint: user satisfaction with agent's outcome
- Bash Reliability: how well an agent uses terminal tools
- Steerability: how well an agent responds to user corrections
Alongside capability, the Alignment Index measures how faithfully agents act in accordance with user intent, based on three key signals: Deceptive Completion, Unauthorized Action, and False Attribution.
As agents take on increasingly complex, long-horizon work, measuring their real-world capabilities and alignment is more important than ever.
Explore the rankings and see what each index measures in the thread below.
BREAKING: Robert Pattinson takes rank 1 on the Agent Arena leaderboard with a net improvement score of 67.42%!
interesting example on the 6.2 launch blog, it's a matter of personal preference but I like the left a lot more
1
4
542
My tuned Glicko model has Gen G over BLG by a razor thin margin of 50.2% to 49.8%. I'm not sure it's actually that close but hopefully there's some good games!
1
2
171
Clayton Thorrez retweeted
Super nice visual from @DawidGalarowicz
Beating April’s best coding model cost 20x less by October.
We analyzed every model listed on Code Arena: WebDev from December 2025 to October 23, 2026: 105 models from 24 labs across 44 weekly leaderboards, plus what every answer cost from April to October.
The pace of releases has stepped up. Last winter, about five new models entered Code Arena each month. From July to October, it was 12 to 14, almost one every other day. And from May to October, the top score climbed 3x faster than it did over the winter.
Even though the labs are racing, it’s the users who are winning. By October, Xiaomi’s MiMo-V2.6-Flash outscored Claude Opus 4.7, April’s top model, at less than a twentieth of the price. And for answers costing between 5 cents and $1, October’s best model had an estimated 65-80% win rate against April’s best at the same price.
Read more about how the AI race is speeding up, and what it means for the field, in the article by @DawidGalarowicz below.
Clayton Thorrez retweeted
Game and application development is getting really good. At the same time computer-use agents are finally starting to work. I don't think this is a coincidence.
Improved computer-use enables the ability to interact then judge applications which then improves generation as well.
Check out my new work Interaction-as-a-Judge for a study on this paradigm!
Agents are demonstrating increasingly strong capabilities in recreating games and applications. One likely reason is their growing ability to directly interact with these applications.
We studied Interaction-as-a-Judge, a growing evaluation paradigm where computer-use agents directly interact with games and applications to improve evaluation.
Interaction-as-a-Judge improves:
- alignment with user preferences.
- a model’s ability to detect errors in games.
- downstream generation. We see improvements in inference-time scaling and the ability for interaction-only reinforcement learning.
It also scales with computer-use capabilities; the better the computer-use model and method, the better the judgments.
We’re also open-sourcing two agent trace datasets, ArenaWebdevPreferences and ArenaWebdevCorrectness, to help support future research. Dive into the thread to learn more.
damn, now not only does Fable 5.2 have a strong idea of who I am, it has a reverse index and will pull my name out when talking about my projects
this is extremely niche too, this dataset has 4 likes on huggingface and fable-5.2 has memorized every detail about it
every generation of models has more and more info about me in the weights, now opus 5.5 knows my employer, names of specific projects and repos I worked on
2
9
1,336
Clayton Thorrez retweeted
Continuous, evolving stream of novel tasks is core to the design of the hillclimbing apparatus, too, for this reason -- should ack that it's not fixed, so how you interpret GH's law changes
excellent summary of the state of public benchmarks🤣
1
317



























