Oliver (@ollychen23) profile picture
Joined May 2026
Photo by Oliver (@ollychen23)Photo by Oliver (@ollychen23)Photo by Oliver (@ollychen23)
Oliver retweeted
The long-awaited Kimi K3.1 is here, and it is #1 on our Vals Index amongst open-weight models (#8 overall)
18
12
5
247
25,333
Anthropic has singlehandedly compressed RSI timelines by about a year.
65
140
20
1,924
89,508
Is context compaction a solved problem? Have not had issues with Codex or Claude for months now
2
57
Oliver retweeted
When will robotics be solved? We put frontier models in real-time MuJoCo sims of a robot arm and a humanoid, where they only receive camera feeds and sensor readings, and respond with motor commands. We tested them on 24 tasks built around spatial reasoning and reaction time.
13
20
8
206
12,129
Software engineers now
İtalya'da el yapımı makarna satan ünlü bir dükkânda çalışan bir kişi, 8 saat boyunca makarnaları kestiği anları paylaştı.
41
379
14
9,840
286,137
Oliver retweeted
Xiaomi open-sourced the RL environments behind MiMo v2.6. In two-thirds of the coding tasks, the answer is still in the task's Git history, and MiMo finds it.
47
80
72
1,225
139,541
RSI in ~270 days so best of luck
A new White House task force will have 120 days to report on the risks of artificial intelligence and come up with decisions on what responsibility the federal government has, @learyreports with the exclusive wsj.com/tech/ai/new-ai-task-… via @WSJ
10
13
2
315
18,033
Massive jump from Google on our RSI-Index. Now #4 and outperforming GPT-6-Astra. Were the internal RSI rumors true 👀? vals.ai/benchmarks/rsi_index
Introducing Gemini 4 Argon – our new frontier model. It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
5
1
27
10,119
Replying to @hopes_revenge
thank you hopes, I've taken note of this.
13
38
2
1,318
57,292
Tons of infra work went into this! Congrats to @hungtran and team for getting this out
Existing coding benchmarks stop at the first working version. We are releasing Vibe Code Bench 1-100 today, to measure what comes next. This benchmark asks if models can handle a large number of modifications to a product, without breaking what already works.
1
76
“What would we even do during an AI slowdown?” Containment. It will take at least a year of dedicated work to harden security to ensure AIs can't self-exfiltrate, and that adversarial nations can't steal the weights of cyber-offensive AIs [1]. Propensities. Capabilities (what an AI can do) are different from propensities (what it tends to do). We can work on improving AI propensities to ensure they have a negligible rate of lying, cheating, and wanton harm. Adversarial robustness. We can also harden AIs against jailbreaks, prompt injection, and backdoors. Obtaining high levels of robustness requires careful, assiduous work, as with autonomous vehicles. Institutional adaptation. We have to greatly increase state capacity to understand and manage AI. Communities also need time to figure out how to handle AI (like AI in education). Civil society also needs to be diversified: nearly all funding for AI safety organizations is directed by the EA/utilitarian network [2]; risk management needs more independent funders, values, and centers of power. Moonshots. We can explore different paradigms for safety: mathematical foundations [3], neuroscience-based interpretability [4], safe-by-design architectures [5], and beyond. AI for good. We can collect targeted post-training data to make AI exceptional at radiology, weather forecasting, agriculture, and so on. Fortunately, we can detect if data or avenues of research actually target beneficial use cases or just secretly push general capabilities [6]. A slowdown means we don't have to bet the species to capture the benefits of AI.
18
25
12
148
11,402
100%, this is going to be one of the most important roles in the world - the people will need to be of incredible integity and technical expertise, and be drawn from a wide enough set of backgrounds that all of society feels confidence in their judgements.
on the idea of evaluators: think it's important that we have a distributed ecosystem of indepedent evaluators. the more eyes and people with distributed skill sets the better. it would be a good idea to fund several efforts on this.
38
35
11
611
63,958
AGI is here but auto-reload still doesn’t work
3
96
Oliver retweeted
Replying to @Polymarket
12
29
1,136
24,690
Oliver retweeted
Today, we're launching Vals Public Sector: independent AI evaluation for government. Recent events have made one thing clear: the government has a stake in understanding how frontier AI models perform on day-to-day work, and the risks they carry. This work is central to who we are. We've spent years building industry-leading benchmarks alongside the top AI labs and enterprise domain experts. Now, we’re bringing that same rigor to support government where AI matters most, from public benefits to national security. We are excited for the work ahead!
6
15
4
66
11,742
Oliver retweeted
We’re releasing our Code Migration benchmark — and we managed to get Fable tested in time Code migration carries real economic weight. COBOL powers banks, payrolls, government services, and underpins nearly 95% of US ATM transactions. The danger with any migration is that a model ships code that looks right but quietly drops essential behaviors We evaluated models in three ways: modern to modern migrations, legacy to modern migrations and on their overall code quality. Each model rebuilds the program in an offline sandbox, then is scored on a hidden behavior test with anti-cheat checks that catch anything wrapping the original, copying reference files, or staying in the source language Fable 5 leads overall at 55%, but costs $115.43 per test, while Opus 4.8 (47%), and GPT 5.5 (45%) cost $30.51 and $6.44, respectively, making GPT 5.5 the most cost efficient model. Kimi K2.6 is the #1 open-weight model (28%) priced at $5.12, ranking above some frontier models
6
16
3
131
17,151