Founder & CEO at Intent Lab. Built caffe, ONNX, PyTorch 1.0. Formerly Google Brain / Meta / Alibaba / Lepton AI / NVIDIA.
Berkeley, CA
Joined April 2009
- Tweets715
- Following386
- Followers21.7K
- Likes577
Our agent fleet just took #1 on ClickBench, 1.2× ahead of the next best engine and 3.1× ahead of the codebase it started from. To our knowledge, the first time an autonomous agent fleet has taken first place on a public systems benchmark where correctness is verified independently.
We stripped out anything that only helped this benchmark - avoiding "AI feature creeps". Reviewer agents stripped out any special snowflake optimizations. ClickBench later added a concurrent QPS measure the fleet never targeted, and our entry is first there too.
Our takeaway: as agents get more capable, the infrastructure around them becomes the main limit on how fast they iterate, and letting the fleet work on its own methodology paid off. We plan to open source the engine. Full write-up: intentlab.ai/blog/an-agent-f…
Autonomous engineering fleet needs strong review, so we built a rigorous benchmark of 284 real-world bugs.
Our review agent caught 93%, beating every baseline at 63% lower cost than the top one.
When bugs get overlooked, nearly 20% of them last over a year.
intentlab.ai/blog/building-r…
Once merged, bugs tend to stay. In our benchmark the median bug took 33 days to fix, nearly 20% lasted over a year, and one went unnoticed for more than 15 years.
As coding agents take on more of the work, verification has to scale with them. That's what we're building at Intent Lab.
Full write-up with the benchmark details: intentlab.ai/blog/building-r…
And learn more about our fleet here: intentlab.ai/blog/turn-your-…
TLA+ is also (one of the) method that we at Intent Lab used to build mission critical infra software like Agent FS. Key is to make it verifiable and scalable - going through millions of states and also ensure the TLA+ to Rust/C conversion doesn’t weaken the end to end correctness. intentlab.ai/blog/turn-your-…
I used Opus 5.5 to formally verify the Claude Agent SDK using Lean. A couple short prompts = 16 PRs fixing various bugs and race conditions. Video attached.
TLA+ also works well. I sometimes combine Lean and TLA+ to look for issues around data flow, concurrency, and state mgmt.
I don't know either language well, but Claude is excellent at both. This approach is super useful for formally modeling your code and finding bugs that a human probably wouldn't have spotted.
Is formal verification the future of coding (or at least, bug finding)?
Excited to see the official launch of 224 Ventures! Also @OriolVinyalsML launching his new startup on the same day - the AI world is never short of excitements.
My 2 cents: one should build the right harness. If a harness dictates how the model works, it fights the model and you get brittleness. If it defines what the task is and checks the result, it enables long horizon task success. That's roughly how we got complex tasks done e2e at IntentLab.
We are starting Intent Lab, building an autonomous team we call "fleet" that turns intent into production software. Today we are sharing some early results: the fastest GLM5.2 inference engine, one shot database creation, and a fully verified agent filesystem.
intentlab.ai/blog/turn-your-…
We've spent our careers building large systems by hand, from open source to cloud infra, to our last startup. The theme was careful architecture, done one system at a time. What interests us now is building the oracle that produces a thousand of them.