Ian Wu retweeted
Computer-use evals like OSWorld still don’t really test personal assistant use cases: logged-in accounts, user data, personalized workflows, or realistic desktop/web environments.
so we made MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents, with 184 tasks across 17 popular website clones, seeded with realistic user data and centered on Michael Scott’s hypothetical desktop.
It’s easy to adopt if you already use OSWorld-style runners - I view it as an personalization-focused, more realistic upgrade for CUA evals.
Website: mypcbench.com/
Paper: arxiv.org/abs/2606.16748
Code: github.com/ljang0/MyPCBench
Ian Wu retweeted
🚨🚨New Paper: Generative Insight Anticipation from Scientific Literature
Human scientists achieve breakthroughs by "standing on the shoulders of giants," synthesizing profound insights from disparate sources. While LMs show promise in scientific discovery, they currently struggle to reliably generate ideas of true impact, diversity, and feasibility.
Introducing "insight anticipation," a novel task challenging LLMs to generate a downstream paper's core scientific contribution directly from its foundational prior works. We also introduce GIANTS-4B, an RL-trained model that significantly outperforms frontier models at this task! 🧵⬇️
Ian Wu retweeted
😈 Today, Microsoft open-sources WebGym: the task set, code, a bunch of visualization tools, and guiding documentations. WebGym is an RL environment with the *first* open-source implementation of the fully asynchronous rollout system designed for multi-step vision-supported web agentic trajectory collection, which speeds up *4x-5x* compared to existing synchronous implementations. This release comes with *300k* realistic web agentic tasks with comprehensive evaluation rubrics and pipeline, together with annotations on difficulty and domains.
🧵 1/6
We post-trained a 4B parameter model to write Olympiad-level math proofs!
Our tiny model, *QED-Nano*, outperforms gpt-oss-120b, and approaches the performance of Gemini 3 Pro when paired with test-time scaffolding.
Check out the blogpost: huggingface.co/spaces/lm-pro…
6/ Our code, datasets, models etc. are all publicly available!
github.com/CMU-AIRe/QED-Nano
huggingface.co/lm-provers/QE…
A huge thank you to my amazing collaborators from CMU @aviral_kumar2 @QuYuxiao @setlur_amrith, Hugging Face @edwardbeeching @_lewtun, ETH Zurich @j_dekoninck and Project Numina @JiaLi52524397.
1
5
152
12/Check out the paper here: arxiv.org/pdf/2602.03773
A huge thank you to my fantastic collaborators @aviral_kumar2 @QuYuxiao @setlur_amrith, and a great start to my PhD!