
Views are my own. researching @Microsoft, @UW, contributed to https://teapaw.org/t.co/VzcJa9Skx3
Seattle, WA
Joined August 2012
- Tweets234
- Following584
- Followers3.1K
- Likes1.5K
Linjie (Lindsey) Li retweeted
Embodied First Exam at #CoRL Late Breaking Results
Agentic Robotics session
Monday 12:00–1:30 PM
#122, Griffin, Level 2, JW Marriott
Welcome to join us!
embodied-first-exam.ai
Introducing Embodied First Exam:
One line of code to run coding agents in 1,090 embodied tasks across 31 simulation suites and 44 robots.
Can your agent go far beyond pick-and-place:
Puzzles with spatial constraints;
Olympic sports with multi-robot collab;
Real-time eval, where the world won't pause;
Dynamic, agile, whole-body control;
AutoResearch with sim2sim
Try your own agent:
harbor run -d embodied-first-exam -a <your_own_harness> -m <your_own_model>
Linjie (Lindsey) Li retweeted
Heading to #CoRL2026 in Austin! 🤖 Sharing three recent works on Robotics and 3D Vision from UW RAIVN Lab @RAIVNLab
🧊 GAM — 3D perception matters for robot learning and robustness. Instead of building on a VLA or WAM backbone, we train policies on WildDet3D, a pretrained 3D detector, achieving stronger robustness to visual changes than most VLA/WAM baselines we evaluated
grounded-action-model.github…
🦾 MolmoAct2 — Connecting embodied reasoning to continuous robot actions for real-world deployment, with open models, data, and training code
allenai.org/blog/molmoact2
📦 Reflex — Visually grounded whole-body control that lets a humanoid catch human-thrown boxes with about one second to react, using onboard RGB-D sensing
toberj.github.io/reflex/
📍 Find us at CoRL — all times Austin local / CST
Nov 9 · MolmoAct2
• Main-conference Spotlight S1: 09:00–09:30 · Hilton Austin
• Poster #28: 17:00–18:30 · JW Marriott, Lone Star, 3rd floor
Nov 12
🏆 MolmoAct2 · TAG/G2A Best Paper Presentation
15:10–15:30 · JW Marriott, Room 208
🎤 Reflex · RoboLetics Presentation
09:50–10:30 · Hilton Austin, Salon D
🎤 Reflex · SimBench2Real Presentation
12:05–12:25 · JW Marriott, Room 212
Grateful to all my collaborators, and looking forward to the discussions in Austin! 👋
#CoRL2026 #RobotLearning #EmbodiedAI #Robotics #3DVision #3DPerception #VisionLanguageAction #RobotFoundationModels #HumanoidRobotics #Sim2Real #PhysicalAI
Linjie (Lindsey) Li retweeted
I've been struggling with what technologies to use and how to bring my ideas together. Everything I tried kept turning into bad design!
This tool has so many great design principles, and the fact that it's open source is just crazy!!!
I've been forking it and adapting it to build the workspace I need for my own research! will share the work soon!
My favorite part is the panels and Tools!! OMG, the design and interactions are mind-blowing! Even the technology choices are excellent!! Great work @leoliuym I got to learn so much from this product design!
Introducing Lattice, a local LaTeX workspace for your research.
A paper is more than a .tex file: the papers you cite, notes, slides, Overleaf co-authors and now an AI agent.
I wanted it all in one app, so I built it.
Free to download, open source:
leo1oel.github.io/lattice
一个专为研究打造的本地LaTeX工作空间:Lattice
对经常写学术论文的人来说,你可能需要用LaTeX编辑器写论文,用Zotero管理参考文献,用Overleaf和导师、合作者协同修改,再打开Claude Code帮忙修改内容、检查代码、或者修复编译错误
文献、笔记、论文、Agent在不同工具里来回切换非常麻烦
Lattice,把论文写作、文献、研究资料、Overleaf协作和现有Coding Agent全部整合进了一个本地科研工作台
它有几个能力:
1、LaTeX写作与PDF预览一体化
本地编辑.tex文件,支持一键编译和PDF并排预览、源码与 PDF双向定位、编译错误诊断,修改效果可以直接检查
2、把参考文献和研究资料放进工作区
支持从arXiv、DOI、BibTeX导入文献,统一管理参考论文、引用、Markdown笔记等资料
3、直接接入Claude Code、Codex等
让Agent读取论文项目和参考文献、修改LaTeX、修复编译错误,并通过Diff检查AI对文件做了哪些改动等
4、支持Overleaf双向同步
自己可以在本地使用Lattice,导师和合作者继续在 Overleaf上协作,无需大家都迁移到同一个工具
5、研究成果和展示材料放在一起
除了论文,还可以在同一个工作区管理研究笔记、白板和演示幻灯片,让写论文、整理资料、准备报告成为连续的工作流程
#Lattice #科研写作AI #本地LaTeX
Everyone’s asking whether AI will replace researchers.
I’m more interested in a different question: what will research look like when every researcher has an AI collaborator?
I’ve been trying Lattice by @leoliuym, a workspace that brings papers, notes, LaTeX, and AI together. As AI evolves, our research workflows should too.
Introducing Lattice, a local LaTeX workspace for your research.
A paper is more than a .tex file: the papers you cite, notes, slides, Overleaf co-authors and now an AI agent.
I wanted it all in one app, so I built it.
Free to download, open source:
leo1oel.github.io/lattice
Introducing Lattice, a local LaTeX workspace for your research.
A paper is more than a .tex file: the papers you cite, notes, slides, Overleaf co-authors and now an AI agent.
I wanted it all in one app, so I built it.
Free to download, open source:
leo1oel.github.io/lattice
Linjie (Lindsey) Li retweeted
Fun fact: RAGEN is the #3 hardest task in FrontierCS 2 🔥
Why so hard?
- Take a 0.5B model.
- Improve it with RL.
- Budget: 0.5 A100 GPU hours.
That is a really high bar. The code was manually written by @wzenus and other students.
Fun to see it up there. 😄
Linjie (Lindsey) Li retweeted
Introducing Microsoft-Decision-1, our new model for fast decision-making.
It delivers top performance on structured decision tasks, outperforming both LLMs and other decision models in latency and quality.
We’re already testing it across Microsoft for everything from incident response and quality control to scientific discovery.
Linjie (Lindsey) Li retweeted
Honored that our RSI-Exam made it into this year's @stateofai 🎉 As AI R&D benchmarks saturate, we need evals with real headroom 📈: 88 RSI tasks 🧪 and the top model (Opus 5.5) still scores ~0.53 🤯 Thanks @nathanbenaich!
welcome to the 9th annual @stateofai report!
ai building better ai, world models, physical ai, geopolitics, cyber defense and 12-month predictions.
read it at stateof(dot)ai or watch my editor’s cut here.
a few of this year’s biggest findings follow:
Linjie (Lindsey) Li retweeted
GPT-6 Astra can solve some seriously tricky spatial constraint tasks: from unlocking interlocked parts to threading a rope through three rings.
Try it yourself now 👇
qinengwang-aiden.github.io/d…
Linjie (Lindsey) Li retweeted
GPT-6 Astra from @OpenAI is here. We asked: how far can it push the frontier of RSI?
On RSI‑Exam, GPT-6 Astra scored 0.5126, taking the #1 spot. That’s 18.4% above GPT-5.6 Sol and 54.8% above GPT-5.5.
RSI just took a generational leap. 🚀
We ran GPT-6 Astra on HumanCLAW-Bench, using the same open-source harness and motion generator.
It is a big step over the previous best:
- FindSR: 64.9% → 75.5% (+10.6)
- NavSR: 42.4% → 57.1% (+14.7)
- InteractSR: 16.8% → 46.6% (+29.8)
The gains compound: Find-to-Nav conversion rises from 65% to 76%, and Nav-to-Sit from 39% to 72%. Astra also solves 147/507 (29%) episodes missed by all nine previous models .
Three things stood out.
1/ Better embodied spatial awareness
GPT-6 Astra's dDtb—how far the objects it disturbs end up moving—is 0.65 m, the lowest on the leaderboard, while its #Dtb, the number of objects it touches, remains in line with the other models. It still bumps into things; it just does not keep shoving them.
2/ Stronger spatial memory
Sitting is the hard case: the agent has to turn away from the target, so the seat leaves its ego view and it must rely on its own history. Earlier models often sat on the wrong object or on thin air. GPT-6 Astra converts 72% of successful approaches into a correct sit, up from 39%. Once it reaches the target, it never forgets to sit (0 of 108 failures).
3/ Better agentic decision continuity
Earlier VLMs often jump between decisions: sit down, stand up again, then wander off. GPT-6 Astra commits. Among episodes where a sit lands on the target, 97% end still seated with a clean stop, compared with 78% for Gemini 3.1. It knows what it is doing and finishes it.
⚠️ Caveats: this is a single run in the low-thinking setting, and 53% of sit episodes still fail.
Embodied self-awareness remains the bottleneck we described in the paper; the ceiling has simply moved up.
Leaderboard: human-claw.github.io
All 1,218 episode videos: huggingface.co/datasets/Choi…
🦾HumanCLAW: Can Vision-Language Models Act Through a Body?
A VLM can spot the sofa in a second. But can it get a body there and sit down?
We call it Action Intelligence, and our work decouples it from motor control so it can finally be measured. Today’s best models turn out to be surprisingly bad at it. 🔻human-claw.github.io/
Linjie (Lindsey) Li retweeted
HumanCLAW: Can VLMs act through a body?
Give VLMs a body, ask it to walk over and sit on the couch.
VLMs always fail, in 1,218 long-horizon tasks across 41 indoor scenes:
walls block...
feet catch on furniture...
objects get knocked aside...
Real robot testing fuse decision and joint control into one system. When they fail, we may not tell what failed. So HumanClaw strips low-level control error out of the result: the body never falls.
In practice: a frozen VLM picks one parametric whole-body skill every 0.5s from egocentric view. A pretrained motion generator turns it into continuous motion.
So here, failure = decision failure.
Where it breaks:
- 1. inefficient exploration: once a target is in view, recognizing it is mostly fine. The more common failure is that the model never brings the target into view at all.
- 2. it sees the couch and still can't get there. The model does not know how far away it is or when it has arrived. It stops early, or walks straight past.
- 3. it is standing in contact with the couch and still does not sit. Sit success ranges from 90% down to 3.5% across models.
- 4. the clearest signal is where the collisions land: mostly on the body parts the model can't see, like legs and feet. VLMs describe a wall correctly, then walk a leg into it.
A clear gap in embodied self-awareness!
Everything is open:
- paper human-claw.github.io
- benchmark huggingface.co/HumanCLAW
- code github.com/Human-CLAW/HumanC…
- leaderboard human-claw.github.io/#leader…
Done by the amazing @siyao94 @Kuvvius 👇
🦾HumanCLAW: Can Vision-Language Models Act Through a Body?
A VLM can spot the sofa in a second. But can it get a body there and sit down?
We call it Action Intelligence, and our work decouples it from motor control so it can finally be measured. Today’s best models turn out to be surprisingly bad at it. 🔻human-claw.github.io/
Linjie (Lindsey) Li retweeted
I couldn't make it to ECCV in person this year, so I teleported my avatar to Malmö instead! 🇸🇪✨
While I'm only there in spirit, you can check out our real-life poster on "imaginative perception tokens" here:
📍 ExHall, Poster #13
⏰ 10:30 AM CEST
📌 Poster Session 3
#ECCV
Linjie (Lindsey) Li retweeted
Can frontier AI agents achieve Recursive Self-Improvement by turning a weak method into one that performs better on hidden data?
Introducing RSI-Exam 🧵: 88 executable research tasks across 6 domains, now open for task contributions from every field🧑🔬.
Its task bank spans virtual cells, TPU kernels, chip design, quantitative finance, agent harnesses, model distillation, and much more.
Across these tasks, every rollout follows the same core protocol:
🔁 An agent inherits a working method and iterates using visible data.
🔒 Only its final artifact enters a fresh-container evaluation on the hidden set.
📊 The result so far: Opus 5 leads the 88-task leaderboard with a mean hidden-set score of 0.464.
📢 Want to help build the next release? Author an executable task, review one, or audit a rollout. Substantial contributions qualify for paper authorship.
Special thanks to @XinyeYee for the tremendous effort in leading this project.
Explore: rsi-exam.ai/
Blog: rsi-exam.ai/blog.html
Contribute: rsi-exam.ai/contribute.html
GitHub: github.com/aiming-lab/RSI-Ex…
Robotics felt different this month. Google DeepMind and Anthropic made the same question hard to ignore: can a reasoning model decide what a whole body/robot should do next?
Gemini Robotics 2 put an embodied reasoning model above a VLA for whole-body control. Anthropic tested general-purpose reasoning models at several levels of control.
- deepmind.google/blog/gemini-…
- anthropic.com/research/claud…
The bad news is clear. General-purpose reasoning models still cannot control a humanoid on their own. They need a pretrained policy underneath.
Still, I think this might actually work. A strong model (w/ excellent multimodal and reasoning capabilities) could really do it one day.
HumanCLAW is our attempt to get an early glimpse of that future.
🦾HumanCLAW: Can Vision-Language Models Act Through a Body?
A VLM can spot the sofa in a second. But can it get a body there and sit down?
We call it Action Intelligence, and our work decouples it from motor control so it can finally be measured. Today’s best models turn out to be surprisingly bad at it. 🔻human-claw.github.io/
🦾HumanCLAW: Can Vision-Language Models Act Through a Body?
A VLM can spot the sofa in a second. But can it get a body there and sit down?
We call it Action Intelligence, and our work decouples it from motor control so it can finally be measured. Today’s best models turn out to be surprisingly bad at it. 🔻human-claw.github.io/
Linjie (Lindsey) Li retweeted
Really cool to see @AnthropicAI benchmarking with MolmoAct 🤖 Wild finding: LLM supervisors actually hurt performance on familiar tasks while MolmoAct solo beats every LLM+VLA combo. But on novel tasks the VLA can’t solve alone, the best models provide net uplift. Judgment > deference.
anthropic.com/research/claud…
Linjie (Lindsey) Li retweeted
we’re excited to learn about your work on multimodal agents and will select one submission for a best paper award 🏆
please submit by the extended deadline Aug 5 (AOE)‼️
📣 Call for Papers — 1st Workshop on Multimodal Digital Agents (MDA) @ ECCV 2026
How should AI agents see and act across web, desktop, and mobile interfaces? We're bringing together work on vision-centric digital agents — perception, reasoning, benchmarks, safety, and more.
🗓️ Sept 8, 2026 (ECCV) · ✍️ Submit by July 29 (AoE)
📄 Full paper (≤14 pages, ECCV-accepted welcome) or extended abstract (≤4 pages) — non-archival, 🏆 best paper award
🎤 Speakers include Graham Neubig, Ranjay Krishna, Qianhui Wu & Alexandre Drouin
Submit / details → mda-workshop.allen.ai/
#ECCV2026 #ComputerVision #AIagents #Multimodal #GUIagents
Linjie (Lindsey) Li retweeted
Excited to share that our paper, Imaginative Perception Tokens, has been accepted to ECCV 2026! 🎉🇸🇪 🌼



























