Assistant Professor@NWU, Amazon Scholar, Postdoc@Stanford, PhD@UIUC #Language+#Vision/#Robotics, #Reasoning, #Planning
Chicago, IL
Joined November 2017
- Tweets939
- Following924
- Followers12.4K
- Likes1.6K
Pinned Tweet
Honored to be named as MIT TR 35 Under 35 @techreview.
Couldn’t have done this without the best PhD advisor @hengjinlp, my academic home @siebelschool, and the most supportive postdoc mentors @jiajunwu_cs @drfeifei @StanfordAILab @StanfordHAI, my forever mentor Shih-Fu Chang, and the most nurturing my new home @northwesterncs and my amazing students!
Reasoning and planning has been entering a new stage. Excited to explore more ahead!
Today in The Download, our daily newsletter: Introducing our 35 Innovators Under 35 list for 2025 trib.al/Yz17AtP
Behavior Uncloning #CoRL2026:
Remove an unwanted behavior from a trained policy by editing its weights.
No retraining from scratch.
No inference-time steering.
📍 #CoRL2026 Poster II, #79
🕔 Tue Nov 10, 5:00–6:30 PM
Interesting findings:
- Edit only before the policy picks a side:
At the start, the left and right paths look the same. A small nudge here can switch the side.
Once it is already curving left, those motions belong to "left", Pulling them toward "right" rarely switches the side and tends to break the motion.
So we edit only the steps where a classifier is <50% sure the policy is going left. Editing every step drops success 80.7% → 72.7%.
- Pushing away from the bad behavior is not enough. You also have to keep the good one:
Push away only: the motion breaks. Task completion 53%, success 27%.
Keep imitating good demos only: the bad side never goes away. Success 49%.
Both: 84%.
So this is how we get it work:
1. Label demos or rollouts by mode (handle-first / blade-first) on mixed knife handovers.
2. Train a small mode classifier on the policy's own features, then freeze it.
3. Fine-tune the policy for <500 steps with two losses:
• Redirect: on unwanted-mode samples, backprop the classifier through the policy so its features move toward the allowed mode.
• Retain: on good samples, keep the usual imitation loss so the skill doesn't degrade.
4. Redirect only on states where the policy hasn't committed to the unwanted mode yet (classifier prob < 0.5). Removing this gate drops success 80.7% → 72.7%.
5. Throw the classifier away. Same model, same latency.
So fun to collaborate with @zhiwen_fan_ and his lab at Texas A&M on this! Led by the amazing @haohw_.👇
Behavior Uncloning is accepted to #CoRL2026! 🎉
Huge thanks to our amazing collaborators and advisors!
Jiuzhou Lei, @EdisonLeeROC, @Neutrino_l, Minghui Zheng, @ManlingLi_, @RuohanZhang76, @zhiwen_fan_
See you in Austin!
📄 arxiv.org/abs/2606.29201
🌐 behavior-uncloning.github.io
Embodied First Exam at #CoRL Late Breaking Results
Agentic Robotics session
Monday 12:00–1:30 PM
#122, Griffin, Level 2, JW Marriott
Welcome to join us!
embodied-first-exam.ai
Introducing Embodied First Exam:
One line of code to run coding agents in 1,090 embodied tasks across 31 simulation suites and 44 robots.
Can your agent go far beyond pick-and-place:
Puzzles with spatial constraints;
Olympic sports with multi-robot collab;
Real-time eval, where the world won't pause;
Dynamic, agile, whole-body control;
AutoResearch with sim2sim
Try your own agent:
harbor run -d embodied-first-exam -a <your_own_harness> -m <your_own_model>
Embodied First Exam at #CoRL Late Breaking Results
Tomorrow 12:00–1:30 PM
Griffin, Level 2, JW Marriott
Welcome to join us!
embodied-first-exam.ai
🤖📜 We will be presenting Embodied First Exam (embodied-first-exam.ai/), a comprehensive evaluation framework for coding agents on robotics tasks, at CoRL Late-Breaking Results at Griffin, Level 2, JW Marriott 12:00–1:30 PM.
Manling Li @ CoRL 2026 retweeted
Looking forward to this distinguished lecture visit at Northwestern (feel free to join if you are in the Chicago area)! Thanks for the invitation Manling, Samir, Han et al. -- and thanks for the kind words Manling, it has been an absolute pleasure to mentor you and to see your amazing success as a faculty member! 🤗
🎉 Welcome @mohitban47 to Northwestern CS for our Distinguished Lecture next Monday!
He is a leading voice in multimodality, and he has been a wonderful mentor to me since my PhD days.
📅planitpurple.northwestern.ed…
🎉 Welcome @mohitban47 to Northwestern CS for our Distinguished Lecture next Monday!
He is a leading voice in multimodality, and he has been a wonderful mentor to me since my PhD days.
📅planitpurple.northwestern.ed…
Manling Li @ CoRL 2026 retweeted
Real time Atlas! Try it now! 🚀
Manling Li @ CoRL 2026 retweeted
⏱️ Introducing Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?
The best model, GPT-6-Astra, solved only 36% of tasks reliably.
Jev achieves sub-1s latency, but its performance remains at 8%.
🔗 Read our full blog: realtimeagents.org/blog/real…
Can coding agents do Scientific Tasks?
We included physical experiments into Embodied First Exam:
- AutoBio
- LabUtopia
- Coming soon: our original Embodied Scientific Exam benchmark. Coding agents run physical/chemistry experiments like in a high school lab. Stay tuned.
Introducing Embodied First Exam:
One line of code to run coding agents in 1,090 embodied tasks across 31 simulation suites and 44 robots.
Can your agent go far beyond pick-and-place:
Puzzles with spatial constraints;
Olympic sports with multi-robot collab;
Real-time eval, where the world won't pause;
Dynamic, agile, whole-body control;
AutoResearch with sim2sim
Try your own agent:
harbor run -d embodied-first-exam -a <your_own_harness> -m <your_own_model>
Game2Player: let AI build its own training ground.
- Builder agents turn web games into tasks. They read the game state, so every step is auto-checked.
- Player agents play them. Some write code. Some just watch the screen.
🔁 When a player finds a broken game, the builder fixes it.
Games get better. Players get better. No humans needed.
The scale:
- 568 games,
- 3,436 tasks,
- 1.56M steps of experience.
We then train a 9B model to learn from it and plays from screenshots alone. Its scores roughly double.
Experience, at scale, without humans.
Led by @RuiYang70669025 👇
Introducing Game2Player 🎮 Scaling gameplay experience for visual agents.
Instead of a handful of games or costly human play data, we use coding agents to turn 568 web games into 3,436 verifiable tasks, then collect 1.56M gameplay steps with both code policies and VLM policies.
🔁 Environments and players improve together: progress is checked from game state at every step, and any problem a player hits goes back to the builder agent for repair.
🧪 We also introduce PlayerBench: 50 held-out games, 287 tasks. Even GPT-6.1 Sol reaches only 62.6% progress.
📈 Our 9B VLM, trained from our gameplay data, plays from pixels alone. It lifts progress from 30.0 to 53.1 on GameWorld (ahead of Qwen3.8-Flash-Next) and more than doubles it on PlayerBench, from 14.3 to 31.2.
🌐 Website: game2player.github.io/
💻 Code: github.com/game2player/Game2…
🤗 Models: huggingface.co/game2player
🚀 More games and results coming soon. Stay tuned!
Manling Li @ CoRL 2026 retweeted
Our latest work in closing the last-mile gap on long horizon robotics tasks.
Long-horizon, whole-body robot policies still fail during deployment. How do we best close this last-mile gap? 🤖
🚨 Introducing I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulation
Website: i-3l.github.io/
Paper: arxiv.org/abs/2610.17748
Manling Li @ CoRL 2026 retweeted
more reasoning doesn’t always mean better agents.
In real-time environments, every extra second of thinking has a cost.
This raises an important question for agent evaluation: should we optimize for reasoning quality alone, or the quality of decisions made under time constraints?
Great work on Realtime-Bench!
Realtime-Bench ⏱️
Key findings:
- Best model (GPT-6-Astra): 36%.
- Fastest model (Jev, answers in under 1s): 8%.
- More reasoning helps only when the clock is paused.
- With a real clock, slow thinking costs the win.
- Even with time frozen, many tasks stay unsolved.
Play with it at realtimeagents.org!
Great work led by @JingyuTang621 @jiaxin_pei and team 👇
Fun fact: RAGEN is the #3 hardest task in FrontierCS 2 🔥
Why so hard?
- Take a 0.5B model.
- Improve it with RL.
- Budget: 0.5 A100 GPU hours.
That is a really high bar. The code was manually written by @wzenus and other students.
Fun to see it up there. 😄
Realtime-Bench ⏱️
Key findings:
- Best model (GPT-6-Astra): 36%.
- Fastest model (Jev, answers in under 1s): 8%.
- More reasoning helps only when the clock is paused.
- With a real clock, slow thinking costs the win.
- Even with time frozen, many tasks stay unsolved.
Play with it at realtimeagents.org!
Great work led by @JingyuTang621 @jiaxin_pei and team 👇
Manling Li @ CoRL 2026 retweeted
my hypothesis for why AI-generated slides for technical presentations are so awful:
people who don't have a lot of experience presenting think that to make a talk you just put all of the stuff you want to talk about on the slides, then you go through each slide and say the stuff the slide says.
how you're actually supposed to prepare a presentation: you decide what to say, write it out as a rough script, then put maybe 3% of the text on the slide, plus a bunch of visual aides (e.g. plots that only include data you're planning to discuss).
AI-generated presentations are mostly the former. they have a ton of stuff the presenter never mentions, and the stuff that is most useful for the presenter is absent or not highlighted. they include enough content for a presentation 10 times longer than the actual presentation is supposed to be.
maybe try doing it the other way: first write the script for what you actually want to say, and make sure you can say all of it in the time you have. then prompt the model to create the slides based on the script. ask it to use as little text a possible, and a minimalist visual style rather than the claude house style. you can thank me later (I accept both venmo and zelle).
