Student Researcher @GoogleDeepMind | PhD student @Penn. Working on robot learning.
Philadelphia, USA
Joined June 2018
- Tweets194
- Following294
- Followers851
- Likes3.7K
Pinned Tweet
Check out my most recent project that I've been working on for more than half a year -- co-led with Tim @TimSong52005757!
We showed a flexible and effective recipe for improving generalist VLAs leveraging non-robotic foundation models! The key is guidance in 3D!
More details @ omniguide.github.io/
VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.
@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.
We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.
This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io/
Paper: arxiv.org/abs/2603.10052
Long Le retweeted
🌟 Excited to share that our paper, RoboReason, has been accepted to EMNLP 2026!
🤖 We study how well Vision-Language Models (VLMs) perform on spatiotemporal ordering tasks that use real-world robotics images.
Our key findings:
🔹 Large VLMs often underperform previously reported results when evaluated on spatiotemporal ordering tasks using real-world robotics images.
🔹 Overlaying specialized computer vision model outputs and increasing reasoning effort generally help improve large VLM performance.
🔹 Finetuning smaller models using GRPO and SFT on examples generated by specialized computer vision models often helps them achieve comparable results to much larger VLMs.
📍 Presenting in Budapest with @utk7arsh at @emnlpmeeting, tomorrow!
🗓️ Poster Session C | 9:00–10:30 AM
#EMNLP2026 #VLMs #LLMs #Robotics #AI
Long Le retweeted
Gemini 4 Argon taught a robot to play cornhole just by watching a video.
It mapped the physics, coded its own simulation, and taught @Apptronik's Apollo humanoid to sink 7 throws in a row.
Long Le retweeted
With the advent of Astra as a policy, we wanted to test it on a difficult dexterous task: tying a shoelace knot.
Given just one video of the human doing the task, Astra figured out how to perform the task on real YAM robots in our custom coding harness. It directly generates end-effector poses and gripper commands, using camera feedback to adjust its actions.
This is an early glimpse into a new era of in-context learning for robotics, where new skills can be acquired at test time.
Code and more demos coming soon!
Super thrilled that our paper has just been accepted to #CoRL2026! 🎉🥳 See you in Texas!
VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.
@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.
We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.
This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io/
Paper: arxiv.org/abs/2603.10052
This continues our PIXIE line on fast 3D physics from pixels (pixie-3d.github.io/) but
- With a flow-matching backbone + generative model loss
- Support for many physics solvers within a shared latent space.
Work with my undergraduate student Qilin Huang, master's student Quynh-Anh Huynh, collaborators, and faculty members: @chenwangcw @MorPhLingXD @burrhhh @EricREaton @LingjieLiu1
Personal news: I’m joining @GoogleDeepMind NYC this summer as a Student Researcher on the Robotics team!
4 years ago, I left Google to start my PhD. Now I get to come back and work on humanoid robot learning.
Full loop closure, as the SLAM nerds would say. NYC folks—say hi!
Long Le retweeted
Really excited to release mjviser, a web-based MuJoCo viewer, powered by Viser. It has almost all the features of the native MuJoCo viewer, but runs in your browser. Load and simulate any MuJoCo model with a single uv command 👇
uvx mjviser <model.xml>
Long Le retweeted
Really exciting work! This is exactly promising for test-time controllability. Agents shall perceive to guide and correct behavior actively.
VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.
@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.
We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.
This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io/
Paper: arxiv.org/abs/2603.10052
Long Le retweeted
Excited about this project: we showed how, for challenging tasks, a VLA can "get by with a little help from its friends"😉 --- powerful perception models that infer geometry and more, by constructing 3D guidance fields for diffusion and flow policies.@LongLeRobot @TimSong52005757
VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.
@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.
We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.
This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io/
Paper: arxiv.org/abs/2603.10052
Long Le retweeted
It was a lot of fun to see this in person. We see so much online, but theres nothing nearly as convincing as just doing a demo, first try, right in front of someone.
And its cool work too, addressing a serious shortcoming of current policies
For this project, we’ve also optimized the speed significantly and shown real-time demos to several visitors coming to Penn including @chris_j_paxton and @DJiafei. Here’s a video of the robot finishing a task in 20 secs autonomously
For this project, we’ve also optimized the speed significantly and shown real-time demos to several visitors coming to Penn including @chris_j_paxton and @DJiafei. Here’s a video of the robot finishing a task in 20 secs autonomously
VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.
@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.
We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.
This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io/
Paper: arxiv.org/abs/2603.10052
Long Le retweeted
Scaling VLAs with more robot data is just not enough.
OmniGuide shows you can fix generalist policies at inference time.
Add guidance fields in 3D space that attract toward goals and repel from obstacles, and steer the policy without retraining.
Long Le retweeted
"Last-mile precision" for VLAs! Check out the cool work led by the amazing Tim @TimSong52005757 and Long @LongLeRobot!
VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.
@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.
We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.
This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io/
Paper: arxiv.org/abs/2603.10052
Long Le retweeted
So honored to have the support of my professors and peers, including the omnipotent Long @LongLeRobot, on my first PhD project. 3D space is the bridge between the task and action space, where the guidance from foundation knowledge flows.
VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.
@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.
We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.
This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io/
Paper: arxiv.org/abs/2603.10052