Pinned Tweet
Excited to share Do as I Do! We turn everyday human videos into physically consistent robot data that can be directly executed in the real world.
This was a fun collaboration with @bhawna_paliwal_ and @willjhliang, with lots of moving parts. More details in Mahi's thread below👇
Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data?
Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵
Haritheja retweeted
As we get closer to a world where we live among robots, it’s becoming more and more valuable for robots to learn quickly with and from people.
I’m very excited to share Steertuning, a novel approach that adapts robots to new tasks from 5 human demonstrations in under 5 minutes, from demonstration to deployment.
Website: steertuning.github.io/
Code: github.com/billy-yibo-yan/st…
Haritheja retweeted
In behavior cloning, we mastered how to learn robotics from "robot-complete" data – but what happens to data that is not so perfect? Intelligence is often about learning from incomplete references; to come up with a "story" given disparate datapoints. Our Pick Anything demo shows an ability of this sort: the policy learned human-like dexterity from absolutely no human-robot interactions, no teleop or handheld tool data, just letting humans be humans and robots be robots. How did we do this?
Learning from humans without actions is often reduced to a semantics problem, a fancy way to prompt the robot. What, if anything, can robots learn about the dynamics from them? RL for manipulation has the opposite problem – often we are stuck with "technically correct" solution that are nevertheless aesthetically unsatisfying. In Pick Anything, we found a way to unite the two in a way for dexterous manipulation that I personally find quite beautiful (and I can't wait to share the paper with you!) I believe what we have here is the key to human level dexterity for robots, and all of that without needing any real robot data whatsoever.
For this Monday/Tuesday, you can visit our demo booth at CoRL (Griffin hall, demo 12) and ask me more questions! It's a labor of love from my past year of work, and I'm very excited to be finally able to talk about it.
Haritheja retweeted
Robots that learn should generalize. Locomotion should show generalization across terrain; manipulation across object categories. Here is a video of “Pick Anything”, demonstrating human like grasps for 66 different objects, ranging in size and shape from air pods to dumbbells to pipe wrenches, using a *single* policy. The system uses the stereo camera head for vision (no wrist cameras) & Sharpa Wave hands for manipulation. It was trained using video and RL (zero teleoperation data). This is research from our Dex-Hands team at Amazon FAR and we will be doing live demos of the system at CoRL 2026 ( Demo 12, Nov. 9 &10). Please come by!
Haritheja retweeted
Back in ~Dec 2025, I really wanted to work on mobile manipulation, but robots were either too expensive or not performant enough.
So we built our own!
Excited to finally share something I’ve had so much fun working on:
1. Pixel: a co-designed full stack system for learning mobile manipulation.
2. Pixel-UMI: environment aware whole body manipulation without robot data.
w/ @JasonJZLiu
(1/N)
Haritheja retweeted
❓I keep coming back to this: what tactile representation does manipulation need?
We introduce PanoTac: A dense three-axis Tactile Sensor.
In the demo, the robot reorients and inserts a tube, then wiggles the stopper free while the tube sits loosely in the rack. The right side show the tactile signals that helps robot detect when the stopper releases and when to switch from wiggling to lifting. (1/7)
Haritheja retweeted
Almost all bimanual robots include wrist cameras. They add bulk, cost, and complexity, but are critical for performance. Are they really needed?
Excited to finally share EyeRobot 2.0, where we use active gaze from an ego stereo camera instead 🧵 (with @kushtimusprime)
Haritheja retweeted
Humans generalize from rich interaction experience. We bring this to humanoid loco-manipulation through scalable real-to-sim-to-real.
Introducing PRISM, we expand 4 real videos into 256 counterfactual variants, recover robot-object trajectories, and train one policy that generalizes across diverse objects, scales, and spatial layouts.
CoRL’26 🚀 Website for details 👉 prism-real2sim2real.github.i…
Work done at Amazon FAR w/ @zhenkirito123 @pabbeel @rocky_duan @JitendraMalikCV @carlo_sferrazza @ckarenliu @GuanyaShi @akanazawa
Haritheja retweeted
One human demonstration. Any multi-fingered hand. Zero-shot sim-to-real visuomotor policy.
morphometricimitation.github…
Collaborators: @he_siming @ckwolfeofficial @HaozhiQ @LeaMue27 Shankar Sastry, Claire Tomlin, @JitendraMalikCV
Haritheja retweeted
Achieving human-like dexterity is the next frontier for robotics, and yet dexterity data is often subtly hard to scale. Real-world dexterity data, including things like finger-pose estimates, is often slightly off, making it physically invalid and hard to execute on real hardware and hard to learn from.
DO AS I DO is an algorithm for reconstructing and retargeting monocular RGB videos to robot hands, outperforming the state of the art and even working from generated videos. @bhawna_paliwal_ @HarithejaE @willjhliang and @notmahi join us to tell us more.
Watch Episode 107 of RoboPapers, with @chris_j_paxton and @DJiafei, now!
Haritheja retweeted
Full episode dropping soon!
Geeking out with @bhawna_paliwal_ @HarithejaE @willjhliang @notmahi on Do as I Do: Dexterous Manipulation Data from Everyday Human Videos do-as-i-do.com/
Co-hosted by @chris_j_paxton @DJiafei
Haritheja retweeted
Grounded API is live.
- SOTA on hand-tracking benchmarks (< 1 cm)
- SOTA on SLAM benchmarks
- In-the-wild ego data -> enriched data in minutes
- Integration with @huggingface @LeRobotHF & @rerundotio
- Built for @BitRobotNetwork RoboCap suite
Technical report & more↓
Haritheja retweeted
I didn't get what GPT-6 meant for robotics until I actually tried it.
GPT-6 Astra just does physical ICL out of the box.
we drop a recording of a human doing a novel task into 𝗰𝗼𝗱𝗲𝘅 app. Prompt it to drive a robot arm the same way.
It just works on the first pass!
Haritheja retweeted
Meet FetchMan: a vision-based humanoid policy trained entirely in simulation that transfers zero-shot to diverse real-world scenes and objects.
Simulation has produced impressive locomotion policies that transfer to the real world. We wanted to see how far the same recipe goes for vision-based loco-manipulation. More below🧵
Haritheja retweeted
Excited to share SPD: simulation pre-training for dexterity. We pre-trained a policy in simulation and fine-tuned with less than 2 hours of real data (with @sarthakkamat)
Shout-out to Kevin for carrying on the lab tradition of live demos!! 🦾🦾
We flew our robot to Seattle for the @meta_aria Summit to prove HUG really works. Thanks to everyone who brought objects!
Ft @BillyYYan building YOR; PC @irmakkguzey @mangahomanga
🌐 HUG: grasping.io
💻 Code: github.com/KevinyWu/hug
🤖 Robot: yourownrobot.ai/
Haritheja retweeted
Your policy doesn't need 7B params. It simply needs dense features.
Introducing Patch Policy: pretrained ViT + small transformer beats OpenVLA-OFT with 0.7% of its params, and trains on a 5090.
Here it inserts a cable (~2mm tol), and does it again as we unplug mid-rollout. 🧵
Haritheja retweeted
Pretrained ViTs see the world in rich, dense detail. Most policies pool it to a single vector before acting, discarding most of it.
We introduce Patch Policy: a minimal architectural extension that enables transformer-based policies to consume dense tokens directly, no billion-param VLM required. It outperforms a fine-tuned 7B VLA by 18% with ~0.7% of its parameters, enabling robust, precise manipulation.
Haritheja retweeted
I'm at #RSS2026 – presenting MolmoSpaces on Tuesday & CAP (Contact Anchored Policies) on Wednesday.
I've been thinking a lot about what robotics 2-5 years from now looks like: beyond teleop and position control. If you're interested about anything from robot free/human data or sim evals to force/torque controlled dexterous hands, let's chat! 🧵
Haritheja retweeted
How can generalist policies adapt to new challenges at deployment using skills they already have?
We optimize VLA *prompt inputs* with reinforcement learning, enabling efficient real-robot adaptation on complex tasks where existing methods struggle. 🧵
semantic-action-rl.github.io
