Robotics PhD Student @ MIT | Minerva Humanoids
Joined November 2015
- Tweets104
- Following355
- Followers700
- Likes1.9K
Pinned Tweet
Introducing Joga, a full-stack humanoid soccer system with active vision.
Agile dribbling. Tight direction changes. Receive → dribble → shoot/pass.
An actuated neck enables ball tracking with onboard vision.
Residual models for perception + actuation close the sim-to-real gap.
Paper coming soon!
Work done in collaboration with @Avi_Narula14 @khai_ngx @venki1899 @JohnMarangola @martinpeticco @pulkitology
Nolan Fey retweeted
Two days in Brussels with @generalcatalyst Institute.
When a drone carrying explosives turned up inside Leipzig/Halle Airport this summer, it shut a major cargo hub for hours. We’re building robots to resolve that in minutes and keep critical infrastructure running.
Nolan Fey retweeted
We found a way to scale humanoid control that reduces GPU-hours by more than 1000×.
Across 129,514 motions, SuperMimic surpasses the state of the art (128 GPUs, 7 days) with 1 GPU in 3 hours.
Project: supermimic.github.io
Demo: supermimic.github.io/demo/
Nolan Fey retweeted
@EkaRobotics has raised a $150M Series A.
Generality v/s Performance is the fundamental tension at the frontier of robotics. Generality without performance is just a prototype. Performance without generality is just a single-purpose tool.
Customers shouldn’t have to choose. Eka breaks the tradeoff. Powered by our Vision-Force-Action (VFA) model, we deliver the performance and versatility customers need to put robots to work every day.
Eka is built on three convictions:
⚡️Force is the language of manipulation.
🚀 Reinforcement learning is how dexterous manipulation scales.
🎂 System 1—fast, reliable physical execution—is the bottleneck to solve.
With this funding, we will scale our model and accelerate customer deployments.
Grateful to be building alongside a dream team and investors who share our ambition.
Nolan Fey retweeted
Actuator networks offer a data-driven alternative to traditional sysid, but until now, they have been difficult to train for humanoids.
In Contact-UAN, @nolan_fey @khai_ngx uncover that the practical bottleneck is state estimation - and that this is solvable with careful design!
(1/9) Same rewards. Less domain randomization. Higher jumps. Stable landings.
Just by improving the actuator model.
Introducing Contact-UAN: a residual world model for humanoids, learned from proprioception. No torque sensing or motion capture.
Project website: contact-uan.csail.mit.edu.
Proprioceptive intelligence is so fun to watch. And the blind policies generalize better OOD than the visual policies!
Every robot already carries contact sensors: its own joints. When a robot touches something, its joints drift slightly off their commands, and the encoders pick it up. Legged robots and humanoids already leverage this signal to walk blind - but is it enough for blind precise assembly?
We find that it is! Our policies can insert pegs, mesh gears, and thread nuts using only the robot's joint positions - no camera, no tactile, and no object poses. And the peg policy works zero-shot in the real world! 🧵
w/ my amazing co-first author @younghyo_park, and advised by @kushalk_ and @pulkitology.
Nolan Fey retweeted
I believe many (including me) have tried to directly apply UAN to their humanoids but it doesn’t work.
Contact-UAN address this via proper sim-real rollout matching with contact. With just 5 minutes of real walking data, we generalize across multiple whole-body control tasks.
(1/9) Same rewards. Less domain randomization. Higher jumps. Stable landings.
Just by improving the actuator model.
Introducing Contact-UAN: a residual world model for humanoids, learned from proprioception. No torque sensing or motion capture.
Project website: contact-uan.csail.mit.edu.
(1/9) Same rewards. Less domain randomization. Higher jumps. Stable landings.
Just by improving the actuator model.
Introducing Contact-UAN: a residual world model for humanoids, learned from proprioception. No torque sensing or motion capture.
Project website: contact-uan.csail.mit.edu.
8/9 We can then use the force-control policy for impedance control: an outer loop turns hand-position error into force commands.
Here we're tracking a circle using onboard sensing.
9/9 Work done with @khai_ngx, along with @iamyashprabhu @gabe_mrgl @pulkitology.
If you're struggling with sim-to-real, please reach out. I'd love to hear what you're running into.
Paper/videos: contact-uan.csail.mit.edu/
Code: github.com/nolie-rolie/conta…
SoftMimic meets general motion tracking! Track any motion while being compliant, and you can even adjust the stiffness as needed for manipulation tasks. Great work, Venki!
As humanoid robots become part of our everyday lives, they need to do more than just perform impressive motions. They should be safe, fun to interact with, and respond naturally to unexpected contact, without sacrificing performance.
Our latest work, Compliant GMT (C-GMT), brings whole-body compliant responses to general motion tracking.
🌐 More details: venki1899.github.io/c-gmt/
Work done at MIT Improbable AI Lab with @michl_wang, Jin, @pulkitology.
Nobody should have to risk their life doing work that a robot could do instead.
With recent progress in humanoid hardware, whole-body control, and real-time teleoperation, that’s finally possible.
Couldn’t be more excited to be building with the team at Minerva 🫡
Very nice work studying manipulation without vision.
Forces are sensed by comparing measured joint positions with target joint positions.
Each contact tells the policy something about the state of the world.
This sets a tough baseline for those adding more sensors.
In my latest PhD paper, we declare WAR on sensor-maxxing.
For the first time, push-resilient humanoid walking, just with joint encoders! No IMU, no F/T sensors.
🥁 Introducing Blind Dexterity 🧵 👇
w/ @OKaidanov @liu_puze @Jan_R_Peters at @DFKI @ias_tudarmstadt
Loved seeing so many strengths across our lab come together here: hardware design for the actuated neck, perception robust enough for outdoor deployment, and sim2real RL to transfer agile behaviors from sim. We all brought different expertise and learned a lot from each other.
Taking humanoid soccer out of the lab: no instrumentation, no controlled environment. Real fields, around people.
Trained in simulation. Deployed in reality.
No post-training. Just pre-training.
Dribbling. Passing. Turning. Scoring goals.
Work led by Avi, Khai, @nolan_fey along with @martinpeticco @JohnMarangola, and Venki.