Researcher @Meta Superintelligence Labs Prev: Undergrad @CILVRatNYU #Robots #ComputerVision #AI
Menlo Park, CA
Joined July 2021
- Tweets74
- Following279
- Followers104
- Likes86
Billy Yan retweeted
Excited to announce Robocap-10k. 10,000 hours of ego with metric actions.
This is the high-fidelity dataset I wished I had in grad school but at industrial scale.
It’s time we make robotics model scaling a science.
Introducing RoboCap-10k. 10,000 hours of open egocentric data with metric hand / head pose on every frame.
Captured on RoboCap across 11 countries and 21 industries.
GroundedAPI produces 3D enrichment giving robot policies high-fidelity action grounding.
with @BitRobotNetwork @huggingface @rerundotio ↓
Billy Yan retweeted
Five human demos. Under five minutes. A robot learns a brand-new task, from recording to deployment.
That's Steertuning from NYU: 71% average success across 12 new tasks, including kitchens in new homes.
Billy Yan retweeted
Even the simplest weight edit, a deletion, can teach an LLM new skills!
Meet Deletion Tuning (#NeurIPS2026): just zeroing <0.1% of weights matches full fine-tuning's accuracy at scale. It's thousands of times smaller and avoids forgetting, a step toward continual learning. 🧵👇
Billy Yan retweeted
Teaching robots new tricks used to take forever. Now it takes under 5 minutes. 🦾⏱️
Meet "Steertuning" from the NYU team, a new method that adapts robot policies to new environments using just 5 human video demos.
Our favorite detail shouted out in the paper? The massive 100+ hour egocentric dataset powering their base model was captured on Aria Gen 2 glasses. 😎🕶️
Check it out and see the bots in action: steertuning.github.io/
Code: github.com/billy-yibo-yan/st…
As we get closer to a world where we live among robots, it’s becoming more and more valuable for robots to learn quickly with and from people.
I’m very excited to share Steertuning, a novel approach that adapts robots to new tasks from 5 human demonstrations in under 5 minutes, from demonstration to deployment.
Website: steertuning.github.io/
Code: github.com/billy-yibo-yan/st…
Billy Yan retweeted
I believe adapting robot policies using just a handful of demonstrations will be crucial for robots and Physical AI to work alongside humans, allowing them to learn new tasks and quickly adapt to variations of familiar ones.
Check out exciting new work led by @BillyYYan and @irmakkguzey !
As we get closer to a world where we live among robots, it’s becoming more and more valuable for robots to learn quickly with and from people.
I’m very excited to share Steertuning, a novel approach that adapts robots to new tasks from 5 human demonstrations in under 5 minutes, from demonstration to deployment.
Website: steertuning.github.io/
Code: github.com/billy-yibo-yan/st…
Billy Yan retweeted
Really impressive demos! have seen how much time and effort went into this over the past months, so happy to see it out. Check out Steertuning!
As we get closer to a world where we live among robots, it’s becoming more and more valuable for robots to learn quickly with and from people.
I’m very excited to share Steertuning, a novel approach that adapts robots to new tasks from 5 human demonstrations in under 5 minutes, from demonstration to deployment.
Website: steertuning.github.io/
Code: github.com/billy-yibo-yan/st…
Billy Yan retweeted
What is going to come next when the small and fast llm is very intelligent?
Our first version of Muse Ember 🔥
Ember aims to maximize intelligence per GW of inference to help bring personal superintelligence to everyone.
We would love all the feedback to keep hill-climbing. Please send it our way. We will keep hill-climbing on efficiency, user experience, and better approach to train with internal frontier!
More to come 🫡
Billy Yan retweeted
As we get closer to a world where we live among robots, it’s becoming more and more valuable for robots to learn quickly with and from people.
I’m very excited to share Steertuning, a novel approach that adapts robots to new tasks from 5 human demonstrations in under 5 minutes, from demonstration to deployment.
Website: steertuning.github.io/
Code: github.com/billy-yibo-yan/st…
Billy Yan retweeted
Robots need to feel where they touch and how forces change. Introducing PanoTac: an open-source, dense 3D tactile sensor for <$10 USD.
Watch as the robot racks a test tube and pulls the stopper: PanoTac tells the tube orientation, and feels when the stopper comes free. 🧵
Billy Yan retweeted
Introducing ID-WM: a hierarchical world model that decouples "where to go" from "how to get there." Finally, we can plan over hundreds to thousands of steps with WMs.
It beats the strongest baseline by 38.5%, single-level WMs by 135%. #NeurIPS2026 🧵
id-wm.github.io
Billy Yan retweeted
After about a year of building and experimenting, I’m excited to share PanoTac, my first research project! 🤖
We built a compliant tactile fingertip that costs under $50 and takes ~30 min to assemble. By combining piezoresistive and magnetic sensing, it captures contact geometry and provides normal and shear feedback in a dense 12×32×3 tactile field.
I’m especially excited about how these sensing modalities complement each other: even simple fusion broadens what either can do alone.
We use touch every day to guide our actions, often without thinking about it. Seeing the robot use tactile feedback to decide how to move and when to change actions has been incredibly rewarding.
This project has also shaped how I think about hardware–learning co-design: what should we build into the hardware to give a policy the information it needs? I see tactile sensor fusion as a promising direction for exploring this connection.💡
Grateful to my coauthors, advisors, and everyone who shared ideas and feedback along the way!
Check out the demos below 👇
panotac.github.io/
❓I keep coming back to this: what tactile representation does manipulation need?
We introduce PanoTac: A dense three-axis Tactile Sensor.
In the demo, the robot reorients and inserts a tube, then wiggles the stopper free while the tube sits loosely in the rack. The right side show the tactile signals that helps robot detect when the stopper releases and when to switch from wiggling to lifting. (1/7)
Billy Yan retweeted
❓I keep coming back to this: what tactile representation does manipulation need?
We introduce PanoTac: A dense three-axis Tactile Sensor.
In the demo, the robot reorients and inserts a tube, then wiggles the stopper free while the tube sits loosely in the rack. The right side show the tactile signals that helps robot detect when the stopper releases and when to switch from wiggling to lifting. (1/7)
Billy Yan retweeted
I believe that we are entering an era of abundant robot data. Therefore, the story of finetuning large pretrained video models as robot world models is the story of the past. We should discover the optimal architecture and training recipe for scaling world model training on robot-native datasets. @_YimingDou and I have been validating this belief at Amazon FAR and have obtained some preliminary success. Proud to share this work with everyone!
Introducing DreamFAR, a world model trained from scratch using only robot data, without internet pretraining. It supports:
🎮 Real-time teleoperation
⏱️ Minutes of long-horizon rollout
🧩 Generalization across 1,000+ tasks
yimingdou.com/DreamFAR
🧵 (1/8)
Billy Yan retweeted
Tactile sensing is one modality that struggles intensely with data scarcity. That's why, I think efforts to unify hardware across the field is very valuable for future research. Very exciting to see this! @arikslep
Billy Yan retweeted
We present OpenTac, an open, full-stack ecosystem for 3D tactile sensing on any robot surface.
Anyone can design and fabricate high-density tactile skins for any part of their robot, then read them out, simulate them, and share data across different sensors.
Billy Yan retweeted
Open-weight. 🇺🇸 Made in the US 🇺🇸. Runs on Mac Studio.
Muse Spark Open is ready for y'all to play with!
Billy Yan retweeted
Full episode dropping soon!
Geeking out with @bhawna_paliwal_ @HarithejaE @willjhliang @notmahi on Do as I Do: Dexterous Manipulation Data from Everyday Human Videos do-as-i-do.com/
Co-hosted by @chris_j_paxton @DJiafei
Billy Yan retweeted
🎉 Excited to share HuRo, accepted to CoRL 2026!
Can robotized human videos provide scalable robot learning supervision?
HuRo turns human videos into robot aligned observations and actions.
In VLA pretraining, OOD completion rises from 34.9% to 72.2%.
🧵 More in replies.
