getting robots to work better @physical_int. incoming assistant prof @nyuniversity
San Diego
Joined September 2014
- Tweets140
- Following344
- Followers1.3K
- Likes886
yisha retweeted
I'm at #CoRL2026! Check out our posters tonight, 5–6:30pm, JW Marriott:
#63 FPO++: Flow Policy Gradients for Robot Control (also an oral this morning by @redstone_hong)
#140 TACTX: Learning Shared Tactile Representations Across Diverse Sensors
#184 PRISM: Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulation @Z1hanW
#189 VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes @wangyenjen @jiaman01
yisha retweeted
Excited to share that our work TactX was accepted to CoRL 2026! 🎉
If you're at CoRL, come check out our poster and chat with us!
📍 Poster #140 | Session 2
🗓️ Tue, Nov 10 | 5:00–6:30 PM
🏨 JW Marriott, Austin
See you there! 🤖
A robot policy trained on one tactile sensor usually breaks the moment you swap the sensor. New hardware → recollect data → retrain.
We built TACTX: train tactile policies on one sensor, deploy zero-shot on a physically different one.
The catch, these sensors don't just look different, they measure contact through totally different physics:
👁️ Daimon — vision-based · 🧲 eFlesh — magnetic · ⚡ FlexiTac — resistive
Same contact but completely different raw signals. 🧵
yisha retweeted
we are presenting pi0.7 tomorrow at CoRL (Oral)!
talk: Monday ~10:50am
poster: Monday 5pm, poster 63
Heading to Austin! I started working on things related to robot learning only around 2 years ago. First time attending CoRL. Excited to meet everyone 🦉
2
55
1,437
I’ve been thinking that with powerful agents now, it’s the best time to scale up design. Amazing work!
1/5 FlexiTac has been accepted to the CoRL 2026 Demo Track, and today we’re releasing FlexiTac-Gen! 🧵
Try it: flexitac.github.io/flexitac-…
Over the past two years, we’ve received many requests to customize FlexiTac for different robots. With help from AI agents, we built a tool that makes it easy to design your own sensor and export the files for fabrication.
A simple piece of tactile hardware to help move your research forward. See you at CoRL!
yisha retweeted
Happy to share Catch It All!
We catch completely unseen, random objects with lightning speed, safe reactions ⚡
Thank you to @arth_shukla, @ncklashansen, @HuazeLiu, @ABuynitsky, @darin_djapri
💻 Code & results available at:
arth-shukla.github.io/catch-…
Really excited to share Catch It All!, my first paper as last author! In this work led by @Kehlani_Fay @arth_shukla we develop a system for catching diverse objects using a single egocentric camera 🧵👇
yisha retweeted
Really excited to share Catch It All!, my first paper as last author! In this work led by @Kehlani_Fay @arth_shukla we develop a system for catching diverse objects using a single egocentric camera 🧵👇
It’s really fun to see how people are using the pi model! Sometimes we are like: wow I can’t believe that’s a potential use case 🤔
Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams.
1
25
903
yisha retweeted
Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams.
What’s the most important aspect that serves a good policy? If you have thoughts, please submit to our workshop!
Robot-learning folks: have you ever found that a seemingly minor system choice beneath the policy—controller tuning, hardware selection, or inference infrastructure—made a much bigger difference to your robot’s performance than an architecture or algorithm tweak? How do you usually isolate and report that effect?
That’s the question behind our Workshop on Everything Beneath the Policy at #CoRL2026.
yisha retweeted
A video from a Pi robot deployed at Dandelion Chocolate, fully autonomous w/ no interventions. 🤖
Deploying robots has taught us surprising lessons about the gap between proof-of-concept (i.e. building one box) and real-world utility (productively building boxes for hours).
I expected that the hard part is building the box, since it’s the most dexterous, but that wasn't what we found. Counterintuitively, the hardest part was reliably stacking the boxes. Our original table-mounted robot had poor visibility of the stack without special separately-mounted cameras. Plus, stacking requires more generalization (each box is placed in a different location), and an imprecisely-placed box can lead the entire stack to collapse many boxes later.
We recently switched this deployment over to a mobile robot, and it's fun to watch the robot being completely self sufficient for multiple hours. 🙂 Data and feedback from real world use-cases like these are quite valuable for the π pre-trained model as we scale!
gripper + 3 finger = 5 finger hand is kind of cute 🥺 congrats on the release! @yifengzhu_ut @zipengfu
Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids.
- learned directly from human manipulation data
- no teleop/robot data
- close to human-level dexterity and efficiency
- multi-robot collab
2
20
1,776
yisha retweeted
My group at Johns Hopkins now has a name and a website
Introducing the Brains, Bots, and Behavior Lab
b3.cs.jhu.edu
We're always looking for creative researchers at all levels to join us!
yisha retweeted
Children learn to walk by falling. Robots fail too—but we usually throw those failures away.
Yet failed rollouts tell us something successful demonstrations cannot: what bad actions actually lead to.
So, can robots learn from their failures too?
Introducing FACT: Failure-Aware Causal Training for World-Action Models.
🔗website: fact-wam.github.io/
📄arxiv: arxiv.org/abs/2608.10232
💻 code: github.com/Bariona/FACT
I’ve been joking with friends that my main job this year has been interviewing 🫠
Now I’m finally excited to share what’s next: I’ll be joining NYU as an Assistant Professor next fall, and before that, I’ll be spending a year at @physical_int !
I feel incredibly fortunate to have spent the past two years at UCSD working on control-hardware co-design.
When I was looking for a postdoc, I told people I wanted to work on robot learning for hardware design. I’m deeply grateful that @xiaolonw and Mike trusted me enough to start something none of us quite knew how to do.
Whenever I mentioned working with both of them, people either knew only one of them (they come from very different communities), or found the combination surprising. I find it to be really fun and I learnt so much more than I would anywhere else.
Although I think jointly optimizing policies and hardware at scale is still somewhat early, I strongly believe in optimizing (or at least iterating) hardware alongside policy training.
Over the past few decades, we’ve developed a lot of depth in individual hardware components: sensors, actuators, mechanisms, materials, etc. But we still haven’t explored enough of their combinations as integrated systems, or understood their actual usefulness for policy.
As coding agents get more and more powerful, learning where and how hardware is bottlenecking robots is really valuable. And the only way to understand this is to scale up and try things :)
I’m getting ready to join the bay area robotics community, but it’s still a bit sad to leave the beautiful San Diego.
I had a lot of fun sampling taco shops on Tuesdays (although I eventually converged to two places on my way home...). I will definitely miss the on campus climbing gym, and the forever-sunny-and-warm outdoor swimming pool next to it 🥺
Yaqi is super kind and sweet 🥹whoever join her lab will have a happy phd life 😉
Very excited to share that I'll be joining UIUC ECE and CS (affiliate) as an Assistant Professor @UofIllinois
I'm recruiting PhD students, interns, and postdocs to build trustworthy AI agents and robots that reason, learn, and grow alongside humans. If you're interested in joining my lab at UIUC, reach out at [email protected]
Many thanks to my advisors and collaborators for all their help and support! @haroldsoh @ Katia_Sycara @RamananDeva @ Alexander_Gray @ David_Hsu @ksmeel @ Mohan_Kankanhalli @joecampb @SimonStepputtis @ Woojun_Kim @yswhynot
Touch has become a key ingredient for dexterous robot manipulation, but which tactile sensor should you actually use?
We're excited to introduce TacO, a benchmark for evaluating tactile sensors across real-world manipulation tasks.
🌮 TacO Benchmark provides:
Cross-modality comparison of different sensors (vision-, magnetic-, acoustic-, and resistive-based tactile sensing)
Standardized imitation learning framework with all kinds of tactile sensors
Open-source code, data, and hardware
Our experiments show that there is no universally best tactile sensor😌(even the expensive ones). The right choice depends on the tasks. We hope TacO helps the community build and evaluate the next generation of tactile sensors and how to use them to improve vision-based robot policies.
If you are interested in the commercial sensors, here are the detailed links!
FSR: a.co/d/05paXSTR or similar
Contact mic: a.co/d/06kEZqlT, sound card: a.co/d/030iYREj (similar ones should work too)
eflesh & flexitac: details in paper, egain: customized
1
341
We also thank again @binghao_huang and @venkyp2000 for donating sensors 😆, and our friends at SNU for making and shipping eGain all the way across the ocean. We won't be able to play with so many sensors without their help!
1
3
842
This is led by @anyazorin , collaborating with Zilin Si, Myungsun Park, @jnsungp , @ABuynitsky , @bhadang_sachin , Taejun Park, Sohee John Yoon, Yong-Lae Park, Oliver Kroemer, Zeynep Temel, Michael T. Tolley, @xiaolonw
1
467