Assistant Professor @Princeton. Developing robots that plan and learn to help people.
Princeton, NJ
Joined October 2011
- Tweets515
- Following387
- Followers3.5K
- Likes779
Adding this to my catalogue of (very) surprising physical strategies discovered by coding agents...
Me: Put the green box on target, clearing obstructions.
Astra: You got it, boss 🫡
(Purely python solution generated in a red teamed sandbox using no any other tools, with $20 budget)
This week's #PaperILike is a break from the usual: "Worms and science," a 2003 interview with Sydney Brenner.
On choosing problems, hypothesis-driven vs data-driven science, and knowing when to move on (in his case, from molecular bio).
PDF: pmc.ncbi.nlm.nih.gov/article…
Tom Silver retweeted
😫Tired of spending hours teleoperating robots?
🚀What if you could collect 2.9× more demonstrations with the same human effort?
Introducing TANDEM: our new work that combines autonomous task and motion planning (TAMP) with human teleoperation only when needed. 🧵 1/n
Check out Samrat’s extension to DATAFARM where there is now a human in the loop to do what TAMP alone cannot!
Robots should spawn MDPs and run RL to learn new skills. But which MDPs, and how can these skills generalize? Check out @Bw_Li1024 's CoRL 2026 paper for answers!
Failures are inevitable for robots that are deployed in the *big* world. We introduce ReSYNC, a system that leverages large-scale RL sim2real to recover from local failures and concept discovery to explain/avoid new failures. Check it out here: jaraxxus-me.github.io/ReSYNC…
Tom Silver retweeted
Failures are inevitable for robots that are deployed in the *big* world. We introduce ReSYNC, a system that leverages large-scale RL sim2real to recover from local failures and concept discovery to explain/avoid new failures. Check it out here: jaraxxus-me.github.io/ReSYNC…
This week's #PaperILike is "Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy" (Allen et al., NeurIPS 2024).
A fresh perspective on the classic RL paradigm. If TD(0) and TD(1) disagree, you need memory!
PDF: arxiv.org/abs/2407.07333
Tom Silver retweeted
Our thread showed EMPIRIC solving test tasks. Here is a complete run, showing how it gets there: it experiments, writes what it learned into its simulator as code, plans in that model, then builds a bridge one block longer than in training.
Agentic real-to-sim is getting increasingly good at rebuilding scenes in a physics simulator. But looking at a scene does not reveal how glue cures, how water heats, or how a fan's wind pushes things. And what if the simulator cannot model these mechanisms at all?
We present EMPIRIC, an agent that learns a residual world model: a physics engine extended with code for the missing mechanisms.
It solves all 25 runs across five simulated domains, where coding-agent baselines solve 14 to 16, and on a real robot it learns wind and domino masses from two gusts.
basisresearch.github.io/empi…
[1/9]
This week's #PaperILike is "Designing Collective Behavior in a Termite-Inspired Robot Construction Team" (Werfel et al., Science 2014).
Robots climbing on a structure as they are building it (12 yrs ago!)
Video: youtube.com/watch?v=LFwk303p…
PDF: ssr.princeton.edu/sites/g/fi…
That's a wrap for the Northeast Robotics Colloquium (NERC) 2026! It was such a blast to bring NERC to Princeton for the first time.
With 300 attendees, this was the largest NERC ever, yet still small by the standards of today's big conferences... (1/n)
@lakshitadodeja closed out the program by announcing that NERC 2027 will be hosted at Brown University! Looking forward to seeing you all there :)
Finally, if you were with us at NERC 2026, we are eager for your feedback! See the form linked at the top of the website: nerc2026.github.io/
Tom Silver retweeted
@yichao_liang 's latest robotics + program synthesis work with @BasisOrg ! Learning programs that describe heat, glue, wind, electrical switches, etc, a step toward robots that go into new environments, and program for themselves a model of how things work.
Agentic real-to-sim is getting increasingly good at rebuilding scenes in a physics simulator. But looking at a scene does not reveal how glue cures, how water heats, or how a fan's wind pushes things. And what if the simulator cannot model these mechanisms at all?
We present EMPIRIC, an agent that learns a residual world model: a physics engine extended with code for the missing mechanisms.
It solves all 25 runs across five simulated domains, where coding-agent baselines solve 14 to 16, and on a real robot it learns wind and domino masses from two gusts.
basisresearch.github.io/empi…
[1/9]
Tom Silver retweeted
Very cool new work; especially cool to see the resulting vla do better than TAMP and the original VLA! I think this overall direction of auto-generating training data for VLAs is very promising.
Also great to see TiPToP being useful for other research (this was our hope when we built it!) 😄
Fine-tuning VLAs on data generated from TAMP seems like an easy win. Turns out it's not that simple! The key issue: distribution mismatch.
@samratsahoo_ has a clever workaround: learn to align TAMP to the pretraining distribution. Works really well.
I also want to give a big shout-out to TiPToP (by @WillShenSaysHi @nishanthkumar23 @MIT_LISLab), the TAMP system we build on. B/c of TiPToP, @samratsahoo_ was able to set up and run TAMP on a new robot on his first day at Princeton. Inconceivable a few years ago!