Varun Bhatt retweeted
We asked a VLA-powered robot to “open top drawer” and “could you open the top drawer, please?”: identical commands, yet yields a 60% performance difference 🤯🤯
How can we scalably red-team robots with such diverse and realistic user instructions? 🤔
Learn more! 🧵 [1/11]
We show how LLM prompts can be algorithmically generated for diverse human-like coordination and communication. Super excited about its implications for scenario generation and training better embodied assistants in the future!
Varun Bhatt retweeted
One of the biggest challenges in designing agents/robots that collaborate effectively with humans is the lack of diverse human data—large scale collection is hard!
Introducing PLAN-QD: a framework that algorithmically generates diverse, human-like agents.
👇 Learn more! [1/9]
ALT PLAN-QD Main Figure
Varun Bhatt retweeted
Ecstatic to announce the release of @pyribs 0.7.1! The absolute highlight of this release is the QDHF (Quality Diversity through Human Feedback) tutorial contributed by @li_ding_! The tutorial is available here and runs on Google Colab in ~1 hour:
docs.pyribs.org/en/stable/tu…
ALT Images generated by QDHF for the prompt "an astronaut riding a horse on mars"
Varun Bhatt retweeted
Excited to announce the release of @pyribs v0.7.0! Pyribs v0.7.0 greatly improves the flexibility of our archives and adds a number of new algorithmic features 🧵
Varun Bhatt retweeted
If you are at CORL, don't miss @vbhatt_cs's oral presentation on Nov 9, from 1:45 to 2:45 pm, about surrogate-assisted generation of HRI scenarios using quality diversity!
Varun Bhatt retweeted
Excited to share a new paper “Arbitrarily Scalable Environment Generators via Neural Cellular Automata” with @tehqin17, @vbhatt_cs, @snikolaidis19, @JiaoyangLi9, accepted to #NeurIPS2023!
Paper:arxiv.org/abs/2310.18622
Website:yulunzhang.net/publication/z…
Code:github.com/lunjohnzhang/ware…
Our work on HRI scenario generation has been accepted at #CoRL 2023 for an oral presentation! Huge thanks to the reviewers and the area chairs for the detailed feedback, and I am excited to share this work with the robotics community at @corl_conf .
Excited to share our new work “Surrogate Assisted Generation of Human-Robot Interaction Scenarios” done in collaboration with @herambnemlekar, @tehqin17, @btjanaka, @Hejia_Zhang_, @SophieHsu13, and @snikolaidis19.
arXiv: arxiv.org/abs/2304.13787 [1/7]
Very happy to see environment generation applied to warehouse design. Layouts can be generated for any given robot planning algorithm, so you only need to move the shelves to benefit from this method!
Excited to share a new paper “Multi-Robot Coordination and Layout Design for Automated Warehousing” with @tehqin17, @vbhatt_cs, @snikolaidis19, @JiaoyangLi9, accepted as a full paper in @IJCAIconf!
arXiv: arxiv.org/abs/2305.06436
Code: github.com/lunjohnzhang/ware…
We are extending the submission deadline for the EGG workshop (teapaw.org/vbhatt_cs/status/16505…) to 25 May 2023 (AoE). It is a hybrid conference, and we encourage submissions from virtually attending authors as well.
Submission link: openreview.net/group?id=robo…
We are excited to announce the first workshop on Environment Generation for Generalizable Robots (EGG) at #RSS2023 (sites.google.com/usc.edu/egg…)! Consider submitting if you are working in any area relevant to environment generation for robotics. Submissions due on May 17, 2023, AoE.
Excited to share our new work “Surrogate Assisted Generation of Human-Robot Interaction Scenarios” done in collaboration with @herambnemlekar, @tehqin17, @btjanaka, @Hejia_Zhang_, @SophieHsu13, and @snikolaidis19.
arXiv: arxiv.org/abs/2304.13787 [1/7]
Finally, we show that the failure scenarios found by our algorithms are not artifacts of simulation, and can be replicated in the real world. We observe incorrect goal predictions and near collisions that cause the robot to reset, lengthening the task completion time. [6/7]
We hope the proposed algorithms will be a valuable tool to test HRI systems before conducting user studies or deploying them in the real world. Feel free to reach out with questions and comments!
arXiv: arxiv.org/abs/2304.13787 [7/7]
Varun Bhatt retweeted
Submissions for the EGG workshop are now open! sites.google.com/usc.edu/egg…
We have an incredible list of speakers: @AleksandraFaust @jeffclune @_rockt @deepakpathak @mark_riedl @mattdeitke
We are excited to announce the first workshop on Environment Generation for Generalizable Robots (EGG) at #RSS2023 (sites.google.com/usc.edu/egg…)! Consider submitting if you are working in any area relevant to environment generation for robotics. Submissions due on May 17, 2023, AoE.
The workshop is being organized by @jparkerholder, @KuanFang, @MinqiJiang, @snikolaidis19, @tehqin17, @SophieHsu13, and myself.