Zhengming Yu

✨RigidFormer: Learning Rigid Dynamics using Transformers — accepted at #NeurIPS2026! 💻 Code is now open source: github.com/Frank-ZY-Dou/Dyna… We propose an object-centric Transformer for learning rigid-body dynamics, enabling efficient and scalable mesh-free simulation. It achieves higher accuracy and faster simulation. We hope this architecture can be extended to a wide range of rigid-body dynamics scenarios and potentially to deformable-body dynamics by adapting its rigid-alignment component. Contributions are very welcome! It’s been a pleasure working with this wonderful team: Minghao Guo (@GuoMh14), Haixu Wu (@Haixu_Wu_1998), Doug Roble, Tuur Stuyck (@TuurStuyck), and Wojciech Matusik (@wojmatusik)! See you in Atlanta! #Simulation #PhysicalAI #MachineLearning #Robotics #AI #DeepLearning #Simulation

I kept seeing #GPT-6 Astra modelling results on here and got curious enough to build the thing myself: an agentic pipeline that turns a plain RGB video into 3D assets. The living room for robots @MIT_CSAIL. Input: one handheld phone walkthrough of our lab kitchen (~20s). Nothing else — no depth sensor, no CAD, no asset library. ~1 day (I actually slept overnight), including human-in-the-loop. Live: frank-zy-dou.github.io/kitch… It is still not perfect — thin and shiny things are still weak, a few objects are drafts, the room shell needs another pass. The loop, roughly: - Monocular video → metric scan (ViPE): camera poses + depth. This is the measuring instrument, not the output. - GPT-6 lists what should exist as separate objects, then open-vocabulary detection + tracking gives per-object masks; each object is fused and measured in metres. - For every asset, GPT-6 works in its own sandbox with tools: it writes the object as a program in a small Blender DSL (closed solids, PBR colours, hinges/drawers), builds it, renders it over the original video frames, compares it with the scan points in 3D, and iterates. - A separate GPT-6 session is the verifier — it can render the model on any frame it wants and must justify every complaint with a frame. The modeller never grades its own work. - A completeness pass renders the whole modelled scene from the video's own cameras, puts it next to the real frames, and says what's still missing. What the detector keeps missing (a row of identical cabinets) gets placed geometrically instead. - Out comes MJCF/URDF with joints. Now, adding simulation to this env.