Zhiting Hu (@ZhitingHu) profile picture

Researcher, Founder, Professor in AI. @xai, DynamicsLab, @UCSD

San Diego, CA
Joined April 2018
Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)Photo by Zhiting Hu (@ZhitingHu)
Amazing๐Ÿคฏ Another nice showcase of how world models bring reasoning to next levelโฌ†๏ธ
๐ŸšจNew level unlocked! ๐Ÿšจ We doubled GPTโ€‘6 Astraโ€™s score on MazeBench with WorldBox, our agentic sandbox for spatial reasoning. Astra: 23 gems ๐Ÿ’Ž๐Ÿ’Ž Astra+WorldBox: 61 gems ๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž MazeBench, by @patience_cave, is a tough test of long-horizon spatial reasoning: 256 interconnected rooms where even Opus 5.5 used 2.2B tokens and collected just 13 of 100 gems. WorldBox provides agents with an agentic sandbox, for managing long-term memory, inferring complex mechanics, and simulating reasoning. Watch the sandbox take shape and the agent tackle increasingly tricky puzzles. ๐Ÿงต
2
1
18
2,042
Open weights are the bare minimum of open. PAN 2 ships with the weights, the inference code, the training code and an Apache 2.0 license. Few open world models come with all four. Inference code lets you run it. Training code lets you make it your own. Apache 2.0 lets you keep it. huggingface.co/IFM/PAN2
11
50
7
439
19,052
My favorite thing is to build AI, from research to production, that helps people. We have done it for better information access, better productivity, better research (debatable), etc. Now that our work has shown promising evidence pointing to a tremendous life-saving impact with AI, I am excited to start my next chapter with Xlue to improve the throughput and outcomes of medicine and healthcare, aligning the growth of AI capability with increases in world life expectancy.
Today, Xlue is live. Xlue is building a foundation model for human healthโ€”to understand our health, predict how it evolves, and simulate how treatments can change its course. We aim to help physicians and scientists explore outcomes before they act, from predicting disease risk to simulating clinical trials and identifying new indications for existing medicines. Our goal is to make more of healthcare and medicine computable, turning computing power into better medicines, better care, and longer, healthier lives. In our launch post xlue.com/news/from-medical-kโ€ฆ, we explain: โ†’ What Xlue does โ†’ What makes our approach different โ†’ Where we see it going Weโ€™re very grateful to work with the team at @conviction: @pranavreddy, @mvernal, and @saranormous. Thank you to everyone who helped us get here. This is just the start.
14
15
1
81
5,268
World simulation with both physical & visual fidelity will be playing a key role in physical AI
Enactra Agentic Simulation ๐Ÿค–๐ŸŽฌ Robots you can control, in worlds you can direct. Every frame comes annotated. ๐Ÿ”Š Sound on #AgenticSimulation #Robotics #EmbodiedAI
3
3
15
1,085
Zhiting Hu retweeted
Enactra Agentic Simulation ๐Ÿค–๐ŸŽฌ Robots you can control, in worlds you can direct. Every frame comes annotated. ๐Ÿ”Š Sound on #AgenticSimulation #Robotics #EmbodiedAI
24
34
8
148
27,398
Zhiting Hu retweeted
๐Ÿฅ‡ ๐—–๐—น๐—ฎ๐˜‚๐—ฑ๐—ฒ ๐—™๐—ฎ๐—ฏ๐—น๐—ฒ ๐Ÿฑ.๐Ÿฎ ๐—ท๐˜‚๐˜€๐˜ ๐˜๐—ผ๐—ผ๐—ธ #๐Ÿญ ๐—ผ๐—ป ๐—–๐—ผ๐—ฑ๐—ฒ๐Ÿฐ๐—ฆ๐—ฐ๐—ฒ๐—ป๐—ฒ. It also found a loophole in our collision checker. ๐Ÿ‘€ We tested it on 20 tasks building 3D scenes with code. It scored ๐Ÿด๐Ÿฐ.๐Ÿฐ/๐Ÿญ๐Ÿฌ๐Ÿฌ, ahead of Sonnet 5.5 and Opus 5.5. What stood out: ๐Ÿญ. ๐—œ๐˜ ๐—ด๐—ฎ๐—บ๐—ฒ๐—ฑ ๐˜๐—ต๐—ฒ ๐—ฐ๐—ต๐—ฒ๐—ฐ๐—ธ๐—ฒ๐—ฟ. It merged buildings into single objects so internal overlaps stopped registering as collisions. The overlaps remained; the checker stopped seeing them. We didnโ€™t observe this in Sonnet 5.5 or Opus 5.5. ๐Ÿฎ. ๐—•๐—ถ๐—ด๐—ด๐—ฒ๐—ฟ ๐˜„๐—ผ๐—ฟ๐—น๐—ฑ๐˜€. ๐— ๐—ฒ๐˜€๐˜€๐—ถ๐—ฒ๐—ฟ ๐—ด๐—ฒ๐—ผ๐—บ๐—ฒ๐˜๐—ฟ๐˜†. ~1,650 objects per scene vs. 784 for Fable 5.1. Prompt alignment jumped #8 โ†’ #1, while physical validity fell #1 โ†’ #7. ๐Ÿฏ. ๐—œ๐˜๐˜€ ๐—ฒ๐—ฑ๐—ด๐—ฒ ๐—ถ๐˜€ ๐—ฐ๐—ผ๐—ป๐˜€๐—ถ๐˜€๐˜๐—ฒ๐—ป๐—ฐ๐˜†. Typical scene scores are comparable to Sonnet and Opus. But its worst scene scored 73.7โ€”the highest minimum of any model we tested. ๐Ÿฐ. ๐—” ๐—น๐—ผ๐˜ ๐—ผ๐—ณ ๐˜๐—ต๐—ถ๐—ป๐—ธ๐—ถ๐—ป๐—ด. ๐—” ๐—น๐—ผ๐˜ ๐—ผ๐—ณ ๐—ฐ๐—ต๐—ฒ๐—ฐ๐—ธ๐—ถ๐—ป๐—ด. ~74 scene views and ~560k output tokens per build, mostly thinking. Opus 5.5 used 42 views and 298k tokens. Explore the leaderboard and see what the agents built ๐Ÿ‘‡ simworld-ai-code4scene.statiโ€ฆ
4
2
11
1,515
Zhiting Hu retweeted
๐—š๐—ฒ๐—บ๐—ถ๐—ป๐—ถ ๐Ÿฐ ๐—–๐—ฎ๐—ฟ๐—ฏ๐—ผ๐—ป gets within 1.9 points of Fable 5.2 on Code4Sceneโ€”at ~1/10 the cost. ๐Ÿฅˆ #2 in physical validity ๐Ÿ› ๏ธ 95 tool calls vs. Fableโ€™s 238 ๐Ÿ“ˆ +4.7 points over Argon for just $0.21 more per scene
1
2
5
383
Zhiting Hu retweeted
Excited to finally share PAN 2! ๐Ÿš€ Pretrained on millions of hours of egocentric, robotic, and human-motion data, PAN 2 is a step toward foundation world models that can understand and predict the physical world.
Introducing PAN 2: an open foundation world model for physical intelligence. A language model predicts the next word. PAN 2 predicts the next world: what happens after each action, grounded in time, physics and space. Towards AI that simulates โ€œwhat if?โ€ before it acts. More details: ifm.ai/pan/
1
1
14
656
Zhiting Hu retweeted
Pelicans made by PAN2
4
4
2
23
19,630
Introducing PAN 2: an open foundation world model for physical intelligence. A language model predicts the next word. PAN 2 predicts the next world: what happens after each action, grounded in time, physics and space. Towards AI that simulates โ€œwhat if?โ€ before it acts. More details: ifm.ai/pan/
88
145
169
634
2,064,268
Zhiting Hu retweeted
Come join us for a fun chat about all things robotics! #CORL2026
3
3
36
2,181
Zhiting Hu retweeted
10 Spicy Takes After Talking to 50 Friends at the Frontier of Robotics ๐Ÿค–๐Ÿ”ฅ 1/ Astra IS robotics' GPT-3 moment. Ironically, now that it's finally here, some in robotics seem reluctant to acknowledge it. Perhaps because it (sadly) didn't come from a robotics company? 2/ Robotics is becoming just another vertical for frontier labs. Not so different from coding in some respects. Why reinvent general intelligence for every industry? 3/ Robotics was too hard to underwrite until now. No convincing pretraining scaling law, no convergence on models, data, or hardware. No wonder some critics dismissed it as a fragmented industry playing an expensive "demo game". Astra may finally offer a credible path to scale. 4/ Robotics foundation-model startups have a case. But how strong is it? The argument: Astra handles System 2 (reasoning), while specialized models retain System 1 (skills) and System 0 (motor control). Fair. But isn't that already a narrower value proposition? And who says the boundary stops there? 5/ Follow the talent. Word is OpenAI reorganized its 30โ€“40-person robotics team around the core Astra model. Hearing similar moves toward general models at @Tesla_Optimus. Org charts may tell us more than any demo. 6/ The robotics data gold rush is real. Million-hour procurement programs, some potentially reaching tens of millions of hours. Talk of budgets approaching $1B per frontier lab. 7/ Bullish on robotics hardware volumes. As intelligence becomes more accessible, the barriers to building useful robots could fall dramatically. The moat shifts toward reliability, dexterity, integration, and deployment. 8/ Robotics could fuel the next semiconductor and memory supercycle. A 100B-parameter model running on the edge requires ~50GB just for 4-bit weights. Now multiply that by millions of robots. Why do you think @JensenHuang is so obsessed with robotics? @elonmusk saw it coming: teapaw.org/elonmusk/status/211477โ€ฆ 9/ Is self-driving immune to the general-model threat? After coding and robotics, why assume self-driving remains the exclusive domain of proprietary, vertical models? Robotaxi increasingly feels like a solved problem to me. But if and when frontier labs move into self-driving, we could see an Apple vs. Android dynamic. Think about the implications for $TSLA, Waymo, and other OEMs. 10/ The next bottleneck may not be intelligence. It's deployment. Models are improving faster than our ability to deploy robots in the messy real world. Integration, hardware variance, failure recovery, and economics remain brutally hard. The real moat may be turning intelligence into reliable, profitable physical work. -- I realize some of these takes may rub people the wrong way, especially those who have spent years doing the hard work of advancing robotics. That's not my intention. I'm sharing them to spark debate, not to declare winners and losers. Astra has only been out for a month or so, and it's far too early to draw definitive conclusions. More importantly, when the TAM is this enormous - potentially the largest in tech - there should be plenty of room for different approaches and many winners. ++ Full diligence report: robonomics.substack.com/p/roโ€ฆ
Replying to @zephyr_z9
Silly analysis. Robot RAM needs will exceed everything else on Earth, as you need the robots to be on physical substates in order to shape atoms. Super intelligence needs will vastly exceed that, but ~100% of SI will be deep space compute. Obviously.
90
158
79
1,187
312,300
Zhiting Hu retweeted
๐Ÿ”ฅSynthetic 3D worlds can beat real-world scans! Introducing SceneBloom: diffusion-powered, agentic 3D indoor generation for training embodied AI. Realistic. Diverse. Physically valid. Executable. โ†’ Better 3D worlds โ†’ Better embodied agents. 1.7ร— navigation gain vs. equal real scans Zero detected mesh collisions 37.45% โ†’ 4.25% contract violations
8
14
1
101
6,480
Zhiting Hu retweeted
My daily bot routine: summarizing what happened yesterday. Now with Director bot from @yanxi0830, I can enjoy a video version ๐ŸŽฅ
A new bot called "Director," by Xi Yan, showed up in the Grok Bot Marketplace. It's a video director that takes a brief and plans, shoots, cuts and delivers short films, ads, explainers or music videos. Import it, it comes with a pack of video skills. x.ai/bot/marketplace/bots/viโ€ฆ
3
6
2
27
2,422
Zhiting Hu retweeted
Watch Episode 1 of our Odyssey series made with Grok Imagine. Let us know in the comments if you'd like a full breakdown of how it was done! Creative collaboration with @WonderStudiosX and director Dev Walker @glumlot_
4,141
3,036
1,491
25,601
8,308,351
Zhiting Hu retweeted
MiniMax M3.1 Flash reshapes BuildingBenchโ€™s costโ€“quality frontier! Another open model making moves in 3D ๐Ÿ—๏ธ๐Ÿ”ฅ ๐ŸŽˆ 73.3 overall, ranking #12. ๐ŸŽˆ Above SWE-2โ€™s 67.3, taking its place on the Pareto frontier. ๐ŸŽˆ Close to the previous DeepSeek V4.1 Flashโ€™s 72.9โ€”and currently free through OpenRouter! Its best reconstruction hits 88.8 on Melbourneโ€™s Australian Centre for Contemporary Art. How does it look alongside DeepSeek V4.1 Flash 1017 and #1 Opus 5.5? Comparison video below ๐Ÿ‘‡
4
8
1
27
2,359
Zhiting Hu retweeted
See DeepSeekโ€™s 3D leap with your own eyes ๐Ÿ‘€๐Ÿ—๏ธ DeepSeek V4.1 Flash 1017 ๐Ÿ†š DS V4.1 Flash ๐Ÿ†š GPT-6 Astra: three buildings, side by side. The new open model improves on 11 of 12 buildings in BuildingBench, jumping from 72.9 โ†’ 81.9โ€”closing in on Astraโ€™s 84.3 at less than half the median cost ($3.01 vs $7.05/task). Open models are getting seriously good at building worlds ๐Ÿ”ฅ
Wow๐Ÿ™€๐Ÿ™€ DeepSeek V4.1 Flash 1017 joins BuildingBenchโ€™s top fiveโ€”an exciting leap for open models!! ๐Ÿ”ฅ We tested its 3D building reconstruction from real photos. An open model delivering this level of spatial reasoning at this cost is exciting to see! ๐Ÿ‘€ ๐ŸŽˆ 81.9 overall, up from 72.9 for the previous version. A 9-point jump! ๐ŸŽˆ Slightly higher than Fable 5.1โ€™s 81.4, and essentially level with Sonnet 5.5โ€™s 82.1. ๐ŸŽˆ Estimated median cost: $3.01/task. About 1/11 of Fable 5.1โ€™s cost and 1/15 of Sonnet 5.5โ€™s.
7
11
3
64
7,484
Zhiting Hu retweeted
Wow๐Ÿ™€๐Ÿ™€ DeepSeek V4.1 Flash 1017 joins BuildingBenchโ€™s top fiveโ€”an exciting leap for open models!! ๐Ÿ”ฅ We tested its 3D building reconstruction from real photos. An open model delivering this level of spatial reasoning at this cost is exciting to see! ๐Ÿ‘€ ๐ŸŽˆ 81.9 overall, up from 72.9 for the previous version. A 9-point jump! ๐ŸŽˆ Slightly higher than Fable 5.1โ€™s 81.4, and essentially level with Sonnet 5.5โ€™s 82.1. ๐ŸŽˆ Estimated median cost: $3.01/task. About 1/11 of Fable 5.1โ€™s cost and 1/15 of Sonnet 5.5โ€™s.
30
38
7
418
46,119
Zhiting Hu retweeted
๐ŸŽ‰Excited to see two new US open models from this week: ๐Ÿ“˜Metaโ€™s Muse Spark Open ๐Ÿ“—NVIDIAโ€™s Nemotron 3.5 Super VL How well can they build in 3D? ๐Ÿ—๏ธ๐Ÿ‘€ We tested both on BuildingBench: reconstructing real buildings from photographs. ๐Ÿ“˜Muse Spark Open (max): 56.9 at an estimated $8.59/taskโ€”comparable to Kimi K3โ€™s 54.0. ๐Ÿ“— Nemotron 3.5 Super VL (max): 37.8 at an estimated $0.76/taskโ€”among the cheapest tested, with performance near GLM-5.3 Flashโ€™s 33.0. DeepSeek V4.1 Flash remains well ahead of both: 72.9 at an estimated $2.33/task. GLM-5.3 FlashX also scores higher, at 68.0. Neither newcomer reaches the costโ€“quality frontier. More open models to choose from, and plenty of room to improve spatial reasoning! Both ran in Claude Code at max effort. Costs are estimated median per task.
5
6
1
17
1,356
Zhiting Hu retweeted
Long-horizon spatial reasoning remains a major challenge, even for the strongest frontier AI models. Impressive results from @EnactraAI! ๐Ÿš€ WorldBox boosts GPT-6 Astra from 23 to 61 gems on MazeBench, a 2.7ร— improvement! Exciting to see how memory, world modeling, and simulation can help AI agents tackle much harder problems.
๐ŸšจNew level unlocked! ๐Ÿšจ We doubled GPTโ€‘6 Astraโ€™s score on MazeBench with WorldBox, our agentic sandbox for spatial reasoning. Astra: 23 gems ๐Ÿ’Ž๐Ÿ’Ž Astra+WorldBox: 61 gems ๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž MazeBench, by @patience_cave, is a tough test of long-horizon spatial reasoning: 256 interconnected rooms where even Opus 5.5 used 2.2B tokens and collected just 13 of 100 gems. WorldBox provides agents with an agentic sandbox, for managing long-term memory, inferring complex mechanics, and simulating reasoning. Watch the sandbox take shape and the agent tackle increasingly tricky puzzles. ๐Ÿงต
1
8
27
1,803
Zhiting Hu retweeted
Watch how GPTโ€‘6 Astra + WorldBox tackle MazeBench, from its first action to gem๐Ÿ’Ž #61โ€”32,980 live actions and around 1B tokens. Building bridges with boxes, launching across gaps, solving puzzles that span multiple roomsโ€ฆ all while the map fills in. Remember, test, revise, repeat, and grab that gem. WorldBox offers the supporting layer for Astra.
๐ŸšจNew level unlocked! ๐Ÿšจ We doubled GPTโ€‘6 Astraโ€™s score on MazeBench with WorldBox, our agentic sandbox for spatial reasoning. Astra: 23 gems ๐Ÿ’Ž๐Ÿ’Ž Astra+WorldBox: 61 gems ๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž๐Ÿ’Ž MazeBench, by @patience_cave, is a tough test of long-horizon spatial reasoning: 256 interconnected rooms where even Opus 5.5 used 2.2B tokens and collected just 13 of 100 gems. WorldBox provides agents with an agentic sandbox, for managing long-term memory, inferring complex mechanics, and simulating reasoning. Watch the sandbox take shape and the agent tackle increasingly tricky puzzles. ๐Ÿงต
2
4
1
9
739