Pinned Tweet
Replying to @LupintheI
Have to correct your misunderstanding of my sentence.
My original sentence: PT *faithfully* simulate the scene. The bottleneck is that the assets are a real-world approximation. This approximation leads to fake feeling.
There does exist many games with great assets, PT makes them look incredible.
This Saturday I'm giving a talk at @saturdayrobotic in SF. Mostly about FLORA, our Data Engine used to train GR00T, and how I think about robot data going forward. Everyone is welcome. Come say hi!
luma.com/ub5ask7s
🚀 @saturdayrobotic Reading Club #39: @nvidia Flora — Scaling Robot Learning to 200,000 Hours of Data Per Day!
🎤 Speaker: @yunzeman (Research Scientist, @nvidia GEAR)
Keynote: NVIDIA Flora — The Open Data Engine for NVIDIA GR00T
📍 San Francisco
📅 Saturday, November 21 | 2–5 PM
🎟️ RSVP: luma.com/ub5ask7s
Host: @junfanzhu98
Robot foundation models need data labels at every stage. Pretraining needs timestamped action captions, both to condition on language and to filter and rebalance large, mixed datasets. RL post-training needs process rewards that show where an episode goes wrong. Deployment needs outcome rewards to monitor fleets and evaluate policies.
Today these labels come either from human annotators, which doesn't scale, or from frontier API models, which are too expensive at robot-data scale. Most existing tools also produce only one kind of label.
This talk presents Flora, the open data engine used to label training data for NVIDIA's GR00T models. Flora consists of three models built on open VLMs. A temporal segmenter and a contextual captioner turn human and robot videos into timestamped action instructions. These range from single-camera egocentric clips to multi-view recordings of bimanual and dexterous-hand robots. A single reward model gives episode outcome scores and dense per-step labels that flag suboptimal actions and failures. It can also grade episodes against rubrics written at test time.
🔥 In production, it labels up to 200,000 hours of data per day.
The talk will close with what an open data engine makes possible next: closing the loop between robot deployment and learning, human-to-robot transfer, and physical auto-research.
See you at Saturday Robotics! 🤖
🎟️ RSVP: luma.com/ub5ask7s
#NVIDIA #GR00T #Flora #PhysicalAI #Robotics #WorldModels #SaturdayRobotics
Yichen Sheng retweeted
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
Yichen Sheng retweeted
Totally agreed with @TimSweeneyEpic
Where is the journalism in @WSJ?
Replying to @WSJ
I can't believe this garbage. The article starts out by saying that NVIDIA's export strategy was lawful, then ignores that and tears into the company and its lawful export strategy with unsourced unsubstantiated suggestions that other companies might have done bad things.
Yichen Sheng retweeted
Replying to @GregoryDiamos
The media said it, and implied they were the source. Every time their claimed source denies it, they make up a new one. And now they are trying to redefine the English language to make their past false statements seem true-ish. I have never seen anything like this in industry.
Yichen Sheng retweeted
A heartbreaking day.
I came to @nvidia as an intern. It was my first job, and I never left. For my whole career, @JensenHuang has been the leader I looked up to.
He taught us to reason from first principles, that the mission is the boss, and that greatness comes from suffering. And he lived it every single day, working harder than any of us.
I've never known a more inspiring leader. Thank you, Jensen. We'll keep building on what you started.
Replying to @WSJ @WSJopinion
Yichen Sheng retweeted
I’m deeply saddened that @JensenHuang is no longer leading NVIDIA.
I once had the opportunity to meet Jensen alongside a few teammates. He was genuinely enthusiastic about our work and warmly shook hands with each of us. It was my first time shaking hands with him. More than the gesture itself, what meant the most to me was knowing that our hard work was recognized and valued by the leader of NVIDIA.
I’ve always admired Jensen’s vision, conviction, and willingness to make bold decisions. He has consistently pushed NVIDIA to think big and do what truly matters.
Thank you, Jensen, for your extraordinary leadership and for inspiring us to keep pushing forward.
In the harshest winter, the plum blossom still blooms.
To Jensen.
🤖 Made with AI
My heart is heavy today.
I still don’t have a photo with @JensenHuang. It’s not that I never had the chance to ask. I just had a small, personal dream: that one day, he would be the one to suggest we take a photo together.
I like to think I made it halfway there. After seeing our work, Jensen warmly shook our hands. We didn’t take a photo, but that moment meant more to me than he probably realized. Now, that little dream feels further out of reach.
I’ve learned so much from Jensen: how to analyze a problem, reason carefully, and make a decision. How to stay resilient when things get difficult. How to approach something entirely new by returning to first principles.
There’s so much wisdom in the way he thinks and leads, and so much of it has shaped the way I approach my own work.
Those lessons will stay with me. They will continue to guide us as we work toward our mission.
Thank you, @JensenHuang, for everything you’ve taught me—and for a handshake I’ll always remember.
Yichen Sheng retweeted
Very inspired way to make a good use of data! Happy to see #DL3DV is contributing to industry level world models!
Excited to finally share C2R#! 🚀
Coarse-to-Real (C2R)# turns rough 3D and anime videos into photorealistic videos, without requiring paired video training data, by learning from much cheaper image pairs instead.
Last year, we introduced C2R, using video generation models to turn rough 3D blockouts into photorealistic videos. But making this work reliably in a game engine is another story. Even models as powerful as Seedance 2.5 often require multiple attempts to get the desired result. For real applications, we need stable and controllable generation. This typically requires large amounts of paired coarse/real videos, which are extremely expensive to collect.
So how can we get around this data bottleneck?
One evening over dinner, while I was struggling with this problem, our intern Daohan proposed an interesting idea: Could we learn video translation from image pairs instead? After all, image pairs are much easier and cheaper to obtain.
Over the following months, we worked hard to turn this idea into reality.
We developed a lightweight Real-to-Coarse model trained only on image pairs. By applying it frame by frame to real videos, we can synthesize pseudo-coarse videos and create large amounts of pseudo-paired video training data. We then use these pairs to train a large Coarse-to-Real video generation model, without needing actual paired coarse/real videos!
The resulting C2R# technology was eventually used at Roblox RDC.
C2R# can also handle other input proxies, including anime-style videos, as long as you have 10K image pairs.
🌐 Project: gonzalognogales.github.io/co…
📄 Paper: arxiv.org/abs/2610.28954
Introducing Flora, an open model suite for scaling robotics data engines.
It watches robot and human video and annotate what happened (timestamped instructions) and how well it went (rewards and failure reasons).
🔗 research.nvidia.com/labs/gea…
We use it to train NVIDIA GR00T. 🧵
Today, Dream Labs is officially out of stealth! 🚀
What an incredible journey these past six months have been.
I’ve always believed in two things: scaling and generalization. That belief led us to introduce World Action Models (WAMs), which learn from videos of the world and use that understanding to act. With enough scale, the right architecture, and the right data, I believe we can build a general-purpose robot brain that works across different robots, tasks, and environments.
Yet despite incredible progress in robotics, most breakthroughs have remained in research labs, out of reach for the broader public.
Six months ago, @jang_yoel , @scott_e_reed , @youliangtan , and I co-founded Dream Labs with a simple dream: What if anyone could give their robot a general-purpose brain, just through an API call?
Today, we’re taking that first step. Dream-1 is now live for everyone
Dream-1 controls bimanual robots with parallel grippers it has never seen before, zero-shot, on unseen tasks in unseen environments. But what excites me most isn’t just what Dream-1 can do today. It’s what we might discover together.
With our public robot brain API, we want to put general-purpose robot intelligence into everyone’s hands. Try it on new robots, tasks, and environments we’ve never imagined. We want to empower everyone to discover their own “wow” moments with robots and explore what’s truly possible.
ChatGPT wasn’t perfect when it first launched, but it opened up AI to the world. I hope today marks a similar beginning for robotics.
Dream-1 is still a research beta. It will make mistakes, and there’s a long way to go. But someone has to take the first step.
None of this would have been possible without our amazing team. I’m incredibly proud of what we’ve accomplished in just six months and grateful to be building alongside such talented people. ❤️
And we’re just getting started. If you share our dream, come build the future of robotics with us! 🚀
Everyday capabilities for everyday robots @dreamlabs_ai
Cheaper experiments → more hypotheses tested. That's the part that actually compounds.
The AI industry has a massive compute efficiency problem. At Physion Labs, we think it's time to challenge it.
Today, we're sharing the technology behind NexTron, our world-intelligence training engine.
Our benchmarks on matched hardware:
⚡ Up to 20.9× SFT throughput vs. DeepSpeed (H100)
⚡ Up to 6.6× SFT throughput vs. Megatron (H200)
⚡ Up to 7.2× SFT throughput vs. Megatron (B200)
⚡ Up to 3.1× end-to-end RL throughput vs. Miles–Megatron
🔥 180B model fine-tuning on just 8 H100 GPUs.
Same hardware. Dramatically higher throughput. Comparable model quality.
We believe the next frontier isn't just building bigger GPU clusters.
It's unlocking more intelligence from every GPU, every watt, and every dollar spent on training.
#PhysionLabs #NexTron #WorldIntelligence #AIInfrastructure
1
2
302
Yichen Sheng retweeted
Please help spread the word!
We’re hiring Research Scientists and Research Interns at NVIDIA Research to work on some of the most exciting problems in generative AI:
🌎 World Models & Video Diffusion
🧠 Diffusion Language Models
🧬 Biomolecule Design
If your research aligns with any of these areas, please apply directly through the links below.
Open positions + application links:
research.nvidia.com/labs/gen…
Yichen Sheng retweeted
“5 years of graphics progress in one toggle”
Now that people have had a chance to play with DLSS5, we can see that neural rendering has redefined real-time graphics.
Ten years of dedicated research and development paying off.
youtu.be/WY5u1-4h48Q
Yichen Sheng retweeted
Yichen Sheng retweeted
Can't wait to see some of you in Berlin!
Open models are how developers build AI they can own.
Not just weights. Model families, post-training recipes, evaluation tools, and supporting data give teams the ability to inspect, adapt, and deploy AI for real applications.
At GTC Berlin, Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA, will show developers what that looks like in practice.
🎙️ Building Open Models That Developers Can Inspect, Adapt, and Deploy: nvda.ws/4ivv0qH
📍 #NVIDIAGTC Berlin | October 20–22, 2026
🔗 Register with code GTC-NVP8A3H2 for 25% off: nvda.ws/4j297iN
Yichen Sheng retweeted
Impressed by GPT-Astra’s CoT ! This Kung Fu demo is built from scrach in Blender and takes the reference video + simple prompt as input. The CoT reasoning really show a grasp of kinematics, joint coordination, and body motion !
Reference video comes from PhysAvater @qingqing_zhao_ @yang_zheng18
Yichen Sheng retweeted
GPT-6 Astra feels deeply personal. In summer 2024, during my internship with #NVIDIACosmos team, we started Scenethesis(research.nvidia.com/labs/cos…) with GPT-4o: an agent that could imagine a 3D scene, ground it through promt/vision, align it with physics, judge the result before replaning. Those were good old days.
Two years later, the dream has grown far beyond a single scene. Astra brings much stronger multimodal reasoning and long-horizon tool use; Blender and Unreal are becoming canvases for intelligence. Watching the base model advance this quickly is astonishing—and incredibly exciting.
The mission ahead excites me most: agents that create, simulate, and explore worlds; accumulate knowledge from experience; and keep evolving—for games, robotics, and embodied AI. Huge congratulations to the brilliant @OpenAI team. What a beautiful moment for the agentic world!
Replying to @OpenAI
GPT-6 Astra is the most intelligent and aligned model in the world, and sets a new state of the art for computer use, browsing, software engineering, cybersecurity, science, and professional work.
openai.com/index/gpt-6-astra…




























