Physical AI & Post-training (RLHF, Evals) Robotic @GenrobotAI | ex. LLM human data @vana prev. VC Investor | IBD @CreditSuisse Math @@UniofOxford 🧙♂️🔮
San Francisco USA
Joined April 2022
- Tweets1.1K
- Following2.4K
- Followers2.9K
- Likes4.7K
Pinned Tweet
𝗗𝗮𝘁𝗮 𝗶𝘀 𝘁𝗵𝗲 𝗻𝗲𝘄 𝗼𝗶𝗹 𝗶𝗻 𝘁𝗵𝗲 𝗔𝗜 𝗲𝗿𝗮 and why matters to AI Agents?
As we head into 2025 - the year of AI agents - one truth stands out: AI models are only as good as their training data. Let's dive into why this matters 🧵
The real bottleneck in AI isn't computing power or model architecture - it's quality data. And here's where things get interesting...
AI agents are exploding on CT, but they all face the same challenge: garbage in = garbage out. Your fancy AI agent is useless without proper training data. @pmarca @virtuals_io @vvaifudotfun
This is where @withvana enters the picture. We're not just another AI project - we're building the foundation for high-quality, user-owned AI data. We recently just introduced Vana's DataDAO rewards system:
• Every 21 days
• Top 16 DataDAOs receive $VANA
• Community-driven selection
• Democratic staking mechanism
teapaw.org/withvana/status/186640…
Here's how it works:
• Stake $VANA on DataDAOs you believe in
• Support quality datasets
• Earn rewards as a staker
• Help shape the future of AI training data
Why this matters for AI agents:
• Better training data = better AI performance
• User-owned data = aligned incentives
• Community curation = quality control
• Decentralized governance = sustainable growth
The power of DataDAOs:
• Users control their data
• Contributors get rewarded
• Quality rises to the top
• Community-driven curation
Want to participate?
1. Get $VANA
2. Choose your favorite DataDAOs
3. Stake your tokens
4. Earn rewards
5. Shape the future of AI
Think of it as liquid democracy for AI training data:
• Your stake = your voice
• Quality rises naturally
• Contributors get rewarded
• Everyone wins
The future of AI isn't just about better models - it's about better data. And better data comes from aligned incentives and community ownership.
Join us in building the foundation for the next generation of AI agents. Your data, your future.
Learn more: datahub.vana.org/
#AI #DataDAO #AIAgents #ai16z
🧵After testing every hand glove on the market, here’s why we rebuilt ours around magnetic encoders：
After seeing just about every hand data glove on the market, I’m convinced most of them are solving the wrong problem first.
Here’s the thing nobody likes to say out loud: if your glove can’t tell you exactly where each finger joint is, every downstream model inherits that error. Hand data is only as good as the joint angles underneath it. So when we built DAS Dex 2.0, we started there.
Why not soft gloves? They deform. The surface changes shape as you grip, so the contact geometry is never fully modeled, and mapping deformation back to a 21-point hand model introduces errors that need repeated recalibration. Comfortable to wear, hard to trust as ground truth.
Why not EMF? Electromagnetic tracking drifts near metal and EM sources — exactly the environments manipulation data gets collected in. Why not pure IMU? It needs constant recalibration and drifts with temperature and time.
So we went with magnetic encoders. An exoskeleton with a built-in skeletal model reads joint angles directly — no soft-body deformation to back out, no line-of-sight dependency, strong immunity to EM interference, and no constant recalibration. You read the joint, you get the angle.
The classic objection to exoskeletons is bulk. That’s the real engineering story here: we built a custom miniature magnetic encoder just super thin and moved the whole structure to the side of the fingers instead of the back of the hand. 21 encoders per hand, 150g total (without wrist cam) — thin enough that it gets out of the way while you work.
On the sensing side: a self-developed full-palm thin-film e-skin, over 980-point high-density array covering the whole palm (not just fingertips), reading normal force down to 0.1N. A 3D silicone surface adds grip. We tuned specifically for low cross-talk — press one point and it stays one point, no phantom spread.
Put together: direct joint readout + full-palm tactile + wrist camera + IMU, in a glove light enough to forget you’re wearing. That’s what we think ground-truth-grade hand data collection should look like.
We just debuted DAS Dex 2.0 at CoRL 2026. If you’re working on dexterous manipulation, teleop, or hand data pipelines and want the technical details — benchmarks, tactile specs, how we validate joint accuracy — my DMs are open. Genuinely love to meet everyone there!
Victoria ✈️ CoRL 2026 Austin retweeted
10 Spicy Takes After Talking to 50 Friends at the Frontier of Robotics 🤖🔥
1/ Astra IS robotics' GPT-3 moment. Ironically, now that it's finally here, some in robotics seem reluctant to acknowledge it. Perhaps because it (sadly) didn't come from a robotics company?
2/ Robotics is becoming just another vertical for frontier labs. Not so different from coding in some respects. Why reinvent general intelligence for every industry?
3/ Robotics was too hard to underwrite until now. No convincing pretraining scaling law, no convergence on models, data, or hardware. No wonder some critics dismissed it as a fragmented industry playing an expensive "demo game". Astra may finally offer a credible path to scale.
4/ Robotics foundation-model startups have a case. But how strong is it? The argument: Astra handles System 2 (reasoning), while specialized models retain System 1 (skills) and System 0 (motor control). Fair. But isn't that already a narrower value proposition? And who says the boundary stops there?
5/ Follow the talent. Word is OpenAI reorganized its 30–40-person robotics team around the core Astra model. Hearing similar moves toward general models at @Tesla_Optimus. Org charts may tell us more than any demo.
6/ The robotics data gold rush is real. Million-hour procurement programs, some potentially reaching tens of millions of hours. Talk of budgets approaching $1B per frontier lab.
7/ Bullish on robotics hardware volumes. As intelligence becomes more accessible, the barriers to building useful robots could fall dramatically. The moat shifts toward reliability, dexterity, integration, and deployment.
8/ Robotics could fuel the next semiconductor and memory supercycle. A 100B-parameter model running on the edge requires ~50GB just for 4-bit weights. Now multiply that by millions of robots. Why do you think @JensenHuang is so obsessed with robotics? @elonmusk saw it coming: teapaw.org/elonmusk/status/211477…
9/ Is self-driving immune to the general-model threat? After coding and robotics, why assume self-driving remains the exclusive domain of proprietary, vertical models? Robotaxi increasingly feels like a solved problem to me. But if and when frontier labs move into self-driving, we could see an Apple vs. Android dynamic. Think about the implications for $TSLA, Waymo, and other OEMs.
10/ The next bottleneck may not be intelligence. It's deployment. Models are improving faster than our ability to deploy robots in the messy real world. Integration, hardware variance, failure recovery, and economics remain brutally hard. The real moat may be turning intelligence into reliable, profitable physical work.
--
I realize some of these takes may rub people the wrong way, especially those who have spent years doing the hard work of advancing robotics. That's not my intention.
I'm sharing them to spark debate, not to declare winners and losers. Astra has only been out for a month or so, and it's far too early to draw definitive conclusions. More importantly, when the TAM is this enormous - potentially the largest in tech - there should be plenty of room for different approaches and many winners.
++
Full diligence report: robonomics.substack.com/p/ro…
Victoria ✈️ CoRL 2026 Austin retweeted
What will be the “RLHF” moment for robotics? What will it take to get robotics to where LLMs are today and beyond?
New blog post with @chelseabfinn sharing some thoughts on the state of RL for frontier robotics models and what's missing 👇
Blog: pd-perry.github.io/posts/pos…
Ok this is wild and I need to share it — today we're open-sourcing Gen-HumanEgo 🎉 DM OPEN🤖🤔
1,800+ hours of real human demonstrations, 10,000+ tasks, first-person view. And it's actually training-ready — which turned out to be the whole hard part.
Here's the thing I keep coming back to: scale alone doesn't make data useful. Throwing more hours at a model doesn't help if it still can't tell how a hand moved, where things are in 3D, or what the task actually meant. That's the gap we went after.
So instead of just dumping video, we ran everything through our Data Foundation Model and aligned three signals in one shared space:
✋ Hands — full 3D reconstruction, not just keypoints. Finger articulation, grasp, palm orientation — the stuff that actually tells you how someone grabbed a thing. Sub-cm, even under blur and occlusion.
📐 Depth — Ego-Depth produces a single large-FOV fisheye depth map, using cross-view attention to recover depth where stereo overlap fails — including mirrors, glass, and reflective surfaces. Full-FOV coverage of 157°×132°, ≤±4% error at 1m, ≥99.5% fill rate.
🧠 Semantics — collect naturally, annotate afterwards — no rigid action scripts, so variation, retries, and recovery survive. Each episode is labeled at three levels (video → task → subtask), aligned to frame ranges, with object attributes, spatial relations, and an is_success flag.
The part I'm proudest of: it's not four separate label dumps. Everything's aligned in the same spatiotemporal frame — one coherent representation of what a human actually did.
And we're not stopping at hands. Whole-body is next — reaching, bending, shifting weight, all the motion an Ego camera only half-sees. That's the piece I'm most excited to build.
It's open. Go break it, train on it, tell me what's missing 👇
teapaw.org/GenrobotAI/status/2102…
Open-source now: huggingface.co/datasets/genr…
Join our Discord community to connect with the team and other builders: discord.gg/rSSb5thgu
Contact Us: [email protected]
Victoria ✈️ CoRL 2026 Austin retweeted
Introducing OM-1, our first robot foundation model: zero-shot generalizing across table-top arms, industrial arms, and humanoids.
- Learned directly from human manipulation data
- No teleop or robot data
- Near human-level dexterity and efficiency
- Multi-robot collaboration
Victoria ✈️ CoRL 2026 Austin retweeted
GEN-1.5, our latest embodied foundation model, can learn new tasks prompted with 3 - 12 seconds of a single demonstration, no gradient updates or fine-tuning. It generalizes prompts to new situations, recovers from mistakes, and improvises new strategies to reach the same goal.
Ok this is TOO cool 😭 Real footage of our rig in the wild — not a demo, not a mockup, an actual customer running actual data collection. This is the good stuff. 🔥
The moat in robot learning is data. But the moat under that is the collection stack that produces it. Most vendors are still optimizing for hours shipped. We optimize for what survives training.
Here is a guide to choose robotics data collection hardware.
Robotics data quality starts with the collection stack. It’s not just about camera resolution. What matters is whether the system can reliably produce synchronized, precise, multimodal, and scalable data.
Explaining precision, data type, continuous collection and efficiency below, using Genrobot as an example.
Victoria ✈️ CoRL 2026 Austin retweeted
One brain. For any robot. 🤖
We’re launching Gemini Robotics 2: our next-generation physical AI bringing full body intelligence to humanoids, advanced dexterity, multi-robot teamwork and more.
It's impressive to see how @GenrobotAI supplied the full 100K hrs of real-world pretraining data behind Xiaomi-Robotics-1.
The data stack includes: 6-camera synced Ego+UMI capture + Data Foundation Model (DFM) + motion-capture-verified ground truth — built for scale, precision, and diversity.
The most common mistake for robot data collectors: optimizing for hours collected, not diversity. If your dataset has 100hrs from one operator in one kitchen, you don't have 100hrs of signal — you have 1 scene overfit 100x. The bar labs are moving to: <10hrs per unique operator × scene pair.
LingBot-VLA 2.0 is open-source. It was pretrained on 60,000 hours of data — 10,000 of which are high-value egocentric human manipulation. We supplied a core part of that. A thread on why hand tracking precision, not data volume, is now the bottleneck. 🧵
1/
Robot learning is fundamentally observation → action. Pure ego video forces a model to infer manipulation from high-dimensional pixels — separating background, lighting, occlusion, camera motion, and hand-object interaction on its own. Hand tracking gives it a shortcut.
2/
Precise hand tracking is a lower-dimensional, structured, action-relevant representation. It doesn't just record what a person saw — it describes how the action happened: how the trajectory formed, how the grasp closed, when contact occurred. Less to learn. Faster convergence.
3/
But it only has training value if it's precise enough. In AR/VR, a few centimeters is fine — you're recognizing a gesture. In robot learning, error doesn't stay in perception. It propagates:tracking error → action representation error → policy error → contact failure → task failure.
4/
The decisive motions in manipulation happen in the last few centimeters before contact. USB insertion. Placing a test tube. Precision grasping. Off by a little on finger position, off by a beat on contact timing — and the model learns the wrong causal structure. Sub-centimeter isn't a nicer perception number. It's the threshold for the data to be trainable at all.
5/
Stable <1cm hand tracking isn't an algorithm result. Three layers of difficulty: Hardware-software co-design. In egocentric view, hands are occluded, move fast, leave the frame. A single view can't recover the hand.
Ground-truth production. 3D calibration, extrinsic stability, mocap alignment, multi-view time sync — a data engineering problem, not a tuning problem.
Compute. Multi-view matching, uncertainty estimation, 3D reconstruction, temporal smoothing — at scale.
6/
Uncertainty-adaptive multi-view fusion. The hard part of a multi-camera rig isn't adding cameras — it's knowing which view, and which keypoint, to trust right now. Naively averaging across cameras injects error from the unreliable ones. We predict per-view, per-keypoint uncertainty and weight the fusion accordingly.
7/
The bigger shift: Human Data sits between internet video (scalable, weakly action-relevant) and teleoperation (action-relevant, unscalable). It's the only path that gets you both — but only if the spatial precision holds. Human Data is becoming infrastructure, not a collection service. From Human, For Human.
Congrats @Robbyant @YinghaoXu1 @GenrobotAI the open-source release. 🤝#EmbodiedAI #RobotLearning #HandTracking #VLA
Personal update: I'm relocating to SF and going all-in on the single most underrated bottleneck in AI right now — robotic data.
Everyone's racing on models and compute. But in embodied AI, data is the real constraint. Robots don't learn from text or internet images — they need real physical demonstrations: precise sequences of what a human does with their hands and body, and what happens in the world as a result. Collecting that at scale is brutally hard, and almost no one has solved it.
I've spent the last few years across crypto and then building human AI data partnerships at @vana, working closely with frontier labs on how training data shapes where models actually go. The throughline was never the asset class — it was always data. Since 2024 I've been pulled deeper into the training-data layer: human data, RL and post-training data, and eventually a bit of my own data venture (lucky to have crossed paths with great data teams along the way — @HelloSurgeAI @turingcom @micro1_ai @mercor @joinHandshake ). Embodied data is just the earliest, hardest version of that same problem. LLM data feels like a market of existing demand; robotic data is where new demand is being created.
Leaving crypto wasn't easy — it means stepping out of circles I spent years building and starting over in a new world. But if there's one thing I trust about myself, it's building 0→1. Coming in humble and genuinely optimistic.
🧵 why I think human-centric embodied data is where LLM data was in ~2020 👇
LLM data had the internet as a starting corpus. Embodied AI has no equivalent — there's no pretrained web of real-world manipulation data. Every hour of high-fidelity human behavior data has to be created, not scraped.
That changes everything. The hard problem isn't labeling — it's defining what good data even is: which modalities matter (ego vision + mm-precise hand tracking + tactile/force + proprioception), how to align them at high precision, and how to make data transfer across different robot embodiments.
Get it right and you unlock the things robotics has been stuck on for a decade: imitation learning at scale, dexterous bimanual manipulation, and finally closing the sim-to-real gap. Then the flywheel kicks in — better data → better models → better data pipelines → better data again.
Embodied AI doesn't just need better models. It needs scalable human data infrastructure. That's the bet.
Nice work — motion alone really isn't enough. Seen a lot of glove designs by now, and the fundamentals matter no matter the approach: tactile sensing (piezoresistive / capacitive / etc.), force direction beyond just pressure, and tight time-sync so contact sits inside the full multimodal state. Same goal, different path. Stay tuned~
Manipulation happens through surfaces. To understand contact-rich dexterous interaction, motion alone is not enough. We also need to know surface properties and contact state.
Excited to share ART-Glove, an articulated tactile glove that captures contact-grounded information while preserving human dexterity. It provides:
- Known Geometry: 16 rigid functional surfaces
- Surface Motion: 22 anatomically aligned joints
- Tactile Contact: 2048 piezoresistive taxels
Huge thanks to my advisor Ding @zhao__ding, and to Yuxiang @yxyang1995, Maria @bauzavillalonga, Marissa, and Peide @peide_huang for the valuable advice and discussions.
Paper: arxiv.org/abs/2606.16370
Victoria ✈️ CoRL 2026 Austin retweeted
The bottleneck of frontier robotics isn’t compute, labeling, or the models themselves.
It’s data collection.
While language models scaled effortlessly on open internet text, robotics requires physical trajectories, motor torques, and tactile forces that cannot simply be scraped from a webpage.
Every token has to be fought for.
Here is a breakdown of the 7 data types shaping the industry today, each representing a trade-off between collection cost and action-label purity:
1. Real Teleoperation (AgiBot World, DROID). Collected by humans guiding hardware, it scales linearly with human hours.
2. Low-cost Capture (Mobile ALOHA, UMI handheld). It drives collection cost down while keeping real physics, though it introduces an embodiment mapping problem when transferring human hand actions to robotic joints.
3. Fleet / Deployment Data (Tesla Optimus, Figure). These are trajectories from robots already working in the field. Tesla is betting its automotive fleet infrastructure transfers to Optimus. It generates powerful, real edge cases, but requires scaled deployment.
4. Simulation (NVIDIA Isaac Sim, Genesis). While offering near-infinite scale, the sim-to-real gap still struggles to model contact-rich dynamics like slipping, twisting, friction.
5. World-Model Synthetic (NVIDIA Cosmos 3). NVIDIA just shipped Cosmos 3, which natively outputs action trajectories, not only video pixels. If a world model can accurately simulate the laws of physics natively, it reduces the need for manual teleop data drastically.
6. Egocentric video (Ego4D, Meta’s Project Aria). First-person human video captured with head-mounted rigs. Far more scalable than teleop and closer to a robot’s own viewpoint. Still carries no robot action signals on its own.
7. Internet video (Youtube, TikTok). Maximum scale, lowest cost, effectively free. It captures the widest range of objects, tasks and physical situations, but with zero action labels and (mostly) a third-person viewpoint.
Collecting data is only the step one.
The next great execution challenge is engineering a coherent training recipe that can blend these heterogeneous data sources into a single model.
Victoria ✈️ CoRL 2026 Austin retweeted
World Labs CEO Dr. Fei-Fei Li: "The world is not made of words."
"Language models have given machines an extraordinary command of concepts, vocabulary, and reasoning, but the physical world, virtual or real, runs on a different substrate."
"Where language models learn the statistical structure of text, world models learn the statistical structure of space and time: how light falls on a surface, how a garden looks from an angle no camera has captured, how objects respond to force and follow the laws of physics."
"Language gave machines a way to talk about that world. World models are how machines will finally come to understand, imagine, reason and interact with it."
Full piece: drfeifei.substack.com/p/a-fu…
Victoria ✈️ CoRL 2026 Austin retweeted
1/ 🧠Humans are the best robot data source!
2/ 👓Human egocentric video is rich in quantity, but poor in quality.
3/ Beyond scaling data, smarter representation and architecture matter just as much.
4/ Want an open-source framework to train your own learn-from-human-data robot policy?
🚀We introduce HumanEgo: Zero-Shot Robot Learning
from Minutes of Human Egocentric Videos⬇️
✦ Zero-Shot Human-to-Robot Transfer
✦ Robot-Data-Free
✦ Just 30 min of data per task
✦ Collect by Anyone, Anytime, Anywhere
✦ Deploy on Any Robot, Any Camera, Any Environment
✦ Open-Source & Easy-to-Implement
Let's squeeze every bit of signal out of human data!
🌐 Website: humanego-ai.github.io
📄 Paper: arxiv.org/pdf/2605.24934
💻 Code: github.com/TX-Leo/HumanEgo
📹 Video: youtu.be/pdL46diijuY
🧵 1/n
Victoria ✈️ CoRL 2026 Austin retweeted
of all RL environment companies in this market map, @benchflow_ai is the only one
> angel funding only
> no yc / a16z accelerator
> no phd / former lab exp / no formal pedigree
also solofounder
yet we shipped two top tier researches, got 8-fig acquisition offers from unicorns
Victoria ✈️ CoRL 2026 Austin retweeted
We are back. After one year of quiet building.
Introducing GENE-26.5, our first robotic brain that takes a major step toward human-level capability.
For years, robotics has struggled to learn from the world’s largest and valuable data source: Humans.
Solving it means rethinking the whole stack from the ground up:
- A robotics-native foundation model.
- A 1:1 human-like robotic hand.
- A noninvasive data collection glove for motion, force, and touch.
- A simulator that turns weeks of experiments into minutes.
GENE-26.5 is trained across language, vision, proprioception, tactile, and action. We designed a set of tasks to test how far we can go with this new paradigm.
Fully autonomous, 1x speed, one model, same weights. (Enjoy with sound on)
We are approaching the endgame for robotics.
And this is just a beginning.