PhD in Robotics @CMU_Robotics @airlabcmu | Visiting Researcher @Meta | Making robots 🤖 see the 🌍

Joined November 2017
Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀 One universal model enables SoTA for: 🔥 Mono Depth Estimation 🔥 Multi-View SfM 🔥 Multi-View Stereo 🔥 Depth Completion 🔥 Registration … and many more possibilities! – plus everything is metric 🎯 We release code for data processing, training, benchmarking & ablations – everything Apache 2.0! Details & Links 👇
29
130
31
734
123,747
Nikhil Keetha retweeted
How your robot moves depends on what it’s driving over, especially when moving at >10m/s over loose gravel or ice 🚙💨 What if dynamics models could be conditioned on exactly the terrain they expect to drive over? Introducing SpatialBEV! Accepted at #CoRL2026 🤠 0/7
2
7
2
26
1,999
Nikhil Keetha retweeted
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks: Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better: Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better: Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better: Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work! In summary: - As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding. - Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.
1,674
6,646
1,544
56,037
7,991,437
Nikhil Keetha retweeted
Today we release Poincar3, our newest 3D model trained entirely through SSL on RGB image sequences. I find the coolest thing is that, without labels, it learns multi-view geometry. Just using the attention map, reliable tracks can be formed (see below). davnords.com/poincar3
9
27
1
193
17,169
Nikhil Keetha retweeted
1/ We introduce TrackEverything: a 3D point tracker that tracks all points across all frames of long videos (1000+ frames). Our key idea is to tie computation to unique 3D scene content rather than redundant 2D pixels in a video. trackeverything.github.io
25
183
14
1,662
139,294
Nikhil Keetha retweeted
Introducing PointZero—a 3D world model pre-trained without robots. Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it? PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics! Our model achieves SOTA results on: 🔥 Zero-shot 3D dynamics 🔥 Action-conditioned 3D dynamics (post-training) 🔥 Imitation learning (post-training) Details and links 👇 1/6
8
46
5
260
29,032
Nikhil Keetha retweeted
World-action models typically imagine the future in RGB – but are pixels really the right representation for robotics? Our bet is no: RGB spends capacity on fine-grained details and variation that are often irrelevant for robot policies. DINO features, point tracks, and depth capture more useful features like semantics, motion, and geometry. But no single modality captures everything — how can we effectively combine them? We introduce ✨ModAR✨, which predicts the future one modality at a time, with each prediction informing the next. We train from scratch and find that this formulation performs best. Our 30.1M scratch-trained model even outperforms a 6B video-model-initialized model finetuned on the same data! 🧵 [1/8]
12
61
5
411
38,940
Nikhil Keetha retweeted
[1/N] Introducing Point4D: dense 4D reconstruction across hundreds of frames from a monocular video. Unlike existing feed-forward 4D methods that operate over short windows, Point4D keeps tracking in 3D, even across occlusions and out-of-frame motion.
6
47
3
332
35,160
Sparse attention ftw!! Stay tuned for what we have to show for vision 👀
🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6
17
1,326
Holy!!! Didn't expect 4D sim to be possible any time soon
GPT-6 gets pretty good at creating one-shot raw animations from scratch too without much detailed prompts. Definitely did not think a VIGA-4D variant would be this close
828
Nikhil Keetha retweeted
Hmm, data is still and will always be the biggest moat. Deepseek is clearly saying that as of now better post-training comes more from "better data + better environments" than from novelties in RL algorithm.
🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6
6
25
8
361
25,735
Replying to @OmarAlama
@OmarAlama tried GPT 6 Astra Xhigh to recreate this from @theworldlabs's Atlas - and 👀 🤯 Video shows the blender reconstruction and fun fact ... Astra used COLMAP! 😳 and pulled the floor plan via search! 🧵👇 They really did train with tons of 3D tool data!!
Atlas is out! What excites me most is the 3D capability, months of work with @bardienus : an AR omni model that beats every specialized model at 3D reconstruction. First clip: fly through the Porsche museum, 500+ frames streamed into a single consistent point cloud; feed-forward Gaussian splats follow.
2
11
1
61
10,625
Colmap output saved by Astra:
1
4
368
The video shared at the top of this thread was an intermediate thing from Astra - it used 40% of the $100 weekly limit Lowk was having an existential crisis for the past day but alas all you need to know is that the magic sauce is in the data 😉 xitter.cf/Nik__V__/status/209774…
Hot take: we keep debating modeling for solving a field - llm, world model, vla, wam, agents … But tbh the real moat is the underlying data powering it 👀👑
3
425
Crazy stuff!!
GPT-6 hard-carrying humanoid loco manipulation 👀
There's a new version of this post
2
223
Hot take: we keep debating modeling for solving a field - llm, world model, vla, wam, agents … But tbh the real moat is the underlying data powering it 👀👑
2
2
1
23
2,369
Nikhil Keetha retweeted
We will give one banked reset for every day you don't have access to Astra on your paid ChatGPT plan, starting today. Team is moving mountains to give access as fast as we can. First one will land in ~ 3 hours. There is still time to create your account if you don't have one.
5,573
3,448
3,541
48,140
9,180,592
Nikhil Keetha retweeted
[1/6] Can one model fix rendering artifacts from any 3D representation? And what does it even mean to "fix" a rendering? At #ECCV2026, we present FixAnything (fix-anything.github.io): a single generalist video model that refines 3DGS/mesh/sparse point-cloud renderings into photorealistic, 3D-consistent videos.
7
44
4
262
16,810
GPT-5.6 Sol seems to currently be the most capable model available for almost every single task (at least the ones I am testing), across coding, math, agentic workflows, prose and creative writing. It's likely going to go down as the Opus 4.6 of OAI, and a much more significant moment than GPT-5 (which was meh). Even if there is indefinite model dev pause (narrator: "there won't be"), optimizing inference for this particular model is good enough of a business as it feels we may be close to the plateau of utility for a non trivial fraction of users.
23
14
4
467
51,760