DPhil in Computer Vision, University of Oxford || Clarendon Scholar @Oxford_VGG @OxfordTVG || Experience with @Snap @AIatMeta
Noxus
Joined October 2022
- Tweets55
- Following210
- Followers157
- Likes294
World models that see only through the players' eyes forget what happens out of view.
OverSeer adds a few fixed cameras and generates every view together, so the world stays consistent over time and across players.
📄 arxiv.org/abs/2610.28554
🌐 runjiali-rl.github.io/OverSe…
🧵👇
Joint work with @kam_woh , @MinghaoChen23 , @han_junlin , @jianyuan_wang , @ZhaochongAn , Juan-Manuel Perez-Rua, Andrea Vedaldi and @liu_shikun at @AIatMeta.
We release the data: 38,780 two-player Minecraft episodes with 8 synchronised views and 19,775 two-player Doom episodes with 10 views.
🤗 huggingface.co/collections/l…
It works in Minecraft, in Doom and on real multi-camera capture: places stay where they were when a player returns, and what one player does is seen by the other.
Fixed cameras barely change, so our δ-VAE keeps each camera's first frame and codes only what changes: 8× fewer tokens per camera. The agents' views stay full frames.
OverSeer then generates all views together, frame by frame, from the agents' actions and camera poses.
A creeper explodes while both agents look away.
A world model built on egocentric views alone loses the crater. OverSeer's fixed cameras keep it in view, so it is still there when the agents turn back.
Runjia Li retweeted
Introducing Code Video Model 🎬
Amazing work led by @raywzy1 and the team!
Agentic coding as the engine for controllable video generation.
Scene layout, motion, timing, camera trajectories, and physical dynamics, all in editable code.
A coding agent turns prompts into executable programs that render proxy videos. Our training-free early-step conditioning lets a pretrained video model follow the proxy’s structure while adopting a reference image’s appearance.
From physics-grounded motion and bullet time to 3D/4D reconstruction and robotics simulation, it all starts with code.
Watch the demo and explore more visual results 👇
Project page: code-video-model.github.io/
Paper: code-video-model.github.io/t…
Code: github.com/code-video-model/…
Data: huggingface.co/datasets/code…
Youtube: youtube.com/watch?v=dTtiVkSR…
Runjia Li retweeted
We share H3-World 🌍
The first to turn MiniMax-H3 itself into world model.
No new action module. We directly convert H3’s pretrained language understanding into world control.
Only 8K samples + 0.199% Trainable Params.
📄 huggingface.co/papers/2609.0…
💻 danzer1xxxxchan.github.io/H3…
Runjia Li retweeted
We live in a multimodal world. We see, talk, act, and dream.
Yet most LLMs still start with language pretraining. Why not train them natively with multimodal I/O from scratch?
Because it’s SUPER HARD, adding modalities triggers training instability, design complexity, and often modal competition
So what’s the path forward?
Introducing: Towards Physics of Multimodal Pretraining (junlinhan.github.io/projects…)
We unpack the underlying mechanics of multimodal pretraining across 4 aspects: Knowledge Flow, Modality Synergy, Early Unification, and Recipe.
Runjia Li retweeted
We trained a video world model on just 15 hours of video of a single-arm robot. 🧵
It generalizes zero-shot to unseen embodiments (and even orangutans). And it picked up something we never trained for: give it the object motion you want, and it synthesizes the robot motion that produces it.
The trick is representing actions as masked video.
Runjia Li retweeted
🎁 We are pleased to introduce Syn4D, a large-scale multiview synthetic dataset for dynamic scenes, accepted at #ECCV2026.
The complete dataset, including its dense geometric annotations, is now publicly available.
🌐 Project: jzr99.github.io/Syn4D/
💻 GitHub: github.com/jzr99/Syn4D
🤗 Dataset: huggingface.co/datasets/Syn4…
🧵1/8
Runjia Li retweeted
AutoPartGen code is now available! 🚀
We are excited to finally release a reimplementation of AutoPartGen, as promised!
Check out the code, models, project page, and paper here:
💻 Code: github.com/facebookresearch/…
🤗 Models: huggingface.co/facebook/auto…
🌐 Project page: silent-chen.github.io/AutoPa…
📎 Paper: arxiv.org/abs/2507.13346
We sincerely apologize for the long delay, and we greatly appreciate your patience and continued interest in our work!
Runjia Li retweeted
SAE interventions can be unreliable. 🧠🔒
We show that even when features are clamped, bad behaviors can still return through alternative residual-space directions. 🧩↩️
Feature control ≠ behavior control. 🚨
Paper: huggingface.co/papers/2606.1…
Page+Code: mingyuee88.github.io/sae-pos…
Runjia Li retweeted
🚀 Introducing Instruct-Particulate, our new model for inferring articulated structures from static 3D meshes, with significantly improved generalization to novel object categories and support for kinematic prompting.
To achieve this, we scaled our training data 40× and redesigned the model to follow kinematic prompts.
The result: diverse, realistic, simulator-compatible articulated 3D assets can now be generated directly from real-world images!
🔗 Project page: instruct-particulate.github.…
🤗 Demo: huggingface.co/spaces/rayli/…
Runjia Li retweeted
🌙 Open-sourcing Yume -- a programmable, explicit world model on Godot.
You build a game by describing the world as JSON -- the things in it + the rules for how they behave -- and one fixed engine runs it.
github.com/kamwoh/yume
Runjia Li retweeted
🚀 Introducing Articraft, a coding agent for articulated 3D asset creation.
Articraft writes code, executes it, receives validation feedback, and refines the result into simulation-ready 3D assets with parts, joints, and motion.
We’re also releasing Articraft-10K: 10,000+ articulated objects across 250 categories, unlocking large-scale interactive scenes for robotics simulation and physical AI.
🔗 Project page: articraft3d.github.io/
💻 Code: github.com/mattzh72/articraf…
Runjia Li retweeted
We made an interactive client-server viewer for LagerNVS with @JonathonLuiten!
You can now interactively explore scenes from just a photo capture - no optimization, no 3D Gaussians, just load your images, run the model on a cloud GPU and stream the renders to your local browser.
Check out the video below for some spaces I recently captured in Oxford, London and beyond!
