Computer Vision Research Scientist at @simulon, music lover , fond of scientific/musical/geeky/useless stuff. I'm posting papers on whatever I found amazing :)
Paris, France
Joined June 2012
- Tweets3.1K
- Following307
- Followers2.4K
- Likes4.3K
Basically, if you want to save money, just do close up shots. Seedance2.5 still better overall imo (as well).
Shoutout @jon_barron for the whole thing + nice visuals and @jbhuang0604 for the video on it! youtu.be/dYxEmKbZfr4?si=FyXG… (worthy subscription!)
"FoundationVIO: Enforcing Metric Scale in Feed-forward Visual-Inertial Odometry"
TLDR: 3D foundation models + IMU measurements and factor-graph optimization to recover metric-scale camera trajectories and dense 3D reconstructions, with loop closure for globally consistent SLAM.
Jin Rhee¹, Yuhao Zhang¹, @fdellaert², @MauriceFallon¹
¹ Oxford Robotics Institute, @UniofOxford
² School of Interactive Computing, @GeorgiaTech
Project page: dynamic.robots.ox.ac.uk/proj…
Paper: arxiv.org/abs/2610.14951
Code: github.com/ori-drs/Foundatio…
Alexandre Morgand retweeted
We’ve had creators reaching out with some really interesting ideas for @Simulon Reality-Aware Rendering, asking what’s possible.
We want to explore these ideas together and learn how we can support the use cases that emerge from real-world experimentation.
So we’re opening up early access to a few creators this weekend, with credits to experiment, ahead of the public launch on Tuesday, October 27.
If you’ve got an exciting idea, sign up in the Simulon app and send us your username along with what you’d like to create.
Let’s explore what’s possible together!
"PURE: Refining Feed-forward 3D Reconstruction with Unlabeled Post-training"
TLDR: improves VGGT, Pi3, and DA3 using unlabeled video post-training, combining relative depth constraints, local geometry distillation, and correspondence-based pose refinement 1/2
to produce more accurate and temporally consistent 3D reconstructions of dynamic scenes without additional inference overhead. 2/2
Project page: funnyguy525.github.io/PURE-P…
Paper: arxiv.org/abs/2610.14698
Code: github.com/IGL-HKUST/PURE
Demo: huggingface.co/spaces/cyun92…
One year ago, I would have never imagined that we could reach this level of Mixed Reality with complex interactions. This is an incredible achievement at @Simulon but you would be surprised by how easy it is to use! Give it a try! (Android version is launching soon!)
Here’s how @Simulon Reality-Aware Rendering works. I placed my raptor next to the water and dropped a stone in to see how he’d respond.
His 3D placement sets his starting position and scale. When the recording starts, he becomes a ghost, leaving me to shape the performance through actions, gestures and spoken instructions. Simulon uses the recorded scene to generate his behaviour and integrate him into the footage afterward.
Instead of typing a prompt, spinning the slot machine and hoping for the best, you can show it, say it and perform it. Your intent, timing and spatial context are right there in the recording. It’s a much more intuitive way to create.
This breakdown is generated procedurally, summarising the actual reasoning behind the result.
Alexandre Morgand retweeted
The shadow and compression of the edge of the couch, plus audio is impressive!
🤖 Made with AI
Alexandre Morgand retweeted
Here’s how @Simulon Reality-Aware Rendering works. I placed my raptor next to the water and dropped a stone in to see how he’d respond.
His 3D placement sets his starting position and scale. When the recording starts, he becomes a ghost, leaving me to shape the performance through actions, gestures and spoken instructions. Simulon uses the recorded scene to generate his behaviour and integrate him into the footage afterward.
Instead of typing a prompt, spinning the slot machine and hoping for the best, you can show it, say it and perform it. Your intent, timing and spatial context are right there in the recording. It’s a much more intuitive way to create.
This breakdown is generated procedurally, summarising the actual reasoning behind the result.
Alexandre Morgand retweeted
Meet Nigel. He’s not enjoying our cause-and-effect demo.
So I let him sit in the DeLorean. Still not happy.
I didn’t write his lines. What he says is generated from his personality and the context of the shot.
It’s an entirely new way to create: place imaginary characters in your world, record your performance, and let Simulon generate their reactions and realistically integrate them into your footage.
Reality-Aware Rendering is coming to @Simulon this week.
Alexandre Morgand retweeted
This wasn’t animated by hand. And it wasn’t generated from a prompt either.
It was created from real world footage.
It’s something entirely new.
We call it Reality-Aware Rendering.
Coming to @Simulon next week.
"Extracting Neural Materials from Images"
TLDR: framework extracting relightable neural materials from multiview images; learned material prior + uncertainty-guided inverse path tracing, capturing complex effects like clearcoat, dust, fuzz, layered reflections beyond classic PBR
@youwangk¹², Jon Hasselgren¹, @Peter4AI¹, Andrea Weidlich¹, Tae-Hyun Oh³, Jacob Munkberg¹
¹ @nvidia
² POSTECH
³ KAIST
Project page: nvlabs.github.io/neumatex/
Paper: arxiv.org/abs/2606.26715
Results: nvlabs.github.io/neumatex/re…
original thread xitter.cf/youwangk/status/207161…
Excited to share our new arXiv paper, "𝗘𝘅𝘁𝗿𝗮𝗰𝘁𝗶𝗻𝗴 𝗡𝗲𝘂𝗿𝗮𝗹 𝗠𝗮𝘁𝗲𝗿𝗶𝗮𝗹𝘀 𝗳𝗿𝗼𝗺 𝗠𝘂𝗹𝘁𝗶-𝘃𝗶𝗲𝘄 𝗜𝗺𝗮𝗴𝗲𝘀."
🔗 Project page: nvlabs.github.io/neumatex/
📄 Paper: arxiv.org/abs/2606.26715
💡𝗧𝗟;𝗗𝗥
We tackle the challenging inverse rendering problem of extracting "neural materials" from images by combining:
1️⃣ Large-scale material reconstruction foundation model
2️⃣ Test-time optimization using differentiable Monte Carlo path tracer
3️⃣ Uncertainty-guided material regularization
What is a "𝗡𝗲𝘂𝗿𝗮𝗹 𝗠𝗮𝘁𝗲𝗿𝗶𝗮𝗹"? It's a universal SVBSDF representation that goes beyond classic PBR. The neural specular latent space encodes complex material-light interactions like haze, dust, clearcoat, fuzz, and scattering, all while being path-traceable in real time.
Classic PBR struggles (diffuse + single-lobe specular component) with these complex multi-layered specular effects. To overcome this and the lack of prior knowledge about the neural material latent space, we combine a Large Material Reconstruction Model (LMRM) for robust initialization with uncertainty-guided Test-Time Optimization (TTO).
This project wraps up an intense and amazing 13-week internship at NVIDIA Real-Time Graphics Research (research.nvidia.com/labs/rtr…). Huge thanks to my amazing mentors and the team: Jon Hasselgren, Jacob Munkberg, Peter Kocsis, Andrea Weidlich, and Tae-Hyun Oh. Super excited to share our efforts on next-generation generative content creation.
