Jeremy van Reenen retweeted
One year ago, I would have never imagined that we could reach this level of Mixed Reality with complex interactions. This is an incredible achievement at @Simulon but you would be surprised by how easy it is to use! Give it a try! (Android version is launching soon!)
Here’s how @Simulon Reality-Aware Rendering works. I placed my raptor next to the water and dropped a stone in to see how he’d respond.
His 3D placement sets his starting position and scale. When the recording starts, he becomes a ghost, leaving me to shape the performance through actions, gestures and spoken instructions. Simulon uses the recorded scene to generate his behaviour and integrate him into the footage afterward.
Instead of typing a prompt, spinning the slot machine and hoping for the best, you can show it, say it and perform it. Your intent, timing and spatial context are right there in the recording. It’s a much more intuitive way to create.
This breakdown is generated procedurally, summarising the actual reasoning behind the result.
Jeremy van Reenen retweeted
Reality-Aware Rendering is a new mode in the Simulon app that turns a phone recording into a rendered scene. The demo runs about a minute and eighteen seconds.
Per the creator, you place neural assets with your phone and act out the scene. When you stop recording, it reads what you do, what you say and the place around you, and the assets respond to all of it. The final is regenerated from the recording as a new performance, not a copy of the footage.
Why it matters: it points to a way of making visual effects that starts from a phone and a real room instead of a studio pipeline. It is available now on iPhone, with Android coming.
Credit: @diveshnaidoo
Jeremy van Reenen retweeted
Check out this Reality-Aware rendering feature from the new @Simulon 2.0. This is a game changer.
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
How it works teapaw.org/diveshnaidoo/status/21…
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
Simulon is insanely good.
Anyone remotely interested in 3D, CG, and cinema should toy with it for fun.
You can do this in 3 minutes, app download included. Try it.
Here’s how @Simulon Reality-Aware Rendering works. I placed my raptor next to the water and dropped a stone in to see how he’d respond.
His 3D placement sets his starting position and scale. When the recording starts, he becomes a ghost, leaving me to shape the performance through actions, gestures and spoken instructions. Simulon uses the recorded scene to generate his behaviour and integrate him into the footage afterward.
Instead of typing a prompt, spinning the slot machine and hoping for the best, you can show it, say it and perform it. Your intent, timing and spatial context are right there in the recording. It’s a much more intuitive way to create.
This breakdown is generated procedurally, summarising the actual reasoning behind the result.
Let the force be with you @Simulon
You can try this right now for free. We are live on the AppStore - apps.apple.com/us/app/simulo…
Jeremy van Reenen retweeted
确实有趣😁
供桌上的蟾蜍也是我的小宠物
这种互动方式让人印象深刻
效果真的不可思议˶╹ꇴ╹˶！
🤖 Made with AI
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
this is truly magical
unlocks so much potential for creation and play
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
You can try this right now for free. We are live on the AppStore - apps.apple.com/us/app/simulo…
Jeremy van Reenen retweeted
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
Video generation is not new but at @Simulon the keyword was always about control. How do you navigate through vfx with the least friction possible? How do you maximise creativity? Impossible question but I think a path of the answer is here!
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
Look at this carefully.
This opens up a whole new world of creative possibilities for creative people.
Here are ten ways this could be used :
1. Filmmakers: shoot a dialogue scene in a real café, drop your character into the booth, and act it out with your phone as the camera.
2. Musicians: record a song in a real room, lock the mood and feeling of that place, then let the listener step inside and choose the next verse.
3. Theatre artists: rehearse a scene on a real street corner, place the other character in the space, and run lines without a stage.
4. Painters and visual artists: hang a work in a real gallery wall, walk around it with the phone, and show collectors how it lives in the room.
5. Novelists and world-builders: drop a character into a real city block and let readers walk the opening chapter as a short scene.
Endless possibilities....
But it's the ones with great IP will make the most of this.
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
看完直接愣住。你在真实空间里演，虚拟资产会「看见」你的动作、「听见」你说话，然后整场戏被重新生成一次。这已经不是传统 AR 或 VFX 了，是人和虚拟角色在真实世界里共同完成一场表演。超前。
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
Watch this, understand it, and embrace what the future of reality will look like
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
This morning I cloned Dune with @Simulon. Naturally and obviously, the next step was turning him into a kaiju and filming him in the city using reality-aware rendering.
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
Been following Divesh since day zero.
This is really ground breaking technology.
I’m so stoked to see what people will create with this.
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
Jeremy van Reenen retweeted
The whole short was made with zero prompting. Just real footage, 3D neural assets and reality-aware rendering.
Here are a few moments side by side. Watch the full short for the ending.
Top: Simulon
Bottom: what I actually filmed
Jeremy van Reenen retweeted
Things that aren’t there can now see and hear you.
Reality-Aware Rendering is now live in Simulon. Place anything in the real world, act out the scene, and it responds to you.
We made this short in an afternoon. Watch to the end to see what was really there.
Available now on iPhone. How it works 👇