Andrew Tao retweeted
🚀 Nemotron 3.5 Super VL is here! Double AA index with no architectural changes! 💚
Same architecture as Super-v3 (120B total / 12B active hybrid Mamba-Transformer MoE). AA Intelligence Index doubles (13 → 26) with the new training recipe:
What we added:
• Joint SFT on vision + language data from the first update
• Logit distillation as continuous pretraining from Ultra for 1T tokens.
• Agentic mid-training (boosts software-engineering / long-horizon work)
• Improved C-RADIO vision encoder for better document analysis.
📃 All details in the tech report:
research.nvidia.com/labs/nem…
🤗 Model weights on HF (NVFP4): huggingface.co/nvidia/NVIDIA…
Blog: developer.nvidia.com/blog/nv…
Nemotron Super 3.5 … Smarter and less verbose than ultra. Also our best vision model by far. This was a good tight sprint. Hope it’s useful to you!
Nvidia’s Nemotron 3.5 Super VL is now the leading US open weight model, scoring 26 on the Artificial Analysis Intelligence Index
Nemotron 3.5 Super VL achieves twice the intelligence of its predecessor, Nemotron 3 Super (13), and does so using ~60% fewer output tokens per task
It launches alongside Nemotron 3.5 Bolt 74B-A9B 24GB, the first model in its new Bolt family, which scores 21 on the Intelligence Index
Congratulations to @NVIDIAAI on the releases!
Key Takeaways:
➤ Leading US open weight model: Nemotron 3.5 Super VL scores 26.5, edging out Thinking Machines' Inkling Small (25.7) and ahead of NVIDIA's Nemotron 3 Ultra (23) and Meta's Muse Glimmer (17)
➤ Bolt comes within 2 points of Ultra: Nemotron 3.5 Bolt 74B-A9B 24GB scores 21, 1.5 points behind Nemotron 3 Ultra 550B A55B (23). The 2 bit memory has about a seventh of Ultra's parameters (74B vs 550B) and a sixth of its active parameters (9B vs 55B)
➤ Super 3.5 VL near lossless in NVFP4: Nemotron 3.5 Super VL releases in NVFP4, scoring within 0.1 points of its BF16 weights on the Intelligence Index, with no component moving by more than 2.6 points
Nemotron 3.5 Super VL details
Context window: 1M tokens
Modalities: Vision and text input; text output
Parameters: 120B / 12B
Nemotron 3.5 Bolt 74B-A9B 24GB details
Context window: 256k tokens
Modalities: Vision and text input; text output
Parameters: 74B / 9B
2
2
327
Andrew Tao retweeted
We've been cooking! 👨🍳
DLSS 5 with 3D-Guided Neural Rendering is being upgraded with a 2X faster model for up to 1.4X faster framerates - available first in NBA 2K27 Season 2 on October 16.
Learn More ➡️ nvda.ws/4dUwbwP
Remember when initial reviews of Ray Tracing tech focused on the 58% performance drop when enabled? techpowerup.com/review/battl…
Games are entertaining not just because they’re pretty but moreso because a team of developers has built an amazing experience and world/story. Agents and NR can help realize a vision but you truly need devs to craft an experience.
4
274
Andrew Tao retweeted
We’re publishing a deeper technical report on why generation is needed for the next leap toward photorealism.
VRAM and compute limit both the scene abstraction and the number of rays we can afford. Previous DLSS reconstructs from that representation—so even perfect reconstruction is still bounded by what the scene contains.
DLSS 5 is the first DLSS to generate beyond that abstraction and break through that ceiling, while staying grounded in the game and controlled by the artist.
We also compare it with offline generative models and renderers at similar scaling cost.
research.nvidia.com/labs/adl…
Andrew Tao retweeted
A NVIDIA diz que as otimizações do DLSS5 serão lançadas de maneira bem mais rápidas, fazendo o modelo ficar mais leve e maior qualidade a cada update.
Inclusive teremos um novo update para o DLSS5 até o final do ano.
Andrew Tao retweeted
DLSS 5 Neural Rendering is a huge step towards the future of real-time graphics and the realization of 10 years of research. It is controllable, consistent, and fast, and I can’t wait to see what game developers accomplish with this new tool.
youtu.be/58FagrSqC4M
What we're trying to do with DLSS5/NR is challenging: to provide uplift while preserving artistic intent. Ultimately, the goal is to give the dev more power to achieve their vision, not less.
2
1
296
Replying to @NVIDIAAI
@NVIDIAAI Nemotron 3 Nano Omni on your DGX Sparks is something wonderful to experience 🤩
My second recipe, this is a quick win for all your DGX Spark and local AI folks
I just replaced gemini-flash, openai, and claude with my own model running on a single DGX spark to give my entire agent fleet vision, audio understanding, and video understanding
A lot of LLM models lack vision, and other modalities,
you can solve it super quick by just spinning this up.
@nvidia 's Nemotron-3-nano-omni-30B is a multimodal perception engine:
Computer-use, vision, audio, and video built into one MoE checkpoint. 31.6B params, 3B active.
It takes 21GB on the spark. you still have 86GB free — run it alongside your main model.
i have it serving as the vision sub-agent for every agent in my fleet, replacing three separate cloud API calls.
Here's what I tested to verify it can replace my cloud AI endpoints:
→ sends it a news screenshot, it reads the headline, identifies the UI, reads the branding. 7s.
→ sends it a photo of toys on a shelf, it catalogs 12+ objects. 9s.
→ sends it a full 2:46 song as raw audio, it transcribes the lyrics, identifies the key (C#m), tempo (~91 BPM), instruments, and song structure. 17s.
→ sends video, it identifies the show (Arcane), reads Netflix branding, classifies the genre. 9s.
The thing I found out: vLLM already supports audio input through the standard API. you just need pip install "vllm[audio]" — the parakeet ASR encoder was sitting there the whole time, nobody turned it on.
One serve command, all modalities:
vllm serve nemotron-omni-nvfp4 \
--limit-mm-per-prompt '{"image":4,"video":1,"audio":4}'
21GB checkpoint. 86GB VRAM still free.
62 tok/s decode. zero cloud. zero API costs. zero rate limits.
full recipe + benchmarks + findings → [reply link]
Andrew Tao retweeted
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
images.nvidia.com/pdf/Open-W…
Andrew Tao retweeted
Why are open technologies for AI so important?
How and why is NVIDIA building Nemotron?
What can we learn from China’s AI efforts?
Great conversation with Matt Turck.
youtu.be/Oojrfdl42LI
Selected as a best paper finalist at #CVPR2026: PixelDiT from NVIDIA Research
In most image generation models, a pretrained autoencoder compresses the image before any diffusion happens, causing quality loss that accumulates across the entire pipeline.
PixelDiT, or Pixel Diffusion Transformers, removes this step entirely. It's a single-stage model that learns the diffusion process directly in pixel space, end-to-end.
Andrew Tao retweeted
Replying to @nvidia
@NVIDIA introduces Nemotron 3 Nano Omni LLMs.
A year ago, we started research on omni-modal LLMs—covering everything from architecture to data—and released OmniVinci (ICLR 2026). We’ve received some feedback since then, but two clear themes emerged: 9B parameters weren't quite enough for many applications, and restrictive licensing was holding back commercial use.
Therefore, we’ve teamed up with a larger group of amazing NVIDIA researchers/engineers to introduce NVIDIA's first commercial omni-LLM Nemotron 3 Nano Omni. This new 30B-A3B model is fully open-source and commercially usable. It also consistently outperforms similar-sized models across image, video, audio, and omni benchmarks.
My primary focus on this project was Omni-RL infra development and model training, besides architecture/hyper-parameter exploration and verification. Here are a few key takeaways from the process:
• Cross-modal synergy is real: In multimodal RL, improvements in one modality reinforce the others, translating directly into stronger overall benchmark scores.
• Data quality beats quantity: When it comes to RL, a steep reward curve slope is often much more critical than sheer training duration.
• Untapped potential in RL infra/algorithm: Especially for multimodal setups, there is still so much room to explore and optimize underlying architectures and scheduling systems.
To better support both the research community and industry developers, we aren't just releasing the model weights—we are also open-sourcing the training datasets and training code.
All LLMs should be omni-modality. We are looking forward to hearing your thoughts and feedback!
Blog: nvda.ws/4tCmhFS
Hugging Face: nvda.ws/4u79ue9
Tech Report: nvda.ws/4tDzHBq
#AI #MultimodalAI #OpenSource #ReinforcementLearning
Andrew Tao retweeted
Today we released Nemotron-3-Nano-Omni-30B-A3B - our first Omni model, with speech and audio understanding capabilities powered by parakeet-tdt-0.6b-v2 encoder.
🫡1st position on VoiceBench
🌏English only
🎙️5.95% WER on Open ASR Leaderboard
📽️Video+audio understanding
Andrew Tao retweeted
Replying to @ctnzr
This looks incredible! Can’t wait to see it in action on my new Alienware Area 51 👽 😎
























