Ass. prof. of Generative AI. PI of Generative Memory Lab (@DondersInst). Generative diffusion, language diffusion and statistical physics. AI realist.

Nijmegen, Nederland
Joined July 2011
1/?) As promised to Sander Dieleman (@sedielem), we’re finally excited to share: Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion We show that continuous diffusion can achieve very strong language modeling performance when operating directly on bitstreams, outperforming masked and uniform diffusion baselines, and essentially matching autoregressive models under our evaluation settings.
6
37
5
238
34,656
Luca Ambrogioni retweeted
Hot take: the community keeps asking whether diffusion LLMs can beat AR. That's the wrong question. Diffusion is a sampling method, and it can sit on top of an AR model to make it faster instead of replacing it. Speculative decoding already speeds up AR, but it has two long-standing problems. Training a drafter takes a long time, and you have to build a complete pretraining corpus for it. Even then, token acceptance drops sharply on out-of-domain data. Uno @ssahoo_ offers an elegant way out: it decouples sampling training from capability training. There's no separate drafter. The model itself, plus one LoRA, drafts several tokens in a single forward pass, and the LoRA learns to match the model's own next-token distribution, so there's no drafter corpus to build and no labels to collect. We applied this to the Looped Transformer Nanbeige4.2-3B: 1 hour of LoRA training gives a lossless 2× speedup on agentic task. We haven't optimized anything yet, and we believe the acceleration can go much further. The takeaway: diffusion doesn't need to beat AR. Any model that can score its next token can get a fast diffusion mode in an hour. Page: fengsxy.github.io/LoopUno Lora Adapters: huggingface.co/liddlefish/Lo…
3
17
5
86
7,653
Luca Ambrogioni retweeted
For the Navier-Stokes proof: "On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding"
Since I'm now blocked by bluey from replying to this thread, I answer here by quote tweet the openAI construction has more to do with convex integration than anything else. In convex integration, the oscillations are typically built by hand and the iteration procedure creates wild weak solutions. The non-uniqueness construction of Albritton–Brué–Colombo used an unstable vortex to split solutions apart. The Córdoba–Martínez-Zoroa singularities amplify vorticity across scales by strain. In the openAI paper, a fluid instability is clever made so that the flow itself generates the oscillations: they grow by drawing energy from the background shear. The Reynolds stress then becomes the force that creates the collapsing smooth vortex. Using a physical instability to create the stress in this way has no precedent, and it is what produces a smooth force rather than a merely rough one. Some on these ideas are in the literature (e.g. Leibovich-Stewartson 1983) but the way they are combined is definitely insane and definitely nowhere in the literature previously. I also don't know of anyone suggesting that convex integration would play a role in the creation of a blow-up solution like this. On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding: there are several different groups of pde experts working on understanding it who say that they more or less understand now. I expect digestions soon td;lr: anyone saying that this OAI result is unsurprising is bullshitting sorry!
7
16
292
19,925
Luca Ambrogioni retweeted
PhD applications at @GatsbyUCL are open! Full funding available regardless of nationality. Includes ✨new✨ Jump Trading studentships in AI with potential application to finance, eligible for employment opportunities with Jump Trading during PhD. Deadline: 🚨November 9th🚨
Interested in a #PhD in machine learning & theoretical neuroscience? Applications for 2027 entry to the Gatsby Unit PhD programme now open! 💰 Fully-funded 4-year programme 🤝 Close links to Sainsbury Wellcome Centre & the ELLIS Unit @ UCL (@uclcsml) ℹ️ ucl.ac.uk/life-sciences/gats…
1
14
118
5,080
Luca Ambrogioni retweeted
1/5 Continuous diffusion may be better at discrete reasoning than standard sampling suggests. We took SRM's MNIST-Sudoku Hard checkpoint (arXiv 2502.21075) and changed ONLY the sampler: ❗ 0.8% → 65.7% valid boards solved, without retraining. Why does standard sampling fail?🧵
6
28
1
214
8,936
Luca Ambrogioni retweeted
The magic of flow map language models is that they capture dependencies between tokens in just 1-2 model passes. Using this as a drafter for speculative decoding makes a ton of sense, and it's exciting to see this working as well as it does! (Love the shuffle test as well btw!)
Announcing FlashMap: a new speculative decoding algorithm based on flow maps! >6x faster inference on Qwen3-8B 🧵 Project page: jshen.net/flashmap arXiv: arxiv.org/abs/2610.24365 w/ @Jay41969851 @_ryantlee @jonathanmay @cosmo_shirley
4
29
233
11,201
Luca Ambrogioni retweeted
A key difference between continuous and discrete diffusion for language is that the former is (in principle) able to represent uncertainty in intermediate noisy states (i.e. superpositions of possible outcomes). Binary discrete diffusion is another way to achieve this. Neat!
🚨 New paper! What if LLMs could reason like we do when solving Sudoku: keep a few candidates per cell and iteratively narrow them down? Introducing Binary Discrete Diffusion for language and reasoning tasks: it denoises sets of candidate tokens using binary representations! 🧵
6
26
2
187
11,143
Luca Ambrogioni retweeted
I like the intuition here: remove strong local correlations to focus on weaker longer-range correlations, which results in better representations to align to. That same intuition inspired masked-self prediction (arxiv.org/abs/1903.04933) for hierarchical generative modelling.
What makes a good representation for diffusion models? Surprisingly, more information isn't always better. In our new work, Purify Before You Align, we show that removing locally predictable information can substantially improve diffusion alignment. 🎥 A quick overview 👇 🧵 1/n
3
15
1
166
9,971
Luca Ambrogioni retweeted
🎊 Excited to give a tutorial on diffusion LLMs at @NeurIPSConf 2026 with @StefanoErmon @JCJesseLai @mittu1204 📆 Mon, Dec 7, 9 AM AEDT 📍 TikTok Ent Centre (Sydney, Australia) Topics covered: > A beginner friendly introduction to diffusion-LLMs > Designing frontier diffusion-LLMs > Using Continuous diffusion for LLMs
2
26
2
90
7,252
Luca Ambrogioni retweeted
Can flow language models scale to billions of parameters and handle real generative tasks? Introducing 🌊 Lafa (Large flow language model), a flow LM scaled up to 8B that matches AR quality, with 🚀 2.5× AR throughput at 7B. Co-led with @ArnavChavan6 @UdbhavBamba. Joint work with @wognsfjq96 @hongseu33 @AdtRaghunathan @jw9730
9
51
9
217
22,192
Luca Ambrogioni retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
134
774
143
6,351
752,303
Luca Ambrogioni retweeted
We’re hiring a summer research intern at the MIT–IBM Computing Research Lab in Boston. We are looking for a PhD student near graduation. We’re interested in non-autoregressive architectures for language generation and reasoning, including diffusion models, flows, and EBMs.
3
11
1
212
10,180
Luca Ambrogioni retweeted
🎉 Excited to bring our book The Principles of Diffusion Models to @NeurIPSConf 2026 as a tutorial—this time, all about diffusion LLMs! 🚀 🔹 How do ideas from continuous diffusion extend to language generation? 🔹 What are the latest advances and open frontiers in diffusion LLMs? From foundational principles to cutting-edge research. Stay tuned! 📆 Mon, Dec 7, 9 AM AEDT 📍 TikTok Ent Centre (Sydney, Australia) w/ organizers @ssahoo_ @mittu1204 @StefanoErmon
🎊 Excited to give a tutorial on diffusion LLMs at @NeurIPSConf 2026 with @StefanoErmon @JCJesseLai @mittu1204 📆 Mon, Dec 7, 9 AM AEDT 📍 TikTok Ent Centre (Sydney, Australia) Topics covered: > A beginner friendly introduction to diffusion-LLMs > Designing frontier diffusion-LLMs > Using Continuous diffusion for LLMs
2
15
1
73
3,370
Luca Ambrogioni retweeted
Can a language model be based purely on self-conditioning? 🤔 Yes! Meet LAW 👋 (a.k.a. Loopholing All the Way!) 🧵1/8
10
75
12
377
35,581
Luca Ambrogioni retweeted
🚀 Are you a PhD student pushing the boundaries of AI, robotics, graphics, or accelerated computing? NVIDIA's Graduate Fellowship offers up to $60K in funding, mentorship from NVIDIA researchers, and a summer research internship! ⏳ Apply by October 30! blogs.nvidia.com/blog/applic…
6
23
130
14,688
Likewise, some glimpses sometimes
confession: i've yet to have that moment of feeling that frontier LLMs are geniuses, or super-human at ~anything broadly coherent that i deeply prize
4
652
Luca Ambrogioni retweeted
THIS IS DISINFORMATION The PHP exploit Nate is talking about was found by AI-enabled defensive operations. There is no evidence of large scale exploitation. This is a huge win for defense running ahead of offensive.
It's striking that this vulnerability was found by Sol and Argon rather than being found and fixed in the wake of Mythos 6mo ago. Seems like a heavy strike against the "defense will run ahead of offense" theory, especially give that domains like bio are much harsher than cyber.
13
39
4
362
14,180
Luca Ambrogioni retweeted
Here we explore the design space of losses for Strong Stochastic Flow Maps improving FD Dino by 10-50% over the original implementation by using a random ODE approach.
Excited to announce the follow-up to our NeurIPS 2026 Oral: Improved Training of Strong Stochastic Flow Maps! We improve the training of SSFMs by unlocking derivative-based losses! Joint work with @antoniofrancaib @rmincam @ChristianKoke @Niklas_TR and @AlexanderTong7
6
46
2,720
Luca Ambrogioni retweeted
From T. Tao slides:
[Public Lecture] Math 2.0 Terence Tao (October 9, 2026) Slides: teorth.github.io/tao-web/sli… Recording: aimath.org/videos/terry-tao-…
5
3
3
50
57,211
Luca Ambrogioni retweeted
Nice to see that the spherical geometry xitter.cf/jdeschena/status/20546… can also be beneficial for fixed-point reasoners!
A looped transformer has to know when to stop, and its state has to stay bounded while it loops. We introduce Spherical Fixed-Point Reasoners (SFPRM) 🌐: loop on the unit sphere, where each step is an angle, and stop once the angle is small enough. 🧵 (1/9)
7
30
1,441