
🐈 pretraining @MistralAI | pure maths & cs @ENS_ULM | problem setter / half retired competitive programmer
Paris, France
Joined December 2016
- Tweets2.1K
- Following350
- Followers387
- Likes2K
I feel so lucky to see Jon von Neumann's dream come to life
Throughout history, mathematics used to be extremely repetitive. Backprop, VC dimension, KKT conditions, and a lot of other things were independently rediscovered several times. Even with google search, it was impossible to search for math statements and proofs because the difference in notation breaks it.
LLMs solved this problem and now we're sitting on a goldmine of possible results. There has never been a better time to be a mathematician if you care about getting the work done rather than about enjoying doing it all by hand.
3
195
Yann Viegas retweeted
Throughout history, mathematics used to be extremely repetitive. Backprop, VC dimension, KKT conditions, and a lot of other things were independently rediscovered several times. Even with google search, it was impossible to search for math statements and proofs because the difference in notation breaks it.
LLMs solved this problem and now we're sitting on a goldmine of possible results. There has never been a better time to be a mathematician if you care about getting the work done rather than about enjoying doing it all by hand.
Yann Viegas retweeted
is the Anthropic IPO chart a better LR schedule than warmup cosine decay?
let's find out! @mridul__agarwal @ananthsiyer and i are running 4 identical pretraining runs, with the LR driven live by ANT, SPCX, SPY, and NVDA.
watch the models train at stockasticdescent.live
Quoting a wise man: "if you value intelligence above all other human qualities, you’re gonna have a bad time"
idk how to say this but Riemann being solved means the world after today is no longer the same world i grew up in. as a math student, the intellectual excitement of thinking about problems like this, the impact - that a lot of scientific workers attach their life’s meaning to - is gone. frontier scientific discovery now is just like tapping on a calculator.
quoting a friend: after today, math class will become PE class. It’s only a way for kids to exercise their brains, a skill like a language, but no longer an essential subject to pursue and devote your life for cuz "you were making progress for humanity". lots of people’s life’s work is now just ashes in history.
this marks a milestone where we can pretty much believe AI has been proven to be smarter than humans in any capacity, including frontier scientific discovery; and it will keep growing to be smarter and better at everything humans can do. physical ai is next.
so many, many de facto social norms and foundational beliefs about a man's value are being redefined by AI progress. our societal value system - what can you do as a human that is unique, what’s your value to society and humanity as a whole and to others around you. what is fulfilling to achieve, when AI now can do everything better than us? we must soul-search for new meaning as a species. and before that consolidates, there will be riots and an uneasy phase. the progress brought by ai can be disruptive to our society's current structures, economy and labor market. people will need to reskill, and regimes need redesign on how value is distributed. i get the doom scenario now - humans may become 2nd class citizens and ai can enslave us if they find a reason to. how are we gonna compete with them in any labor or intellectual game?
13
643
Yann Viegas retweeted
Yep
Yann Viegas retweeted
Wow. It looks like Grigori Perelman is going down in history as the only human to ever solve a Millennium Prize problem.
Still waiting for a useful deep learning theory result to be proven by AI !
Why I think it didn't happen yet: it's incredibly tricky to design a toy model simple enough to be tractable yet complex enough to capture what happens at scale.We don't even have much human data on this
Yann Viegas retweeted
Dmitry Kovalev, who did his phd at the same time as me at kaust, once said
"if you wake me up in the middle of the night and ask me to give a proof convergence for Nesterov’s accelerated gradient method, I will first give you a piece of my mind and then I’ll write down the proof"
Yann Viegas retweeted
It's sadly appropriate that the founding father of the GPU seems to have been banging mistresses in parallel, rather than sequentially.
Language models process each token through a single residual stream (blue arrows): reading / writing memory and applying MLPs at each layer, and finally decoding the next token.
Can we scale up this single stream computation?
Introducing Latent Lattice Language Models (LLLMs): augmenting this vertical stream with a horizontal axis of computation—the Lattice of Latents (orange arrows).
RNN 🤝 Transformers 🧵
2
491
Yann Viegas retweeted
please help me farm aura on goodfire
New neurons research: introducing Neuron Communities.
We discovered communities of neurons that work together inside LLMs. They’re surprisingly small, but powerful – just a few neurons work together to perform common behaviors, like arithmetic, ALL CAPS, or recalling a date.
Work led by research fellow @_igorshilov:
Yann Viegas retweeted
Replying to @ziv_ravid
You forgot to compare against what everyone uses arxiv.org/abs/2506.02285
Yann Viegas retweeted
You can 2x Qwen-0.8b's performance on Math-500 by adding Gaussian noise to the last 4 layers.
RSI is here 🔥🔥🔥 and it's Gaussian!
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Yann Viegas retweeted
The year is 2050. The math departments are full of 80+ year old emeritus professors prompting the machines. Professor Gigachad (75) walks into the a16z Department of Compute down the hallway to show the VC kids his latest finding that will revolutionize mathematics.
Yann Viegas retweeted
Replying to @xuanalogue
fyi I don't think it's super surprising to me. I & coauthors explained a three-layer neural network weight in full in 2023 (2306.17844); I think the demos here are much easier than that (e.g. reading what positions have large weights). Goodfire's silico for example is probably more capable due to their tools. Still a great benchmark anyways!





























