Sam Acquaviva (@Sam_Acqua) header
Sam Acquaviva (@Sam_Acqua) profile picture

now @ spacexai, ex-cursor, after thought, cocosci @ mit

Joined February 2014
Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)Photo by Sam Acquaviva (@Sam_Acqua)
"Do not panic or hallucinate like a fragile Large Language Model."
3
9
1
157
5,622
Sam Acquaviva retweeted
Modded-NanoGPT optimization result #14 (2026/05/04): @Sam_Acqua has achieved a new record of 3150 steps (-60), by adding SOAP preconditioning before Muon orthogonalization for the MLP weights (SOAP-Muon).
3
5
1
75
6,038
Sam Acquaviva retweeted
New NanoGPT Speedrun WR at 84.3 (-0.3s) from @Sam_Acqua, who found that Muon Orthogonalization on Q and K performed better when ran over pairs of heads instead of on the full 6-head weight matrix. (many people had previously gotten negative results with single-head ortho). github.com/KellerJordan/modd…
5
10
1
87
8,675
Sam Acquaviva retweeted
New NanoGPT Speedrun WR at 86.8 (-0.4s) from @.samacqua on Github, by tuning and reusing the transpose_copy kernel during the cross entropy backward calc. Outside the main speedrun track, Sam did an interesting experiment in Jan showing how test-time training can improve perplexity. github.com/KellerJordan/modd…. github.com/KellerJordan/modd…
2
14
78
7,521
Sam Acquaviva retweeted
New paper: World models + Program synthesis by @topwasu 1. World modeling on-the-fly by synthesizing programs w/ 4000+ lines of code 2. Learns new environments from minutes of experience 3. Positive score on Montezuma's Revenge 4. Compositional generalization to new environments topwasu.github.io/poe-world [1/n]
17
103
14
567
58,985
I've recently started my job as an asst professor at NTU, Singapore. If you are ever in town come say hi :)
28
11
1
672
37,973
Sam Acquaviva retweeted
1/7 Wondered what happens when you permute the layers of a language model? In our recent paper with @tegmark, we swap and delete entire layers to understand how models perform inference - in doing so we see signs of four universal stages of inference!
21
89
17
544
121,759