
Maker of *those* diagrams for deep learning algorithms | @mit @mitlids PhD student under @GioeleZardini
Boston
Joined July 2022
- Tweets668
- Following433
- Followers7.6K
- Likes3.5K
Something nice about abstract representations is we can just *derive* where the KV-cache should go to get the inference form. There is an expressible relationship between prefill and inference! The little symbols in the top right switch between modes.
Posted Mixtral here cause it fits, but check out other models here ~~
zardini.mit.edu/diagrams/mix…
Doing an SF trip from Thursday Oct 1st to Tuesday Oct 6th! DM if you’re around and want to talk about GPU software/hardware :)
2
9
1,157
The updated diagrams page with @GioeleZardini lets us "see like a GPU". We can see how attention is usually expressed, but this doesn't give much info. How are we applying the Matmul? Over which axis is the SoftMax operating? (1/4)
But we can do even better! GPUs see everything in terms of axes - they are what we batch, parallelize, stream, and reduce over. Notice how the q-axis is parallelized in each expression? Well, by drawing the axes of arrays, we can immediately see this pattern. (3/4)
All of these diagrams can be found on our updated, interactive website. Switch between the forms at the top of the page to start seeing like a GPU yourself! (4/4)
zardini.mit.edu/diagrams/att…
Big update to the diagrams page :) will make a series of posts with @GioeleZardini
zardini.mit.edu/diagrams/att…
Vincent Abbott retweeted
Check this out!!! 🚀
At long last, the interactive DeepSeek-v4.1-Flash diagram is up ~
vtabbott.io/diagrams/deepsee…
At long last, the interactive DeepSeek-v4.1-Flash diagram is up ~
vtabbott.io/diagrams/deepsee…
damn deepseek v4.1 flash engram got hands, at least at the end of the interactive deepseek v4.1 flash project I'll have an understanding of the entire architecture
18
1,189
Vincent Abbott retweeted
FlashAttention on an actual napkin: very Boston-coded experience with @vtabbott_
arxiv.org/pdf/2412.03317
Working on a diagram for DeepSeek V4.1-Flash. Some really interesting features that I'll write about, the KV compression is next level.
Fable is past the critical point where I can interact with my codebase via plain English commands. The modern gen of models is crazy. Like, we’re at takeoff now. This is it.
The critical point is a competence + reliability. It needs to not make mistakes, not make a mess, and parse instructions well. Opus doesn’t quite make the cut, and Astra is ime VERY spikey. I’m curious about others’ experience though.
1
5
576




















