we launched the most comprehensive ai performance engineering repo in the world now we'll be posting every single resource follow and save to keep up with the series. links in thread 🧵 part 5: Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures the authors introduced a model for relating arithmetic throughput, memory bandwidth, and data reuse. applied to AI workloads, it gives a way to evaluate batching and kernel optimizations against the hardware's limits. the paper's concepts connect to these AI performance decisions: operational intensity measures floating-point work per byte of DRAM traffic after cache reuse. during dense transformer decoding, a linear layer processing a small token batch can read a large weight matrix for little arithmetic. compute and bandwidth roofs bound throughput. if weight reads limit an inference kernel, reducing transferred bytes can lower its memory-time bound. increasing peak arithmetic throughput alone leaves that bound unchanged. the ridge point marks the minimum intensity needed for peak compute throughput to be possible. batching tokens that share a weight matrix can increase work per weight byte, moving the matrix multiplication toward that threshold. computation ceilings account for limits in instruction parallelism and operation mix. for neural-network matrix multiplication, check Tensor Core use and compare throughput with the roof for the kernel's execution path and precision. bandwidth ceilings account for memory access patterns and data placement. for GPU tensor operations, changing the layout or thread-to-data mapping to coalesce scattered accesses can improve useful bandwidth. data reuse raises intensity when it reduces DRAM traffic for the same arithmetic work. a tiled matrix multiplication can reuse operands on chip; measure whether that reuse reduces DRAM byte traffic. the memory level determines which bytes to count. if a matrix multiplication reuses operands from L2, compare its L2 traffic with L2 bandwidth and its DRAM traffic with DRAM bandwidth. the authors demonstrated the model with 4 floating-point kernels on 4 multicore systems. for AI performance work, use it to assess weight reuse, memory layout, and arithmetic execution before choosing an optimization to test.