Prithvi Dixit

We’ve been assuming that extreme attention sparsity must destroy model quality. That assumption is now false. New SkyLight Release: vAttention ⚡️ We added vAttention to the Tier1A leaderboard. The data is striking: 👑 #1 Practical Method: Surpasses the previous lead (PQCache), closing the gap to dense models within 1% at up to 10x sparsity.. 👑 Saturates Benchmarks: Dominates the sparsity–quality frontier (w/ oracle top-k). 💡 Introducing Verified-X: A new paradigm for reliable inference-time sparsity. In this week’s blog, we share an in-depth look at the algorithm and how to run it via SkyLight 👇

Selecting the top-k highest scores is the intuitive standard for sparse attention. However, our latest SkyLight Tier 1A benchmarks show a non-trivial gap between Oracle Top-k and dense baselines at high sparsity. PQCache currently tops the leaderboard as the strongest practical method, closing the gap between approximate-top-k and oracle-top-k for upto 10x sparsity levels. Closing the gap between dense and practical sparse attention methods will clearly require new ideas. In this week’s blog, PQCache authors share an in-depth look at the algorithm, and how to run it via SkyLight 👇

We are open-sourcing the full Tier-1A implementation and the first standardized leaderboards today. We invite the community to benchmark your methods against the frontier, prove your improvements on the leaderboard, and help us close the gap between sparse and dense. 👉 Leaderboard: sky-light.eecs.berkeley.edu 👉 Code: github.com/skylight-org/spar… 5/N 🧵

Sparse attention offers a viable path to solving the computational and memory issues of long-context LLM serving. However, we observe a surprising gap between the rapid pace of academic research and the reality of open-source adoption. We identify the core barrier as fragmentation where new methods are published rapidly, but with missing comparative grounding. Today we’re excited to share SkyLight, where we are building a unified framework designed to bridge this gap. SkyLight standardizes the implementation and evaluation of sparse attention to enable rigorous, controlled benchmarking. Our first major effort is to understand the frontier of inference time sparse-decoding in LLMs. 1/N 🧵

To learn more about SkyLight check out 👉 Blog: sky-light.eecs.berkeley.edu/… This is a joint collaboration from Sky Computing lab with  @apd10desai ,  @kumarkagrawal, @pd172945, @luisgschroeder  @profjoeyg ,  @matei_zaharia , @istoica05  and @RethinkingAILab