kvtop (@kvtop) profile picture

kvtop

@kvtop
Joined October 2026
  • Tweets1
  • Following3
  • Followers0
When is it better to fetch a KV cache than to recompute it? New on the LMCache blog: an Internet for the KV cache, by Siddhant Ray, adapted from his arXiv paper with Nick Feamster and Junchen Jiang of the University of Chicago. One example: DeepSeek-V4-Pro, about 100,000 tokens, about 1 GB of compressed KV cache. Sending it over a 10 Gbps link takes about 0.8 s and $0.09. Recomputing it on AWS 8×B200 takes about 15 s and $0.47. Under the post’s assumptions, transfer is about 19× faster and 5.2× cheaper ⚡ (AWS prices are examples only.) blog.lmcache.ai/en/2026/10/2…
An Internet for the KV Cache: Store Anywhere, Use Everywhere

An Internet for the KV Cache: Store Anywhere, Use Everywhere

Sending a 1 GB KV cache over the Internet can be 19× faster and 5.2× cheaper than recomputing it. A proposal to store it anywhere, use it everywhere.

blog.lmcache.ai
11
14
7
144
11,731
Load more