Space Bunny is here! 🐰 MiniMax M3.1 Flash hits >1K tok/s, bs1 decode with SGLang, production-ready from Day 0. This integration is a joint effort by @MiniMax_AI, @NVIDIAAI, @AMD, and @radixark, and you can also run RL with Miles! -Broad hardware support: NVIDIA Blackwell and Hopper (GB300/B300/B200/H200), plus AMD Instinct MI350X and MI355X. -Agentic serving at scale: ~100K total tok/s per GPU on the AgentX coding-agent trace with 96 concurrent sessions. -Speculative decoding: Supports MiniMax's DSpark draft and a DFlash2 draft trained by @radixark; supports HiCache and multimodal input. -Ultra-low-latency inference: MegaKernels deliver 2.15× faster single-request decode.

Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models. RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale. Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc. Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware. Here is what we built, and why teams picked Miles🧵

🚀 New blog: Advanced CUDA Graph Techniques in Inference and the design story behind them. CUDA Graph removes kernel-launch overhead by recording GPU work once and replaying it. But bringing it into a real inference engine introduces challenges in ops compatibility, graph management, and memory overhead. Together with the @AIatMeta team, we refactored SGLang's CUDA Graph support around a common runner/backend interface to tackle these problems systematically. On top of this architecture, we highlight several techniques first developed in SGLang: 1️⃣Breakable CUDA Graph (BCG), our new default prefill CUDA Graph backend, removes the dependency on torch.compile for faster setup and broader compatibility. 2️⃣Experimental Full CUDA Graph for prefill, first introduced in SGLang for FA4 and FlashInfer, using request padding to enable full graph capture for dynamic prefill workloads. 3️⃣Graph memory reuse across shapes and segments, keeping the CUDA Graph memory footprint fixed and making memory management more predictable as graph coverage expands.

The king of small models is back! Qwen3.8-27B from @Alibaba_Qwen is open source, and Day-0 support is live in SGLang: - 206.1 tok/s decode on a single RTX 5090, with our NVFP4 plus DSpark - 38.28 tok/s decode on DGX Spark Qwen3.8-27B raises the bar again for what a small model can do on agentic planning and long-horizon tasks. Long live the (small model) king! Run it locally with SGLang 👇

🚀 New blog: Unified Radix Cache: One Tree for Hybrid Model Prefix Caching Hybrid models complicate prefix caching: each attention type has its own cache reuse semantics, and specialized cache classes multiply combinatorially, duplicating tree logic as caching features grow. Unified Radix Cache brings everything into one shared tree: 1️⃣ One token-keyed radix topology for canonical prefix identity, with composable components for architecture-specific reuse semantics 2️⃣ HiCache and session-aware eviction are natively supported in the shared tree, built for agentic workloads 3️⃣ A clean separation between the controller and tree core makes the core replaceable, including an experimental Rust tree core implementation One tree, composable cache semantics, and a unified foundation for hybrid model prefix caching.

DSpark for Kimi K3 just got an upgrade: we further improved the model for long context and agentic use cases. - Accept length 4.2 in RULER v2 1M context length - Accept length 4.66 on SWE-rebench - Improvements across most of the benchmarks we tested (one small regression on GSM8K) And a milestone worth celebrating: DSpark crossed 140k downloads in the first few days after launch. Seeing a model we trained get picked up this quickly means a lot to us 🌟 Thank you to everyone who tried it, filed issues, and pushed it into real workloads.

🚀 SGLang now officially supports @thinkymachines Inkling-Small running on 2× @NVIDIAAI DGX Spark systems connected via ConnectX-7. Huge thanks to @realJerryzhou @ispobaoke for implementing the support, with valuable community contributions from @u1tra_instinct and @TechMDAI. This enables strong agentic capabilities to run locally on a compact platform. The current setup achieves 24 tok/s (concurrency=1) without MTP. DSpark support is coming soon. Check out the cookbook below to get started 👇

Inkling-small is out today! With SGLang, you can get 648 tok/s decode with DSpark (simulated acc len=4) and 288 tok/s w/o DSpark, under the same setup (8x @NVIDIAAI B200, TP 8, NVFP4, bs=1). What makes this model different is the size. 276B total with 12B active is a sweet spot for RL, and both LoRA and full-parameter training become well within reach. Miles is ready and verified for multimodal RL on Inkling-small, so you can turn your multimodal data into real capability gains. At ~1/4 the size, Inkling-small matches the bigger version in capability and even wins on some benchmarks. Run Inkling-small with SGLang, and customize it with Miles.