David Wang retweeted
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
David Wang retweeted
DFlash: Keep Drafting Parallel
My talk at @modal Runtime in SF on Oct 1. The drafter was the serial bottleneck, so DFlash made it parallel. That moved the bottleneck to selection, so DFlash 2 fixed that.
DFlash 2 is here! Qwen3.8-27B at 70 tok/s on an M5 Max MacBook Pro.
⚡ Up to 4.6× the speed of autoregressive decoding, with the same output.
This is the next generation of DFlash, seeded at Z Lab and upgraded at Inco AI. Get one more accepted token on every pass, for free!
inco.ai/blog/dflash2/
David Wang retweeted
DFlash to the moon(shot)!
David Wang retweeted
Frontier models require frontier speculators. Our team moved mountains to achieve 460 tok/s with our custom DFlash model.
Try it day 0 at modal.com/endpoints.
David Wang retweeted
Kimi K3 is live on Modal.
Moonshot has shipped the world's first open 3T-class model, and we're a day zero launch partner.
We trained a custom DFlash speculator for K3's novel architecture so you can run it faster, losslessly. The most capable open model we've worked with by far.
David Wang retweeted
Inkling by @thinkymachines is now available on Modal, backed by a custom DFlash speculator for 67% higher throughput and interactivity.
Running on Modal Auto Endpoints with SGLang today.
Today, we are introducing Inkling.
Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.
thinkingmachines.ai/news/int…
Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
David Wang retweeted
Speculation Is All You Need.
In this blog post, we announce the co-release (w/ Z Lab) of six more state-of-the-art DFlash speculators for @Alibaba_Qwen 3.x.
Over 1k output tps for 3.5 122B-A10B on a B200.
Read the blog for why we're all-in on spec dec.
modal.com/blog/spec-is-all-u…
David Wang retweeted
Try them out! 🚀 Really enjoyed the amazing collaboration with the Modal team. The retrained DFlash draft models now support longer contexts, making them a better fit for agentic workloads.
Speculation Is All You Need.
In this blog post, we announce the co-release (w/ Z Lab) of six more state-of-the-art DFlash speculators for @Alibaba_Qwen 3.x.
Over 1k output tps for 3.5 122B-A10B on a B200.
Read the blog for why we're all-in on spec dec.
modal.com/blog/spec-is-all-u…
David Wang retweeted
🚀 New blog: The next generation of speculative decoding: DFlash and Spec V2
DFlash + Spec V2 hit >4.3X baseline throughput for LLM inference, now the default speculative decoding engine in SGLang! Together with @modal and z-lab.ai, our jointly-released DFlash drafter for Qwen 3.5 397B-A17B beats both baseline and native MTP in every setting we benchmarked:
1️⃣ >4.3X baseline & 1.5X native MTP throughput (concurrency 1, HumanEval, 8xB200)
2️⃣ Block diffusion drafter: a full token block in one forward pass
3️⃣ KV injection: target-model features fed into every draft layer’s KV cache for higher acceptance
4️⃣ Spec V2 overlap scheduler: +33% end-to-end
Read the code, deploy a DFlash server, and start experimenting!
David Wang retweeted
🚀 DFlash now runs on SGLang's new default speculative-decoding engine, Spec V2.
⚡️ Hitting >4.3× baseline throughput (1.5× over native MTP) on Qwen 3.5 397B-A17B. Same quality, more speed!
⭐ github.com/z-lab/dflash
David Wang retweeted
Great to see DFlash is being used in MiMo!
🚀 1,000+ TOKENS/S ON A 1T MODEL! 🚀
We are thrilled to release Xiaomi MiMo-V2.5-Pro-UltraSpeed in collaboration with @TileRT_AI , breaking the 1,000 tokens/s output speed on a 1 Trillion parameter model for the FIRST TIME!
Not wafer-scale integration like Cerebras. Not pure on-chip SRAM chips like Groq. We achieve 1,000 tps on a 1T MoE model using just a SINGLE, STANDARD 8-GPGPU NODE.
Read the full technical deep dive：mimo.xiaomi.com/blog/mimo-ti…
Want to experience the future of real-time AI?
👉 Apply for UltraSpeed now: platform.xiaomimimo.com/ultr…
⏳ Limited-Time Access: Application-based · Jun 8 – Jun 23 (PDT)
💬 Chat Experience: Completely FREE for a limited time — try the blazing-fast web chat now.
⚡ UltraSpeed API: Just 3x the price for a ~10x boost in output experience.
🤝 Enterprise & Large-Scale Needs: [email protected]
David Wang retweeted
Amazing to see DFlash helping push Xiaomi MiMo to 1,000+ tokens/s on a 1T model.
Happy to see our work supercharging one of the best open-source LLMs! 🚀
🚀 1,000+ TOKENS/S ON A 1T MODEL! 🚀
We are thrilled to release Xiaomi MiMo-V2.5-Pro-UltraSpeed in collaboration with @TileRT_AI , breaking the 1,000 tokens/s output speed on a 1 Trillion parameter model for the FIRST TIME!
Not wafer-scale integration like Cerebras. Not pure on-chip SRAM chips like Groq. We achieve 1,000 tps on a 1T MoE model using just a SINGLE, STANDARD 8-GPGPU NODE.
Read the full technical deep dive：mimo.xiaomi.com/blog/mimo-ti…
Want to experience the future of real-time AI?
👉 Apply for UltraSpeed now: platform.xiaomimimo.com/ultr…
⏳ Limited-Time Access: Application-based · Jun 8 – Jun 23 (PDT)
💬 Chat Experience: Completely FREE for a limited time — try the blazing-fast web chat now.
⚡ UltraSpeed API: Just 3x the price for a ~10x boost in output experience.
🤝 Enterprise & Large-Scale Needs: [email protected]
David Wang retweeted
Added a fun lil widget to the LLM Engineer's Almanac -- a "Token Timing Simulator" so you can get a visceral feel for what a benchmark perf number means.
Here's @_dcw02's latest work with @zhijianliu's DFlash technique in @sgl_project -- ~1k TPS!
modal.com/llm-almanac/token-…
David Wang retweeted
DFlash⚡ meets OpenClaw🦞 = FlashClaw
Same Claw. >4X faster or cheaper.
DFlash support for Qwen3.5 is live — outperforming native MTP by up to 2.3X.
More to come! 🔥
David Wang retweeted
Check out our paper and code!
Follow our Hugging Face — more draft models for popular LLMs coming soon 🚀
HF: huggingface.co/collections/z…
Code: github.com/z-lab/dflash
Paper: arxiv.org/abs/2602.06036
The paper is now available: huggingface.co/papers/2602.0…
More updates coming soon!
David Wang retweeted
The paper is now available: huggingface.co/papers/2602.0…
More updates coming soon!
Holiday cooking finally ready to serve! 🥳
Introducing DFlash — speculative decoding with block diffusion.
🚀 6.2× lossless speedup on Qwen3-8B
⚡ 2.5× faster than EAGLE-3
Diffusion vs AR doesn’t have to be a fight.
At today’s stage:
• dLLMs = fast, highly parallel, but lossy
• AR LLMs = accurate, sequential, but slow
DFlash = diffusion drafts, AR verifies.
