Your open-source AI ally. We are committed to making production-grade AI inference as accessible and reliable as electricity, powered by vLLM.
Joined October 2023
- Tweets620
- Following1.4K
- Followers1.2K
- Likes988
Pinned Tweet
Tomorrow at #SFTechWeek with @tensorwave, @CosmicLabsTech, @weka and @makora. Hardware, watts, kernel and token.
Silicon doesn't serve itself. That's the layer we run.
10am, SF. If you're shipping tokens, meet our team.
Our @Techweek_ Inference at SCALE panel is growing!!
Excited to have our friends from @makora, @weka, and @EmbeddedLLM joining our @tensorwave x @CosmicLabsTech teams for a conversation around hardware, inference, market dynamics and more.
partiful.com/e/Xb2BSFjKC6Wtw…
EmbeddedLLM retweeted
Outstanding work by the team!!! Optimizing World Action Model on AMD GPU by combining kernel level optimization via GEAK (github.com/AMD-AGI/GEAK), fewer denoising steps and compact action expert to reduce the latency from 222.55ms to 8.83ms. @AIatAMD
Fully open
📝 Blog: rocm.blogs.amd.com/artificia…
💻 Code: github.com/AMD-AGI/faster-ac…
🤗 Checkpoint: huggingface.co/collections/a…
EmbeddedLLM retweeted
High throughput on the grill. Zero latency on the drinks.
vLLM maintainers and friends gathered during the PyTorch Conference for an evening of Brazilian steak, drinks, and great conversation! 🥩🥂
Grateful for this community and everyone who keeps vLLM moving forward. 🚀
EmbeddedLLM retweeted
vLLM is all you need to reproduce DeepSeek's performance.
🚨🚨🚨 vLLM is all you need to reproduce DeepSeek’s performance.
With Wide-EP and EPLB, vLLM can match DeepInfer’s decode performance within 6% and generate $39/gpu/hr in revenue at DeepSeek’s off-peak pricing, at 200 P90 TPS on the AgentX workload.
Specifically:
• Under DeepSeek’s decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 achieves throughput within 6% of DeepInfer’s published results across batch sizes 8–256.
• On AgentX, vLLM pushes the Pareto frontier further on GB300, reaching up to 1M tokens/s/GPU and generating $38.96/gpu/hr in revenue at 196 P90 TPS, based on DeepSeek’s off-peak API pricing.
• Also, vLLM extended low-latency kernel support for DeepSeek-V4.1-Flash, achieving up to 826 TPS per user on AgentX.
We helped close the performance gap.
@SemiAnalysis_ Kimi K3 estimator puts MI355X at 36% higher revenue per chip-hour than HGX B300, with a 38.8% profit margin vs 7.7%.
Performance = profit.
🚀 @AMD MI355X just showed competitive performance to NVIDIA B300 on @Kimi_Moonshot K3 serving!
8× MI355X delivers 24.4% more throughput than HGX B300 (7,567 vs 6,083 tok/s/GPU) at 50 tok/s/user on @SemiAnalysis_ AgentX agentic coding benchmark, and 87.4% more tokens per dollar.
We cover the vLLM optimizations, AMD kernels, and hardware details that closed the gap, plus patterns for spotting bottlenecks so you can get more out of vLLM on your own GPUs.
Thanks @AMD @embeddedLLM @SemiAnalysis_
Full writeup: 🔗 vllm.ai/blog/2026-10-16-kimi…
EmbeddedLLM retweeted
🎉 Step 5 Preview has Day-0 support in vLLM.
Sparse attention, MoE, multi-token prediction and vision, with serving work across Blackwell and Hopper.
The work spans the sparse indexer, FP8 MegaMoE and cache transfer. Here is how the pieces fit together, with kernel results and validation below. 🧵
1/6
EmbeddedLLM retweeted
🎉 Day-0 support for Ling-3.1-flash is here in vLLM!
~560B total parameters, ~25B active per token, and up to 1M context. From a rough brief to websites, mobile UIs, and interactive prototypes—serve it on your own infrastructure with vLLM.
Congrats to @AntLingAGI!
Building the future of AI on @AMD takes the whole stack working in lockstep.
From @tensorwave's Instinct GPU clusters and @weka's data layer to our work at @EmbeddedLLM pushing vLLM on ROCm and TokenVisor into production, the shift from raw GPU hours to governed, high-throughput inference and agent services is happening right now.
Great sharing the stage with @CosmicLabsTech, @makora_ai, @miguelsaavedra, and Prithvi Mattur during #SFTechWeek! 🚀
Thank you to everyone who came out for our “From Training to Inference” panel with @CosmicLabsTech, @makora_ai, @EmbeddedLLM, and @weka during #SFTechWeek.
It was a packed room and a great conversation about where AI compute is heading, what builders need now, and how the infrastructure behind it all is evolving.
Shoutout to our speakers @miguelsaavedra and Prithvi Mattur!
ALT “From Training to Inference” panel with Cosmic Labs, Makora, Embedded LLM, and Weka.
EmbeddedLLM retweeted
ALERT🚨: AMD JUST LANDED SUPPORT FOR MEGAKERNEL DECODE IN UPSTREAM VLLM WELL BEFORE NVIDIA. This marks one of the first times that a major vLLM feature is only supported on ROCm and not on CUDA yet.
MegaKernels is a technique that is massively important for high-interactivity decoding by reducing the # of kernel launches by 10x.
This marks an❗️important milestone❗️for ROCm vLLM to land major ROCm-exclusive features. CUDA vLLM doesn't even have draft PRs for upstream vLLM megakernel support yet.
EmbeddedLLM retweeted
It was an incredible honor to receive the National Medal of Science from @POTUS today.
My parents came to America with big dreams and a deep belief in education and hard work. This country gave me the opportunity to pursue my passion for semiconductors and work with extraordinary people to push the boundaries of what is possible.
I am so excited about what we can accomplish together as America continues to lead the world in science and technology.
.@POTUS presents the National Medal of Science to Sergey Brin, @JensenHuang, @elonmusk, and @LisaSu, and the National Medal of Technology and Innovation to @MichaelDell and @satyanadella.
Meet us at #COLM2026 in San Francisco.
Oct 9 at DAIH: our research on deploying clinical AI where the data lives across Nvidia and AMD GPU, with energy and cost in view.
Building models or putting them into production? Come say Hi! @manpree59181175
DM us to meet.
EmbeddedLLM retweeted
AMD MI355X posts the best margin on every model in SemiAnalysis InferenceX agentic profit estimates:
Kimi K3: 45.7% vs 43.8% (GB300)
GLM 5.3: 63.4% vs 63.2% (GB200)
MiniMax M3: 51.5% vs 48.4% (B300)
DeepSeek V4 Pro: 87.9% (no green data)
inferencex.semianalysis.com
Massive respect to @AMD, @roaner and @AnushElangovan for investing where it counts: dedicated hardware, CI, and the room for core maintainers @Rxday000 to push ROCm to its limit.
The partnership is in a better place than it has ever been. 502 commits is the baseline. When software and silicon move at the same speed, the whole open source ecosystem wins. Proud of what we’re building together on @vllm_project.
#1 company contributing code to @vllm_project right now?
@AMD: 502 commits to core vLLM in 90 days. 13% of all org contributions, almost 4x Nvidia.
Also grateful to Red Hat, IBM, Embedded LLM & Inferact for building it with us.
Open source wins when hardware has a choice.
EmbeddedLLM retweeted
Zero mismatches. Bit-for-bit aligned. 🔥
Incredibly proud of our team's work integrating with vime @vllm_project. Rock-solid RL pipelines are now a reality on @AMD ROCm.
Read the deep dive below and star our repo⭐ github.com/RL-Align/RL-Kerne… – at Singapore
Training and rollout logprobs matched bit for bit on ROCm. The @RLKernel team integrated RL-Align/RL-Kernel with vllm-project/vime. A 200-step Qwen3-8B GRPO run on 8× @AMD MI300X recorded zero logprob mismatches between Megatron training and vLLM rollout.
The strict path aligns reduction order, intermediate precision, rounding points, and math primitives across both sides.
Deep dive:
vllm.ai/blog/2026-09-14-rl-k…
EmbeddedLLM retweeted
Training and rollout logprobs matched bit for bit on ROCm. The @RLKernel team integrated RL-Align/RL-Kernel with vllm-project/vime. A 200-step Qwen3-8B GRPO run on 8× @AMD MI300X recorded zero logprob mismatches between Megatron training and vLLM rollout.
The strict path aligns reduction order, intermediate precision, rounding points, and math primitives across both sides.
Deep dive:
vllm.ai/blog/2026-09-14-rl-k…
EmbeddedLLM retweeted
Massive shoutout to TJ & Team at @EmbeddedLLM; they are the lead maintainers & CODEOWNERS of vLLM ROCm! Up until May 2026, they did not even have persistent access to an MI355X cluster, but after SemiAnalysis worked with & convinced AMD leadership of something that is quite obvious in hindsight, they finally got a persistent cluster to make MI355X great on AMD!
Super proud of the @EmbeddedLLM Team, highly respected within the @vllm_project community.
EmbeddedLLM retweeted
🎉 Congrats to @XiaomiMiMo on MiMo-V2.6: day-0 support in vLLM for both sizes.
Two MoEs with text, image, video and audio in one checkpoint: Pro is 1.02T total with 42B active, Flash is 309B with 15B active. 1M context, native FP8 weights. Built-in DFlash spec decoding drafts 7 tokens a step.
Same architecture class as V2.5, and the serving pieces carry over: hybrid sliding-window plus global attention, the mimo reasoning and tool-call parsers, DFlash spec decoding.
Thanks to the @XiaomiMiMo team for opening the weights and the RL stack! 🙌
🔗 recipes.vllm.ai/XiaomiMiMo/M…
Introducing Xiaomi MiMo-V2.6 — Pro & Flash.
Frontier intelligence, all the modalities, built in public.
🔹 Two omnimodal models, advancing through scaled reinforcement learning
🔹 Pro performs on par with Claude Opus 5 and GPT-5.6 Sol across most agent benchmarks
🔹 Pro scores 46 on the Artificial Analysis Intelligence Index — the highest among open-source models
🔹 Stronger coding, computer use, 3D reasoning and creative capabilities
🔹 Open model weights, technical report, RL environments and training code
Blog：mimo.xiaomi.com/mimo-v2-6
The best kept secret on the industry. Profit margin exceeds GB200 NVL72 and B300.
Proud of our team and our partners at @AIatAMD!
AMD ALERT🚨🚨: On Kimi K3 2.8T, AMD has even better profit margins per gigawatt compared to B300, GB200 NVL72, and B200. This is due to AMD's lower TCO and better performance, leading to better profit margins of 53.6% on Mi355X compared to 44.3% on GB200 NVL72. This is from our open-source Agentic Inference benchmark, AgentX.
Shoutout to @roaner & @EmadBarsoumPi 10x engineers for optimizing this 2.8T model!
EmbeddedLLM retweeted
10 things to know before joining the AMD AI Developer Program 👇
1. It’s free. Anyone can join, whether you are a student starting out or a seasoned ML engineer.
2. You get $100 in cloud credits on day one. Enough to prototype, benchmark, and get moving on AMD tech.
3. You don’t need any hardware to start. Build immediately with cloud access.
4. There’s a private Discord with a community of engineers who will provide technical discussion and help.
5. You get early invites to dev events and workshops. Before they’re public. Before they fill up.
6. Monthly Ask the Expert hours with open-source partners. Ask questions. Go deep. Learn the tricks of the trade.
7. Your projects can get featured on official AMD Developer channels and at events.
8. You’re automatically entered into monthly hardware raffles. No forms. No extra steps.
9. You get a free month of @DeepLearningAI Pro. High quality courses for you to expand your skills.
10. The more you participate, the more you unlock. More access, more visibility, more opportunities.
If you’re building, training, or optimizing AI models, we’d love to have you join us!