Suvinay Subramanian

Introducing Learned Asynchronous Decoding w/ friends from MIT/Google! LLM responses often have chunks of tokens that are semantically independent. We train LLMs to identify and decode them in parallel, speeding up inference by 1.46x geomean (AlpacaEval) w/ only 1.3% quality loss.

Really excited to see Midjourney enter physical AI. Ultrasound is the ripest candidate for expanding medical imaging beyond the hospital. Full-body ultrasound is a new modality, but the design inherits the limitations of Butterfly's sensing architecture, which may limit its clinical usefulness and potential for a data flywheel. To truly scale medical imaging out of the hospital and to a billion people, we need to do much better. I think that's a solvable problem, but it requires starting with a fundamentally different physical design. teapaw.org/ergodicthought/status/…

Asynchronous decoding: multiple LLM threads write different parts of an answer in parallel. In Feb we (MIT×Google) introduced PASTA—the first async-dec method that uses policy learning to optimize latency & quality end-to-end. See us @ E-2600, East Hall A-B, Tue 11pm #ICML.

📣 The Journey Matters: Our #ICLR2025 paper shows how to pretrain sparse LLMs with half the size of dense LLMs while maintaining quality. We found that the average parameter count during sparse pre-training predicts quality, not final size. An MIT/Rice/Google/ISTA collab 🧵 1/N

Breaking news: Google is winning on every AI front. This is not just about Gemini 2.5 but about a reality that OpenAI and Anthropic fans have ignored for too long. Here's a non-exhaustive list: - Gemini 2.5 Pro is the best model in the world according to benchmarks, vibe checks, high-taste testers, and firsthand testimonies. It's also fast and cheap compared to similar models (Google offers it for free on the Gemini app!) - Gemini 2.5 Flash (to be announced soon) is much faster and much cheaper, so it captures perfectly the Pareto frontier of cost-performance of cost-efficient models. - Gemma 3 is a highly competitive open-source model, as good or better than Llama 4 and DeepSeek models. - That's just LLMs. Google is world-class in image (Imagen 3), video (Veo 2), voice (Chirp 3), and music (Lyria). They're integrating them all in Vertex AI. - Deep Research with Gemini 2.5 Pro is *twice as good* as OpenAI's Deep Research, according to human testers. Other agents? Yes: Project Astra (assistant) and Project Mariner (computer interaction) - They just launched Agent2Agent, compatible and complementary to Anthropic's MCP, which they will build in-house as well. - And they keep publishing papers in top journals (Nature) and going to the top conferences (ICLR, NeurIPS), whereas others jealously keep their most important stuff for themselves. - That's just the AI stuff, but Google is also a consumer software company with seven 2+ billion monthly users: Search, YouTube, Gmail, Android, Chrome, Maps, and Play Store - A hyperscaler (Google Cloud) - A hardware company (TPUs, Ironwood) - And a phone company (Pixel). How can OpenAI or Anthropic or even Meta fight such a beast? Let’s wait for their responses to this. I’ll be here to cover any newsworthy release—even if I’ve already made my bet on who’s most likely to win. (Read the full post in the link below.)

Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n

Please consider submitting your best work. MLArchSys is the best place to showcase your work at the intersection of ML, Computer Architecture, and System. Check out the call for paper and look for new topics we included this year 🚀🔥 sites.google.com/corp/view/m… 1/3

We're excited to launch the 𝐀𝐫𝐜𝐡𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 𝐂𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐞 (bit.ly/4idtYN6). Design complex, reasoning-based questions that expose the current limitations of LLMs and contribute to the broader effort of improving AI reasoning for comp. arch. and systems.