David Dorman

Is the compute market still the Wild West? What will happen to GPU prices over the next few years? And how will the inference market evolve? @bernhardsson, co-founder and CEO of @Redpoint portfolio company @modal, was the perfect person to discuss this and more. Modal has crossed a $1B revenue run rate and recently raised $750M at a $15.75B valuation. A few takeaways from my conversation with Erik, alongside @patrickachase: ▪️ Why compute won’t become a commodity GPU capacity varies in disk, networking, location, and especially reliability. That variation makes a true exchange hard to build, so much of the market still runs on late-night WhatsApp groups and hallway deals at events like GTC. Erik expects it to get somewhat more efficient over time, but to stay structurally messy. ▪️ NeoClouds, NeoLabs, and the cost of capital Erik disagrees with the idea that NeoClouds are going to die, and he sees them as sound businesses built on leasing hard assets against locked-in contracts. Their real disadvantage is cost of capital, which is why the successful hyperscalers all sit inside cash-rich public companies. NeoLabs carry a different risk. GPU deals often require only about 30% upfront, so a lab that raises $1B can commit to far more compute than it can pay for today. If funding tightens, Erik expects some NeoLabs to resell capacity and start to look more like NeoClouds. ▪️ Where durable advantage lives in inference Erik sees pure token serving as a race to the bottom, with routers already commoditizing the underlying providers. Better kernels and smarter KV caching still matter, but those techniques spread quickly. He believes the more durable edge comes from offering a cluster of adjacent building blocks. ▪️ Who is actually adopting coding agents Erik expected junior engineers to adopt agents first, but he has seen the opposite. For senior engineers, agents are like having 10 interns. Give one a full POSIX test suite and it can write a new file system in three days. He also sees training data creating a new kind of incumbency in software, with developers defaulting to frameworks like React because agents already know them well. ▪️ Why GPU prices may go up Erik changed his mind on this 48 hours before we recorded. After reading Anthropic’s S1, he concluded that the largest buyers of compute are financially strong enough to keep acquiring capacity, which points to higher prices over the next 12 months. Full Unsupervised Learning Episode 👇 0:00 Intro 1:26 The Compute Market Today 5:16 Modal’s Path to LLM Inference 10:02 Building the Hard Infrastructure 12:32 Unsolved Problems in Inference 16:07 Chips & Model Sizes 21:21 What Wins in Inference 24:56 Prioritizing a Multi-Product Roadmap 30:20 Agents as Modal’s Users 33:19 The Next Big Bet After Sandboxes 38:58 Robotics & Multimodal Inference 41:04 Scaling From $2M to $1B 42:51 NeoClouds & NeoLabs 48:15 RL as a Service 51:40 Quickfire YouTube: youtu.be/HJDekXH-h5c Spotify: bit.ly/4dZ0O4e Apple: bit.ly/4inZJnE