Gustavo Rivero

Some thoughts on AI ROI, which I believe is misunderstood and severely understated. TL;DR: AI ROI is inflecting for end customers (most important) and infra providers. Pure model provider ROI is less clear, but models will keep improving regardless, driving the industry forward. Of all the debates on AI, the biggest and most persistent is around ROI. The problem is that the debate revolves around a single, inconsistently applied definition, when in reality there are three: 1) ROI for model providers, 2) ROI for model customers, and 3) ROI for infrastructure providers While cooperative, #1 and #2 are fundamentally at odds with one another. All else equal, the higher the margin a model provider captures on the tokens it sells, the lower the margin the customer of those tokens captures on the work being done. This is driving enterprises to rationalize spend where they can: if a job can be done as effectively with a cheaper model, the ROI on that job is proportionally higher than if it were done with a more expensive model. That’s not 'bad' for AI. It just represents margin shifting from the provider to the customer. Meanwhile, the ROI for infrastructure providers is a collective function of both. And while there are nuances around returns for individual infra providers depending on the concentration of and share between the model and customer cohorts, the reality is that infrastructure ROI overall benefits as long as demand for tokens keeps growing, regardless of the ongoing push-pull between model providers and customers. In the last couple of months, we seem to be heading in a certain direction: aggregate model customer ROI is accelerating as customers benefit from the utility of increasingly intelligent tokens (and apps) at falling prices. This is evidenced by nearly all of the token-volume metrics across the industry, which continue to show growth and/or acceleration. Both the qualitative and quantitative data points on positive customer ROI are growing in concert, most recently with AWS CFO John Felton remarking that he studied AWS’ own token usage and found that, in general, more is better. And for those who still don’t believe in return on token spend, I implore you to model out the number of tokens needed to complete a job of your choice and measure the value of that job relative to the underlying cost of the tokens, with or without the model provider margin. Sure, there can always be waste. But when utilized properly, the returns on tokens are extraordinary, which is exactly what's driving continued volume growth. We’ve subsequently seen this flow through to the cloud infrastructure providers, whose payback periods have compressed and whose returns have also increased, primarily as a function of rising rental prices for GPUs/accelerators. And as those returns have increased, so has demand for the underlying infrastructure, benefiting the AI supply chain at large. So far, so good. That leaves the model providers, where the biggest question really lies. The fact that inference gross margins can be so high is certainly positive in addition to being obvious: the prices at which tokens are sold are multiples higher than the miniscule cost of the underlying FLOPs needed to generate them. And again, the value of the work being done by the end customer is multiples higher than the token prices, so everybody wins. However, inference gross margins paint an incomplete picture, because token prices are primarily a function of model quality, which is primarily a function of training spend. That spend, in aggregate, is causing model providers to burn lots of cash. And because competition remains so intense, it’s unlikely in my view that any model provider can retain its inference margins without continued aggressive training spend. So the two go hand in hand and should be assessed together. While Anthropic’s reported positive FCF in 3Q was a nice surprise, I don’t expect it to continue, especially as OpenAI remains committed to investing aggressively and burning cash for the foreseeable future. So that’s the predicament. The market is unsure what the long-term business and economic model of a 'lab' is, and rightly so. That raises the question: if the labs can’t make money, who’s going to fund the training, and how will the models keep getting better, which is the critical ingredient in continued AI adoption and end-customer ROI? I see a few outcomes to this question: Scenario 1: Concerns about model-provider economics prove overstated, and the market expands rapidly enough to accommodate multiple profitable frontier developers alongside a thriving open-source and open-weight ecosystem, supporting broad-based growth with attractive margins. This is probably the cleanest outcome which is why the market tends to like it so much. I still think this is the most likely path in the fullness of time, but it’s admittedly unclear. The mega-pilled RSI scenario also fits within this outcome, mainly in the sense that the investment pays off. Scenario 2: Because of heightened competitive intensity and elevated training costs, the aggregate model provider ecosystem simply operates at maturity with razor-thin margins and very tight/minimal free cash flow generation, like other extremely competitive and/or commoditized industries. This is at least where we seem to be today, and where we will likely remain for some time. Scenario 3: Financial pressure causes weaker participants to exit, merge, or scale back large training programs, eventually giving way to consolidation and reduced frontier training intensity, which allows the remaining players to expand margins. This seems both logical and likely to happen to some unknown degree, which would be positive for the value of the remaining players. Note that megacap tech hyperscalers, who collectively generate hundreds of billions in operating cash flow from their core businesses and are also training models, can continue to tap into that cash flow for ongoing training (ie GOOGL, META, and more recently NVDA). That will persist regardless of what happens with OpenAI or Anthropic. Scenario 4: Frontier labs eventually convert into true public benefit corporations (in more than just name), owned and funded by a broad coalition of companies and the USG, all with incentives aligned toward the ongoing training and development of new models, effectively making the labs public utilities rather than private corporations. I do believe there is tremendous value in the IP at both OpenAI and Anthropic, and consequently see this as a far more likely outcome than either of them “going bust” and ceasing operations. While this may seem scarier to markets and in the headlines, the actual end result could be quite positive and well distributed across the entire AI ecosystem (rather than concentrated in the two labs). And at the very least, I believe there is strong merit in a pool of government-funded dollars for model development, much like the CHIPS and Science Act of 2022, which authorized $280 billion in federal funding and incentives. The US federal government spends >$7 trillion annually. Point being, there is both ample budget and precedent for the USG (and megacaps/others) to contribute to this national cause. All in, some combination of these four outcomes should play out. And while some paths are smoother than others and there will be inevitable volatility and hiccups along the way, they all ultimately lead to positive ROI on AI spend: better models drive better end-customer returns, which drive better returns on infrastructure. That flywheel is already in motion (as evidenced by rising token volumes, GPU rental prices, and infra returns), and it’s much more sustainable than the market fears. It will take time to get from here to there, but the endgame appears to be quite positive, almost any way you slice it.