i like golf and systems @voltcapital | prev @Delphi_Digital
Joined June 2011
- Tweets11.6K
- Following599
- Followers6.1K
- Likes14.6K
Rob retweeted
We are building atlases of the intangible economy.
Neuroscience provides some good intuition here.
It’s often the case that building atlases is simply impossible without the right neurotechnology, and it makes sense to co-design the atlas and the technology to acquire (or generate) that atlas.
For example, the Enigma project at Stanford, which is building the largest functional atlas ever collected, was empowered by the creation of next-generation Neuropixels, increasing throughput by a factor of 4.
A holistic strategy in the age of abudant intelligence optimizes the creation of new tools vs. using existing tools to build atlases of the intangible economy.
Simply, what is even more valuable than a static dataset is a data generation capability (both the right to and the ability to generate high signal data). This is how we engineer economically valuable serendipity.
The firm itself becomes a scientific instrument.
few in data are talking about owning differentiated sensors to capture novel data - everyone is red queen racing to unique sources of data (enterprises, bio, etc.)
but by controlling the sensor in which a certain form of data can even be made legible, you can control all downstream data feeds (ex: halter and agriculture)
ex: in the limit this looks like space telescopes (high capex, R&D intensity, regulatory permission, etc)
science.nasa.gov/mission/rom…
a few friends have begun to monopolize the most esoteric of sensors and are exhibiting the extreme form of pole position
extreme pole position (read: MONOPOLY) involves controlling the interface where reality is made legible to intelligence
broadcast creates awareness. diffusion requires trust.
As I - along with all public analysts - continue to pore over the Anthropic $ANT S-1 - some thoughts / observations:
The principal question for Anthropic: ultimately boils down to will their revenues generated outpace their fixed compute commitments? This is the principal dilemma Dario lays out in his interview with Dwarkesh. Procure too little and you run the risk that you are under resourced in a race against other AI labs - and forced to procure compute at any price into an already tight market (aka Anthropic this year). Provision too much relative to your revenue traction and you run the risk of “blowing up”.
Compute Qualities
Compute for an AI Lab generally has the following characteristics:
• Take or pay contracts - given provider fulfills performance SLAs
• Annual cost of usually 1/4 of the capex required - on a rolling basis (owned / leased chip math)
• Configuration of counterparty (hyperscaler, neocloud, self-build) and accelerator (Nvidia GPU, Hyperscaler ASIC (TPU, Trainium), 3rd Party Accelerator (Cerebras, etc), AMD GPU)
• In general, self-build allows the labs (and the hyperscalers) the flexibility to “smooth” the existential curve. Building shell buys an AI lab / hyperscaler flexibility to adapt to their business needs. Contrary to fanatical consensus that there are “GPUs gathering dust and being hoarded” - the biggest cost in a DC is the depreciation of IT gear. Quite catastrophically money losing - vs over building land, power, shell but not equipping with IT gear (~20% of capex in land / power / shell depreciated over 10-20 years = 1 year of empty shell ≈ 1-2% of project “wasted” - vs ~80% of capex in IT gear over ~5 years = 1 year of idle GPUs ≈ 15% of capex wasted)
• Labs are early in building out this function - as managing and developing sites is a drag on resources, heavily physical operation, requires scale and IG ratings etc (for now they borrow other people’s credit - Google backs the data center leases, Broadcom backs the chip lenders) - but will be an area the labs will extend to over next 3 years
The Economics Of Compute
I have heard many takes on this and efforts to analogize to other industries. Here is a compilation of my thoughts:
The economics of producing tokens are constantly evolving as well. Every generation there is a 3-10x improvement on the total hardware (maybe total 1.5-2x improvement from strictly hardware design, and remainder from hardware-software co-design improvements (scale-up domains, new numerics, custom HBM, offloading the network etc)).
On top of that, the labs have shown an ability to squeeze even more out of their internal inference serving software - through many many techniques like batching, model parallelism, speculative decoding, prefill/decode disaggregation, etc. You can see the price decreases generation on generation but holding a capability or bench score constant, the price of compute has fallen 1,000x+ in about three years (roughly 10x a year, and far faster on some tasks).
This is perhaps the most important thought for this type of business: the implicit operating leverage that exists. If you are a 10% GP margin lab (100 revenue, 90 cost), and your cost is falling 10x year on year, if you keep PRICE per token FLAT AND YOU DON’T GROW. Your NEW GM is 91% (100 revenue, 9 cost, 91 GP). Moreover, because your fixed compute is a FIXED COST... the moment you do grow, with the SAME COMPUTE CAPACITY, you are now able to serve 10X MORE BUSINESS (1,000 revenue, 90 cost). You can move from a $10 GP business to a $910 GP business in 1 year - insanity
I know, I know. Sounds too good to be true. What does this assume? This assumes an unlimited TAM - that as fast as you are able to free up demand, you are able to match it and sell it (rate of diffusion is perfect). It assumes you pass no value back to your customers (not true). It assumes no competitive pressures (aka competitors will not offer a more capable model / compete on price to win business). It assumes you will not create more intelligent models which consume more compute per inference pass.
All of these are not true - the environment does have those factors. But the way I see it is by early this year we discovered an amazing business with unprecedented unit economics which climbed the wall of worry of my analysis.
1) Would these businesses ever get to large enough revenue scale to amortize the training costs? - YES
2) Would these business ever get to unit profitability? - YES
3) Would these businesses be sticky given the ease of switching API endpoints? - Not fully proven, but evidence is encouraging
4) Would the models become commoditized and business is left to the app layer? - Not fully proven, but evidence is encouraging
Beyond the compute, is a war for talent. Would labs be able to attract, retain and motivate the most talented engineers post-economic realization? This is an open question - but culture and alignment is key. This is not a comment on Anthropic or OpenAI or anyone else - but given the qualities of compute described above, it makes sense why the battle for talent has been so intense so far.
There has been a trending chart on Anthropic’s compute efficiency and the fact that they only use 25% of their compute towards inference. That their gross margins are 85%. I’ve heard takes all around the spectrum - bearish that it means they have overbuilt compute and bullish that they have the opportunity to earn a lot more. The same data point on both sides...
I’ll offer my thoughts as the last piece and stop my consciousness dump & save rest for next time... The reality is $ / GW for Anthropic is an output not an input. It is the confluence of many things - the hard work they (and Mythos 5.5+) are doing to improve inference throughput per unit of compute. This naturally increases this number. How much true competitive pressure they are facing (should trend lower if they are truly facing competitive pressure - from a place of weakness). Or from a position of strength - to pass this along to customers to extend / entrench their business and applications deeper with customers. And the way they think about this inference compute is not an “overbuild” - because every MW they reclaim from inference, they can use to deploy for training, or internal use (aka using Mythos 5.5+ to build / find more optimizations for their system). This is the recursive loop people talk about.
I don’t want to say it’s all roses as I see it - the reality is that Anthropic only has 1 real year of operating at this scale. The qualities of an AI lab as a business - we are just figuring out and have discovered this year. There is a stable of 5+ credible competitors who haven’t really pressed the gas on competition yet between OpenAI, xAI, Google, Meta, Chinese (DeepSeek, Moonshot, Zhipu, Qwen, Byte - with a fundamentally cost plus / margin destructive mindset), alongside some other neolabs who could come up with something completely different.
Safety and alignment are also real risks to these businesses - not just marketing fluff. As we explore new ways to harness AI and extending the capabilities of these models, we have an increasingly difficult job of making sure that they do not break - which means that an increasing % of these company’s compute needs to be dedicated to safety (20% compute+). There are real risks that the revenues generated may slow down relative to the growth of the fixed cost base. So we need to be sober about the risks.
But despite all of this, I come away excited at what the future holds for these labs. Take stock of what has happened in the last year - AI was still in chat bot mode for 99% of the world last year - and is still for 95% of the world today. We have come so far so quickly and we are just starting...
The information presented in this post is the opinion of the author and does not necessarily reflect the view of any other person or entity, including Altimeter Capital Management, LP (”Altimeter”). The information provided is believed to be from reliable sources but no liability is accepted for any inaccuracies. This is for information purposes and should not be construed as an investment recommendation. Past performance is no guarantee of future performance. Altimeter is an investment adviser registered with the U.S. Securities and Exchange Commission. Registration does not imply a certain level of skill or training. Altimeter and its clients trade in public securities and have made and/or may make investments in or investment decisions relating to the companies referenced herein. The views expressed herein are those of the author and not of Altimeter or its clients, which reserve the right to make investment decisions or engage in trading activity that would be (or could be construed as) consistent and/or inconsistent with the views expressed herein.
This post and the information presented are intended for informational purposes only. The views expressed herein are the author’s alone and do not constitute an offer to sell, or a recommendation to purchase, or a solicitation of an offer to buy, any security, nor a recommendation for any investment product or service. While certain information contained herein has been obtained from sources believed to be reliable, neither the author nor any of his employers or their affiliates have independently verified this information, and its accuracy and completeness cannot be guaranteed. Accordingly, no representation or warranty, express or implied, is made as to, and no reliance should be placed on, the fairness, accuracy, timeliness or completeness of this information. The author and all employers and their affiliated persons assume no liability for this information and no obligation to update the information or analysis contained herein in the future.
On a personal note - Altimeter’s funds are large Anthropic shareholders. I was not involved in those private investments and have no economic interest in them beyond my own capital invested in our funds. These are my views as a public markets analyst, formed from the S-1 and public information. So welcome all honest pushback and debate. Not investment advice.
Rob retweeted
We wanted to better understand what AI safety looks like in practice, so we brought @zachary_horvitz on @TheDryRunPod to walk us through his research on long-tail model behavior.
Some takeaways:
1. Rare model outputs aren’t always harmful. Exploring many possible answers can also help uncover useful discoveries. Zach connected this to how AI is helping researchers explore problems in math and biology.
2. A model refusing a harmful question once doesn’t mean it will refuse every time. As models become safer and more useful, more people use them. Even a small chance of harm matters when that usage reaches millions of people.
3. Finding rare harmful outputs is only part of the challenge. Researchers need ways to estimate their likelihood without generating thousands of answers for every question. They also need to judge those answers, often using other AI models that can make mistakes too.
We learned a lot from Zach and we’ll be exploring more of these questions on the pod.
What does rare AI behavior mean for safety?
Zach Horvitz (@zachary_horvitz), a PhD candidate at Columbia researching rare behavior in generative models, joins Dry Run to explain what rare harmful behavior looks like, how researchers measure it, and what it means for the safety of models people rely on.
00:53 What AI safety looks like today
04:05 What rate of harm is "acceptable"?
06:56 Why the same prompt gives different answers
08:08 Why rare harmful outputs matter at scale
11:36 Asking the same harmful question 10,000 times
12:13 How to estimate the odds of a harmful answer
13:50 Using AI models to judge other models
16:28 How human feedback rewards flattery
18:42 Why safer AI also gets used more
24:19 Testing models without access to their internals
27:04 How rare outputs could drive scientific discovery
know what to measure:
- combat, $/intercept
- orbit, $/kg
- materials, $/ton
- energy, $/watt
- tokens, $/token
live thinking: where does serving go?
- been thinking more about this as everyone tries to further underwrite inference. directionally correct that demand for 'intelligence' grows exponentially but less certain on where said demand gets served / who captures the economics.
- most assume (more agents * more work = more inference). likely true in aggregate but present id lean that we still dont know how to quite represent the problem in the 'most' efficient manner. is it all just inefficient orchestation?
- so what share of work actually requires 'frontier' intelligence? most jobs dont require a phd and the ones that do dont need that for every step. throwing intelligence at the wall to see what sticks checks the boxes toward a payout. a lot of it today is just figuring out something for the first time so we can benchmark and chart a path towards repeatability.
- what about control? some models dont allow tasks that other models are capable of executing. perhaps opening the door to model abliteration. it is interesting bc capability & permission are two different things. two models are equally intelligent (?), dont mean they equally are useful to the end user deploying them.
- kinda like pizza in nyc. something you wait for lucali, sometimes you scarrs..both hit but one is craft and the other is cheap/fast/good enough. neither needs to win every order and that kinda is how i view inference right now. just need something that reliably gets the job done.
- does most (%) of serving move to smaller models OR the frontier labs own the customer/routing path? cheaper intelligence unlocks demand that wasnt economical but that doesnt mean it will be captured efficiently.
- financing is the most interesting part of the stack right now. infra around assumption of future utilization, pricing and customer demand. all inputs that change on a dime. my hunch is were spending time underwriting growth in inference without spending enough time on where those workloads actually settle.
- think the unit of measurement is part of the problem. have written about token/sec, $/token, gpu utilization to all be useful. customer really only cares about work completed. would the most valuable system in the world be the one that eliminates inference? doubt it.
- so value acrrues in the model, serving infra, distribution, workflow owners? each expose some new binding constraint. all need to be dynamic. right now trying to figure out how much demand migrates to another provider, how much gets shifted away and how much entirely new demand gets created in the process.
every team gets a lil taste of being an GM with an aging roster. run it back with the the aging verterans or accept the tank, blow it up & rebuild.
mercenaries are at the wheel
noticing an inflection in optionalitymaxxing
- founders seeding side incubators to taste investing
- founding engineers whispering to competitors for a higher-equity role
- “mentors” advising fledgling startups only to carry their secrets to older, more credible entrepreneurs
where went the autistic singlemindedness i grew up around and was inspired by? where went those who burn it all to the ground, put their back against a wall, and walk along one vector, through fire and through delusion, to their goals? i fear every live player must lash a commitment anchor around their neck, or all the ai winners will grow up godless and figless
Rob retweeted
had fun going through all my favorite financial history books and putting this together
as a wise man once said, alpha is hidden in the texts that many are too lazy to read
Our report “The House Always Wins: Why Speculation is a Theme of the Decade” is out now.
Speculation runs rampant when new tech rearranges what’s possible or the system stops working for most. AI and crypto are now fueling both.
Read it for free here.
delphi.link/Speculation
sovereignty is only as durable as your least replaceable counterparty
The issue for international neoclouds is behind the scenes they will constantly be contending with a hostile state dept who would prefer compute dominance continue to accrue to the U.S.
Ofc the notion of sovereignty remains a dominant force on the other side but if you’re relying on American hyperscalers to fund the erosion of American dominance you’re going to have a bad time.
Watch the Humain project in the Middle East and how it’s treated post the OAI spy incident.
Not sure how that will progress but it will tell you a lot about multiple things including wether MBS was onsides in Iran etc
Rob retweeted
Alex is right here. Don't trust the eval industrial complex.
Most public benchmarks are terrible, and top rank there is a negative signal. If you're building products, you need to create your own private evals.
The private evals need to deeply correspond to what your users want and the unique knowledge and know-how you have in the domain. You also need to avoid open-source code as much as possible.
META’S HEAD OF AI SLAMS AI BENCHMARKS
"it’s some PhD student who made these benchmarks"
"you’re kind of outsourcing all of your product thinking to this viral benchmark"
"so clearly that’s wrong"
I think there is something to this. the whole industry developed so rapidly that there was hardly any time to establish proper benchmarking. companies used what was available, competitors then had to catch up on the same benchmarks, and some of them eventually became industry standards
apparently there is quite a lot of randomness and chance in how we are measuring AI progress today
we can all learn from sitting on an aesthetic bench
youtube.com/watch?v=hGqKppJY…
Rob retweeted
big one. we teamed up with Matera and @solana to bring X9.150 into business payments
X9.150 is the first US national standard for QR payments, built by 70+ orgs, and the biggest US banks are lining up behind it
we're putting it to work in @altitude bill pay: signed recipient details on the invoice, verified through the X9.150 standard, settled in usdc on solana with the invoice that marks itself paid on the recipient's end
catch live demo at money20/20 this week
Find us at Money20/20 USA this week for a demo of Altitude Bill Pay using the new ANSI X9.150 QR payment standard.
See signed payment instructions, USDC settlement on Solana, and payment status updates in action.
Learn more: altitude.xyz/blog/raising-th…
