primalglenn 🌐🖥️

One would think that because blobs are almost free, L2s should have no problem landing blobs. But one interesting thing I learned from interviewing L2s is that they in fact have a lot of trouble landing blobs, EVEN WHEN there's no congestion. I want to explain why, as that reveals some very interesting facts about the protocol. Recall that blob pricing works similar to gas pricing (EIP-1559), where there's a "base fee" which goes up or down each block, depending on whether the previous block hit the target number of blobs (14 out of 21). The idea is that in a well-functioning market, the base fee will track the "market price" of blobs, so that you can land blobs by just paying the base fee, with perhaps a little priority fee if there's congestion. However, whereas this mechanism works well for regular gas, it turns out to be an issue for blobs, because blob demand is relatively inelastic to price. In other words, just because the blob price is low does not mean that L2 will send more blobs -- they still need to fill those blobs with L2 transactions, which may have low demand even when L1 blob fees are low. As a result, blob usage has been consistently below target, such that the blob price sat at 1 wei -- the theoretical minimum -- for a long time. A later EIP raised the floor higher, but it's still so low as to be effectively free. Why is this an issue? Normally it's not. People love free blobs! But at times of congestion, because the base price starts so low, it takes a HUGE amount of time for the base price to rise enough to discover the true market price. For example, if the base price is 1 gwei and the (hidden) market price is 40 gwei, it would take 47 consecutive full blocks (~10 minutes) for the base price to catch up. By the time it catches up, the congestion may already be over. So what do L2s do during congestion? They bid with the priority fee, but without knowing what the true market clearing price is, they have to overbid in order to be sure that their blobs will be included. This is precisely the unreliability / wastefulness that EIP-1559 was set out to fix in the first place! But price instability is not the only reason why L2s have trouble landing blobs. There's another more subtle and interesting reason -- MEV. Blobs are large chunks of data, and as such they cost bandwidth to propagate. Since bandwidth is a finite resource, everything being equal a block with blobs will take longer to propagate than a block without blobs. This is an issue because a block builder only has a limited amount of time to propagate blocks before missing the attestation deadline, which would cause their block to be reorg-ed. Therefore, block builders play a timing game where they wait as long as possible so they can pack the most valuable (high MEV) transactions into their block, but not waiting so long that they miss the attestation deadline. Therefore, blobs have a hidden cost -- because they make blocks larger, builders who include blobs must send out their blocks sooner, causing them to miss out on some MEV. As such, some builders refuse to include blobs EVEN WHEN there's no congestion in the blob market, so from the L2's perspective they just see their blob transactions being silently dropped, even though they appear to be paying adequate fees. To truly ensure blob inclusion, the L2 would have to include a high-enough priority fee to compensate for the lost MEV from the builder, but once again this value is hidden and hard to discover. In short, there are at least two forces against L2s landing blobs reliably: - The blob price being so low that it kills price discovery and causes bidding wars during congestion. - Blobs cause builders to lose MEV, so they ignore blobs unless priority fees are high enough to compensate for the MEV. So what can we do to make blobs more reliable? Good news is that things will already get better in Glamsterdam due to ePBS giving blobs more time to propagate, but more fixes are required if we want to completely fix these issues. Thankfully many researchers including @fradamt @potuz_eth @barnabemonnot are already thinking about this -- I may talk about some of their ideas in a future post!