a16z Crypto's latest research: What is the key to large-scale adoption of DeFi?

Author: PGarimidi, jneu_net, @MaxResnic

Compiled by: Jiahua, ChainCatcher

Blockchain can now genuinely claim that it has the capability to compete with existing financial infrastructures. Today’s production systems can process tens of thousands of transactions per second and are about to see an order of magnitude increase. However, beyond raw throughput, financial applications also require predictability. Whether it’s a trade, a bid in an auction, or an option exercise, the normal operation of a financial system demands a definite answer: when will this transaction be executed? If transactions face unpredictable delays (whether malicious or accidental), many applications will become unusable.

To make on-chain financial applications competitive, blockchains must provide short-term packaging guarantees, meaning that if a valid transaction is submitted to the network, it can guarantee that it will be packaged as quickly as possible. For example, consider an on-chain order book. An efficient order book requires market makers to continually provide liquidity by maintaining buy and sell orders for assets on the ledger.

The key challenge faced by market makers is to minimize the bid-ask spread while avoiding the risk of “adverse selection” due to quotes deviating from the market. To do this, market makers must continually update their orders to reflect the state of the real world. For instance, if an announcement from the Federal Reserve causes asset prices to spike, market makers need to react immediately and update their orders to the new prices. In this scenario, if the transaction updating the market maker’s orders does not get executed immediately, they will suffer losses as arbitrageurs execute their orders at outdated prices. The market makers then need to set wider spreads to reduce their risk exposure in such events, which in turn diminishes the competitiveness of on-chain trading venues.

Predictable transaction packaging provides a strong guarantee for market makers, enabling them to respond quickly to off-chain events and maintain the efficiency of on-chain markets.

What We Have vs. What We Need

Today, existing blockchains only provide robust final packaging guarantees, typically effective within a few seconds. While these guarantees are sufficient for applications like payments, they are too weak to support large classes of financial applications where market participants need to react in real-time to information.

Taking the aforementioned order book as an example: For market makers, if arbitrageurs’ transactions can land in earlier blocks, then guaranteeing they will be packaged “within the next few seconds” is meaningless. Without strong packaging guarantees, market makers are forced to widen their spreads and offer worse prices to users to mitigate the increased risk of adverse selection. This, in turn, significantly reduces the attractiveness of on-chain trading compared to other venues that provide stronger guarantees.

To truly realize the vision of blockchain as a modern infrastructure for capital markets, builders need to address these issues so that high-value applications like order books can thrive.

What Makes Achieving Predictability Difficult?

Enhancing the packaging guarantees of existing blockchains to support these use cases is a challenge. Some current protocols might rely on a node that can decide which transactions to package at any given time (the “leader”). While this simplifies the engineering challenge of building high-performance blockchains, it also introduces a potential economic bottleneck where these leaders can extract value. Typically, during the window when a node is selected as a leader, they have complete authority over which transactions to package into blocks. For a blockchain handling financial activities of any scale, the leader holds a privileged position. If this single leader decides not to package a transaction, the only remedy is to wait for the next leader willing to package that transaction.

In permissionless networks, leaders have an incentive to extract value, commonly referred to as MEV (Miner Extractable Value). MEV extends far beyond merely sandwiching AMM trades. Even if the leader can merely delay transaction packaging by a few milliseconds, it can yield substantial profits and reduce the efficiency of the underlying application. An order book that only prioritizes certain traders’ transactions creates an unfair competitive environment for everyone else. In the worst case, the leader may become so hostile that traders completely abandon the platform.

Imagine a rate hike occurs, causing the ETH price to drop 5% instantly. Each market maker on the order book rushes to cancel their open orders and place new ones at the new price. Meanwhile, each arbitrageur submits orders to sell ETH at the outdated order price. If this order book operates on a protocol with a single leader, that leader wields tremendous power. The leader can simply choose to censor all market makers’ cancellation operations, allowing arbitrageurs to reap massive profits. Alternatively, the leader might not directly censor cancellation operations but delay them until after the arbitrageurs’ transactions are executed. The leader could even insert their own arbitrage transactions to fully exploit price discrepancies.

Two Fundamental Demands

In the face of these advantages, market makers’ active participation becomes economically unviable; they risk being taken advantage of whenever prices fluctuate. This issue boils down to the leader possessing excessive privileges in two critical aspects: 1) the leader can censor any other person’s transactions, and 2) the leader can see others’ transactions and submit their own in response. Any one of these issues could prove disastrous.

An Example

We can pinpoint the problem with the following example. Consider an auction with two bidders, Alice and Bob, where Bob is also the leader of the block in which the auction takes place. (The setup with only two bidders is for illustrative purposes; the same reasoning applies regardless of the number of bidders.)

The auction accepts bids over the time it takes to generate a block, from time t=0 to t=1. Alice submits a bid bA at time tA, and Bob submits a bid bB at time tB > tA. Since Bob is the leader of this block, he can always ensure he acts last. Alice and Bob also have a continuously updated source of truth for asset prices (e.g., a centralized exchange’s mid-price). At time t, let’s say this price is pt. We assume that at any given time t, the market’s expectation of the asset price at the end of the auction (t=1) is always equal to the current real-time price pt. The auction rules are simple: the higher bidder between Alice and Bob wins the auction and pays their bid amount.

The Need for Censorship Resistance

Now let’s consider what happens when Bob can leverage his advantage as the leader of this auction. If Bob can censor Alice’s bid, it’s clear that the auction would fail. With no other bidders, Bob can simply bid an arbitrarily small amount to ensure he wins the auction. This results in the auction settling with essentially zero profit.

The Hidden Demand

A more complex situation arises if Bob cannot directly censor Alice’s bid but can still see Alice’s bid before making his own. In this case, Bob has a simple strategy. When he bids, he merely checks if ptB > bA holds. If it does, Bob’s bid is just slightly above bA; if not, Bob does not bid at all.

By executing this strategy, Bob subjects Alice to adverse selection. The only scenario in which Alice wins is if price updates lead her bid to eventually exceed the asset’s expected value. Whenever Alice wins the auction, she anticipates losing money and would be better off not participating at all. As all competitors disappear, Bob can once again simply bid an arbitrarily small amount and win, while the auction effectively generates zero profit.

The key takeaway here is that the duration of this auction is irrelevant. As long as Bob can censor Alice’s bid or see Alice’s bid before making his own, this auction is doomed to fail.

The same principle applies to any high-frequency trading asset environment, whether spot trading, perpetual contracts, or derivatives exchanges: If there exists a leader with the power Bob has in this example, that leader could cause the market to collapse entirely. To make on-chain products serving these use cases viable, they must not grant leaders these powers.

How Do These Issues Arise in Practice Today?

The story above paints a bleak picture for on-chain transactions on any permissionless single-leader protocol. However, many decentralized exchanges (DEXs) operating on single-leader protocols continue to maintain healthy trading volumes; why is that? In practice, a combination of two forces has mitigated the aforementioned issues:

• Leaders do not fully exploit their economic power, as they themselves are often heavily invested in the success of the underlying blockchain;
• Applications have built workarounds to avoid being so vulnerable in the face of these issues.

While these two factors have so far kept decentralized finance (DeFi) operational, they are insufficient for on-chain markets to genuinely compete with off-chain markets in the long run.

To qualify as a leader in a blockchain with substantial economic activity, a significant amount of staking is required. Therefore, either the leader must own a considerable amount of stake or have enough reputation for other token holders to delegate their stakes to them. In either case, large node operators are typically known entities whose reputations are at stake. Not only reputation, but this staking also means these operators have a financial incentive to ensure their blockchain runs well. For this reason, we have not yet seen leaders fully exploit their market power as described above—but that does not mean these issues do not exist.

Firstly, relying on the good will of node operators through social pressure and appealing to their long-term motivations is not a solid foundation for the future of finance. As the scale of on-chain financial activity increases, the potential profits for leaders also rise accordingly. The more this potential grows, the harder it becomes on a social level to prevent leaders from acting against their immediate interests.

Secondly, the degree to which leaders can leverage their market power exists on a spectrum from benign to leading to complete market collapse. Node operators can unilaterally push to utilize their power for greater profits. When some operators push the recognized limits, others will quickly follow suit. The actions of a single node may seem inconsequential, but when everyone changes, the impact is evident.

Perhaps the best example of this phenomenon is Timing Games: leaders attempt to announce block creation as late as possible while the protocol is still valid to earn higher rewards. When leaders are too aggressive, this leads to longer block times and block skipping. While the profitability of these strategies is well-known, leaders choose not to engage in these games mainly to act as good stewards of the blockchain. However, this is a fragile social equilibrium. Once a single node operator starts playing these strategies to earn higher rewards without any repercussions, other operators will quickly join in.

Timing Games are just one example of how leaders can increase profits without fully exploiting their market power. Leaders can also take many other actions to increase their rewards at the expense of applications. Isolated, these actions may be feasible for applications, but ultimately the balance will tip to a point where the on-chain costs outweigh the benefits.

Another factor that keeps DeFi operational is that applications move significant logic off-chain and only publish results on-chain. For instance, any protocol requiring rapid auctions executes this off-chain. These applications typically run their necessary mechanisms on a set of permissioned nodes to avoid encountering the issues of malicious leaders. For example, UniswapX runs its Dutch auctions off the Ethereum mainnet to complete transactions, similarly, CowSwap runs its batch auctions off-chain.

While this is workable for applications, it places the foundational layer and the value proposition built on-chain in a precarious position. In a world where application execution logic resides off-chain, the foundational layer becomes purely a settlement layer. One of DeFi’s strongest selling points is composability. In a world where all execution occurs off-chain, these applications effectively exist in isolated environments. Relying on off-chain execution also adds new assumptions to the trust model of these applications. The operation of applications no longer depends solely on the activity of the underlying chain; this off-chain infrastructure must also operate correctly.

How to Achieve Predictability

To address these issues, we need protocols that satisfy two properties: consistent transaction packaging and ordering rules, along with transaction privacy before confirmation (for strict definitions and extended discussions on these properties, please refer to this paper).

Basic Demand 1: Censorship Resistance

We summarize the first property as short-term censorship resistance. If any transaction that arrives at an honest node is guaranteed to be included in the next possible block, then the protocol is considered short-term censorship-resistant:

Short-term Censorship Resistance: Any valid transaction that arrives on time at any honest node will necessarily be packaged into the next possible block.

More precisely, we assume the protocol runs on a fixed clock with each block generated at set intervals, for example, every 100 milliseconds. What we need to ensure is that if a transaction arrives at an honest node at t=250ms, it will be included in the block generated at t=300ms. Adversaries should not have the discretion to selectively package certain transactions they hear while omitting others.

The spirit of this definition is that users and applications should have an extremely reliable way to land transactions at any point in time. There should be no scenario where a single node coincidentally drops packets (whether due to malice or simple operational failure) that prevents a transaction from landing. While this definition requires packaging guarantees for transactions arriving at any honest node, the overhead of achieving this in practice may be too high. The important feature is that the protocol should be robust so that the behavior of on-chain entry points is highly predictable and easy to reason about.

Permissionless single-leader protocols clearly do not satisfy this property, as there is no way to land transactions if at any point the single leader is a Byzantine node. However, even a set of four nodes that can guarantee packaging transactions during each time slot greatly improves the options available for users and applications to land transactions. It is worthwhile to sacrifice a certain amount of performance for a protocol that can reliably allow applications to thrive. There is still more work to be done to find the right trade-off between robustness and performance, but the guarantees provided by existing protocols are insufficient.

Given that protocols can guarantee packaging, ordering is somewhat a natural consequence. Protocols can freely use any deterministic ordering rule they prefer to ensure consistent ordering. The simplest solution is to sort by priority fees or perhaps allow applications to flexibly order transactions interacting with their state. The best way to order transactions remains an active area of research, but in any case, ordering rules only make sense based on the landing of transactions that need ordering.

Basic Demand 2: Hiding

Following short-term censorship resistance, the next most important property is that the protocol provides a form of privacy we refer to as “hiding.”

Hiding: Before the protocol finalizes the packaging of a transaction, no party other than the node receiving that transaction submission can obtain any information regarding that transaction.

Protocols with the “hiding” property may allow nodes to see all transactions submitted to them in plaintext, but require that the rest of the protocol stays blind until consensus is reached and the order of transactions is finalized in the final log. For example, the protocol might use time-lock encryption to hide the entire contents of a block until a certain deadline; or the protocol might use threshold encryption to immediately decrypt the block once a committee agrees that it is irreversibly confirmed.

This means nodes may misuse the information gleaned from any transaction submitted to them, but the rest of the protocol only knows what they reached consensus on afterward. By the time transaction information is disclosed to the rest of the network, the transaction has already been ordered and confirmed, so no other party can front-run it. To make this definition useful, it does indeed mean multiple nodes can land transactions during any given time period.

We abandon the stronger concept of only users knowing any information about the transaction before its confirmation (e.g., in a cryptographic memory pool) because the protocol needs to take some steps as a garbage transaction filter. If the content of transactions is completely hidden from the entire network, then the network cannot filter out garbage transactions from meaningful ones. The only way to resolve this issue is to leak some non-hidden metadata as part of the transaction, such as the address of the fee payer, which will be charged regardless of whether the transaction is valid.

However, this metadata might leak enough information for adversaries to exploit. Therefore, we prefer that a single node has complete visibility of the transaction while other nodes in the network have no visibility of it. But this also means that to make this property useful, users must have at least one honest node as an on-chain entry point to land transactions during each time period.

A protocol that possesses both short-term censorship resistance and hiding provides an ideal foundation for building financial applications. Returning to our earlier example of running an auction on-chain, these two properties directly address the issues that Bob could cause for market collapse. Bob cannot censor Alice’s bid, nor can he leverage Alice’s bid to inform his own bidding.

With short-term censorship resistance, anyone submitting a transaction (whether a trade or an auction bid) can be assured of immediate packaging. Market makers can alter their orders; bidders can bid quickly; settlements can land efficiently. Users can be confident that any action they take will be executed immediately. This, in turn, will allow the next generation of low-latency real-world financial applications to be wholly built on-chain.

To enable blockchain to truly compete with and even surpass existing financial infrastructures, we need to tackle issues far beyond just throughput.