[ethresear.ch] [Execution Layer Research] How large can EIP-8141's validation budget be? A mempool DoS benchmark

How large can EIP-8141’s validation budget be? A mempool DoS benchmark

Authors: Stefano De Angelis, Daniil Ankushin, Manuel Arto. All at Nethermind. Thanks to Thomas for feedback and comments on the initial benchmarking plan.

TL;DR

• EIP-8141 caps unpaid validation at MAX_VERIFY_GAS = 100k; too low for a Groth16 privacy transfer or post-quantum signatures.
• Raising the cap is a DoS risk, since validation and block processing share the node’s CPU. An attacker can send transactions that use the whole cap and then fail, taking CPU from block processing for free.
• We tested this risk on a Nethermind node, flooding it with such transactions at caps from 100k to 500k, on one CPU core (the worst case) and three machines.
• Without in-client protections, block processing on our slowest machine stays within a 25 ms delay budget only up to 15M gas/s. With Nethermind allowing up to about 100 failing tx/s per connection, that limits the cap to the current 100k.
• We show that a MAX_VERIFY_GAS = 300k is viable with an in-client protection that pauses transaction validation while the node processes a block. At 100 tx/s (30M gas/s), block processing then stays within 50 ms, even on p90 mainnet blocks.
• The cost moves to the mempool, which admits only 12–27% of honest frame transactions under the flood.

1. Introduction

In an ordinary Ethereum transaction the signature check is cheap and fixed. In an EIP-8141 frame transaction the check is arbitrary EVM code in a VERIFY frame. Before admitting the transaction to its mempool, a node simulates its validation prefix, the frames up to and including the first one that sets who pays. The node must run that code to learn whether the transaction is valid, and if the check fails, nobody pays for that work. This opens a denial-of-service (DoS) vector. An attacker sends transactions whose VERIFY frame burns significant CPU and then fails, so the sender pays nothing. As a protection against this risk, EIP-8141 bounds the work with the MAX_VERIFY_GAS parameter, currently set at 100,000 gas per transaction.

However, a 100k validation budget admits limited use cases. For example, a privacy transaction with a single-note Groth16 spend needs at least 211,828 gas even after every optimization. Post-quantum accounts hit the same limit as verifying hash-based SPHINCS+ variants in the EVM costs up to 276k gas. Hence, raising that budget is a requirement to make privacy, post-quantum signatures and other relevant use cases practical with frames. This leads to the critical research question for this post:

Can the block processing of a node be slowed down with DoS campaigns when the frame validation cap goes up?

To answer this question, we ran a DoS campaign against a Nethermind client with EIP-8141 activated. We set the frame validation budget at caps from 100,000 to 500,000, and tested input loads with two types executed in the VERIFY frame: (i) EVM hashing loops and (ii) stuffed secp256k1 signatures. We evaluated the impact on block processing through three metrics: (i) block import delay, (ii) block building delay, and (iii) share of honest transactions admitted. We ran the tests across different hardware tiers covering local builder profiles and our slowest machine. We call an input load safe when the extra time it adds to importing or building a median mainnet block stays within a delay budget of 25 ms. On today’s network that much extra delay makes a node miss the attestation deadline about 0.2 percentage points more often (§3.5). In short, we found the following:

1. A local builder absorbs about 30M gas/s of validation. Block processing stays within a 25 ms delay budget up to about 30M gas/s of VERIFY work. Each cap reaches that limit at a different transaction rate: 300 tx/s at 100k, 120 at 250k, 100 at 300k and 60 at 500k. At 100 tx/s, 300k is the largest cap within budget.
2. A full node absorbs only about 15M gas/s. Every node must import each block on time. On our slowest machine, the full-node profile, import stays within 25 ms only up to about 15M gas/s of VERIFY work. At 100 tx/s that is 10M gas/s at 100k but 30M at 300k, so only 100,000 is within budget.
3. In-client DoS protections unlock 300,000 within budget at 100 tx/s. We implemented protections in the Nethermind client that make mempool validation give way to block import. While the node processes a block, it starts no frame validation of gossiped frame transactions (#13994). Import delay drops to about 2 ms or less on every machine, so a 300,000 cap stays within 25 ms at 100 tx/s.
4. Honest transaction admission under flood. Failing transactions compete for the same CPU as honest ones. Under a 100 tx/s flood at 300k, the slowest machine admits only 12% of honest frame transactions.

1.1. Why 100k is too low

With frames, transactions can adopt different authorization schemes, each with its own verification cost. The table below shows that many of them do not fit under the 100k `MAX_VERIFY_GAS` limit.

2. Model and definitions

2.1 Threat model

We consider an adversary that mounts a DoS attack on a node by sending it transactions that fail validation, with the aim of disrupting its operation. We first define the attack and the transactions it uses.

Block-delay attack. The attacker wants to make a node late to import or build blocks, without producing any invalid block. The attacker connects to the node like any peer and writes the VERIFY code of the frame transactions it sends. Since a transaction pays fees only if it passes validation, the attacker crafts transactions that consume the whole validation budget and then fail. Throughout the document we refer to them as failing transactions. Each one names a solvent payer, declares a VERIFY budget equal to the cap C, runs the most CPU-expensive work that budget buys, and fails just before approving payment. The node has spent the CPU, and nobody pays for it.

At r failing transactions per second the node runs r × C gas per second of unpaid validation, and that CPU is taken from block processing. The attack succeeds once the delay it adds to processing one block exceeds a delay budget B (§2.2).

The attack works at any cap, since at a lower cap the attacker just sends more transactions. Raising the cap still raises the risk, because a node’s defenses limit transactions, not gas. Nethermind, for example, processes about 100 failing transactions per second from one connection (Appendix A), and at that rate a 300k cap delivers three times the load of 100k. The benchmark therefore asks how many failing transactions per second an attacker needs at each cap before r × C exceeds what the node can absorb.

2.2 Notations and metrics

Load. At rate r and cap C the node runs r × C gas per second of unpaid validation.

Metrics. We measure three metrics.

• Block-import delay and block-building delay. Δ(C, r) is the time the attack adds to processing one block. We measure it separately for import, which every node does, and for building, which only a proposing node does. It depends on f, the CPU share the attack takes at the load r × C, and on T, the time the block needs without the attack. A block that needs T of CPU gets only 1 − f of the core and takes T / (1 − f), so the attack adds Δ = T · f / (1 − f). For example, if the attack takes 30% of the core, the block gets the remaining 70% and takes T / 0.7 ≈ 1.43 T, so Δ ≈ 0.43 T.
• Honest admission. The share of honest frame transactions the node admits to its mempool while under attack. It measures what a node gives up when it limits the CPU spent on validation.

We judge the two delays against an acceptable delay budget, set as benchmark parameter B:

• Delay budget. B is the largest block delay we accept.
• Maximum cap. C_{max(r)} is the largest tested cap whose block delay stays within B at rate r.

3. Methodology

3.1 Hardware

We tested the worst case, a node with no spare core to absorb the attack. A single Nethermind node ran on one CPU core, shared by block processing and the validation of failing transactions. We ran it on three machines:

By referencing EIP-7870, High and Mid stand for local builders. Low is the closest we had to a full node (§4.1).

3.2 Load types

We built failing transactions from the most CPU-expensive work we found for a validation frame, in two load types:

• Keccak-wide. KECCAK256 over 4 KB, repeated N times, then fail. We fixed N so that each transaction consumes nearly the full cap before it fails.
• Sig-stuffed. A transaction that verifies many secp256k1 signatures. The verification checks signatures one by one, with the last signature failing, so that the node must process every signature before rejecting the transaction.

3.3 Benchmarks

We do not target mainnet blocks directly. Each benchmark repeats a fixed piece of block work (Appendix B), first with no attack and then under a steady load of failing transactions, fixed in gas per second. The attack determines f, how much CPU it takes on hardware. Thus, we apply the same f to mainnet blocks measured at length T (§3.1), and we calculate Δ(C, r) on mainnet values.

We ran four benchmarks:

• Block building (§4.1). The node builds blocks while it validates a flood of failing transactions, at 5M to 40M gas/s in 5M steps and at every tested cap.
• Block import (§4.2). The node imports blocks while it validates a flood of failing transactions, at 5M, 10M, 30M and 50M gas/s and at caps from 250,000 to 400,000.
• Honest admission (§5.2). The share of honest frame transactions the node admits under a load of 10M, 30M and 50M gas/s.
• Per-connection rate (Appendix A). The node is flooded from one connection at increasing rates, to find how many failing transactions per second one connection gets processed. This is the reference rate r.

We report MAX_VERIFY_GAS caps of 100,000 (today’s value), 250,000, 300,000, 400,000 and 500,000. A run is one pass of a benchmark on one machine through every load type, cap and load level. For each load type and cap, the node first processes blocks with no load, which gives the baseline, and then under each load level. Every benchmark ran at least twice on each machine. We report the median over blocks and, where runs differ, the worst run.

The benchmark is public in Nethermind #14135. Our runs used the client code at master commit ccb07d17. Block building ran at commit 01cc0069, block import and the per-connection rate (Appendix A) at f859b806, and honest admission at 6de10bc9. Each result can be reproduced at its commit.

3.4 Delay budget

We judge the delays against two budgets, B = 25 ms and B = 50 ms. Any ms of delay counts, as a node that imports a block late may miss the attestation deadline, 4 s into the slot. Xatu measures that cost. It records 133 nodes imports over a mainnet block. With no added delay, a node imported the slot’s block only after the deadline in 3.86% of slots. Adding 10 ms of block delay raises that share by 0.07 percentage points, 25 ms by 0.19, 50 ms by 0.39, 100 ms by 0.85 and 250 ms by 2.79. The analysis and its data are public in block-import-delay-budget. There is no threshold below which delay is free, so the budget B is a policy choice. In this work we use 25 ms as the main budget and report 50 ms as a more tolerant one.

4. Benchmark results

We now report the two delay benchmarks, block building and block import.

4.1 Block building

The table below gives the block-building delay on our three machines. Each cell is the delay added to a median mainnet block with keccak-wide / sig-stuffed load types, at a cap of 300,000. At the same gas per second we obtained the same delay results with the other caps (see Appendix C). A cell marked “≥” means the core was saturated, with f within 0.01 of 0.5.

Once block processing and validation both keep the core busy, the operating system splits it about evenly, so f stops near 0.5 and any excess load queues. The delay is then at least T. The figure below shows f itself, the CPU share the attack takes from building on each machine.

Why slower machines suffer more. A slower core needs more of its time to validate the same gas/s, so the attack takes a larger share of it. Below saturation, f is roughly inversely proportional to single-thread performance. At 10M gas/s of sig-stuffed, for example, Low has about two thirds of High’s single-thread rating (1,700 against 2,600) and loses about 1.6 times its share (0.21 against 0.13).

Which machine matters. Building delay is judged on High and Mid. Within B = 25 ms both machines hold 30M gas/s for both load types, and 35M within 50 ms. On High and Mid, sig-stuffed is the harder load type, 1.3 to 1.6 times keccak-wide at every load. Low would hold only 15M gas/s within 25 ms and 20M within 50 ms.

On the p90 block. A p90 mainnet block takes about 1.8 times as long to process as the median one (§3.1). The delay is proportional to T, so we assume here every delay in the table grows by the same factor. High and Mid then hold 20M gas/s within 25 ms and 30M within 50 ms, and Low 10M and 15M.

4.2 Block import

The table below gives the block-import delay on our three machines. Each cell is the delay added to a median mainnet block, keccak-wide / sig-stuffed, at a cap of 300,000. Caps 250,000 and 400,000 give the same delays (Appendix C).

At 50M gas/s every machine is at or near the 0.5 limit.

Import loses about the same CPU share as building. Both benchmarks ran at 5M, 10M and 30M gas/s with caps 250,000 to 400,000. Figure below shows that, at the same machine, load type, cap and gas rate, the share f the attack takes from import matches roughly the share it takes from building.

On the p90 block. Import delays grow by the same factor of about 1.8. Low then stays within 25 ms only up to 10M gas/s, and within 50 ms up to 15M.

4.3 Input rate per cap

The limits above are in gas per second. Since `r` failing transactions per second at cap C deliver r × C gas per second, dividing each machine’s limit by the cap gives the highest transaction rate that cap sustains on the median mainnet block, in tx/s within B = 25 ms / 50 ms:

Each value is the worst case over runs and load types, and holds for both building and import, which lose about the same CPU share (§4.2). On High and Mid the 50 ms values are lower bounds, because their core saturates at 40M gas/s, the highest load we measured.

The figure below shows the same results for each load type, on the median block. Every cap uses the 300k measurements, which hold for all caps at the same gas per second (Appendix C).

100 tx/s at 300k is 30M gas/s. Assuming 100 tx/s as the reference rate (Appendix A), High and Mid add 11 and 13 ms with keccak-wide failing transactions and 17 and 22 ms with sig-stuffed ones, hence all stay within 25 ms on the median block (§4.1). Low is saturated, at least 42 ms. Along the 100 tx/s line in the figure, High and Mid stay within 25 ms up to 300k with both load types, and up to 400k with keccak-wide. Sig-stuffed exceeds 25 ms at 400k.

On the p90 block. At the 100 tx/s reference rate, 300k stays within 25 ms only with keccak-wide failing transactions, 20 ms on High and 23 on Mid. Sig-stuffed ones add 31 and 38 ms, within 50 ms but not 25.

4.4 Discussion

We ran the benchmarks in the worst case, with block processing and validation sharing one CPU core, on our High, Mid and Low machines. On each, failing transactions, whose validation prefix burns the whole cap and then fails, delay block processing. We observed that the delay grows with the gas they burn per second as expected. The attack takes about the same CPU share from block import as from block building, and at the same gas/s the cap does not change it.

The results therefore answer two questions.

1. How much validation can be absorbed, in gas/s? On the median mainnet block High absorbs 35M gas/s, Mid 30M and Low 15M. On the p90 block every delay is about 1.8 times larger, and they absorb 25M, 20M and 10M.
2. How many failing transactions per second an attacker needs to exceed the budget? At MAX_VERIFY_GAS = 100k the attacker needs more than 350, 300 and 150 tx/s on the median block, and more than 250, 200 and 100 on the p90 block. At 300k it needs more than 117, 100 and 50 on the median block, and more than 83, 67 and 33 on the p90 block. At the 100 tx/s reference rate, every machine therefore absorbs MAX_VERIFY_GAS = 100k. At 300k Low exceeds the budget, while High and Mid stay within it on the median block, and on the p90 block only with keccak-wide failing transactions.

We note that the operating system splits the core between frames validation and block processing. So the block gets only 1 − f of it and takes longer. As a countermeasure, a client can keep the core for block processing when a block needs it. It can, for example, pause or preempt validation while a block is imported (§5.1), cap the CPU it spends on frame validation per head (§5.2), and downgrade or disconnect a connection whose transactions keep failing validation, which bounds r (Appendix A). None of these was active in our runs above. We implemented them in the Nethermind client and we discuss their effect in the next section.

5. What protections change

As part of Nethermind’s frames implementation, we added protections that limit DoS exposure when MAX_VERIFY_GAS increases.

5.1 Block import: pause and preemption

We implemented a protection mechanism inside the Nethermind client to prioritize block processing when necessary. With #13994, validation of frame transactions gives way to block processing. No validation prefix starts while a block is processed, and one that is already running is stopped mid-execution. Notably, the interrupted transaction is not rejected but deferred.

We ran the import benchmark with and without this protection and measured the delay that failing transactions add to block import. The figure shows the median over runs of the delay on the synthetic import block (Appendix B), which takes 3.3 to 4.2 times as long as a median mainnet block.

With sig-stuffed failing transactions at 30M gas/s, the protections remove the 55 ms delay on High, and cut it from 69 to 0.8 ms on Mid and from 173 to 1.9 ms on Low. Due to its limited resources, on Low the protected node validated only about 40% of the transactions sent, so it handled about 12M gas/s rather than 30M and its 1.9 ms must be interpreted as a lower bound. Keccak-wide ones leave at most 2.3 ms on every machine.

5.2 The per-head CPU budget, and honest transactions

Nethermind limits the CPU it spends on frame transaction validation per new head with a fixed budget. This is a node policy. Within that budget the node runs the validation prefix of each incoming frame transaction to decide whether to admit it. Failing transactions cannot take more than that budget, so they cannot slow block processing beyond it. But under attack the honest frame transactions, those whose validation prefix succeeds, compete for the same budget. Once failing transactions have spent it, the node defers every honest frame transaction until the next head.

In our benchmark we set that budget at 1 s of CPU per head, about 8% of one core over a 12 s slot. Then, alongside the load of failing transactions, we submitted to the node a steady 5 honest frame transactions per second, each with a 50,000 gas validation prefix. The failing transactions were keccak-wide at a cap of 300k. The table below shows the share of these honest transactions the node admitted over three slots, as the median over two runs.

The figure follows the 12 s between one head and the next, split into four 3s windows. It shows the average share admitted in each window. The × on each curve marks when the budget runs out. On High at 10M gas/s, for example, that happens at 10.6 s. From 6 to 10.6 s the node still admits honest transactions and from 10.6 to 12 s it admits none, so the 6–12 s window averages 73%. At 30M and 50M it runs out at 3.6 and 2.2 s, and on Low at 5.5, 1.9 and 1.2 s. The admitted share in the table is roughly the fraction of those 12 s the budget lasts, except on Low, which is too busy to admit every honest transaction even before the budget runs out.

The budget is CPU time, so a slower CPU spends it on fewer transactions, and the loss depends on the load of failing transactions. At MAX_VERIFY_GAS 300k, 100 failing tx/s are 30M gas/s and leave 27% of our honest frame transactions admitted on High and 12% on Low. The budget turns a CPU problem into an inclusion problem, and the trade falls hardest on slower hardware.

6. Picking the number

To withstand a flood of 100 failing transactions per second, we identify 300,000 as a safe MAX_VERIFY_GAS, provided block import is protected.

With this configuration, we showed that block import stays within a 25 ms delay budget on every machine, and block building on High and Mid. Under flooding, with a per-head CPU budget of about 8% of one core, the node still admits 27% of honest frame transactions on High, 25% on Mid and 12% on Low (§5.2). We measured these shares with keccak-wide failing transactions. The rest of this section shows how that number follows from the measurements, and how it moves with the transaction rate and the delay budget.

To pick a number we have to identify the configuration that does not exceed the desired delay budget. Without Nethermind’s protections, the bottleneck is block import on Low, which absorbs 15M gas/s within 25 ms and 20M within 50 ms. With import protected, the bottleneck is block building on High and Mid, which absorb 30M gas/s within 25 ms and 35M gas/s within 50 ms. Writing G_B for the gas per second the bottleneck absorbs within B, the largest cap is:

C_{max(r)} = G_B / r, rounded down to a tested cap.

For example, with import protected, r = 100 tx/s and B = 25 ms give C_{max} = 30M / 100 = 300,000.

The table below gives C_{max} for a few rates, on the median mainnet block with sig-stuffed failing transactions. The first row is the gas per second G_B each setup absorbs within its delay budget B, and every other cell is the largest tested MAX_VERIFY_GAS, in gas per transaction.

* By the formula the cap is potentially higher than what we tested. For example, High and Mid with protections at 150 tx/s within 50 ms result in ~233k cap, but no cap was tested in our configurations between 100,000 and 250,000.

When measured on the p90 block, the configuration with 300,000 at 100 tx/s still holds within the B = 25 ms delay budget for keccak-wide failing transactions only, while for sig-stuffed ones the budget needs to increase to B = 50 ms (§4.3).

To conclude, at the 100 tx/s reference rate 300,000 is the largest tested cap that holds on High and Mid, and it admits the use cases that motivate raising the cap, for example privacy transfers with Groth16 proofs and hash-based post-quantum signatures (§1). This result holds only when the execution client protects block import as described in §5.1. Without protections, block import on Low brings the cap back to 100,000 (§4.2).

7. Limits of the study

Our benchmarks have the following limitations:

• They ran on one client, with a single core shared by block processing and frame validation.
• We tested three hardware profiles but none as slow as EIP-7870’s full node tier.
• The benchmarks ran on synthetic blocks, not mainnet ones (Appendix B). We measured on them the CPU share f the attack takes, and applied it to the time T for mainnet blocks. This assumes the attack takes the same share from any block work. Appendix B checks it on a much longer block, where it holds within 25% and errs toward overstating the delay, but not on blocks of different content.
• Every attack ran with zero state gas. Cold reads from a large existing contract are unmeasured.
• Our load counts each failing transaction once, rejected at admission. A frame transaction that is admitted and fails later costs more. It is gossiped to peers that validate it again, which we did not measure. And if it fails VERIFY at build time, it can be re-executed on later builds without paying.

Appendix

A. Transactions per connection

Transactions reach the mempool over the node’s network connections, and Nethermind limits how many it processes from each. Above 600 rejects in 60 s a connection is downgraded and most of its messages ignored. Above 6,000 it is disconnected. The result is that one connection gets at most about 100 failing tx/s processed, which it reaches when it offers about 800 tx/s. Above that it is disconnected. That is the reference rate used throughout this post.

The figure shows how many failing transactions per second one connection gets processed, against the rate it offers. A connection offering 50 tx/s is already downgraded, and about 89% of its messages are then ignored, which is why processed throughput climbs so slowly with offered rate. A connection offering 1,000 tx/s or more is disconnected within the 60 s window. The reference rate is a Nethermind DoS-protection policy, not a property of the attack. An attacker holding more connections, or a client that limits a connection less or not at all, delivers a higher rate. The verification load on the node then scales with the cap C and the rate r, as r × C (§4.3).

B. The benchmark blocks

We ran every benchmark in a test environment, on a single node processing synthetic blocks. While the node built or imported these blocks, a separate stream of failing transactions, the load, reached its mempool, and the node validated them on the same CPU core. Hence, we measured how much of the CPU core the attack takes from block work, the share f (§2.2). Our assumption is that f depends on the machine and on the gas/s load and load type, but not on which block the node is processing. Under that assumption, f measured on a synthetic block carries over to a mainnet block, and the delay follows from the mainnet block time T as Δ = T · f / (1 − f). Below we describe how we estimated f for block building and for block import, and then how we validated the assumption.

Block building. The building benchmark repeatedly built a synthetic block holding a single failing transaction of its own, separate from the load. Building it takes about 1 ms of CPU on High at 100k and 3 ms at 300k. If the node completes \text{N}0 builds in a time window without the attack and \text{N} under it, then f = 1 − \text{N} / \text{N}0.

Block import. The import benchmark repeatedly imported a synthetic 10M-gas block of keccak-wide hashing. One import takes about 98 ms on High, 98 ms on Mid and 178 ms on Low, about 3.6, 3.3 and 4.2 times a median mainnet block. If one import takes \text{W}0 without the attack and \text{W}0 + Δ under it, then f = Δ / (\text{W}0 + Δ).

Validating the assumption.

• Different blocks, same f. The two benchmark blocks showed that at the same machine, load type, cap and gas rate they lose the same f within three percentage points of the core on Mid and Low, while on High import loses up to five points less (§4.2).
• Synthetic blocks, same f. Our synthetic build takes only 1 to 3 ms, far shorter than a real block. To check that f still holds, we also built a block longer than a p90 mainnet block under the same load. Its delay matched the prediction within 10% in 31 of 36 cases and within 25% in all.
• Different caps, same f. At the same gas/s, caps from 100k to 500k take about the same CPU share f, differing by at most 4 percentage points. Hence, we conclude that the CPU consumption follows the load, not the size of each transaction.

These checks do not cover block content. A mainnet block mixes storage access, calls and precompiles, and none of our blocks does.

C. The delay across caps

In §4.1 and §4.2 we set the cap at 300,000. In the figure below we show measurements for different caps, and same input load (gas/s). We notice that what impacts the delay is not the cap itself, but depends on the input load.

Block building was run at all five caps and block import at 250,000, 300,000 and 400,000. At the same gas per second, f differs across caps by at most 4 percentage points in both benchmarks, on every machine and load type. The figure shows only building, where the largest gap is on Low with sig-stuffed failing transactions at 5M gas/s.