Description
Problem
The bandwidth consumption in CometBFT is high. In particular, this comes from the fact that the reactor of the mempool is a basic flooding gossip protocol (à la Bitcoin). Because the network topology is dense, it is very likely that a node receives a transaction multiple times from different neighbors. Notice that such a redundancy is useful to prevent byzantine or selfish nodes to stop the dissemination of a transaction in the network.
Some evidence of this redundancy is seen in #613
Target audience
Consensus developers
Upside
Small improvements could cut bandwidth usage in CometBFT by (at least) 25%, based on other results.
Downside
There are many possible improvements to test. To mitigate the risk of spending too much time on discussing improvements, we will divide the improvements into two batches, one with quick wins and one with ideas that require further though and refinement.
For each of the approaches that are chosen on the brainstorm we will
- prototype the approach
- experiment with improvements using e2e and fast prototyping
- experiment with the approaches that turned out promising in the QA infra
- document the solutions as ADRs
The remaining ideas will become part of #11 and if there is room to try these and other ideas on that issue, we will iterate on brainstorming and prototyping.
Consider if #1048 is pre-requisite for this task of if backlog.
Definition of done
- Provide in the form of an ADR a new gossip protocol, or an improvement to the current one, that reduces the bandwidth of mempool transactions, as demonstrated by experiments on a testnet that resembles as close as possible to a real-world mainnet.
- Implement, test, and merge into main the solution proposed in the ADR.
- Protocols or ideas discussed but not implemented or prototyped should also be documented.