Storage optimization #44
Assignees
Labels
major-priority
A major, long-running priority for the team
storage
tracking
A complex issue broken down into sub-problems
Comments
5 tasks
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
High-level tracking issue for general storage optimization efforts. This issue can be expanded over time.
At present (mid 2023), depending on their configuration, Tendermint-based nodes use large quantities of storage space. This has significant cost implications for operators. We aim to implement strategies to reduce and/or offload certain data stored in order to reduce operators' costs.
The two main problems that are present in the CometBFT storage layer:
To address these problems, we first need to build understanding of:
The work to be done can be broken down in the following main subsections:
The end result of this work should be CometBFT optimized for a single storage backend which ultimately results in a significant reduction in both storage access time and on disk storage footprint.
To reach this goal we envision the following steps :
Preliminary investigation to identify the users and workloads of the storage backend (how they query the nodes, what are their common pain points with regards to storage, collection of issues to address).
Tune CometBFT to address storage related bottlenecks
Part of this section covers addressing issues found during the benchmarking and investigation process outlined above. Another part addresses concrete issues reported by users. While part of this issues cannot be fully addressed before the analysis above, some optimizations can be performed on CometBFT as it is today - marked with * .
The Genesis file can be large and surpass internal DB file size limitations (3GB for RocksDB).
It seems that for users pruning the indexer is not as high a priority and well understood as reducing the footprint and reducing the potential DoS vector querying it can be.
CometBFT stores and allows querying of data not essential for consensus
We need to Identify the functionalities we want to support within Tendermint and offload non-critical data and functionality.
main#816This implementation provides users with an API to implement their own event indexing and prune the full nodes who store events at the moment.
CometBFT currently maintains its own [WAL](https://github.com/cometbft/cometbft/blob/101bf50e715d6a10c8135392166c35bdae94972e/consensus/wal.go) - is this even necessary, given that the underlying database should actually be taking care of this? It is another source of complexity and potential point of failure in the system that the team has to maintain.
Original issue: tendermint/tendermint#9881
The text was updated successfully, but these errors were encountered: