Establish and implement the relevant metrics to understand storage workloads #46
Closed
3 tasks done
Assignees
Labels
metrics
P:storage-optimization
Priority: Give operators greater control over storage and storage optimization
storage
Milestone
Comments
4 tasks
Closed
4 tasks
6 tasks
|
For Q4 2023, we will focus on:
This list might be altered after an in person meeting of the team deciding on the exact measurments that will help us achieve our Q4 goals. The main result of this testing should be :
|
The throughput can be hard to measure as it depends on many factors. Is it better to measure the duration of pruning (if possible)? This is similar to how the Golang core team optimised their garbage collector by looking at the duration of GC pauses. |
3 tasks
github-merge-queue bot
pushed a commit
that referenced
this issue
Feb 13, 2024
This PR superceeds #79 with some adjustments to the segments of code timed as well as bucket sizes. The majority of the code was done by William in #79. I tried to fine tune the measurements to exclude proto marshalling/unmarshalling where I think it made sense. Closes #46 Blocked on benchmarking to confirm it is measuring what we want. (follow up) The metrics gave us nice and realistic measurements in our benchmarks. <!-- Please add a reference to the issue that this PR addresses and indicate which files are most critical to review. If it fully addresses a particular issue, please include "Closes #XXX" (where "XXX" is the issue number). If this PR is non-trivial/large/complex, please ensure that you have either created an issue that the team's had a chance to respond to, or had some discussion with the team prior to submitting substantial pull requests. The team can be reached via GitHub Discussions or the Cosmos Network Discord server in the #cometbft channel. GitHub Discussions is preferred over Discord as it allows us to keep track of conversations topically. https://github.com/cometbft/cometbft/discussions If the work in this PR is not aligned with the team's current priorities, please be advised that it may take some time before it is merged - especially if it has not yet been discussed with the team. See the project board for the team's current priorities: https://github.com/orgs/cometbft/projects/1 --> --- #### PR checklist - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments --------- Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
mergify bot
pushed a commit
that referenced
this issue
Feb 22, 2024
This PR superceeds #79 with some adjustments to the segments of code timed as well as bucket sizes. The majority of the code was done by William in #79. I tried to fine tune the measurements to exclude proto marshalling/unmarshalling where I think it made sense. Closes #46 Blocked on benchmarking to confirm it is measuring what we want. (follow up) The metrics gave us nice and realistic measurements in our benchmarks. <!-- Please add a reference to the issue that this PR addresses and indicate which files are most critical to review. If it fully addresses a particular issue, please include "Closes #XXX" (where "XXX" is the issue number). If this PR is non-trivial/large/complex, please ensure that you have either created an issue that the team's had a chance to respond to, or had some discussion with the team prior to submitting substantial pull requests. The team can be reached via GitHub Discussions or the Cosmos Network Discord server in the #cometbft channel. GitHub Discussions is preferred over Discord as it allows us to keep track of conversations topically. https://github.com/cometbft/cometbft/discussions If the work in this PR is not aligned with the team's current priorities, please be advised that it may take some time before it is merged - especially if it has not yet been discussed with the team. See the project board for the team's current priorities: https://github.com/orgs/cometbft/projects/1 --> --- #### PR checklist - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments --------- Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> (cherry picked from commit dfd3f6c) # Conflicts: # internal/store/store.go
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Was tendermint/tendermint#9773
We need to identify the set of metrics to understand the storage workloads of CometBFT. The metrics should help us identify:
Open questions: Do we want information on which CometBFT Blockstore / StateStore method call triggered the access or is read/write/delete count and timing enough?
Draft implementation: tendermint/tendermint#9774
DoD
The text was updated successfully, but these errors were encountered: