8000 test: fuzz test for foyer-memory by xiaguan · Pull Request #337 · foyer-rs/foyer · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

test: fuzz test for foyer-memory #337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 264 commits into
base: main
Choose a base branch
from

Conversation

xiaguan
Copy link
Contributor
@xiaguan xiaguan commented Apr 14, 2024

What's changed and what's your intention?

Please explain IN DETAIL what the changes are in this PR and why they are needed. :D

As title

Checklist

  • I have written the necessary rustdoc comments
  • I have added the necessary unit tests and integration tests
  • I have passed make all (or make fast instead if the old tests are not modified) in my local environment.

Related issues or PRs (optional)

MrCroxx added 30 commits May 26, 2023 16:35
* chore: add license checker

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix license checker config

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: introduce instrusive indexers and collections

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: introduce FTL-like storage engine

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* update ci

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* sort cargo file

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix memory leak

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: enable direct i/o on linux target

- enable direct i/o on linux target
- refine flusher and reclaimer

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix unit test

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
…s#30)

* chore: remove unused old storage engien and other components

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* update ci

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: impl storage recovery

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* chore

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: add segment fifo eviction policy

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* export fifo fs store

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: export mods

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* make cargo sort happy

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* fix: deadlock

fix deadlocks:

1. exclusive lock was not `Send` but unsafe impled.
2. submit flush task await blocks `set_region_evictable`
3. no new reclamation task created after `set_region_evctable` if all
   write process are waiting

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* update ci

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* fix: gracefullly shutdown runners

Use `Store::shutdown_runners()` to gracefully shutdown runners.`

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* log warn if send failed instead of panic

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
- fix region advance
- fix recovery slice destroy
- seal last dirty region when shutdown

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: introduce prometheus metrics

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* calc metrics

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix registry metrics

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
…oyer-rs#56)

- support multiple admission or reinsertion policies
- loose flush exclusive lock

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* feat: introduce rated random admission policy

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* make clippy happy

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
MrCroxx and others added 22 commits April 8, 2024 07:53
…#307)

* chore: bump foyer-memory to 0.1.3

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* chore: bump foyer-intrusive to 0.3.1

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* refactor: remove usage of unstable features

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* refactor: some more

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
…er-rs#317)

* refactor: use crate allocator_api2 to make foyer build on stable

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: use stable pipeline for ci tests, use nightly for sanitizer

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: update ci prefix key

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: try fix ci

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: update ci nightly version

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* chore: try resolve github issue

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* test: run CI on both linux and macos

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: fix build on macos

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: try fix ci run on targets

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* doc: update README and add a simple example

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* chore: update Cargo.toml

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* chore: fix ci coverage

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* chore: fix check

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
)

* refactor: remove Handle bound from Eviction

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* refactor: loose bound of in-memory eviction related traits

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
* test: upload binaries when asan/lsan test fails

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: fix if condition

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

* fix: use different name for different artifacts

Signed-off-by: MrCroxx <mrcroxx@outlook.com>

---------

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: xiaguan <751080330@qq.com>
Signed-off-by: xiaguan <751080330@qq.com>
@MrCroxx
Copy link
Member
MrCroxx commented Apr 16, 2024

Sorry for the delay. I've been working on some refactors these days.

What's the size of the fuzzy test dataset? Should it also be uploaded?

Comment on lines +110 to +115
#[derive(Debug, Arbitrary)]
struct Input {
capacity: usize,
cache_type: CacheType,
operations: Vec<Op>,
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cargo-fuzz just random generate the Input to max the code coverage with sanitizer.
The afl fuzz seems better one, it need some input to start,and it is hard to use and maintain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

6 participants
0