8000 Check compaction output file sizes · Issue #13619 · facebook/rocksdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Check compaction output file sizes #13619
Open
@pdillinger

Description

@pdillinger

I believe that if a compaction output is somehow corrupted to have extra data after the actual RocksDB written data, we don't detect that until the DB is re-opened and you get "Sst file size mismatch: ...". We should include this sanity check on installing compaction results so that we don't end up with an un-openable DB.

Remote compactions currently have kind of the opposite problem: the file size is read from the FileSystem and treated as authoritative. So you might get a "Bad magic number" error when really the problem is an incomplete file was read/queried. We should be reporting the size mismatch by propagating the file size from the remote compaction worker.

Both kinds of compaction should be cross-checking between when was independently recorded as the size written and the size reported by the FileSystem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0