An embedded Key-Value storage engine. Both key (coded as "PageId") and value (coded as "Page") are arbitrary bytes.
Some features which make PageHouse unique among all kinds of K-V engines:
- PageHouse can store values with any size. And storing large values (e.g. 64KiB to 64 MiBs) is welcome in PageHouse.
- Support snapshot. You can read on a stable view without worrying about the consistency.
- Support reference. You can store a ref-page of another page. Similar to hard link in a file system.
- Support external values. You can store your real value data in anywhere, and then put a link in PageHouse. E.g. files on local disk or object storage.
- Resource efficiency.
- Each
read(key)requires exactly 1 system read. The data could be already in system page cache if you are lucky. - Each
write(key, value)requires 2 system writes - The background tasks are carefully designed and try to avoid the influence of foreground tasks.
- Each
- PageHouse cache all page ids and page meta (including offset, size, and other information) in memory. It could become a bottleneck if your memory is limited.
- 1 page meta consumes ~500 bytes memory currently.
- We plan to dramatically reduce the memory usage in the near future.
PageHouse is under the Apache 2.0 license. See the LICENSE file for details.
- PageHouse is not production ready yet! Don't use it in your production environment!
- PageHouse is used to be part of TiFlash