Article

RocolSys: An Automatic Row-Column Data Storage System for HTAP

Authors:

Hao Wu,

Wei ZhaoAuthors Info & Claims

Web and Big Data: 8th International Joint Conference, APWeb-WAIM 2024, Jinhua, China, August 30 – September 1, 2024, Proceedings, Part V

Pages 368 - 372

https://doi.org/10.1007/978-981-97-7244-5_27

Published: 31 August 2024 Publication History

Abstract

Hybrid Transactional and Analytical Processing (HTAP) brings great challenges to data storage systems. However, traditional approaches have struggled to dynamically adapt storage structures to data and complex workloads. Fortunately, machine learning can provide new methods to guide decisions on data storage structure selection. Motivated by this, we develop RocolSys, an automatic hybrid data storage system for HTAP that can predict workloads and select storage structure automatically based on machine learning. RocolSys predicts workloads accurately and selects storage structures efficiently. It also provides the user-friendly interface that allows users to connect to their own databases. We demonstrate the efficiency of RocolSys on public benchmarks.

References

[1]

Zhang C, Li GL, Feng JH, and Zhang JT Survey of key techniques of HTAP databases Ruan Jian Xue Bao/J. Softw. 2023 34 2 761-785

Google Scholar

[2]

Abadi, D.J., Madden, S.R., Hachem, N.: Column stores vs. row stores: how different are they really? In: Proceedings of the 2008 ACM SIGMOD (2008)

Google Scholar

[3]

Domingos, P.: Machine learning for data management: problems and solutions. In: SIGMOD, pp. 629–629 (2018)

Google Scholar

[4]

OpenGauss. https://opengauss.org/zh/

Google Scholar

[5]

Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD (2016)

Google Scholar

[6]

Ma, L., Aken, D.V., Hefny, A., et al.: Query-based workload forecasting for selfdriving database management systems. In: The International Conference (2018)

Google Scholar

[7]

TPC-H (2023). https://www.tpc.org/tpch/

Google Scholar

Index Terms

RocolSys: An Automatic Row-Column Data Storage System for HTAP
1. Information systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Cost-Based Lightweight Storage Automatic Decision for In-Database Machine Learning
Web Information Systems Engineering – WISE 2021
Abstract
Storage structure decision for a database aims to automatically determine the effective storage structure according to the data distribution and workload. With the integration of machine learning and database becoming closer, complex machine ...
A Comparative Study of Row and Column Storage for Time Series Data
Spatial Data and Intelligence
Abstract
Over the past few decades, researchers have done massive research on data storage structures of relational databases. The existing research mainly focuses on the analysis and optimization of row and column storage structures in relational ...
Automatic Document Data Storage System Based on Machine Learning
Web and Big Data
Abstract
Document storage management plays a significant role in the field of database. With the advent of big data, making storage management manually becomes more and more difficult and inefficient. There are many researchers to develop algorithms for ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Web and Big Data: 8th International Joint Conference, APWeb-WAIM 2024, Jinhua, China, August 30 – September 1, 2024, Proceedings, Part V

Aug 2024

530 pages

ISBN:978-981-97-7243-8

DOI:10.1007/978-981-97-7244-5

Editors:
Wenjie Zhang
University of New South Wales, Sydney, NSW, Australia
,
Anthony Tung
National University of Singapore, Queenstown, Singapore
,
Zhonglong Zheng
Zhejiang Normal University, Jinhua, China
,
Zhengyi Yang
University of New South Wales, Sydney, NSW, Australia
,
Xiaoyang Wang
University of New South Wales, Sydney, NSW, Australia
,
Hongjie Guo
Zhejiang Normal University, Jinhua, China

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 31 August 2024

Author Tags

Qualifiers

Article

Index Terms

Recommendations

Cost-Based Lightweight Storage Automatic Decision for In-Database Machine Learning

A Comparative Study of Row and Column Storage for Time Series Data

Automatic Document Data Storage System Based on Machine Learning