More Web Proxy on the site http://driver.im/

research-article

Understanding the effect of data center resource disaggregation on production DBMSs

Authors:

Sebastian Angel,

Boon Thau LooAuthors Info & Claims

Proceedings of the VLDB Endowment, Volume 13, Issue 9

Pages 1568 - 1581

https://doi.org/10.14778/3397230.3397249

Published: 01 May 2020 Publication History

Abstract

Resource disaggregation is a new architecture for data centers in which resources like memory and storage are decoupled from the CPU, managed independently, and connected through a high-speed network. Recent work has shown that although disaggregated data centers (DDCs) provide operational benefits, applications running on DDCs experience degraded performance due to extra network latency between the CPU and their working sets in main memory. DBMSs are an interesting case study for DDCs for two main reasons: (1) DBMSs normally process data-intensive workloads and require data movement between different resource components; and (2) disaggregation drastically changes the assumption that DBMSs can rely on their own internal resource management.

We take the first step to thoroughly evaluate the query execution performance of production DBMSs in disaggregated data centers. We evaluate two popular open-source DBMSs (MonetDB and PostgreSQL) and test their performance with the TPC-H benchmark in a recently released operating system for resource disaggregation. We evaluate these DBMSs with various configurations and compare their performance with that of single-machine Linux with the same hardware resources. Our results confirm that significant performance degradation does occur, but, perhaps surprisingly, we also find settings in which the degradation is minor or where DDCs actually improve performance.

References

[1]

Apache spark - unified analytics engine for big data. https://spark.apache.org.

[2]

Big data analytics on-premises, in the cloud, or on hadoop --- vertica. https://www.vertica.com.

[3]

M. K. Aguilera, N. Amit, I. Calciu, X. Deguillard, J. Gandhi, S. Novakovic, A. Ramanathan, P. Subrahmanyam, L. Suresh, K. Tati, R. Venkatasubramanian, and M. Wei. Remote regions: a simple abstraction for remote memory. In Proceedings of the USENIX Annual Technical Conference (ATC), 2018.

[4]

M. K. Aguilera, N. Amit, I. Calciu, X. Deguillard, J. Gandhi, P. Subrahmanyam, L. Suresh, K. Tati, R. Venkatasubramanian, and M. Wei. Remote memory in the age of fast networks. In Proceedings of the ACM Symposium on Cloud Computing (SOCC), 2017.

Digital Library

[5]

M. K. Aguilera, K. Keeton, S. Novakovic, and S. Singhal. Designing far memory data structures: Think outside the box. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS), 2019.

Digital Library

[6]

Alibaba. ApsaraDB for POLARDB: A next-generation relational database - Alibaba cloud. https://www.alibabacloud.com/products/apsaradb-for-polardb, 2019.

[7]

Amazon-Aurora. Amazon aurora - Relational database built for the cloud - AWS. https://aws.amazon.com/rds/aurora/, 2019.

[8]

S. Angel, M. Nanavati, and S. Sen. Disaggregation and the application. In Proceedings of the USENIX Workshop on Hot Topics in Cloud Computing (HotCloud), July 2020.

[9]

J. Arulraj and A. Pavlo. How to build a non-volatile memory database management system. In S. Salihoglu, W. Zhou, R. Chirkova, J. Yang, and D. Suciu, editors, Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, Chicago, IL, USA, May 14-19, 2017, pages 1753--1758. ACM, 2017.

[10]

C. Barthels, S. Loesing, G. Alonso, and D. Kossmann. Rack-scale in-memory join processing using RDMA. In Proceedings of the ACM SIGMOD Conference, 2015.

Digital Library

[11]

Q. Cai, W. Guo, H. Zhang, D. Agrawal, G. Chen, B. C. Ooi, K. Tan, Y. M. Teo, and S. Wang. Efficient distributed memory management with RDMA and caching. PVLDB, 11(11):1604--1617, 2018.

Digital Library

[12]

A. Carbonari and I. Beschastnikh. Tolerating Faults in Disaggregated Datacenters. In Proceedings of the ACM Workshop on Hot Topics in Networks (HotNets), 2017.

Digital Library

[13]

Citus-Data. Citus data: Worry-free postgres. built to scale out. https://www.citusdata.com/, 2019.

[14]

A. Dragojević, D. Narayanan, O. Hodson, and M. Castro. FaRM: Fast Remote Memory. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2014.

[15]

D. Duplyakin, R. Ricci, A. Maricq, G. Wong, J. Duerig, E. Eide, L. Stoller, M. Hibler, D. Johnson, K. Webb, A. Akella, K. Wang, G. Ricart, L. Landweber, C. Elliott, M. Zink, E. Cecchet, S. Kar, and P. Mishra. The design and operation of CloudLab. In Proceedings of the USENIX Annual Technical Conference (ATC), July 2019.

[16]

M. J. Franklin, M. J. Carey, and M. Livny. Global memory management in client-server database architectures. In L. Yuan, editor, 18th International Conference on Very Large Data Bases, August 23-27, 1992, Vancouver, Canada, Proceedings, pages 596--609. Morgan Kaufmann, 1992.

[17]

P. X. Gao, A. Narayan, S. Karandikar, J. Carreira, S. Han, R. Agarwal, S. Ratnasamy, and S. Shenker. Network requirements for resource disaggregation. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2016.

[18]

J. Giceva, G. Zellweger, G. Alonso, and T. Roscoe. Customized OS support for data-processing. In Proceedings of the 12th International Workshop on Data Management on New Hardware, DaMoN 2016, San Francisco, CA, USA, June 27, 2016, pages 2:1--2:6. ACM, 2016.

Digital Library

[19]

J. Gu, Y. Lee, Y. Zhang, M. Chowdhury, and K. G. Shin. Efficient memory disaggregation with Infiniswap. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2017.

[20]

F. Li, S. Das, M. Syamala, and V. R. Narasayya. Accelerating relational databases by leveraging remote memory and RDMA. In Proceedings of the ACM SIGMOD Conference, 2016.

Digital Library

[21]

K. T. Lim, J. Chang, T. N. Mudge, P. Ranganathan, S. K. Reinhardt, and T. F. Wenisch. Disaggregated memory for expansion and sharing in blade servers. In Proceedings of the International Symposium on Computer Architecture (ISCA), 2009.

Digital Library

[22]

K. T. Lim, Y. Turner, J. R. Santos, A. AuYoung, J. Chang, P. Ranganathan, and T. F. Wenisch. System-level implications of disaggregated memory. In Proceedings of the IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2012.

Digital Library

[23]

X. Liu, A. Aboulnaga, K. Salem, and X. Li. CLIC: client-informed caching for storage servers. In M. I. Seltzer and R. Wheeler, editors, 7th USENIX Conference on File and Storage Technologies, February 24-27, 2009, San Francisco, CA, USA. Proceedings, pages 297--310. USENIX, 2009.

[24]

S. Manegold, P. A. Boncz, and N. Nes. Cache-conscious radix-decluster projections. In M. A. Nascimento, M. T. Özsu, D. Kossmann, R. J. Miller, J. A. Blakeley, and K. B. Schiefer, editors, (e)Proceedings of the Thirtieth International Conference on Very Large Data Bases, VLDB 2004, Toronto, Canada, August 31 - September 3 2004, pages 684--695. Morgan Kaufmann, 2004.

[25]

Microsoft-SQL-Database. Sql database - cloud database as a service --- Microsoft Azure. https://azure.microsoft.com/en-us/services/sql-database/, 2019.

[26]

MonetDB. Monetdb - the column-store pioneer. https://www.monetdb.org/Home, 2019.

[27]

I. Müller, P. Sanders, A. Lacurie, W. Lehner, and F. Färber. Cache-efficient aggregation: Hashing is sorting. In T. K. Sellis, S. B. Davidson, and Z. G. Ives, editors, Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, Victoria, Australia, May 31 - June 4, 2015, pages 1123--1136. ACM, 2015.

[28]

K. Ousterhout, R. Rasti, S. Ratnasamy, S. Shenker, and B. Chun. Making sense of performance in data analytics frameworks. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2015.

[29]

J. M. Patel, H. Deshmukh, J. Zhu, N. Potti, Z. Zhang, M. Spehlmann, H. Memisoglu, and S. Saurabh. Quickstep: A data platform based on the scaling-up approach. PVLDB, 11(6):663--676, 2018.

[30]

Postgres-XL. Postgres-xl: Open source scalable sql database cluster. https://www.postgres-xl.org/, 2019.

[31]

PostgreSQL. PostgreSQL: The world's most advanced open source relational database. https://www.postgresql.org/, 2019.

[32]

A. Shamis, M. Renzelmann, S. Novakovic, G. Chatzopoulos, A. Dragojevic, D. Narayanan, and M. Castro. Fast general distributed transactions with opacity. In P. A. Boncz, S. Manegold, A. Ailamaki, A. Deshpande, and T. Kraska, editors, Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019, pages 433--448. ACM, 2019.

[33]

Y. Shan, Y. Huang, Y. Chen, and Y. Zhang. LegoOS: A Disseminated, Distributed OS for Hardware Resource Disaggregation. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2018.

[34]

V. Shrivastav, A. Valadarsky, H. Ballani, P. Costa, K. S. Lee, H. Wang, R. Agarwal, and H. Weatherspoon. Shoal: A Network Architecture for Disaggregated Racks. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2019.

[35]

M. Stonebraker. Operating system support for database management. Communications of the ACM, 24(7), June 1981.

[36]

A. van Renen, V. Leis, A. Kemper, T. Neumann, T. Hashida, K. Oe, Y. Doi, L. Harada, and M. Sato. Managing non-volatile memory in database systems. In G. Das, C. M. Jermaine, and P. A. Bernstein, editors, Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, June 10-15, 2018, pages 1541--1555. ACM, 2018.

[37]

G. Yadgar, M. Factor, K. Li, and A. Schuster. Management of multilevel, multiclient cache hierarchies with application hints. ACM Trans. Comput. Syst., 29(2):5:1--5:51, 2011.

Digital Library

[38]

Q. Zhang, Y. Cai, S. Angel, A. Chen, V. Liu, and B. T. Loo. Rethinking data management systems for disaggregated data centers. In Proceedings of Conference on Innovative Data Systems Research (CIDR), Jan. 2020.

Cited By

Liu YXie MShi SXu YLitz HQian C(2024)Outback: Fast and Communication-Efficient Index for Key-Value Store on Disaggregated MemoryProceedings of the VLDB Endowment10.14778/3705829.370584918:2(335-348)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.14778/3705829.3705849
Lu BHuang KLiang CWang TLo E(2024)DEX: Scalable Range Indexing on Disaggregated MemoryProceedings of the VLDB Endowment10.14778/3675034.367505017:10(2603-2616)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.14778/3675034.3675050
Wu CAmiri MQin HMehta BMarcus RLoo B(2024)Towards Full Stack Adaptivity in Permissioned BlockchainsProceedings of the VLDB Endowment10.14778/3641204.364121617:5(1073-1080)Online publication date: 2-May-2024
https://dl.acm.org/doi/10.14778/3641204.3641216
Show More Cited By

Recommendations

Resource Bricolage for Parallel DBMSs on Heterogeneous Clusters

Running parallel database systems in an environment with heterogeneous resources has become increasingly common, due to cluster evolution and increasing interest in moving applications into public clouds or shared infrastructures. For database systems ...
Optimizing Resource Utilization of a Data Center

To provision IT solutions with reduced operating expenses, many businesses are moving their IT infrastructures into public data centers or are starting to build their own private data centers. Data centers can provide flexible resource provisioning in ...
Towards workload-aware dbmss: identifying workload type and predicting its change

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment

Proceedings of the VLDB Endowment Volume 13, Issue 9

May 2020

295 pages

ISSN:2150-8097

Editors:
Magdalena Balazinska
University of Washington
,
Xiaofang Zhou
University of Queensland, Australia

Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 May 2020

Published in PVLDB Volume 13, Issue 9

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
430
Total Downloads

Downloads (Last 12 months)69
Downloads (Last 6 weeks)3

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YXie MShi SXu YLitz HQian C(2024)Outback: Fast and Communication-Efficient Index for Key-Value Store on Disaggregated MemoryProceedings of the VLDB Endowment10.14778/3705829.370584918:2(335-348)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.14778/3705829.3705849
Lu BHuang KLiang CWang TLo E(2024)DEX: Scalable Range Indexing on Disaggregated MemoryProceedings of the VLDB Endowment10.14778/3675034.367505017:10(2603-2616)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.14778/3675034.3675050
Wu CAmiri MQin HMehta BMarcus RLoo B(2024)Towards Full Stack Adaptivity in Permissioned BlockchainsProceedings of the VLDB Endowment10.14778/3641204.364121617:5(1073-1080)Online publication date: 2-May-2024
https://dl.acm.org/doi/10.14778/3641204.3641216
Xiong ZJiang DXiong J(2024)DiStore: A Fully Memory Disaggregation Friendly Key-Value Store with Improved Tail Latency and Space EfficiencyProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673088(607-617)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3673038.3673088
Widmoser MKocher DAugsten N(2024)Scalable Distributed Inverted List Indexes in Disaggregated MemoryProceedings of the ACM on Management of Data10.1145/36549742:3(1-27)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654974
Hao XZhou XYu XStonebraker M(2024)Towards Buffer Management with Tiered Main MemoryProceedings of the ACM on Management of Data10.1145/36392862:1(1-26)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639286
An HWang FFeng DZou XLiu ZZhang J(2024)A Scalable and Write-Optimized Disaggregated B+-Tree With Adaptive Cache AssistanceIEEE Transactions on Cloud Computing10.1109/TCC.2024.343747212:4(1074-1087)Online publication date: Oct-2024
https://doi.org/10.1109/TCC.2024.3437472
Ng DLin AKashyap ALi GLu X(2024)NVMe-oPF: Designing Efficient Priority Schemes for NVMe-over-Fabrics with Multi-Tenancy Support2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00052(519-531)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPS57955.2024.00052
Zhao YLi YXu ZYang TYang KChen LYao XZhang G(2024) Fat-B + Tree: Fast B + tree Indexing with In-Network Memory 2024 IEEE International Performance, Computing, and Communications Conference (IPCCC)10.1109/IPCCC59868.2024.10850425(1-10)Online publication date: 22-Nov-2024
https://doi.org/10.1109/IPCCC59868.2024.10850425
Wang RGao CWang JKadam PTamerÖzsu MAref W(2024)Optimizing LSM-based indexes for disaggregated memoryThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-024-00863-y33:6(1813-1836)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1007/s00778-024-00863-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents