More Web Proxy on the site http://driver.im/

research-article

Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication for Enterprise Applications

Authors:

Matthias Uflacker,

Hasso PlattnerAuthors Info & Claims

IMDM '15: Proceedings of the 3rd VLDB Workshop on In-Memory Data Mangement and Analytics

Article No.: 7, Pages 1 - 7

https://doi.org/10.1145/2803140.2803147

Published: 31 August 2015 Publication History

Abstract

In-memory database systems are well-suited for enterprise workloads, consisting of transactional and analytical queries. A growing number of users and an increasing demand for enterprise applications can saturate or even overload single-node database systems at peak times. Better performance can be achieved by improving a single machine's hardware but it is often cheaper and more practicable to follow a scale-out approach and replicate data by using additional machines.

In this paper we present Hyrise-R, a lazy master replication system for the in-memory database Hyrise. By setting up a snapshot-based Hyrise cluster, we increase both performance by distributing queries over multiple instances and availability by utilizing the redundancy of the cluster structure. This paper describes the architecture of Hyrise-R and details of the implemented replication mechanisms. We set up Hyrise-R on instances of Amazon's Elastic Compute Cloud and present a detailed performance evaluation of our system, including a linear query throughput increase for enterprise workloads.

References

[1]

Amazon Web Services, Inc. Amazon elastic compute cloud - user guide for linux (api version 2014-10-01), Feb. 2015.

[2]

J. Benzi and M. Damodaran. Parallel three dimensional direct simulation monte carlo for simulating micro flows. In Parallel Computational Fluid Dynamics 2007, pages 91--98. Springer, 2009.

[3]

Y. Breitbart, R. Komondoor, R. Rastogi, S. Seshadri, and A. Silberschatz. Update propagation protocols for replicated databates. In Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, SIGMOD '99, pages 97--108, New York, NY, USA, 1999. ACM.

Digital Library

[4]

E. Cecchet, G. Candea, and A. Ailamaki. Middleware-based database replication: the gaps between theory and practice. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 739--752. ACM, 2008.

Digital Library

[5]

W. Chen, S. Toueg, and M. K. Aguilera. On the quality of service of failure detectors. Computers, IEEE Transactions on, 51(5):561--580, 2002.

Digital Library

[6]

D. DeWitt and J. Gray. Parallel database systems: The future of high performance database systems. Commun. ACM, 35(6):85--98, June 1992.

Digital Library

[7]

W. Emmerich. Software engineering and middleware: a roadmap. In Proceedings of the Conference on The future of Software engineering, pages 117--129. ACM, 2000.

Digital Library

[8]

M. G. Gouda and T. M. McGuire. Accelerated heartbeat protocols. In Distributed Computing Systems, 1998. Proceedings. 18th International Conference on, pages 202--209. IEEE, 1998.

Digital Library

[9]

J. Gray, P. Helland, P. O'Neil, and D. Shasha. The dangers of replication and a solution. In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data, SIGMOD '96, pages 173--182, New York, NY, USA, 1996. ACM.

Digital Library

[10]

M. Grund, P. Cudre-Mauroux, J. Krüger, S. Madden, and H. Plattner. An overview of hyrise - a main memory hybrid storage engine. IEEE Data Engineering Bulletin, 2012.

[11]

M. Grund, J. Krüger, H. Plattner, A. Zeier, P. Cudre-Mauroux, and S. Madden. Hyrise: A main memory hybrid storage engine. Proc. VLDB Endow., 4(2):105--116, Nov. 2010.

Digital Library

[12]

N. Hayashibara, X. Defago, R. Yared, and T. Katayama. The ϕ accrual failure detector. In Reliable Distributed Systems, 2004. Proceedings of the 23rd IEEE International Symposium on, pages 66--78. IEEE, 2004.

Digital Library

[13]

T. Hoefler, C. Siebert, and W. Rehm. A practically constant-time MPI Broadcast Algorithm for large-scale InfiniBand Clusters with Multicast. page 232, 03 2007.

[14]

B. Kemme and G. Alonso. Don't be lazy, be consistent: Postgres-r, a new way to implement database replication. In Proceedings of the 26th International Conference on Very Large Data Bases, VLDB '00, pages 134--143, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc.

Digital Library

[15]

A. Kemper and T. Neumann. Hyper: A hybrid oltp&olap main memory database system based on virtual memory snapshots. In Data Engineering (ICDE), 2011 IEEE 27th International Conference on, pages 195--206. IEEE, 2011.

Digital Library

[16]

J. Krueger, C. Kim, M. Grund, N. Satish, D. Schwalb, J. Chhugani, H. Plattner, P. Dubey, and A. Zeier. Fast updates on read-optimized databases using multi-core cpus. Proc. VLDB Endow., 5(1):61--72, Sept. 2011.

Digital Library

[17]

M. M. Michael, J. E. Moreira, D. Shiloach, and R. W. Wisniewski. Scale-up x scale-out: A case study using nutch/lucene. In IPDPS, pages 1--8. IEEE, 2007.

[18]

T. Mühlbauer, W. Rödiger, A. Reiser, A. Kemper, and T. Neumann. Scyper: Elastic olap throughput on transactional data. In Proceedings of the Second Workshop on Data Analytics in the Cloud, DanaC '13, pages 11--15, New York, NY, USA, 2013. ACM.

Digital Library

[19]

S. Müller and H. Plattner. An in-depth analysis of data aggregation cost factors in a columnar in-memory database. In ACM Fifteenth International Workshop On Data Warehousing and OLAP colocated with ACM CIKM, Maui (HI), USA, 2012.

Digital Library

[20]

H. Plattner. A common database approach for oltp and olap using an in-memory column database. SIGMOD, 2009.

Digital Library

[21]

H. Plattner. The impact of columnar in-memory databases on enterprise systems: Implications of eliminating transaction-maintained aggregates. Proc. VLDB Endow., 7(13):1722--1729, Aug. 2014.

Digital Library

[22]

H. Plattner. Efficient transaction processing for hyrise in mixed workload environments. In In Memory Data Management and Analysis: First and Second International Workshops, IMDM 2013, Riva del Garda, Italy, August 26, 2013, IMDM 2014, Hongzhou, China, September 1, 2014, Revised Selected Papers, volume 8921, page 112. Springer, 2015.

[23]

A. Thomson, T. Diamond, S.-C. Weng, K. Ren, P. Shao, and D. J. Abadi. Calvin: Fast distributed transactions for partitioned database systems. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD '12, pages 1--12, New York, NY, USA, 2012. ACM.

Digital Library

[24]

C. Tinnefeld, S. Müller, H. Kaltegärtner, S. Hillig, L. Butzmann, D. Eickhoff, S. Klauck, D. Taschik, B. Wagner, O. Xylander, A. Zeier, H. Plattner, and C. Tosun. Available-to-promise on an in-memory column store. In T. Härder, W. Lehner, B. Mitschang, H. Schöning, and H. Schwarz, editors, BTW, volume 180 of LNI, pages 667--686. GI, 2011.

[25]

J. Wust, M. Grund, and H. Plattner. Tamex: A task-based query execution framework for mixed enterprise workloads on in-memory databases. In IMDM, INFORMATIK 2013, 2013.

Cited By

Hiller JKimmerlin MPlauth MHeikkila SKlauck SLindfors VEberhardt FBursztynowski DSantos JHohlfeld OWehrle K(2018)Giving Customers Control Over Their Data: Integrating a Policy Language into the Cloud2018 IEEE International Conference on Cloud Engineering (IC2E)10.1109/IC2E.2018.00050(241-249)Online publication date: Apr-2018
https://doi.org/10.1109/IC2E.2018.00050
Hayakawa YEggert LHonda MSantry D(2017)PrismProceedings of the 2017 Symposium on Cloud Computing10.1145/3127479.3127480(181-188)Online publication date: 24-Sep-2017
https://dl.acm.org/doi/10.1145/3127479.3127480
Kimmerlin MPlauth MHeikkila SNiemi T(2017)A practical evaluation of a network expansion mechanism in an openstack cloud federation2017 IEEE 6th International Conference on Cloud Networking (CloudNet)10.1109/CloudNet.2017.8071540(1-6)Online publication date: Sep-2017
https://doi.org/10.1109/CloudNet.2017.8071540
Show More Cited By

Recommendations

Borealis-R: a replication-transparent stream processing system for wide-area monitoring applications
SIGMOD '08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data

Borealis-R is a replication-based system for both fast and

highly-available processing of data streams over wide-area networks. In Borealis-R, multiple operator replicas send outputs to downstream replicas, allowing each replica to use whichever data ...
Postgres-R(SI): Combining Replica Control with Concurrency Control Based on Snapshot Isolation
ICDE '05: Proceedings of the 21st International Conference on Data Engineering

Replicating data over a cluster of workstations is a powerful tool to increase performance, and provide fault-tolerance for demanding database applications. The big challenge in such systems is to combine replica control (keeping the copies consistent) ...
MIDDLE-R: Consistent database replication at the middleware level

The widespread use of clusters and Web farms has increased the importance of data replication. In this article, we show how to implement consistent and scalable data replication at the middleware level. We do this by combining transactional concurrency ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

IMDM '15: Proceedings of the 3rd VLDB Workshop on In-Memory Data Mangement and Analytics

August 2015

63 pages

ISBN:9781450337137

DOI:10.1145/2803140

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SAMSUNG: SAMSUNG
VLDB Endowment: Very Large Database Endowment
Microsoft: Microsoft

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 August 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

IMDM '15

IMDM '15: VLDB Workshop on In-Memory Data Mangement and Analytics

August 31, 2015

HI, Kohala Coast, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
146
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hiller JKimmerlin MPlauth MHeikkila SKlauck SLindfors VEberhardt FBursztynowski DSantos JHohlfeld OWehrle K(2018)Giving Customers Control Over Their Data: Integrating a Policy Language into the Cloud2018 IEEE International Conference on Cloud Engineering (IC2E)10.1109/IC2E.2018.00050(241-249)Online publication date: Apr-2018
https://doi.org/10.1109/IC2E.2018.00050
Hayakawa YEggert LHonda MSantry D(2017)PrismProceedings of the 2017 Symposium on Cloud Computing10.1145/3127479.3127480(181-188)Online publication date: 24-Sep-2017
https://dl.acm.org/doi/10.1145/3127479.3127480
Kimmerlin MPlauth MHeikkila SNiemi T(2017)A practical evaluation of a network expansion mechanism in an openstack cloud federation2017 IEEE 6th International Conference on Cloud Networking (CloudNet)10.1109/CloudNet.2017.8071540(1-6)Online publication date: Sep-2017
https://doi.org/10.1109/CloudNet.2017.8071540
Wang JDuan HMin GYing GZheng S(2016)Goldfish: In-Memory Massive Parallel Processing SQL Engine Based on Columnar Store2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData)10.1109/iThings-GreenCom-CPSCom-SmartData.2016.49(142-149)Online publication date: Dec-2016
https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData.2016.49

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents