research-article

Open access

Spanner: Becoming a SQL System

Authors:

Marcel van der Holst,

Dale WoodfordAuthors Info & Claims

SIGMOD '17: Proceedings of the 2017 ACM International Conference on Management of Data

Pages 331 - 343

https://doi.org/10.1145/3035918.3056103

Published: 09 May 2017 Publication History

PDF eReader

Abstract

Spanner is a globally-distributed data management system that backs hundreds of mission-critical services at Google. Spanner is built on ideas from both the systems and database communities. The first Spanner paper published at OSDI'12 focused on the systems aspects such as scalability, automatic sharding, fault tolerance, consistent replication, external consistency, and wide-area distribution. This paper highlights the database DNA of Spanner. We describe distributed query execution in the presence of resharding, query restarts upon transient failures, range extraction that drives query routing and index seeks, and the improved blockwise-columnar storage format. We touch upon migrating Spanner to the common SQL dialect shared with other systems at Google.

References

[1]

A. Ailamaki, D. J. DeWitt, and M. D. Hill.href10.1007/s00778-002-0074-9Data Page Layouts for Relational Databases on Deep Memory Hierarchies. The VLDB Journal, 11(3):198--215, Nov. 2002.

Digital Library

Google Scholar

[2]

J. Baker, C. Bond, J. C. Corbett, J. Furman, A. Khorlin, J. Larson, J.-M. Leon, Y. Li, A. Lloyd, and V. Yushprakh.hrefhttp://www.cidrdb.org/cidr2011/Paper/CIDR11_Paper32.pdfMegastore: Providing Scalable, Highly Available Storage for Interactive Services. In Proceedings of the Conference on Innovative Data system Research (CIDR), pp. 223--234, 2011.

Google Scholar

[3]

E. Brewer.hrefhttps://research.google.com/pubs/pub45855.htmlSpanner, TrueTime and the CAP Theorem. Technical Report, https://research.google.com/pubs/pub45855.html, 2017.

Google Scholar

[4]

F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber.hrefhttp://www.usenix.org/events/osdi06/tech/chang.htmlBigtable: A Distributed Storage System for Structured Data. In 7th Symposium on Operating Systems Design and Implementation (OSDI '06), November 6--8, Seattle, WA, USA, pp. 205--218, 2006.

Digital Library

Google Scholar

[5]

J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, W. Hsieh, S. Kanthak, E. Kogan, H. Li, A. Lloyd, S. Melnik, D. Mwaura, D. Nagle, S. Quinlan, R. Rao, L. Rolig, Y. Saito, M. Szymaniak, C. Taylor, R. Wang, and D. Woodford.hrefhttps://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbettSpanner: Google Globally-Distributed Database. In Tenth USENIX Symposium on Operating Systems Design and Implementation (OSDI 12), pp. 261--264, Hollywood, CA, Oct. 2012.

Digital Library

Google Scholar

[6]

L. Lamport. The part-time parliament. ACM Transactions on Computer Systems (TOCS), 16(2):133--169, 1998.

Digital Library

Google Scholar

[7]

S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shivakumar, M. Tolton, and T. Vassilakis.hrefhttp://cacm.acm.org/magazines/2011/6/108648-dremel-interactive-analysis-of-web-scale-datasets/fulltextDremel: Interactive Analysis of Web-Scale Datasets. Communications of the ACM, 54:114--123, 2011.

Digital Library

Google Scholar

[8]

P. O'Neil, E. Cheng, D. Gawlick, and E. O'Neil. The log-structured merge-tree (LSM-tree). Acta Informatica, 33(4):351--385, 1996.

Digital Library

Google Scholar

[9]

J. Shute, M. Oancea, S. Ellner, B. Handy, E. Rollins, B. Samwel, R. Vingralek, C. Whipkey, X. Chen, B. Jegerlehner, et al.hrefhttps://research.google.com/pubs/pub38125.htmlF1: The Fault-Tolerant Distributed RDBMS Supporting Google's Ad Business. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 777--778. ACM, 2012.

Digital Library

Google Scholar

[10]

M. Stonebraker and U. Cetintemel. "One size fits all": an idea whose time has come and gone. In 21st International Conference on Data Engineering (ICDE'05), pp. 2--11. IEEE, 2005.

Digital Library

Google Scholar

Cited By

View all

Zhang RYe ZCai PZhou XCai DQian L(2024)Hybrid Shared-Buffer for Multi-Master DatabasesJournal of Database Management10.4018/JDM.35692035:1(1-27)Online publication date: 7-Nov-2024
https://dl.acm.org/doi/10.4018/JDM.356920
Bashtovyi AFechan A(2024)Distributed Transactions in Microservice Architecture: Informed Decision-making StrategiesVìsnik Nacìonalʹnogo unìversitetu "Lʹvìvsʹka polìtehnìka". Serìâ Ìnformacìjnì sistemi ta merežì10.23939/sisn2024.15.44915(449-459)Online publication date: 15-Jul-2024
https://doi.org/10.23939/sisn2024.15.449
Xu QYang CZhou A(2024)Native Distributed Databases: Problems, Challenges and OpportunitiesProceedings of the VLDB Endowment10.14778/3685800.368583917:12(4217-4220)Online publication date: 8-Nov-2024
https://doi.org/10.14778/3685800.3685839
Show More Cited By

Index Terms

Spanner: Becoming a SQL System
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing
2. Information systems
  1. Data management systems

Recommendations

Efficient Processing of Semantic Web Queries in HBase and MySQL Cluster

With rapid growth of the Semantic Web, the authors' research focuses on designing a scalable resource description framework (RDF) database system that can efficiently process SPARQL queries over large RDF datasets. In this article, they present their ...
Comparing NoSQL MongoDB to an SQL DB
ACMSE '13: Proceedings of the 51st annual ACM Southeast Conference

NoSQL database solutions are becoming more and more prevalent in a world currently dominated by SQL relational databases. NoSQL databases were designed to provide database solutions for large volumes of data that is not structured. However, the ...
Incorporating NoSQL into a database course

This article introduces the concepts of Big Data and NoSQL and describes a semester long web-based project that uses both a relational database (Oracle 11g) and a NoSQL (MongoDB) database for an undergraduate database course. The relational database ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGMOD '17: Proceedings of the 2017 ACM International Conference on Management of Data

May 2017

1810 pages

ISBN:9781450341974

DOI:10.1145/3035918

General Chairs:
Rada Chirkova
North Carolina State University, USA
,
Jun Yang
Duke University, USA
,
Program Chair:
Dan Suciu
University of Washington, USA

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 May 2017

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS'17

Sponsor:

SIGMOD

SIGMOD/PODS'17: International Conference on Management of Data

May 14 - 19, 2017

Illinois, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
14,324
Total Downloads

Downloads (Last 12 months)894
Downloads (Last 6 weeks)75

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhang RYe ZCai PZhou XCai DQian L(2024)Hybrid Shared-Buffer for Multi-Master DatabasesJournal of Database Management10.4018/JDM.35692035:1(1-27)Online publication date: 7-Nov-2024
https://dl.acm.org/doi/10.4018/JDM.356920
Bashtovyi AFechan A(2024)Distributed Transactions in Microservice Architecture: Informed Decision-making StrategiesVìsnik Nacìonalʹnogo unìversitetu "Lʹvìvsʹka polìtehnìka". Serìâ Ìnformacìjnì sistemi ta merežì10.23939/sisn2024.15.44915(449-459)Online publication date: 15-Jul-2024
https://doi.org/10.23939/sisn2024.15.449
Xu QYang CZhou A(2024)Native Distributed Databases: Problems, Challenges and OpportunitiesProceedings of the VLDB Endowment10.14778/3685800.368583917:12(4217-4220)Online publication date: 8-Nov-2024
https://doi.org/10.14778/3685800.3685839
Shute JBales SBrown MBrowne JDolphin BKudtarkar RLitvinov AMa JMorcos JShen MWilhite DWu XYu L(2024)SQL Has Problems. We Can Fix Them: Pipe Syntax In SQLProceedings of the VLDB Endowment10.14778/3685800.368582617:12(4051-4063)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.14778/3685800.3685826
Mao YYin RLei LYe PZou STang SGuo YYuan YYu XWan BGong YGao CZhang GShen JShi RMa R(2024)ByteMQ: A Cloud-native Streaming Data Layer in ByteDanceProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698536(774-791)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698536
Liu JWang FMo DLuo S(2024)Structural Designs Meet Optimality: Exploring Optimized LSM-tree Structures in a Colossal Configuration SpaceProceedings of the ACM on Management of Data10.1145/36549782:3(1-26)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654978
Levandoski JCasto GDeng MDesai REdara PHottelier THormati AJohnson AJohnson JKurzyniec DMcVeety SRamanathan PSaxena GShanmugan VVolobuev YBarcelo PSanchez-Pi NMeliou ASudarshan S(2024)BigLake: BigQuery's Evolution toward a Multi-Cloud LakehouseCompanion of the 2024 International Conference on Management of Data10.1145/3626246.3653388(334-346)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3626246.3653388
Zambrano MGonzalez VSimmonds M(2024)The database quest for the Square Kilometre ArraySoftware and Cyberinfrastructure for Astronomy VIII10.1117/12.3019760(60)Online publication date: 25-Jul-2024
https://doi.org/10.1117/12.3019760
Song HZhou WCui HPeng XLi F(2024)A survey on hybrid transactional and analytical processingThe VLDB Journal10.1007/s00778-024-00858-933:5(1485-1515)Online publication date: 4-Jun-2024
https://doi.org/10.1007/s00778-024-00858-9
Xu QChen MYang YWang C(2024)Flutist: Parallelizing Transaction Processing for LSM-Tree-Based Relational DatabaseWeb and Big Data10.1007/978-981-97-7238-4_29(460-476)Online publication date: 28-Aug-2024
https://doi.org/10.1007/978-981-97-7238-4_29
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Efficient Processing of Semantic Web Queries in HBase and MySQL Cluster

Comparing NoSQL MongoDB to an SQL DB

Incorporating NoSQL into a database course