More Web Proxy on the site http://driver.im/

research-article

DBMS metrology: measuring query time

Authors:

Richard T. Snodgrass,

Young-Kyoon Suh,

Matthew Wong Johnson,

Cheng YiAuthors Info & Claims

SIGMOD '13: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

Pages 421 - 432

https://doi.org/10.1145/2463676.2465331

Published: 22 June 2013 Publication History

Abstract

It is surprisingly hard to obtain accurate and precise measurements of the time spent executing a query. We review relevant process and overall measures obtainable from the Linux kernel and introduce a structural causal model relating these measures. A thorough correlational analysis provides strong support for this model. Using this model, we developed a timing protocol, which (1) performs sanity checks to ensure validity of the data, (2) drops some query executions via clearly motivated predicates, (3) drops some entire queries at a cardinality, again via clearly motivated predicates, (4) for those that remain, for each computes a single measured time by a carefully justified formula over the underlying measures of the remaining query executions, and (5) performs post-analysis sanity checks. The resulting query time measurement procedure, termed the Tucson Protocol, applies to proprietary and open-source DBMSes.

References

[1]

A. G. Ailamaki, D. J. DeWitt, M. D. Hill, and D. A. Wood, "DBMSes on modern processors: Where does time go?" University ofWisconsin Computer Sciences Technical Report 1394, February 25, 1999.

[2]

M. Akswew, U. Cetintemel, M. Riondato, E. Upfal, and S. B. Zdonik, "Learning-based Query Performance Modeling and Prediction," in Proceedings of the IEEE International Concerence on Data Engineering, pp. 390--411, 2012.

Digital Library

[3]

D. Bovet, and M. Cesati, "Understanding the Linux Kernel, Third Edition," O'Reilly, 2003.

Digital Library

[4]

B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears, "Benchmarking Cloud Serving Systems with YCSB," in Proceedings of the ACM Symposium on Cloud Computing, New York, June 2010, pp. 143--154.

Digital Library

[5]

R. F. Forman, J. M. Pechacek, and W. H. Schwane, "Apparatus and Methodfor Measure Transaction Time in a Computer System," IBM Patent, January 2001.

[6]

A. Ganapathi, H. A. Kuno, U. Dayal, J. L. Wiener, A. Fox, M. I. Jordan, and D. A. Patterson, "Predicting multiple performance metrics for queries: Better decisions enabledby machine learning," in Proceedings of IEEE International Conference on Data Engineering, 2009.

Digital Library

[7]

L. Gikoumakis and C. Galindo-Legaria, "Testing SQL Server's Query Optimizer: Challenges, Techniques andExperiences, IEEE Data Eng. Bull. 31(1): 36--43 (2008).

[8]

H.-Y. Hwang and Y.-T. Yu, "An Analytical Method for Estimating and Interpreting Query Time," in Proceedings of VLDB International Conference, Brighton, 1978, pp. 347--358.

Digital Library

[9]

Working Group 2 of the Joint Committee for Guides in Metrology (JCGM/WG 2), International Vocabulary of Metrology--Basic and General Concepts and Associated Terms, 2008.

[10]

K. Kant, Introduction to Computer System Performance Evaluation, McGraw-Hill, 1992.

[11]

M. Stillger, G. M. Lohman, V. Markl, and M. Kaqndil, "LEO - DB2's LearningOptimizer," in Proceedings of the VLDB International Conference, 2001.

Digital Library

[12]

TPC, "Transaction Processing Performance Council-TPC-C," http://www.tpc.org/tpcc/. (accessed August 29, 2010)

[13]

TPC, "TPC Transaction Processing Performance Council-TPC-H," http://www.tpc.org/tpch/. (accessed August 29, 2010)

[14]

Valgrind Developers, "Callgrind: A Call-Graph Generating Cache and BranchPrediction Profiler," http://valgrind.org/docs/manual/cl-manual.html.(accessed October 27, 2010)

[15]

B. B. Yao, M. T. Özsu, and N. Khandelwal, N., "XBench benchmark and performance testing of XML DBMSs," in Proceedings of the International Conference on Data Engineering, pp. 621--632, 2004.

Digital Library

[16]

P. Yu, M.-S. Chen, H.-U. Heiss, and S. Lee, "On workload characterization of relational database environments," IEEE Transactions on Software Engineering, 18:347--355, 1992.

Digital Library

[17]

N. Zhang, J. Tatemura, J. M. Patel, and H. Hacõgumus, "Towards Cost-Effective Storage Provisioning for DBMSs," Proceedings of the VLDB Endowment (PVLDB), Vol. 5,No. 4, pp. 274--285 (2011)

Digital Library

[18]

R. Zhang, R. T. Snodgrass and S. Debray, "Micro-Specialization in DBMSes," in Proceedings of the IEEE International Conference on Data Engineering, pp. 690--701, April 2012.

Digital Library

Cited By

Currim SSnodgrass RSuh Y(2024)Identifying the Root Causes of DBMS SuboptimalityACM Transactions on Database Systems10.1145/363642549:1(1-40)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3636425
Suh YAn JTak BNa G(2022)A Comprehensive Empirical Study of Query Performance Across GPU DBMSesProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/35080246:1(1-29)Online publication date: 28-Feb-2022
https://dl.acm.org/doi/10.1145/3508024
Suh YKim SKim J(2020)CLUTCH: A Clustering-Driven Runtime Estimation Scheme for Scientific SimulationsIEEE Access10.1109/ACCESS.2020.30425968(220710-220722)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3042596
Show More Cited By

Index Terms

DBMS metrology: measuring query time
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database query processing
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

DBMS Metrology: Measuring Query Time
Invited Paper from ICDT 2014, Invited Paper from EDBT 2015, Regular Papers and Technical Correspondence

It is surprisingly hard to obtain accurate and precise measurements of the time spent executing a query because there are many sources of variance. To understand these sources, we review relevant per-process and overall measures obtainable from the ...
Investigation on Three Dimensional Image Measurement for Geometrical Parameters of the Cutter
GCIS '09: Proceedings of the 2009 WRI Global Congress on Intelligent Systems - Volume 03

In this investigation, image measurement technology is applied for development of an image measurement system for geometrical parameters of the cutter. CCD, LED light source, servo motor and x-y bi-axial stage have been integrated in this system. It can ...
Measurement of End-effector Pose Errors and the Cable Profile of Cable-Driven Robot using Monocular Camera
Abstract
Measurement of performance metrics namely repeatability and directional accuracy of an under-actuated cable-driven robot using a monocular camera is proposed. Experiments were conducted on a cable-driven robot used for construction and the results ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '13: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

June 2013

1322 pages

ISBN:9781450320375

DOI:10.1145/2463676

General Chairs:
Kenneth Ross
Columbia University
,
Divesh Srivastava
AT&T Research
,
Program Chair:
Dimitris Papadias
HKUST

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 June 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGMOD/PODS'13

Sponsor:

SIGMOD

SIGMOD/PODS'13: International Conference on Management of Data

June 22 - 27, 2013

New York, New York, USA

Acceptance Rates

SIGMOD '13 Paper Acceptance Rate 76 of 372 submissions, 20%;

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
385
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Currim SSnodgrass RSuh Y(2024)Identifying the Root Causes of DBMS SuboptimalityACM Transactions on Database Systems10.1145/363642549:1(1-40)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3636425
Suh YAn JTak BNa G(2022)A Comprehensive Empirical Study of Query Performance Across GPU DBMSesProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/35080246:1(1-29)Online publication date: 28-Feb-2022
https://dl.acm.org/doi/10.1145/3508024
Suh YKim SKim J(2020)CLUTCH: A Clustering-Driven Runtime Estimation Scheme for Scientific SimulationsIEEE Access10.1109/ACCESS.2020.30425968(220710-220722)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3042596
Islam MGovernatori G(2018)RuleRSArtificial Intelligence and Law10.1007/s10506-018-9218-026:4(315-344)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s10506-018-9218-0
Currim SSnodgrass RSuh YZhang R(2016)DBMS MetrologyACM Transactions on Database Systems10.1145/299645442:1(1-42)Online publication date: 9-Nov-2016
https://dl.acm.org/doi/10.1145/2996454
Suh YSnodgrass RZhang R(2014)AZDBLabProceedings of the VLDB Endowment10.14778/2733004.27330507:13(1641-1644)Online publication date: 1-Aug-2014
https://dl.acm.org/doi/10.14778/2733004.2733050

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten