Why and how to benchmark XML databases

A Schmidt, F Waas, M Kersten, D Florescu… - ACM Sigmod …, 2001 - dl.acm.org
A Schmidt, F Waas, M Kersten, D Florescu, MJ Carey, I Manolescu, R Busse
ACM Sigmod Record, 2001dl.acm.org
Benchmarks belong to the very standard repertory of tools deployed in database
development. Assessing the capabilities of a system, analyzing actual and potential
bottlenecks, and, naturally, comparing the pros and cons of different systems architectures
have become indispensable tasks as databases management systems grow in complexity
and capacity. In the course of the development of XML databases the need for a benchmark
framework has become more and more evident: a great many different ways to store XML …
Benchmarks belong to the very standard repertory of tools deployed in database development. Assessing the capabilities of a system, analyzing actual and potential bottlenecks, and, naturally, comparing the pros and cons of different systems architectures have become indispensable tasks as databases management systems grow in complexity and capacity. In the course of the development of XML databases the need for a benchmark framework has become more and more evident: a great many different ways to store XML data have been suggested in the past, each with its genuine advantages, disadvantages and consequences that propagate through the layers of a complex database system and need to be carefully considered. The different storage schemes render the query characteristics of the data variably different. However, no conclusive methodology for assessing these differences is available to date.In this paper, we outline desiderata for a benchmark for XML databases drawing from our own experience of developing an XML repository, involvement in the definition of the standard query language, and experience with standard benchmarks for relational databases.
ACM Digital Library