The HathiTrust Digital Library’s potential for musicology research

The HathiTrust Digital Library (HTDL) is one of the largest digital libraries in the world, containing seventeen million volumes from the collections of major academic and research libraries. In this paper, we discuss the HTDL’s potential for musicology research by providing a bibliometric analysis of the collection as a whole, and of the music materials in particular. A series of case studies illustrates the kinds of musicological research that may be conducted using the HTDL. We highlight several opportunities for improvement and discuss promising future directions for new knowledge creation through the processing and analysis of large amounts of retrospective data. The HTDL presents significant new opportunities to the study of music that will continue to expand as data, metadata and collection enhancements are introduced.

1. For example, the Bach digital database portal, designed to provide Bach researchers with “solid information on the works of Johann Sebastian Bach and other composers from the Bach family and their whereabouts,” http://www.bach-digital.de/content/infos.xml.

2. For example, Alexander Street’s Classical Scores Library, http://alexanderstreet.com/products/classical-scores-library-package.

3. For example, a virtual library of nineteenth-century California sheet music (from between the years 1852 and 1900), http://people.ischool.berkeley.edu/~mkduggan/neh.html.

4. See http://hathitrust.org/about.

5. See https://www.hathitrust.org/technology.

6. See https://www.hathitrust.org/trac.

7. See “What are the different copyright statuses of items in HathiTrust, and what do they mean?” https://www.hathitrust.org/help_copyright#RightsCodes.

8. See http://creativecommons.org/publicdomain/zero/1.0/.

9. We first converted the records we obtained from the HathiTrust into MARCXML (see: http://www.loc.gov/standards/marcxml/) using a Perl module (see: http://search.cpan.org/~gmcharlt/MARC-File-MiJ/lib/MARC/File/MiJ.pm). Then we used a Library of Congress XSLT stylesheet to convert the records to the Metadata Object Description Schema (MODS) format. The stylesheet was enhanced locally by consolidating information encoded in multiple MARC data and control fields, to reduce data loss and retain more detail about the conceptual characterizations of the items. Finally, a locally developed XSLT style sheet was used to transform the records into Structured Query Language (SQL) insert statements for populating the customized MODS database tables.

10. Of the 14.6 million item records that we examined, 6,672,311 (46%) had Library of Congress Classification numbers and 2,461,361 (16.8%) had Dewey Decimal Classification numbers. 2,165,140 had both, leaving only 296,221 (2% of the total items) that had a Dewey Decimal number but not a Library of Congress classification number. Of the volumes with a recorded classification authority (approximately 50% of the total records), only a very small number had any authority other than Library of Congress or Dewey Decimal. Classification authorities were not ascertainable for the remaining volumes, primarily because the HTDL does not retain local call numbers that can help in determining classification information. Although not retained, the local call number for a volume can, however, be retrieved if needed—via the holding record for the volume, using the contributing library’s local system number, which is stored by the HTDL.

11. An important source of inconsistency is the significant variation that is found in the format of the date field across records, originating from variations in the local standards used by contributing institutions for their own bibliographic records. For example, some bibliographic records use wildcard characters in the date field, which are not consistent with each other as, sometimes, different wildcard characters have been used. Ranges of years often appear, and their formats, too, are frequently different. (For example, “1904–1924,” “between 1920 and 1950,” etc.). Other sources of variation include the fact that some records use the character ‘u’, for ‘unknown’, in place of a digit—as in ‘18uu’ to denote an year which is not precisely known but is from the nineteenth century—other records may use a dash (for example, ‘198–’).

12. See https://www.hathitrust.org/visualizations_dates for up-to-date chronological information.

13. See: http://www.loc.gov/standards/mods/v3/mods2marc-mapping.html#genre.

14. The footprints of languages in the overall HTDL collection are shown in Fig. 3.

15. Consistent with the relative decline of German as a language of international scholarship starting in the 1920s, the proportion of German-language material in the music-class in the HTDL collection declines from about 61% for texts that are considered to be in public domain for researchers in the USA (which are mostly pre-1923 publications) to about 30% for texts that are in-copyright (which are mostly post-1923).

16. Each distinct Library of Congress Subject Heading in the HathiTrust metadata explored in this paper is counted separately, so that “Piano Music,” “Vocal Music,” etc. are categories distinct from the category “Music.”

17. Tune books constituted a genre of early American music publication that had a pedagogical aim, and they usually contained an instructional preface. That so few tune books are classified as such by subject in the HTDL is probably due to their having often been cataloged by libraries as ‘hymns.’

18. ‘Music’ and ‘Musical Scores’ are overlapping categories for a small number items in the HTDL, which is an artifact of how these metadata are generated from the MARC bibliographic records.

19. Important early histories and reference works related to music and contained in the HTDL collection include Jean-Jacques Rousseau’s Dictionnaire de Musique (1768), Sir John Hawkins’s General History of the Science and Practice of Music (1776), and Charles Burney’s A General History of Music: From the Earliest Ages to the Present Period (1789).

20. This was the focus of a collaborative project carried out under the auspices of the HTRC in 2015; details can be found at: https://www.hathitrust.org/htrc_acs_awards_spring2015.

21. For simple examples of how comparison and contrast between two corpora created from the HTDL collection can be performed by using the algorithmic tools provided by the HathiTrust Research Center, see: ‘Workset Builder and Portal of the HathiTrust Research Center’. HathiTrust Research Center UnCamp. Ann Arbor, Michigan. 30–31 March 2015, http://bit.ly/1NF7QLi.

22. Sag [28] notes: ‘The HathiTrust aims to develop and facilitate the development of data mining and analysis of its digital collection. This activity would have qualified as “non-consumptive research” under the now defunct Amended Settlement Agreement [ASA]. “Non-consumptive research” as defined in the ASA is a form of non-expressive use...’

23. See: https://htrcbookworm.wordpress.com/.

We gratefully acknowledge the contributions of Kirstin (Dougan) Johnson and Colleen Fallaw, who were co-authors of a preliminary version of this paper, which appeared in the Proceedings of the 1st International Workshop on Digital Libraries for Musicology. We wish to thank Janina Sarol for her help in generating the data for this paper. Xiao Hu provided valuable suggestions and critique. Mike Furlough of the HathiTrust, as well as Tim Cole of the HathiTrust Research Center, helpfully answered our queries. We are grateful to the anonymous reviewers who provided detailed and generous commentary. This work was made possible with generous support from the HathiTrust Foundation.

Authors and Affiliations

1. University of Illinois at Urbana-Champaign, Urbana-Champaign, USA

   J. Stephen Downie

2. Singapore University of Technology and Design, Singapore, 487372, Singapore

   Sayan Bhattacharyya

3. Rutgers, The State University of New Jersey, New Brunswick, USA

   Francesca Giannetti

4. University of Michigan, Ann Arbor, USA

   Eleanor Dickson Koehl

5. University of Denver, Denver, USA

   Peter Organisciak

Corresponding author

Correspondence to Francesca Giannetti.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Reprints and permissions