More Web Proxy on the site http://driver.im/

short-paper

Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis

Authors:

Heiko SchuldtAuthors Info & Claims

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Pages 4465 - 4468

https://doi.org/10.1145/3394171.3414538

Published: 12 October 2020 Publication History

Abstract

Multimedia retrieval and analysis are two important areas in "Big data" research. They have in common that they work with feature vectors as proxies for the media objects themselves. Together with metadata such as textual descriptions or numbers, these vectors describe a media object in its entirety, and must therefore be considered jointly for both storage and retrieval.

In this paper we introduce Cottontail DB, an open source database management system that integrates support for scalar and vector attributes in a unified data and query model that allows for both Boolean retrieval and nearest neighbour search. We demonstrate that Cottontail DB scales well to large collection sizes and vector dimensions and provide insights into how it proved to be a valuable tool in various use cases ranging from the analysis of MRI data to realizing retrieval solutions in the cultural heritage domain.

Supplementary Material

MP4 File (3394171.3414538.mp4)

This is the presentation video for our paper "Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis" at ACM MM 2020, in which we give an overview of the system as well as a brief demo as to how the system can be used.

Download
269.99 MB

References

[1]

Fabian Berns, Luca Rossetto, Klaus Schoeffmann, Christian Beecks, and George Awad. 2019. V3C1 Dataset: An Evaluation of Content Characteristics. In Proceedings of the 2019 ACM International Conference on Multimedia Retrieval (Ottawa ON, Canada) (ICMR '19). Association for Computing Machinery, New York, NY, USA, 334--338. https://doi.org/10.1145/3323873.3325051

Digital Library

[2]

Michael J Carey, Laura M Haas, Peter M Schwarz, Manish Arya, William F Cody, Ronald Fagin, Myron Flickner, Allen W Luniewski, Wayne Niblack, Dragutin Petkovic, et al. 1995. Towards heterogeneous multimedia information systems: The Garlic approach. In Proceedings RIDE-DOM'95. Fifth International Workshop on Research Issues in Data Engineering-Distributed Object Management. IEEE, Taipei, Taiwan, 124--131.

[3]

Arjen P de Vries and HM Blanken. 1998. Database technology and the management of multimedia data in the Mirror project. In Multimedia Storage and Archiving Systems III, Vol. 3527. International Society for Optics and Photonics, Boston, MA, USA, 443--453.

[4]

Ralph Gasser, Luca Rossetto, and Heiko Schuldt. 2019. Multimodal multimedia retrieval with Vitrivr. In Proceedings of the 2019 ACM International Conference on Multimedia Retrieval. ACM, Ottawa, ON, Canada, 391--394.

Digital Library

[5]

Ivan Giangreco and Heiko Schuldt. 2016. ADAMpro: Database support for big multimedia retrieval. Datenbank-Spektrum, Vol. 16, 1 (2016), 17--26.

[6]

Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Andreas Leibetseder, Liting Zhou, Aaron Duane, Duc-Tien Dang-Nguyen, Michael Riegler, Luca Piras, Minh-Triet Tran, et al. 2019. [Invited papers] Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018). ITE Transactions on Media Technology and Applications, Vol. 7, 2 (2019), 46--59.

[7]

Rui Hu, Stefan Ruger, Dawei Song, Haiming Liu, and Zi Huang. 2008. Dissimilarity measures for content-based image retrieval. In IEEE International Conference on Multimedia and Expo. IEEE, Hannover, Germany, 1365--1368.

[8]

Piotr Indyk and Rajeev Motwani. 1998. Approximate nearest neighbors: towards removing the curse of dimensionality. In Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing. ACM, Dallas, TX, USA, 604--613.

Digital Library

[9]

Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, 1 (2010), 117--128.

Digital Library

[10]

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with GPUs. CoRR, Vol. abs/1702.08734 (2017).

[11]

Mathias Lux. 2011. Content Based Image Retrieval with LIRe. In Proceedings of the 19th ACM International Conference on Multimedia (Scottsdale, Arizona, USA) (MM '11). Association for Computing Machinery, New York, NY, USA, 735--738. https://doi.org/10.1145/2072298.2072432

Digital Library

[12]

Dan Ma, Vikas Gulani, Nicole Seiberlich, Kecheng Liu, Jeffrey L Sunshine, Jeffrey L Duerk, and Mark A Griswold. 2013. Magnetic resonance fingerprinting. Nature, Vol. 495, 7440 (2013), 187--192.

[13]

Luca Rossetto, Ralph Gasser, Silvan Heller, Mahnaz Amiri Parian, and Heiko Schuldt. 2019. Retrieval of structured and unstructured data with vitrivr. In Proceedings of the ACM Workshop on Lifelog Search Challenge. ACM, Ottawa, ON, Canada, 27--31.

Digital Library

[14]

Luca Rossetto, Ralph Gasser, Jakub Lokoc, Werner Bailer, Klaus Schoeffmann, Bernd Muenzer, Tomas Soucek, Phuong Anh Nguyen, Paolo Bolettieri, Andreas Leibetseder, et al. 2020. Interactive Video Retrieval in the Age of Deep Learning-Detailed Evaluation of VBS 2019. IEEE Transactions on Multimedia (2020).

[15]

Luca Rossetto, Ivan Giangreco, and Heiko Schuldt. 2014. Cineast: a multi-feature sketch-based video retrieval engine. In 2014 IEEE International Symposium on Multimedia. IEEE, Taichung, Taiwan, 18--23.

Digital Library

[16]

Luca Rossetto, Ivan Giangreco, Claudiu Tanase, and Heiko Schuldt. 2016. vitrivr: A flexible retrieval stack supporting multiple query modes for searching in multimedia collections. In Proceedings of the 24th ACM international Conference on Multimedia. ACM, Amsterdam, The Netherlands, 1183--1186.

Digital Library

[17]

Loris Sauter, Mahnaz Amiri Parian, Ralph Gasser, Silvan Heller, Luca Rossetto, and Heiko Schuldt. 2020. Combining Boolean and Multimedia Retrieval in vitrivr for Large-Scale Video Search. In International Conference on Multimedia Modeling. Springer, 760--765.

Digital Library

[18]

Hans-Jörg Schek. 1980. Methods for the administration of textual data in database systems. In Proceedings of the 3rd annual ACM Conference on Research and Development in Information Retrieval. Butterworth & Co., Cambridge, UK, 218--235.

[19]

Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2016. YFCC100M: The New Data in Multimedia Research. Commun. ACM, Vol. 59, 2 (Jan. 2016), 64--73. https://doi.org/10.1145/2812802

Digital Library

[20]

Marco Vogt, Alexander Stiemer, and Heiko Schuldt. 2018. Polypheny-DB: Towards a Distributed and Self-Adaptive Polystore. In 2018 IEEE International Conference on Big Data (Big Data). IEEE, Seattle, WA, USA, 3364--3373.

[21]

Roger Weber, Hans-Jörg Schek, and Stephen Blott. 1998. A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In VLDB, Vol. 98. Morgan Kaufmann, New York City, NY, USA, 194--205.

Cited By

Redkina N(2024)Impact of Open Science Infrastructure on the Development of the World Information Resources MarketScientific and Technical Information Processing10.3103/S014768822470009651:2(161-172)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.3103/S0147688224700096
Sauter LGasser RRettig LSchuldt HRossetto L(2024)General Purpose Multimedia Retrieval with vitrivr at LSC'24Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661120(47-52)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661120
Spiess FSchuldt HRossetto L(2024)Spatiotemporal Lifelog Analytics in Virtual Reality with vitrivr-VRProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661113(7-11)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661113
Show More Cited By

Index Terms

Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis
1. Information systems

Recommendations

Multimedia Retrieval and Analysis with Cottontail DB

Analysis and retrieval of media collections get more and more challenging the larger the collections become. Keeping everything in the main memory becomes less feasible, and more and more time and effort have to be spent to deal with the data ...
Multimodal Multimedia Retrieval with vitrivr
ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

The steady growth of multimedia collections - both in terms of size and heterogeneity - necessitates systems that are able to conjointly deal with several types of media as well as large volumes of data. This is especially true when it comes to ...
Content-based multimedia information retrieval: State of the art and challenges

Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

October 2020

4889 pages

ISBN:9781450379885

DOI:10.1145/3394171

General Chairs:
Chang Wen Chen
Chinese University of Hong Kong, Shenzhen, China
,
Rita Cucchiara
UNIMORE, Italy
,
Xian-Sheng Hua
Alibaba Group, China
,
Program Chairs:
Guo-Jun Qi
Futurewei Technologies, USA
,
Elisa Ricci
UNITN & Fondazione Bruno Kessler, Italy
,
Zhengyou Zhang
Tencent, China
,
Roger Zimmermann
National University of Singapore, Singapore

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

MM '20

Sponsor:

SIGMM

MM '20: The 28th ACM International Conference on Multimedia

October 12 - 16, 2020

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
332
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)9

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Redkina N(2024)Impact of Open Science Infrastructure on the Development of the World Information Resources MarketScientific and Technical Information Processing10.3103/S014768822470009651:2(161-172)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.3103/S0147688224700096
Sauter LGasser RRettig LSchuldt HRossetto L(2024)General Purpose Multimedia Retrieval with vitrivr at LSC'24Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661120(47-52)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661120
Spiess FSchuldt HRossetto L(2024)Spatiotemporal Lifelog Analytics in Virtual Reality with vitrivr-VRProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661113(7-11)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661113
Arnold RSchuldt H(2024)Multimedia Retrieval in Mixed Reality: Leveraging Live Queries for Immersive Experiences2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)10.1109/AIxVR59861.2024.00048(289-293)Online publication date: 17-Jan-2024
https://doi.org/10.1109/AIxVR59861.2024.00048
Taipalus T(2024)Vector database management systemsCognitive Systems Research10.1016/j.cogsys.2024.10121685:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.cogsys.2024.101216
Parian-Scherb MUhrig PRossetto LDupont SSchuldt H(2024)Gesture retrieval and its application to the study of multimodal communicationInternational Journal on Digital Libraries10.1007/s00799-023-00367-025:4(585-601)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s00799-023-00367-0
Gasser RArnold RFaber FSchuldt HWaltenspül RRossetto L(2024)A New Retrieval Engine for VitrivrMultiMedia Modeling10.1007/978-3-031-53302-0_28(324-331)Online publication date: 29-Jan-2024
https://doi.org/10.1007/978-3-031-53302-0_28
Spiess FRossetto LSchuldt H(2024)Exploring Multimedia Vector Spaces with vitrivr-VRMultiMedia Modeling10.1007/978-3-031-53302-0_27(317-323)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1007/978-3-031-53302-0_27
Sauter LSchuldt HWaltenspül RRossetto L(2023)Novice-Friendly Text-based Video Search with vitrivrProceedings of the 20th International Conference on Content-based Multimedia Indexing10.1145/3617233.3617262(163-167)Online publication date: 20-Sep-2023
https://dl.acm.org/doi/10.1145/3617233.3617262
Spiess FGasser RSchuldt HRossetto L(2023)The Best of Both Worlds: Lifelog Retrieval with a Desktop-Virtual Reality Hybrid SystemProceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593107(65-68)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593107
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents