Methods of Multi-Modal Data Exploration
Pages 34 - 37
Abstract
Techniques and tools designed for information retrieval, data exploration or data analytical tasks are based on the relational and text-search model, and cannot be easily applied to unstructured data such as images or videos. Researcher communities have been trying to reveal the semantics of multimedia in the last decades with ever-improving results in various tasks, dominated by the latest success of deep learning. Limits of object retrieval models drive the need for data exploration methods that support multi-modal data, like multimedia surrounded by structured attributes. In this paper, we describe, implement and evaluate exploration methods using multiple modalities and retrieval models in the context of multimedia. We apply the techniques in e-commerce product search and recommending, and demonstrate benefit for different retrieval scenarios. Lastly, we propose a method for extending database schema by latent visual attributes learned from image data. This enables closing the loop by going back to relational data, and potentially benefiting a range of industrial applications.
References
[1]
Christian Beecks, Tomávs Skopal, Klaus Schoeffmann, and Thomas Seidl. 2011. Towards Large-Scale Multimedia Exploration. In Proceedings of the 5th International Workshop on Ranking in Databases (DBRank 2011), Gautam Das, Vagelis Hsristidis, and Ihab Ilyas (Eds.). VLDB, Seattle, WA, USA, 31--33.
[2]
Jürgen Bernard, Matthias Zeppelzauer, Markus Lehmann, Martin Müller, and Michael Sedlmair. 2018. Towards User-Centered Active Learning Algorithms. Comput. Graph. Forum, Vol. 37 (2018), 121--132.
[3]
P. vC ech and T. Grovs up. 2015. Comparison of metric space browsing strategies for efficient image exploration. In 2015 13th International Workshop on Content-Based Multimedia Indexing (CBMI). 1--6.
[4]
Tomávs Grovs up, Pvr emysl Cech, Jakub Lokovc, and Tomávs Skopal. 2015. A Web Portal for Effective Multi-model Exploration. In MultiMedia Modeling, Xiangjian He, Suhuai Luo, Dacheng Tao, Changsheng Xu, Jie Yang, and Muhammad Abul Hasan (Eds.). Springer International Publishing, Cham, 315--318.
[5]
T. Grovs up, J. Movs ko, and P. Cech. 2015. Continuous hierarchical exploration of multimedia collections. In 2015 13th International Workshop on Content-Based Multimedia Indexing (CBMI). 1--4.
[6]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25. Curran Associates, Inc., 1097--1105.
[7]
Hongzhi Li, Joseph G. Ellis, Lei Zhang, and Shih-Fu Chang. 2018. PatternNet: Visual Pattern Mining with Deep Neural Network. In Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval (ICMR '18). ACM, New York, NY, USA, 291--299.
[8]
Yao Li, Lingqiao Liu, Chunhua Shen, and Anton van den Hengel. 2014. Mid-level Deep Pattern Mining. CoRR, Vol. abs/1411.6382 (2014). arxiv: 1411.6382 http://arxiv.org/abs/1411.6382
[9]
Jakub Lokovc, Tomávs Grovs up, and Tomávs Skopal. 2012. SIR: The Smart Image Retrieval Engine. In Similarity Search and Applications, Gonzalo Navarro and Vladimir Pestov (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 240--241.
[10]
Jakub Lokovc, Tomávs Grovs up, and Tomávs Skopal. 2013. On Scalable Approximate Search with the Signature Quadratic Form Distance. In Similarity Search and Applications, Nieves Brisaboa, Oscar Pedreira, and Pavel Zezula (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 312--318.
[11]
Jakub Lokovc, Tomávs Grovsup, and Tomávs Skopal. 2012. Image Exploration Using Online Feature Extraction and Reranking. In Proceedings of the 2Nd ACM International Conference on Multimedia Retrieval (ICMR '12). ACM, New York, NY, USA, Article 66, bibinfonumpages2 pages.
[12]
Juraj Movs ko, Jakub Lokovc, Tomávs Grovs up, Pvr emysl vC ech, Tomávs Skopal, and Jan Lánský. 2015. MLES: Multilayer Exploration Structure for Multimedia Exploration. In New Trends in Databases and Information Systems, Tadeusz Morzy, Patrick Valduriez, and Ladjel Bellatreche (Eds.). Springer International Publishing, Cham, 135--144.
[13]
Giang NGuyên and Marcel Worring. 2008. Interactive access to large image collections using similarity-based visualization. Journal of Visual Languages & Computing Journal of Visual Languages and Computing, Vol. 19 (04 2008), 203--224.
[14]
Burr Settles. 2009. Active Learning Literature Survey.
[15]
Tomás Skopal, Ladislav Peska, and Tomás Grovsup. 2018. Interactive Product Search Based on Global and Local Visual-Semantic Features. In Similarity Search and Applications - 11th Int. Conference, SISAP 2018, Lima, Peru. 87--95.
[16]
Tomávs Skopal, Ladislav Pevska, Gregor Kovalvcik, Tomávs Grovs up, and Jakub Lokovc. 2017. Product Exploration Based on Latent Visual Attributes. In Proc. of the 2017 ACM on Conference on Information and Knowledge Management (CIKM '17). ACM, 2531--2534.
[17]
Z. Wang, J. Meng, T. Yu, and J. Yuan. 2017. Common visual pattern discovery and search. In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). 1011--1018.
Index Terms
- Methods of Multi-Modal Data Exploration
Recommendations
Random Methods for Fast Exploration of the Raw Video Material
DEXA '09: Proceedings of the 2009 20th International Workshop on Database and Expert Systems ApplicationIn this work we propose a visualization and access method for exploring a database of raw video material, where the span of information stored can be huge and difficult to understand at a glance. Developed visual mining tool presents the overview of the ...
Index support for content-based multimedia exploration
MM '10: Proceedings of the 18th ACM international conference on MultimediaContent-based multimedia exploration systems support users in browsing and searching voluminous multimedia databases in an interactive and playful way. Guiding the user navigation and exploration process through the database contents, similarity-based ...
Accentuating visualization parameters to guide exploration
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing SystemsWe present a new method for displaying visualization parameters to guide casual data exploration. When visualizing datasets with large parameter spaces it can be difficult to move between data views. Building on social navigation and degree-of-interest ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2019
427 pages
ISBN:9781450367653
DOI:10.1145/3323873
- General Chairs:
- Abdulmotaleb El Saddik,
- Alberto Del Bimbo,
- Zhongfei Zhang,
- Program Chairs:
- Alexander Hauptmann,
- K. Selcuk Candan,
- Marco Bertini,
- Lexing Xie,
- Xiao-Yong Wei
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 June 2019
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Grantová Agentura ðeské Republiky
Conference
ICMR '19
Sponsor:
Acceptance Rates
Overall Acceptance Rate 254 of 830 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 133Total Downloads
- Downloads (Last 12 months)4
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in