More Web Proxy on the site http://driver.im/

short-paper

Towards Evaluating and Simulating Keyword Queries for Development of Interactive Known-item Search Systems

Authors:

Ladislav Peška,

František Mejzlík,

Tomáš Souček,

Jakub LokočAuthors Info & Claims

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

Pages 281 - 285

https://doi.org/10.1145/3372278.3390726

Published: 08 June 2020 Publication History

Abstract

Searching for memorized images in large datasets (known-item search) is a challenging task due to a limited effectiveness of retrieval models as well as limited ability of users to formulate suitable queries and choose an appropriate search strategy. A popular option to approach the task is to automatically detect semantic concepts and rely on interactive specification of keywords during the search session. Nonetheless, employed instances of such search models are often set arbitrarily in existing KIS systems as comprehensive evaluations with reals users are time demanding. This paper envisions and investigates an option to simulate keyword queries in a selected "toy'' (yet competitive) keyword search model relying on a deep image classification network. Specifically, two properties of such keyword-based model are experimentally investigated with our known-item search benchmark dataset: which output transformation and ranking models are effective for the utilized classification model and whether there are some options for simulations of keyword queries. In addition to the main objective, the paper inspects also the effect of interactive query reformulations for the considered keyword search model.

References

[1]

Claudiu Cobâ rzan, Klaus Schoeffmann, Werner Bailer, Wolfgang Hü rst, Adam Blavzek, Jakub Lokovc, Stefanos Vrochidis, Kai Uwe Barthel, and Luca Rossetto. 2017. Interactive video search tools: a detailed analysis of the video browser showdown 2015. Multimedia Tools Appl., Vol. 76, 4 (2017), 5539--5571. https://doi.org/10.1007/s11042-016--3661--2

[2]

J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 248--255. https://doi.org/10.1109/CVPR.2009.5206848

[3]

Cathal Gurrin, Klaus Schoeffmann, Hideo Joho, Andreas Leibetseder, Liting Zhou, Aaron Duane, Duc-Tien Dang-Nguyen, Michael Riegler, Luca Piras, Minh-Triet Tran, Jakub Lokovc, and Wolfgang Hürst. 2019. [Invited papers] Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018). ITE Transactions on Media Technology and Applications, Vol. 7, 2 (2019), 46--59. https://doi.org/10.3169/mta.7.46

[4]

K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778. https://doi.org/10.1109/CVPR.2016.90

[5]

Björn Þór Jónsson, Omar Shahbaz Khan, Dennis C. Koelma, Stevan Rudinac, Marcel Worring, and Jan Zahálka. 2020. Exquisitor at the Video Browser Showdown 2020. In MultiMedia Modeling, Yong Man Ro, Wen-Huang Cheng, Junmo Kim, Wei-Ta Chu, Peng Cui, Jung-Woo Choi, Min-Chun Hu, and Wesley De Neve (Eds.). Springer International Publishing, Cham, 796--802.

[6]

Andreas Leibetseder, Bernd Münzer, Jürgen Primus, Sabrina Kletz, and Klaus Schoeffmann. 2020. diveXplore 4.0: The ITEC Deep Interactive Video Exploration System at VBS2020. In MultiMedia Modeling, Yong Man Ro, Wen-Huang Cheng, Junmo Kim, Wei-Ta Chu, Peng Cui, Jung-Woo Choi, Min-Chun Hu, and Wesley De Neve (Eds.). Springer International Publishing, Cham, 753--759.

[7]

Xirong Li, Chaoxi Xu, Gang Yang, Zhineng Chen, and Jianfeng Dong. 2019. W2VV+: Fully Deep Learning for Ad-hoc Video Search. In Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, Nice, France, October 21--25, 2019. 1786--1794. https://doi.org/10.1145/3343031.3350906

Digital Library

[8]

Jakub Lokoc, Gregor Kovalc'i k, Tomá s Soucek, Jaroslav Moravec, and Premysl Cech. 2019. A Framework for Effective Known-item Search in Video. In Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, Nice, France, October 21--25, 2019. 1777--1785. https://doi.org/10.1145/3343031.3351046

Digital Library

[9]

Jakub Lokovc, Werner Bailer, Klaus Schoeffmann, Bernd Münzer, and George Awad. 2018. On Influential Trends in Interactive Video Retrieval: Video Browser Showdown 2015--2017. IEEE Trans. Multimedia, Vol. 20, 12 (2018), 3361--3376. https://doi.org/10.1109/TMM.2018.2830110

Digital Library

[10]

Jakub Lokovc, Gregor Kovalvcik, Bernd Münzer, Klaus Schöffmann, Werner Bailer, Ralph Gasser, Stefanos Vrochidis, Phuong Anh Nguyen, Sitapa Rujikietgumjorn, and Kai Uwe Barthel. 2019. Interactive Search or Sequential Browsing? A Detailed Analysis of the Video Browser Showdown 2018. ACM Trans. Multimedia Comput. Commun. Appl., Vol. 15, 1, Article 29 (Feb. 2019), 18 pages. https://doi.org/10.1145/3295663

Digital Library

[11]

Tomas Mikolov, Kai Chen, Greg S. Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. http://arxiv.org/abs/1301.3781

[12]

Phuong Anh Nguyen, Jiaxin Wu, Chong-Wah Ngo, Danny Francis, and Benoit Huet. 2020. VIREO @ Video Browser Showdown 2020. In MultiMedia Modeling, Yong Man Ro, Wen-Huang Cheng, Junmo Kim, Wei-Ta Chu, Peng Cui, Jung-Woo Choi, Min-Chun Hu, and Wesley De Neve (Eds.). Springer International Publishing, Cham, 772--777.

[13]

Ladislav Pevs ka, Gregor Kovalcík, and Jakub Lokovc. 2019. Towards Automatic Configuration of Interactive Known-Item Search Systems. In Similarity Search and Applications, Giuseppe Amato, Claudio Gennaro, Vincent Oria, and Milovs Radovanović (Eds.). Springer International Publishing, Cham, 340--347.

[14]

Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. CoRR, Vol. abs/1804.02767 (2018). arxiv: 1804.02767 http://arxiv.org/abs/1804.02767

[15]

Luca Rossetto, Heiko Schuldt, George Awad, and Asad A. Butt. 2019. V3C - A Research Video Collection. In MultiMedia Modeling - 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8--11, 2019, Proceedings, Part I. 349--360. https://doi.org/10.1007/978--3-030-05710--7_29

[16]

Loris Sauter, Mahnaz Amiri Parian, Ralph Gasser, Silvan Heller, Luca Rossetto, and Heiko Schuldt. 2020. Combining Boolean and Multimedia Retrieval in vitrivr for Large-Scale Video Search. In MultiMedia Modeling, Yong Man Ro, Wen-Huang Cheng, Junmo Kim, Wei-Ta Chu, Peng Cui, Jung-Woo Choi, Min-Chun Hu, and Wesley De Neve (Eds.). Springer International Publishing, Cham, 760--765.

[17]

A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, 12 (Dec 2000), 1349--1380. https://doi.org/10.1109/34.895972

Digital Library

[18]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7--12, 2015. 1--9.

[19]

Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning Spatiotemporal Features with 3D Convolutional Networks. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) (ICCV '15). IEEE Computer Society, Washington, DC, USA, 4489--4497. https://doi.org/10.1109/ICCV.2015.510

Digital Library

[20]

Kazuya Ueki, Kotaro Kikuchi, Susumu Saito, and Tetsunori Kobayashi. 2016. Waseda at TRECVID 2016: ad-hoc video search. In TRECVID 2016 Workshop, Gaithersburg, MD, USA, Vol. 24.

[21]

Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2017. Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 39, 4 (2017), 652--663.

Digital Library

[22]

Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. 2017. Learning Transferable Architectures for Scalable Image Recognition. CoRR, Vol. abs/1707.07012 (2017). arxiv: 1707.07012 http://arxiv.org/abs/1707.07012

Cited By

Lokoč JVopálková ZDokoupil PPeška L(2023)Video Search with CLIP and Interactive Text Query ReformulationMultiMedia Modeling10.1007/978-3-031-27077-2_50(628-633)Online publication date: 9-Jan-2023
https://dl.acm.org/doi/10.1007/978-3-031-27077-2_50
Peška LLokoč J(2022)Rating-Aware Self-Organizing MapsMultiMedia Modeling10.1007/978-3-030-98358-1_10(119-130)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-030-98358-1_10

Index Terms

Towards Evaluating and Simulating Keyword Queries for Development of Interactive Known-item Search Systems
1. Information systems
  1. Information retrieval

Recommendations

Evaluating leading web search engines on children's queries
HCII'11: Proceedings of the 14th international conference on Human-computer interaction: users and applications - Volume Part IV

This study compared retrieved results, relevance ranking, and overlap across Google, Yahoo!, Bing, Yahoo Kids!, and Ask Kids on 15 queries constructed by middle school children. Queries included one word, two words, and multiple words/phrases/natural ...
Towards an Effective XML Keyword Search

Inspired by the great success of information retrieval (IR) style keyword search on the web, keyword search on XML has emerged recently. The difference between text database and XML database results in three new challenges: 1) Identify the user search ...
Towards An Interactive Keyword Search over Relational Databases
WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web

Keyword search over relational databases has been widely studied for the exploration of structured data in a user-friendly way. However, users typically have limited domain knowledge or are unable to precisely specify their search intention. Existing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

June 2020

605 pages

ISBN:9781450370875

DOI:10.1145/3372278

General Chairs:
Cathal Gurrin
Dublin City University, Ireland
,
Björn Þór Jónsson
IT University of Copenhagen, Denmark
,
Noriko Kando
National Institute of Informatics, Tokyo
,
Program Chairs:
Klaus Schoeffmann
Klagenfurt University, Austria
,
Phoebe Chen
La Trobe University, Australia
,
Noel E. O'Connor
Dublin City University, Ireland

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Grantová Agentura ðeské Republiky

Conference

ICMR '20

Sponsor:

SIGMM

ICMR '20: International Conference on Multimedia Retrieval

June 8 - 11, 2020

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
79
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lokoč JVopálková ZDokoupil PPeška L(2023)Video Search with CLIP and Interactive Text Query ReformulationMultiMedia Modeling10.1007/978-3-031-27077-2_50(628-633)Online publication date: 9-Jan-2023
https://dl.acm.org/doi/10.1007/978-3-031-27077-2_50
Peška LLokoč J(2022)Rating-Aware Self-Organizing MapsMultiMedia Modeling10.1007/978-3-030-98358-1_10(119-130)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-030-98358-1_10

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten