Fast object instance search in videos from one example
Pages 4381 - 4385
Abstract
We present an efficient approach to search for and locate all occurrences of a specific object in large video volumes, given a single query example. Locations of object occurrences are returned as spatio-temporal trajectories in the 3D video volume. Despite much work on object instance search in image datasets, these methods locate the object independently in each image, therefore do not preserve the spatio-temporal consistency in consecutive video frames. This results in sub-optimal performance if directly applied to videos, as will be shown in our experiments. We propose to locate the object jointly across video frames using spatio-temporal search. The efficiency and effectiveness of the proposed approach is demonstrated on a consumer video dataset consisting of crawled YouTube videos and mobile captured consumer clips. Our method significantly improves the localized search accuracy over the baseline, which treats each frame independently. Moreover, it is able to find the top 100 object trajectories in the 5.5-hour dataset within 30 seconds.
References
[1]
Ran Tao, E. Gavves, C.G.M. Snoek, and A.W.M. Smeulders, “Locality in generic instance search from one example,” in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, June 2014, pp. 2099–2106.
[2]
Yuning Jiang, Jingjing Meng, Junsong Yuan, and Jiebo Luo, “Randomized spatial context for object search,” Image Processing, IEEE Transactions on, p. to appear, 2015.
[3]
Xiaohui Shen, Zhe Lin, Jon Brandt, Shai Avidan, and Ying Wu, “Object retrieval and localization with spatially-constrained similarity measure and k-nn reranking,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2012.
[4]
Josef Sivic and Andrew Zisserman, “Video google: A text retrieval approach to object matching in videos,” in Proc. IEEE Intl. Conf.on Computer Vision, 2003.
[5]
Christoph H. Lampert, Matthew B. Blaschko, and Thomas Hofmann, “Beyond sliding windows: Object localization by efficient subwindow search,” in Proc. IEEE Conf.on Computer Vision and Pattern Recognition, 2008.
[6]
Christoph H. Lampert, “Detecting objects in large image collections and videos by efficient subimage retrieval,” in Proc. IEEE Intl. Conf.on Computer Vision, 2009.
[7]
James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman, “Object retrieval with large vocabularies and fast spatial matching,” in Proc. IEEE Conf.on Computer Vision and Pattern Recognition, 2007.
[8]
Josef Sivic and Andrew Zisserman, “Efficient visual search of videos cast as text retrieval,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 591–606, 2009.
[9]
Jingjing Meng, Junsong Yuan, Yuning Jiang, Nitya Narasimhan, Venu Vasudevan, and Ying Wu, “Inter-active visual object search through mutual information maximization,” in Proc. ACM Multimedia, 2010.
[10]
Yuning Jiang, Jingjing Meng, and Junsong Yuan, “Ran-domized visual phrases for object search,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2012.
[11]
Jingjing Meng, Junsong Yuan, Gang Wang, Yap-Peng Tan, and Jianbo Xu, “Object instance search in videos,” in Proc. International Conf. on Information, Communications and Signal Processing, 2013.
[12]
Takahito Kawanishi, Akisato Kimura, Kunio Kashino, Shin'ichi Satoh, Duy-Dinh Le, Xiaomeng Wu, and Sbastien PoulIot, “Ntt communication science laboratories and nii in treevid 2010 instance search task,” in TRECVID'10, 2010, pp. -1–1.
[13]
Duy-Dinh Le, Cai-Zhi Zhu, Sebastien Poullot, Vu Q Lam, Vu H Nguyen, Nhan C Duong, Thanh D Ngo, Duc A Duong, and Shinichi Satoh, “National institute of informatics, japan at trecvid 2012,”.
[14]
Cai-Zhi Zhu and Shin'ichi Satoh, “Large vocabulary quantization for searching instances from videos,” in Proceedings of the 2Nd ACM International Conference on Multimedia Retrieval, New York, NY, USA, 2012, ICMR '12, pp. 52:1–52:8, ACM.
[15]
Yan Yang and Shinichi Satoh, “Efficient instance search from large video database via sparse filters in subspaces,” in Image Processing (ICIP), 2013 20th IEEE International Conference on, 2013, pp. 3972–3976.
[16]
Du Tran, Junsong Yuan, and David Forsyth, “Video event detection: From subvolume localization to spatiotemporal path search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 404–416, 2014.
[17]
Du Tran and Junsong Yuan, “Optimal spatio-temporal path discovery for video event detection,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2011.
[18]
Michal Perd'och, Ondrej Chum, and Jiri Matas, “Ef-ficient representation of local geometry for large scale object retrieval,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition. IEEE, 2009, pp. 9–16.
[19]
R. Arandjelovió and A. Zisserman, “Three things everyone should know to improve object retrieval,” in IEEE Computer Vision and Pattern Recognition, 2012.
[20]
Marius Muja and David G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” in International Conference on Computer Vision Theory and Application VISSAPP'09). 2009, pp. 331–340, INSTICC Press.
[21]
R. Kasturi, D. Goldgof, P. Soundararajan, V. Manohar, J. Garofolo, R. Bowers, M. Boonstra, V. Korzhova, and Jing Zhang, “Framework for performance evaluation of face, text, and vehicle detection and tracking in video: Data, metrics, and protocol,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 31, no. 2, pp. 319–336, Feb 2009.
Index Terms
- Fast object instance search in videos from one example
Index terms have been assigned to the content through auto-classification.
Recommendations
Object Instance Search in Videos via Spatio-Temporal Trajectory Discovery
Given a specific object as query, object instance search aims to not only retrieve the images or frames that contain the query, but also locate all its occurrences. In this work, we explore the use of spatio-temporal cues to improve the quality of object ...
Query Adaptive Instance Search using Object Sketches
MM '16: Proceedings of the 24th ACM international conference on MultimediaSketch-based object search is a challenging problem mainly due to two difficulties: (1) how to match the binary sketch query with the colorful image, and (2) how to locate the small object in a big image with the sketch query. To address the above ...
Is my object in this video? reconstruction-based object search in videos
IJCAI'17: Proceedings of the 26th International Joint Conference on Artificial IntelligenceThis paper addresses the problem of videolevel object instance search, which aims to retrieve the videos in the database that contain a given query object instance. Without prior knowledge about "when" and "where" an object of interest may appear in a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
5242 pages
Copyright © 2015.
Publisher
IEEE Press
Publication History
Published: 01 September 2015
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 27 Feb 2025