Instance Image Retrieval by Aggregating Sample-based Discriminative Characteristics
Authors: 
Zhongyan Zhang, Lei Wang, Yang Wang, Luping Zhou, Jianjia Zhang, Fang ChenAuthors Info & Claims
Pages 91 - 99
Abstract
Identifying the discriminative characteristic of a query is important for image retrieval. For retrieval without human interaction, such characteristic is usually obtained by average query expansion (AQE) or its discriminative variant (DQE) learned from pseudo-examples online, among others. In this paper, we propose a new query expansion method to further improve the above ones. The key idea is to learn a "unique'' discriminative characteristic for each database image, in an offline manner. During retrieval, the characteristic of a query is obtained by aggregating the unique characteristics of the query-relevant images collected from an initial retrieval result. Compared with AQE which works in the original feature space, our method works in the space of the unique characteristics of database images, significantly enhancing the discriminative power of the characteristic identified for a query. Compared with DQE, our method needs neither pseudo-labeled negatives nor the online learning process, leading to more efficient retrieval and even better performance. The experimental study conducted on seven benchmark datasets verifies the considerable improvement achieved by the proposed method, and also demonstrates its application to the state-of-the-art diffusion-based image retrieval.
References
[1]
Relja Arandjelović and Andrew Zisserman. 2012. Three things everyone should know to improve object retrieval Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2911--2918.
[2]
Artem Babenko, Anton Slesarev, Alexandr Chigorin, and Victor Lempitsky. 2014. Neural codes for image retrieval. In European conference on computer vision. Springer, 584--599.
[3]
Sergey Brin and Lawrence Page. 2012. Reprint of: The anatomy of a large-scale hypertextual web search engine. Computer networks, Vol. 56, 18 (2012), 3825--3833.
[4]
Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, and Fei Sha. 2016. Synthesized classifiers for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5327--5336.
[5]
Ondvrej Chum, Andrej Mikulik, Michal Perdoch, and Jivrí Matas. 2011. Total recall II: Query expansion revisited. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 889--896.
[6]
Ondrej Chum, James Philbin, Josef Sivic, Michael Isard, and Andrew Zisserman. 2007. Total recall: Automatic query expansion with a generative feature model for object retrieval Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, 1--8.
[7]
Ritendra Datta, Jia Li, and James Z Wang. 2005. Content-based image retrieval: approaches and trends of the new age Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval. ACM, 253--262.
[8]
Agni Delvinioti, Hervé Jégou, Laurent Amsaleg, and Michael E Houle. 2014. Image retrieval with reciprocal and shared nearest neighbors Computer Vision Theory and Applications (VISAPP), 2014 International Conference on, Vol. Vol. 2. IEEE, 321--328.
[9]
Michael Donoser and Horst Bischof. 2013. Diffusion processes for retrieval revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1320--1327.
[10]
Albert Gordo, Jon Almazan, Jerome Revaud, and Diane Larlus. 2017. End-to-end learning of deep visual representations for image retrieval. International Journal of Computer Vision Vol. 124, 2 (2017), 237--254.
[11]
Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Teddy Furon, and Ondvrej Chum. 2017. Efficient diffusion on region manifolds: Recovering small objects with compact cnn representations 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 926--935.
[12]
Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2008. Hamming embedding and weak geometric consistency for large scale image search. Computer Vision--ECCV 2008 (2008), 304--317.
[13]
Herve Jegou, Hedi Harzallah, and Cordelia Schmid. 2007. A contextual dissimilarity measure for accurate and efficient image search Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on. IEEE, 1--8.
[14]
Albert Jiménez, Jose M Alvarez, and Xavier Giró Nieto. 2017. Class-weighted convolutional features for visual instance search Proceedings of the 28th British Machine Vision Conference 2017. 1--12.
[15]
Yannis Kalantidis, Clayton Mellina, and Simon Osindero. 2016. Cross-dimensional weighting for aggregated deep convolutional features European Conference on Computer Vision. Springer, 685--701.
[16]
Tomasz Malisiewicz, Abhinav Gupta, and Alexei A Efros. 2011. Ensemble of exemplar-svms for object detection and beyond Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 89--96.
[17]
Panagiotis Papadakis, Ioannis Pratikakis, Theoharis Theoharis, and Stavros Perantonis. 2010. PANORAMA: A 3D shape descriptor based on panoramic views for unsupervised 3D object retrieval. International Journal of Computer Vision Vol. 89, 2 (2010), 177--192.
[18]
James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman. 2007. Object retrieval with large vocabularies and fast spatial matching Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on. IEEE, 1--8.
[19]
James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman. 2008. Lost in quantization: Improving particular object retrieval in large scale image databases Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 1--8.
[20]
Danfeng Qin, Stephan Gammeter, Lukas Bossard, Till Quack, and Luc Van Gool. 2011. Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 777--784.
[21]
Filip Radenović, Giorgos Tolias, and Ondvrej Chum. 2016. CNN image retrieval learns from BoW: Unsupervised fine-tuning with hard examples European Conference on Computer Vision. Springer, 3--20.
[22]
Matthew Richardson and Pedro Domingos. 2002. The intelligent surfer: Probabilistic combination of link and content information in pagerank Advances in neural information processing systems. 1441--1448.
[23]
Yong Rui, Thomas S Huang, Michael Ortega, and Sharad Mehrotra. 1998. Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Transactions on circuits and systems for video technology, Vol. 8, 5 (1998), 644--655.
[24]
Xiaohui Shen, Zhe Lin, Jonathan Brandt, and Ying Wu. 2014. Spatially-constrained similarity measurefor large-scale object retrieval. IEEE transactions on pattern analysis and machine intelligence, Vol. 36, 6 (2014), 1229--1241.
[25]
Arnold WM Smeulders, Marcel Worring, Simone Santini, Amarnath Gupta, and Ramesh Jain. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on pattern analysis and machine intelligence, Vol. 22, 12 (2000), 1349--1380.
[26]
Giorgos Tolias and Hervé Jégou. 2014. Visual query expansion with or without geometry: refining local descriptors by feature aggregation. Pattern recognition, Vol. 47, 10 (2014), 3466--3476.
[27]
Giorgos Tolias, Ronan Sicre, and Hervé Jégou. 2015. Particular object retrieval with integral max-pooling of CNN activations. arXiv preprint arXiv:1511.05879 (2015).
[28]
Shuang Wang and Shuqiang Jiang. 2015. INSTRE: a new benchmark for instance-level object retrieval and recognition. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 11, 3 (2015), 37.
[29]
Joe Yue-Hei Ng, Fan Yang, and Larry S Davis. 2015. Exploiting local features from deep networks for image retrieval Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 53--61.
[30]
Shaoting Zhang, Ming Yang, Timothee Cour, Kai Yu, and Dimitris N Metaxas. 2015. Query specific rank fusion for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, 4 (2015), 803--815.
[31]
Liang Zheng, Shengjin Wang, Lu Tian, Fei He, Ziqiong Liu, and Qi Tian. 2015. Query-adaptive late fusion for image search and person re-identification Proceedings of the IEEE conference on computer vision and pattern recognition. 1741--1750.
[32]
Liang Zheng, Yi Yang, and Qi Tian. 2018. SIFT Meets CNN: A Decade Survey of Instance Retrieval. IEEE Trans. Pattern Anal. Mach. Intell. Vol. 40, 5 (2018), 1224--1244.
[33]
Denny Zhou, Olivier Bousquet, Thomas N Lal, Jason Weston, and Bernhard Schölkopf. 2004 a. Learning with local and global consistency. In Advances in neural information processing systems. 321--328.
[34]
Denny Zhou, Jason Weston, Arthur Gretton, Olivier Bousquet, and Bernhard Schölkopf. 2004 b. Ranking on data manifolds. In Advances in neural information processing systems. 169--176.
[35]
Xiang Sean Zhou and Thomas S Huang. 2003. Relevance feedback in image retrieval: A comprehensive review. Multimedia systems, Vol. 8, 6 (2003), 536--544.
Index Terms
- Instance Image Retrieval by Aggregating Sample-based Discriminative Characteristics
Recommendations
Graph-based multiple-instance learning for object-based image retrieval
MIR '08: Proceedings of the 1st ACM international conference on Multimedia information retrievalWe study in this paper the problem of using multiple-instance semi-supervised learning to solve object-based image retrieval problem, in which the user is only interested in a portion of the image, and the rest of the image is considered as irrelevant. ...
Relevance-Feedback Image Retrieval Based on Multiple-Instance Learning
ICIS '08: Proceedings of the Seventh IEEE/ACIS International Conference on Computer and Information Science (icis 2008)This paper describes the development of a novel content-based image retrieval system using Multiple-Instance Learning (MIL). MIL is designed for learning on bags, each composed of a number of instances (i.e., feature vectors). For a given bag, one or ...
Instance Image Retrieval with Generative Adversarial Training
MultiMedia ModelingAbstractWhile generative adversarial training becomes promising technology for many computer vision tasks especially in image processing domain, it has few works so far on instance level image retrieval domain. In this paper, we propose an instance level ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2018
550 pages
Copyright © 2018 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 June 2018
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ICMR '18
Sponsor:
Acceptance Rates
ICMR '18 Paper Acceptance Rate 44 of 136 submissions, 32%;
Overall Acceptance Rate 254 of 830 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 212Total Downloads
- Downloads (Last 12 months)6
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in