Abstract
A novel approach to colour-based object recognition and image retrieval -the multimodal neighbourhood signature- is proposed. Object appearance is represented by colour-based features computed from image neighbourhoods with multi-modal colour density function. Stable invariants are derived from modes of the density function that are robustly located by the mean shift algorithm. The problem of extracting local invariant colour features is addressed directly, without a need for prior segmentation or edge detection. The signature is concise - an image is typically represented by a few hundred bytes, a few thousands for very complex scenes.
The algorithm’s performance is first tested on a region-based image retrieval task achieving a good (92%) hit rate at a speed of 600 image comparisons per second. The method is shown to operate successfully under changing illumination, viewpoint and object pose, as well as non-rigid object deformation, partial occlusion and the presence of background clutter dominating the scene. The performance of the multimodal neighbourhood signature method is also evaluated on a standard colour object recognition task using a publicly available dataset. Very good recognition performance (average match percentile 99.5%) was achieved in real time (average 0.28 seconds for recognising a single image) which compares favourably with results reported in the literature.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Silverman B.W. Density Estimation for Statistics and Data Analysis. New York: Chapman and Hall, 1986.
D. Comaniciu and P. Meer. Mean Shift Analysis and Applications. In Proceedings of the International Conf. On Computer Vision, pages 1197–1203, 1999.
M. Das, E. Riseman, and B. Draper. FOCUS: Searching for Multi-coloured Objects in a Diverse Image Database. In IEEE Proceedings in Computer Vision and Pattern Recognition, pages 756–761, 1997.
G. Finlayson, M. Drew, and B. Funt. Diagonal transforms suffice for color constancy. In Proceedings of the International Conference on Computer Vision, Berlin, pages 164–171, 1993.
K. Fukunaga and L. Hostetler. The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition. In IEEE Transactions in Information Theory, pages 32–40, 1975.
B. Funt, K. Barnard, and L. Martin. Is Machine Colour Constancy Good Enough? In Proceedings of the 5th European Conference on Computer Vision, pages 445–459, 1998.
B. Funt and G. Finlayson. Color Constant Color Indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(5):522–529, 1995.
T. Gevers and W. M. Smeulders. Color-based Object Recognition. Pattern Recognition, 32(3):453–464, 1999.
G. Healey and D. Slater. Global Color Constancy-Recognition Of Objects By Use Of Illumination Invariant Properties Of Color Distributions. Journal Of The Optical Society Of America A-optics Image Science And Vision, 11(11):3003–3010, 1994.
D. Jacobs, P. Belhumeur, and R. Basri. Comparing Images Under Variable Illumination. In IEEE Proceedings in Computer Vision and Pattern Recognition, pages 610–616, 1998.
D. Koubaroulis, J. Matas, and J. Kittler. MNS: A Novel Method for Colour-Based Object Recognition and Image Retrieval. Technical Report VSSP-TR-6/99, University of Surrey, December 1999. (available at http://www.ee.surrey.ac.uk/Personal/D.Koubaroulis/thesis/pub/koubaroulistr699.ps.gz).
E. Land and J. McCann. Lightness and the retinex theory. Journal of the Optical Society of America, 61(1):1–11, 1971.
J. Matas. Colour Object Recognition. PhD thesis, University Of Surrey, 1995.
J. Matas, D. Koubaroulis, and J. Kittler. Performance Evaluation of the Multimodal Neighbourhood Signature Method for Colour Object Recognition. In Proceedings of the Czech Pattern Recognition Workshop, Perslac, Czech Republic, pages 27–34, 2000. (available at http://www.ee.surrey.ac.uk/Personal/D.Koubaroulis/-thesis/pub/matas-cprw00.ps.gz).
K. Messer, J. Kittler, and M. Kraaijveld. Selecting Features for Neural Networks to Aid an Iconic Search Through an Image Database. In Proceedings of the IEE 6th International Conference on Image Processing and Its Applications, pages 428–432, 1997.
F. Mindru, T. Moons, and L. Van Gool. Recognizing Color Patterns Irrespective of Viewpoint and Illumination. In Proceedings of the Computer Vision and Pattern Recognition,Fort Collins, Colorado, pages 368–373, 1999.
K. Nagao and W. Grimson. Recognizing 3D Objects Using Photometric Invariant. Technical report, Massachusetts Institute of Technology Artificial Intelligence Lab, 1995.
S. Nayar and R. Bolle. Reflectance Based Object Recognition. International Journal of Computer Vision, 17(3):219–240, 1996.
K. Park, Il-Dong Yun, and Sang Uk Lee. Color Image Retrieval Using a Hybrid Graph Representation. Journal of Image and Vision Computing, 17(7):465–474, 1999.
Y. Rui, T.S. Huang, and S.-F. Chang. Image retrieval: Current techniques, promising directions, and open issues. Journal Of Visual Communication And Image Representation, 10(1):39–62, 1999.
R. J. Smith and S.-F. Chang. Integrated Spatial and Feature Image Query. Multimedia Systems, 7(2):129–140, 1999.
M. J. Swain and D. H. Ballard. Color Indexing. International Journal of Computer Vision, 7(1):11–32, 1991.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Matas, J.G., Koubaroulis, D., Kittler, J. (2000). Colour Image Retrieval and Object Recognition Using the Multimodal Neighbourhood Signature. In: Computer Vision - ECCV 2000. ECCV 2000. Lecture Notes in Computer Science, vol 1842. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45054-8_4
Download citation
DOI: https://doi.org/10.1007/3-540-45054-8_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67685-0
Online ISBN: 978-3-540-45054-2
eBook Packages: Springer Book Archive