Abstract
Recent machine learning techniques have demonstrated their capability for identifying image categories using image features. Among these techniques, Support Vector Machines (SVM) present good results for example in Pascal Voc challenge 2011 [8], particularly when they are associated with a kernel function [28, 35]. However, nowadays image categorization task is very challenging owing to the sizes of benchmark datasets and the number of categories to be classified. In such a context, lot of effort has to be put in the design of the kernel functions and underlying semantic features. In the following of the paper we call semantic features the features describing the (semantic) content of an image. In this paper, we propose a framework to learn an effective kernel function using the Boosting paradigm to linearly combine weak kernels. We then use a SVM with this kernel to categorize image databases. More specifically, this method create embedding functions to map images in a Hilbert space where they are better classified. Furthermore, our algorithm benefits from boosting process to learn this kernel with a complexity linear with the size of the training set. Experiments are carried out on popular benchmarks and databases to show the properties and behavior of the proposed method. On the PASCAL VOC2006 database, we compare our method to simple early fusion, and on the Oxford Flowers databases we show that our method outperforms the best Multiple Kernel Learning (MKL) techniques of the literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
\(\langle \mathbf{v}\mathbf{v}^\top, \mathbf{M}\mathbf{M}^\top \rangle = \text{trace}(\mathbf{v}\mathbf{v}^\top\mathbf{M}\mathbf{M}^\top) = \text{trace}(\mathbf{M}^\top\mathbf{v}\left(\mathbf{M}^\top\mathbf{v}\right)^\top) = \|\mathbf{M}^\top\mathbf{v}\|^2\).
References
Awais M, Yan F, Mikolajczyk K, Kittler J (2011) Augmented kernel matrix vs classifier fusion for object recognition. In: British machine vision conference. BMVA Press, pp 60.1–60.11
Bach, FR, Lanckriet GRG (2004) Multiple kernel learning, conic duality, and the smo algorithm. In: International conference on machine learning
Cortes C (2009) Invited talk: can learning kernels help performance? In: International conference on machine learning, p 161
Cortes C, Mohri M, Rostamizadeh A (2010) Two-stage learning kernel algorithms. In: International conference on machine learning
Crammer K, Keshet J, Singer Y (2001) Kernel design using boosting. In: Advances in neural information processing systems. MIT Press, pp 537–544
Cristianini N, Shawe-Taylor J, Elisseff A, Kandola J (2001) On kernel target alignement. In: Advances in neural information processing systems. Vancouver, Canada, pp 367–373
Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learning. In: International conference on machine learning, vol 227. Corvalis, Oregon
Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2011) The Pascal visual object classes challenge 2011 (VOC2011) results. http://www.pascal-network.org/challenges/VOC/voc2011/workshop/index.html
Figueiras-Vidal AR, Rokach L (2012) An exploration of research directions in machine ensemble theory and applications. In: European symposium on artificial neural networks, computational intelligence and machine learning
Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: a statistical view of boosting. Annals Stat 2:337–374
Gehler P, NowoziS S (2009) On feature combination for multiclass object classification. In: IEEE international conference on computer vision, pp 221–228
Gong Y, Lazebnik S, Gordo A, Perronnin F (2012) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans pattern Anal Mach Intell 99:1 (PrePrints)
Gosselin PH, Cord M (2006) Feature based approach to semi-supervised similarity learning. Pattern Recognit 39:1839–1851. Special issue on similarity-based pattern recognition
Gosselin PH, Cord M, Philipp-Foliguet S (2008) Combining visual dictionary, kernel-based similarity and learning strategy for image category retrieval. Comput Vis Image Underst 110(3):403–417. Special issue on similarity matching in computer vision and multimedia
Gosselin PH, Precioso F, Philipp-Foliguet S (2010) Incremental kernel learning for active image retrieval without global dictionaries. Pattern Recognit 44(10–11):2244–2254
Hazen TJ (2010) Multi-class svm optimization using mce training with application to topic identification. In: IEEE International conference on acoustics, speech, and signal processing, pp 5350–5353
Kawanabe M, Nakajima S, Binder A (2009) A procedure of adaptive kernel combination with kernel-target alignment for object classification. In: ACM international conference on image and video retrieval
Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12:953–997
Lanckriet GRG, Cristianini N, Bartlett N, El Ghaoui L, Jordan MI (2004) Learning the kernel matrix with semi-definite programming. Int J Mach Learn Res 5:27–72
Madzarov G, Gjorgjevikj D, Chorbev I (2008) A multi-class svm classifier utilizing binary decision tree. Informatica 33(1):233–242
Nilsback ME, Zisserman A (2008) Automated flower classification over a large number of classes. In: Proceedings of the Indian conference on computer vision, graphics and image processing
Orabona F, Jie L (2011) Ultra-fast optimization algorithm for sparse multi kernel learning. In: Getoor L, Scheffer T (eds) Proceedings of the 28th international conference on machine learning (ICML-11), ICML ’11. New York, NY, USA, ACM, pp. 249–256
Picard D, Thome N, Cord M, Rakotomamonjy A (2012) Learning geometric combinations of gaussian kernels with alternating quasi-newton algorithm. In: European symposium on artificial neural networks, computational intelligence and machine learning
Rakotomamonjy A, Bach F, Canu S, Grandvalet Y (2008) Simplemkl. J Mach Learn Res 9:2491–2521
Sánchez J, Perronnin F (2011) High-dimensional signature compression for large-scale image classification. In: CVPR, pp 1665–1672
Schölkopf B, Smola A (2002) Learning with Kernels. MIT Press, Cambridge, MA
Shawe-Taylor J, Cristianini N (2004) Kernel methods for Pattern Analysis. Cambridge University Press, ISBN 0-521-81397-2
Song Z, Chen Q, Huang ZY, Hua Y, Yan S (2011) Contextualizing object detection and classification. In: CVPR, pp 1585–1592
Sonnenburg S, Rtsh G, Schfer C, Schlkopf B (2006) Large scale multiple kernel learning. In J Mach Learn Res 7:1531–1565
Tieu K, Viola P (2000) Boosting image retrieval. In: IEEE international conference on computer vision and pattern recognition, pp 228–235
Vapnik V (1982) Estimation of dependences based on empirical data. Springer-Verlag
Varma M, Babu BR (2009) More generality in efficient multiple kernel learning. In: International conference on machine learning
Vert R (2002) Designing a m-svm kernel for protein secondary structure prediction. Master’s thesis, DEA informatique de Lorraine
Yu S, Tranchevent LC, De Moor B, Moreau Y (2011) Kernel-based data fusion for machine learning - Methods and applications in bioinformatics and text mining, vol 345. Studies in computational intelligence. Springer
Yu Y, Zhang J, Huang Y, Zheng S, Ren W, Wang C, Huang K, Tan T (2010) Object detection by context and boosted hog-lbp. In: ECCV workshop on PASCAL VOC
Author information
Authors and Affiliations
Corresponding author
Additional information
Thanks to DGA agency for funding.
Rights and permissions
About this article
Cite this article
Lechervy, A., Gosselin, PH. & Precioso, F. Boosted kernel for image categorization. Multimed Tools Appl 69, 471–490 (2014). https://doi.org/10.1007/s11042-012-1328-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-012-1328-1