More Web Proxy on the site http://driver.im/

research-article

Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations

Authors:

Roberto CipollaAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 29, Issue 6

Pages 1005 - 1018

https://doi.org/10.1109/TPAMI.2007.1037

Published: 01 June 2007 Publication History

Abstract

We address the problem of comparing sets of images for object recognition, where the sets may represent variations in an object's appearance due to changing camera pose and lighting conditions. Canonical Correlations (also known as principal or canonical angles), which can be thought of as the angles between two d\hbox{-}\rm dimensional subspaces, have recently attracted attention for image set matching. Canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the two main classical methods: parametric distribution-based and nonparametric sample-based matching of sets. Here, this is first demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effectiveness, a novel discriminative learning method over sets is proposed for set classification. Specifically, inspired by classical Linear Discriminant Analysis (LDA), we develop a linear discriminant function that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. Image sets transformed by the discriminant function are then compared by the canonical correlations. Classical orthogonal subspace method (OSM) is also investigated for the similar purpose and compared with the proposed method. The proposed method is evaluated on various object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of 500 general objects taken at different views. The method is also applied to object category recognition using ETH-80 database. The proposed method is shown to outperform the state-of-the-art methods in terms of accuracy and efficiency.

References

[1]

H. Hotelling, “Relations between Two Sets of Variates,” Biometrika, vol. 28, no. 34, pp. 321-372, 1936.

[2]

Å. Björck and G.H. Golub, “Numerical Methods for Computing Angles between Linear Subspaces,” Math. Computation, vol. 27, no. 123, pp. 579-594, 1973.

[3]

T. Kailath, “A View of Three Decades of Linear Filtering Theory,” IEEE Trans. Information Theory, vol. 20, no. 2, pp. 146-181, 1974.

[4]

E. Oja, Subspace Methods of Pattern Recognition. Research Studies Press, 1983.

[5]

R. Gittins, Canonical Analysis: A Review with Applications in Ecology. Springer-Verlag, 1985.

[6]

T.M. Cover and J.A. Thomas, Elements of Information Theory. Wiley, 1991.

Digital Library

[7]

P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman, “Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.

Digital Library

[8]

O. Yamaguchi, K. Fukui, and K. Maeda, “Face Recognition Using Temporal Image Sequence,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 318-323, 1998.

Digital Library

[9]

W.Y. Zhao, R. Chellappa, and A. Krishnaswamy, “Discriminant Analysis of Principal Components for Face Recognition,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 336-341, 1998.

Digital Library

[10]

D.D. Lee and H.S. Seung, “Learning the Parts of Objects by Non-Negative Matrix Factorization,” Nature, vol. 401, no. 6755, pp. 788-791, 1999.

[11]

Y. Li, S. Gong, and H. Liddell, “Recognising the Dynamics of Faces Across Multiple Views,” Proc. British Machine Vision Conf., pp. 242-251, 2000.

[12]

S. Satoh, “Comparative Evaluation of Face Sequence Matching for Content-Based Video Access,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 163-168, 2000.

Digital Library

[13]

D.M. Blackburn, M. Bone, and P.J. Phillips, Facial Recognition Vendor Test 2000: Evaluation Report, 2000.

[14]

R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. John Wily and Sons, 2000.

Digital Library

[15]

D.D. Lee and H.S. Seung, “Algorithms for Non-Negative Matirx Factorization,” Advances in Neural Information Processing Systems, pp. 556-562, 2001.

[16]

P.J. Phillips, P. Grother, R.J. Micheals, D.M. Blackburn, E. Tabassi, and J.M. Bone, Facial Recognition Vendor Test 2002: Evaluation Report, http://www.frvt.org/FRVT2002/, Mar. 2003

[17]

G. Shakhnarovich, J.W. Fisher, and T. Darrel, “Face Recognition from Long-Term Observations,” Proc. European Conf. Computer Vision, pp. 851-868, 2002.

Digital Library

[18]

M.-H. Yang, “Kernel Eigenfaces vs. Kernel Fisherfaces: Face Recognition Using Kernel Methods,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 215-220, 2002.

Digital Library

[19]

M. Bressan and J. Vitria, “Nonparametric Discriminant Analysis and Nearest Neighbor Classification,” Pattern Recognition Letters, vol. 24, no. 15, pp. 2743-2749, 2003.

Digital Library

[20]

K. Lee, M. Yang, and D. Kriegman, “Video-Based Face Recognition Using Probabilistic Apperance Manifolds,” Proc. Computer Vision and Pattern Recognition Conf., pp. 313-320, 2003.

Digital Library

[21]

S. Zhou, V. Krueger, and R. Chellappa, “Probabilistic Recognition of Human Faces from Video,” Computer Vision and Image Inderstanding, vol. 91, no. 1, pp. 214-245, 2003.

Digital Library

[22]

X. Liu and T. Chen, “Video-Based Face Recognition Using Adaptive Hidden Markov Models,” Proc. Computer Vision and Pattern Recognition Conf., pp. 340-345, 2003.

Digital Library

[23]

L. Wolf and A. Shashua, “Learning over Sets Using Kernel Principal Angles,” J. Machine Learning Research, vol. 4, no. 10, pp.913-931, 2003.

Digital Library

[24]

K. Fukui and O. Yamaguchi, “Face Recognition Using Multi-Viewpoint Patterns for Robot Vision,” Proc. Int'l Symp. Robotics Research, pp. 192-201, 2003.

[25]

B. Leibe and B. Schiele, “Analyzing Appearance and Contour Based Methods for Object Categorization,” Proc. Computer Vision and Pattern Recognition Conf., pp. 409-415, 2003.

[26]

D. Hardoon, S. Szedmak, and J. Shawe-Taylor, “Canonical Correlation Analysis; An Overview with Application to Learning Methods,” Neural Computation, vol. 16, no. 12, pp. 2639-2664, 2004.

Digital Library

[27]

T. Kozakaya, O. Yamaguchi, and K. Fukui, “Development and Evaluation of Face Recognition System Using Constrained Mutual Subspace Method,” IPSJ J., vol. 45, no. 3 pp. 951-959, 2004.

[28]

A. Hadid and M. Pietikainen, “From Still Image to Video-Based Face Recognition: An Experimental Analysis,” Proc. Sixth IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 813-818, 2004.

Digital Library

[29]

M.T. Sadeghi and J.V. Kittler, “Decision Making in the LDA Space: Generalised Gradient Direction Metric,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 248-253, 2004.

Digital Library

[30]

X. Wang and X. Tang, “Random Sampling LDA for Face Recognition,” Proc. Computer Vision and Pattern Recognition Conf., pp. 259-265, 2004.

Digital Library

[31]

P. Viola and M. Jones, “Robust Real-Time Face Detection,” Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.

Digital Library

[32]

J. Via, I. Santamaria, and J. Perez, “Canonical Correlation Analysis (CCA) Algorithms for Multiple Data Sets: Application to Blind SIMO Equalization,” Proc. 13th European Signal Processing Conf., 2005.

[33]

J.M. Geusebroek, G.J. Burghouts, and A.W.M. Smeulders, “The Amsterdam Library of Object Images,” Int'l J. Computer Vision, vol. 61, no. 1, pp. 103-112, Jan. 2005.

Digital Library

[34]

O. Arandjelović, G. Shakhnarovich, J. Fisher, R. Cipolla, and T. Darrell, “Face Recognition with Image Sets Using Manifold Density Divergence,” Proc. Computer Vision and Pattern Recognition Conf., pp. 581-588, 2005.

Digital Library

[35]

T.-K. Kim and J. Kittler, “Locally Linear Discriminant Analysis for Multimodally Distributed Classes for Face Recognition with a Single Model Image,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 318-327, Mar. 2005.

Digital Library

[36]

T.-K. Kim, O. Arandjelović, and R. Cipolla, “Learning over Sets Using Boosted Manifold Principal Angles (BoMPA),” Proc. British Machine Vision Conf., pp. 779-788, 2005.

[37]

M. Nishiyama, O. Yamaguchi, and K. Fukui, “Face Recognition with the Multiple Constrained Mutual Subspace Method,” Proc. Audio and Video-Based Biometric Person Authentication, pp. 71-80, 2005.

Digital Library

[38]

T.-K. Kim, J. Kittler, and R. Cipolla, “Learning Discriminative Canonical Correlations for Object Recognition with Image Sets,” Proc. European Conf. Computer Vision, pp. 251-262, 2006.

Digital Library

[39]

K. Fukui, B. Stenger, and O. Yamaguchi, “A Framework for 3D Object Recognition Using the Kernel Constrained Mutual Subspace Method,” Proc. Asian Conf. Computer Vision, pp. 315-324, 2006.

Digital Library

[40]

Toshiba Corporation “Facepass,” http://www.toshiba.co.jp/rdc/mmlab/tech/w31e.htm, 2005.

Cited By

Fei YLiu YJia CLi ZWei XChen M(2025)A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian ManifoldACM Computing Surveys10.1145/370849857:5(1-37)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3708498
Mohammadi MBabai MWilkinson M(2025)Generalized Relevance Learning Grassmann QuantizationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.346631547:1(502-513)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TPAMI.2024.3466315
Shen XSong PYuan YZheng YWooldridge MDy JNatarajan S(2024)Distributed manifold hashing for image set classification and retrievalProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i5.28282(4802-4810)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i5.28282
Show More Cited By

Index Terms

Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition
  2. Machine learning
    1. Machine learning approaches

Recommendations

Face recognition using discriminative locality preserving vectors

We proposed an effective face recognition method based on the discriminative locality preserving vectors method (DLPV). Using the analysis of eigenspectrum modeling of locality preserving projections, we selected the reliable face variation subspace of ...
Kernel Grassmannian distances and discriminant analysis for face recognition from image sets

We address the problem of face recognition from image sets, where subject-specific subspaces instead of image vectors are compared. Previous methods based on Grassmannian subspace distances mainly take linear subspaces as input. The non-linearity exists ...
Discriminant clustering embedding for face recognition with image sets
ACCV'07: Proceedings of the 8th Asian conference on Computer vision - Volume Part II

In this paper, a novel local discriminant embedding method, Discriminant Clustering Embedding (DCE), is proposed for face recognition with image sets. DCE combines the effectiveness of submanifolds, which are extracted by clustering for each subject's ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 29, Issue 6

June 2007

190 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © 2007.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 June 2007

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

155
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fei YLiu YJia CLi ZWei XChen M(2025)A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian ManifoldACM Computing Surveys10.1145/370849857:5(1-37)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3708498
Mohammadi MBabai MWilkinson M(2025)Generalized Relevance Learning Grassmann QuantizationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.346631547:1(502-513)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TPAMI.2024.3466315
Shen XSong PYuan YZheng YWooldridge MDy JNatarajan S(2024)Distributed manifold hashing for image set classification and retrievalProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i5.28282(4802-4810)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i5.28282
Shen XWu WWang XZheng Y(2024)Multiple Riemannian Kernel Hashing for Large-Scale Image Set Classification and RetrievalIEEE Transactions on Image Processing10.1109/TIP.2024.341941433(4261-4273)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3419414
Hachiya HSaito Y(2024)Set representative vector and its asymmetric attention-based transformation for heterogeneous set-to-set matchingNeurocomputing10.1016/j.neucom.2024.127372578:COnline publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127372
Ma JTang YShang Z(2024)Discriminative latent subspace learning with adaptive metric learningNeural Computing and Applications10.1007/s00521-023-09159-836:4(2049-2066)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s00521-023-09159-8
Yan KLi D(2023)Enterprise Management Data Acquisition System Based on WoTInternational Journal of Information System Modeling and Design10.4018/IJISMD.31357713:2(1-11)Online publication date: 31-Mar-2023
https://dl.acm.org/doi/10.4018/IJISMD.313577
Wei DShen XSun QGao XRen Z(2023)Sparse Representation Classifier Guided Grassmann Reconstruction Metric Learning With Applications to Image Set AnalysisIEEE Transactions on Multimedia10.1109/TMM.2022.317353525(4307-4322)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3173535
Wax MAdler A(2023)Vector Set Classification by Signal Subspace MatchingIEEE Transactions on Information Theory10.1109/TIT.2022.320768669:3(1853-1865)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TIT.2022.3207686
Lu YWong WZeng BLai ZLi X(2023)Guided Discrimination and Correlation Subspace Learning for Domain AdaptationIEEE Transactions on Image Processing10.1109/TIP.2023.326175832(2017-2032)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIP.2023.3261758
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents