More Web Proxy on the site http://driver.im/

article

Analysis of correlation based dimension reduction methods

Authors:

Cheong ParkAuthors Info & Claims

International Journal of Applied Mathematics and Computer Science, Volume 21, Issue 3

Pages 549 - 558

Published: 01 September 2011 Publication History

Abstract

Analysis of correlation based dimension reduction methodsDimension reduction is an important topic in data mining and machine learning. Especially dimension reduction combined with feature fusion is an effective preprocessing step when the data are described by multiple feature sets. Canonical Correlation Analysis CCA and Discriminative Canonical Correlation Analysis DCCA are feature fusion methods based on correlation. However, they are different in that DCCA is a supervised method utilizing class label information, while CCA is an unsupervised method. It has been shown that the classification performance of DCCA is superior to that of CCA due to the discriminative power using class label information. On the other hand, Linear Discriminant Analysis LDA is a supervised dimension reduction method and it is known as a special case of CCA. In this paper, we analyze the relationship between DCCA and LDA, showing that the projective directions by DCCA are equal to the ones obtained from LDA with respect to an orthogonal transformation. Using the relation with LDA, we propose a new method that can enhance the performance of DCCA. The experimental results show that the proposed method exhibits better classification performance than the original DCCA.

References

[1]

Anton, H. and Busby, R. (2003). Contemporary Linear Algebra, John Wiley and Sons, Denver, CO.

[2]

Baudat, G. and Anouar, F. (2000). Generalized discriminant analysis using a kernel approach, Neural Computation 12(10): 2385-2404.

Digital Library

[3]

Billings, S. and Lee, K. (2002). Nonlinear fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm, Neural Networks 15(2): 263-270.

Digital Library

[4]

Chen, L., Liao, H., M.Ko, Lin, J. and Yu, G. (2000). A new LDA-based face recognition system which can solve the small sample size problem, Pattern Recognition 33(10): 1713-1726.

[5]

Duda, R., Hart, P. and Stork, D. (2001). Pattern Classification, Wiley Interscience, New York, NY.

[6]

Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition, 2nd Edn., Academic Press, San Diego, CA.

[7]

Fukunaga, K. and Mantock, J. (1983). Nonparametric discriminant analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 5(6): 671-678.

Digital Library

[8]

Garthwaite, P. (1994). An interpretation of partial least squares, Journal of the American Statistical Society 89(425): 122- 127.

[9]

He, X. and Niyogi, P. (2003). Locality preserving projections, Proceedings of the Advances in Neural Information Pro cessing Systems Conference, Vancouver, Canada, pp. 153- 160.

[10]

Hotelling, H. (1936). Relations between two sets of variates, Biometrika 28(3): 321-377.

[11]

Hou, C., Nie, F., Zhang, C. and Wu, Y. (2009). Learning an orthogonal and smooth subspace for image classification, IEEE Signal Processing Letters 16(4): 303-306.

[12]

Howland, P. and Park, H. (2004). Generalizing discriminant analysis using the generalized singular value decomposition, IEEE Transactions on Pattern Analysis and Machine Intelligence 26(8): 995-1006.

Digital Library

[13]

Jolliffe, I. (1986). Principal Component Analysis, Springer, New York, NY.

[14]

Nie, F., Xiang, S. and Zhang, C. (2007). Neighborhood minmax projections, Proceedings of the International Joint Conference on Artificial Intelligence, Hyderabad, India, pp. 993- 998.

[15]

Pardalos, P. and Hansen, P. (2008). Data Mining and Mathematical Programming, CRM Proceedings & Lecture Notes, Vol. 45, American Mathematical Society, Montreal.

[16]

Park, C. and Park, H. (2008). A comparison of generalized linear discriminant analysis algorithms, Pattern Recognition 41(3): 1083-1097.

Digital Library

[17]

Roweis, S.T. and Saul, L.K. (2000). Nonlinear dimensionality reduction by locally linear embedding, Science 290(5500): 2323-2326.

[18]

Sugiyama, M. (2006). Local fisher discriminant analysis for supervised dimensionality reduction, Proceedings of the IEEE International Conference on Machine Learning, Pittsburgh, PA, USA, pp. 905-912.

[19]

Sun, Q., Zeng, S., Liu, Y., Heng, P. and Xia, D. (2005). A new method of feature fusion and its application in image recognition, Pattern Recognition 38(12): 2437-2448.

Digital Library

[20]

Sun, T. and Chen, S. (2007). Class label versus sample label-based CCA, Applied Mathematics and Computation 185(1): 272-283.

[21]

Sun, T., Chen, S., Yang, J. and Shi, P. (2008). A supervised combined feature extraction method for recognition, Procedings of the IEEE International Conference on Data Mining, Pisa, Italy, pp. 1043-1048.

[22]

Tenenbaum, J.B., de Silva, V. and Langford, J.C. (2000). A global geometric framework for nonlinear dimensionality reduction, Science 290(5500): 2319-2323.

[23]

Wegelin, J. (2000). A survey of partial least squares (PLS) methods, with emphasis on the two block case, Technical report, Department of Statistics, University of Washington, Seattle, WA.

[24]

Yan, S., Xu, D., Zhang, B., Zhang, H.-J., Yang, Q. and Lin, S. (2007). Graph embedding and extensions: A general framework for dimensionlity reduction, IEEE Transactions on Pattern Analysis and Machine Intelligence 29(1): 40- 51.

Digital Library

[25]

Yang, J. and Yang, J.-Y. (2003). Why can LDA be performed in PCA transformed space?, Pattern Recognition 36(2): 563- 566.

[26]

Yang, J., Yang, J., Zhang, D. and Lu, J. (2003). Feature fusion: Parallel strategy vs. serial strategy, Pattern Recognition 36(6): 1369-1381.

[27]

Ye, J. (2005). Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems, Journal of Machine Learning Research 6(4): 483-502.

Digital Library

[28]

Yu, H. and Yang, J. (2001). A direct LDA algorithm for highdimensional data-with application to face recognition, Pattern Recognition 34(10): 2067-2070.

Cited By

Pujol FMora HGirona-Selva J(2016)A connectionist computational method for face recognitionInternational Journal of Applied Mathematics and Computer Science10.5555/3060518.306052126:2(451-465)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.5555/3060518.3060521
Karbauskaitź RDzemyda G(2015)Optimization Of The Maximum Likelihood Estimator For Determining The Intrinsic Dimensionality Of High-Dimensional DataInternational Journal of Applied Mathematics and Computer Science10.5555/3060538.306054725:4(895-913)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.5555/3060538.3060547
Goel NSingh K(2013)A modified convolution and product theorem for the linear canonical transform derived by representation transformation in quantum mechanicsInternational Journal of Applied Mathematics and Computer Science10.2478/amcs-2013-005123:3(685-695)Online publication date: 1-Sep-2013
https://dl.acm.org/doi/10.2478/amcs-2013-0051
Show More Cited By

Analysis of correlation based dimension reduction methods
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Multiview dimension reduction via Hessian multiset canonical correlations

We integrates Hessian regularization into multiset CCA for multiview dimension reduction.We provide the optimization of HesMCC algorithm.We conduct extensive experiments to verify the proposed HesMCC and the results are promising. Canonical correlation ...
Max-K-Min Distance Analysis for Dimension Reduction
ICPR '14: Proceedings of the 2014 22nd International Conference on Pattern Recognition

We propose a new criterion for discriminative dimension reduction, Max-K-Min Distance Analysis (MKMDA). Given a data set with C classes, MKMDA maximizes the sum of the K minimum pair wise distance of these C classes on the selected one-dimensional ...
Core-Sets For Canonical Correlation Analysis
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

Canonical Correlation Analysis (CCA) is a technique that finds how "similar" are the subspaces that are spanned by the columns of two different matrices A έℜ(of size ^m-x-n) and B έℜ(of size ^m-x-l). CCA measures similarity by means of the cosines of the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal of Applied Mathematics and Computer Science

International Journal of Applied Mathematics and Computer Science Volume 21, Issue 3

9 2011

151 pages

EISSN:2083-8492

Issue’s Table of Contents

Publisher

Walter de Gruyter & Co.

United States

Publication History

Published: 01 September 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pujol FMora HGirona-Selva J(2016)A connectionist computational method for face recognitionInternational Journal of Applied Mathematics and Computer Science10.5555/3060518.306052126:2(451-465)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.5555/3060518.3060521
Karbauskaitź RDzemyda G(2015)Optimization Of The Maximum Likelihood Estimator For Determining The Intrinsic Dimensionality Of High-Dimensional DataInternational Journal of Applied Mathematics and Computer Science10.5555/3060538.306054725:4(895-913)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.5555/3060538.3060547
Goel NSingh K(2013)A modified convolution and product theorem for the linear canonical transform derived by representation transformation in quantum mechanicsInternational Journal of Applied Mathematics and Computer Science10.2478/amcs-2013-005123:3(685-695)Online publication date: 1-Sep-2013
https://dl.acm.org/doi/10.2478/amcs-2013-0051
Górecki TŁuczak M(2013)Linear discriminant analysis with a generalization of the Moore-Penrose pseudoinverseInternational Journal of Applied Mathematics and Computer Science10.2478/amcs-2013-003523:2(463-471)Online publication date: 1-Jun-2013
https://dl.acm.org/doi/10.2478/amcs-2013-0035

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents