Abstract
Principal component analysis (PCA) is fundamental in many pattern recognition applications. Much research has been performed to minimize the reconstruction error in L1-norm based reconstruction error minimization (L1-PCA-REM) since conventional L2-norm based PCA (L2-PCA) is sensitive to outliers. Recently, the variance maximization formulation of PCA with L1-norm (L1-PCA-VM) has been proposed, where new greedy and nongreedy solutions are developed. Armed with the gradient ascent perspective for optimization, we show that the L1-PCA-VM formulation is problematic in learning principal components and that only a greedy solution can achieve robustness motivation, which are verified by experiments on synthetic and real-world datasets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bishop, C.M., 2006. Pattern Recognition and Machine Learning. Springer.
de la Torre, F., Black, M.J., 2001. Robust Principal Component Analysis for Computer Vision. Proc. 8th IEEE Int. Conf. on Computer Vision, 1:362–369.
Ding, C., Zhou, D., He, X.F., Zha, H.Y., 2006. R1-PCA: Rotational Invariant L1-Norm Principal Component Analysis for Robust Subspace Factorization. Proc. 23rd Int. Conf. on Machine Learning, p.281–288. [doi:10.1145/1143844.1143880]
Duda, R.O., Hart, P.E., Stork, D.G., 2001. Pattern Classification. Wiley-Interscience.
Hastie, T., Tibshirani, R., Friedman, J., Franklin, J., 2005. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer.
Hyvärinen, A., Hurri, J., Hoyer, P.O., 2009. Natural Image Statistics. Springer.
Ke, Q.F., Kanade, T., 2003. Robust Subspace Computation Using L1 Norm. Technical Report, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.
Ke, Q.F., Kanade, T., 2005. Robust L1 Norm Factorization in the Presence of Outliers and Missing Data by Alternative Convex Programming. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.592–599.
Kwak, N., 2008. Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell., 30(9):1672–1680. [doi:10.1109/TPAMI.2008.114]
Le, Q.V., Karpenko, A., Ngiam, J., Ng, A.Y., 2011. ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning. Advances in Neural Information Processing Systems 24, p.1017–1025.
Liu, J., Ji, S., Ye, J., 2009. Multi-task Feature Learning via Efficient L2,1-Norm Minimization. Proc. 25th Conf. on Uncertainty in Artificial Intelligence, p.339–348.
Nakajima, S., Sugiyama, M., Babacan, D., 2011. On Bayesian PCA: Automatic Dimensionality Selection and Analytic Solution. Proc. 28th Int. Conf. on Machine Learning, p.497–504.
Nie, F., Huang, H., Cai, X., Ding, C., 2010. Efficient and Robust Feature Selection via Joint L2,1-Norms Minimization. Advances in Neural Information Processing Systems 23, p.1813–1821.
Nie, F., Huang, H., Ding, C., Luo, D., Wang, H., 2011. Robust Principal Component Analysis with Non-greedy L1-Norm Maximization. Proc. 22nd Int. Joint Conf. on Artificial Intelligence, p.1433–1438.
Wright, J., Ganesh, A., Rao, S., Peng, Y.G., Ma, Y., 2009. Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization. Advances in Neural Information Processing Systems 22, p.2080–2088.
Zass, R., Shashua, A., 2007. Nonnegative Sparse PCA. Advances in Neural Information Processing Systems 19, p.1561–1568.
Zhang, Y., Teng, Y., 2010. Adaptive multiblock kernel principal component analysis for monitoring complex industrial processes. J. Zhejiang Univ.-Sci. C (Comput. & Electron.), 11(12):948–955. [doi:10.1631/jzus.C1000148]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the National Natural Science Foundation of China (Nos. 61071131 and 61271388), the Beijing Natural Science Foundation (No. 4122040), the Research Project of Tsinghua University (No. 2012Z01011), and the United Technologies Research Center (UTRC)
Rights and permissions
About this article
Cite this article
Feng, Dc., Chen, F. & Xu, Wl. Learning robust principal components from L1-normmaximization. J. Zhejiang Univ. - Sci. C 13, 901–908 (2012). https://doi.org/10.1631/jzus.C1200180
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C1200180