Abstract
Nonnegative Matrix Factorization (NMF) has recently received much attention both in an algorithmic aspect as well as in applications. Text document clustering and supervised classification are important applications of NMF. Various types of numerical optimization algorithms have been proposed for NMF, which includes multiplicative, projected gradient descent, alternating least squares and active-set ones. In this paper, we discuss the selected Non-Negatively constrained Least Squares (NNLS) algorithms (a family of the NNLS algorithm proposed by Lawson and Hanson) that belong to a class of active-set methods. We noticed that applying the NNLS algorithm to the Tikhonov regularized LS objective function with a regularization parameter exponentially decreasing considerably increases the accuracy of data clustering as well as it reduces the risk of getting stuck into unfavorable local minima. Moreover, the experiments demonstrate that the regularized NNLS algorithm is superior to many well-known NMF algorithms used for text document clustering.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Benthem, M.H.V., Keenan, M.R.: J. Chemometr. 18, 441–450 (2004)
Berry, M., Browne, M., Langville, A., Pauca, P., Plemmons, R.: Comput. Stat. Data An. 52, 155–173 (2007)
Bro, R., Jong, S.D.: J. Chemometr. 11, 393–401 (1997)
Buciu, I., Pitas, I.: Application of non-negative and local nonnegative matrix factorization to facial expression recognition. In: Proc. Intl. Conf. Pattern Recognition (ICPR), pp. 288–291 (2004)
Cai, D., He, X., Wu, X., Bao, H., Han, J.: Locality preserving nonnegative matrix factorization. In: Proc. IJCAI 2009, pp. 1010–1015 (2009)
Cai, D., He, X., Wu, X., Han, J.: Nonnegative matrix factorization on manifold. In: Proc. 8th IEEE Intl. Conf. Data Mining (ICDM), pp. 63–72 (2008)
Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons, Chichester (2009)
Ding, C., Li, T., Jordan, M.I.: IEEE T. Pattern. Anal. 32, 45–55 (2010)
Ding, C., Li, T., Peng, W.: Nonnegative matrix factorization and probabilistic latent semantic indexing: Equivalence, chi-square statistic, and a hybrid method. In: Proc. AAAI National Conf. Artificial Intelligence (AAAI 2006) (2006)
Ding, C., Li, T., Peng, W., Park, H.: Orthogonal nonnegative matrix tri-factorizations for clustering. In: Proc 12th ACM SIGKDD Intl. Conf. Knowledge Discovery and Data Mining, pp. 126–135. ACM Press, New York (2006)
Du, Q., Kopriva, I.: Neurocomputing 72, 2682–2692 (2009)
Heiler, M., Schnoerr, C.: J. Mach. Learn. Res. 7, 1385–1407 (2006)
Jain, A.K., Murty, M.N., Flynn, P.J.: ACM Comput. Surv. 31, 264–323 (1999)
Jankowiak, M.: Application of nonnegative matrix factorization for text document classification. MSc thesis (supervised by Dr. R. Zdunek), Wroclaw University of Technology, Poland (2010) (in Polish)
Kim, H., Park, H.: Bioinformatics 23, 1495–1502 (2007)
Kim, H., Park, H.: SIAM J. Matrix Anal. A 30, 713–730 (2008)
Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. Prentice-Hall, Englewood Cliffs (1974)
Lee, D.D., Seung, H.S.: Nature 401, 788–791 (1999)
Li, T., Ding, C.: The relationships among various nonnegative matrix factorization methods for clustering. In: Proc. 6th Intl. Conf. Data Mining (ICDM 2006), pp. 362–371. IEEE Computer Society, Washington DC, USA (2006)
O’Grady, P., Pearlmutte, B.: Neurocomputing 72, 88–101 (2008)
Sajda, P., Du, S., Brown, T.R., Stoyanova, R., Shungu, D.C., Mao, X., Parra, L.C.: IEEE T. Med. Imaging 23, 1453–1465 (2004)
Shahnaz, F., Berry, M., Pauca, P., Plemmons, R.: Inform. Process. Manag. 42, 373–386 (2006)
Sra, S., Dhillon, I.S.: Nonnegative matrix approximation: Algorithms and Applications. UTCS Technical Report TR-06-27, Austin, USA (2006), http://www.cs.utexas.edu/ftp/pub/techreports/tr06-27.pdf
Xu, W., Liu, X., Gong, Y.: Document clustering based on non-negative matrix factorization. In: SIGIR 2003: Proc 26th Annual Intl ACM SIGIR Conf. Research and Development in Informaion Retrieval, pp. 267–273. ACM Press, New York (2003)
Zdunek, R., Cichocki, A.: Comput. Intel. Neurosci. (939567) (2008)
Zdunek, R., Phan, A.H., Cichocki, A.: Aust. J. Intel. Inform. Process. Syst. 12, 16–22 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zdunek, R. (2011). Regularized NNLS Algorithms for Nonnegative Matrix Factorization with Application to Text Document Clustering. In: Burduk, R., Kurzyński, M., Woźniak, M., Żołnierek, A. (eds) Computer Recognition Systems 4. Advances in Intelligent and Soft Computing, vol 95. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20320-6_77
Download citation
DOI: https://doi.org/10.1007/978-3-642-20320-6_77
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20319-0
Online ISBN: 978-3-642-20320-6
eBook Packages: EngineeringEngineering (R0)