Semi-Supervised Sequential Kernel Regression Models with Penalty Functions

Hengjin Tang, Sadaaki Miyamoto, and Yasunori Endo

Department of Risk Engineering, School of Systems and Information Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan

Received:

April 20, 2014

Accepted:

August 25, 2014

Published:

January 20, 2015

Keywords:

kernel regression, switching regression models, semi-supervised clustering, sequential clustering, penalty functions

Abstract

Switching regression models can output multiple clusters and regression models. However, there is one problem: the results have a strong dependency on the predefined number of clusters. To avoid these drawbacks, we have researched sequential extractions. In sequential extractions process, one cluster is extracted at a time using a method of noise-detection, and the number of clusters are determined automatically. We propose semi-supervised sequential kernel regression models with penalty functions. Additionally, we also find that the sensitivity against the regularization parameter λ can be alleviated by semi-supervisions using penalty functions. We show the effectiveness of the proposed method by using numerical examples.

Cite this article as:

H. Tang, S. Miyamoto, and Y. Endo, “Semi-Supervised Sequential Kernel Regression Models with Penalty Functions,” J. Adv. Comput. Intell. Intell. Inform., Vol.19 No.1, pp. 51-57, 2015.

Data files:

References

[1] R. E. Quandt, “A New Approach to Estimating Switching Regressions,” J. of the American Statistical Association, Vol.67, pp. 306-310, 1972.
[2] S. M. Goldfeld and R. E. Quandt, “Techniques for Estimating Switching Regressions,” in Studies in Nonlinear Estimation (eds. by S. M. Goldfeld and R. E. Quandt), pp. 3-35, Ballinger, Cambridge, Massachusetts, 1976.
[3] B. Mirkin, “The Iterative Extraction Approach to Clustering,” Lecture Notes in Computational Science and Engineering, 58, pp. 153-179, Springer, New York, 2007.
[4] S. Theodoridis and K. Koutroumbas, “Pattern Recognition (4th Ed.),” Academic Press, Massachusetts, 2008.
[5] S. Miyamoto, Y. Kuroda, and K. Arai, “Algorithms for Sequential Extraction of Clusters by Possibilistic Method and Comparison with Mountain Clustering,” J. of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.5, pp. 448-453, 2008.
[6] R. N. Davé and R. Krishnapuram, “Robust clustering methods: a unified view,” IEEE Trans. on Fuzzy Systems, Vol.5, No.2, pp. 270-293, 1997.
[7] R. N. Davé and S. Sen, “On Generalizing the Noise Clustering Algorithms,” Proc. of the 7th IFSA World Congress, Vol.3, pp. 205-210, 1997.
[8] H. Tang and S. Miyamoto, “Semi-supervised Sequential Kernel Regression Models with Pairwise Constraints,” Lecture Notes in Artificial Intelligence, 8234, pp. 166-178, 2013.
[9] M. Girolami, “Mercer Kernel-Based Clustering in Feature Space,” IEEE Trans. on Neutral Networks, Vol.13, No.3, pp. 780-784, 2002.
[10] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.
[11] Kernel-Machines.Org, address:
http://kernel-machines.org [Accessed March 15, 2014]
[12] K. Wagstaff, C. Cardie, S. Rogers, and S. Schröedl, “Constrained K-means Clustering with Background Knowledge,” Proc. of the 8th Int. Conf. on Machine Learning, pp. 577-584, 2001.
[13] S. Basu,M. Bilenko, and R. J. Mooney, “A Probabilistic Framework for Semi-Supervised Clustering,” Proc. of the 10th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 59-68, 2004.
[14] H. Tang and S. Miyamoto, “Sequential Extraction of Fuzzy Regression Models: Least Squares and Least Absolute Deviations,” Int. J. of Uncertainty, Fuzziness and Knowledge-Based Systems, Vol.19, Suppl.1, pp. 53-63, 2011.
[15] H. Tang and S. Miyamoto, “Sequential Regression Models with Pairwise Constraints Using Noise Clusters,” J. of Advanced Computational Intelligence and Intelligent Informatics (JACIII), Vol.16, No.7, pp. 814-818, 2012.
[16] O. Chapelle, B. Schölkopf, and A. Zien, “Semi-Supervised Learning,” The MIT Press, Cambridge, Massachusttes, 2006.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.