Abstract
Kernel-based learning presents a unified approach to machine learning problems such as classification and regression. The selection of a kernel and associated parameters is a critical step in the application of any kernel-based method to a problem. This paper presents a data-driven evolutionary approach for constructing kernels, named KTree. An application of KTree to the Support Vector Machine (SVM) classifier is described. Experiments on a synthetic dataset are used to determine the best evolutionary strategy, e.g. what fitness function to use for kernel evaluation. The performance of an SVM based on KTree is compared with that of standard kernel SVMs on a synthetic dataset and on a number of real-world datasets. KTree is shown to outperform or match the best performance of all the standard kernels tested.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. Journal of Machine Learning Research 2 (2002)
Howley, T., Madden, M.G.: The Genetic Kernel Support Vector Machine: Description and Evaluation. Artificial Intelligence Review 24 (2005)
Scholkopf, B.: Statistical Learning and Kernel Methods. Technical Report MSR-TR-2000- 23, Microsoft Research, Microsoft Corporation (2000)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Bahlmann, C., Haasdonk, B., Burkhardt, H.: On-line Handwriting Recognition with Support Vector Machines - A Kernel Approach. In: Proc. of the 8th Intl. Workshop on Frontiers in Handwriting Recognition (2002)
Lin, H., Lin, C.: A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods. Technical report, Dept. of Computer Science and Information Engineering, National Taiwan University (2003)
Cristianini, N., Shawe-Taylor, J.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Mangasarian, O., Musicant, D.: Lagrangian Support Vector Machines. Journal of Machine Learning Research 1 (2001)
Luke, S., Spector, L.: A Comparison of Crossover and Mutation in Genetic Programming. In: Genetic Programming: Proc. of the 2nd Annual Conference. Morgan Kaufmann, San Francisco (1997)
Newman, D., Hettich, S., Blake, C., Merz, C.: UCI Repository of machine learning databases (1998)
Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics 1, 80–83 (1945)
Frolich, H., Chapelle, O., Scholkopf, B.: Feature Selection for SVMs by Means of Genetic Algorithms. In: Proc. of the Intl. IEEE Conference on Tools with AI, pp. 142–148 (2003)
Runarsson, T., Sigurdsson, S.: Asynchronous Parallel Evolutionary Model Selection for Support Vector Machines. Neural Information Processing - Letters and Reviews 3 (2004)
Friedrichs, F., Igel, C.: Evolutionary Tuning of Multiple SVM Parameters. In: Proc. of the 12th European Symposium on Artificial Neural Network, pp. 519–524 (2004)
Lessmann, S., Stahlbock, R., Crone, S.: Genetically constructed kernels for support vector machines. In: Proc. of German Operations Research, GOR (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Howley, T., Madden, M.G. (2006). An Evolutionary Approach to Automatic Kernel Construction. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds) Artificial Neural Networks – ICANN 2006. ICANN 2006. Lecture Notes in Computer Science, vol 4132. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840930_43
Download citation
DOI: https://doi.org/10.1007/11840930_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38871-5
Online ISBN: 978-3-540-38873-9
eBook Packages: Computer ScienceComputer Science (R0)