Abstract
In the design of support vector machines an important step is to select the optimal hyperparameters. One of the most used estimators of the performance is the Radius-Margin bound. Some modifications of this bound have been made to adapt it to soft margin problems, giving a convex optimization problem for the L2 soft margin formulation. However, it is still interesting to consider the L1 case due to the reduction in the support vector number. There have been some proposals to adapt the Radius-Margin bound to the L1 case, but the use of gradient descent to test them is not possible in some of them because these bounds are not differentiable. In this work we propose to use simulated annealing as a method to find the optimal hyperparameters when the bounds are not differentiable, have multiple local minima or the kernel is not differentiable with respect to its hyperparameters.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vapnik, N.V.: The Natureof Statistical Learning Theory (1 edn. 1998). Springer, Heidelberg (2000)
Joachims, T.: Estimating the generalization performance of a SVM efficiently. In: Langley, P. (ed.) Proc. of ICML-00, pp. 431–438. Morgan Kaufmann, San Francisco (2000), citeseer.ist.psu.edu/article/joachims99estimating.html
Vapnik, V., Chapelle, O.: Bounds on error expectation for support vector machines. Neural Computation 12(9), 2013–2036 (2000)
Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing multiple parameters for support vector machines. Machine Learning 46(1), 131–159 (2002)
Duan, K., Sathiya, S., Poo, A.: Evaluation of simple performance measures for tuning the svm hyperparameters. Neurocomputing 51, 41–59 (2003)
Chung, K.-M., Kao, W.-C., Sun, C.-L., Wang, L.-L., Lin, C.-J.: Radius margin bounds for support vector machines with the rbf kernel. Neural Computation 15, 2643–2681 (2003)
Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)
Kirkpatrick, S., Gellat, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, New York (2004)
Ingber, A.L.: Adaptive simulated annealing (asa): Lessons learned. Control and Cybernetics 25(1), 33–54 (1996)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Acevedo, J., Maldonado, S., Siegmann, P., Lafuente, S., Gil, P. (2007). Tuning L1-SVM Hyperparameters with Modified Radius Margin Bounds and Simulated Annealing. In: Sandoval, F., Prieto, A., Cabestany, J., Graña, M. (eds) Computational and Ambient Intelligence. IWANN 2007. Lecture Notes in Computer Science, vol 4507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73007-1_35
Download citation
DOI: https://doi.org/10.1007/978-3-540-73007-1_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73006-4
Online ISBN: 978-3-540-73007-1
eBook Packages: Computer ScienceComputer Science (R0)