Abstract
Support Vector Machines constitute a Machine Learning technique originally designed for the solution of two-class problems. This paper investigates and proposes strategies for the generalization of SVMs to problems with more than two classes. The focus of this work is on strategies that decompose the original multiclass problem into binary subtasks, whose outputs are combined. The proposed strategies aim to investigate the adaptation of the decompositions for each multiclass application considered, using information of the performance obtained in its solution or extracted from its examples. The implemented algorithms were evaluated using benchmark datasets and real applications from the Bioinformatics domain. Among the benefits observed is the obtainment of simpler decompositions, which require less binary classifiers in the multiclass solution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Allwein, E.L., Shapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. In: Proc 17th Int Conf on Machine Learning, pp. 9–16 (2000)
Aluha, R.K., Magnanti, T.L., Orlin, J.B.: Network flows: theory, algorithms and applications. Prentice Hall, Englewood Cliffs (1993)
Asuncion, A., Newman, D.J.: UCI repository of machine learning databases, http://www.ics.uci.edu/~mlearn/MLRepository.html
Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. Machine Learning 47(2-3), 201–233 (2002)
Cristianini, N., Taylor, J.S.: An introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Dietterich, T.G., Bariki, G.: Solving multiclass learning problems via error-correcting output codes. J. Artificial Intelligence Research 2, 263–286 (1995)
Feelders, A., Verkooijen, W.: On the statistical comparison of inductive learning methods. In: Fisher, D., Lenz, H.-J. (eds.) Learning from data: artificial intelligence and statistics V, pp. 272–279. Springer, Heidelberg (1996)
Hsu, C.-W., Lin, C.-J.: A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 13, 415–425 (2002)
Kijsirikul, B., Ussivakul, N.: Multiclass Support Vector Machines using adaptive directed acyclic graph. In: Proc. Int. Joint Conf. on Neural Networks, pp. 980–985 (2002)
Kreβel, U.: Pairwise classification and Support Vector Machines. In: Advances in Kernel Methods - Support Vector Learning, pp. 185–208 (1999)
Lorena, A.C., Carvalho, A.C.P.L.F.: Minimum spanning trees in hierarchical multiclass Support Vector Machines generation. In: Ali, M., Esposito, F. (eds.) IEA/AIE 2005. LNCS (LNAI), vol. 3533, pp. 422–431. Springer, Heidelberg (2005)
Lorena, A.C., Carvalho, A.C.P.L.F.: Protein cellular localization prediction with multiclass Support Vector Machines and Decision Trees. Computers in Biology and Medicine 37, 115–125 (2007)
Lorena, A.C., Carvalho, A.C.P.L.F.: Multiclass SVM design and parameter selection with genetic algorithms. In: IEEE Digital Proc. IX Brazilian Symp. on Neural Networks (2006)
Lorena, A.C., Carvalho, A.C.P.L.F.: Evolutionary design of multiclass support vector machines. J. Intelligent and Fuzzy Systems 18, 445–454 (2007)
Lorena, A.C., Carvalho, A.C.P.L.F.: Design of Directed Acyclic Graph Multiclass Structures. Neural Network World 17, 657–674 (2007)
Mitchell, T.: Machine learning. McGraw Hill, New York (1997)
Mitchell, M.: An introduction to Genetic Algorithms. MIT Press, Cambridge (1999)
Platt, J.C., Cristianini, N., Shawe-Taylor, J.: Large margin DAGs for multiclass classification. In: Advances in Neural Information Processing Systems 12, 547–553 (2000)
Quilan, J.R.: Induction of Decision Trees. Machine Learning 1(1), 81–106 (1986)
Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Machine Learning Research 5, 1533–7928 (2004)
Schwenker, F.: Hierarquical support vector machines for multi-class pattern recognition. In: Proc. 4th Int. Conf. on Knowledge-Based Intelligent Systems and Allied Technologies, pp. 561–565 (2000)
Takahashi, F., Abe, S.: Decision-tree-based multiclass support vector machines. In: Proc. 9th Int. Conf. on Neural Information Processing, vol. 3, pp. 1418–1422 (2002)
Vural, V., Dy, J.G.: A hierarchical method for multi-class Support Vector Machines. In: Proc. 21st Int. Conf. on Machine Learning, pp. 831–838 (2004)
Weston, J., Watkins, V.: Multi-class Support Vector Machines. Tech Rep CSD-TR-98-04, Dep. Computer Science, University of London (1998)
Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: improving the strength pareto evolutionary algorithm. In: Evolutionary methods for design, optimization, and control, pp. 95–100 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Lorena, A.C., de Carvalho, A.C.P.L.F. (2008). Investigation of Strategies for the Generation of Multiclass Support Vector Machines. In: Nguyen, N.T., Katarzyniak, R. (eds) New Challenges in Applied Intelligence Technologies. Studies in Computational Intelligence, vol 134. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79355-7_31
Download citation
DOI: https://doi.org/10.1007/978-3-540-79355-7_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-79354-0
Online ISBN: 978-3-540-79355-7
eBook Packages: EngineeringEngineering (R0)