Abstract
Discussions about the trade-off between accuracy and diversity when designing Multiple Classifier Systems is an active topic in Machine Learning. One possible way of considering the design of Multiple Classifier Systems is to select the ensemble members from a large pool of classifiers focusing on predefined criteria, which is known as the Overproduce and Choose paradigm. In this paper, a genetic algorithm is proposed to design Multiple Classifier Systems under this paradigm while controlling the trade-off between accuracy and diversity of the ensemble members. The proposed algorithm is compared with several classifier selection methods from the literature on different UCI Repository datasets. This paper specifies several conditions for which it is worth using diversity during the design stage of Multiple Classifier Systems.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2001)
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)
Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)
Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: a survey and categorisation. Information Fusion 6, 5–20 (2005)
Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)
Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)
Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting and randomization. Machine Learning 40, 139–157 (2000)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, Chichester (2001)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156 (1996)
Gacquer, D., Delmotte, F., Delcroix, V., Piechowiak, S.: A genetic approach for training diverse classifier ensembles. In: Proceedings of the 12th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU 2008), Malaga, Spain, pp. 798–805 (2008)
Hagan, M.T., Demuth, H.B., Beale, M.: Neural Network Design. PWS Publishing, Boston (1996)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 832–844 (1998)
Kuncheva, L.I.: That elusive diversity in classifier ensembles. In: Proceedings of the 1st Iberian Conference on Pattern Recognition and Image Analysis, pp. 1126–1138 (2003)
Kuncheva, L.I., Whitaker, C.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51, 181–207 (2003)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley Interscience, Hoboken (2004)
Margineantu, D.D., Dietterich, T.G.: Pruning adaptive boosting. In: Proceedings 14th International Conference on Machine Learning, pp. 211–218 (1997)
Melville, P., Mooney, R.J.: Creating diversity in ensembles using artificial data. Information Fusion 6, 99–111 (2005)
Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)
Opitz, D., Maclin, R.: Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Research 11, 169–198 (1999)
Partridge, D., Yates, W.B.: Engineering multiversion neural-net systems. Neural Computation 8, 869–893 (1996)
Quinlan, J.R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)
Quinlan, J.R.: Bagging, boosting, and c4.5. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725–730 (1996)
Rodriguez, J.J., Kuncheva, L.I., Alonso, C.J.: Rotation forest: A new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 1619–1630 (2006)
Roli, F., Giacinto, G., Vernazza, G.: Methods for designing multiple classifier systems. In: Proceedings of the 2nd International Workshop on Multiple Classifier Systems, Cambridge, UK, pp. 78–87 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gacquer, D., Delcroix, V., Delmotte, F., Piechowiak, S. (2009). On the Effectiveness of Diversity When Training Multiple Classifier Systems. In: Sossai, C., Chemello, G. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2009. Lecture Notes in Computer Science(), vol 5590. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02906-6_43
Download citation
DOI: https://doi.org/10.1007/978-3-642-02906-6_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02905-9
Online ISBN: 978-3-642-02906-6
eBook Packages: Computer ScienceComputer Science (R0)