Abstract
We consider strategies for building classifier ensembles for non-stationary environments where the classification task changes during the operation of the ensemble. Individual classifier models capable of online learning are reviewed. The concept of ”forgetting” is discussed. Online ensembles and strategies suitable for changing environments are summarized.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aha, D., Kibler, D., Albert, M.K.: Instance-based learning algorithms. Machine Learning 6, 37–66 (1991)
Bezdek, J.C., Kuncheva, L.I.: Nearest prototype classifier designs: An experimental study. International Journal of Intelligent Systems 16(12), 1445–1473 (2001)
Blum, A.: Empirical support for Winnow and weighted-majority based algorithms: results on a calendar scheduling domain. In: Proc. 12th International Conference on Machine Learning, pp. 64–72. Morgan Kaufmann, San Francisco (1995)
Breiman, L.: Pasting small votes for classification in large databases and on-line. Machine Learning 36, 85–103 (1999)
Dasarathy, B.V.: Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1990)
Domingos, P., Hulten, G.: Mining high-speed data streams. In: Knowledge Discovery and Data Mining, pp. 71–80 (2000)
Domingos, P., Hulten, G.: A general framework for mining massive data streams. Journal of Computational and Graphical Statistics 12, 945–949 (2003)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)
Ganti, V., Gehrke, J., Ramakrishnan, R.: Mining data streams under block evolution. ACM SIGKDD Explorations Newsletter 3, 1–10 (2002)
Hart, P.E.: The condensed nearest neighbor rule. IEEE Transactions on Information Theory 16, 515–516 (1968)
Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proc. 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 97–106. ACM Press, New York (2001)
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3, 79–87 (1991)
Kelly, M.G., Hand, D.J., Adams, N.M.: The impact of changing populations on classifier performance. In: Proc. 5th ACM SIGDD International Conference on Knowledge Discovery and Data Mining, pp. 367–371. ACM Press, San Diego (1999)
Kohonen, T.: Self-Organizing Maps, 3rd edn. Springer, New York (2000)
Lim, C.P., Harrison, R.E.: Online pattern classification with multiple neural network systems: An experimental study. IEEE Transactions on Systems, Man, and Cybernetics 35, 235–247 (2003)
Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear threshold algorithm. Machine Learning 2, 285–318 (1988)
Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Inform. Computation 108, 212–261 (1994)
Markou, M., Singh, S.: Novelty detection: a review. Part 1: statistical approaches. Signal Processing 83(12), 2481–2497 (2003)
Oza, N.C.: Online Ensemble Learning. PhD thesis, University of California, Berkeley (2001)
Salganicoff, M.: Density-adaptive learning and forgetting. In: Proceedings of the 10th International Conference on Machine Learning, pp. 276–283 (1993)
Street, W.N., Kim, Y.S.: A streaming ensemble algorithm (SEA) for large-scale classification. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 377–382. ACM Press, New York (2001)
Wang, H., Fan, W., Yu, P.S., Han, J.: Mining concept drifting data streams using ensemble classifiers. In: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 226–235. ACM Press, New York (2003)
Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23, 69–101 (1996)
Widmer, G., Kubat, M.: Special Issue on Context Sensitivity and Concept Drift. Machine Learning 32 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kuncheva, L.I. (2004). Classifier Ensembles for Changing Environments. In: Roli, F., Kittler, J., Windeatt, T. (eds) Multiple Classifier Systems. MCS 2004. Lecture Notes in Computer Science, vol 3077. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-25966-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-25966-4_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22144-9
Online ISBN: 978-3-540-25966-4
eBook Packages: Springer Book Archive