Cited By
View all- (2018)Improving blog spam filters via machine learningInternational Journal of Data Analysis Techniques and Strategies10.1504/IJDATS.2017.0859019:2(99-121)Online publication date: 14-Dec-2018
Rotation-based model trees for classification
Pages 22 - 37
Abstract
Structurally, a model tree is a regression method that takes the form of a decision tree with linear regression functions instead of terminal class values at its leaves. In this study, model trees were coupled with a rotation-based ensemble for solving classification problems. In order to apply this regression technique to classification problems, we considered the conditional class probability function and sought a model-tree approximation to it. During classification, the class whose model tree generated the greatest approximated probability value was chosen as the predicted class. We performed a comparison with other well-known ensembles of decision trees on standard benchmark data sets, and the performance of the proposed technique was greater in most cases.
References
[1]
Asuncion, A. and Newman, D.J. (2007) UCI Machine Learning Repository, Irvine, CA: School of Information and Computer Science, University of California, http://www.ics.uci.edu/~mlearn/ MLRepository.html.
[2]
Bauer, E. and Kohavi, R. (1999) 'An empirical comparison of voting classification algorithms: bagging, boosting, and variants', Machine Learning, Vol. 36, pp.105-139.
[3]
Breiman, L. (1996) 'Bagging predictors', Machine Learning, Vol. 24, pp.123-140.
[4]
Breiman, L. (2001) 'Random forests', Machine Learning, Vol. 45, pp.5-32.
[5]
Cabrera, J.B.D. (2006) 'On the impact of fusion strategies on classification errors for large ensembles of classifiers', Pattern Recognition (PR), Vol. 39, No. 11, pp.1963-1978.
[6]
Cohen, W. and Singer, Y. (1999) 'A simple, fast, and effective rule learner', AAAI/IAAI, pp.335-342.
[7]
Frank, E., Wang, Y., Inglis, S., Holmes, G. and Witten, I. (1998) 'Using model trees for classification', Machine Learning, Vol. 32, pp.63-76.
[8]
Hernández-Lobato, D., Martínez-Muñoz, G. and Suárez, A. (2009) 'Statistical instance-based pruning in ensembles of independent classifiers', IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 31, No. 2, pp.364-369.
[9]
Ho, T. (1998) 'The random subspace method for constructing decision forests', IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No. 8, pp.832-844.
[10]
Holmes, G., Pfahringer, B., Kirkby, R., Frank, E. and Hall, M. (2001) 'Multiclass alternating decision trees', ECML, pp.161-172.
[11]
Keerthi, S. and Shevade, S.K. (2003) 'SMO algorithm for least-squares SVM formulation', Neural Computation (NECO), Vol. 15, No. 2, pp.487-507.
[12]
Kotsiantis, S., Pierrakeas, C. and Pintelas, P. (2004) 'Predicting students' performance in distance learning using machine learning techniques', Applied Artificial Intelligence (AAI), Vol. 18, pp.411-426.
[13]
Landwehr, N., Hall, M. and Frank, E. (2005) 'Logistic model trees', Machine Learning, Vol. 59, Nos. 1-2, pp.161-205.
[14]
Luo, R. and Wang, H. (2008) 'A composite logistic regression approach for ordinal panel data regression', Int. J. Data Analysis Techniques and Strategies, Vol. 1, No. 1.
[15]
Malerba, D., Esposito, F., Ceci, M. and Appice, A. (2004) 'Top-down induction of model trees with regression and splitting nodes', IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 5, pp.612-625.
[16]
Melville, P. and Mooney, R. (2005) 'Creating diversity in ensembles using artificial data', Information Fusion, Vol. 6, No. 1, pp.99-111.
[17]
Nadeau, C. and Bengio, Y. (2003) 'Inference for the generalization error', Machine Learning, Vol. 52, pp.239-281.
[18]
Opitz, D. and Maclin, R. (1999) 'Popular ensemble methods: an empirical study', Artificial Intelligence Research, Vol. 11, pp.169-198.
[19]
Quinlan, J.R. (1992) 'Learning with continuous classes', Proc. of Australian Joint Conf. on AI, Singapore: World Scientific, pp.343-348.
[20]
Quinlan, J.R. (1996) 'Bagging, boosting, and C4.5', Proceedings of the Thirteenth National Conference on Artificial Intelligence, AAAI/MIT Press, pp.725-730.
[21]
Renooij, S. and van der Gaag, L.C. (2008) 'Evidence and scenario sensitivities in naive Bayesian classifiers', Int. J. Approx. Reasoning (IJAR), Vol. 49, No. 2, pp.398-416.
[22]
Rodriguez, J., Kuncheva, L. and Alonso, C. (2006) 'Rotation forest: a new classifier ensemble method', IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, No. 10, pp.1619-1630.
[23]
Rudin, C., Daubechies, I. and Schapire, R. (2004) 'The dynamics of AdaBoost: cyclic behavior and convergence of margins', Journal of Machine Learning Research, Vol. 5, pp.1557-1595.
[24]
Ting, K.M. and Witten, I.H. (1997) 'Stacking bagged and dagged models', Fourteenth International Conference on Machine Learning, San Francisco, California, pp.367-375.
[25]
Wang, Y. and Witten, I.H. (1997) 'Induction of model trees for predicting continuous classes', Proc. of the Poster Papers of the European Conference on ML, Prague: Faculty of Informatics and Statistics, University of Economics, pp.128-137.
[26]
Webb, G.I. (2000) 'MultiBoosting: a technique for combining boosting and wagging', Machine Learning, Vol. 40, pp.159-196.
[27]
Witten, I. and Frank, E. (2005) Data Mining: Practical Machine Learning Tools and Techniques, 2nd ed., San Francisco: Morgan Kaufmann.
- Rotation-based model trees for classification
Recommendations
Disturbing Neighbors Ensembles of Trees for Imbalanced Data
ICMLA '12: Proceedings of the 2012 11th International Conference on Machine Learning and Applications - Volume 02Disturbing Neighbors (DN) is a method for generating classifier ensembles. Moreover, it can be combined with any other ensemble method, generally improving the results. This paper considers the application of these ensembles to imbalanced data: ...
Forests of nested dichotomies
Ensemble methods are often able to generate more accurate classifiers than the individual classifiers. In multiclass problems, it is possible to obtain an ensemble combining binary classifiers. It is sensible to use a multiclass method for constructing ...
Using model trees and their ensembles for imbalanced data
CAEPIA'11: Proceedings of the 14th international conference on Advances in artificial intelligence: spanish association for artificial intelligenceModel trees are decision trees with linear regression functions at the leaves. Although originally proposed for regression, they have also been applied successfully in classification problems. This paper studies their performance for imbalanced ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Publisher
Inderscience Publishers
Geneva 15, Switzerland
Publication History
Published: 01 December 2010
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Jan 2025
Other Metrics
Citations
Cited By
View all- (2018)Improving blog spam filters via machine learningInternational Journal of Data Analysis Techniques and Strategies10.1504/IJDATS.2017.0859019:2(99-121)Online publication date: 14-Dec-2018