Abstract
Most of regression learning methods aim to reduce various metrics of prediction errors. However, in many real-life applications it is prediction cost, which should be minimized as the under-prediction and over-prediction errors have different consequences. In this paper, we show how to extend the evolutionary algorithm (EA) for global induction of model trees to achieve a cost-sensitive learner. We propose a new fitness function which allows minimization of the average misprediction cost and two specialized memetic operators that search for cost-sensitive regression models in the tree leaves. Experimental validation was performed with bank loan charge-off forecasting data which has asymmetric costs. Results show that Global Model Trees with the proposed extensions are able to effectively induce cost-sensitive model trees with average misprediction cost significantly lower than in popular post-hoc tuning methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Barros, R.C., Ruiz, D.D., Basgalupp, M.: Evolutionary model trees for handling continuous classes in machine learning. Information Sciences 181, 954–971 (2011)
Barros, R.C., Basgalupp, M.P., Carvalho, A.C., Freitas, A.A.: A Survey of Evolutionary Algorithms for Decision-Tree Induction. IEEE Transactions on Systems Man and Cybernetics, Part C 42(3), 291–312 (2012)
Bansal, G., Sinha, A.P., Zhao, H.: Tuning data mining methods for cost-sensitive regression: a study in loan charge-off forecasting. Journal of Management Information Systems 25(3), 317–338 (2008)
Bradford, J., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.E.: Pruning decision trees with misclassification costs. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 131–136. Springer, Heidelberg (1998)
Cain, M., Janssen, C.: Real estate price prediction under asymmetric loss. Annals of the Institute of Statistical Mathematics 47(3), 401–414 (1995)
Crone, S.F., Lessmann, S., Stahlbock, R.: Utility based data mining for time series analysis: Cost-sensitive learning for neural network predictors. In: Proc. of 1st UDBM, Chicago, IL, pp. 59–68 (2005)
Czajkowski, M., Kretowski, M.: An Evolutionary Algorithm for Global Induction of Regression Trees with Multivariate Linear Models. In: Kryszkiewicz, M., Rybinski, H., Skowron, A., Raś, Z.W. (eds.) ISMIS 2011. LNCS, vol. 6804, pp. 230–239. Springer, Heidelberg (2011)
Czajkowski, M., Kretowski, M.: Does Memetic Approach Improve Global Induction of Regression and Model Trees? In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) SIDE 2012 and EC 2012. LNCS, vol. 7269, pp. 174–181. Springer, Heidelberg (2012)
Domingos, P.: MetaCost: A general method for making classifiers cost-sensitive. In: Proc. of KDD 1999, pp. 155–164. ACM Press (1999)
Elkan, C.: The Foundations of Cost-Sensitive Learning. In: Proc. of IJCAI, pp. 973–978 (2001)
Gagne, P., Dayton, C.M.: Best Regression Model Using Information Criteria. Journal of Modern Applied Statistical Methods 1, 479–488 (2002)
Gendreau, M., Potvin, J.Y.: Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol. 146 (2010)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Data Mining, Inference, and Prediction, 2nd edn. Springer (2009)
Kahneman, D., Tversky, A.: Prospect Theory: An Analysis of Decisions under Risk. Econometrica 47(2), 263–292 (1979)
Kotsiantis, S.B.: Decision trees: a recent overview. Artificial Intelligence Review, 1–23 (2011)
Krętowski, M., Grześ, M.: Evolutionary Induction of Cost-Sensitive Decision Trees. In: Esposito, F., Raś, Z.W., Malerba, D., Semeraro, G. (eds.) ISMIS 2006. LNCS (LNAI), vol. 4203, pp. 121–126. Springer, Heidelberg (2006)
Kretowski, M., Grześ, M.: Evolutionary Induction of Mixed Decision Trees. International Journal of Data Warehousing and Mining 3(4), 68–82 (2007)
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer (1996)
Murthy, S.: Automatic construction of decision trees from data: A multi-disciplinary survey. Data Mining and Knowledge Discovery 2, 345–389 (1998)
Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Numerical Recipes in C. Cambridge University Press (1988)
Rokach, L., Maimon, O.Z.: Top-down induction of decision trees classifiers - A survey. IEEE Transactions on Systems Man and Cybernetics, Part C 35(4), 476–487 (2005)
Rokach, L., Maimon, O.Z.: Data mining with decision trees: theory and application. Machine Perception Arfitical Intelligence 69 (2008)
Schwarz, G.: Estimating the Dimension of a Model. The Annals of Statistics 6, 461–464 (1978)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Parallel Distributed Processing, pp. 318–362. MIT Press, Cambridge (1986)
Shefrin, H., Statman, M.: The Disposition to Sell Winners Too Early and Ride Losers Too Long: Theory and Evidence. Journal of Finance 40, 777–790 (1985)
Ting, K.: An instance-weighting method to induce cost-sensitive trees. IEEE Transactions on Knowledge and Data Engineering 14(3), 659–665 (2002)
Torgo, L., Ribeiro, R.: Utility-based regression. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 597–604. Springer, Heidelberg (2007)
Turney, P.: Types of cost in inductive concept learning. In: Proc. of ICML 2000 Workshop on Cost-Sensitive Learning, Stanford, CA (2000)
Quinlan, J.: Learning with Continuous Classes. In: Proc. of AI 1992, pp. 343–348. World Scientific (1992)
Varian, H.R.: A Bayesian Approach to Real Estate Assessment. In: Fienberg, S.E., Zellner, A. (eds.) Studies in Bayesian Econometrics and Statistics: In honor of L.J. Savage, North-Holland, Amsterdam, pp. 195–208 (1974)
Zhao, H., Sinha, A.P., Bansal, G.: An extended tuning method for cost-sensitive regression and forecasting. Decision Support Systems 51, 372–383 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Czajkowski, M., Czerwonka, M., Kretowski, M. (2014). Cost-Sensitive Extensions for Global Model Trees: Application in Loan Charge-Off Forecasting. In: Swiątek, J., Grzech, A., Swiątek, P., Tomczak, J. (eds) Advances in Systems Science. Advances in Intelligent Systems and Computing, vol 240. Springer, Cham. https://doi.org/10.1007/978-3-319-01857-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-01857-7_30
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-01856-0
Online ISBN: 978-3-319-01857-7
eBook Packages: EngineeringEngineering (R0)