Abstract
Regression methods aim at inducing models of numeric data. While most state-of-the-art machine learning methods for regression focus on inducing piecewise regression models (regression and model trees), we investigate the predictive performance of regression models based on polynomial equations. We present Ciper, an efficient method for inducing polynomial equations and empirically evaluate its predictive performance on standard regression tasks. The evaluation shows that polynomials compare favorably to linear and piecewise regression models, induced by standard regression methods, in terms of degree of fit and complexity. The bias-variance decomposition of predictive error shows that Ciper has lower variance than methods for inducing regression trees.
Chapter PDF
Similar content being viewed by others
References
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Chaudhuri, P., Huang, M.C., Loh, W.Y., Yao, R.: Piecewise-polynomial regression trees. Statistica Sinica 4, 143–167 (1994)
Chen, Y., Dong, G., Han, J., Wah, B., Wang, J.: Multidimensional regression analysis of time-series data streams. In: Proceedings of the Twentyeighth International Conference on Very Large Data Bases, pp. 323–334. Morgan Kaufmann, San Francisco (2002)
Džeroski, S., Todorovski, L., Ljubič, P.: Using constraints in discovering dynamics. In: Proceedings of the Sixth International Conference on Discovery Science, pp. 297–305. Springer, Berlin (2003)
Falkenhainer, B., Michalski, R.: Integrating quantitative and qualitative discovery in the ABACUS system. In: Machine Learning: An Artificial Intelligence Approach, vol. 3, pp. 153–190. Morgan Kaufmann, San Mateo (1990)
Frank, E., Witten, I.H.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Mateo (1999)
Friedman, J.: Multivariate adaptive regression splines (with discussion). Annals of Statistics 19, 1–141 (1991)
Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neural Computation 4, 1–58 (1992)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Berlin (2001)
Langley, P., Simon, H.A., Bradshaw, G.L., Żythow, J.M.: Scientific Discovery. MIT Press, Cambridge (1987)
Todorovski, L., Džeroski, S.: Using domain knowledge on population dynamics modeling for equation discovery. In: Proceedings of the Twelfth European Conference on Machine Learning, pp. 478–490. Springer, Heidelberg (2001)
Todorovski, L., Džeroski, S., Ljubič, P.: Discovery of polynomial equations for regression. In: Proceedings of the Sixth International Multi-Conference Information Society, Ljubljana, Slovenia, vol. A, pp. 151–154. Jozef Stefan Institute (2003)
Torgo, L.: Regression data sets (2001), http://www.liacc.up.pt/~ltorgo/Regression/DataSets.html
Torgo, L., da Costa, J.P.: Clustered partial linear regression. In: Proceedings of the Eleventh European Conference on Machine Learning, pp. 426–436. Springer, Heidelberg (2000)
Wang, Y., Witten, I.H.: Induction of model trees for predicting continuous classes. In: The Proceedings of the Poster Papers of the Eighth European Conference on Machine Learning, pp. 128–137. University of Economics, Faculty of Informatics and Statistics, Prague (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Todorovski, L., Ljubič, P., Džeroski, S. (2004). Inducing Polynomial Equations for Regression. In: Boulicaut, JF., Esposito, F., Giannotti, F., Pedreschi, D. (eds) Machine Learning: ECML 2004. ECML 2004. Lecture Notes in Computer Science(), vol 3201. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30115-8_41
Download citation
DOI: https://doi.org/10.1007/978-3-540-30115-8_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23105-9
Online ISBN: 978-3-540-30115-8
eBook Packages: Springer Book Archive