Abstract
By their very nature, regression problems can be transformed into classification problems by discretizing their target variable. Within this perspective, in this work we investigate the possibility of improving the performance of deep machine learning models in regression scenarios through a training strategy that combines different classification and regression objectives. In particular, we train deep neural networks using the mean squared error along with categorical cross-entropy and the novel Fisher loss as companion losses. Finally, we will compare experimentally the results of these companion loss methods with the ones obtained using the standard mean squared loss.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, software available from tensorflow.org
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Caruana, R.: Multitask learning. Mach. Learn. 28, 41–75 (1997)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 1–27 (2011)
Chen, J., Cheng, L., Yang, X., Liang, J., Quan, B., Li, S.: Joint learning with both classification and regression models for age prediction. In: Journal of Physics: Conference Series, vol. 1168, p. 032016. IOP Publishing (2019)
Chollet, F., et al.: Keras (2015). https://github.com/fchollet/keras
Christensen, R.H.B.: ordinal-regression models for ordinal data. R Packag. Version 28, 2015 (2015)
Diaz-Vico, D., Dorronsoro, J.R.: Deep least squares fisher discriminant analysis. IEEE Trans. Neural Netw. Learn. Syst. 31(8), 2752–2763 (2019)
Díaz-Vico, D., Fernández, A., Dorronsoro, J.R.: Companion losses for deep neural networks. In: Sanjurjo González, H., Pastor López, I., García Bringas, P., Quintián, H., Corchado, E. (eds.) HAIS 2021. LNCS (LNAI), vol. 12886, pp. 538–549. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86271-8_45
Díaz-Vico, D., Fernández, A., Dorronsoro, J.R.: Companion losses for ordinal regression. In: Garcia Bringas, P., et al. (eds.) Hybrid Artificial Intelligent Systems. HAIS 2022. LNCS, vol. 13469, pp. 211–222. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-15471-3_19
Glocker, B., Pauly, O., Konukoglu, E., Criminisi, A.: Joint classification-regression forests for spatially structured multi-object segmentation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 870–881. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33765-9_62
Liu, M., Zhang, J., Adeli, E., Shen, D.: Deep multi-task multi-channel learning for joint classification and regression of brain status. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 3–11. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_1
Liu, M., Zhang, J., Adeli, E., Shen, D.: Joint classification and regression via deep multi-task multi-channel learning for Alzheimer’s disease diagnosis. IEEE Trans. Biomed. Eng. 66(5), 1195–1206 (2018)
Paszke, A., et al.: Automatic differentiation in pytorch. In: Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019)
Schulter, S., Leistner, C., Wohlhart, P., Roth, P.M., Bischof, H.: Alternating regression forests for object detection and pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2013
Schulter, S., Leistner, C., Wohlhart, P., Roth, P.M., Bischof, H.: Accurate object detection with joint classification-regression random forests. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014
Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945). https://www.jstor.org/stable/3001968
Zhang, Z., Dai, G., Xu, C., Jordan, M.I.: Regularized discriminant analysis, ridge regression and beyond. J. Mach. Learn. Res. 11, 2199–2228 (2010)
Acknowledgments
The authors acknowledge financial support from the European Regional Development Fund and the Spanish State Research Agency of the Ministry of Economy, Industry, and Competitiveness under the project PID2019-106827GB-I00. They also thank the support of the UAM–ADIC Chair for Data Science and Machine Learning and gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at UAM. They finally acknowledge the financial support of the Department of Education of the Basque Government under the grant PRE_2022_1_0103.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sánchez-Ferrera, A., Dorronsoro, J.R. (2023). Companion Classification Losses for Regression Problems. In: García Bringas, P., et al. Hybrid Artificial Intelligent Systems. HAIS 2023. Lecture Notes in Computer Science(), vol 14001. Springer, Cham. https://doi.org/10.1007/978-3-031-40725-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-40725-3_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40724-6
Online ISBN: 978-3-031-40725-3
eBook Packages: Computer ScienceComputer Science (R0)