Abstract
Objective feature selection is an important component in the machine learning framework, which has addressed problems like computational burden increasing and unnecessary high-dimensional representations. Most of feature selection techniques only perform individual feature evaluations and ignore the structural relationships between features of the same nature, causing relations to break and harming the algorithm performance. In this paper a feature group selection technique is proposed with the aim of objectively identify the relevance that a feature group carries out in a classification task. The proposed method uses Multiple Kernel Learning with a penalization rule based on the \(\ell _1\)-norm and a Support Vector Machine as base learner. Performance evaluation is carried out using two binarized configurations of the freely available MFEAT dataset. It provides six different feature groups allowing to develop multiple feature group analysis. The experimental results show that the implemented methodology is stable in the identification of the relevance of each feature group during all experiments, what allows to outperform the classification accuracy of state-of-the-art methods.
H. J. Areiza-Laverde—Supported by Colciencias and the Instituto Tecnológico Metropolitano.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brahim, A.B., Khanchel, R., Limam, M.: Robust ensemble based algorithms for multi-source data classification. Int. J. Comput. Inf. Syst. Ind. Manag. Appl. 4, 420–427 (2012)
Cilia, N.D., De Stefano, C., Fontanella, F., di Freca, A.S.: A ranking-based feature selection approach for handwritten character recognition. Pattern Recognit. Lett. (2018)
Cristianini, N., Shawe-Taylor, J.: An introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)
Culache, O., Obadă, D.R.: Multimodality as a premise for inducing online flow on a brand website: a social semiotic approach. Procedia-Soc. Behav. Sci. 149, 261–268 (2014)
Damoulas, T., Girolami, M.A.: Pattern recognition with a Bayesian Kernel combination machine. Pattern Recognit. Lett. 30(1), 46–54 (2009)
De Stefano, C., Fontanella, F., Marrocco, C., Di Freca, A.S.: A GA-based feature selection approach with an application to handwritten character recognition. Pattern Recognit. Lett. 35, 130–141 (2014)
Dheeru, D., Karra Taniskidou, E., Duin, R.: UCI machine learning repository - multiple features data set (2017). https://archive.ics.uci.edu/ml/datasets/Multiple+Features
Dhifli, W., Aridhi, S., Nguifo, E.M.: MR-SimLab: scalable subgraph selection with label similarity for big data. Inf. Syst. 69, 155–163 (2017)
Foresti, L., Tuia, D., Timonin, V., Kanevski, M.F.: Time series input selection using multiple Kernel learning. In: ESANN, pp. 123–128 (2010)
Gönen, G.B., Gönen, M., Gürgen, F.: Probabilistic and discriminative group-wise feature selection methods for credit risk analysis. Expert Syst. Appl. 39(14), 11709–11717 (2012)
Gönen, M., Alpaydın, E.: Multiple Kernel learning algorithms. J. Mach. Learn. Res. 12(Jul), 2211–2268 (2011)
Gu, X., Angelov, P.P.: Self-organising fuzzy logic classifier. Inf. Sci. 447, 36–51 (2018)
Hernández-Muriel, J.A., Álvarez-Meza, A.M., Echeverry-Correa, J.D., Orozco-Gutierrez, A.A., Álvarez-López, M.A.: Feature relevance estimation for vibration-based condition monitoring of an internal combustion engine. Tecno Lóg. 20, 159–174 (2017). http://www.scielo.org.co/scielo.php?script=sciarttext&pid=S0123-77992017000200011&nrm=iso
Koç, M., Barkana, A.: Application of linear regression classification to low-dimensional datasets. Neurocomputing 131, 331–335 (2014)
Kyunghoon, K.: Approaches to the design of machine learning system. Ph.D. thesis, Escuela de Graduados de la Universidad Nacional de Seúl (2016)
Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R.P., Tang, J., Liu, H.: Feature selection: a data perspective. ACM Comput. Surv. (CSUR) 50(6), 94 (2017)
Mosek, A.: The mosek optimization software. 54(2-1), 5 (2010). http://www.mosek.com
Raza, H., Cecotti, H., Prasad, G.: Optimising frequency band selection with forward-addition and backward-elimination algorithms in EEG-based brain-computer interfaces. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2015)
Subrahmanya, N., Shin, Y.C.: Automated sensor selection and fusion for monitoring and diagnostics of plunge grinding. J. Manuf. Sci. Eng. 130(3), 031014 (2008)
Symons, C.T., Arel, I.: Multi-view budgeted learning under label and feature constraints using label-guided graph-based regularization. In: International Conference on Machine Learning, Workshop on Combining Learning Strategies to Reduce Label Cost. Citeseer (2011)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58, 267–288 (1996)
Wang, J., Wang, M., Li, P., Liu, L., Zhao, Z., Hu, X., Wu, X.: Online feature selection with group structure analysis. IEEE Trans. Knowl. Data Eng. 27(11), 3029–3041 (2015)
Witten, I.H., Frank, E., Hall, M.A., Pal, C.J.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2016)
Xiang, S., Yang, T., Ye, J.: Simultaneous feature and feature group selection through hard thresholding. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 532–541. ACM (2014)
Xu, Z., Jin, R., Yang, H., King, I., Lyu, M.R.: Simple and efficient multiple Kernel learning by group lasso. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 1175–1182. Citeseer (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Areiza-Laverde, H.J., Díaz, G.M., Castro-Ospina, A.E. (2018). Feature Group Selection Using MKL Penalized with \(\ell _1\)-norm and SVM as Base Learner. In: Figueroa-García, J., López-Santana, E., Rodriguez-Molano, J. (eds) Applied Computer Sciences in Engineering. WEA 2018. Communications in Computer and Information Science, vol 915. Springer, Cham. https://doi.org/10.1007/978-3-030-00350-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-00350-0_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00349-4
Online ISBN: 978-3-030-00350-0
eBook Packages: Computer ScienceComputer Science (R0)