Abstract
In this paper, we address the supervised dimensionality reduction problem for improved hyperspectral image (HSI) classification. In particular, we design a novel information theoretic ranking measure and further propose a greedy strategy that provides a more principled way of ranking the features. Note that the use of information theory is prevalent in the area of filtering based dimensionality reduction. However, the existing ranking measures in this regard are predominantly based on the notion of enforcing non-redundancy among the selected features while simultaneously ensuring relevance. In the process, it is not considered whether the selected feature dimensions are representatives of the discarded ones. However, those abandoned dimensions may contain subtle information regarding some of the land-cover classes. Inspired by this analogy, we propose a new ranking function in this paper which generates an ordering of the dimensions based on non-redundancy, relevancy while ensuring that the selected feature dimensions are representatives of all the original feature dimensions. We subsequently propose a graph-based greedy algorithm for feature selection in this regard. Experimental results on widely studied HSI datasets confirm the superiority of the proposed method compared to the existing information theoretic feature selection strategies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Networks 5(4), 537–550 (1994). https://doi.org/10.1109/72.298224
Baumgardner, M.F., Biehl, L.L., Landgrebe, D.A.: 220 band aviris hyperspectral image data set: June 12, 1992 Indian pine test site 3, September 2015. https://doi.org/10.4231/R7RX991C
Cheng-San, Y., Chuang, L.Y., Ke, C.H., Yang, C.H.: A hybrid feature selection method for microarray classification. IAENG Int. J. Comput. Sci. 35 (2008)
Chicharro, D., Panzeri, S.: Synergy and redundancy in dual decompositions of mutual information gain and information loss. Entropy 19(2), 71 (2017). https://doi.org/10.3390/e19020071
Ding, C., Peng, H.: Minimum redundancy feature selection from microarray gene expression data. In: Computational Systems Bioinformatics. Proceedings of the 2003 IEEE Bioinformatics Conference, CSB2003, August 2003, pp. 523–528 (2003). https://doi.org/10.1109/CSB.2003.1227396
Estevez, P.A., Tesmer, M., Perez, C.A., Zurada, J.M.: Normalized mutual information feature selection. IEEE Trans. Neural Networks 20(2), 189–201 (2009). https://doi.org/10.1109/TNN.2008.2005601
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, vol. 9, pp. 249–256. PMLR, 13–15 May 2010. http://proceedings.mlr.press/v9/glorot10a.html
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005). https://doi.org/10.1109/TPAMI.2005.159
Huang, J.J., Lv, N., Li, S.Q., Cai, Y.Z.: Feature selection for classificatory analysis based on information-theoretic criteria. Acta Automatica Sinica 34(3), 383–392 (2008). https://doi.org/10.3724/SP.J.1004.2008.00383
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations, December 2014. https://arxiv.org/abs/1412.6980
Kwak, N., Choi, C.-H.: Input feature selection for classification problems. IEEE Trans. Neural Networks 13(1), 143–159 (2002). https://doi.org/10.1109/72.977291
Lin, P., Thapa, N., Omer, I., Zhang, J.: Feature selection: a preprocess for data perturbation. IAENG Int. J. Comput. Sci. 38(2), 168–175 (2011)
Medjahed, S.A., Ouali, M.: Band selection based on optimization approach for hyperspectral image classification. Egypt. J. Remote Sens. Space Sci. 21(3), 413–418 (2018). https://doi.org/10.1016/j.ejrs.2018.01.003
Novovičová, J., Somol, P., Haindl, M., Pudil, P.: Conditional mutual information based feature selection for classification task. In: Rueda, L., Mery, D., Kittler, J. (eds.) CIARP 2007. LNCS, vol. 4756, pp. 417–426. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76725-1_44
Wang, W., Jones, P., Partridge, D.: A comparative study of feature-salience ranking techniques. Neural Comput. 13(7), 1603–1623 (2001). https://doi.org/10.1162/089976601750265027
Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. Chemometr. Intell. Lab. Syst. 2(1), 37–52 (1987). https://doi.org/10.1016/0169-7439(87)80084-9. Proceedings of the Multivariate Statistical Workshop for Geologists and Geochemists
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kookna, V., Singh, A.K., Raj, A., Banerjee, B. (2020). A Novel Information Theoretic Cost Measure for Filtering Based Feature Selection from Hyperspectral Images. In: Nain, N., Vipparthi, S., Raman, B. (eds) Computer Vision and Image Processing. CVIP 2019. Communications in Computer and Information Science, vol 1147. Springer, Singapore. https://doi.org/10.1007/978-981-15-4015-8_10
Download citation
DOI: https://doi.org/10.1007/978-981-15-4015-8_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-4014-1
Online ISBN: 978-981-15-4015-8
eBook Packages: Computer ScienceComputer Science (R0)