More Web Proxy on the site http://driver.im/

research-article

Multi-feature fusion: : Graph neural network and CNN combining for hyperspectral image classification

Authors:

Weiwei CaiAuthors Info & Claims

Volume 501, Issue C

Pages 246 - 257

https://doi.org/10.1016/j.neucom.2022.06.031

Published: 28 August 2022 Publication History

Abstract

Due to its impressive representation power, the graph convolutional network (GCN) has attracted increasing attention in the hyperspectral image (HSI) classification. However, the most of available GCN-based methods for HSI classification utilize superpixels as graph nodes, which ignore the pixel-wise spectral-spatial features. To overcome the issues, we propose a novel multi-feature fusion network (MFGCN), where two different convolutional networks, i.e., multi-scale GCN and multi-scale convolutional neural network (CNN), are utilized in two branches, separately. The multi-scale superpixel-based GCN can reduce the computing power requirements, deal with the problem of labeled deficiency, and refine the multi-scale spatial features from HSI. The multi-scale CNN can extract the multi-scale pixel-wise local features for HSI classification. Furthermore, we introduced a 1D CNN to extract the spectral features for superpixels (nodes), which is different from most existing methods. Finally, a concatenate operation is employed to fuse the complementary multi-scale features. In comparison with the state-of-the-art models on three datasets, the proposed method achieves superior experimental results and outperforms competitive methods.

References

[1]

Y. Ding, X. Zhao, Z. Zhang, W. Cai, N. Yang, Graph sample and aggregate-attention network for hyperspectral image classification, IEEE Geosci. Remote Sens. Lett. 19 (2022) 1–5.

[2]

D. Hong, W. He, N. Yokoya, J. Yao, L. Gao, L. Zhang, J. Chanussot, X. Zhu, Interpretable hyperspectral artificial intelligence: When nonconvex modeling meets hyperspectral remote sensing, IEEE Geosci. Remote Sens. Magazine 9 (2) (2021) 52–87.

[3]

D. Yao, Z. Zhi-li, Z. Xiao-feng, C. Wei, H.e. Fang, C. Yao-ming, W.-W. Cai, Deep hybrid: multi-graph neural network collaboration for hyperspectral image classification, Defence Technol. (2022).

[4]

Y. Cai, Z. Zhang, P. Ghamisi, Y. Ding, X. Liu, Z. Cai, R. Gloaguen, Superpixel Contracted Neighborhood Contrastive Subspace Clustering Network for Hyperspectral Images, IEEE Trans. Geosci. Remote Sens (2022) 1–13 5530113.

[5]

H. Hu, F. He, F. Zhang, Y. Ding, X. Wu, J. Zhao, M. Yao, Unifying Label Propagation and Graph Sparsification for Hyperspectral Image Classification, IEEE Geosci. Remote Sens. Lett. (2022) 15.

[6]

S. Li, W. Song, L. Fang, Y. Chen, P. Ghamisi, J.A. Benediktsson, Deep learning for hyperspectral image classification: an overview, IEEE Trans. Geosci. Remote Sens. 57 (9) (2019) 6690–6709.

[7]

D. Hong, N. Yokoya, J. Chanussot, X.X. Zhu, CoSpace: Common subspace learning from hyperspectral-multispectral correspondences, IEEE Trans. Geosci. Remote Sens. 57 (7) (2019) 4349–4359.

[8]

Y. Ding, Z. Zhang, X. Zhao, D. Hong, W. Li, W. Cai, Y. Zhan, AF2GNN: graph convolution with adaptive filters and aggregator fusion for hyperspectral image classification, Inf. Sci. 602 (2022) 201–219.

[9]

Y. Cai, M. Zeng, Z. Cai, X. Liu, Z. Zhang, Graph regularized residual subspace clustering network for hyperspectral image clustering, Inf. Sci. 578 (2021) 85–101.

[10]

Y. Ding, X. Zhao, Z. Zhang, W. Cai, N. Yang, Y. Zhan, Semi-supervised locality preserving dense graph neural network with ARMA filters and context-aware learning for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens. 60 (2022) 1–12.

[11]

J. Bai, A. Yuan, Z. Xiao, H. Zhou, D. Wang, H. Jiang, L. Jiao, Class incremental learning with few-shots based on linear programming for hyperspectral image classification, IEEE Trans. Cybernetics (2020).

[12]

J. Peng, L. Li, Y.Y. Tang, Maximum likelihood estimation-based joint sparse representation for the classification of hyperspectral remote sensing images, IEEE Trans. Neural Networks Learn. Syst. 30 (6) (2018) 1790–1802.

[13]

Y. Cai, Z. Zhang, Z. Cai, X. Liu, X. Jiang, Q. Yan, Graph convolutional subspace clustering: A robust subspace clustering framework for hyperspectral image, IEEE Trans. Geosci. Remote Sens. 59 (5) (2020) 4191–4202.

[14]

J. Bai, S. Huang, Z. Xiao, X. Li, Y. Zhu, A.C. Regan, L. Jiao, Few-Shot Hyperspectral Image Classification Based on Adaptive Subspaces and Feature Transformation, IEEE Trans. Geosci. Remote Sens. 60 (2022) 1–17.

[15]

C. Chen, W. Li, H. Su, K. Liu, Spectral-spatial classification of hyperspectral image based on kernel extreme learning machine, Remote Sens. 6 (6) (2014) 5795–5814.

[16]

J. Li, J.M. Bioucas-Dias, A. Plaza, Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning, IEEE Trans. Geosci. Remote Sens. 48 (11) (2010) 4085–4098.

[17]

C. Bo, H. Lu, D. Wang, Hyperspectral image classification via JCR and SVM models with decision fusion, IEEE Geosci. Remote Sens. Lett. 13 (2) (2015) 177–181.

[18]

Y. Cai, X. Liu, Z. Cai, BS-Nets: an end-to-end framework for band selection of hyperspectral image, IEEE Trans. Geosci. Remote Sens. 58 (3) (2019) 1969–1984.

[19]

M. Fauvel, J.A. Benediktsson, J. Chanussot, J.R. Sveinsson, Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles, IEEE Trans. Geosci. Remote Sens. 46 (11) (2008) 3804–3814.

[20]

L. Fang, N. He, S. Li, P. Ghamisi, J.A. Benediktsson, Extinction profiles fusion for hyperspectral images classification, IEEE Trans. Geosci. Remote Sens. 56 (3) (2017) 1803–1815.

[21]

S. Jia, L. Shen, Q. Li, Gabor feature-based collaborative representation for hyperspectral imagery classification, IEEE Trans. Geosci. Remote Sens. 53 (2) (2014) 1118–1129.

[22]

Z. Tang, M. Ling, H. Yao, Z. Qian, X. Zhang, J. Zhang, S. Xu, Robust image hashing via random Gabor filtering and DWT, Comput., Mater. Continua 55 (2) (2018) 331–344.

[23]

P. Quesada-Barriuso, F. Argüello, D.B. Heras, Spectral–spatial classification of hyperspectral images using wavelets and extended morphological profiles, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7 (4) (2014) 1177–1185.

[24]

J.A. Benediktsson, J.A. Palmason, J.R. Sveinsson, Classification of hyperspectral data from urban areas based on extended morphological profiles, IEEE Trans. Geosci. Remote Sens. 43 (3) (2005) 480–491.

[25]

X. Kang, S. Li, J.A. Benediktsson, Spectral–spatial hyperspectral image classification with edge-preserving filtering, IEEE Trans. Geosci. Remote Sens. 52 (5) (2013) 2666–2677.

[26]

X.X. Zhu, D. Tuia, L. Mou, G.-S. Xia, L. Zhang, F. Xu, F. Fraundorfer, Deep learning in remote sensing: A comprehensive review and list of resources, IEEE Geosci. Remote Sens. Mag. 5 (4) (2017) 8–36.

[27]

L. Mou, P. Ghamisi, X.X. Zhu, Deep recurrent neural networks for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens. 55 (7) (2017) 3639–3655.

[28]

Y. Chen, Z. Lin, X. Zhao, G. Wang, Y. Gu, Deep Learning-Based Classification of Hyperspectral Data, IEEE J. Selected Top. Appl. Earth Observ. Remote Sens. 7 (6) (2014) 2094–2107.

[29]

K. Makantasis, K. Karantzalos, A. Doulamis, N. Doulamis, Deep supervised learning for hyperspectral data classification through convolutional neural networks, in: IEEE International Geoscience and Remote Sensing Symposium (IGARSS), IEEE, 2015, pp. 4959–4962.

[30]

S.K. Roy, G. Krishna, S.R. Dubey, B.B. Chaudhuri, HybridSN: Exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification, IEEE Geosci. Remote Sens. Lett. 17 (2) (2019) 277–281.

[31]

W. Hu, Y. Huang, L.i. Wei, F. Zhang, H. Li, Deep convolutional neural networks for hyperspectral image classification, J. Sens. 2015 (2015) 1–12.

[32]

Y. Li, H. Zhang, Q. Shen, Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network, Remote Sens. 9 (1) (2017) 67.

[33]

J. Yang, Y.-Q. Zhao, J.-C.-W. Chan, Learning and transferring deep joint spectral–spatial features for hyperspectral classification, IEEE Trans. Geosci. Remote Sens. 55 (8) (2017) 4729–4742.

[34]

C. Chen, J.-J. Zhang, C.-H. Zheng, Q. Yan, L.-N. Xun, Classification of hyperspectral data using a multi-channel convolutional neural network, in: International Conference on Intelligent Computing, Springer, 2018, pp. 81–92.

[35]

J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.

[36]

L. Zhu, Y. Chen, P. Ghamisi, J.A. Benediktsson, Generative adversarial networks for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens. 56 (9) (2018) 5046–5063.

[37]

H. Hu, M. Yao, F. He, F. Zhang, Graph neural network via edge convolution for hyperspectral image classification, IEEE Geosci. Remote Sens. Lett. 19 (2022) 1–5.

[38]

M.W. T. N. Kipf, Semi-supervised classification with graph convolutional networks, narXiv preprint arXiv:1609.02907 (2016).

[39]

A. Qin, Z. Shang, J. Tian, Y. Wang, T. Zhang, Y.Y. Tang, Spectral–spatial graph convolutional networks for semisupervised hyperspectral image classification, IEEE Geosci. Remote Sens. Lett. 16 (2) (2018) 241–245.

[40]

J. Bai, B. Ding, Z. Xiao, L. Jiao, H. Chen, A.C. Regan, Hyperspectral image classification based on deep attention graph convolutional network, IEEE Trans. Geosci. Remote Sens. 60 (2021) 1–16.

[41]

D. Hong, L. Gao, J. Yao, B. Zhang, A. Plaza, J. Chanussot, Graph convolutional networks for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens. 59 (7) (2020) 5966–5978.

[42]

S. Wan, C. Gong, P. Zhong, B. Du, L. Zhang, J. Yang, Multiscale dynamic graph convolutional network for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens. 58 (5) (2019) 3162–3177.

[43]

Y. Ding, X. Zhao, Z. Zhang, W. Cai, N. Yang, Multiscale graph sample and aggregate network with context-aware learning for hyperspectral image classification, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 14 (2021) 4561–4572.

[44]

A.J. Izenman, Linear discriminant analysis, Modern multivariate statistical techniques, Springer, 2013, pp. 237–280.

[45]

R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Süsstrunk, SLIC superpixels compared to state-of-the-art superpixel methods, IEEE Trans. Pattern Anal. Mach. Intell. 34 (11) (2012) 2274–2282.

Digital Library

[46]

T. Dozat, Incorporating nesterov momentum into adam, (2016).

[47]

K. Djerriri, A. Safia, R. Adjoudj, M.S. Karoui, Improving hyperspectral image classification by combining spectral and multiband compact texture features, in: IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 2019, pp. 465–468.

[48]

W. Li, G. Wu, F. Zhang, Q. Du, Hyperspectral image classification using deep pixel-pair features, IEEE Trans. Geosci. Remote Sens. 55 (2) (2016) 844–853.

[49]

C. Zhang, G. Li, S. Du, Multi-scale dense networks for hyperspectral remote sensing image classification, IEEE Trans. Geosci. Remote Sens. 57 (11) (2019) 9201–9222.

[50]

M. Zhang, W. Li, Q. Du, Diverse region-based CNN for hyperspectral image classification, IEEE Trans. Image Process. 27 (6) (2018) 2623–2634.

Cited By

Huang WZhang ZZhao WPeng JXu WLiao YZhou SWang Z(2025)Auditing privacy budget of differentially private neural network modelsNeurocomputing10.1016/j.neucom.2024.128756614:COnline publication date: 21-Jan-2025
https://dl.acm.org/doi/10.1016/j.neucom.2024.128756
Filho IGôlo MMarcacini RRezende S(2025)How do financial time series enhance the detection of news significance in market movements? A study using graph neural networks with heterogeneous representationsNeural Computing and Applications10.1007/s00521-024-10418-537:3(1307-1319)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s00521-024-10418-5
Zhang YWilker K(2024)Visual-and-Language Multimodal Fusion for Sweeping Robot Navigation Based on CNN and GRUJournal of Organizational and End User Computing10.4018/JOEUC.33838836:1(1-21)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.4018/JOEUC.338388
Show More Cited By

Index Terms

Multi-feature fusion: Graph neural network and CNN combining for hyperspectral image classification
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Feature Fusion: Graph Attention Network and CNN Combing for Hyperspectral Image Classification
ICCCV '22: Proceedings of the 5th International Conference on Control and Computer Vision

Graph convolutional networks (GCNs) have attracted increasing attention in hyperspectral image classification. However, most of the available GCN-based HSI classification methods treat superpixels as graph nodes, ignoring pixel-level spectral spatial ...
Multiple convolutional layers fusion framework for hyperspectral image classification
Abstract
Hyperspectral images (HSIs) often contain complex structures with different scales, and thus capturing the structural information of various scales is very important for HSIs classification. The convolutional neural network can ...
Integrating Convolutional Neural Network and Gated Recurrent Unit for Hyperspectral Image Spectral-Spatial Classification
Pattern Recognition and Computer Vision
Abstract
In this paper, we propose a novel deep learning framework for hyperspectral image (HSI) spectral-spatial classification. This framework mainly consists of two components: convolutional neural network (CNN) and gated recurrent unit (GRU). CNN is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 501, Issue C

Aug 2022

890 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 28 August 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Huang WZhang ZZhao WPeng JXu WLiao YZhou SWang Z(2025)Auditing privacy budget of differentially private neural network modelsNeurocomputing10.1016/j.neucom.2024.128756614:COnline publication date: 21-Jan-2025
https://dl.acm.org/doi/10.1016/j.neucom.2024.128756
Filho IGôlo MMarcacini RRezende S(2025)How do financial time series enhance the detection of news significance in market movements? A study using graph neural networks with heterogeneous representationsNeural Computing and Applications10.1007/s00521-024-10418-537:3(1307-1319)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s00521-024-10418-5
Zhang YWilker K(2024)Visual-and-Language Multimodal Fusion for Sweeping Robot Navigation Based on CNN and GRUJournal of Organizational and End User Computing10.4018/JOEUC.33838836:1(1-21)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.4018/JOEUC.338388
Zhang WGeng XWang QHan JJiang H(2024)A Low-Power and High-Accuracy Approximate Adder for Logarithmic Number SystemProceedings of the Great Lakes Symposium on VLSI 202410.1145/3649476.3658706(125-131)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3649476.3658706
Li XSu YYao JGu HJiao Y(2024)LCFE-XKnowledge-Based Systems10.1016/j.knosys.2024.112215301:COnline publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.112215
Atlan FPence I(2024)The effect of noise removal filters on classifying different types of medical imagesDigital Signal Processing10.1016/j.dsp.2024.104613153:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.dsp.2024.104613
Wang DLi ZDey NCrespo RShi FSherratt R(2024)Design element extraction of plantar pressure imaging employing meta-learning-based graphic convolutional neural networksApplied Soft Computing10.1016/j.asoc.2024.111598158:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111598
Xu ZShen KCai PYang THu YChen SZhu YWu ZDai YWang JYang F(2024)Parallel proportional fusion of a spiking quantum neural network for optimizing image classificationApplied Intelligence10.1007/s10489-024-05786-354:22(11876-11891)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05786-3
Zhang JTsai PTsai M(2024)Semantic2Graph: graph-based multi-modal feature fusion for action segmentation in videosApplied Intelligence10.1007/s10489-023-05259-z54:2(2084-2099)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05259-z
Yan YLi JXu SChen XLiu GFu Y(2024)LightCapsGNN: light capsule graph neural network for graph classificationKnowledge and Information Systems10.1007/s10115-024-02170-y66:10(6363-6386)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10115-024-02170-y
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents