CN114581789A - Hyperspectral image classification method and system - Google Patents
Hyperspectral image classification method and system Download PDFInfo
- Publication number
- CN114581789A CN114581789A CN202210158771.8A CN202210158771A CN114581789A CN 114581789 A CN114581789 A CN 114581789A CN 202210158771 A CN202210158771 A CN 202210158771A CN 114581789 A CN114581789 A CN 114581789A
- Authority
- CN
- China
- Prior art keywords
- hyperspectral image
- image classification
- compression
- expansion
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 108091006146 Channels Proteins 0.000 claims abstract description 42
- 230000003993 interaction Effects 0.000 claims abstract description 40
- 230000006835 compression Effects 0.000 claims abstract description 35
- 238000007906 compression Methods 0.000 claims abstract description 35
- 230000003595 spectral effect Effects 0.000 claims abstract description 28
- 238000013507 mapping Methods 0.000 claims abstract description 27
- 238000001228 spectrum Methods 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000000605 extraction Methods 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 33
- 230000004913 activation Effects 0.000 claims description 27
- 239000011159 matrix material Substances 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 15
- 238000010606 normalization Methods 0.000 claims description 11
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 9
- 238000012360 testing method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000012821 model calculation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- VACHUYIREGFMSP-UHFFFAOYSA-N 9,10-dihydroxyoctadecanoic acid Chemical compound CCCCCCCCC(O)C(O)CCCCCCCC(O)=O VACHUYIREGFMSP-UHFFFAOYSA-N 0.000 description 1
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- VXPARNCTMSWSHF-DNVSUFBTSA-N Dihydrosanguinarine Natural products O=C1[C@H](C(C)=C)C[C@]2(CO)[C@@H](C)[C@H](O)CCC2=C1 VXPARNCTMSWSHF-DNVSUFBTSA-N 0.000 description 1
- 235000008331 Pinus X rigitaeda Nutrition 0.000 description 1
- 235000011613 Pinus brutia Nutrition 0.000 description 1
- 241000018646 Pinus brutia Species 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a hyperspectral image classification method and a hyperspectral image classification system, which comprise the following steps: acquiring a hyperspectral image, and preprocessing the hyperspectral image; performing channel interaction and compression of pixel spectral dimensions and expansion and alignment of a spatial window of pixel sample spatial dimensions on the preprocessed hyperspectral image to obtain mapping characteristics; based on the mapping characteristics, after two times of information interaction are carried out in sequence, spectrum-space characteristics are obtained; obtaining the category of each pixel in the acquired hyperspectral image by adopting a classifier based on the spectrum-space characteristics; the information interaction is to perform spectrum characteristic channel compression, non-local space information extraction and spectrum characteristic channel expansion on the input characteristics in sequence, and then fuse the input characteristics with the information interaction to obtain output characteristics. The receptive field for capturing the spatial information is enlarged, and richer and more robust spectral spatial information can be captured to efficiently complete the hyperspectral image classification task.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a hyperspectral image classification method and system.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The hyperspectral image classification is a core technology of hyperspectral image processing and interpretation, and a hyperspectral image classification task aims to assign a unique semantic label to each hyperspectral image pixel according to a given ground object class set and based on the spectrum and spatial characteristics of a hyperspectral image.
According to the traditional hyperspectral image classification method, the spectral space features which are discriminant and robust enough are difficult to extract efficiently from the aspects of spectral dimension and spatial dimension to complete the task of assigning the pixel labels, and in a complex scene, the generalization capability of the traditional hyperspectral image classification method is often tested. In recent years, a hyperspectral image classification method based on deep learning often extracts stacked spectrum-space characteristics in an end-to-end mode to realize a more robust pixel classification result, and the development of a hyperspectral image classification task is promoted.
Among the numerous deep learning models, the classification method based on the convolutional neural network has attracted extensive attention in the hyperspectral image classification task by the characteristics of local perception and parameter sharing and proves the excellent performance of the classification method. Similar to typical computer vision tasks such as semantic segmentation, target detection and instance segmentation, in the hyperspectral image classification task, modeling of long-range or non-local spatial information is also beneficial to assisting in completing the pixel classification task. To globally aggregate locally captured filtered values, convolution-based structures require deep stacking of convolution layers to capture spatial features of different granularity.
However, hyperspectral images are inherently accompanied by small sample problems due to their dual difficulties of image acquisition and data annotation, which limits the convolutional layer stacking depth of the model used to handle the hyperspectral image classification task. In addition, the deeper level convolution model also affects the efficiency of model task processing in terms of model parameters and runtime.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention provides a hyperspectral image classification method and system, which expand the receptive field of capturing spatial information and can capture richer and more robust spectral spatial information to efficiently complete a hyperspectral image classification task.
In order to achieve the purpose, the invention adopts the following technical scheme:
a first aspect of the present invention provides a hyperspectral image classification method, which includes:
acquiring a hyperspectral image, and preprocessing the hyperspectral image;
performing channel interaction and compression of pixel spectral dimensions and expansion and alignment of a spatial window of pixel sample spatial dimensions on the preprocessed hyperspectral image to obtain mapping characteristics;
based on the mapping characteristics, after two times of information interaction are carried out in sequence, spectrum-space characteristics are obtained;
obtaining the category of each pixel in the acquired hyperspectral image by adopting a classifier based on the spectrum-space characteristics;
the information interaction is to perform spectrum characteristic channel compression, non-local space information extraction and spectrum characteristic channel expansion on the input characteristics in sequence, and then fuse the input characteristics with the information interaction to obtain output characteristics.
Further, the channel interaction and compression of the pixel spectral dimension are realized by a spectral compression group consisting of a 1 × 1 convolution layer, batch normalization operation and a Relu activation function;
or,
the spatial window expansion and alignment of the pixel sample spatial dimension is realized by a spatial expansion group consisting of a 2 × 2 convolution layer, a batch normalization operation and a Relu activation function.
Further, the spectral feature channel compression is realized by a first 1 × 1 convolutional layer;
the output channel of the first 1 x 1 convolutional layer is configured with a batch normalization operation and a Relu activation function.
Further, the non-local spatial information extraction is realized through a double-head self-attention layer;
the dot product multiplication and element-by-element addition operations of the double-headed self-attention layer configuration matrix, the Softmax activation function, and the relative position encoding computation operations.
Further, the spectral feature channel expansion is realized by a second 1 × 1 convolutional layer;
the output channel of the second 1 × 1 convolutional layer is configured with a Relu activation function.
Further, the fusing with the mapping feature specifically includes: and the mapping characteristic and the output of the second 1 × 1 convolutional layer are subjected to fusion addition operation and Relu activation function in sequence to obtain interactive information.
Further, the preprocessing comprises respectively performing mean-variance standardization processing on each spectral channel in the acquired hyperspectral image.
A second aspect of the present invention provides a hyperspectral image classification system comprising:
a pre-processing module configured to: acquiring a hyperspectral image, and preprocessing the hyperspectral image;
a compression expansion module configured to: performing channel interaction and compression of pixel spectral dimensions and expansion and alignment of a spatial window of pixel sample spatial dimensions on the preprocessed hyperspectral image to obtain mapping characteristics;
a self-attention module configured to: based on the mapping characteristics, after two times of information interaction are carried out in sequence, spectrum-space characteristics are obtained;
a classification module configured to: obtaining the category of each pixel in the acquired hyperspectral image by adopting a classifier based on the spectrum-space characteristics;
the information interaction is to perform spectrum characteristic channel compression, non-local space information extraction and spectrum characteristic channel expansion on the input characteristics in sequence, and then fuse the input characteristics with the information interaction to obtain output characteristics.
A third aspect of the invention provides a computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of a method for hyperspectral image classification as described above.
A fourth aspect of the invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of a method for hyperspectral image classification as described above when executing the program.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a hyperspectral image classification method, which is characterized in that after spectrum characteristic channel compression, non-local spatial information extraction and spectrum characteristic channel expansion are sequentially carried out on mapping characteristics, the mapping characteristics are fused with the mapping characteristics to obtain interaction information, and the model performance and generalization capability of a model on a hyperspectral image classification task are further improved; in particular, a double-head self-attention mechanism is used for capturing non-local space interaction and correlation information, the mechanism enlarges the receptive field for capturing space information, and efficient matrix dot product multiplication is used for reducing the model calculation amount and the model parameter amount, so that the efficiency of classification is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a flow chart of a method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of model training according to a first embodiment of the present invention;
FIG. 3 is a structural diagram of a Bottleneck transform sub-block according to a first embodiment of the present invention;
fig. 4 is a diagram of a dual-headed self-attention layer according to a first embodiment of the invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Example one
The embodiment provides a hyperspectral image classification method, as shown in fig. 1, which specifically includes the following steps:
step 1, acquiring a hyperspectral image, and preprocessing the hyperspectral image.
Specifically, the specific method of pretreatment is as follows: and respectively carrying out mean-variance standardization treatment on each spectrum channel in the acquired hyperspectral image so as to accelerate the convergence speed of the proposed network model in the training process.
And 2, inputting the preprocessed hyperspectral images into a hyperspectral image classification network to obtain the category of each pixel in the acquired hyperspectral images. As shown in fig. 1, the hyperspectral image classification network includes a compression and expansion block (including a compression and expansion module), a self-attention block (self-attention module), and a classifier.
And for image blocks input by the network model, adopting a compression and expansion block to perform channel interaction and compression of pixel spectral dimensions and expansion and alignment of a spatial window of pixel sample spatial dimensions to obtain mapping characteristics. As shown in fig. 1, the companding block includes a spectral compression set consisting of a 1 × 1 convolution layer, a batch normalization operation, and a Relu activation function, and a spatial expansion set consisting of a 2 × 2 convolution layer, a batch normalization operation (BN), and a Relu activation function. The spectrum compression group is used for realizing channel interaction and compression of the spectrum dimension of the pixel sample, the spectrum compression coefficient alpha is used for controlling the channel interaction and compression degree of the spectrum dimension, and the alpha is set to be 0.64 through experimental verification; and the space expansion group is used for realizing the expansion and alignment of a space window of a pixel sample space dimension, and the original mapping characteristics (characteristic mapping blocks or mapping characteristic maps) of 9 multiplied by alpha C are transformed into the mapping characteristics of 12 multiplied by alpha C.
The mapping features are input from the attention block, and interaction information (spectral interaction and non-local spatial interaction information, or spectral-spatial features) of the mapping features is extracted. Specifically, based on the mapping characteristics, after two times of information interaction are carried out in sequence, the spectrum-space characteristics are obtained; the information interaction is to perform spectrum characteristic channel compression, non-local space information extraction and spectrum characteristic channel expansion on the input characteristics in sequence, and then fuse the input characteristics with the information interaction to obtain output characteristics. As shown in fig. 1, the self-attention block is composed of two information interactive sub-blocks (Bottleneck Transformer, i.e. bottleeck Transformer sub-blocks) connected in sequence. The bottleeck Transformer subblock, as shown in fig. 3, is derived from a bottleeck variant, one of the basic units of classical ResNet, and consists of a 1 × 1 convolution layer set, a 3 × 3 convolution layer set, and a 1 × 1 convolution layer set, with targeted matching to batch normalization operations and Relu activation functions, plus a residual connection. The Bottleneck Transformer sub-block replaces the 3 × 3 convolutional layer set in Bottleneck with a double-headed Self-Attention (Dual-Head Self-Attention) layer, and the rest of the settings remain unchanged. Specifically, the Bottleneck transducer subblock is composed of an input layer, an output layer and a plurality of output layers which are sequentially connected,A first 1 x 1 convolutional layer, a dual-headed self-attentive layer, a second 1 x 1 convolutional layer, and an output layer. The output channel of the first 1 × 1 convolutional layer is configured with batch normalization operations and Relu activation functions, and the output channel of the second 1 × 1 convolutional layer is configured with Relu activation functions. And the output of the input layer and the output of the second 1 multiplied by 1 convolutional layer are input into the output layer after fusion addition operation and Relu activation function. The first 1 x 1 convolutional layer is used to perform compression of the spectral feature channel, the second 1 x 1 convolutional layer is used to perform expansion of the spectral feature channel, and the dual-headed self-attention layer is used to model non-local spatial information. As shown in fig. 4, the double-headed self-attention layer splits the input data into three same-dimension shared matrices q, k, and v after the input data is subjected to dimension transformation by the full connection layer; specifically, x represents the dot product operation of the matrix, + represents the element-by-element addition operation of the matrix, 1 x 1 is the dot convolution operation, Softmax represents the nonlinear activation operation of the Softmax function, Wq,WkAnd WvTrainable weighting matrices, R, representing respective corresponding matrices q, k and vhAnd RwThe relative position codes of the height and the width of the characteristic input diagram are respectively represented, r represents a position code matrix generated for the output characteristic diagram, and besides, a T corner with upper left corner in capital is used in the diagram to mark a transpose matrix of a torque conversion matrix. Dot product multiplication and element-by-element addition operations of the double-headed self-attention layer configuration matrix, Softmax activation functions and relative position encoding calculation operations. Specifically, firstly, W is alignedqAnd WkThe similarity of a matrix q and a matrix k is calculated by carrying out nonlinear activation on the dot product result of the matrix q by using a softmax activation function, wherein the q matrix is embedded with relative position codes firstly, then proper dimensionality scaling is carried out on the similarity result, the corrected result is also the self weight of the v matrix, and the matrix v subjected to self-attention weighting correction is input into an output layer by using the self weight. In other words, the input map is processed by 1 × 1 convolutional layer to obtain initialized same matrixes q, k and v, and the matrixes q and k are dot-product-operated to obtain a matrix qkTRelative position codes R corresponding to height and width of input maphAnd RwPerforming element-by-element addition operation to obtain a matrix r, and performing dot product operation on the matrix q and the matrix r to obtain a matrix qrTMatrix qkTAnd qrTAnd after element-by-element addition operation, performing nonlinear activation by using a softmax activation function to obtain a similarity matrix of the matrix q and the matrix k, and performing dot product operation on the similarity matrix and the matrix v to obtain an output result. The self-attentive body portion of the dual-headed self-attentive layer used may be represented as:
where DHSA (q, k, v) represents the output result (mutual information) of the double-headed self-attention layer, dkFor the dimension of the k vector used as the dimension scaling of the intermediate result, T denotes the vector transpose and softmax is the softmax activation function.
And inputting the interaction information into a classifier to obtain the category of each pixel in the acquired hyperspectral image. As shown in figure 1, the method uses a classic three-layer classifier to perform final hyperspectral image pixel classification, wherein the hyperspectral image pixel classification comprises a mean pooling layer, a flattening layer and a full-link layer, and the final classifier uses a softmax activation function to predict and generate a label classification probability vector.
As shown in fig. 2, the method for training the hyperspectral image classification network includes:
(1) and carrying out mean-variance standardization processing on each spectral channel of the hyperspectral image.
(2) Filling 0 in the preprocessed image boundary; and splitting the data set according to the ground object class sample proportion of the data set. For each category in the data scene, according to the training sample: and (3) verifying the sample: test specimen r1:r2:r3For example, for the Indian Pines dataset and the Kennedy Space Center dataset, r1:r2:r30.1:0.01:0.89 for the Pavia University dataset r1:r2:r30.05:0.01: 0.945; when the number of certain sample classes is too small to meet the sampling requirement of the verification set, the minimum sampling number is set to ensure that each class is uniformly sampled according to the proportion of the number of class samples; specifically, when each pixel sample is sampled, the image is takenCutting an image block with the size of 9 multiplied by C by taking the element as a center, wherein 9 multiplied by 9 represents the size of a space window of the image block, and C is the original spectral dimension of the image; the training samples, the verification samples and the test samples of each category are respectively aggregated into a training set, a verification set and a test set.
(3) For each image block input into the network model, realizing spectral domain compression and spatial domain expansion of the pixel sample through a compression expansion block; and extracts the interactive information through the self-attention block.
(4) And (5) judging whether the iteration number of the hyperspectral image classification network reaches the set training iteration number, if so, performing the step (5), and otherwise, returning to the step (3).
(5) And sending the interaction information (spectrum-space characteristics) to a classifier for pixel classification, and calculating a loss value according to the label. The classifier uses a softmax activation function to predict and generate a label classification probability vector, then uses a cross entropy loss function to calculate a loss value, iteratively trains the proposed hyperspectral image classification network model in a back propagation mode, updates parameters of the proposed hyperspectral image classification network model, and expresses the cross entropy loss function used by the proposed hyperspectral image classification network model as follows:
wherein,
representing the calculated loss value, N representing the number of samples of a single batch training set in a small batch training mode adopted by the model, where the value is 32, K represents the number of classes in a data scene, N and K index the nth sample of the current batch training set and the kth class in the class label set, ynRepresenting the true value, In (y), of the nth hyperspectral image patch sample In the current batch of training setnK) denotes the indicator function, when ynWhen k is, In (y)nK) is 1; otherwise, In (y)nK) is 0. In addition to this, the present invention is,
and outputting a probability value representing that the nth hyperspectral image block sample under consideration belongs to the softmax function of the class k.
The network model method provided by the invention uses an Adam optimizer, the learning rate is set to be 0.001, and the size of a single-batch training set in a small-batch training mode is 32. In model training, calculating loss values corresponding to a training set and a verification set after each iteration is completed, and using a model after 100 iterations as a final model. In the model testing stage, the cut image block testing set is used for performing model testing in different hyperspectral scenes, the classification performance of the model in each hyperspectral scene can be quantitatively measured according to the corresponding truth value label, and the model can obtain the visual image of the whole scene by assigning class labels to each pixel in the scene.
The method makes up the defects of the existing deep learning-based method on the hyperspectral image classification task to a certain extent, obtains richer and more robust spectrum-space characteristics, and further improves the model performance and generalization capability of the model on the hyperspectral image classification task; in particular, the model uses a double-headed self-attention mechanism to capture non-local spatial interaction and correlation information, the mechanism enlarges the receptive field for capturing spatial information, and efficient matrix dot product multiplication is used to reduce the model calculation amount and the model parameter amount, which is beneficial to improving the efficiency of the model.
Example two
The embodiment provides a hyperspectral image classification system, which specifically comprises the following modules:
a pre-processing module configured to: acquiring a hyperspectral image, and preprocessing the hyperspectral image;
a compression expansion module configured to: performing channel interaction and compression of pixel spectral dimensions and expansion and alignment of a spatial window of pixel sample spatial dimensions on the preprocessed hyperspectral image to obtain mapping characteristics;
a self-attention module configured to: based on the mapping characteristics, after two times of information interaction are carried out in sequence, spectrum-space characteristics are obtained;
a classification module configured to: obtaining the category of each pixel in the acquired hyperspectral image by adopting a classifier based on the spectrum-space characteristics;
the information interaction is to perform spectrum characteristic channel compression, non-local space information extraction and spectrum characteristic channel expansion on the input characteristics in sequence, and then fuse the input characteristics with the information interaction to obtain output characteristics.
It should be noted that, each module in the present embodiment corresponds to each step in the first embodiment one to one, and the specific implementation process is the same, which is not described herein again.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in a hyperspectral image classification method as described in the first embodiment above.
Example four
The present embodiment provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps in the hyperspectral image classification method according to the first embodiment are implemented.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A hyperspectral image classification method is characterized by comprising the following steps:
acquiring a hyperspectral image, and preprocessing the hyperspectral image;
performing channel interaction and compression of pixel spectral dimensions and expansion and alignment of a spatial window of pixel sample spatial dimensions on the preprocessed hyperspectral image to obtain mapping characteristics;
based on the mapping characteristics, after two times of information interaction are carried out in sequence, spectrum-space characteristics are obtained;
obtaining the category of each pixel in the acquired hyperspectral image by adopting a classifier based on the spectrum-space characteristics;
the information interaction is to perform spectrum characteristic channel compression, non-local space information extraction and spectrum characteristic channel expansion on the input characteristics in sequence, and then fuse the input characteristics with the information interaction to obtain output characteristics.
2. The hyperspectral image classification method according to claim 1, wherein the channel interaction and compression of the spectral dimensions of the pixels are realized by a spectral compression group consisting of a 1 x 1 convolutional layer, a batch normalization operation and a Relu activation function;
or,
the spatial window expansion and alignment of the pixel sample spatial dimension is achieved by a spatial expansion group consisting of a 2 x 2 convolution layer, a batch normalization operation and a Relu activation function.
3. The hyperspectral image classification method according to claim 1, wherein the spectral feature channel compression is realized by a first 1 x 1 convolutional layer;
the output channel of the first 1 x 1 convolutional layer is configured with a batch normalization operation and a Relu activation function.
4. The hyperspectral image classification method according to claim 1, wherein the non-local spatial information extraction is realized by a double-headed self-attention layer;
the dot product multiplication and element-by-element addition operations of the double-headed self-attention layer configuration matrix, the Softmax activation function, and the relative position encoding computation operations.
5. The hyperspectral image classification method according to claim 1, wherein the spectral feature channel expansion is realized by a second 1 x 1 convolutional layer;
the output channel of the second 1 × 1 convolutional layer is configured with a Relu activation function.
6. The hyperspectral image classification method according to claim 5, wherein the fusing with the mapping features specifically is: and the mapping characteristic and the output of the second 1 × 1 convolutional layer are subjected to fusion addition operation and Relu activation function in sequence to obtain interactive information.
7. The hyperspectral image classification method according to claim 1, wherein the preprocessing comprises performing mean-variance normalization on each spectral dimension of all pixels in the acquired hyperspectral image.
8. A hyperspectral image classification system, comprising:
a pre-processing module configured to: acquiring a hyperspectral image, and preprocessing the hyperspectral image;
a compression expansion module configured to: performing channel interaction and compression of pixel spectral dimensions and expansion and alignment of a spatial window of pixel sample spatial dimensions on the preprocessed hyperspectral image to obtain mapping characteristics;
a self-attention module configured to: based on the mapping characteristics, after two times of information interaction are carried out in sequence, spectrum-space characteristics are obtained;
a classification module configured to: obtaining the category of each pixel in the acquired hyperspectral image by adopting a classifier based on the spectrum-space characteristics;
the information interaction is to perform spectrum characteristic channel compression, non-local space information extraction and spectrum characteristic channel expansion on the input characteristics in sequence, and then fuse the input characteristics with the information interaction to obtain output characteristics.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of a method for hyperspectral image classification according to any of the claims 1 to 7.
10. Computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor realizes the steps in a method for hyperspectral image classification according to any of the claims 1 to 7 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158771.8A CN114581789A (en) | 2022-02-21 | 2022-02-21 | Hyperspectral image classification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158771.8A CN114581789A (en) | 2022-02-21 | 2022-02-21 | Hyperspectral image classification method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114581789A true CN114581789A (en) | 2022-06-03 |
Family
ID=81770430
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210158771.8A Pending CN114581789A (en) | 2022-02-21 | 2022-02-21 | Hyperspectral image classification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114581789A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117237341A (en) * | 2023-11-13 | 2023-12-15 | 山东第一医科大学第一附属医院(山东省千佛山医院) | Human body peripheral blood sample detection method and system based on hyperspectral image |
| CN117765297A (en) * | 2023-11-20 | 2024-03-26 | 中国地质大学（武汉） | Hyperspectral image classification method, hyperspectral image classification device, hyperspectral image classification equipment and storage medium |
-
2022
- 2022-02-21 CN CN202210158771.8A patent/CN114581789A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117237341A (en) * | 2023-11-13 | 2023-12-15 | 山东第一医科大学第一附属医院(山东省千佛山医院) | Human body peripheral blood sample detection method and system based on hyperspectral image |
| CN117765297A (en) * | 2023-11-20 | 2024-03-26 | 中国地质大学（武汉） | Hyperspectral image classification method, hyperspectral image classification device, hyperspectral image classification equipment and storage medium |
| CN117765297B (en) * | 2023-11-20 | 2024-06-07 | 中国地质大学（武汉） | Hyperspectral image classification method, hyperspectral image classification device, hyperspectral image classification equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109190752B (en) | Image semantic segmentation method based on global features and local features of deep learning | |
| CN107564025B (en) | Electric power equipment infrared image semantic segmentation method based on deep neural network | |
| CN108647585B (en) | Traffic identifier detection method based on multi-scale circulation attention network | |
| CN110929736B (en) | Multi-feature cascading RGB-D significance target detection method | |
| CN111950453A (en) | Optional-shape text recognition method based on selective attention mechanism | |
| CN112489164B (en) | Image coloring method based on improved depth separable convolutional neural network | |
| CN106250931A (en) | A kind of high-definition picture scene classification method based on random convolutional neural networks | |
| CN112163447B (en) | Multi-task real-time gesture detection and recognition method based on Attention and Squeezenet | |
| CN111368637B (en) | Transfer robot target identification method based on multi-mask convolutional neural network | |
| CN113269224A (en) | Scene image classification method, system and storage medium | |
| CN114581789A (en) | Hyperspectral image classification method and system | |
| CN115908793A (en) | Coding and decoding structure semantic segmentation model based on position attention mechanism | |
| CN117372853A (en) | Underwater target detection algorithm based on image enhancement and attention mechanism | |
| CN111126155B (en) | Pedestrian re-identification method for generating countermeasure network based on semantic constraint | |
| CN111008570B (en) | Video understanding method based on compression-excitation pseudo-three-dimensional network | |
| CN115937693A (en) | Road identification method and system based on remote sensing image | |
| CN116563683A (en) | Remote sensing image scene classification method based on convolutional neural network and multi-layer perceptron | |
| CN115512100A (en) | Point cloud segmentation method, device and medium based on multi-scale feature extraction and fusion | |
| CN114463340A (en) | Edge information guided agile remote sensing image semantic segmentation method | |
| CN117173595A (en) | Unmanned aerial vehicle aerial image target detection method based on improved YOLOv7 | |
| CN116468638A (en) | Face image restoration method and system based on generation and balance countermeasure identification | |
| CN116386042A (en) | Point cloud semantic segmentation model based on three-dimensional pooling spatial attention mechanism | |
| CN115761268A (en) | Pole tower key part defect identification method based on local texture enhancement network | |
| CN114331894A (en) | Face image restoration method based on potential feature reconstruction and mask perception | |
| CN111914852B (en) | Multi-mode saliency object detection method based on coding and decoding structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |