CN114092832B - High-resolution remote sensing image classification method based on parallel hybrid convolutional network - Google Patents
High-resolution remote sensing image classification method based on parallel hybrid convolutional network Download PDFInfo
- Publication number
- CN114092832B CN114092832B CN202210065211.8A CN202210065211A CN114092832B CN 114092832 B CN114092832 B CN 114092832B CN 202210065211 A CN202210065211 A CN 202210065211A CN 114092832 B CN114092832 B CN 114092832B
- Authority
- CN
- China
- Prior art keywords
- network
- remote sensing
- sample data
- value
- resolution remote
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000012549 training Methods 0.000 claims abstract description 60
- 238000012360 testing method Methods 0.000 claims abstract description 46
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 41
- 238000006243 chemical reaction Methods 0.000 claims abstract description 19
- 230000004927 fusion Effects 0.000 claims abstract description 17
- 238000013528 artificial neural network Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 13
- 238000010606 normalization Methods 0.000 claims description 11
- 238000012795 verification Methods 0.000 claims description 11
- 238000012937 correction Methods 0.000 claims description 10
- 230000002779 inactivation Effects 0.000 claims description 9
- 230000000877 morphologic effect Effects 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 7
- 238000012805 post-processing Methods 0.000 claims description 6
- 239000000654 additive Substances 0.000 claims description 3
- 230000000996 additive effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000007797 corrosion Effects 0.000 claims description 3
- 238000005260 corrosion Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 210000002569 neuron Anatomy 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims 1
- 238000011160 research Methods 0.000 abstract description 7
- 230000003595 spectral effect Effects 0.000 abstract description 6
- 238000001228 spectrum Methods 0.000 abstract description 4
- 239000000284 extract Substances 0.000 abstract description 3
- 238000000605 extraction Methods 0.000 abstract description 3
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a high-resolution remote sensing image classification method based on a parallel hybrid convolution network, which comprises the following specific steps of: inputting a high-resolution remote sensing image and corresponding sample label data, wherein the sample label data comprises a training sample data set and a test sample data set; establishing a three-dimensional convolutional neural network and a two-dimensional convolutional neural network in parallel, and establishing an information fusion conversion network to realize space spectrum characteristic information fusion and deep extraction; inputting training sample data sets in batches to train the network, constructing a cross entropy loss function and a random gradient descent algorithm to optimize the network and updating parameters until the network converges; and inputting a test sample data set into the hybrid network model, outputting a predicted value of a test sample label, and finishing high-resolution image classification. The method can simultaneously extract the spatial characteristics and the spectral characteristics of the high-resolution remote sensing images, perform characteristic fusion to realize high-efficiency and high-precision classification of the images, and provide an important role in the researches of natural resource monitoring, general survey of geographical national conditions, urban planning, climate change and the like.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a high-resolution remote sensing image classification method based on a parallel hybrid convolution network.
Background
With the rapid development of the remote sensing satellite technology, the resolution of remote sensing images is gradually improved. The high-resolution remote sensing technology in China develops rapidly, and a high-resolution earth observation system is developed to improve the capability of acquiring high-resolution remote sensing images and promote the progress of the spatial information technology. The release of the high-resolution second-grade satellite sub-meter resolution remote sensing image indicates that the remote sensing earth observation of China enters the sub-meter age, and the spatial resolution of the sub-satellite point can reach 0.8 meter. The high-spatial-resolution remote sensing image contains abundant information such as texture, shape, structure, spectrum and the like, and provides abundant data sources for researches such as remote sensing image classification, target identification and extraction, change detection and the like. The high-resolution remote sensing image classification is taken as a current research hotspot and plays a key role in applications such as natural resource monitoring, general survey of geographical national conditions, urban planning research, environmental disaster assessment, global climate change research and the like.
The classification accuracy of visual interpretation in the traditional remote sensing image classification method is highest, but the workload is large, the efficiency is low, and visual interpreters are required to have professional knowledge and rich interpretation experience, so that the method is difficult to apply on a large scale. The development of computer technology provides a new direction for remote sensing image classification, can realize automatic classification of remote sensing images, and reduces time cost. The computer classification method can be divided into supervised classification and unsupervised classification according to whether the training samples are used or not, and a large number of researches show that the supervised classification method using the training samples has higher precision. The machine learning method uses samples for training, can realize automatic classification and improve image classification efficiency, but the traditional machine learning method only relates to low-level features. The high-resolution images have abundant space spectrum characteristics, the traditional classification method is low in precision, and a method suitable for quickly and accurately classifying the high-resolution remote sensing images needs to be developed.
The deep learning is a typical and most advanced machine learning framework developed from a traditional neural network, can extract high-level features of the remote sensing image, has remarkable advantages in multi-scale and multi-level feature extraction, and is beneficial to high-resolution remote sensing image ground feature classification. Among a plurality of deep learning models, the convolutional neural network is an efficient and widely-applied method, can learn deep features and share weights, and effectively avoids the influence of variation and noise in high-resolution images. Meanwhile, the robustness and the generalization capability of the network structure are good, and the method has a good research prospect on the high-resolution remote sensing image classification.
The current convolutional neural network for classifying high-resolution remote sensing images has higher precision than other methods, but still has some problems, and the defects are mainly expressed in two aspects. On one hand, the network structure is complex, the parameter quantity is too large, the training time is long, over-fitting or under-fitting training is easy to cause, and the classification efficiency is not high and the effect is not good. On the other hand, spectral information is not fully utilized, most of convolutional neural networks for high-resolution image classification are two-dimensional convolutional neural networks, although spatial information in an image can be well extracted, partial spectral information is lost, and the classification accuracy is low.
Disclosure of Invention
The invention provides a high-resolution remote sensing image classification method based on a parallel hybrid convolution network aiming at the defects of the existing high-resolution remote sensing image classification method, combines the advantages of three-dimensional convolution and two-dimensional convolution, extracts the deep spatial spectral features of the image and performs fusion conversion, and improves the classification precision and efficiency of the high-resolution remote sensing image.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a high-resolution remote sensing image classification method based on a parallel hybrid convolution network comprises the following steps:
step 1: and (4) preprocessing data. Inputting high-resolution remote sensing image data to be processed, carrying out preprocessing such as data normalization, extremum processing and data enhancement, and obtaining a high-resolution remote sensing image to be marked and tested;
step 2: and constructing a sample data set. Constructing a multi-class sample data set and a corresponding sample label set of the high-resolution remote sensing image, and performing data segmentation to construct three types of sample data sets: a training sample data set with a label, a verification sample data set and a test sample data set;
and step 3: and building a parallel hybrid convolutional neural network. Respectively building a three-dimensional convolution neural network, a two-dimensional convolution neural network and an information fusion conversion network, and connecting the three-dimensional convolution neural network, the two-dimensional convolution neural network and the information fusion conversion network into a parallel hybrid convolution network;
and 4, step 4: and (5) network training. Inputting training sample data sets with labels in batches into the parallel hybrid convolution network built in the step 3, guiding by label category information, training the network by using a random gradient descent algorithm and optimizing parameters, and realizing network convergence after multiple times of complete training;
and 5: and (6) network adjustment. Evaluating the network precision on the verification sample data set, and training the weight;
step 6: and classifying the test sample data set. Inputting a sample data set to be tested of the high-resolution remote sensing image based on a trained network model, obtaining a plurality of predicted probability values of each pixel in the test sample through the network model, and outputting a predicted label value of the test sample after processing;
and 7: and (5) classifying and post-processing. And removing speckle noise and smooth boundaries of the classification result by adopting a classification post-processing method of morphological open operation, and outputting a final predicted label value of the test sample data set to obtain the classification result.
Further, the specific implementation of step 1 is:
step 1.1: inputting high-resolution remote sensing image data to be processed, processing the data by using a maximum and minimum normalization method, and normalizing all pixel values to a range of 0-1, wherein the maximum value of the pixel is set to be P, and the normalization formula is as follows:
wherein,xrepresenting the pixel value of a pixel in the input high-resolution remote sensing image data;x max andx min the maximum value and the minimum value of all pixel values in the high-resolution remote sensing image are obtained;x * representing the pixel value of the pixel after the normalization of the maximum value and the minimum value;
step 1.2: the pixel with the pixel value larger than P in the original image is taken as an extreme value, the normalized value is larger than 1 after the processing of the step 1.1, the normalized value is converted into 1, and the extreme value processing of the normalized image data is realized;
step 1.3: the high-resolution remote sensing image data are input, and data enhancement and image category separability are improved through horizontal or vertical random overturning, noise adding and the like.
Further, the specific implementation of step 2 is:
step 2.1: constructing a multi-class sample data set of the high-resolution remote sensing image, and manufacturing a corresponding sample label set, wherein the label class of the sample set is C class according to GDPJ 01-2013 geographic national condition census content and indexes;
step 2.2: dividing a high-resolution remote sensing image sample data set and a sample label set into a training set part and a test set part, wherein the training set part is provided with high-resolution image data of a plurality of areas and a plurality of time phases and comprises all categories, and the size of an image division area is set to be M multiplied by M; before network training, partially dividing a training set into 95% of training sample data sets and 5% of verification sample data sets;
step 2.3: the test set part only comprises a test sample data set, which is derived from the high-resolution remote sensing image and the label data except the training set part, and the size of the test sample data set is set to be NxN.
Further, the specific implementation of step 3 is:
step 3.1: building a three-dimensional convolution neural network with a six-layer network structure: the first, third, fifth and sixth layers comprise three-dimensional convolution layers and correction linear unit layers; the second and the fourth layers only comprise three-dimensional convolution layers, and the step length is 2;
step 3.2: building a two-dimensional convolution neural network of multi-scale convolution and attention convolution: the first layer comprises a two-dimensional convolution layer and a modified linear unit layer; the second layer comprises three multi-scale two-dimensional depth separable convolution layers and a correction linear unit layer, the sizes of convolution kernels of the three multi-scale two-dimensional depth separable convolution layers and the correction linear unit layer are respectively set to be 3 multiplied by 3, 5 multiplied by 5 and 7 multiplied by 7, the convolution kernels are distributed in a parallel mode and are output through point-by-point addition layer connection; the third layer has the same structure as the first layer, and is different from the first layer in parameter setting; the fourth layer comprises an attention convolution layer, wherein the attention convolution block comprises a channel attention module, a space attention module and a global context attention block, and is connected and output by a point-by-point additive layer;
step 3.3: constructing an information fusion conversion network: the first layer comprises a characteristic information splicing layer, and the characteristic information of the three-dimensional convolutional neural network and the characteristic information of the two-dimensional convolutional neural network are fused; the second layer comprises a two-dimensional convolution layer and a modified linear unit layer; the third layer comprises a maximum pooling layer and a random inactivation layer, the size and the step length of a sampling core of the maximum pooling layer are both 2, and the inactivation rate of the random inactivation layer is 0.5; the fourth layer comprises a fully connected layer;
step 3.4: the established three-dimensional convolutional neural network and the two-dimensional convolutional neural network are distributed in a parallel mode, and the information fusion conversion network is used for connecting the first two networks to build a parallel hybrid convolutional neural network.
Further, the specific implementation of step 4 is:
step 4.1: selecting a training sample data set and corresponding sample label data of a high-resolution remote sensing image, inputting the training sample data set and the corresponding sample label data into a built hybrid convolutional neural network in batch, calculating an output value of each neuron of the convolutional neural network in a forward direction, and calculating and outputting a label prediction value of the training sample, wherein the batch size is set to be B multiplied by B;
step 4.2: calculating a loss function of the hybrid convolutional neural network and performing back propagation, wherein a calculation formula of a cross entropy loss function CELoss is as follows:
wherein,Losscalculating a loss value obtained by taking the predicted value and the tag value as input;classis a sample tag value as one of the input parameters;xis the sample label prediction value, j represents the number of categories,x[j]label prediction values representing the jth category;
step 4.3: the optimizer uses a random gradient descent algorithm to minimize a loss function and updates each parameter in the convolutional neural network, wherein the momentum parameter in the optimizer is set to be 0.8, and the learning rate is set to be 0.01; the expression formula of the update parameter of the stochastic gradient descent algorithm is as follows:
wherein, W is a model parameter,W t the network model parameters representing the t-th set,W t+1representing the t +1 th group of updated network model parameters;
is the learning rate of the convolutional neural network;
the partial derivatives of the network model parameters W are calculated by the loss functions CELoss.
Further, the specific method for network adjustment in step 5 is as follows: after each generation of training in the step 4, evaluating the precision value of the network model on the verification sample data set; then training the weight of the network, and adjusting the parameters of the network model according to the change of the classification precision of the network; and realizing network fitting convergence after multi-generation training.
Further, the specific implementation of step 6 is:
step 6.1: inputting sample data sets to be tested of the high-resolution remote sensing images into the network model in batches based on the model trained in the step 4 and the step 5, performing category prediction, and outputting prediction probability values of the batch test samples;
step 6.2: after the batch prediction of all the test sample data sets in the step 6.1, updating the prediction results, and outputting the prediction probability value of each pixel in the test sample data sets belonging to each class;
step 6.3: and based on the predicted classification probability value of the test sample, after the predicted probability value is screened, the class with the maximum probability value is regarded as the class to which the pixel belongs, dimension conversion is carried out to obtain a predicted label value of the test sample data set, and a classification result is output.
Further, the morphological open operation post-processing in step 7 specifically includes: and inputting a prediction classification result of the test sample data set, sequentially performing morphological corrosion operation and expansion operation processing, and outputting a final classification result.
Compared with the prior art, the invention has the following beneficial effects:
according to the method, the three-dimensional convolutional neural network and the two-dimensional convolutional neural network are used in a combined mode, the defect that only spatial information is processed in a traditional high-resolution image classification method is overcome, meanwhile, abundant spatial features and spectral features of the high-resolution images are extracted, the high-resolution remote sensing images are accurately classified, and training cost and time cost are reduced. Compared with the existing high-resolution remote sensing image classification method based on deep learning, the automatic classification method based on deep learning has the advantages of higher automatic classification efficiency and higher classification precision.
Drawings
FIG. 1 is a flow chart of a high-resolution remote sensing image classification method based on a parallel hybrid convolution network.
FIG. 2 is a structural diagram of a parallel hybrid convolutional neural network constructed in the present invention.
Detailed Description
The invention is described in further detail below with reference to the following figures and examples:
as shown in fig. 1, the high resolution remote sensing image classification method based on the parallel hybrid convolution network provided by the invention comprises the following implementation steps:
step 1: inputting high-resolution remote sensing image data to be processed, performing a series of data preprocessing, and obtaining a high-resolution remote sensing image to be marked and tested, wherein the specific method comprises the following steps:
step 1.1: inputting high-resolution remote sensing image data to be processed, processing the high-resolution remote sensing image data by using a maximum and minimum normalization method, normalizing all pixel values to a range of 0-1, wherein the maximum value of a pixel is set to be P, and the normalization formula is as follows:
wherein,xrepresenting the pixel value of a pixel in the input high-resolution remote sensing image data;x max andx min the maximum value and the minimum value of all pixel values in the high-resolution remote sensing image are obtained;x * representing the pixel value of the pixel after the normalization of the maximum value and the minimum value;
step 1.2: the pixel with the pixel value larger than P in the original image is taken as an extreme value, the pixel becomes a value beyond the range of 0-1 after being processed in the step 1.1, and the value is converted into 1, so that the extreme value processing of the normalized image data is realized;
step 1.3: the high-resolution remote sensing image data are input, and data enhancement and image category separability are improved through horizontal or vertical random overturning, noise adding and the like.
Step 2: constructing a multi-class sample data set and a corresponding sample label set of the high-resolution remote sensing image, and performing data segmentation to construct three types of sample data sets, wherein the specific method comprises the following steps:
step 2.1: constructing a multi-class sample data set of the high-resolution remote sensing image, and manufacturing a corresponding sample label set, wherein the label class of the sample set is C class according to GDPJ 01-2013 geographic national condition census content and indexes;
step 2.2: dividing a high-resolution remote sensing image sample data set and a sample label set into a training set part and a test set part, wherein the training set part is provided with high-resolution image data of a plurality of areas and a plurality of time phases and comprises all categories, and the size of an image division area is set to be M multiplied by M; before network training, partially dividing a training set into 95% of training sample data sets and 5% of verification sample data sets;
step 2.3: the test set part only comprises a test sample data set, which is derived from the high-resolution remote sensing image and the label data except the training set part, and the size of the test sample data set is set to be NxN.
And step 3: respectively building a three-dimensional convolution neural network, a two-dimensional convolution neural network and an information fusion conversion network, and connecting the three networks into a parallel hybrid convolution neural network, wherein the network structure is shown in figure 2, and the specific method comprises the following steps:
step 3.1: building a three-dimensional convolution neural network with a six-layer network structure: the first, third, fifth and sixth layers comprise three-dimensional convolution layers and correction linear unit layers; the second and fourth layers only contain three-dimensional convolution layers, the step length is 2, and the pooling function is realized;
step 3.2: building a two-dimensional convolution neural network of multi-scale convolution and attention convolution: the first layer comprises a two-dimensional convolution layer and a modified linear unit layer; the second layer comprises three multi-scale two-dimensional depth separable convolution layers and a correction linear unit layer, the sizes of convolution kernels of the three multi-scale two-dimensional depth separable convolution layers and the correction linear unit layer are respectively set to be 3 multiplied by 3, 5 multiplied by 5 and 7 multiplied by 7, the convolution kernels are distributed in a parallel mode and are output through point-by-point addition layer connection; the third layer has the same structure as the first layer, and is different from the first layer in parameter setting; the fourth layer comprises an attention convolution layer, wherein the attention convolution layer comprises a channel attention module, a space attention module and a global context attention module, and is connected and output by a point-by-point additive layer;
step 3.3: constructing an information fusion conversion network: the first layer comprises a characteristic information splicing layer, and the characteristic information of the three-dimensional convolutional neural network and the characteristic information of the two-dimensional convolutional neural network are fused; the second layer comprises a two-dimensional convolution layer and a correction linear unit layer for characteristic information conversion; the third layer comprises a maximum pooling layer and a random inactivation layer, the size and the step length of a sampling core of the maximum pooling layer are both 2, and the inactivation rate of the random inactivation layer is 0.5; the fourth layer comprises a fully connected layer;
step 3.4: the established three-dimensional convolutional neural network and the two-dimensional convolutional neural network are distributed in a parallel mode, and the information fusion conversion network is used for connecting the first two networks to build a parallel hybrid convolutional neural network.
And 4, step 4: inputting a training sample data set with labels in batches into the parallel hybrid convolution network established in the step 3, training the network by using a random gradient descent algorithm and optimizing parameters, and realizing network convergence after multiple times of complete training, wherein the specific method comprises the following steps:
step 4.1: selecting a training sample data set and corresponding sample label data of a high-resolution remote sensing image, inputting the training sample data set and the corresponding sample label data into a built hybrid convolutional neural network in batch, calculating an output value of each neuron in the convolutional neural network in a forward direction, and calculating and outputting a label prediction value of a training sample, wherein the batch size is set to be B multiplied by B;
step 4.2: calculating a loss function of the hybrid convolutional neural network and performing back propagation, wherein a calculation formula of a cross entropy loss function CELoss is as follows:
wherein,Losscalculating a loss value obtained by taking the predicted value and the tag value as input;classis a sample tag value as one of the input parameters;xis the sample label prediction value, j represents the number of categories,x[j]label prediction values representing the jth category;
step 4.3: the optimizer uses a random gradient descent algorithm to minimize a loss function and updates each parameter in the convolutional neural network, wherein the momentum parameter in the optimizer is 0.8, and the learning rate is set to be 0.0.1; the expression formula of the update parameter of the stochastic gradient descent algorithm is as follows:
wherein, W is a model parameter,W t the network model parameters representing the t-th set,W t+1representing the t +1 th group of updated network model parameters;
is the learning rate of the convolutional neural network;
the partial derivatives of the network model parameters W are calculated by the loss functions CELoss.
And 5: evaluating the network precision on the verification sample data set, and training the network weight, wherein the specific method comprises the following steps:
and 4, after each generation of training in the step 4, evaluating the precision value and the loss value of the network model on the verification sample data set, then training the network weight, adjusting the model parameters according to the network classification precision change, and realizing network fitting convergence after multi-generation training.
Step 6: the method comprises the following steps of inputting a sample data set to be tested of the high-resolution remote sensing image based on a trained network model, obtaining a plurality of predicted probability values of each pixel in a test sample through the network model, and outputting a classification result of the sample data set after processing, wherein the specific method comprises the following steps:
step 6.1: inputting sample data sets to be tested in the high-resolution remote sensing images into the network model in batches based on the model trained in the step 4 and the step 5, performing class prediction, and outputting class prediction probability values of the batch test samples;
step 6.2: after the batch prediction of all the test sample data sets in the step 6.1, updating the prediction result, and outputting the prediction probability value of each pixel in the test sample data sets belonging to each category;
step 6.3: and based on the predicted classification probability value of the test sample data set, after the predicted probability value is screened, the class with the maximum probability value is regarded as the class to which the pixel belongs, dimension conversion is carried out, the label predicted value of the test sample data set is obtained, and a classification result is output.
And 7: and (3) carrying out classification post-processing by adopting morphological open operation, wherein the specific method comprises the steps of inputting a prediction classification result of the test sample data set, sequentially carrying out morphological corrosion operation and expansion operation, removing speckle noise and smooth boundary in the classification result, and outputting the final prediction classification result of the test sample data set.
In the embodiment, the three-dimensional convolution and the two-dimensional convolution are connected in parallel to form the hybrid convolution neural network, so that the spatial information and the spectral information of the high-resolution remote sensing image are fully utilized, and the deep high-level features can be extracted; and the information fusion conversion network is used for completing the characteristic information fusion and conversion in the aspects of space and spectrum, further extracting the distinguishing characteristics, realizing the rapid classification of the high-resolution remote sensing images and improving the classification precision.
Although specific embodiments of the present invention have been described in detail above, it is not to be understood that the scope of the invention is limited thereby. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (7)
1. A high-resolution remote sensing image classification method based on a parallel hybrid convolution network is characterized by comprising the following steps:
step 1, inputting high-resolution remote sensing image data to be processed, carrying out data normalization and extreme value processing pretreatment, and obtaining a high-resolution remote sensing image to be marked and tested;
step 2, constructing a multi-class sample data set and a corresponding sample label set of the high-resolution remote sensing image, and performing data segmentation to construct three types of sample data sets: a training sample data set with a label, a verification sample data set and a test sample data set;
step 3, respectively building a three-dimensional convolution neural network, a two-dimensional convolution neural network and an information fusion conversion network, and connecting the three-dimensional convolution neural network, the two-dimensional convolution neural network and the information fusion conversion network into a parallel hybrid convolution network;
the specific treatment process of the step 3 is as follows;
step 3.1, building a three-dimensional convolutional neural network with a six-layer network structure: the first, third, fifth and sixth layers comprise three-dimensional convolution layers and correction linear unit layers; the second and the fourth layers only comprise three-dimensional convolution layers, and the step length is 2;
step 3.2, building a two-dimensional convolution neural network of multi-scale convolution and attention convolution: the first layer comprises a two-dimensional convolution layer and a modified linear unit layer; the second layer comprises three multi-scale two-dimensional depth separable convolution layers and a correction linear unit layer, the sizes of convolution kernels of the three multi-scale two-dimensional depth separable convolution layers and the correction linear unit layer are respectively set to be 3 multiplied by 3, 5 multiplied by 5 and 7 multiplied by 7, the convolution kernels are distributed in a parallel mode and are output through point-by-point addition layer connection; the third layer has the same structure as the first layer, and is different from the first layer in parameter setting; the fourth layer comprises an attention convolution layer, wherein the attention convolution layer comprises a channel attention module, a space attention module and a global context attention module, and is connected and output by a point-by-point additive layer;
step 3.3, constructing an information fusion conversion network: the first layer comprises a characteristic information splicing layer, and the characteristic information of the three-dimensional convolutional neural network and the characteristic information of the two-dimensional convolutional neural network are fused; the second layer comprises a two-dimensional convolution layer and a modified linear unit layer; the third layer comprises a maximum pooling layer and a random inactivation layer, the size and the step length of a sampling core of the maximum pooling layer are both 2, and the inactivation rate of the random inactivation layer is 0.5; the fourth layer comprises a fully connected layer;
step 3.4, distributing the built three-dimensional convolutional neural network and two-dimensional convolutional neural network in a parallel mode, connecting the first two networks by using an information fusion conversion network, and building a parallel hybrid convolutional neural network;
step 4, inputting training sample data sets with labels in batches into the parallel hybrid convolution network built in the step 3, guiding the training sample data sets with label type information, training the network by using a random gradient descent algorithm, optimizing parameters, and realizing network convergence after multiple times of complete training;
step 5, evaluating the network precision on the verification sample data set, and training the weight to realize network adjustment;
step 6, inputting a sample data set to be tested of the high-resolution remote sensing image based on the trained network model, obtaining a plurality of prediction probability values of each pixel in the test sample through the network model, and outputting a prediction label value of the test sample after processing;
and 7, removing speckle noise and smooth boundaries of the classification result by adopting a classification post-processing method of morphological open operation, and outputting a final predicted label value of the test sample data set to obtain the classification result.
2. The high-resolution remote sensing image classification method based on the parallel hybrid convolutional network as claimed in claim 1, which is characterized in that: the specific treatment process of the step 1 is as follows;
step 1.1, inputting high-resolution remote sensing image data to be processed, processing the data by using a maximum and minimum normalization method, and normalizing all pixel values to a range of 0-1, wherein the maximum pixel value is set as P, and a normalization formula is as follows:
wherein x represents the pixel value of a pixel in the input high-resolution remote sensing image data; x is the number ofmaxAnd xminThe maximum value and the minimum value of all pixel values in the high-resolution remote sensing image are obtained; x is the number of*Representing the pixel value of the pixel after the normalization of the maximum value and the minimum value;
step 1.2, the pixel with the pixel value larger than P in the original image is taken as an extreme value, the normalized value is larger than 1 after the processing of the step 1.1, the normalized value is converted into 1, and the extreme value processing of the normalized image data is realized;
and step 1.3, inputting the high-resolution remote sensing image data, and realizing data enhancement and increasing image category separability through horizontal or vertical random overturning and noise adding processing.
3. The high-resolution remote sensing image classification method based on the parallel hybrid convolutional network as claimed in claim 1, which is characterized in that: the specific treatment process of the step 2 is as follows;
step 2.1, constructing a multi-class sample data set of the high-resolution remote sensing image, and manufacturing a corresponding sample label set, wherein the label class of the sample set is C class according to GDPJ 01-2013 geographic national conditions general survey content and indexes;
step 2.2, dividing a high-resolution remote sensing image sample data set and a sample label set into a training set part and a test set part, wherein the training set part is provided with high-resolution image data of a plurality of areas and a plurality of time phases and comprises all categories, and the size of an image division area is set to be M multiplied by M; before network training, partially dividing a training set into 95% of training sample data sets and 5% of verification sample data sets;
and 2.3, the test set part only contains a test sample data set, the test sample data set is derived from the high-resolution remote sensing image and the label data except the training set part, and the size of the test sample data set is set to be NxN.
4. The high-resolution remote sensing image classification method based on the parallel hybrid convolutional network as claimed in claim 1, which is characterized in that: the specific treatment process of the step 4 is as follows;
step 4.1, selecting a training sample data set and corresponding sample label data of the high-resolution remote sensing image, inputting the training sample data set and the corresponding sample label data into the built parallel hybrid convolutional neural network in batch, calculating an output value of each neuron of the convolutional neural network in a forward direction, and calculating and outputting a label predicted value of the training sample, wherein the batch size is set to be BxB;
step 4.2, calculating a loss function of the hybrid convolutional neural network and performing back propagation, wherein a calculation formula of a cross entropy loss function CELoss is as follows:
wherein, Loss is calculated by taking a predicted value and a tag value as input; class is a sample tag value, as one of the input parameters; x is a sample label predicted value, j represents the number of categories, and x [ j ] represents the label predicted value of the jth category;
and 4.3, minimizing a loss function by using a random gradient descent algorithm by the optimizer, and updating each parameter in the convolutional neural network, wherein the expression formula of the updated parameter of the random gradient descent algorithm is as follows:
Wt+1＝Wt-ηtgt
wherein W is a model parameter, WtNetwork model parameters, W, representing the t-th groupt+1Representing the t +1 th group of updated network model parameters; etatIs the learning rate of the convolutional neural network; gtThe partial derivatives of the network model parameters W are calculated by the loss functions CELoss.
5. The high-resolution remote sensing image classification method based on the parallel hybrid convolutional network as claimed in claim 1, which is characterized in that: the specific method of network adjustment in step 5 is as follows: after each generation of training, evaluating the precision value of the network model on the verification sample data set, then training the weight of the network, and adjusting the parameters of the network model according to the change of the classification precision of the network; and realizing network fitting convergence after multi-generation training.
6. The high-resolution remote sensing image classification method based on the parallel hybrid convolutional network as claimed in claim 1, which is characterized in that: the specific treatment process of the step 6 is as follows;
6.1, inputting sample data sets to be tested of the high-resolution remote sensing images into a network model in batches based on the model trained in the step 4 and the step 5, performing class prediction, and outputting prediction probability values of the batch test samples;
6.2, after the batch prediction of all the test sample data sets in the step 6.1, updating the prediction result, and outputting the prediction probability value of each pixel in the test sample data sets belonging to each category;
and 6.3, screening the prediction probability value based on the prediction classification probability value of the test sample, taking the class with the maximum probability value as the class to which the pixel belongs, performing dimension conversion to obtain the prediction label value of the test sample data set, and outputting a classification result.
7. The high-resolution remote sensing image classification method based on the parallel hybrid convolutional network as claimed in claim 1, which is characterized in that: the specific method of morphological open operation classification post-processing in the step 7 is as follows: and inputting a predicted classification result of the test sample data set, sequentially performing morphological corrosion operation and expansion operation processing to obtain a predicted label value of the test sample data set, and outputting a final classification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210065211.8A CN114092832B (en) | 2022-01-20 | 2022-01-20 | High-resolution remote sensing image classification method based on parallel hybrid convolutional network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210065211.8A CN114092832B (en) | 2022-01-20 | 2022-01-20 | High-resolution remote sensing image classification method based on parallel hybrid convolutional network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114092832A CN114092832A (en) | 2022-02-25 |
| CN114092832B true CN114092832B (en) | 2022-04-15 |
Family
ID=80308905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210065211.8A Active CN114092832B (en) | 2022-01-20 | 2022-01-20 | High-resolution remote sensing image classification method based on parallel hybrid convolutional network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114092832B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114894819B (en) * | 2022-05-07 | 2023-06-23 | 中国石油大学(华东) | Satellite-borne GNSS-R soil humidity inversion method |
| CN115131680B (en) * | 2022-07-05 | 2024-08-20 | 西安电子科技大学 | Remote sensing image water body extraction method based on depth separable convolution and jump connection |
| CN115049936B (en) * | 2022-08-12 | 2022-11-22 | 深圳市规划和自然资源数据管理中心(深圳市空间地理信息中心) | High-resolution remote sensing image-oriented boundary enhanced semantic segmentation method |
| CN115249332B (en) * | 2022-09-23 | 2023-01-20 | 山东锋士信息技术有限公司 | Hyperspectral image classification method and device based on space spectrum double-branch convolution network |
| CN115661655B (en) * | 2022-11-03 | 2024-03-22 | 重庆市地理信息和遥感应用中心 | Southwest mountain area cultivated land extraction method with hyperspectral and hyperspectral image depth feature fusion |
| CN116434065B (en) * | 2023-04-19 | 2023-12-19 | 北京卫星信息工程研究所 | Water body segmentation method for panchromatic geometric correction remote sensing image |
| CN116593980B (en) * | 2023-04-20 | 2023-12-12 | 中国人民解放军93209部队 | Radar target recognition model training method, radar target recognition method and device |
| CN116503744B (en) * | 2023-06-28 | 2023-09-29 | 航天宏图信息技术股份有限公司 | Height grade-guided single-view remote sensing image building height estimation method and device |
| CN116778104B (en) * | 2023-08-16 | 2023-11-14 | 江西省国土资源测绘工程总院有限公司 | Mapping method and system for dynamic remote sensing monitoring |
| CN117036984B (en) * | 2023-10-09 | 2024-01-09 | 武汉大学 | Cascade U-shaped network cloud detection method and system integrating attention mechanisms |
| CN118230176B (en) * | 2024-05-23 | 2024-08-23 | 江西师范大学 | Wetland classification method and system based on multisource remote sensing and space-time depth integrated learning |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108510437A (en) * | 2018-04-04 | 2018-09-07 | 科大讯飞股份有限公司 | A kind of virtual image generation method, device, equipment and readable storage medium storing program for executing |
| WO2018214195A1 (en) * | 2017-05-25 | 2018-11-29 | 中国矿业大学 | Remote sensing imaging bridge detection method based on convolutional neural network |
| CN109410242A (en) * | 2018-09-05 | 2019-03-01 | 华南理工大学 | Method for tracking target, system, equipment and medium based on double-current convolutional neural networks |
| CN110334765A (en) * | 2019-07-05 | 2019-10-15 | 西安电子科技大学 | Remote Image Classification based on the multiple dimensioned deep learning of attention mechanism |
| CN111898543A (en) * | 2020-07-31 | 2020-11-06 | 武汉大学 | Building automatic extraction method integrating geometric perception and image understanding |
| CN112232280A (en) * | 2020-11-04 | 2021-01-15 | 安徽大学 | Hyperspectral image classification method based on self-encoder and 3D depth residual error network |
| CN112560894A (en) * | 2020-11-16 | 2021-03-26 | 武汉光谷信息技术股份有限公司 | Improved 3D convolutional network hyperspectral remote sensing image classification method and device |
| CN112836666A (en) * | 2021-02-19 | 2021-05-25 | 河海大学 | Hyperspectral image classification and identification method |
| CN112883900A (en) * | 2021-03-12 | 2021-06-01 | 中科三清科技有限公司 | Method and device for bare-ground inversion of visible images of remote sensing images |
| CN113011499A (en) * | 2021-03-22 | 2021-06-22 | 安徽大学 | Hyperspectral remote sensing image classification method based on double-attention machine system |
| CN113361355A (en) * | 2021-05-28 | 2021-09-07 | 同济大学 | Remote sensing image earth surface element identification method combining multi-temporal characteristics and storage medium |
| CN113486859A (en) * | 2021-08-03 | 2021-10-08 | 张博宙 | Human behavior identification method based on multi-class spectrogram and composite convolutional neural network |
| CN113705526A (en) * | 2021-09-07 | 2021-11-26 | 安徽大学 | Hyperspectral remote sensing image classification method |
-
2022
- 2022-01-20 CN CN202210065211.8A patent/CN114092832B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018214195A1 (en) * | 2017-05-25 | 2018-11-29 | 中国矿业大学 | Remote sensing imaging bridge detection method based on convolutional neural network |
| CN108510437A (en) * | 2018-04-04 | 2018-09-07 | 科大讯飞股份有限公司 | A kind of virtual image generation method, device, equipment and readable storage medium storing program for executing |
| CN109410242A (en) * | 2018-09-05 | 2019-03-01 | 华南理工大学 | Method for tracking target, system, equipment and medium based on double-current convolutional neural networks |
| CN110334765A (en) * | 2019-07-05 | 2019-10-15 | 西安电子科技大学 | Remote Image Classification based on the multiple dimensioned deep learning of attention mechanism |
| CN111898543A (en) * | 2020-07-31 | 2020-11-06 | 武汉大学 | Building automatic extraction method integrating geometric perception and image understanding |
| CN112232280A (en) * | 2020-11-04 | 2021-01-15 | 安徽大学 | Hyperspectral image classification method based on self-encoder and 3D depth residual error network |
| CN112560894A (en) * | 2020-11-16 | 2021-03-26 | 武汉光谷信息技术股份有限公司 | Improved 3D convolutional network hyperspectral remote sensing image classification method and device |
| CN112836666A (en) * | 2021-02-19 | 2021-05-25 | 河海大学 | Hyperspectral image classification and identification method |
| CN112883900A (en) * | 2021-03-12 | 2021-06-01 | 中科三清科技有限公司 | Method and device for bare-ground inversion of visible images of remote sensing images |
| CN113011499A (en) * | 2021-03-22 | 2021-06-22 | 安徽大学 | Hyperspectral remote sensing image classification method based on double-attention machine system |
| CN113361355A (en) * | 2021-05-28 | 2021-09-07 | 同济大学 | Remote sensing image earth surface element identification method combining multi-temporal characteristics and storage medium |
| CN113486859A (en) * | 2021-08-03 | 2021-10-08 | 张博宙 | Human behavior identification method based on multi-class spectrogram and composite convolutional neural network |
| CN113705526A (en) * | 2021-09-07 | 2021-11-26 | 安徽大学 | Hyperspectral remote sensing image classification method |
Non-Patent Citations (2)
| Title |
|---|
| Classification of hyperspectral remote sensing images using different dimension reduction methods with 3D/2D CNN;Huseyin Firat等;《Remote Sensing Applications:Society and Environment 25（2022）》;20220111;第1-17页 * |
| 关节点时空信息融合降维的人体动作识别方法;李博文等;《微电子学与计算机》;20210924;第1-4页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114092832A (en) | 2022-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114092832B (en) | High-resolution remote sensing image classification method based on parallel hybrid convolutional network | |
| CN110533631B (en) | SAR image change detection method based on pyramid pooling twin network | |
| CN112347970B (en) | Remote sensing image ground object identification method based on graph convolution neural network | |
| CN115063573B (en) | Multi-scale target detection method based on attention mechanism | |
| CN105975573B (en) | A kind of file classification method based on KNN | |
| CN113033520B (en) | Tree nematode disease wood identification method and system based on deep learning | |
| CN108052966A (en) | Remote sensing images scene based on convolutional neural networks automatically extracts and sorting technique | |
| CN112950780B (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN106295613A (en) | A kind of unmanned plane target localization method and system | |
| CN116206185A (en) | Lightweight small target detection method based on improved YOLOv7 | |
| CN106408030A (en) | SAR image classification method based on middle lamella semantic attribute and convolution neural network | |
| CN111178304B (en) | High-resolution remote sensing image pixel level interpretation method based on full convolution neural network | |
| CN111612051A (en) | Weak supervision target detection method based on graph convolution neural network | |
| Chen et al. | Agricultural remote sensing image cultivated land extraction technology based on deep learning | |
| CN111652835A (en) | Method for detecting insulator loss of power transmission line based on deep learning and clustering | |
| CN114693064B (en) | Building group scheme generation performance evaluation method | |
| CN111461121A (en) | Electric meter number identification method based on YO L OV3 network | |
| CN114283285A (en) | Cross consistency self-training remote sensing image semantic segmentation network training method and device | |
| CN113111716A (en) | Remote sensing image semi-automatic labeling method and device based on deep learning | |
| CN115759389A (en) | Day-ahead photovoltaic power prediction method based on weather type similar day combination strategy | |
| CN114359702A (en) | Method and system for identifying building violation of remote sensing image of homestead based on Transformer | |
| CN117152427A (en) | Remote sensing image semantic segmentation method and system based on diffusion model and knowledge distillation | |
| CN115690549A (en) | Target detection method for realizing multi-dimensional feature fusion based on parallel interaction architecture model | |
| CN111242028A (en) | Remote sensing image ground object segmentation method based on U-Net | |
| Qin et al. | A wireless sensor network location algorithm based on insufficient fingerprint information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |