[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN115375548A - Super-resolution remote sensing image generation method, system, equipment and medium - Google Patents

Super-resolution remote sensing image generation method, system, equipment and medium Download PDF

Info

Publication number
CN115375548A
CN115375548A CN202211043000.0A CN202211043000A CN115375548A CN 115375548 A CN115375548 A CN 115375548A CN 202211043000 A CN202211043000 A CN 202211043000A CN 115375548 A CN115375548 A CN 115375548A
Authority
CN
China
Prior art keywords
image
resolution
low
model
super
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211043000.0A
Other languages
Chinese (zh)
Inventor
叶武剑
林振溢
刘怡俊
劳俊明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202211043000.0A priority Critical patent/CN115375548A/en
Publication of CN115375548A publication Critical patent/CN115375548A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a method, a system, equipment and a medium for generating a remote sensing image with super-resolution, wherein the method comprises the following steps: responding to a model construction request sent by a demand end, constructing an initial image generation model and obtaining an image identification model, obtaining a low-resolution sample image and a corresponding high-resolution sample image, processing the low-resolution sample image through the initial image generation model to generate an initial super-resolution image, adopting the initial super-resolution image and the high-resolution sample image to construct a sample image pair to be input into the image identification model, determining an overall loss function value, carrying out iterative optimization on model parameters of the numerical initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model, and inputting the low-resolution remote sensing image into the target image generation model to obtain the target super-resolution image. The super-resolution remote sensing image obtained by the target image generation model has the advantages of clearer and smoother imaging at texture details and better image quality.

Description

Super-resolution remote sensing image generation method, system, equipment and medium
Technical Field
The invention relates to the technical field of image processing, in particular to a method, a system, equipment and a medium for generating a super-resolution remote sensing image.
Background
The remote sensing image has abundant detail information and perception information, and has wide application in the fields of disaster monitoring, resource exploration and the like, and the remote sensing image obtained from the remote sensing satellite at present is often low in resolution and cannot meet the application requirement of the quality of the remote sensing image. The image super-resolution aims to restore the low-resolution image into a corresponding high-resolution image with higher visual quality through a certain algorithm, and the obtained remote sensing image can be optimized under the existing remote sensing satellite system, so that the quality of the remote sensing image is improved.
The image super-resolution method based on deep learning extracts effective high-frequency features of images by training and learning in a large number of low-resolution and high-resolution images, solves the bottleneck that deep features of images are difficult to learn by a traditional method, but the obtained images still have a large difference from actual requirements and are mainly reflected in lack of details and strong fuzzy sense of reconstructed images.
With the rapid development of big data, the generation of confrontation networks becomes one of the most popular deep learning algorithms at present, and the existing remote sensing image super-resolution method based on the GAN can learn local details by utilizing the confrontation characteristics of the GAN so as to generate images with more reality. At present, a mainstream remote sensing image super-resolution network is nESRGAN +, which has a certain effect on reconstruction effect, but the image quality of the generated super-resolution image still needs to be optimized.
Disclosure of Invention
The invention provides a method, a system, equipment and a medium for generating a remote sensing image with super-resolution, which solve the technical problem that the image quality of the generated super-resolution image is not high when the remote sensing image is processed by the existing remote sensing image super-resolution network.
The invention provides a super-resolution remote sensing image generation method, which comprises the following steps:
responding to the model construction request, constructing an initial image generation model and acquiring an image identification model;
acquiring a low-resolution sample image and a corresponding high-resolution sample image;
inputting the low-resolution sample image into the initial image generation model to generate an initial super-resolution image;
constructing a sample image pair by using the initial super-resolution image and the high-resolution sample image, inputting the sample image pair into the image identification model, and determining an overall loss function value;
performing iterative optimization on model parameters in the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model;
and when receiving the low-resolution remote sensing image, inputting the low-resolution remote sensing image into the target image generation model, and outputting a target super-resolution image.
Optionally, the step of constructing an initial image generation model and acquiring an image identification model in response to the model construction request includes:
responding to the model construction request, and sequentially connecting the feature extraction convolution layer and the LeakyReLU activation layer to generate a low-layer feature extraction module;
the method comprises the following steps of constructing a high-level feature extraction module by serially connecting ten COT _ RDB modules consisting of COTNet-Conv-LeakyReLU layers, wherein residual errors are connected between every two COT _ RDB modules;
two Conv-PixelShuffle-LeakyReLU layers are connected in series to construct an up-sampling module;
and the low-layer feature extraction module, the high-layer feature extraction module, the up-sampling module and the image reconstruction convolution module are sequentially connected to construct an initial image generation model, and an image identification model constructed by the BoTNet-50 module is obtained.
Optionally, the step of inputting the low resolution sample image into the initial image generation model to generate an initial super-resolution image includes:
inputting the low-resolution sample image into the image generation model, extracting low-layer features of the low-resolution sample image through the feature extraction convolutional layer, and performing nonlinear mapping on the low-layer features through the LeakyReLU activation layer to generate a low-layer feature map;
successively extracting high-level features of the low-level feature map by adopting the COT _ RDB module, and mapping and outputting to obtain a high-level feature map;
continuously upsampling the high-level feature map through the upsampling module to obtain an upsampling map meeting the target size;
and mapping the low-layer features and the high-layer features to the up-sampling image by using an image reconstruction convolution module to generate an initial super-resolution image.
Optionally, the step of constructing a sample image pair using the initial super-resolution image and the high-resolution sample image, and inputting the sample image pair to the image identification model, and determining the overall loss function value includes:
constructing a sample image sample pair by using the low-resolution image and the super-resolution image;
inputting the sample image pair into the image identification model, calculating the true probability of the image and acquiring edge information;
acquiring a feature extraction network, and respectively extracting features of the image pair on different convolution layers by using the feature extraction network to obtain total target perception information, background texture information and corner information which respectively correspond to the different convolution layers;
inputting the edge information, the total target perception information, the background texture information, the corner information and the image true probability into a preset integral loss function model for simulation calculation by combining a preset information characteristic coefficient and a preset loss function balance coefficient to obtain an integral loss function value;
the overall loss function model is as follows:
Figure BDA0003821531660000031
wherein,
Figure BDA0003821531660000032
Figure BDA0003821531660000033
Figure BDA0003821531660000034
Figure BDA0003821531660000035
Figure BDA0003821531660000036
Figure BDA0003821531660000037
in the form of an overall loss of energy,
Figure BDA0003821531660000038
in order to sense the loss of power,
Figure BDA0003821531660000039
in order for the generator to combat the loss,
Figure BDA00038215316600000310
in order to minimize the absolute error of the signal,
Figure BDA00038215316600000311
is edge loss, lambda is the balance coefficient of the generator against loss, eta is the balance coefficient of the minimum absolute error, mu is the balance coefficient of edge loss,
Figure BDA00038215316600000312
the overall target perception information is characterized in that,
Figure BDA00038215316600000313
the information characterizing the texture of the background is,
Figure BDA00038215316600000314
representing corner information, alpha is a characteristic coefficient of overall target perception information, beta is a characteristic coefficient of background texture information, gamma is a characteristic coefficient of the corner information, N is the number of samples, i is the ith sample,
Figure BDA00038215316600000315
is a low resolution image of the ith sample, theta G Model parameters of the model are generated for the image,
Figure BDA00038215316600000316
is a super-resolution image obtained by performing image generation on the low-resolution image of the ith sample, theta D The model parameters of the model are identified for the image,
Figure BDA00038215316600000317
the image true probability of the super-resolution image generated for the low-resolution image of the ith sample,
Figure BDA00038215316600000318
a high resolution image representing the ith sample,
Figure BDA00038215316600000319
in order to extract the operator for the edge,
Figure BDA00038215316600000320
for the edge information of the high resolution image of the mth channel of the ith sample,
Figure BDA00038215316600000321
is the edge information of the super-resolution image of the ith sample channel, M is the number of image channels, M is the mth channel,
Figure BDA00038215316600000322
for the high resolution image of the mth channel of the ith sample,
Figure BDA00038215316600000323
is a super-resolution image of the m channel of the ith sample.
Optionally, the step of performing iterative optimization on the model parameters in the initial image generation model according to the overall loss function value until the overall loss function value converges to obtain a target image generation model includes:
carrying out convergence judgment on the overall loss function value;
if the overall loss function value is not converged, analyzing the overall loss function value to obtain a characteristic error of the image pair;
minimizing the characteristic error, iteratively updating model parameters in the initial image generation model in a back propagation mode, and skipping to execute the step of acquiring the low-resolution sample image and the corresponding high-resolution sample image;
and if the overall loss function value is converged, outputting a target image generation model according to the current model parameters.
Optionally, when a low-resolution remote sensing image is received, the step of inputting the low-resolution remote sensing image to the target image generation model and outputting a target super-resolution image includes:
when receiving a low-resolution remote sensing image, inputting the low-resolution remote sensing image into the target image generation model;
extracting target low-level features and target high-level features of the low-resolution remote sensing image through the target image generation model, and generating a target up-sampling image;
and mapping the target low-layer features and the target high-layer features to the target up-sampling graph by using the target image generation model, and outputting a target super-resolution image.
The second aspect of the invention provides a super-resolution remote sensing image generation system, which comprises:
the model construction module is used for responding to the model construction request, constructing an initial image generation model and acquiring an image identification model;
the sample image acquisition module is used for acquiring a low-resolution sample image and a corresponding high-resolution sample image;
the initial super-resolution image generation module is used for inputting the low-resolution sample image into the initial image generation model to generate an initial super-resolution image;
the loss function calculation module is used for constructing a sample image pair by adopting the initial super-resolution image and the high-resolution sample image, inputting the sample image pair into the image identification model and determining an integral loss function value;
the model optimization module is used for carrying out iterative optimization on model parameters in the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model;
and the target super-resolution image generation module is used for inputting the low-resolution remote sensing image into the target image generation model and outputting the target super-resolution image when the low-resolution remote sensing image is received.
Optionally, the model building module is specifically configured to:
responding to the model construction request, and sequentially connecting the feature extraction convolution layer and the LeakyReLU activation layer to generate a low-layer feature extraction module;
the method comprises the following steps of constructing a high-level feature extraction module by serially connecting ten COT _ RDB modules consisting of COTNet-Conv-LeakyReLU layers, wherein residual connection is arranged between every two COT _ RDB modules;
two Conv-PixelShuffle-LeakyReLU layers are connected in series to construct an up-sampling module;
and sequentially connecting the low-layer feature extraction module, the high-layer feature extraction module, the upsampling module and the image reconstruction convolution module to construct an initial image generation model and obtain an image identification model constructed by the BoTNet-50 module.
A third aspect of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to execute the steps of the super-resolution remote sensing image generation method according to any one of the first aspect of the present invention.
A fourth aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, wherein the computer program is configured to implement the method for generating a super-resolution remote sensing image according to any one of the first aspect of the present invention when executed.
According to the technical scheme, the invention has the following advantages:
the method comprises the steps of constructing an initial image generation model and obtaining an image identification model by responding to a model construction request sent by a demand end, then obtaining a low-resolution sample image serving as a training sample and a corresponding high-resolution sample image, processing the low-resolution sample image through the initial image generation model to generate an initial super-resolution image, adopting the initial super-resolution image and the high-resolution sample image to construct a sample image, inputting the sample image into the image identification model, determining an overall loss function value, carrying out iterative optimization on model parameters of the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model, and inputting a low-resolution remote sensing image into the target image generation model to obtain a target super-resolution image. The super-resolution remote sensing image obtained by the target image generation model has the advantages of clearer and smoother imaging at texture details and better image quality.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a flowchart illustrating steps of a super-resolution remote sensing image generation method according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating steps of a super-resolution remote sensing image generation method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of an image generation model according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a high-level feature extraction module according to a second embodiment of the present invention;
fig. 5 is a block diagram of a super-resolution remote sensing image generation system according to a third embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a system, equipment and a medium for generating a remote sensing image with super-resolution, which are used for solving the technical problem that the image quality of the generated super-resolution image is not high when the remote sensing image is subjected to image processing by the existing remote sensing image super-resolution network.
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating steps of a method for generating a super-resolution remote sensing image according to an embodiment of the present invention.
The invention provides a super-resolution remote sensing image generation method, which comprises the following steps:
and step 101, responding to a model construction request, constructing an initial image generation model and acquiring an image identification model.
The model building request refers to request information for building an image generation model sent by a demand side platform of the remote sensing image generation application capable of supporting super-resolution, and specifically comprises a building instruction for building the image generation model and composition module information of the image generation model.
The initial image generation model refers to an initial model which is not trained and is not subjected to iterative optimization, and the initial model can be used for generating a super-resolution remote sensing image.
The image identification model is used for identifying the degree of consistency of image data distribution of the super-resolution image generated by the image generation model and the remote sensing image serving as the real image.
In the embodiment of the invention, when a model construction request sent by a demand side platform of any remote sensing image generation application supporting super-resolution is received, the model construction request can be analyzed to obtain the information of the composition modules of the image generation model, and the initial image generation model is started to be constructed according to the construction instruction and the requirements of the information of the composition modules of the image generation model.
Step 102, acquiring a low-resolution sample image and a corresponding high-resolution sample image.
The low-resolution sample image and the high-resolution sample image are pairs of a low-resolution image and a high-resolution image which are training samples of the remote sensing image.
In the embodiment of the invention, after a high-resolution sample image serving as a training sample of the remote sensing image is obtained, the high-resolution sample image is subjected to downsampling preprocessing to obtain a corresponding low-resolution sample image.
Preferably, the down-sampling is performed by bicubic down-sampling.
And 103, inputting the low-resolution sample image into an initial image generation model to generate an initial super-resolution image.
The initial super-resolution image is a super-resolution image corresponding to the low-resolution sample image generated in the initial image generation model.
In the embodiment of the invention, the acquired low-resolution sample image is input into the initial image generation model, the image characteristics of the low-resolution sample image are extracted through the initial image generation model, the image characteristics are mapped and subjected to up-sampling processing, and the initial super-resolution image is generated.
And 104, constructing a sample image pair by using the initial super-resolution image and the high-resolution sample image, inputting the sample image pair into an image identification model, and determining an integral loss function value.
The sample image pair refers to an image combination composed of a high-resolution sample image and a super-resolution image generated from a low-resolution sample image.
The overall loss function value refers to a simulation calculation result of the overall loss function model.
In the embodiment of the invention, the initial super-resolution image and the high-resolution sample image are input to the image identification model in the form of an image pair, the image identification model is used for judging the distribution consistency degree of the image data of the initial super-resolution image and the high-resolution sample image, and the characteristics extracted in the image identification model processing process are input to the preset integral loss function model for calculation according to the sample image, so that the integral loss function value is determined.
And 105, performing iterative optimization on model parameters in the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model.
The model parameters include the weight and bias of the model.
The target image generation model refers to a target model which is used for training an initial image generation model and performing iterative optimization to reach a preset target, and the target model can be used for generating a super-resolution remote sensing image.
In the embodiment of the invention, according to the characteristic error condition reflected by the overall loss function value, the weight and the bias in the initial image generation model are iteratively updated by minimizing the characteristic error until the overall loss function value is converged, and the target image generation model is obtained.
It can be understood that, in the training process, the image generation model and the image identification model can play mutually, and when the overall loss function value converges, the image generation model and the image identification model basically reach nash balance.
And 106, when the low-resolution remote sensing image is received, inputting the low-resolution remote sensing image into the target image generation model, and outputting the target super-resolution image.
The low-resolution remote sensing image is a low-resolution image obtained by preprocessing an image acquired by a remote sensing satellite.
The target super-resolution image is a super-resolution image corresponding to a low-resolution remote sensing image generated in the target image generation model.
In the embodiment of the invention, after a high-resolution remote sensing image is obtained from a remote sensing satellite, downsampling pretreatment is carried out on the high-resolution remote sensing image to obtain a corresponding low-resolution remote sensing image, the low-resolution remote sensing image is input into a target image generation model, the image characteristics of the low-resolution remote sensing image are extracted through the target image generation model, the image characteristics are mapped and upsampled, and a target super-resolution image is generated.
In the embodiment of the invention, an initial image generation model is constructed and an image identification model is obtained by responding to a model construction request sent by a demand end, then a low-resolution sample image and a corresponding high-resolution sample image which are used as training samples are obtained, the low-resolution sample image is processed by the initial image generation model to generate an initial super-resolution image, the initial super-resolution image and the high-resolution sample image are adopted to construct a sample image and are input to the image identification model, an overall loss function value is determined, model parameters of the initial image generation model are subjected to iterative optimization according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model, and a low-resolution remote sensing image is input to the target image generation model to obtain a target super-resolution image. The super-resolution remote sensing image obtained by the target image generation model has the advantages of clearer and smoother imaging at texture details and better image quality.
Referring to fig. 2, fig. 2 is a flowchart illustrating steps of a method for generating a super-resolution remote sensing image according to a second embodiment of the present invention.
The invention provides a super-resolution remote sensing image generation method, which comprises the following steps:
step 201, responding to a model building request, building an initial image generation model and obtaining an image identification model.
Optionally, step 201 comprises the sub-steps of:
responding to the model construction request, and sequentially connecting the feature extraction convolution layer and the LeakyReLU activation layer to generate a low-layer feature extraction module;
the method comprises the following steps of constructing a high-level feature extraction module by serially connecting ten COT _ RDB modules consisting of COTNet-Conv-LeakyReLU layers, wherein residual errors are connected between every two COT _ RDB modules;
two Conv-PixelShuffle-LeakyReLU layers are connected in series to construct an up-sampling module;
and sequentially connecting the low-layer feature extraction module, the high-layer feature extraction module, the up-sampling module and the image reconstruction convolution module to construct an initial image generation model and obtain an image identification model constructed by the BoTNet-50 module.
As shown in fig. 3, in the embodiment of the present invention, the image generation model includes four different modules, which are a low-level feature extraction module, a high-level feature extraction module, an upsampling module, and an image reconstruction convolution module. Where, conv is a convolution layer, leakrelu is an active layer, pixelShuffle is a pixel reconstruction layer, and k, n, and s respectively represent convolution kernel size, channel number, and sliding step size of the convolution layer, for example, a convolution layer of k9n64s1 represents a convolution layer of 3 × 3 convolution kernels, 64 output channels, and step size 1. The low-layer feature extraction module is composed of a convolution layer of k9n64s1 and a LeakyReLU activation layer. The high-level feature extraction module comprises ten COT _ RDB modules, and the convolution layer in the COT _ RDB module is a k3n64s1 convolution layer, so that the high-level feature extraction module can consider the global information characteristic of the image and also has local area correlation brought by the convolution local receptive field. The upsampling module comprises two upsampling sub-modules, each of which consists of a convolution layer of k3n256s1, a PixelShuffle pixel reconstruction layer and a LeakyReLU activation layer. The image reconstruction convolution module consists of a convolution layer of k9n64s 1.
As shown in fig. 4, in the embodiment of the present invention, each COT _ RDB module includes a COTNet layer, a convolutional layer, and an leakyreu active layer, which are sequentially connected, where a residual connection is provided between each two COT _ RDB modules, the COTNet layer carries a self-attention mechanism, and the residual connection can improve and ensure identity mapping capability of a network, and promote aggregation of features of different layers.
The BOTNet-50 module comprises a plurality of residual blocks containing convolution, so that the image identification model has the feature extraction capability, and also comprises a multi-head self-attention mechanism, so that the image identification model can better pay attention to details in the image when extracting features.
Step 202, a low resolution sample image and a corresponding high resolution sample image are acquired.
In the embodiment of the present invention, the specific implementation process of step 201 is similar to that of step 102, and is not described herein again.
And 203, inputting the low-resolution sample image into the initial image generation model to generate an initial super-resolution image.
Optionally, step 203 comprises the sub-steps of:
inputting the low-resolution sample image into an image generation model, extracting low-layer features of the low-resolution sample image through a feature extraction convolution layer, and performing nonlinear mapping on the low-layer features through a LeakyReLU activation layer to generate a low-layer feature map;
successively extracting high-level features of the low-level feature map by adopting a COT _ RDB module, and mapping and outputting to obtain a high-level feature map;
continuously upsampling the high-level characteristic diagram through an upsampling module to obtain an upsampling diagram meeting the target size;
and mapping the low-layer features and the high-layer features to the upper sampling map by using an image reconstruction convolution module to generate an initial super-resolution image.
The low-level features refer to low-level features of the image, and include, but are not limited to, features such as color, lines, texture, and the like of the image. The low-level feature map refers to an image carrying low-level features.
The high-level features refer to deep-level features of the image, including but not limited to more detailed color, line, texture, and other features of the image. The high-level feature map refers to an image carrying low-level features and high-level features.
The target size refers to the size of the high resolution image corresponding to the low resolution image. The upsampled map refers to an upsampled image carrying low-level features and high-level features.
In the embodiment of the invention, the acquired low-resolution sample image is input into an initial image generation model, the low-layer features of the low-resolution sample image are extracted through a feature convolution layer, a LeakyReLU activation layer performs nonlinear mapping on the low-layer features, and a low-layer feature map is output to a COT _ RDB module. And while keeping the low-level features, sequentially extracting deeper features from the low-level feature map by a COT _ RDB module under a COTNet self-attention mechanism, and outputting a high-level feature map to an upper sampling module. Because the image size is gradually reduced in the processes of obtaining the low-resolution sample image from the low-resolution sample image by the down-sampling processing of the high-resolution sample image and extracting the features of the low-resolution sample image by the low-layer feature extraction module and the high-layer feature extraction module, in order to compare the super-resolution image generated by the image generation model with the high-resolution sample image, the two up-sampling sub-modules are used for carrying out continuous up-sampling to obtain the up-sampling image with the same size as the high-resolution sample image. And finally, the low-layer features and the high-layer features are mapped to the pixel points of the upper sampling graph in a supplementing manner through linear transformation by using an image reconstruction convolution module, and the image after feature fusion is the initial super-resolution image.
And step 204, constructing a sample image pair by adopting the initial super-resolution image and the high-resolution sample image, inputting the sample image pair into an image identification model, and determining an integral loss function value.
Optionally, step 204 comprises the sub-steps of:
constructing a sample image sample pair by adopting the low-resolution image and the super-resolution image;
inputting the sample image pair into an image identification model, calculating the true probability of the image and acquiring edge information;
acquiring a feature extraction network, and respectively extracting features of the image pair on different convolution layers by using the feature extraction network to obtain total target perception information, background texture information and corner information which respectively correspond to the different convolution layers;
edge information, overall target perception information, background texture information, corner information and image true probability are adopted, preset information characteristic coefficients and preset loss function balance coefficients are combined, the input is carried out on a preset overall loss function model for simulation calculation, and an overall loss function value is obtained.
The overall loss function model is:
Figure BDA0003821531660000111
wherein,
Figure BDA0003821531660000112
Figure BDA0003821531660000113
Figure BDA0003821531660000121
Figure BDA0003821531660000122
Figure BDA0003821531660000123
Figure BDA0003821531660000124
in the form of an overall loss of energy,
Figure BDA0003821531660000125
in order to sense the loss of power,
Figure BDA0003821531660000126
in order for the generator to combat the loss,
Figure BDA0003821531660000127
in order to minimize the absolute error of the signal,
Figure BDA0003821531660000128
is edge loss, lambda is the balance coefficient of the generator against loss, eta is the balance coefficient of the minimum absolute error, mu is the balance coefficient of edge loss,
Figure BDA0003821531660000129
the overall target perception information is characterized in that,
Figure BDA00038215316600001210
the information characterizing the texture of the background is,
Figure BDA00038215316600001211
representing corner information, alpha is a characteristic coefficient of overall target perception information, beta is a characteristic coefficient of background texture information, gamma is a characteristic coefficient of the corner information, N is the number of samples, i is the ith sample,
Figure BDA00038215316600001212
is a low resolution image of the ith sample, theta G Model parameters of the model are generated for the image,
Figure BDA00038215316600001213
is a super-resolution image obtained by performing image generation on the low-resolution image of the ith sample, theta D The model parameters of the model are identified for the image,
Figure BDA00038215316600001214
the image true probability of the super-resolution image generated for the low-resolution image of the ith sample,
Figure BDA00038215316600001215
a high resolution image representing the ith sample,
Figure BDA00038215316600001216
in order to extract the operator for the edge,
Figure BDA00038215316600001217
edge information of the high resolution image of the mth channel for the ith sample,
Figure BDA00038215316600001218
is the edge information of the super-resolution image of the ith sample channel, M is the number of image channels, M is the mth channel,
Figure BDA00038215316600001219
for the high resolution image of the mth channel of the ith sample,
Figure BDA00038215316600001220
is a super-resolution image of the m channel of the ith sample.
Preferably, the balance coefficient of edge loss is set to 0.001.
The image true probability is a probability that the super-resolution image generated by the image generation model is determined as a remote sensing image corresponding to the true image in the image discrimination model.
The edge information is information describing a boundary feature of an image extracted by an edge extraction operator.
Overall object perception information focuses on completeness and refers to image information of each distinct object that makes up an image.
Background texture information focuses on more detailed parts of an image, such as a large leaf of a tree in an image, which usually looks more complex and detailed to the eye.
The corner information is focused on the line information of each target in the image and is closely related to the image sharpening degree.
And the information characteristic coefficient and the balance coefficient are used for adjusting the proportion of each loss in the total loss and influencing the performance of the model.
The feature extraction network is a trained VGG19 network and comprises a plurality of convolution-activation function-pooling layers, each convolution layer utilizes the output of the previous layer to further extract more complex features, and each layer can be regarded as a plurality of extractors of local features.
In the embodiment of the invention, the sample image pair is input to the image identification model, the image true probability is calculated according to the difference degree of the image pair on the image characteristic data distribution, and the edge information is obtained based on the image identification model. And acquiring a feature extraction network, extracting features obtained by a convolutional layer before an activation function by using the feature extraction network, and respectively extracting front, middle and rear features as shallow layer, deeper layer and deep layer pre-activation features, wherein the shallow layer, deeper layer and deep layer pre-activation features respectively correspond to the general target perception information, the background texture information and the corner information. And performing simulation calculation on the overall loss function model by adopting edge information, overall target perception information, background texture information, corner information and image true probability and combining a preset information characteristic coefficient and a preset loss function balance coefficient to obtain an overall loss function value.
And step 205, performing iterative optimization on model parameters in the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain the target image generation model.
Optionally, step 205 comprises the sub-steps of:
carrying out convergence judgment on the overall loss function value;
if the overall loss function value is not converged, analyzing the overall loss function value to obtain a characteristic error of the image pair;
minimizing the characteristic error, iteratively updating model parameters in the initial image generation model in a back propagation mode, and skipping to execute the step 202;
and if the overall loss function value is converged, outputting the target image generation model by using the current model parameters.
The characteristic error refers to a region where the high-resolution image and the super-resolution image differ in the distribution of characteristic data.
In the embodiment of the present invention, convergence determination is performed on the overall loss function value obtained in each training, and if the overall loss function value is not converged, the loss included in the overall loss function value is analyzed to obtain the characteristic error of the image pair. And with the minimized characteristic error as a target, feeding back the information of the characteristic error to the image generation model in a back propagation mode, adjusting the model parameters, skipping to execute the step 202, continuously performing iterative optimization on the image generation model until the overall loss function value is converged, and outputting the target image generation model by using the current model parameters.
And step 206, inputting the low-resolution remote sensing image into the target image generation model when receiving the low-resolution remote sensing image.
In the embodiment of the invention, after the high-resolution remote sensing image is obtained from the remote sensing satellite, the low-resolution remote sensing image is obtained by down-sampling pretreatment of the high-resolution remote sensing image, the low-resolution remote sensing image is input into the target image generation model,
and step 207, extracting target low-level features and target high-level features of the low-resolution remote sensing image through the target image generation model, and generating a target up-sampling image.
The target low-level features refer to low-level features of the target remote sensing image, and include but are not limited to features such as color, lines, textures and the like of the image.
The target high-level features refer to high-level features of the target remote sensing image, and include but are not limited to more detailed features such as colors, lines and textures of the image.
The target up-sampling image is an up-sampled image carrying target low-level features and target high-level features.
In the embodiment of the invention, the low-resolution remote sensing image is input into a target image generation model, the target low-level features of the low-resolution remote sensing image are extracted through a feature extraction convolution layer, and the target low-level features are subjected to nonlinear mapping through a LeakyReLU active layer to generate a target low-level feature map. And successively extracting target high-level features of the low-level feature map by adopting a COT _ RDB module, and mapping and outputting to obtain a target high-level feature map. And continuously upsampling the target high-level characteristic diagram through an upsampling module to obtain a target upsampling diagram meeting the target size.
And step 208, mapping the target low-layer features and the target high-layer features to a target up-sampling image by using the target image generation model, and outputting a target super-resolution image.
In the embodiment of the invention, the target low-layer feature and the target high-layer feature are mapped to the pixel points of the upper sampling graph in a supplementing manner by an image reconstruction convolution module in the target image generation model through linear transformation, and the image after feature fusion is a target super-resolution image.
In the embodiment of the invention, an initial image generation model is constructed and an image identification model is obtained by responding to a model construction request sent by a demand end, then a low-resolution sample image serving as a training sample and a corresponding high-resolution sample image are obtained, the low-resolution sample image is processed by a low-layer feature extraction module, a high-layer feature extraction module, an up-sampling module and an image reconstruction convolution module in the initial image generation model to generate an initial super-resolution image, a sample image pair is constructed by adopting the initial super-resolution image and the high-resolution sample image and is input to the image identification model, and an integral loss function value is determined. And carrying out convergence judgment on the overall loss function value, if the overall loss function value is not converged, carrying out iterative optimization on the model parameters of the initial image generation model according to the characteristic error reflected by the overall loss function value until the overall loss function value is converged to obtain the target image generation model. Inputting the low-resolution remote sensing image into a target image generation model, extracting target low-level features and target high-level features of the low-resolution remote sensing image through the target image generation model, generating a target up-sampling image, and mapping the target low-level features and the target high-level features to the target up-sampling image to obtain a target super-resolution image. The super-resolution remote sensing image obtained by the target image generation model has the advantages of clearer and smoother imaging at texture details and better image quality.
Referring to fig. 5, fig. 5 is a block diagram of a super-resolution remote sensing image generation system according to a third embodiment.
The invention provides a remote sensing image generation system with super-resolution, which comprises:
and the model building module 501 is configured to build an initial image generation model and obtain an image identification model in response to the model building request.
A sample image obtaining module 502 for obtaining a low resolution sample image and a corresponding high resolution sample image.
An initial super-resolution image generation module 503, configured to input the low-resolution sample image to the initial image generation model, and generate an initial super-resolution image.
And the loss function calculation module 504 is configured to construct a sample image pair by using the initial super-resolution image and the high-resolution sample image, input the sample image pair to the image identification model, and determine an overall loss function value.
And the model optimization module 505 is configured to perform iterative optimization on the model parameters in the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain the target image generation model.
And the target super-resolution image generation module 506 is used for inputting the low-resolution remote sensing image into the target image generation model and outputting the target super-resolution image when the low-resolution remote sensing image is received.
Optionally, the model building module 501 is specifically configured to:
responding to the model construction request, and sequentially connecting the feature extraction convolution layer and the LeakyReLU activation layer to generate a low-layer feature extraction module;
the method comprises the following steps of constructing a high-level feature extraction module by serially connecting ten COT _ RDB modules consisting of COTNet-Conv-LeakyReLU layers, wherein residual errors are connected between every two COT _ RDB modules;
two Conv-PixelShuffle-LeakyReLU layers are connected in series to construct an up-sampling module;
and sequentially connecting the low-layer feature extraction module, the high-layer feature extraction module, the up-sampling module and the image reconstruction convolution module to construct an initial image generation model and obtain an image identification model constructed by the BoTNet-50 module.
Optionally, the initial super-resolution image generation module 503 is specifically configured to:
inputting the low-resolution sample image into an image generation model, extracting low-layer features of the low-resolution sample image through a feature extraction convolution layer, and performing nonlinear mapping on the low-layer features through a LeakyReLU activation layer to generate a low-layer feature map;
successively extracting high-level features of the low-level feature map by adopting a COT _ RDB module, and mapping and outputting to obtain a high-level feature map;
continuously upsampling the high-level feature map through an upsampling module to obtain an upsampling map meeting the target size;
and mapping the low-layer features and the high-layer features to the upper sampling map by using an image reconstruction convolution module to generate an initial super-resolution image.
Optionally, the loss function calculating module 504 is specifically configured to:
constructing a sample image sample pair by adopting the low-resolution image and the super-resolution image;
inputting the sample image pair into an image identification model, calculating the true probability of the image and acquiring edge information;
acquiring a feature extraction network, and respectively extracting features of the image pair on different convolution layers by using the feature extraction network to obtain total target perception information, background texture information and corner information which respectively correspond to the different convolution layers;
edge information, overall target perception information, background texture information, corner information and image true probability are adopted, preset information characteristic coefficients and preset loss function balance coefficients are combined, the input is carried out on a preset overall loss function model for simulation calculation, and an overall loss function value is obtained.
The overall loss function model is:
Figure BDA0003821531660000161
wherein,
Figure BDA0003821531660000162
Figure BDA0003821531660000163
Figure BDA0003821531660000164
Figure BDA0003821531660000165
Figure BDA0003821531660000166
Figure BDA0003821531660000167
in the form of an overall loss of energy,
Figure BDA0003821531660000168
in order to sense the loss of power,
Figure BDA0003821531660000169
in order for the generator to combat the losses,
Figure BDA00038215316600001610
in order to minimize the absolute error of the signal,
Figure BDA0003821531660000171
is edge loss, lambda is the balance coefficient of the generator against loss, eta is the balance coefficient of the minimum absolute error, mu is the balance coefficient of edge loss,
Figure BDA0003821531660000172
the overall target perception information is characterized in that,
Figure BDA0003821531660000173
the information characterizing the texture of the background is,
Figure BDA0003821531660000174
representing corner information, alpha is a characteristic coefficient of overall target perception information, and beta is background texture informationThe characteristic coefficient of information, gamma is the characteristic coefficient of the corner information, N is the number of samples, i is the ith sample,
Figure BDA0003821531660000175
is a low resolution image of the ith sample, θ G Model parameters of the model are generated for the image,
Figure BDA0003821531660000176
is a super-resolution image obtained by performing image generation on the low-resolution image of the ith sample, theta D The model parameters of the model are identified for the image,
Figure BDA0003821531660000177
the image true probability of the super-resolution image generated for the low-resolution image of the ith sample,
Figure BDA0003821531660000178
a high resolution image representing the ith sample,
Figure BDA0003821531660000179
an operator is extracted for the edges of the edges,
Figure BDA00038215316600001710
edge information of the high resolution image of the mth channel for the ith sample,
Figure BDA00038215316600001711
is the edge information of the super-resolution image of the ith sample channel, M is the number of image channels, M is the mth channel,
Figure BDA00038215316600001712
for the high resolution image of the mth channel of the ith sample,
Figure BDA00038215316600001713
is a super-resolution image of the m channel of the ith sample.
Optionally, the model optimization module 505 is specifically configured to:
carrying out convergence judgment on the overall loss function value;
if the overall loss function value is not converged, analyzing the overall loss function value to obtain a characteristic error of the image pair;
minimizing the characteristic error, iteratively updating model parameters in the initial image generation model in a back propagation mode, and skipping to execute the step of acquiring a low-resolution sample image and a corresponding high-resolution sample image;
and if the overall loss function value is converged, outputting the target image generation model by using the current model parameters.
Optionally, the target super-resolution image generation module 506 is specifically configured to:
when receiving the low-resolution remote sensing image, inputting the low-resolution remote sensing image into a target image generation model;
extracting target low-level features and target high-level features of the low-resolution remote sensing image through a target image generation model, and generating a target up-sampling image;
and mapping the target low-level features and the target high-level features to a target up-sampling image by using a target image generation model, and outputting a target super-resolution image.
An embodiment of the present invention further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the super-resolution remote sensing image generation method according to any one of the above embodiments.
The memory may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory has a memory space for program code for performing any of the method steps of the above-described method. For example, the memory space for the program code may comprise respective program codes for implementing the respective steps in the above method, respectively. The program code can be read from and written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. The program code may be compressed, for example, in a suitable form. The codes, when executed by a computing processing device, cause the computing processing device to perform the steps of the street view text recognition method described above.
The embodiment of the invention also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed, the method for generating the super-resolution remote sensing image is implemented according to any one of the embodiments.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the module described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A super-resolution remote sensing image generation method is characterized by comprising the following steps:
responding to the model construction request, constructing an initial image generation model and acquiring an image identification model;
acquiring a low-resolution sample image and a corresponding high-resolution sample image;
inputting the low-resolution sample image into the initial image generation model to generate an initial super-resolution image;
constructing a sample image pair by using the initial super-resolution image and the high-resolution sample image, inputting the sample image pair into the image identification model, and determining an overall loss function value;
performing iterative optimization on model parameters in the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model;
and when a low-resolution remote sensing image is received, inputting the low-resolution remote sensing image into the target image generation model, and outputting a target super-resolution image.
2. The remote sensing image generation method with super resolution according to claim 1, wherein the step of constructing an initial image generation model and acquiring an image discrimination model in response to a model construction request includes:
responding to the model construction request, and sequentially connecting the feature extraction convolution layer and the LeakyReLU activation layer to generate a low-layer feature extraction module;
the method comprises the following steps of constructing a high-level feature extraction module by serially connecting ten COT _ RDB modules consisting of COTNet-Conv-LeakyReLU layers, wherein residual errors are connected between every two COT _ RDB modules;
two Conv-PixelShuffle-LeakyReLU layers are connected in series to construct an up-sampling module;
and the low-layer feature extraction module, the high-layer feature extraction module, the up-sampling module and the image reconstruction convolution module are sequentially connected to construct an initial image generation model, and an image identification model constructed by the BoTNet-50 module is obtained.
3. The method for generating a remotely sensed image at super resolution as claimed in claim 2, wherein the step of inputting the low resolution sample image into the initial image generation model to generate an initial super-resolution image comprises:
inputting the low-resolution sample image into the image generation model, extracting low-layer features of the low-resolution sample image through the feature extraction convolutional layer, and performing nonlinear mapping on the low-layer features through the LeakyReLU activation layer to generate a low-layer feature map;
successively extracting high-level features of the low-level feature map by adopting the COT _ RDB module, and mapping and outputting to obtain a high-level feature map;
continuously upsampling the high-level feature map through the upsampling module to obtain an upsampling map meeting the target size;
and mapping the low-layer features and the high-layer features to the up-sampling image by using an image reconstruction convolution module to generate an initial super-resolution image.
4. The method for generating a remote sensing image at super resolution according to claim 1, wherein the step of constructing a sample image pair using the initial super-resolution image and the high-resolution sample image, inputting the sample image pair to the image discrimination model, and determining the overall loss function value comprises:
constructing a sample image sample pair by using the low-resolution image and the super-resolution image;
inputting the sample image pair into the image identification model, calculating the true probability of the image and acquiring edge information;
acquiring a feature extraction network, and respectively extracting features of the image pair on different convolution layers by using the feature extraction network to obtain total target perception information, background texture information and corner information which respectively correspond to the different convolution layers;
inputting the edge information, the total target perception information, the background texture information, the corner information and the image true probability into a preset integral loss function model for simulation calculation by combining a preset information characteristic coefficient and a preset loss function balance coefficient to obtain an integral loss function value;
the overall loss function model is as follows:
Figure FDA0003821531650000021
wherein,
Figure FDA0003821531650000022
Figure FDA0003821531650000023
Figure FDA0003821531650000024
Figure FDA0003821531650000025
Figure FDA0003821531650000026
Figure FDA0003821531650000027
in the form of an overall loss of energy,
Figure FDA0003821531650000028
in order to sense the loss of power,
Figure FDA0003821531650000029
in order for the generator to combat the losses,
Figure FDA00038215316500000210
in order to minimize the absolute error of the signal,
Figure FDA00038215316500000211
is edge loss, lambda is the balance coefficient of the generator against loss, eta is the balance coefficient of the minimum absolute error, mu is the balance coefficient of edge loss,
Figure FDA00038215316500000212
the overall target perception information is characterized in that,
Figure FDA00038215316500000213
the information characterizing the texture of the background is,
Figure FDA00038215316500000214
representing corner information, alpha is a characteristic coefficient of overall target perception information, beta is a characteristic coefficient of background texture information, gamma is a characteristic coefficient of the corner information, N is the number of samples, i is the ith sample,
Figure FDA00038215316500000215
is a low resolution image of the ith sample, θ G Model parameters of the model are generated for the image,
Figure FDA0003821531650000031
is a super-resolution image obtained by performing image generation on the low-resolution image of the ith sample, theta D The model parameters of the model are identified for the image,
Figure FDA0003821531650000032
the image true probability of the super-resolution image generated for the low-resolution image of the ith sample,
Figure FDA0003821531650000033
a high resolution image representing the ith sample,
Figure FDA0003821531650000034
in order to extract the operator for the edge,
Figure FDA0003821531650000035
for the edge information of the high resolution image of the mth channel of the ith sample,
Figure FDA0003821531650000036
is the edge information of the super-resolution image of the ith sample channel, M is the number of image channels, M is the mth channel,
Figure FDA0003821531650000037
for the high resolution image of the mth channel of the ith sample,
Figure FDA0003821531650000038
is a super-resolution image of the m channel of the ith sample.
5. The remote sensing image generation method with super resolution according to claim 1, wherein the step of performing iterative optimization on model parameters in the initial image generation model according to the overall loss function value until the overall loss function value converges to obtain a target image generation model comprises:
carrying out convergence judgment on the overall loss function value;
if the overall loss function value is not converged, analyzing the overall loss function value to obtain a characteristic error of the image pair;
minimizing the characteristic error, iteratively updating model parameters in the initial image generation model in a back propagation mode, and skipping to execute the step of acquiring the low-resolution sample image and the corresponding high-resolution sample image;
and if the overall loss function value is converged, outputting a target image generation model according to the current model parameters.
6. The method for generating a remote sensing image with super resolution according to claim 1, wherein the step of inputting a low resolution remote sensing image to the target image generation model and outputting a target super resolution image when the low resolution remote sensing image is received comprises:
when receiving a low-resolution remote sensing image, inputting the low-resolution remote sensing image into the target image generation model;
extracting target low-level features and target high-level features of the low-resolution remote sensing image through the target image generation model, and generating a target up-sampling image;
and mapping the target low-layer features and the target high-layer features to the target up-sampling graph by using the target image generation model, and outputting a target super-resolution image.
7. A super-resolution remote sensing image generation system is characterized by comprising:
the model construction module is used for responding to the model construction request, constructing an initial image generation model and acquiring an image identification model;
the sample image acquisition module is used for acquiring a low-resolution sample image and a corresponding high-resolution sample image;
the initial super-resolution image generation module is used for inputting the low-resolution sample image into the initial image generation model to generate an initial super-resolution image;
the loss function calculation module is used for constructing a sample image pair by adopting the initial super-resolution image and the high-resolution sample image, inputting the sample image pair into the image identification model and determining an integral loss function value;
the model optimization module is used for carrying out iterative optimization on model parameters in the initial image generation model according to the overall loss function value until the overall loss function value is converged to obtain a target image generation model;
and the target super-resolution image generation module is used for inputting the low-resolution remote sensing image into the target image generation model and outputting the target super-resolution image when the low-resolution remote sensing image is received.
8. The remote sensing image generation system with super resolution of claim 7, wherein the model construction module is specifically configured to:
responding to the model construction request, and sequentially connecting the feature extraction convolution layer and the LeakyReLU activation layer to generate a low-layer feature extraction module;
the method comprises the following steps of constructing a high-level feature extraction module by serially connecting ten COT _ RDB modules consisting of COTNet-Conv-LeakyReLU layers, wherein residual errors are connected between every two COT _ RDB modules;
two Conv-PixelShuffle-LeakyReLU layers are connected in series to construct an up-sampling module;
and the low-layer feature extraction module, the high-layer feature extraction module, the up-sampling module and the image reconstruction convolution module are sequentially connected to construct an initial image generation model, and an image identification model constructed by the BoTNet-50 module is obtained.
9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, causes the processor to execute the steps of the super-resolution remote sensing image generation method according to any one of claims 1 to 6.
10. A computer-readable storage medium on which a computer program is stored, wherein the computer program is executed to implement the method for generating a super-resolution remote sensing image according to any one of claims 1 to 6.
CN202211043000.0A 2022-08-29 2022-08-29 Super-resolution remote sensing image generation method, system, equipment and medium Pending CN115375548A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211043000.0A CN115375548A (en) 2022-08-29 2022-08-29 Super-resolution remote sensing image generation method, system, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211043000.0A CN115375548A (en) 2022-08-29 2022-08-29 Super-resolution remote sensing image generation method, system, equipment and medium

Publications (1)

Publication Number Publication Date
CN115375548A true CN115375548A (en) 2022-11-22

Family

ID=84069887

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211043000.0A Pending CN115375548A (en) 2022-08-29 2022-08-29 Super-resolution remote sensing image generation method, system, equipment and medium

Country Status (1)

Country Link
CN (1) CN115375548A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116782041A (en) * 2023-05-29 2023-09-19 武汉工程大学 Image quality improvement method and system based on liquid crystal microlens array
CN117314756A (en) * 2023-11-30 2023-12-29 中国平安财产保险股份有限公司 Verification and protection method and device based on remote sensing image, computer equipment and storage medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116782041A (en) * 2023-05-29 2023-09-19 武汉工程大学 Image quality improvement method and system based on liquid crystal microlens array
CN116782041B (en) * 2023-05-29 2024-01-30 武汉工程大学 Image quality improvement method and system based on liquid crystal microlens array
CN117314756A (en) * 2023-11-30 2023-12-29 中国平安财产保险股份有限公司 Verification and protection method and device based on remote sensing image, computer equipment and storage medium
CN117314756B (en) * 2023-11-30 2024-04-05 中国平安财产保险股份有限公司 Verification and protection method and device based on remote sensing image, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
EP3961484B1 (en) Medical image segmentation method and device, electronic device and storage medium
Wang et al. Ultra-dense GAN for satellite imagery super-resolution
CN109166130B (en) Image processing method and image processing device
Chen et al. MICU: Image super-resolution via multi-level information compensation and U-net
Chen et al. The face image super-resolution algorithm based on combined representation learning
CN112651978B (en) Sublingual microcirculation image segmentation method and device, electronic equipment and storage medium
CN112668494A (en) Small sample change detection method based on multi-scale feature extraction
CN109146788A (en) Super-resolution image reconstruction method and device based on deep learning
CN111860233B (en) SAR image complex building extraction method and system based on attention network selection
CN115375548A (en) Super-resolution remote sensing image generation method, system, equipment and medium
CN114298997B (en) Fake picture detection method, fake picture detection device and storage medium
CN117058367A (en) Semantic segmentation method and device for high-resolution remote sensing image building
CN113744136A (en) Image super-resolution reconstruction method and system based on channel constraint multi-feature fusion
CN114897136A (en) Multi-scale attention mechanism method and module and image processing method and device
CN115565045A (en) Hyperspectral and multispectral image fusion method based on multi-scale space-spectral transformation
CN117422619A (en) Training method of image reconstruction model, image reconstruction method, device and equipment
CN116385902A (en) Remote sensing big data processing method, system and cloud platform
CN111402131A (en) Method for acquiring super-resolution land cover classification map based on deep learning
CN114092803A (en) Cloud detection method and device based on remote sensing image, electronic device and medium
CN117576483B (en) Multisource data fusion ground object classification method based on multiscale convolution self-encoder
CN117521742B (en) Lightweight deployment image processing method based on deep neural network model
CN111325756A (en) Three-dimensional image artery and vein segmentation method and system based on deep learning network
CN113920323B (en) Different-chaos hyperspectral image classification method based on semantic graph attention network
CN114445280B (en) Point cloud downsampling method based on attention mechanism
Nanavati et al. Generative adversarial networks: A comparative analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination