CN118657674A - Weak light image enhancement analysis method based on attention mechanism - Google Patents
Weak light image enhancement analysis method based on attention mechanism Download PDFInfo
- Publication number
- CN118657674A CN118657674A CN202411154572.5A CN202411154572A CN118657674A CN 118657674 A CN118657674 A CN 118657674A CN 202411154572 A CN202411154572 A CN 202411154572A CN 118657674 A CN118657674 A CN 118657674A
- Authority
- CN
- China
- Prior art keywords
- weak light
- training
- enhancement
- images
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 30
- 238000004458 analytical method Methods 0.000 title abstract description 8
- 238000012549 training Methods 0.000 claims abstract description 94
- 238000000034 method Methods 0.000 claims abstract description 75
- 230000008569 process Effects 0.000 claims abstract description 58
- 238000012795 verification Methods 0.000 claims abstract description 21
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 15
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 10
- 238000011156 evaluation Methods 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000005286 illumination Methods 0.000 claims description 13
- 230000000694 effects Effects 0.000 claims description 10
- 230000002708 enhancing effect Effects 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 6
- 238000012804 iterative process Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 4
- 230000008447 perception Effects 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 230000006583 body weight regulation Effects 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000012797 qualification Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 206010040030 Sensory loss Diseases 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of computer digital image processing, and relates to a weak light image enhancement analysis method based on an attention mechanism, which comprises the steps of constructing an enhancement network model consisting of a decomposition unit, an enhancement unit and a reconstruction unit introducing a multi-head self-attention mechanism, inputting a training set into the enhancement network model for performing traversal training, evaluating the loss performance of enhancement images output by weak light images in batches corresponding to each iteration process of each traversal training and corresponding normal light images in the traversal training process, and carrying out real-time reverse updating of the parameters of the enhancement network model by the ADAM optimizer according to the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training, further obtaining the optimal configuration parameters of the enhancement network model according to the coincidence result of the early-stop condition, and immediately confirming the enhancement result of the weak light image of the enhancement network model under the optimal configuration parameters by using the verification set, thereby realizing efficient and accurate enhancement processing of the weak light image.
Description
Technical Field
The invention belongs to the technical field of computer digital image processing, and relates to a dim light image enhancement analysis method based on an attention mechanism.
Background
With the popularization of intelligent devices and the continuous development of scientific technology, the acquisition of multimedia data becomes increasingly simple. However, in practical application, due to factors such as insufficient ambient light, limited equipment performance, improper manual operation, etc., the obtained image often has problems such as low brightness, poor contrast, dull color, unclear details, obvious noise, etc., which not only affects the visual effect of the image, but also reduces the performance of advanced visual tasks such as object detection, face recognition, automatic driving, etc., so that the weak light image enhancement technology has been developed.
In the prior art, an attention mechanism is introduced on the basis of a developed weak light image enhancement method to make up for the problems of noise introduction, color distortion and insufficient adaptability of a specific scene, for example, a patent application of a weak light image enhancement method of a double attention mechanism with a publication number of CN117974476A is adopted, EDAformer is adopted to integrate a convolution self-attention module and a self-attention module, so that a network focuses on local and global information of a low-illumination image at the same time, a network architecture is realized in a recursive manner, finally, network output characteristics are fused into a reconstructed image, the weight of a loss function is dynamically adjusted to restrict the network to achieve an optimal fusion result, and a final enhanced image is obtained from a visible light image in a weak light environment, so that the problem that the enhancement method in the prior art is not easy to obtain an image with clear detail outline structure and excellent denoising result is solved, but the limitation expression exists, and the method is specifically as follows: 1. the performance of the dual-attention mechanism adopted by the scheme is mainly local and global fusion, and although the wider characteristic representation can be covered from a macroscopic view, the defect that the full coverage is not realized still exists in a detail level, so that the enhancement processing result of the weak light image is not fine and accurate enough.
2. The model training process of the scheme completely executes the preset number of traversal training rounds, does not have the training early-stop strategy execution capability, means that the model can continue to train even if the model reaches better performance and starts to show signs of fitting in the earlier rounds, and not only can result in the waste of calculation resources, but also can reduce the generalization capability of the model due to the additional training rounds.
Disclosure of Invention
In view of this, in order to solve the problems presented in the above-mentioned background art, a weak light image enhancement analysis method based on an attention mechanism is now proposed.
The aim of the invention can be achieved by the following technical scheme: the invention provides a weak light image enhancement analysis method based on an attention mechanism, which comprises the following steps: s1, constructing an enhanced network model by a decomposition unit, an enhancement unit and a reconstruction unit introducing a multi-head self-attention mechanism, and enhancing an input weak light image through the enhanced network model.
S2, extracting each weak light image and the corresponding normal light image in the WEB cloud storage LOL data set, dividing a training set and a verification set according to a predefined principle, inputting the training set into an enhanced network model, performing traversal training on the model, and enabling the upper limit of the traversal training round to be preset coherence.
S3, evaluating the loss performance of the enhanced images output by the weak light images in the corresponding batches of each iteration process of each traversal training of the enhanced network model and the corresponding normal light images, and acquiring the optimal configuration parameters of the enhanced network model according to the conforming result of the early-stop condition.
S4, inputting the verification set into an enhanced network model under the optimal configuration parameters, and performing image enhancement result verification on the model.
S5, if the enhancement result of the enhanced network model image is verified to be qualified, the optimal configuration parameters of the enhanced network model are stored, otherwise, the steps S2-S4 are repeated until the enhancement result is verified to be qualified.
Compared with the prior art, the invention has the following beneficial effects: (1) The invention introduces a multi-head self-attention mechanism to help the enhancement network model to extract information of different feature subspaces of the input weak light image, and compared with other attention mechanisms cited in the prior art, the invention captures more kinds of dependency relationships, thereby being beneficial to generating richer and comprehensive feature representation in the weak light image enhancement processing.
(2) According to the invention, through combining objective loss and sensory loss, the loss performance evaluation of the enhancement image output by the weak light image and the corresponding normal light image in the enhancement network model training process is developed, the model development parameters of the ADAM optimizer are updated reversely in real time, the enhancement network model is prevented from being trained and fitted on the basis of early-stop strategies, and meanwhile, the optimal configuration parameters of the enhancement network model are accurately refined, so that the enhancement image is ensured to be natural and comfortable in visual perception while the optimization of the enhancement image on numerical indexes is ensured.
(3) The invention adopts the verification set to verify the weak light image processing effect of the enhanced network model with the increased optimal configuration parameters, and ensures the generalization, stability and reliability of the enhanced network model with the increased optimal configuration parameters in practical application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of the steps of the method of the present invention.
Fig. 2 is a flowchart illustrating a detailed implementation step of step S1 in fig. 1.
FIG. 3 is a schematic diagram showing the early stop condition compliance of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the present invention provides a method for enhancing and analyzing a weak light image based on an attention mechanism, comprising: s1, constructing an enhanced network model by a decomposition unit, an enhancement unit and a reconstruction unit introducing a multi-head self-attention mechanism, and enhancing an input weak light image through the enhanced network model.
Referring to fig. 2, specifically, the enhancement processing of the input weak light image by the enhanced network model includes: s11, decomposing the input weak light image into an illumination map and a reflection map through a decomposition unit.
S12, splicing the illumination components of the illumination map and the reflection components of the reflection map to generate an input tensor, transmitting the input tensor into a built-in convolution layer of the enhancement unit to conduct primary feature extraction, continuously conducting convolution operation for preset times on the extracted primary features, enhancing the illumination features and the reflection characteristics through convolution kernel weight regulation and ReLU activation functions respectively, and enabling feature maps output by each convolution operation to recover the original size of a weak light image through up-sampling processing and be fused with feature maps output by the previous convolution operation.
S13, splicing the fusion feature images obtained after the convolution operation of the preset times, transmitting the fusion feature images into a reconstruction unit introducing a multi-head self-attention mechanism, obtaining multi-head self-attention calculation results of the spliced feature images, performing splicing adjustment and linear transformation on the multi-head self-attention calculation results, and then reconstructing the spliced feature images by utilizing a built-in transpose convolution layer of the reconstruction unit, thereby obtaining enhanced images of the input weak light images, and realizing enhancement processing of the input weak light images.
Specifically, the decomposing the input low-light image into an illumination map and a reflection map by the decomposing unit includes: and selecting the maximum value of the channel of each pixel of the input weak light image at the inlet of the decomposition unit, reserving the maximum channel dimension value of each pixel of the input weak light image to obtain a maximum value image, carrying out channel dimension splicing on the maximum value image and the input weak light image to form an input tensor with an expanded channel number, transmitting the input tensor into a network structure built in the decomposition unit and composed of a preset number of convolution layers, decomposing the weak light image into corresponding components of the preset number of output channels by using a specific convolution operation after the combination operation of the convolution and the activation function of the preset number of times, extracting the corresponding components of each output channel, carrying out slicing operation, converting the value range of the corresponding components into a preset interval, and extracting a reflection graph and an illumination graph from the reflection graph according to the types of the corresponding components of each output channel.
Specifically, the obtaining the multi-head self-attention calculation result of the spliced feature map includes: dividing the spliced feature map into area blocks with preset window sizes, carrying out convolution operation on the area blocks by using a preset multi-self-attention function, setting the number of output channels to be equal to the number of attention heads, generating a primary attention score map of each area block for each attention head, carrying out secondary convolution operation on the area blocks, setting the number of output channels to be 1, generating an integrated attention score map of each area block, forming an attention score matrix of the spliced feature map according to the positions of the area blocks, and carrying out weighting treatment on the spliced feature map so as to obtain a multi-head self-attention calculation result of the spliced feature map.
It should be noted that the preset multi-self-attention function is specifically tf.layers.conv2d function.
The embodiment of the invention introduces a multi-head self-attention mechanism to help the enhancement network model to extract information of different feature subspaces of the input weak light image, and compared with other attention mechanisms quoted in the prior art, the embodiment of the invention captures more kinds of dependency relations, thereby being beneficial to generating richer and comprehensive feature representation in the weak light image enhancement processing.
S2, extracting each weak light image and the corresponding normal light image in the WEB cloud storage LOL data set, dividing a training set and a verification set according to a predefined principle, inputting the training set into an enhanced network model, performing traversal training on the model, and enabling the upper limit of the traversal training round to be preset coherence.
It should be noted that the predefined principle is that the number of the weak light images and the corresponding normal light images in the training set is the total number of the weak light images and the corresponding normal light images in the LOL data setThe quantity ratio of the weak light images and the corresponding normal light images in the verification set is LOL data set weak light images and the total quantity of the corresponding normal light images。
Specifically, the performing traversal training on the model includes: dividing each weak light image in the training set in batches according to the number of preset images, setting the iteration number of single traversal training of the enhanced network model to be equal to the number of divided batches of the training set, selecting a preset video memory GPU as the traversal training environment of the enhanced network model, and setting the initial learning rate of the existing ADAM optimizer in the enhanced network model before the training is startedAnd the initial state variables are used for inputting all the weak light images in the corresponding batches to the enhancement network model in each iteration process of each traversal training, enhancing the weak light images to output enhanced images, evaluating the loss performance of the enhanced images output by all the weak light images in the corresponding batches of each iteration process of each traversal training and the corresponding normal light images, and the ADAM optimizer is used for carrying out reverse updating on the parameters of the enhancement network model according to the integral loss performance of the weak light images in the corresponding batches of each iteration process of each traversal training, and throwing the updated parameters into the next iteration process until the early-stop condition is met, and ending the traversal training.
Specifically, the evaluating the performance of the loss of the enhanced image output by each weak light image and the corresponding normal light image in each batch corresponding to each iteration process of each traversal training comprises: enhanced images for outputting each weak light image in each iteration process corresponding batch of each traversal trainingCorresponding to the normal light imageThe pre-training network is put into the process of the pre-training network together,For each number of traversals of the training,，For the numbering of the individual iterations,，For the iterative process to correspond to the numbering of each low-light image within a batch,From the formulaObtaining the perception loss coefficients of the enhanced images output by the weak light images in the corresponding batches of each iteration process of each traversal training and the corresponding normal light images, whereinRepresenting a feature extractor of the pre-training network,Representation ofNorms.
From the formulaObtaining objective loss coefficients of enhanced images output by each weak light image and corresponding normal light images in each iteration process corresponding batch of each traversal training, whereinRespectively represent the firstFirst pass trainingThe iterative process corresponds to the first in the batchThe peak signal-to-noise ratio and the structural similarity of the enhanced image output by the weak light image and the corresponding normal light image,Is a natural constant.
The above-mentionedThe specific analysis process of (2) is as follows: converting the enhanced images output by the weak light images in the batches corresponding to the iterative processes of each traversal training and the corresponding normal light images into gray images, and obtaining gray difference values of the pixels of the enhanced images output by the weak light images in the batches corresponding to the iterative processes of each traversal training and the corresponding pixels of the corresponding normal light imagesWhereinIn order to enhance the numbering of the pixels of the image,From the formulaObtaining peak signal-to-noise ratio of enhanced images output by each weak light image in each iteration process corresponding batch of each traversal training and corresponding normal light image thereof, whereinFor a preset image maximum pixel gray value,To enhance the number of image pixels.
It should also be noted that the aboveThe specific analysis process of (2) is as follows: converting the enhanced images output by the weak light images in the batches corresponding to the iteration processes of each traversal training and the corresponding normal light images into gray images, acquiring local average brightness and local standard deviation of the sliding windows respectively corresponding to the enhanced images output by the weak light images in the batches corresponding to the iteration processes of each traversal training and the corresponding normal light images, and respectively marking asAndAnd enhancing covariance between the image and its corresponding sliding window corresponding to the normal light imageFrom the formula、、And respectively obtaining local brightness similarity, local contrast similarity and local structural similarity between the enhanced images output by the weak light images in the corresponding batches of each iteration process of each traversal training and the corresponding normal light images, and taking the product of the local brightness similarity, the local contrast similarity and the local structural similarity as the structural similarity between the enhanced images output by the weak light images in the corresponding batches of each iteration process of each traversal training and the corresponding normal light images.
It should be further noted that the local average brightness, the local standard deviation and the covariance can be obtained through the existing mature calculation formula, which is not described herein.
Will be、The enhanced images output by the weak light images in the corresponding batches of each iteration process of each traversal training and the loss performance evaluation indexes of the corresponding normal light images are used together.
Specifically, the ADAM optimizer performs inverse update of the enhanced network model parameters according to the overall loss performance of the batch of low-light images corresponding to each iteration process of each traversal training, including: for a pair of、Respectively carrying out average value calculation to obtain、Will be、Respectively accumulating the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training with the corresponding preset weight products to obtain an evaluation index of the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training, and performing automatic differential processing on the evaluation index to obtain the gradient of the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training on the enhanced network model parameters。
It should be noted that, the automatic differentiation process to obtain the gradient is a mature technology, and specifically can be obtained by inputting a loss function and a model in pytorch.
Extracting state variables in each iteration process of each traversal training of ADAM optimizer, including first moment estimation of gradientSum and second moment estimationBy the following constitutionUpdating the state variable of the ADAM optimizer so as to obtain the updated state variable of the integral loss expression of the batch of weak light images corresponding to each iteration process of each traversal training of the ADAM optimizer, and recording the updated state variable asAnd。
The above-mentioned、The calculation formulas of (a) are respectively as follows:、 Wherein The super parameters of the ADAM optimizer are respectively 0.9 and 0.999.
Extracting enhanced network model parameters in each iteration process of each traversal trainingFrom the formulaObtaining parameters of the ADAM optimizer for reversely updating the enhanced network model according to the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training, whereinIs a preset constant for preventing the denominator from being 0.
S3, obtaining optimal configuration parameters of the enhanced network model according to the meeting result of the early-stop condition.
Referring to fig. 3, specifically, the obtaining, according to the early-stop condition meeting result, the optimal configuration parameters of the enhanced network model includes: if the reduction rate of the evaluation index of the overall loss performance of the batch of weak light images corresponding to a certain iteration process of the traversal training and the evaluation index of the overall loss performance of the batch of weak light images corresponding to a subsequent continuous preset iteration process is smaller than a preset reduction rate reasonable threshold value, the iteration process of the traversal training is indicated to reach a fitting state and accords with early stop conditions, and the network model parameter enhanced by the iteration process of the traversal training is taken as an optimal configuration parameter.
According to the embodiment of the invention, through combining objective loss and sensory loss, the loss performance evaluation of the enhancement image output by the weak light image and the corresponding normal light image in the training process of the enhancement network model is developed, the parameters of the enhancement network model are reversely updated in real time by the ADAM optimizer, the enhancement network model is prevented from being trained and fitted on the basis of the early-stop strategy, and meanwhile, the optimal configuration parameters of the enhancement network model are accurately refined, so that the enhancement image is ensured to be optimized in numerical indexes and the visual perception of the enhancement image is ensured to be natural and comfortable.
S4, inputting the verification set into an enhanced network model under the optimal configuration parameters, and performing image enhancement result verification on the model.
Specifically, the image enhancement result verification of the model comprises: as described above、The calculation method is consistent, and the perception loss coefficient of the enhancement network model under the optimal configuration parameters aiming at the enhancement image output by each weak light image in the verification set and the corresponding normal light image is obtainedAnd objective loss coefficient，To verify the numbering of the low-light images within the set,Calculating an image enhancement effect evaluation coefficient of an enhancement network model under optimal configuration parameters，Image enhancement outcome verification is performed on the model by the model, wherein、The perceived loss coefficient and the allowable threshold value of the objective loss coefficient are respectively defined for the image enhancement effect evaluation criterion stored in the WEB cloud,To verify the number of low-light images in the set.
S5, if the enhancement result of the enhanced network model image is verified to be qualified, the optimal configuration parameters of the enhanced network model are stored, otherwise, the steps S2-S4 are repeated until the enhancement result is verified to be qualified.
Specifically, the enhancement network model image enhancement result verification qualification condition includes: and the image enhancement effect evaluation coefficient of the enhancement network model under the optimal configuration parameters is larger than or equal to the image enhancement effect evaluation coefficient standard threshold stored in the WEB cloud.
The embodiment of the invention adopts the verification set to verify the weak light image processing effect of the enhanced network model with the increased optimal configuration parameters, and ensures the generalization, stability and reliability of the enhanced network model with the increased optimal configuration parameters in practical application.
The foregoing is merely illustrative and explanatory of the principles of this invention, as various modifications and additions may be made to the specific embodiments described, or similar arrangements may be substituted by those skilled in the art, without departing from the principles of this invention or beyond the scope of this invention as defined in the claims.
Claims (10)
1. The method for enhancing and analyzing the weak light image based on the attention mechanism is characterized by comprising the following steps of:
s1, constructing an enhanced network model by a decomposition unit, an enhancement unit and a reconstruction unit introducing a multi-head self-attention mechanism, and enhancing an input weak light image through the enhanced network model;
s2, extracting each weak light image and corresponding normal light image in the LOL data set stored in the WEB cloud, dividing a training set and a verification set according to a predefined principle, inputting the training set into an enhanced network model, and performing traversal training on the model, wherein the upper limit of the traversal training round is preset coherence;
S3, obtaining optimal configuration parameters of the enhanced network model according to the meeting result of the early-stop condition;
S4, inputting the verification set into an enhanced network model under the optimal configuration parameters, and performing image enhancement result verification on the model;
s5, if the enhancement result of the enhanced network model image is verified to be qualified, the optimal configuration parameters of the enhanced network model are stored, otherwise, the steps S2-S4 are repeated until the enhancement result is verified to be qualified.
2. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 1, wherein: the enhancement processing of the input weak light image through the enhancement network model comprises the following steps: s11, decomposing an input weak light image into an illumination map and a reflection map through a decomposition unit;
S12, splicing the illumination components of the illumination map and the reflection components of the reflection map to generate an input tensor, transmitting the input tensor into a built-in convolution layer of the enhancement unit to perform primary feature extraction, continuously performing convolution operation for preset times on the extracted primary features, enhancing the illumination features and the reflection characteristics through convolution kernel weight regulation and ReLU activation functions respectively, and restoring the original size of the weak light image through upsampling of the feature map output by each convolution operation and fusing the feature map with the feature map output by the previous convolution operation;
S13, splicing the fusion feature images obtained after the convolution operation of the preset times, transmitting the fusion feature images into a reconstruction unit introducing a multi-head self-attention mechanism, obtaining multi-head self-attention calculation results of the spliced feature images, performing splicing adjustment and linear transformation on the multi-head self-attention calculation results, and then reconstructing the spliced feature images by utilizing a built-in transpose convolution layer of the reconstruction unit, thereby obtaining enhanced images of the input weak light images, and realizing enhancement processing of the input weak light images.
3. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 2, wherein: the decomposing unit is used for decomposing the input weak light image into an illumination graph and a reflection graph, and comprises the following steps: and selecting the maximum value of the channel of each pixel of the input weak light image at the inlet of the decomposition unit, reserving the maximum channel dimension value of each pixel of the input weak light image to obtain a maximum value image, carrying out channel dimension splicing on the maximum value image and the input weak light image to form an input tensor with an expanded channel number, transmitting the input tensor into a network structure built in the decomposition unit and composed of a preset number of convolution layers, decomposing the weak light image into corresponding components of the preset number of output channels by using a specific convolution operation after the combination operation of the convolution and the activation function of the preset number of times, extracting the corresponding components of each output channel, carrying out slicing operation, converting the value range of the corresponding components into a preset interval, and extracting a reflection graph and an illumination graph from the reflection graph according to the types of the corresponding components of each output channel.
4. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 2, wherein: the multi-head self-attention calculation result of the spliced feature map is obtained, and the multi-head self-attention calculation result comprises: dividing the spliced feature map into area blocks with preset window sizes, carrying out convolution operation on the area blocks by using a preset multi-self-attention function, setting the number of output channels to be equal to the number of attention heads, generating a primary attention score map of each area block for each attention head, carrying out secondary convolution operation on the area blocks, setting the number of output channels to be 1, generating an integrated attention score map of each area block, forming an attention score matrix of the spliced feature map according to the positions of the area blocks, and carrying out weighting treatment on the spliced feature map so as to obtain a multi-head self-attention calculation result of the spliced feature map.
5. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 1, wherein: the model is subjected to traversal training, which comprises the following steps: dividing each weak light image in the training set in batches according to the number of preset images, setting the iteration number of single traversal training of the enhanced network model to be equal to the number of divided batches of the training set, selecting a preset video memory GPU as the traversal training environment of the enhanced network model, and setting the initial learning rate of the existing ADAM optimizer in the enhanced network model before the training is startedAnd the initial state variables are used for inputting all the weak light images in the corresponding batches to the enhancement network model in each iteration process of each traversal training, enhancing the weak light images to output enhanced images, evaluating the loss performance of the enhanced images output by all the weak light images in the corresponding batches of each iteration process of each traversal training and the corresponding normal light images, and the ADAM optimizer is used for carrying out reverse updating on the parameters of the enhancement network model according to the integral loss performance of the weak light images in the corresponding batches of each iteration process of each traversal training, and throwing the updated parameters into the next iteration process until the early-stop condition is met, and ending the traversal training.
6. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 5, wherein: the evaluation of the loss performance of the enhanced image output by each weak light image in each iteration process corresponding batch of each traversal training and the corresponding normal light image comprises the following steps: enhanced images for outputting each weak light image in each iteration process corresponding batch of each traversal trainingCorresponding to the normal light imageThe pre-training network is put into the process of the pre-training network together,For each number of traversals of the training,，For the numbering of the individual iterations,，For the iterative process to correspond to the numbering of each low-light image within a batch,From the formulaObtaining the perception loss coefficients of the enhanced images output by the weak light images in the corresponding batches of each iteration process of each traversal training and the corresponding normal light images, whereinRepresenting a feature extractor of the pre-training network,Representation ofA norm;
From the formula Obtaining objective loss coefficients of enhanced images output by each weak light image and corresponding normal light images in each iteration process corresponding batch of each traversal training, whereinRespectively represent the firstFirst pass trainingThe iterative process corresponds to the first in the batchThe peak signal-to-noise ratio and the structural similarity of the enhanced image output by the weak light image and the corresponding normal light image,Is a natural constant;
Will be 、The enhanced images output by the weak light images in the corresponding batches of each iteration process of each traversal training and the loss performance evaluation indexes of the corresponding normal light images are used together.
7. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 6, wherein: the ADAM optimizer performs reverse updating of the enhanced network model parameters according to the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training, and the method comprises the following steps: for a pair of、Respectively carrying out average value calculation to obtain、Will be、Respectively accumulating the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training with the corresponding preset weight products to obtain an evaluation index of the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training, and performing automatic differential processing on the evaluation index to obtain the gradient of the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training on the enhanced network model parameters；
Extracting state variables in each iteration process of each traversal training of ADAM optimizer, including first moment estimation of gradientSum and second moment estimationBy the following constitutionUpdating the state variable of the ADAM optimizer so as to obtain the updated state variable of the integral loss expression of the batch of weak light images corresponding to each iteration process of each traversal training of the ADAM optimizer, and recording the updated state variable asAnd；
Extracting enhanced network model parameters in each iteration process of each traversal trainingFrom the formulaObtaining parameters of the ADAM optimizer for reversely updating the enhanced network model according to the integral loss performance of the batch of weak light images corresponding to each iteration process of each traversal training, whereinIs a preset constant.
8. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 7, wherein: the obtaining the optimal configuration parameters of the enhanced network model according to the early-stop condition conforming result comprises the following steps: if the reduction rate of the evaluation index of the overall loss performance of the batch of weak light images corresponding to a certain iteration process of the traversal training and the evaluation index of the overall loss performance of the batch of weak light images corresponding to a subsequent continuous preset iteration process is smaller than a preset reduction rate reasonable threshold value, the iteration process of the traversal training is indicated to reach a fitting state and accords with early stop conditions, and the network model parameter enhanced by the iteration process of the traversal training is taken as an optimal configuration parameter.
9. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 6, wherein: the image enhancement result verification of the model comprises the following steps: obtaining a perception loss coefficient of an enhanced network model under optimal configuration parameters aiming at enhanced images output by each weak light image in a verification set and corresponding normal light imagesAnd objective loss coefficient，To verify the numbering of the low-light images within the set,Calculating an image enhancement effect evaluation coefficient of an enhancement network model under optimal configuration parameters，Image enhancement outcome verification is performed on the model by the model, wherein、The perceived loss coefficient and the allowable threshold value of the objective loss coefficient are respectively defined for the image enhancement effect evaluation criterion stored in the WEB cloud,To verify the number of low-light images in the set.
10. The method for analyzing low-light image enhancement based on an attention mechanism according to claim 9, wherein: the enhancement network model image enhancement result verification qualification condition comprises the following steps: and the image enhancement effect evaluation coefficient of the enhancement network model under the optimal configuration parameters is larger than or equal to the image enhancement effect evaluation coefficient standard threshold stored in the WEB cloud.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411154572.5A CN118657674B (en) | 2024-08-22 | 2024-08-22 | Weak light image enhancement analysis method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411154572.5A CN118657674B (en) | 2024-08-22 | 2024-08-22 | Weak light image enhancement analysis method based on attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118657674A true CN118657674A (en) | 2024-09-17 |
| CN118657674B CN118657674B (en) | 2024-10-25 |
Family
ID=92706722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411154572.5A Active CN118657674B (en) | 2024-08-22 | 2024-08-22 | Weak light image enhancement analysis method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118657674B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112712481A (en) * | 2021-01-11 | 2021-04-27 | 中国科学技术大学 | Structure-texture sensing method aiming at low-light image enhancement |
| WO2022077605A1 (en) * | 2020-10-15 | 2022-04-21 | 青岛理工大学 | Wind turbine blade image-based damage detection and localization method |
| CN116129124A (en) * | 2023-03-16 | 2023-05-16 | 泰州市人民医院 | Image segmentation method, system and equipment |
| CN117132500A (en) * | 2023-09-13 | 2023-11-28 | 西安理工大学 | Weak light enhancement method based on sparse conversion network |
| WO2023236565A1 (en) * | 2022-06-10 | 2023-12-14 | 南京航空航天大学 | Low-light image enhancement method based on reinforcement learning and aesthetic evaluation |
| CN117649368A (en) * | 2023-08-30 | 2024-03-05 | 西安理工大学 | Weak light enhancement method based on self-adaptive sparse attention diffusion model |
| WO2024060321A1 (en) * | 2022-09-22 | 2024-03-28 | 之江实验室 | Joint modeling method and apparatus for enhancing local features of pedestrians |
| CN117935202A (en) * | 2024-01-30 | 2024-04-26 | 华南理工大学 | Low-illumination environment lane line detection method based on deep learning |
| CN117974476A (en) * | 2024-01-31 | 2024-05-03 | 西安理工大学 | Weak light image enhancement method of double-attention mechanism |
| CN118365548A (en) * | 2024-06-19 | 2024-07-19 | 江南大学 | Low-light image enhancement method and system based on customized expansion enhancer |
-
2024
- 2024-08-22 CN CN202411154572.5A patent/CN118657674B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022077605A1 (en) * | 2020-10-15 | 2022-04-21 | 青岛理工大学 | Wind turbine blade image-based damage detection and localization method |
| CN112712481A (en) * | 2021-01-11 | 2021-04-27 | 中国科学技术大学 | Structure-texture sensing method aiming at low-light image enhancement |
| WO2023236565A1 (en) * | 2022-06-10 | 2023-12-14 | 南京航空航天大学 | Low-light image enhancement method based on reinforcement learning and aesthetic evaluation |
| WO2024060321A1 (en) * | 2022-09-22 | 2024-03-28 | 之江实验室 | Joint modeling method and apparatus for enhancing local features of pedestrians |
| CN116129124A (en) * | 2023-03-16 | 2023-05-16 | 泰州市人民医院 | Image segmentation method, system and equipment |
| CN117649368A (en) * | 2023-08-30 | 2024-03-05 | 西安理工大学 | Weak light enhancement method based on self-adaptive sparse attention diffusion model |
| CN117132500A (en) * | 2023-09-13 | 2023-11-28 | 西安理工大学 | Weak light enhancement method based on sparse conversion network |
| CN117935202A (en) * | 2024-01-30 | 2024-04-26 | 华南理工大学 | Low-illumination environment lane line detection method based on deep learning |
| CN117974476A (en) * | 2024-01-31 | 2024-05-03 | 西安理工大学 | Weak light image enhancement method of double-attention mechanism |
| CN118365548A (en) * | 2024-06-19 | 2024-07-19 | 江南大学 | Low-light image enhancement method and system based on customized expansion enhancer |
Non-Patent Citations (4)
| Title |
|---|
| 侯向丹;赵一浩;刘洪普;郭鸿湧;于习欣;丁梦园;: "融合残差注意力机制的UNet视盘分割", 中国图象图形学报, no. 09, 16 September 2020 (2020-09-16) * |
| 姚哲维;杨丰;黄靖;刘娅琴;: "改进型循环生成对抗网络的血管内超声图像增强", 计算机科学, no. 05, 15 May 2019 (2019-05-15) * |
| 毛雪宇;彭艳兵;: "增量角度域损失和多特征融合的地标识别", 中国图象图形学报, no. 08, 12 August 2020 (2020-08-12) * |
| 秦淑婧;杨关;: "增强视觉特征的视觉问答任务研究", 中原工学院学报, no. 01, 25 February 2020 (2020-02-25) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118657674B (en) | 2024-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108986050B (en) | Image and video enhancement method based on multi-branch convolutional neural network | |
| CN112614077B (en) | Unsupervised low-illumination image enhancement method based on generation countermeasure network | |
| CN113313657B (en) | Unsupervised learning method and system for low-illumination image enhancement | |
| Panetta et al. | Tmo-net: A parameter-free tone mapping operator using generative adversarial network, and performance benchmarking on large scale hdr dataset | |
| CN113298744B (en) | End-to-end infrared and visible light image fusion method | |
| CN113724354B (en) | Gray image coloring method based on reference picture color style | |
| CN116757986A (en) | Infrared and visible light image fusion method and device | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| CN112614070B (en) | defogNet-based single image defogging method | |
| CN117197627B (en) | Multi-mode image fusion method based on high-order degradation model | |
| CN111275638A (en) | Face restoration method for generating confrontation network based on multi-channel attention selection | |
| CN115205196A (en) | No-reference image quality evaluation method based on twin network and feature fusion | |
| CN115063318A (en) | Adaptive frequency-resolved low-illumination image enhancement method and related equipment | |
| CN113128517B (en) | Tone mapping image mixed visual feature extraction model establishment and quality evaluation method | |
| CN118657674B (en) | Weak light image enhancement analysis method based on attention mechanism | |
| CN116563133A (en) | Low-illumination color image enhancement method based on simulated exposure and multi-scale fusion | |
| CN118710557B (en) | Image restoration method and system guided by image structure and texture information | |
| CN111914119B (en) | Video abstraction method based on dynamic and static feature fusion | |
| CN118710507B (en) | Underwater image enhancement method based on Mamba hybrid architecture of space-frequency fusion | |
| Gong et al. | Image Style Conversion Model Design based on Generative Adversarial Networks | |
| Chen et al. | Multi-scale Feature Recovery of Low-Light Enhancement Algorithm Based on U-net Network and Perceptual Loss | |
| CN118154894A (en) | Layered significance guide visual feature extraction model establishment and quality evaluation method | |
| CN118037614A (en) | Infrared image enhancement method based on lightweight self-supervision network | |
| CN118135446A (en) | Lightweight short video scene classification method based on data enhancement | |
| CN117893400A (en) | Cartoon style conversion method and system based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |