WO2022178975A1

WO2022178975A1 - Noise field-based image noise reduction method and apparatus, device, and storage medium

Info

Publication number: WO2022178975A1
Application number: PCT/CN2021/090430
Authority: WO
Inventors: 陈昊
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-02-26
Filing date: 2021-04-28
Publication date: 2022-09-01
Also published as: CN112950501B; CN112950501A

Abstract

The present application relates to a computer vision technology in the technical field of artificial intelligence, and discloses a noise field-based image noise reduction method and apparatus, a device, and a storage medium. In the present application, the method comprises: obtaining an image to be subjected to noise reduction and a noise-free image corresponding to the image to be subjected to noise reduction; inputting the noise-free image into a preset noise distribution model, and constructing a Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model; obtaining, according to the Gauss-Poisson joint noise distribution function, noise distribution information of the image to be subjected to noise reduction; importing the image to be subjected to noise reduction and the noise distribution information to a pre-trained noise reduction model; and performing, in the noise reduction model according to the noise distribution information, noise reduction on the image to be subjected to noise reduction to obtain a denoised image. In addition, the present application further relates to a blockchain technology, and the image to be subjected to noise reduction and the noise-free image can be stored in a blockchain. According to the present application, not only can a clear and clean noise-reduced image be obtained, but the distortion generated during image noise reduction can also be prevented.

Description

Image noise reduction method, device, device and storage medium based on noise field

This application claims the priority of the Chinese patent application filed on February 26, 2021 with the application number 202110219477.9 and the invention titled "Method, Apparatus, Equipment and Storage Medium for Image Noise Reduction Based on Noise Field", all of which The contents are incorporated herein by reference.

technical field

The present application relates to the technical field of artificial intelligence, and in particular, to a noise field-based image noise reduction method, device, device, and storage medium.

Background technique

With the widespread use of artificial intelligence in the financial field, there are more and more scenarios in which financial behaviors are carried out on mobile terminals. These scenarios involve relatively strict approval services, such as face recognition services, which require customer Image extraction, for the sake of experience, this kind of background is often based on the picture obtained by extracting frames from the contact video shot by the camera, but in the process of image noise reduction, the inventor realized that the current obtained in this way The quality of the pictures is often not guaranteed, especially the movement of the mobile terminal itself will cause the blurring of the images. Such image defects are unfavorable for subsequent tasks such as face recognition. Therefore, in order to effectively improve the accuracy of subsequent services such as face recognition, it is imperative to optimize image quality.

A necessary step in the process of optimizing image quality is to denoise the image. Usually, image denoising is divided into two ideas: one is to use artificially designed models to denoise images; the other is to use deep learning estimation for image noise reduction. The former is often not applicable in most cases because the designed model can only be aimed at a specific situation; the image noise reduction method of deep learning convolutional neural network emerging in recent years, although it has good adaptability, However, the actual principle of the deep learning prediction method is to make the model predict the image as a whole. After the image is predicted, the overall image is denoised according to the prediction result. However, this overall denoising method is often localized Regions cause distortion of the image.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to propose an image noise reduction method, device, computer equipment and storage medium based on a noise field, so as to solve the problem that the existing image noise reduction scheme has poor applicability and local noise reduction after noise reduction. Technical issues of regional image distortion.

In order to solve the above technical problems, the embodiments of the present application provide an image noise reduction method based on a noise field, which adopts the following technical solutions:

An image noise reduction method based on noise field, comprising:

Obtain the image to be denoised and the noise-free image corresponding to the image to be denoised;

Input the noise-free image into the preset noise distribution model, and construct the Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model;

Obtain the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function;

Import the image to be denoised and the noise distribution information into the pre-trained denoising model, and denoise the image to be denoised in the denoising model according to the noise distribution information to obtain a denoised image.

Further, a Gaussian kernel and a Poisson kernel are preset in the noise distribution model, the noise-free image is input into the preset noise distribution model, and the Gauss-Poisson joint noise distribution function of the noise-free image is constructed in the noise distribution model. steps, including:

Input the noise-free image into the noise distribution model, and obtain the pixel information of the noise-free image through the noise distribution model;

The Gauss-Poisson joint noise distribution function of the noise-free image is constructed based on the pixel information of the noise-free image, the Poisson kernel and the Gaussian kernel.

Further, before the step of inputting the noise-free image into the noise distribution model, and obtaining the pixel information of the noise-free image through the noise distribution model, the method further includes:

Obtain a sample image from a preset image database, input the sample image into a preset initial noise distribution model, and obtain the output result of the initial noise distribution model;

Construct the loss function of the initial noise distribution model, and use the loss function of the noise distribution model to calculate the error based on the output results and the preset standard results to obtain the recognition error;

The noise distribution model is iterated based on the recognition error until the model is fitted, and the output fitted noise distribution model is obtained.

Further, the steps of iterating the noise distribution model based on the recognition error until the model is fitted, and obtaining the fitted noise distribution model, specifically includes:

Compare the recognition error with the preset error threshold;

If the recognition error is greater than the preset error threshold, the initial noise distribution model is iteratively updated based on the back-propagation algorithm until the recognition error is less than or equal to the preset error threshold, and the output fitted noise distribution model is obtained;

Output noise distribution model.

Further, the step of obtaining the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function specifically includes:

The Gauss-Poisson joint noise distribution function is randomly sampled based on the pixel information of the noise-free image, and the sampling matrix is obtained;

The noise distribution information of the image to be denoised is obtained according to the sampling matrix.

Further, the step of obtaining the noise distribution information of the image to be denoised according to the sampling matrix specifically includes:

Obtain the image matrix of the noise-free image, and fuse the image matrix and the sampling matrix of the noise-free image to obtain the first noise image;

obtaining a camera response function corresponding to the image to be denoised, and processing the first noise image based on the camera response function to obtain a second noise image;

acquiring format conversion information corresponding to the image to be denoised, and performing color interpolation on the second noise image based on the format conversion information to obtain a third noise image;

obtaining compression parameters corresponding to the image to be denoised, and compressing the third noise image based on the compression parameters to obtain a fourth noise image;

An image matrix of the fourth noise image is obtained, and noise distribution information of the image to be denoised is obtained based on the image matrix of the fourth noise image.

Further, import the image to be denoised and the noise distribution information into the pre-trained denoising model, and denoise the image to be denoised in the denoising model according to the noise distribution information to obtain the denoised image, specifically including:

Obtain the noise distribution matrix based on the noise distribution information;

Perform matrix splicing on the image matrix and noise distribution matrix of the image to be denoised to obtain a matrix splicing tensor;

Use the convolution check of the noise reduction model to perform the convolution operation on the matrix splicing tensor to obtain the result of the convolution operation;

Image reconstruction is performed based on the result of the convolution operation to obtain a denoised image.

In order to solve the above technical problems, the embodiments of the present application also provide an image noise reduction device based on a noise field, which adopts the following technical solutions:

An image noise reduction device based on noise field, comprising:

an image acquisition module for acquiring the image to be denoised and the noise-free image corresponding to the image to be denoised;

The function building module is used to input the noise-free image into the preset noise distribution model, and construct the Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model;

The noise simulation module is used to obtain the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function;

The image denoising module is used to import the image to be denoised and the noise distribution information into the pre-trained denoising model, and denoise the denoising image in the denoising model according to the noise distribution information to obtain a denoised image.

In order to solve the above-mentioned technical problems, the embodiment of the present application also provides a computer device, which adopts the following technical solutions:

A computer device includes a memory and a processor, wherein computer-readable instructions are stored in the memory, and the processor implements the following noise field-based image noise reduction method when executing the computer-readable instructions:

In order to solve the above technical problems, the embodiments of the present application also provide a computer-readable storage medium, which adopts the following technical solutions:

A computer-readable storage medium, where computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the following image noise reduction method based on noise field is implemented:

Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects:

The present application discloses an image noise reduction method, device, equipment and storage medium based on noise field, belonging to the technical field of artificial intelligence. The noise distribution model obtains a Gauss-Poisson joint noise distribution function. By randomly sampling the Gauss-Poisson joint noise distribution function, the noise distribution information of the image to be denoised is obtained by simulation, and then the pre-trained noise reduction model is based on the noise distribution information. Noise reduction is performed on the image to be denoised to obtain a denoised image. Compared with the existing unified denoising scheme for the whole image, the present application performs denoising based on the noise distribution, that is, the noise reduction intensity is increased for the places with more noise, and the noise reduction intensity is reduced for the places with less noise. , which can not only obtain a clear and clean noise-reduced image, but also prevent the distortion generated in the process of image noise reduction.

Description of drawings

In order to illustrate the solutions in the present application more clearly, the following will briefly introduce the accompanying drawings used in the description of the embodiments of the present application. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 shows an exemplary system architecture diagram to which the present application can be applied;

FIG. 2 shows a flowchart of an embodiment of a noise field-based image noise reduction method according to the present application;

FIG. 3 shows a schematic structural diagram of an embodiment of a noise field-based image noise reduction apparatus according to the present application;

FIG. 4 shows a schematic structural diagram of an embodiment of a computer device according to the present application.

Detailed ways

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of this application; the terms used herein in the specification of the application are for the purpose of describing specific embodiments only It is not intended to limit the application; the terms "comprising" and "having" and any variations thereof in the description and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings.

As shown in FIG. 1 , the system architecture 100 may include

terminal devices

101 , 102 , and 103 , a network 104 and a server 105 . The network 104 is a medium used to provide a communication link between the

terminal devices

101 , 102 , 103 and the server 105 . The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the

terminal devices

101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages and the like. Various communication client applications may be installed on the

terminal devices

101 , 102 and 103 , such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like.

The

terminal devices

101, 102, and 103 can be various electronic devices that have a display screen and support web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) Players, Laptops and Desktops, etc.

The server 105 may be a server that provides various services, such as a background server that provides support for the pages displayed on the

terminal devices

101 , 102 , and 103 .

It should be noted that the noise field-based image noise reduction method provided by the embodiments of the present application is generally performed by a server or a terminal device, and accordingly, the noise field-based image noise reduction apparatus is generally set in the server or terminal device.

It should be understood that the numbers of terminal devices, networks and servers in FIG. 1 are merely illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.

Continuing to refer to FIG. 2 , a flowchart of an embodiment of a method for noise field-based image denoising according to the present application is shown. The image noise reduction method based on noise field includes the following steps:

S201, acquiring an image to be denoised and a noise-free image corresponding to the image to be denoised;

Specifically, an image noise reduction instruction is received, and an image to be denoised and a noise-free image corresponding to the noise-reduced image are obtained, wherein the noise-free image is the same or similar to the image content of the image to be de-noised, for example, in the same scene, using Images captured by the same camera, noise-free images refer to images whose image indicators meet the preset requirements. Image indicators such as resolution, exposure, saturation, etc., can be preset to meet the requirements of the image indicators and will meet the preset image requirements The image required by the index is determined as a noise-free image, the image noise of the image to be denoised is simulated on the noise-free image, and the noise distribution information is generated based on the simulated image noise, and the denoised image is to be denoised based on the noise distribution information and the pre-trained noise reduction model. Noise reduction is performed to obtain a denoised image.

In this embodiment, the electronic device (for example, the server or terminal device shown in FIG. 1 ) on which the noise field-based image noise reduction method runs may receive the image noise reduction instruction through wired connection or wireless connection. It should be pointed out that the above wireless connection methods may include but are not limited to 3G/4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future .

S202: Input the noise-free image into a preset noise distribution model, and construct a Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model.

Among them, image noise is divided into two categories, namely signal-related noise and signal-unrelated noise. Signal-related noise is mainly shot noise, and the noise increases with the increase of the signal, which can be simulated by Poisson distribution. The uncorrelated noise of the signal is mainly random noise. The noise does not increase significantly when the signal increases. It conforms to the Gaussian distribution and can be simulated by the Gaussian distribution. Therefore, in a specific embodiment of the present application, the image noise can be fitted by using a joint Poisson-Gaussian distribution.

In a specific embodiment of the present application, the noise of the image to be denoised is simulated by constructing a noise distribution model. According to the above theoretical photon sensing, the noise generated by photon sensing, that is, the signal-related noise, can be modeled as Poisson noise, while the rest of the static disturbances can be modeled as Poisson noise. Noise, i.e. signal uncorrelated noise, can be modeled as a Gaussian distribution. Input the noise-free image into the preset noise distribution model, and construct the Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model, and generate the image noise of the image to be denoised by simulating the Gauss-Poisson joint noise distribution function. , the Gauss-Poisson joint noise distribution function can be defined as follows:

Among them, L is considered to be an ideal noise-free image, σ _s is considered to be the multiplicative noise related to the signal, σ _c is considered to be the additive noise independent of the signal, and σ ² is the image noise generated by the simulation.

S203: Acquire noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function.

Specifically, the Gauss-Poisson joint noise distribution function is randomly sampled based on the pixel information of the noise-free image to obtain a sampling matrix, and the noise distribution information of the image to be denoised is obtained according to the sampling matrix. For example, if the input noise-free image is an image of size 512*512, the Gauss-Poisson joint noise distribution function needs to be randomly sampled 512*512 times, and the results obtained by random sampling are combined to obtain a 512*512 sampling matrix, The sampling matrix is added to the noise-free image, and the camera noise, format conversion noise and compression noise related to the image to be de-noised are added, and finally a real image with noise corresponding to the image to be de-noised is obtained by simulation. The noise distribution information of the image to be denoised is obtained from the noise-free real image and the noise-free image.

S204 , import the image to be denoised and the noise distribution information into the pre-trained denoising model, and denoise the image to be denoised in the denoising model according to the noise distribution information to obtain a denoised image.

Among them, the noise reduction module here is a U-shaped convolutional network, which is divided into an encoding layer encoder and a decoding layer decoder, and the encoding layer encoder contains a total of three layers of convolutional networks, and the number of convolution channels of each layer of encoding layer encoder is in order are 64, 128, 256. The corresponding decoding layer decoder also includes a three-layer convolutional network, and the number of convolution channels of each layer of the decoding layer decoder is 256, 128, and 64 in turn.

Specifically, the image matrix of the image to be denoised is obtained, the noise distribution matrix is obtained based on the noise distribution information, and the image matrix of the image to be denoised and the noise distribution matrix are matrix spliced to obtain a matrix splicing tensor, where the matrix splicing tensor is A three-dimensional tensor. After obtaining the matrix splicing tensor, the three-dimensional tensors are encoded by the three-layer encoding layer encoder, and then the corresponding encoding results are decoded by the three-layer decoding layer decoder respectively, and finally the decoding results are combined to obtain the output of the noise reduction model. This output is the denoised image.

The present application discloses an image noise reduction method based on noise field, which belongs to the technical field of artificial intelligence. The present application obtains a Gauss- Poisson joint noise distribution function, by randomly sampling the Gauss-Poisson joint noise distribution function, the noise distribution information of the image to be denoised is obtained by simulation, and then the pre-trained noise reduction model denoises the image to be denoised according to the noise distribution information. , to get the denoised image. Compared with the existing unified denoising scheme for the whole image, the present application performs denoising based on the noise distribution, that is, the noise reduction intensity is increased for the places with more noise, and the noise reduction intensity is reduced for the places with less noise. , which can not only obtain a clear and clean noise-reduced image, but also prevent the distortion generated in the process of image noise reduction.

Specifically, a Gaussian kernel and a Poisson kernel are preset in the noise distribution model, the noise-free image is input into the noise distribution model, the pixel information of the noise-free image is obtained through the noise distribution model, and the pixel information of the noise-free image is imported Perform Gaussian operation in the Gaussian kernel to obtain the Gaussian operation result, import the pixel information of the noise-free image into the Poisson kernel to perform the Poisson operation, obtain the Poisson operation result, and construct the Gauss-Poisson joint noise based on the Gaussian operation result and the Poisson operation result. Distribution function.

In the above embodiment, a Gaussian kernel and a Poisson kernel preset in the noise distribution model are used to process the pixel information of the noise-free image respectively, and the Gauss-Poisson joint noise distribution function is constructed and constructed through the obtained processing results, The Gauss-Poisson joint noise distribution function can be used to simulate image noise.

Specifically, the sample image may be a noise-free image, and the noise-free image is input into a preset initial noise distribution model to obtain the output result of the initial noise distribution model, where the output result of the initial noise distribution model is a matrix For example, if the sample image is an image of size 512*512, then the output result of the initial noise distribution model should also be a matrix of size 512*512. Based on the output result of the initial noise distribution model and the preset standard result, use the loss function of the noise distribution model to perform error calculation to obtain the recognition error. The preset standard result here may be an image containing noise corresponding to the noise-free image. Image. Based on the above identification error, the noise distribution model is iterated by using the back-propagation algorithm until the model is fitted, and the noise distribution model is obtained.

Among them, the initial noise distribution model can be a prediction model using the ResNet structure, ResNet refers to the abbreviation of Residual Network, and the ResNet prediction model is a classic neural network as the backbone of many computer vision tasks. A Gaussian kernel and a Poisson kernel are added to the ResNet prediction model to construct a noise distribution model, and the noise distribution of the image is simulated by the constructed noise distribution model. In the specific embodiment of this application, in order to truly describe the image noise to be denoised, a new loss function L needs to be defined here. The new loss function L includes the L1 loss function and the L2 loss function, and the specific definitions are as follows:

The advantage of designing L1 like this here is that it exhibits the asymmetry of simulated image noise, where i refers to the coordinates of the image matrix,

represents the output result of the initial noise distribution model, and σ represents the preset standard result, that is, the image matrix containing the noise image corresponding to the input sample image. I represents a step function, and when the calculation formula of I subscript is less than 0, its value is 1, and when the calculation formula of I subscript is greater than or equal to 0, its value is 0, α is a manually set constant, usually Can be set between 0 and 0.5 to modulate the value of the loss function.

In addition, in order to prevent the randomness from being too strong during the training process, the continuity of the prediction noise is too poor. According to the noise theory, the image structure has complete continuity, and the noise should also have continuity, rather than abrupt changes in some areas. Therefore, it is necessary to set the L2 loss function. L2 is the guarantee to prevent the distortion of subsequent image synthesis, among which, here

and

Refers to the horizontal and vertical differentiation of the output results, respectively. The weighting of the above two L1 and L2 constitutes the loss function L of our noise distribution model. The specific form of the loss function L is as follows:

L=ωL ₁ +βL ₂

Among them, ω and β are weighting coefficients. In the specific embodiment of the present application, the initial weighting coefficients of L1 and L2 are set to 0.5 and 0.5 respectively, and then the initial weighting coefficients are continuously adjusted according to the output result of the initial noise distribution model. .

In the above embodiment, the initial noise distribution model is trained through sample images, and the loss function of the initial noise distribution model is constructed, and the output error of the initial noise distribution model is calculated based on the constructed loss function, and the initial noise distribution model is calculated based on the output error. Iterate to obtain a noise distribution model that meets the requirements.

Compare the recognition error with the preset error threshold;

Output noise distribution model.

Among them, the backpropagation algorithm, that is, the error backpropagation algorithm (Backpropagationalgorithm, BP algorithm) is a learning algorithm suitable for multi-layer neuron networks. It is based on the gradient descent method and is used for the error calculation of deep learning networks. . The input and output relationship of BP network is essentially a mapping relationship: the function completed by a BP neural network with n input and m output is a continuous mapping from n-dimensional Euclidean space to a finite field in m-dimensional Euclidean space. A map is highly nonlinear. The learning process of BP algorithm consists of forward propagation process and back propagation process. In the process of forward propagation, the input information is processed layer by layer through the hidden layer through the input layer and transmitted to the output layer, and then transferred to the back propagation, and the partial derivative of the objective function to the weight of each neuron is obtained layer by layer, which constitutes The gradient of the objective function to the weight vector is used as the basis for modifying the weight.

Specifically, the identification error is compared with a preset error threshold, and if the identification error is greater than the preset error threshold, the initial noise distribution model after training is iteratively updated based on the back-propagation algorithm until the identification error is less than or equal to the preset error Up to the threshold, obtain the noise distribution model of the output fitting. The preset error threshold may be set in advance. In the above embodiment, the initial noise distribution model that has been trained is verified and iterated through the back-propagation algorithm to obtain a noise distribution model that meets the requirements.

In the above embodiment, a sampling matrix is obtained by randomly sampling the Gauss-Poisson joint noise distribution function, and the sampling matrix is added to the noise-free image, and the camera noise, format conversion noise and Compress the noise, and finally simulate a real image with noise corresponding to the image to be denoised. Based on the real image with noise and the noise-free image, the noise distribution information of the image to be de-noised can be obtained.

Specifically, the image matrix of the noise-free image is obtained, and the image matrix and the sampling matrix of the noise-free image are fused to obtain the first noise image. Considering the camera shooting factors, format conversion factors and image JPEG compression factors in the actual situation, it is necessary to take into account the above influences The factors process the first noise image in turn to simulate a real image with noise corresponding to the image to be denoised. The process of format conversion from the bayer image of the original camera to the RGB image, the format conversion process will generate Certain noise, image JPEG compression refers to the compression process before image transmission, and image compression will produce certain noise. In a specific embodiment of the present application, the specific operation process of sequentially processing the first noise image according to the above-mentioned influencing factors is as follows:

y=JPEG{f[DM(L+n(L))]}

Among them, L is considered to be an ideal noise-free image, where y is the real image with noise, f is the camera response function, and DM refers to the process from the bayer image to the RGB image, that is, the color interpolation process. For the JPEG format, JPEG is the image compression process. So far, the step of simulating image noise is completed, and a real image containing noise corresponding to the image to be denoised is obtained.

In the above embodiment, the first noise image is processed based on the camera shooting factor, the format conversion factor and the image JPEG compression factor, so that the first noise image obtains the camera noise, the format conversion noise and the compression noise, and finally obtains an image with The noise-containing real image corresponding to the image to be de-noised can be obtained based on the noise-containing real image and the noise-free image to obtain noise distribution information of the image to be de-noised.

Specifically, the noise distribution matrix is obtained based on the noise distribution information, and the image matrix of the image to be denoised and the noise distribution matrix are matrix spliced to obtain a matrix splicing tensor. The matrix splicing tensor is a three-dimensional condition tensor. The convolution kernel performs the convolution operation on the matrix splicing tensor to obtain the convolution operation result, and fills the convolution operation result into the matrix body of a blank matrix in turn, where the blank matrix is the same size as the image matrix of the image to be denoised, for example They are all 512*512-sized matrices. The above process is equivalent to reconstructing the image, and the denoised image is obtained after reconstruction. Here, the matrix splicing tensor is convolved in a condition-guided manner, and then the image is reconstructed based on the result of the convolution operation to obtain a denoised image.

In most noise reduction scenes, the default is uniform noise distribution for noise reduction, which is applicable in general scenes of natural images, but in mobile scenes, due to shooting angle, light distribution and other reasons, it will cause The noise distribution is not uniform. At this time, noise reduction should be carried out based on the noise distribution. In short, the noise reduction should be increased in places with more noise, and the noise reduction should be reduced in places with less noise. The obtained images are sharp, clean, and distorted images. To perform noise reduction based on the noise distribution, it is necessary to first estimate the noise distribution of the image, and then perform noise reduction based on the noise distribution.

In view of the above technical problems, the present application discloses an image noise reduction method, device, equipment and storage medium based on noise field, which belong to the technical field of artificial intelligence. Input the preset noise distribution model to obtain a Gauss-Poisson joint noise distribution function. By randomly sampling the Gauss-Poisson joint noise distribution function, the noise distribution information of the image to be denoised is obtained by simulation, and then the pre-trained denoising The model denoises the denoised image according to the noise distribution information, and obtains the denoised image. Compared with the existing unified denoising scheme for the whole image, the present application performs denoising based on the noise distribution, that is, the noise reduction intensity is increased for the places with more noise, and the noise reduction intensity is reduced for the places with less noise. , which can not only obtain a clear and clean noise-reduced image, but also prevent the distortion generated in the process of image noise reduction.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned images to be denoised and noise-free images, the above-mentioned images to be de-noised and noise-free images can also be stored in a node of a blockchain.

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. , when the computer-readable instructions are executed, the processes of the above-mentioned method embodiments may be included. Wherein, the aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

It should be understood that although the various steps in the flowchart of the accompanying drawings are sequentially shown in the order indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order and may be performed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution sequence is also It does not have to be performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of sub-steps or stages of other steps.

Further referring to FIG. 3 , as an implementation of the method shown in FIG. 2 above, the present application provides an embodiment of an image noise reduction device based on a noise field, and the device embodiment corresponds to the method embodiment shown in FIG. 2 . , the device can be specifically applied to various electronic devices.

As shown in FIG. 3 , the noise field-based image noise reduction device in this embodiment includes:

An image acquisition module 301, configured to acquire an image to be denoised and a noise-free image corresponding to the image to be denoised;

A function construction module 302, configured to input the noise-free image into a preset noise distribution model, and construct a Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model;

A noise simulation module 303, configured to obtain the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function;

The image denoising module 304 is used to import the image to be denoised and the noise distribution information into the pre-trained denoising model, and denoise the denoising image in the denoising model according to the noise distribution information to obtain a denoised image.

Further, a Gaussian kernel and a Poisson kernel are preset in the noise distribution model, and the function building module 302 specifically includes:

an information extraction unit, used for inputting the noise-free image into the noise distribution model, and obtaining pixel information of the noise-free image through the noise distribution model;

The function construction unit is used to construct the Gauss-Poisson joint noise distribution function of the noise-free image based on the pixel information of the noise-free image, the Poisson kernel and the Gaussian kernel.

Further, the image noise reduction device based on noise field also includes:

a sample acquisition module, configured to acquire a sample image from a preset image database, input the sample image into a preset initial noise distribution model, and obtain an output result of the initial noise distribution model;

The error calculation module is used to construct the loss function of the initial noise distribution model, and based on the output result and the preset standard result, use the loss function of the noise distribution model to perform error calculation to obtain the identification error;

The model iteration module is used to iterate the noise distribution model based on the recognition error until the model is fitted, and obtain the output fitted noise distribution model.

Further, the model iteration module specifically includes:

The error comparison unit is used to compare the recognition error with the preset error threshold;

The model iteration unit is used to iteratively update the initial noise distribution model based on the back-propagation algorithm when the recognition error is greater than the preset error threshold, until the recognition error is less than or equal to the preset error threshold, and obtain the output fitted noise distribution model ;

Model output unit for outputting the noise distribution model.

Further, the noise simulation module 303 specifically includes:

The random sampling unit is used to randomly sample the Gauss-Poisson joint noise distribution function based on the pixel information of the noise-free image to obtain a sampling matrix;

The noise simulation unit is used to obtain the noise distribution information of the image to be denoised according to the sampling matrix.

Further, the noise simulation unit specifically includes:

The matrix fusion subunit is used to obtain the image matrix of the noise-free image, and fuse the image matrix and the sampling matrix of the noise-free image to obtain the first noise image;

a camera noise simulation subunit, configured to obtain a camera response function corresponding to the image to be denoised, and process the first noise image based on the camera response function to obtain a second noise image;

a format conversion noise simulation subunit, used for acquiring format conversion information corresponding to the image to be denoised, and performing color interpolation on the second noise image based on the format conversion information to obtain a third noise image;

A compression noise simulation subunit, used for acquiring compression parameters corresponding to the image to be denoised, and compressing the third noise image based on the compression parameters to obtain a fourth noise image;

The noise distribution subunit is configured to acquire an image matrix of the fourth noise image, and obtain noise distribution information of the image to be denoised based on the image matrix of the fourth noise image.

Further, the image noise reduction module 304 specifically includes:

a distribution matrix unit, used to obtain a noise distribution matrix based on the noise distribution information;

The matrix splicing unit is used to perform matrix splicing of the image matrix and the noise distribution matrix of the image to be denoised to obtain a matrix splicing tensor;

The convolution operation unit is used to perform the convolution operation on the matrix splicing tensor by using the convolution check of the noise reduction model to obtain the result of the convolution operation;

The image reconstruction unit is used for image reconstruction based on the result of the convolution operation to obtain a denoised image.

The present application discloses an image noise reduction device based on noise field, which belongs to the technical field of artificial intelligence. The present application obtains a Gauss- Poisson joint noise distribution function, by randomly sampling the Gauss-Poisson joint noise distribution function, the noise distribution information of the image to be denoised is obtained by simulation, and then the pre-trained noise reduction model denoises the image to be denoised according to the noise distribution information. , to get the denoised image. Compared with the existing unified denoising scheme for the whole image, the present application performs denoising based on the noise distribution, that is, the noise reduction intensity is increased for the places with more noise, and the noise reduction intensity is reduced for the places with less noise. , which can not only obtain a clear and clean noise-reduced image, but also prevent the distortion generated in the process of image noise reduction.

To solve the above technical problems, the embodiments of the present application also provide computer equipment. Please refer to FIG. 4 for details. FIG. 4 is a block diagram of a basic structure of a computer device according to this embodiment.

The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that communicate with each other through a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure, but it should be understood that it is not required to implement all of the shown components, and more or less components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, special-purpose Integrated circuit (Application Specific Integrated Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital processor (Digital Signal Processor, DSP), embedded equipment, etc.

The computer equipment may be a desktop computer, a notebook computer, a palmtop computer, a cloud server and other computing equipment. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touch pad or a voice control device.

The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4 , such as a hard disk or a memory of the computer device 4 . In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Of course, the memory 41 may also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4 , such as computer-readable instructions for an image noise reduction method based on a noise field. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

The processor 42 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. This processor 42 is typically used to control the overall operation of the computer device 4 . In this embodiment, the processor 42 is configured to execute computer-readable instructions stored in the memory 41 or process data, for example, computer-readable instructions for executing the noise field-based image noise reduction method.

The network interface 43 may include a wireless network interface or a wired network interface, and the network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

The present application discloses a computer device, which belongs to the technical field of artificial intelligence. The present application obtains a Gauss-Poisson joint noise distribution function by inputting a noise-free image with similar or identical content to the image to be denoised into a preset noise distribution model , by randomly sampling the Gauss-Poisson joint noise distribution function, the noise distribution information of the image to be denoised is obtained by simulation, and then the pre-trained noise reduction model denoises the denoised image according to the noise distribution information to obtain a denoised image. Compared with the existing unified denoising scheme for the whole image, the present application performs denoising based on the noise distribution, that is, the noise reduction intensity is increased for the places with more noise, and the noise reduction intensity is reduced for the places with less noise. , which can not only obtain a clear and clean noise-reduced image, but also prevent the distortion generated in the process of image noise reduction.

The present application also provides another implementation manner, that is, to provide a computer-readable storage medium, the computer-readable storage medium may be non-volatile or volatile, and the computer-readable storage medium stores Computer readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the noise field based image noise reduction method as described above.

The present application discloses a storage medium, which belongs to the technical field of artificial intelligence. The present application obtains a Gauss-Poisson joint noise distribution function by inputting a noise-free image that is similar or identical to the content of the image to be de-noised into a preset noise distribution model , by randomly sampling the Gauss-Poisson joint noise distribution function, the noise distribution information of the image to be denoised is obtained by simulation, and then the pre-trained noise reduction model denoises the denoised image according to the noise distribution information to obtain a denoised image. Compared with the existing unified denoising scheme for the whole image, the present application performs denoising based on the noise distribution, that is, the noise reduction intensity is increased for the places with more noise, and the noise reduction intensity is reduced for the places with less noise. , which can not only obtain a clear and clean noise-reduced image, but also prevent the distortion generated in the process of image noise reduction.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the scope of the patent of the present application. This application may be embodied in many different forms, rather these embodiments are provided so that a thorough and complete understanding of the disclosure of this application is provided. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements for some of the technical features. . Any equivalent structure made by using the contents of the description and drawings of the present application, which is directly or indirectly used in other related technical fields, is also within the scope of protection of the patent of the present application.

Claims

An image noise reduction method based on noise field, comprising:

acquiring an image to be denoised and a noise-free image corresponding to the image to be denoised;

Inputting the noise-free image into a preset noise distribution model, and constructing a Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model;

Acquiring noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function;

Import the image to be denoised and the noise distribution information into a pre-trained denoising model, and denoise the image to be denoised in the denoising model according to the noise distribution information to obtain a denoised image. noisy image.
The noise field-based image denoising method according to claim 1, wherein a Gaussian kernel and a Poisson kernel are preset in the noise distribution model, and the noise-free image is input into a preset noise distribution The steps of constructing the Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model include:

inputting the noise-free image into the noise distribution model, and obtaining pixel information of the noise-free image through the noise distribution model;

A Gauss-Poisson joint noise distribution function of the noise-free image is constructed based on the pixel information of the noise-free image, the Poisson kernel, and the Gaussian kernel.
The noise-field-based image denoising method according to claim 2, wherein, when the noise-free image is input into the noise distribution model, pixels of the noise-free image are obtained through the noise distribution model Before the steps of the information, also include:

Obtain a sample image from a preset image database, input the sample image into a preset initial noise distribution model, and obtain an output result of the initial noise distribution model;

constructing a loss function of the initial noise distribution model, and using the loss function of the noise distribution model to perform error calculation based on the output result and the preset standard result to obtain an identification error;

The noise distribution model is iterated based on the identification error until the model is fitted, and the fitted noise distribution model is output.
The noise-field-based image denoising method according to claim 1, wherein the step of iterating the noise distribution model based on the identification error until the model is fitted to obtain a fitted noise distribution model, Specifically include:

comparing the identification error with a preset error threshold;

If the identification error is greater than a preset error threshold, the initial noise distribution model is iteratively updated based on the back-propagation algorithm until the identification error is less than or equal to the preset error threshold, and an output fitted noise distribution model is obtained ;

The noise distribution model is output.
The noise field-based image denoising method according to any one of claims 1 to 4, wherein the step of acquiring the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function, Specifically include:

Randomly sample the Gauss-Poisson joint noise distribution function based on the pixel information of the noise-free image to obtain a sampling matrix;

The noise distribution information of the image to be denoised is obtained according to the sampling matrix.
The noise field-based image denoising method according to claim 5, wherein the step of obtaining the noise distribution information of the image to be denoised according to the sampling matrix specifically includes:

obtaining the image matrix of the noise-free image, and fusing the image matrix of the noise-free image and the sampling matrix to obtain a first noise image;

acquiring a camera response function corresponding to the image to be denoised, and processing the first noise image based on the camera response function to obtain a second noise image;

acquiring format conversion information corresponding to the image to be denoised, and performing color interpolation on the second noise image based on the format conversion information to obtain a third noise image;

Acquiring compression parameters corresponding to the image to be denoised, and compressing the third noise image based on the compression parameters to obtain a fourth noise image;

An image matrix of the fourth noise image is acquired, and noise distribution information of the image to be denoised is obtained based on the image matrix of the fourth noise image.
The image noise reduction method based on the noise field according to claim 5, wherein the image to be denoised and the noise distribution information are imported into a pre-trained noise reduction model, and the noise distribution information is based on the noise distribution information. The steps of denoising the image to be denoised in the denoising model to obtain the denoising image specifically include:

obtaining a noise distribution matrix based on the noise distribution information;

performing matrix splicing on the image matrix of the image to be denoised and the noise distribution matrix to obtain a matrix splicing tensor;

Use the convolution kernel of the noise reduction model to perform a convolution operation on the matrix splicing tensor to obtain a convolution operation result;

Image reconstruction is performed based on the result of the convolution operation to obtain a denoised image.
An image noise reduction device based on noise field, comprising:

an image acquisition module, configured to acquire an image to be denoised and a noise-free image corresponding to the image to be denoised;

a function building module for inputting the noise-free image into a preset noise distribution model, and constructing a Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model;

a noise simulation module, configured to obtain the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function;

An image noise reduction module, configured to import the image to be denoised and the noise distribution information into a pre-trained noise reduction model, and use the noise reduction model to denoise the noise reduction model according to the noise distribution information The image is denoised to obtain a denoised image.
A computer device, comprising a memory and a processor, wherein computer-readable instructions are stored in the memory, and the processor implements the following noise field-based image noise reduction method when executing the computer-readable instructions:

acquiring an image to be denoised and a noise-free image corresponding to the image to be denoised;

Inputting the noise-free image into a preset noise distribution model, and constructing a Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model;

Acquiring noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function;

Import the image to be denoised and the noise distribution information into a pre-trained denoising model, and denoise the image to be denoised in the denoising model according to the noise distribution information to obtain a denoised image. noisy image.
The computer device according to claim 9, wherein a Gaussian kernel and a Poisson kernel are preset in the noise distribution model, the noise-free image is input into the preset noise distribution model, and the noise-free image is input in the preset noise distribution model. The steps of constructing the Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model specifically include:

inputting the noise-free image into the noise distribution model, and obtaining pixel information of the noise-free image through the noise distribution model;

A Gauss-Poisson joint noise distribution function of the noise-free image is constructed based on the pixel information of the noise-free image, the Poisson kernel, and the Gaussian kernel.
The computer device of claim 10, wherein before the step of inputting the noise-free image into the noise distribution model, and obtaining pixel information of the noise-free image through the noise distribution model, further include:

Obtain a sample image from a preset image database, input the sample image into a preset initial noise distribution model, and obtain an output result of the initial noise distribution model;

constructing a loss function of the initial noise distribution model, and using the loss function of the noise distribution model to perform error calculation based on the output result and the preset standard result to obtain an identification error;

The noise distribution model is iterated based on the identification error until the model is fitted, and the fitted noise distribution model is output.
The computer device according to claim 9, wherein the step of iterating the noise distribution model based on the identification error until the model is fitted to obtain the output fitted noise distribution model specifically includes:

comparing the identification error with a preset error threshold;

If the identification error is greater than a preset error threshold, the initial noise distribution model is iteratively updated based on the back-propagation algorithm until the identification error is less than or equal to the preset error threshold, and an output fitted noise distribution model is obtained ;

The noise distribution model is output.
The computer device according to any one of claims 9 to 12, wherein the step of acquiring the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function specifically includes:

Randomly sample the Gauss-Poisson joint noise distribution function based on the pixel information of the noise-free image to obtain a sampling matrix;

The noise distribution information of the image to be denoised is obtained according to the sampling matrix.
The computer device according to claim 13, wherein the step of obtaining the noise distribution information of the image to be denoised according to the sampling matrix specifically comprises:

obtaining the image matrix of the noise-free image, and fusing the image matrix of the noise-free image and the sampling matrix to obtain a first noise image;

acquiring a camera response function corresponding to the image to be denoised, and processing the first noise image based on the camera response function to obtain a second noise image;

acquiring format conversion information corresponding to the image to be denoised, and performing color interpolation on the second noise image based on the format conversion information to obtain a third noise image;

Acquiring compression parameters corresponding to the image to be denoised, and compressing the third noise image based on the compression parameters to obtain a fourth noise image;

An image matrix of the fourth noise image is acquired, and noise distribution information of the image to be denoised is obtained based on the image matrix of the fourth noise image.
A computer-readable storage medium, storing computer-readable instructions on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, implements the following noise field-based image noise reduction method:

acquiring an image to be denoised and a noise-free image corresponding to the image to be denoised;

Inputting the noise-free image into a preset noise distribution model, and constructing a Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model;

Acquiring noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function;

Import the image to be denoised and the noise distribution information into a pre-trained denoising model, and denoise the image to be denoised in the denoising model according to the noise distribution information to obtain a denoised image. noisy image.
The computer-readable storage medium of claim 15, wherein a Gaussian kernel and a Poisson kernel are preset in the noise distribution model, the noise-free image is input into the preset noise distribution model, and The step of constructing the Gauss-Poisson joint noise distribution function of the noise-free image in the noise distribution model specifically includes:

inputting the noise-free image into the noise distribution model, and obtaining pixel information of the noise-free image through the noise distribution model;

A Gauss-Poisson joint noise distribution function of the noise-free image is constructed based on the pixel information of the noise-free image, the Poisson kernel, and the Gaussian kernel.
The computer-readable storage medium of claim 16, wherein, in the step of inputting the noise-free image into the noise distribution model, the step of obtaining pixel information of the noise-free image through the noise distribution model Before, also included:

Obtain a sample image from a preset image database, input the sample image into a preset initial noise distribution model, and obtain an output result of the initial noise distribution model;

constructing a loss function of the initial noise distribution model, and using the loss function of the noise distribution model to perform error calculation based on the output result and the preset standard result to obtain an identification error;

The noise distribution model is iterated based on the identification error until the model is fitted, and the fitted noise distribution model is output.
The computer-readable storage medium according to claim 15, wherein the step of iterating the noise distribution model based on the identification error until the model is fitted to obtain a fitted noise distribution model specifically comprises:

comparing the identification error with a preset error threshold;

If the identification error is greater than a preset error threshold, the initial noise distribution model is iteratively updated based on the back-propagation algorithm until the identification error is less than or equal to the preset error threshold, and an output fitted noise distribution model is obtained ;

The noise distribution model is output.
The computer-readable storage medium according to any one of claims 9 to 18, wherein the step of acquiring the noise distribution information of the image to be denoised according to the Gauss-Poisson joint noise distribution function specifically includes:

Randomly sample the Gauss-Poisson joint noise distribution function based on the pixel information of the noise-free image to obtain a sampling matrix;

The noise distribution information of the image to be denoised is obtained according to the sampling matrix.
The computer-readable storage medium according to claim 19, wherein the step of obtaining the noise distribution information of the image to be denoised according to the sampling matrix specifically comprises:

obtaining the image matrix of the noise-free image, and fusing the image matrix of the noise-free image and the sampling matrix to obtain a first noise image;

acquiring a camera response function corresponding to the image to be denoised, and processing the first noise image based on the camera response function to obtain a second noise image;

acquiring format conversion information corresponding to the image to be denoised, and performing color interpolation on the second noise image based on the format conversion information to obtain a third noise image;

Acquiring compression parameters corresponding to the image to be denoised, and compressing the third noise image based on the compression parameters to obtain a fourth noise image;

An image matrix of the fourth noise image is acquired, and noise distribution information of the image to be denoised is obtained based on the image matrix of the fourth noise image.