CN115242933B - Video image denoising method, device, equipment and storage medium - Google Patents
Video image denoising method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115242933B CN115242933B CN202110437115.7A CN202110437115A CN115242933B CN 115242933 B CN115242933 B CN 115242933B CN 202110437115 A CN202110437115 A CN 202110437115A CN 115242933 B CN115242933 B CN 115242933B
- Authority
- CN
- China
- Prior art keywords
- pixel
- calculating
- denoising
- images
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000001914 filtration Methods 0.000 claims abstract description 22
- 238000004364 calculation method Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 11
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a video image denoising method, device, equipment and storage medium. The motion area of the reference frame image is subjected to spatial denoising in advance, the superposition weights of the pixels at the same coordinate positions in the multiple continuous images are calculated based on wiener filtering, then the pixel values of the pixels of the reference frame image subjected to spatial denoising are superposed with the pixel values of the pixels at the same coordinate positions in other frame images in the multiple continuous images according to the corresponding superposition weights, the motion area can be effectively denoised, and the problems of insufficient denoising effect of the motion area and 'stay' of the motion area are solved.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to a video image denoising method, device, equipment and storage medium.
Background
In the wide distribution of noise and natural images, when noise points in the images are serious, the image quality and visual effect can be seriously affected, and the denoising technology for images and video scenes is always a core research subject. The denoising technique can be widely applied to a large number of video and image scenes, such as video conference, video call, photographing, image beautification and the like.
Video denoising techniques require more consideration for the utilization of multi-frame information and the processing of moving regions than general image denoising techniques. If the effective information of the multi-frame is not considered in the presence of motion conditions, the noise is assumed to accord with Gaussian distribution, and the aligned pixel points can be subjected to noise reduction processing through good multi-frame average. However, due to the existence of moving objects in the scene, after the superposition of continuous video frames, if the solving or superposition weights of the moving areas are unreasonable, a phenomenon of non-overlapping exists in the moving areas, and the phenomenon of film is reflected in the video.
Aiming at the problem, most of the existing schemes adopt a method for self-adaptive adjustment of the weight of a moving area to solve the problem of edge smear, but if the moving area has a flat area (namely an area with the same pixel value in an image), the image of the part cannot be judged as the moving area, so that the moving area is treated as a static area, obvious rupture exists between the moving area and the video effect, and a certain 'stay' phenomenon exists.
Disclosure of Invention
The invention provides a video image denoising method, device, equipment and storage medium, which can effectively denoise a motion area and solve the problem of insufficient denoising effect of the motion area.
In a first aspect, an embodiment of the present invention provides a method for denoising a video image, including:
Acquiring a video image sequence, wherein the video image sequence comprises a plurality of continuous images;
extracting a motion region from the sequence of video images;
Spatial domain denoising of a motion area of a reference frame image, wherein the reference frame image is the last frame image in the image sequence;
calculating the superposition weight of pixel points with the same coordinate position in the multi-frame continuous images based on wiener filtering;
And superposing the pixel value of the pixel point of the reference frame image after spatial domain denoising with the pixel value of the pixel point of the same coordinate position in other frame images in the multi-frame continuous image according to the corresponding superposition weight to obtain the denoised video image.
In a second aspect, an embodiment of the present invention further provides a video image denoising apparatus, including:
the image acquisition module is used for acquiring a video image sequence, wherein the video image sequence comprises a plurality of continuous images;
a motion region extraction module for extracting a motion region from the video image sequence;
The spatial domain denoising module is used for spatial domain denoising of a motion area of a reference frame image, wherein the reference frame image is the last frame image in the image sequence;
The superposition weight calculation module is used for calculating the superposition weight of the pixel points with the same coordinate position in the multi-frame continuous images based on wiener filtering;
and the superposition module is used for superposing the pixel value of the pixel point of the reference frame image after spatial domain denoising and the pixel value of the pixel point of the same coordinate position in other frame images in the multi-frame continuous image according to the corresponding superposition weight to obtain the denoised video image.
In a third aspect, an embodiment of the present invention further provides a computer apparatus, including:
One or more processors;
a storage means for storing one or more programs;
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement a video image denoising method as provided in the first aspect of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a video image denoising method as provided in the first aspect of the present invention.
According to the video image denoising method provided by the embodiment of the invention, after a video image sequence is acquired, a motion region is extracted from the video image sequence, spatial domain denoising is performed on the motion region of a reference frame image, then superposition weights of pixel points with the same coordinate positions in multiple continuous images are calculated based on wiener filtering, and then the pixel values of the pixel points of the reference frame image after spatial domain denoising and the pixel values of the pixel points with the same coordinate positions in other frame images in the multiple continuous images are superposed according to the corresponding superposition weights, so that the denoised video image is obtained. The motion area of the reference frame image is subjected to spatial denoising in advance, the superposition weights of the pixels at the same coordinate positions in the multiple continuous images are calculated based on wiener filtering, then the pixel values of the pixels of the reference frame image subjected to spatial denoising are superposed with the pixel values of the pixels at the same coordinate positions in other frame images in the multiple continuous images according to the corresponding superposition weights, the motion area can be effectively denoised, and the problems of insufficient denoising effect of the motion area and 'stay' of the motion area are solved.
Drawings
Fig. 1 is a flowchart of a video image denoising method according to a first embodiment of the present invention;
fig. 2A and 2B are flowcharts of a video image denoising method according to a second embodiment of the present invention;
Fig. 3 is a video image denoising apparatus according to a third embodiment of the present invention;
Fig. 4 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Example 1
Fig. 1 is a flowchart of a video image denoising method according to an embodiment of the present invention, where the method is applicable to a case where a flat area exists in a video image, and the method may be performed by a video image denoising apparatus according to an embodiment of the present invention, where the apparatus may be implemented by software and/or hardware, and is typically configured in a computer device, as shown in fig. 1, and the method specifically includes the following steps:
s101, acquiring a video image sequence, wherein the video image sequence comprises a plurality of continuous images.
In the embodiment of the invention, the video stream can be acquired in real time, and the video image sequence can be acquired from the video stream, and the image video sequence can comprise a plurality of continuous images.
S102, extracting a motion region from the video image sequence.
The moving region is a region in which a moving object exists in a video image, and there is a difference in pixel values of pixel points in the video image which appear as two frame images at the same coordinate position. In an embodiment of the invention, after the video image sequence is acquired, the motion region is extracted from the video image sequence. For example, in some embodiments of the present invention, an optical flow method, a method of determining a difference value pixel by pixel and thresholding, or other methods may be specifically used, and embodiments of the present invention are not limited herein.
S103, spatial domain denoising is performed on a motion area of a reference frame image, wherein the reference frame image is the last frame image in the image sequence.
In the embodiment of the invention, the reference frame image is the last frame image in the image sequence. For a certain frame of image, the independent denoising is also called spatial domain denoising, and the essence of spatial domain denoising is that the pixel value weighted superposition of a plurality of similar positions in a reference frame of image is consistent with the principle of the pixel value weighted superposition at different moments of the same position in the time domain.
S104, calculating superposition weights of pixel points with the same coordinate positions in multiple continuous images based on wiener filtering.
The image collected in the real scene is usually a superposition of noise signals and image signals, and can be described by an additive model:
x＝s+n
Where x is the acquired image, s is the original signal, and n is the superimposed noise signal. If the noise is too large, the original signal is submerged in the noise, and the obtained image x noise point is very large; if the noise is small relative to the original signal, the resulting image quality will be very good, i.e. we want to get a de-noised image of the real signal.
Based on the idea of wiener filtering (WIENER FILTERING), a filter is designed, and acts on each frame of image to obtain the superposition weight of the pixel points with the same coordinate position in the multi-frame continuous image, wherein the expression of the wiener filtering is as follows:
Wherein H (z) is the superposition weight, P ss represents the correlation coefficient between the original signal and the original signal itself, P nn represents the correlation coefficient between the noise signal and the noise signal itself, and z represents the image position coordinate.
For example, for the pixel points of the coordinate positions (a, b), the corresponding superposition weights of the images of each frame are (w 1,w2…wn).
S105, overlapping the pixel value of the pixel point of the reference frame image after spatial denoising with the pixel value of the pixel point of the same coordinate position in other frame images in the multi-frame continuous image according to the corresponding overlapping weight, and obtaining the denoised video image.
In the embodiment of the invention, after the superposition weights of the pixel points of each frame image at the same coordinate position are obtained, the pixel value of the pixel point of the reference frame image at the coordinate position after spatial domain denoising is multiplied by the superposition weights of the reference frame image at the coordinate position, the pixel value of the pixel point of other frame images in a plurality of continuous frames of images is multiplied by the superposition weights of the frame image at the coordinate position, and then the pixel values of each frame image weighted at the coordinate position are added, so that the denoised video image is obtained.
Because in the flat area, P ss of the non-motion area of each frame of image is smaller, the superposition weight approaches 0, and the weight of each frame of image is almost equal, namely the average denoising effect. The P ss of the motion area is large, the reference frame can obtain a large weight, the superposition weight is close to 1, the superposition result of the motion area is closer to the reference frame, and the final result shows the effect that the motion area is close to the original pixel value of the reference frame after weighted superposition. This brings a significant problem, the stationary region can be well denoised according to the wiener filter formula, and the moving region basically retains the noise of the reference frame. Therefore, the embodiment of the invention can remove the noise of the motion area in the final result by independently removing the noise of the motion area of the reference frame image in advance.
According to the video image denoising method provided by the embodiment of the invention, after a video image sequence is acquired, a motion region is extracted from the video image sequence, spatial domain denoising is performed on the motion region of a reference frame image, then superposition weights of pixel points with the same coordinate positions in multiple continuous images are calculated based on wiener filtering, and then the pixel values of the pixel points of the reference frame image after spatial domain denoising and the pixel values of the pixel points with the same coordinate positions in other frame images in the multiple continuous images are superposed according to the corresponding superposition weights, so that the denoised video image is obtained. The motion area of the reference frame image is subjected to spatial denoising in advance, the superposition weights of the pixels at the same coordinate positions in the multiple continuous images are calculated based on wiener filtering, then the pixel values of the pixels of the reference frame image subjected to spatial denoising are superposed with the pixel values of the pixels at the same coordinate positions in other frame images in the multiple continuous images according to the corresponding superposition weights, the motion area can be effectively denoised, and the problems of insufficient denoising effect of the motion area and 'stay' of the motion area are solved.
Example two
Fig. 2A and 2B are flowcharts of a video image denoising method according to a second embodiment of the present invention, where the details of the steps in the foregoing embodiment are described in detail based on the first embodiment, and as shown in fig. 2A and 2B, the method specifically includes the following steps:
s201, acquiring a video image sequence, wherein the video image sequence comprises a plurality of continuous images.
In the embodiment of the invention, the video stream can be acquired in real time, and the video image sequence can be acquired from the video stream, and the image video sequence can comprise a plurality of continuous images.
For image processing of a motion scene, the solution of the motion region is a very critical step. The method for solving the motion area has a plurality of different schemes in terms of effect and time consumption, and it is difficult to have a method for integrating the time consumption and the effect. The traditional optical flow method has the defects that the solving result is accurate, but the time consumption is serious; another very simple method is to judge the difference value pixel by pixel and thresholde, which can achieve a fast solution, but basically cannot solve for flat areas, and is greatly affected by noise fluctuations itself. An exemplary process of extracting a motion region from a video image sequence in an embodiment of the present invention will be described below with reference to steps S202 to S205.
S202, selecting a first neighborhood taking each pixel point on the reference frame image as a center.
And traversing all pixel points on the reference frame image, and selecting a first neighborhood taking each pixel point on the reference frame image as a center. Illustratively, the first neighborhood may be a rectangular region of 5×5 in size centered on each pixel point on the reference frame image.
S203, calculating the similarity between the target area with the same coordinate position as the first neighborhood and the first neighborhood in other frame images.
Specifically, according to the position of the center of the first neighborhood and the size of the first neighborhood, determining target areas with the same coordinate position and the same size as those of the first neighborhood in other frame images, and then calculating the similarity between the first neighborhood and the corresponding target area.
Specifically, a normalized correlation coefficient between the first neighborhood and the corresponding target region may be calculated as a similarity, where a calculation formula of the normalized correlation coefficient is as follows:
wherein ρ is a normalized correlation coefficient of the first neighborhood and the corresponding target region, x i is a pixel value of the i-th pixel in the first neighborhood, y i is a pixel value of the i-th pixel in the target region, u x is an average value of pixel values of all pixels in the first neighborhood, u y is an average value of pixel values of all pixels in the target region, σ x is a standard deviation of pixel values of all pixels in the first neighborhood, and σ y is a standard deviation of pixel values of all pixels in the target region.
S204, judging whether the similarity between the target area in the other frame images and the first neighborhood is smaller than a similarity threshold value.
And comparing the similarity between the target area in the other frame images and the first neighborhood with a preset similarity threshold value, and judging whether the similarity between the target area in the other frame images and the first neighborhood is smaller than the similarity threshold value or not. The normalized correlation coefficients of the target area and the first neighborhood in the other frame images are compared with a preset correlation coefficient threshold, and whether the normalized correlation coefficients of the target area and the first neighborhood in the other frame images are smaller than the correlation coefficient threshold is judged.
S205, when the similarity between the target area of any one frame image in other frame images and the first neighborhood is lower than a similarity threshold value, determining that the first neighborhood belongs to a motion area.
And when the similarity between the target area of any one of the other frame images and the corresponding first neighborhood is lower than a similarity threshold value, determining that the first neighborhood belongs to the motion area.
And calculating the similarity between each first neighborhood and the corresponding target area, so as to obtain the whole motion area.
In some embodiments of the present invention, before selecting the first neighborhood centered on each pixel point on the reference frame image, downsampling may be performed on multiple frames of continuous images to obtain downsampled images after downsampling, then a motion region is extracted from the downsampled images, and then upsampling is performed on the motion region to obtain a motion region with the same resolution as the reference frame image. Therefore, the extraction speed of the motion area can be improved, the real-time noise reduction requirement is met, in addition, the robustness of the motion area solution on a small scale is better, the situation that the motion flat area cannot be judged is greatly improved, and the problem of residence of the motion area is further solved.
An exemplary process of spatial denoising of a motion region of a reference frame image will be described below, and in the embodiment of the present invention, denoising is performed by adopting a method of similar pixel superposition based on a neighborhood gradient direction, and the steps S206 to S210 are referred to.
S206, selecting a second neighborhood which takes each pixel point on the motion area as a center.
Specifically, each pixel point on the motion area is traversed, and a second neighborhood with a preset size (for example, 5×5) is selected with each pixel point as a center. Illustratively, after the motion region is obtained in the above step, a binarized mask having the same size as the reference image is generated, wherein the value of the element of the motion region in the mask is 1, and the value of the element of the non-motion region is 0. And when the second neighborhood is selected, selecting a pixel point on the reference frame image corresponding to the element with the value of 1 on the mask as a center based on the mask.
S207, calculating the gradient direction of the second neighborhood.
Specifically, the gradient direction in the current second neighborhood is solved by using the Laplacian. In other embodiments of the present invention, the gradient direction of the second neighborhood may also be calculated by other methods, which are not described herein. The gradient direction may be horizontal, vertical or diagonal.
S208, selecting a plurality of pixel points in the gradient direction as target pixel points.
Specifically, along the obtained gradient direction, a plurality of pixel points (for example, 5) in the gradient direction are taken as target pixel points.
S209, calculating weights of a plurality of target pixel points based on wiener filtering.
Specifically, the correlation coefficient of the pixel values of the plurality of target pixel points is calculated as the term P ss in the wiener filter formula in the foregoing embodiment, the term P nn is unchanged, and the correlation coefficient of the noise signal and the noise signal is still adopted as the term P nn. Substituting the P ss item and the P nn item into a wiener filter formula to obtain weights of a plurality of target pixel points.
The step S209 may include the following sub-steps:
s2091 calculates the variance of the pixel values of the plurality of target pixel points as the first variance.
Specifically, the variance of the pixel values of the plurality of target pixel points is calculated as a first variance, and the first variance is changed to be a correlation coefficient of the pixel values of the plurality of target pixel points.
S2092 calculates the variance of the pixel values of the pixel points having the same coordinate positions in the flat area of the multi-frame continuous image as the second variance.
Specifically, firstly, a Sobel operator is adopted to extract flat areas from a plurality of continuous images respectively, so that the flat areas in each frame of image are obtained, and then, the intersection of a plurality of flat areas is taken as a final flat area. Then, the variance of the pixel values of the pixel points with the same coordinate positions in the intersection of the flat areas of the plurality of continuous images is calculated as a second variance.
In general, we consider the real signal of the flat area as a constant, and the fluctuation of the image pixels is brought about by noise only, so the second variance can be taken as the correlation coefficient P nn of the noise signal and the noise signal itself.
S2094, calculates the sum of the first variance and the second variance as the first sum value.
Substituting the first variance and the second variance into a wiener filter formula, and calculating the sum of the first variance and the second variance as a first sum value.
S2095, calculates the quotient of the first variance and the first sum as the weight of the plurality of target pixels.
Substituting the first variance and the second variance into a wiener filter formula, and calculating the quotient of the first variance and the first sum value as the weight of a plurality of target pixel points.
And S210, adding the pixel values of the plurality of target pixel points according to the corresponding weights, and taking the added pixel values as the pixel value of the pixel point in the center of the second neighborhood.
Specifically, the pixel values of the plurality of target pixel points are multiplied by the weights, and the sum is performed, and the obtained pixel value is used as the pixel value of the pixel point in the center of the second neighborhood.
And denoising the motion region of the reference frame image by performing the denoising process on the second neighborhood of each pixel point in the motion region of the reference frame image.
In another embodiment of the present invention, spatial denoising of the motion region of the reference frame image may also be performed by adopting a non-local mean value manner. The basic idea behind the non-local mean is that the estimate of the current pixel is obtained by a weighted average of pixels in the image that have a similar neighborhood structure to it. However, the method for overlapping similar pixels based on the neighborhood gradient direction provided by the embodiment of the invention has higher efficiency compared with a non-local mean method, and the effect is similar to the non-local mean method.
An exemplary process of calculating the superposition weights of pixel points of the same coordinate position in a plurality of continuous images based on wiener filtering is described below, referring to steps S211 to S214.
S211, calculating variances of pixel values of pixel points at the same coordinate positions in a plurality of continuous images as a first coefficient.
The variance of the pixel values of the pixel points at the same coordinate position in the multi-frame continuous image is calculated, and illustratively, the variance of the pixel values of the pixel points at the same coordinate position in the 5-frame image is calculated and used as a first coefficient, and the first coefficient is used as a P ss term in a wiener filter formula.
S212, calculating variances of pixel values of pixel points with the same coordinate positions in a flat area of a plurality of continuous images as a second coefficient.
Illustratively, the variance of the pixel values of the pixel points with the same coordinate positions in the flat areas of the multiple continuous images is already obtained in the aforementioned step S2092, and this step is only required to be invoked. The second coefficient (or second variance) is taken as the term P nn in the wiener filter formula.
S213, calculating the sum of the first coefficient and the second coefficient as a second sum value.
Substituting the first coefficient and the second coefficient into a wiener filter formula, and calculating the sum of the first coefficient and the second coefficient as a second sum value.
S214, calculating the quotient of the first coefficient and the second sum value to obtain the superposition weight.
Substituting the first coefficient and the second coefficient into a wiener filter formula, and calculating the quotient of the first coefficient and the second sum value as the superposition weight.
And S215, overlapping the pixel value of the pixel point of the reference frame image after spatial denoising with the pixel value of the pixel point of the same coordinate position in other frame images in the multi-frame continuous image according to the corresponding overlapping weight to obtain the denoised video image.
After the superposition weights of the pixel points of each frame image at the same coordinate position are obtained, the pixel value of the pixel point of the reference frame image at the coordinate position after spatial domain denoising is multiplied by the superposition weights of the reference frame image at the coordinate position, the pixel value of the pixel point of other frame images in multiple continuous frames of images at the coordinate position is multiplied by the superposition weights of the frame image at the coordinate position, and then the pixel values of each frame image after weighting at the coordinate position are added, so that the denoised video image is obtained.
According to the video image denoising method provided by the embodiment of the invention, the motion area of the reference frame image is spatially denoised in advance and the superposition weights of the pixel points with the same coordinate positions in the multiple continuous images are calculated based on wiener filtering, then the pixel values of the pixel points of the reference frame image after spatially denoised and the pixel values of the pixel points with the same coordinate positions in other frame images in the multiple continuous images are superposed according to the corresponding superposition weights, so that the motion area can be effectively denoised, and the problems of insufficient denoising effect of the motion area and 'residence' of the motion area are solved. In addition, the extraction speed of the motion area can be improved, the real-time noise reduction requirement is met, the situation that the motion flat area cannot be judged is improved, and the problem of residence of the motion area is further solved.
Example III
Fig. 3 is a video image denoising apparatus according to a third embodiment of the present invention, including:
an image acquisition module 301, configured to acquire a video image sequence, where the video image sequence includes multiple continuous frames of images;
a motion region extraction module 302, configured to extract a motion region from the video image sequence;
The spatial domain denoising module 303 is configured to spatially denoise a motion region of a reference frame image, where the reference frame image is a last frame image in the image sequence;
the superposition weight calculating module 304 is configured to calculate a superposition weight of pixel points at the same coordinate position in multiple continuous images based on wiener filtering;
And the superposition module 305 is configured to superimpose the pixel value of the pixel point of the reference frame image after spatial denoising with the pixel value of the pixel point of the same coordinate position in other frame images in the multiple continuous frames of images according to the corresponding superposition weight, so as to obtain the denoised video image.
In some embodiments of the present invention, the motion region extraction module 302 includes:
A first neighborhood selection sub-module, configured to select a first neighborhood centered on each pixel point on the reference frame image;
The similarity calculation submodule is used for calculating the similarity between a target area which is the same as the coordinate position of the first neighborhood and the first neighborhood in other frame images;
The judging submodule is used for judging whether the similarity between the target area in other frame images and the first neighborhood is smaller than a similarity threshold value or not;
And the motion region determining submodule is used for determining that the first neighborhood belongs to a motion region when the similarity between the target region of any one frame image in other frame images and the first neighborhood is lower than a similarity threshold value.
In some embodiments of the present invention, the video image denoising apparatus further includes:
and the downsampling module is used for downsampling a plurality of continuous frames of images before selecting a first neighborhood taking each pixel point on the reference frame image as a center to obtain downsampled images after downsampling.
In some embodiments of the present invention, the video image denoising apparatus further includes:
And the up-sampling module is used for carrying out up-sampling processing on the motion region after determining that the first neighborhood belongs to the motion region, so as to obtain the motion region with the same resolution as the reference frame image.
In some embodiments of the present invention, the spatial domain denoising module 303 includes:
A second neighborhood selection sub-module, configured to select a second neighborhood centered on each pixel point on the motion region;
The gradient direction calculation sub-module is used for calculating the gradient direction of the second neighborhood;
The target pixel point determining submodule is used for selecting a plurality of pixel points in the gradient direction as target pixel points;
the weight calculation sub-module is used for calculating weights of a plurality of target pixel points based on wiener filtering;
And the summation submodule is used for summing the pixel values of the plurality of target pixel points according to the corresponding weights and taking the pixel values as the pixel values of the pixel points in the center of the second neighborhood.
In some embodiments of the invention, the weight calculation submodule includes:
A first variance calculating unit configured to calculate variances of pixel values of a plurality of the target pixel points as first variances;
A second variance calculating unit for calculating variances of pixel values of pixel points having the same coordinate positions in a flat area of a plurality of continuous images as second variances;
A first sum value calculation unit configured to calculate a sum of the first variance and the second variance as a first sum value;
And the weight calculation unit is used for calculating the quotient of the first variance and the first sum value as the weight of a plurality of target pixel points.
In some embodiments of the present invention, the second variance calculating unit includes:
The flat region extraction subunit is used for respectively extracting flat regions from a plurality of continuous images by adopting a Sobel operator;
An intersection determination subunit for taking intersections of the plurality of flat areas;
And a second variance calculating subunit for calculating variances of pixel values of pixel points having the same coordinate positions within an intersection of flat areas of the plurality of continuous images as the second variances.
In some embodiments of the present invention, the superposition weight calculation module includes:
A first coefficient calculation sub-module, configured to calculate, as a first coefficient, a variance of pixel values of pixel points at the same coordinate position in multiple continuous images;
A second coefficient calculation sub-module for calculating, as a second coefficient, a variance of pixel values of pixel points having the same coordinate positions in a flat area of a plurality of continuous images;
A second sum calculation sub-module for calculating a sum of the first coefficient and the second coefficient as a second sum;
And the superposition weight calculation sub-module is used for calculating the quotient of the first coefficient and the second sum value to obtain superposition weight.
The video image denoising device can execute the video image denoising method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
A fourth embodiment of the present invention provides a computer device, and fig. 4 is a schematic structural diagram of the computer device provided in the fourth embodiment of the present invention, as shown in fig. 4, where the computer device includes:
A processor 401, a memory 402, a communication module 403, an input device 404, and an output device 405; the number of processors 401 in the computer device may be one or more, one processor 401 being exemplified in fig. 4; the processor 401, memory 402, communication module 403, input means 404 and output means 405 in the computer device may be connected by a bus or other means, in fig. 4 by way of example. The processor 401, the memory 402, the communication module 403, the input means 404 and the output means 405 may be integrated on a computer device.
The memory 402 is a computer readable storage medium, and may be used to store a software program, a computer executable program, and a module, such as a module corresponding to the video image denoising method in the above embodiment. The processor 401 executes various functional applications of the computer device and data processing, i.e., implements the video image denoising method described above, by running software programs, instructions, and modules stored in the memory 402.
Memory 402 may include primarily a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for functionality; the storage data area may store data created according to the use of the microcomputer, and the like. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 402 may further include memory remotely located relative to processor 401, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The communication module 403 is configured to establish a connection with an external device (e.g. an intelligent terminal), and implement data interaction with the external device. The input device 404 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the computer device.
The computer equipment provided by the embodiment can execute the video image denoising method provided by any embodiment of the invention, and has corresponding functions and beneficial effects.
Example five
A fifth embodiment of the present invention provides a storage medium containing computer executable instructions, where a computer program is stored, the program when executed by a processor implementing a video image denoising method as provided in any of the above embodiments of the present invention, the method comprising:
Acquiring a video image sequence, wherein the video image sequence comprises a plurality of continuous images;
extracting a motion region from the sequence of video images;
Spatial domain denoising of a motion area of a reference frame image, wherein the reference frame image is the last frame image in the image sequence;
Calculating the superposition weight of pixel points with the same coordinate position in a plurality of continuous images based on wiener filtering;
And superposing the pixel value of the pixel point of the reference frame image after spatial domain denoising with the pixel value of the pixel point of the same coordinate position in other frame images in the multi-frame continuous image according to the corresponding superposition weight to obtain the denoised video image.
It should be noted that, for the apparatus, device and storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference should be made to the description of the method embodiments for relevant points.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a FLASH Memory (FLASH), a hard disk, or an optical disk of a computer, where the instructions include a plurality of instructions for causing a computer device (which may be a robot, a personal computer, a server, or a network device, etc.) to perform the video image denoising method according to any embodiment of the present invention.
It should be noted that, in the above apparatus, each module, sub-module, unit, and sub-unit included is only divided according to the functional logic, but is not limited to the above division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (10)
1. A method for denoising a video image, comprising:
Acquiring a video image sequence, wherein the video image sequence comprises a plurality of continuous images;
extracting a motion region from the sequence of video images;
Spatial domain denoising of a motion area of a reference frame image, wherein the reference frame image is the last frame image in the image sequence;
calculating the superposition weight of pixel points with the same coordinate position in the multi-frame continuous images based on wiener filtering;
Overlapping the pixel value of the pixel point of the reference frame image after spatial domain denoising with the pixel value of the pixel point of the same coordinate position in other frame images in the multi-frame continuous image according to the corresponding overlapping weight to obtain a denoised video image;
the calculating the superposition weight of the pixel points with the same coordinate position in the multi-frame continuous images based on wiener filtering comprises the following steps:
calculating variances of pixel values of pixel points at the same coordinate positions in the continuous images of the multiple frames as first coefficients;
Calculating the variance of pixel values of pixel points with the same coordinate positions in a flat area of the multi-frame continuous images as a second coefficient;
Calculating a sum of the first coefficient and the second coefficient as a second sum value;
and calculating the quotient of the first coefficient and the second sum value to obtain the superposition weight.
2. The method of denoising video images according to claim 1, wherein extracting a motion region from the sequence of video images comprises:
Selecting a first neighborhood taking each pixel point on the reference frame image as a center;
Calculating the similarity between a target area with the same coordinate position as the first neighborhood and the first neighborhood in other frame images;
judging whether the similarity between a target area in other frame images and the first neighborhood is smaller than a similarity threshold value;
And when the similarity between the target area of any one of the other frame images and the first neighborhood is lower than a similarity threshold value, determining that the first neighborhood belongs to a motion area.
3. The method of denoising a video image according to claim 2, further comprising, prior to selecting a first neighborhood centered at each pixel on the reference frame image:
and carrying out downsampling processing on the multi-frame continuous images to obtain downsampled images after the downsampling processing.
4. A method of denoising a video image according to claim 3, further comprising, after determining that the first neighborhood belongs to a motion region:
and carrying out up-sampling processing on the motion region to obtain the motion region with the same resolution as the reference frame image.
5. The method of denoising video image according to any one of claims 1 to 4, wherein spatial denoising for a motion region of a reference frame image comprises:
selecting a second neighborhood taking each pixel point on the motion area as a center;
calculating the gradient direction of the second neighborhood;
selecting a plurality of pixel points in the gradient direction as target pixel points;
calculating weights of a plurality of target pixel points based on wiener filtering;
And adding the pixel values of the plurality of target pixel points according to the corresponding weights to serve as the pixel value of the pixel point in the center of the second neighborhood.
6. The method of denoising video image according to claim 5, wherein calculating weights for a plurality of the target pixel points based on wiener filtering comprises:
calculating variances of pixel values of a plurality of target pixel points as first variances;
Calculating variances of pixel values of pixel points with the same coordinate positions in a flat area of the multi-frame continuous images as second variances;
Calculating a sum of the first variance and the second variance as a first sum value;
and calculating the quotient of the first variance and the first sum value as the weight of a plurality of target pixel points.
7. The method according to claim 6, wherein calculating the variance of pixel values of pixel points having the same coordinate positions in a flat area of the plurality of continuous images as the second variance comprises:
respectively extracting flat areas from the continuous images of the multiple frames by adopting a Sobel operator;
Taking intersections of a plurality of flat regions;
and calculating the variance of the pixel values of the pixel points with the same coordinate positions in the intersection of the flat areas of the continuous images as a second variance.
8. A video image denoising apparatus, comprising:
the image acquisition module is used for acquiring a video image sequence, wherein the video image sequence comprises a plurality of continuous images;
a motion region extraction module for extracting a motion region from the video image sequence;
The spatial domain denoising module is used for spatial domain denoising of a motion area of a reference frame image, wherein the reference frame image is the last frame image in the image sequence;
The superposition weight calculation module is used for calculating the superposition weight of the pixel points with the same coordinate position in the multi-frame continuous images based on wiener filtering;
The superposition module is used for superposing the pixel value of the pixel point of the reference frame image after spatial domain denoising and the pixel value of the pixel point of the same coordinate position in other frame images in the multi-frame continuous image according to the corresponding superposition weight to obtain a denoised video image;
The superposition weight calculation module comprises:
A first coefficient calculation sub-module, configured to calculate, as a first coefficient, a variance of pixel values of pixel points at the same coordinate position in the multiple continuous frames of images;
a second coefficient calculation sub-module, configured to calculate, as a second coefficient, a variance of pixel values of pixel points with the same coordinate positions in a flat area of the multiple continuous images;
A second sum calculation sub-module for calculating a sum of the first coefficient and the second coefficient as a second sum;
And the superposition weight calculation sub-module is used for calculating the quotient of the first coefficient and the second sum value to obtain superposition weight.
9. A computer device, comprising:
One or more processors;
a storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the video image denoising method of any one of claims 1-7.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a video image denoising method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110437115.7A CN115242933B (en) | 2021-04-22 | 2021-04-22 | Video image denoising method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110437115.7A CN115242933B (en) | 2021-04-22 | 2021-04-22 | Video image denoising method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115242933A CN115242933A (en) | 2022-10-25 |
| CN115242933B true CN115242933B (en) | 2024-11-01 |
Family
ID=83666196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110437115.7A Active CN115242933B (en) | 2021-04-22 | 2021-04-22 | Video image denoising method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115242933B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117011193B (en) * | 2023-09-28 | 2023-12-05 | 生态环境部长江流域生态环境监督管理局生态环境监测与科学研究中心 | Light staring satellite video denoising method and denoising system |
| CN117830142B (en) * | 2024-03-06 | 2024-05-14 | 深圳市云影天光科技有限公司 | Video frame-by-frame denoising method and system based on intelligent recognition image processing |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105472204A (en) * | 2014-09-05 | 2016-04-06 | 南京理工大学 | Inter-frame noise reduction method based on motion detection |
| EP3454560A1 (en) * | 2008-04-10 | 2019-03-13 | QUALCOMM Incorporated | Advanced interpolation techniques for motion compensation in video coding |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010064316A1 (en) * | 2008-12-05 | 2010-06-10 | オリンパス株式会社 | Image processing device, image processing method, and image processing program |
-
2021
- 2021-04-22 CN CN202110437115.7A patent/CN115242933B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3454560A1 (en) * | 2008-04-10 | 2019-03-13 | QUALCOMM Incorporated | Advanced interpolation techniques for motion compensation in video coding |
| CN105472204A (en) * | 2014-09-05 | 2016-04-06 | 南京理工大学 | Inter-frame noise reduction method based on motion detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115242933A (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Single image rain streak decomposition using layer priors | |
| Kim et al. | Optimized contrast enhancement for real-time image and video dehazing | |
| Zhang et al. | Spatially variant defocus blur map estimation and deblurring from a single image | |
| US20160048952A1 (en) | Algorithm and device for image processing | |
| US20150319373A1 (en) | Method and device to compose an image by eliminating one or more moving objects | |
| CN102968814B (en) | A kind of method and apparatus of image rendering | |
| CN115242933B (en) | Video image denoising method, device, equipment and storage medium | |
| Patil et al. | Motion saliency based generative adversarial network for underwater moving object segmentation | |
| CN107194946B (en) | FPGA-based infrared salient object detection method | |
| CN112633274A (en) | Sonar image target detection method and device and electronic equipment | |
| WO2022233252A1 (en) | Image processing method and apparatus, and computer device and storage medium | |
| Roy et al. | A comprehensive survey on computer vision based approaches for moving object detection | |
| Xu et al. | Improved Canny Edge Detection Operator | |
| Patro | Design and implementation of novel image segmentation and BLOB detection algorithm for real-time video surveillance using DaVinci processor | |
| Tezuka et al. | A precise and stable foreground segmentation using fine-to-coarse approach in transform domain | |
| Chen et al. | Saliency-directed image interpolation using particle swarm optimization | |
| Chowdhury et al. | A background subtraction method using color information in the frame averaging process | |
| Ahn et al. | Segmenting a noisy low-depth-of-field image using adaptive second-order statistics | |
| CN111931793A (en) | Saliency target extraction method and system | |
| Wang et al. | Image haze removal using a hybrid of fuzzy inference system and weighted estimation | |
| Zhang et al. | A new image filtering method: Nonlocal image guided averaging | |
| Heinze et al. | Joint multi-frame demosaicing and super-resolution with artificial neural networks | |
| CN113592801A (en) | Method and device for detecting stripe interference of video image | |
| Vijayakumar et al. | Moving object segmentation using enhanced laplacian thresholding method for video surveillance | |
| CN112614100A (en) | Ocean plankton image segmentation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |