CN109348088B - Image noise reduction method and device, electronic equipment and computer readable storage medium - Google Patents
Image noise reduction method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN109348088B CN109348088B CN201811399524.7A CN201811399524A CN109348088B CN 109348088 B CN109348088 B CN 109348088B CN 201811399524 A CN201811399524 A CN 201811399524A CN 109348088 B CN109348088 B CN 109348088B
- Authority
- CN
- China
- Prior art keywords
- noise reduction
- level
- compensation level
- high compensation
- original image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The application provides an image noise reduction method and device, wherein the method comprises the following steps: acquiring a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation level comprises a high compensation level, a low compensation level smaller than the high compensation level, and a transition level between the high compensation level and the low compensation level; determining the weight corresponding to the high compensation level and the transition level according to the brightness of the face area in the original image shot at the high compensation level; and according to the weights corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction by multi-frame original image synthesis. Therefore, multi-frame or single-frame noise reduction is dynamically selected for the face area according to the brightness of the face area in the multi-frame original image shot at the high compensation level, and then the image with good face definition and less noise is obtained through fusion.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image denoising method and apparatus, an electronic device, and a computer-readable storage medium.
Background
With the development of intelligent terminal technology, the use of mobile terminal devices (such as smart phones, tablet computers, and the like) is becoming more and more popular. Most mobile terminal devices are internally provided with cameras, and with the enhancement of the processing capability of the mobile terminal and the development of camera technology, the quality of shot images is higher and higher. In daily life, more and more users use mobile terminal devices such as smart phones and tablet computers to shoot images, and the requirement on the shooting quality is higher and higher.
However, due to the limitation of the shooting scene, especially in a special scene such as a night scene, noise may be introduced when the image is shot, so that the face area in the shot image is not clear. Therefore, under the condition of the maximum definition of the human face region, the noise reduction processing is performed on the image, which is a problem to be solved urgently.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the application provides an image noise reduction method, an image noise reduction device, an electronic device and a computer readable storage medium, so as to solve the technical problems that in the related art, when a night scene is shot, the shot image has a high noise level, a clear portrait cannot be obtained, and user experience is affected.
An embodiment of an aspect of the present application provides an image denoising method, including:
acquiring a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level;
determining the weight corresponding to the high compensation level and the transition level according to the brightness of a face area in the original image shot at the high compensation level;
and according to the weight corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction synthesized by multiple frames of original images.
The image denoising method of the embodiment of the application obtains a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation level comprises a high compensation level, a low compensation level smaller than the high compensation level, and a transition level between the high compensation level and the low compensation level; determining the weight corresponding to the high compensation level and the transition level according to the brightness of the face area in the original image shot at the high compensation level; and according to the weights corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction by multi-frame original image synthesis. Therefore, multi-frame or single-frame noise reduction is dynamically selected for the face area according to the brightness of the face area in the multi-frame original image shot at the high compensation level, and then the image with good face definition and less noise is obtained through fusion.
In another aspect, an embodiment of the present application provides an image noise reduction apparatus, including:
the acquisition module is used for acquiring a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level;
the determining module is used for determining the weight corresponding to the high compensation level and the transition level according to the brightness of a face area in an original image shot at the high compensation level;
and the fusion module is used for fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level according to the weight corresponding to the high compensation level and the transition level to obtain the face region subjected to noise reduction synthesized by the multi-frame original images.
The image noise reduction device of the embodiment of the application obtains the multi-frame original image shot by adopting the corresponding exposure compensation grade; wherein the exposure compensation level comprises a high compensation level, a low compensation level smaller than the high compensation level, and a transition level between the high compensation level and the low compensation level; determining the weight corresponding to the high compensation level and the transition level according to the brightness of the face area in the original image shot at the high compensation level; and according to the weights corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction by multi-frame original image synthesis. Therefore, multi-frame or single-frame noise reduction is dynamically selected for the face area according to the brightness of the face area in the multi-frame original image shot at the high compensation level, and then the image with good face definition and less noise is obtained through fusion.
An embodiment of another aspect of the present application provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the image noise reduction method as described in the foregoing embodiments when executing the program.
Yet another aspect of the present application is directed to one or more non-transitory computer-readable storage media containing computer-executable instructions, wherein the computer program is stored thereon, and when executed by a processor, implements the image denoising method as described in the above embodiments.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of an image denoising method according to an embodiment of the present application;
FIG. 2 is a schematic flowchart of another image denoising method according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an image noise reduction apparatus according to an embodiment of the present disclosure;
FIG. 4 is a block diagram of an electronic device according to some embodiments of the present application;
FIG. 5 is a block diagram of an image processing circuit according to some embodiments of the present disclosure.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
In the related art, when an image is shot in a night scene, the image is usually obtained by increasing the photosensitivity value because the light is dark, but noise is introduced by using high sensitivity in the shooting process, so that the face area of the image is unclear. Therefore, the image shot in the night scene has poor imaging quality and high noise level, and influences the user experience.
In order to solve the problems, the application provides an image noise reduction method, which comprises the steps of obtaining a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation level comprises a high compensation level, a low compensation level smaller than the high compensation level, and a transition level between the high compensation level and the low compensation level; determining the weight corresponding to the high compensation level and the transition level according to the brightness of the face area in the original image shot at the high compensation level; and according to the weights corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction by multi-frame original image synthesis.
An image noise reduction method and apparatus according to an embodiment of the present application are described below with reference to the drawings.
Fig. 1 is a schematic flowchart of an image denoising method according to an embodiment of the present application.
As shown in fig. 1, the image noise reduction method includes the steps of:
In the embodiment of the application, the electronic device for shooting the image can be a hardware device with various operating systems and imaging devices, such as a smart phone, a tablet computer, a personal digital assistant and a wearable device.
The original image refers to an image obtained by shooting through an electronic device without any processing.
The exposure compensation is a control mode of exposure, and is a method that after a subject is measured by electronic equipment, parameters of a shutter and aperture combination are obtained, and then the shutter speed obtained by the measurement is artificially changed by the exposure compensation. The exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level.
Because of the influence of factors such as light intensity and the shaking degree of the electronic device in a shooting scene, when the electronic device shoots an image, the image may be unclear or noise may be introduced, and therefore, multiple frames of original images are generally shot by adopting corresponding exposure compensation levels and are used for selecting multiple frames or single frames of original images with clear pictures to perform synthesis noise reduction processing.
It can be understood that a plurality of frames of original images photographed under the high compensation level, the low compensation level and the transition compensation level respectively to obtain the corresponding compensation levels are obtained.
As an example, suppose a set of exposure compensation values for acquiring a plurality of frames of original images is [ EV +1, EV0, EV-2, EV-4], where EV +1 may be a high compensation level, EV0 may be a transitional compensation level, and EV-2 and EV-4 may be low compensation levels.
And 102, determining the weight corresponding to the high compensation level and the transition level according to the brightness of the face area in the original image shot at the high compensation level.
Wherein, the original image shot at the high compensation level is at least two frames.
Due to the influence of factors such as the light intensity in a shooting scene and the shaking degree of electronic equipment, the brightness of a face area in an original image obtained by shooting according to a high compensation level may be appropriate, the face area is over-exposed or the face area is slightly bright but not over-exposed.
As a possible scenario, when the luminance of the face region in the original image captured at the high compensation level is appropriate, and it is determined that the degree of shaking of the electronic device is small, noise reduction processing of the face region may be performed on a plurality of frames of original images captured at the high compensation level.
In the embodiment of the application, displacement information of the imaging device in the shooting process can be acquired through a displacement sensor arranged on the electronic device, and then the current shaking degree of the electronic device is determined according to the acquired displacement information. As an example, the current shaking degree of the electronic device may be determined by acquiring current gyroscope (Gyro-sensor) information of the electronic device.
The gyroscope is also called an angular velocity sensor, and can measure the rotation angular velocity of the physical quantity during deflection and inclination. In the imaging device, the gyroscope can well measure the actions of rotation and deflection, so that the actual actions of a user can be accurately analyzed and judged. The gyroscope information (gyro information) of the electronic device may include motion information of the electronic device in three dimensional directions in a three-dimensional space, and the three dimensions of the three-dimensional space may be respectively expressed as three directions of an X axis, a Y axis, and a Z axis, where the X axis, the Y axis, and the Z axis are in a pairwise perpendicular relationship.
Therefore, in the embodiment of the application, the current jitter degree of the electronic equipment can be determined according to the current gyro information of the electronic equipment. The larger the absolute value of gyro motion of the electronic device in three directions, the larger the degree of shake of the electronic device. Specifically, absolute value thresholds of gyro motion in three directions may be preset, and the current shake degree of the imaging device may be determined according to a relationship between the sum of the acquired absolute values of gyro motion in the three directions and the preset threshold.
For example, it is assumed that the preset threshold values are a first threshold value a, a second threshold value B, and a third threshold value C, where a < B < C, and the sum of absolute values of gyro motion in three directions currently acquired is S. If S is less than A, determining that the current jitter degree of the electronic equipment is 'no jitter'; if A < S < B, the current shaking degree of the electronic equipment can be determined to be 'slight shaking'; if B < S < C, the current jitter degree of the electronic equipment can be determined to be 'small jitter'; if S > C, it may be determined that the current degree of shaking of the electronic device is "large shaking".
It should be noted that the above examples are only illustrative and should not be construed as limiting the present application. In actual use, the number of the threshold values and the specific numerical values of the threshold values can be preset according to actual needs, and the mapping relationship between gyro information and the jitter degree of the electronic device can be preset according to the relationship between gyro information and the threshold values.
As another possible scene, the brightness of the face area in the original image shot according to the high compensation level is moderate, but the face area obtained by shooting cannot be registered due to the fact that camera shaking is severe during shooting, or the face area in the synthesized image is prone to ghost, and at the moment, one frame of original image with the highest picture definition in the image shot at the high compensation level is selected to perform noise reduction processing on the face area.
As another possible scenario, the brightness of the face region in the original image captured according to the high compensation level is overexposed due to overexposure, and at this time, the image captured at the overexposure level in the obtained multiple frames of original images is selected to perform the noise reduction processing on the face region.
As another possible scenario, when the luminance of the face region in the original image captured according to the high compensation level is too bright due to overexposure but the face region is not yet overexposed, the original image captured at the high compensation level and the original image captured at the transition level may be selected to be subjected to noise reduction processing of the face region respectively.
And 103, fusing the denoised face region in the original image shot at the high compensation level and the denoised face region in the original image shot at the transition level according to the weight corresponding to the high compensation level and the transition level to obtain the face region synthesized by the multi-frame original images and subjected to denoising.
In the embodiment of the application, firstly, multi-frame noise reduction or single-frame airspace noise reduction of a human face region is carried out on at least two frames of original images shot at a high compensation level according to shooting stability when the at least two frames of original images are collected. And performing single-frame spatial noise reduction processing on the face region of the original image shot by adopting the transition level.
The shooting stability can be measured by the shake degree of the electronic device during shooting, and how to determine the shake degree of the electronic device is described in detail in step 102, which is not described herein again.
As a possible scene, if the stability of the electronic device is good when at least two frames of original images are collected, multi-frame noise reduction of a face region is performed on at least two frames of original images shot at a high compensation level, and then the face regions subjected to noise reduction are fused to obtain the face region subjected to multi-frame original image synthesis noise reduction, so that a clear image can be shot in a dark light environment.
The multi-frame noise reduction means that in a night scene or dark light environment, electronic equipment can collect multiple/multi-frame photos or images when imaging is carried out according to a shutter, different pixel points with noise properties are found in different frame numbers, and a clean and pure night scene or dark light photo is obtained through weighting synthesis. It can be understood that when the electronic device shoots a night scene or a dark light environment, the electronic device can calculate and screen the number and the positions of noise points with a plurality of frames, replace the positions of the noise points with the frames without the noise points, and obtain a clear picture through repeated weighting and replacement.
Taking the example in step 101 as an example, the face brightness in the original image captured according to the high compensation level EV +1 is detected, and if it is detected that the face brightness is appropriate and the shake of the electronic device is small, the multi-frame noise reduction of the face region is performed on the multi-frame original image captured at the high compensation level EV + 1.
As another possible scene, if the shooting of the electronic device is unstable when at least two frames of original images are shot and collected, selecting one frame of original image with the highest definition from the at least two frames of original images shot at the high compensation level to perform single-frame spatial noise reduction processing on the face region. The neighborhood size adopted by the single-frame airspace noise reduction is determined according to the sensitivity adopted by shooting the corresponding original image.
The neighborhood is a pixel block which is far smaller than the image size and regular in shape. The shape of the neighborhood may be a square of 2 × 2 or 3 × 3, or may be a polygon having a shape like a circle, an ellipse, or the like. For example, a neighborhood of a pixel point may be a set of the inside or boundaries of a circle centered on the pixel point. The specific shape of the neighborhood is not limited, but the size of all neighborhoods in a frame image should be the same.
Because the image sensor in the electronic device is subjected to different degrees of photo-electromagnetic interference between peripheral circuits and pixels of the image sensor in the electronic device during shooting, noise inevitably exists in the shot original image, and the definition of the shot image is different due to different interference degrees. For example, in a night scene shooting scene, an image is usually shot by using a large aperture and a long exposure time, and if the exposure time is reduced by selecting a high sensitivity, the shot image inevitably generates noise and a mottle. The exposure time refers to the time when light passes through the lens.
It can be understood that the higher the sensitivity of the original image is, the more the noise of the original image is, and the higher the noise reduction strength is, i.e. the larger neighborhood size is adopted in the single-frame spatial domain noise reduction process, for obtaining a clear image. However, the size of the neighborhood is directly related to the image smoothing effect, the larger the size of the neighborhood is, the better the smoothing effect is, but the larger the size of the neighborhood is, the larger the edge information loss is caused by the smoothing, so that the output image becomes blurred, and therefore, the size of the neighborhood needs to be reasonably selected.
The image denoising method of the embodiment of the application obtains a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation level comprises a high compensation level, a low compensation level smaller than the high compensation level, and a transition level between the high compensation level and the low compensation level; determining the weight corresponding to the high compensation level and the transition level according to the brightness of the face area in the original image shot at the high compensation level; and according to the weights corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction by multi-frame original image synthesis. Therefore, the weights corresponding to the high compensation level and the transition level are determined according to the brightness of the face area in the multi-frame original image shot at the high compensation level, and the face area is subjected to noise reduction processing, so that an image with good face definition and low noise is obtained, and the user experience is improved.
For clearly explaining the above embodiment, the present embodiment provides another image denoising method, and fig. 2 is a schematic flow chart of the another image denoising method provided in the embodiment of the present application.
As shown in fig. 2, the image noise reduction method may include the steps of:
In the embodiment of the present application, the method for obtaining multiple frames of original images captured by using corresponding exposure compensation levels is described in step 101 in the foregoing embodiment, and is not described herein again.
The preset brightness interval refers to an interval with proper brightness of a preset human face area.
In the embodiment of the application, whether the brightness of the face area in the original image shot at the high compensation level is in the preset brightness interval or not and whether the brightness of the face area in the original image shot at the high compensation level is overexposed or not are detected, so that multi-frame noise reduction or single-frame noise reduction is dynamically selected for the obtained multi-frame original image, the face area subjected to noise reduction by multi-frame original image synthesis is obtained, and the image with high face definition and less noise is obtained.
And 203, if the brightness of the face area in the original image shot by the high compensation level is not in the preset brightness interval and the brightness of the face area in the original image shot by the high compensation level is not over-exposed, performing multi-frame noise reduction processing on at least two frames of original images shot by the high compensation level, and performing single-frame airspace noise reduction processing on the original image shot by the transition level to obtain the face area synthesized by the multi-frame original images and subjected to noise reduction.
In the embodiment of the application, when the brightness of the face region in the original image shot according to the high compensation level is not in the preset brightness interval and the brightness of the face region in the original image shot according to the high compensation level is not overexposed, multi-frame noise reduction processing is performed on the face region in the original image shot according to the high compensation level, single-frame airspace noise reduction processing of the face region is performed on the original image shot according to the transition level, and then the images subjected to noise reduction processing are fused to obtain the face region subjected to noise reduction by multi-frame original image synthesis.
The fusion proportion is determined according to the brightness and the photosensitive value of the face area in the shot original image after the noise reduction processing is carried out on the face area in the original image shot at the high compensation level and the face area in the original image shot at the transition level.
As an example, if a set of exposure compensation values of a plurality of frames of original images obtained by shooting is [ EV +1, EV0, EV-2, EV-4], if the brightness of a face region in a plurality of frames of original images obtained by shooting with the high compensation level EV +1 at this time is not in a preset brightness interval, and the brightness of a face region in an original image obtained by shooting with the high compensation level is not overexposed. Further, multi-frame noise reduction processing is carried out on the face region in the original image shot according to the high compensation level EV +1, single-frame spatial noise reduction processing of the face region is carried out on the original image shot according to the transition level EV0, and then the images subjected to noise reduction processing are fused to obtain the face region subjected to noise reduction by multi-frame original image synthesis.
And 204, if the brightness of the face region in the at least two frames of original images shot at the high compensation level accords with a preset brightness interval, performing multi-frame noise reduction or single-frame noise reduction on the at least two frames of original images shot at the high compensation level according to the shooting stability when the at least two frames of original images are collected, and obtaining the face region subjected to multi-frame original image synthesis noise reduction.
As a possible scene, if the brightness of the face region in at least two frames of original images shot at a high compensation level meets a preset brightness interval and the shooting is stable when the at least two frames of original images are collected, performing multi-frame noise reduction on the face region in the at least two frames of original images shot at the high compensation level to obtain a face region subjected to multi-frame original image synthesis noise reduction.
As another possible scene, if the brightness of the face region in at least two frames of original images shot at a high compensation level meets a preset brightness interval and shooting is unstable when the at least two frames of original images are collected, performing single-frame spatial noise reduction on the face region in one frame of original image with the highest definition in the at least two frames of original images shot at the high compensation level to obtain a face region synthesized by multiple frames of original images and subjected to noise reduction.
And step 205, if the face area in the original image shot by the high compensation level is overexposed, performing face area noise reduction processing on the original image shot by the transition level to obtain a face area synthesized by multiple frames of original images and subjected to noise reduction.
As a possible scene, if the face area in the original image shot by adopting the high compensation level is overexposed, the face area noise reduction processing is carried out on the original image shot by adopting the transition level, and the face area subjected to the synthesis noise reduction of the multi-frame original image is obtained. The noise reduction degree is adjusted according to the sensitization value when the original image is shot by adopting the transition grade.
As an example, if a group of exposure compensation values of a plurality of frames of original images obtained by shooting are [ EV +1, EV0, EV-2 and EV-4], if a face area in an original image shot by adopting a high compensation level EV +1 is overexposed, then the face area noise reduction processing is carried out on the original image shot by adopting a transition level EV0, and then the face area subjected to noise reduction by multi-frame original image synthesis is obtained.
The image noise reduction method of the embodiment of the application judges whether the brightness of a face area in an original image shot by a high compensation level is in a preset brightness interval or not and whether the brightness of the face area in the original image shot by the high compensation level is overexposed or not by obtaining a plurality of frames of original images shot by a corresponding exposure compensation level, if the brightness of the face area in the original image shot by the high compensation level is not in the preset brightness interval and is not overexposed, performs multi-frame noise reduction processing on at least two frames of original images shot by the high compensation level, performs single-frame spatial noise reduction processing on the face area of the original image shot by a transition level, further fuses to obtain the face area synthesized by a plurality of frames of original images and subjected to noise reduction, if the brightness of the face area accords with the preset brightness interval, performs multi-frame noise reduction or single-frame noise reduction on at least two frames of original images shot by the high compensation level according to the shooting stability when the at least two frames of original images are, and if the face area in the original image shot by adopting the high compensation level is overexposed, carrying out face area noise reduction processing on the original image shot by adopting the transition level to obtain the face area subjected to multi-frame original image synthesis noise reduction. Therefore, according to the brightness of a face area in a multi-frame original image shot at a high compensation level, multi-frame noise reduction or single-frame noise reduction is dynamically selected, and noise reduction processing is performed on the face area, so that an image with good face definition and less noise is obtained, and user experience is improved.
In order to implement the above embodiments, the present application further provides an image noise reduction apparatus.
Fig. 3 is a schematic structural diagram of an image noise reduction apparatus according to an embodiment of the present disclosure.
As shown in fig. 3, the image noise reduction apparatus 100 includes: an acquisition module 110, a determination module 120, and a fusion module 130.
An obtaining module 110, configured to obtain multiple frames of original images captured with corresponding exposure compensation levels; wherein the exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level.
And the determining module 120 is configured to determine weights corresponding to the high compensation level and the transition level according to the brightness of the face region in the original image captured at the high compensation level.
And the fusion module 130 is configured to fuse the noise-reduced face region in the original image shot at the high compensation level and the noise-reduced face region in the original image shot at the transition level according to the weight corresponding to the high compensation level and the transition level, so as to obtain a noise-reduced face region synthesized by multiple frames of original images.
As another possible implementation manner, the image noise reduction apparatus may further include:
and the first noise reduction module is used for carrying out multi-frame noise reduction or single-frame airspace noise reduction on the human face region of the at least two frames of original images shot at the high compensation level according to the shooting stability when the at least two frames of original images are collected.
And the second noise reduction module is used for carrying out single-frame spatial domain noise reduction processing on the original image shot by adopting the transition level.
The neighborhood size adopted by the single-frame airspace noise reduction is determined according to the sensitivity adopted by shooting the corresponding original image.
As another possible implementation manner, the first noise reduction module may further include:
and the multi-frame noise reduction unit is used for performing multi-frame noise reduction on the human face area of the at least two original images shot at the high compensation level if the shooting is stable when the at least two original images are collected.
And the single-frame airspace noise reduction unit is used for performing single-frame airspace noise reduction on a frame of original image with highest definition in the at least two frames of original images shot at a high compensation level if shooting is unstable when the at least two frames of original images are collected.
As another possible implementation manner, the image noise reduction apparatus may further include:
the first determining module is used for determining that the brightness of the face area in the original image shot at the high compensation level is not in a preset brightness interval, and the brightness of the face area in the original image shot at the high compensation level is not over exposed.
As another possible implementation manner, the image noise reduction apparatus 100 may further include:
and the third noise reduction module is used for performing multi-frame noise reduction or single-frame noise reduction on the at least two frames of original images shot at the high compensation level according to the shooting stability when the at least two frames of original images are collected, so as to obtain the face area subjected to multi-frame original image synthesis noise reduction.
As another possible implementation manner, the image noise reduction apparatus may further include:
and the fourth noise reduction module is used for carrying out face region noise reduction processing on the original image shot by adopting the transition level to obtain a face region subjected to noise reduction by multi-frame original image synthesis in the original image shot by adopting the high compensation level.
The image noise reduction device of the embodiment of the application obtains the multi-frame original image shot by adopting the corresponding exposure compensation grade; wherein the exposure compensation level comprises a high compensation level, a low compensation level smaller than the high compensation level, and a transition level between the high compensation level and the low compensation level; determining the weight corresponding to the high compensation level and the transition level according to the brightness of the face area in the original image shot at the high compensation level; and according to the weights corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction by multi-frame original image synthesis. Therefore, the weights corresponding to the high compensation level and the transition level are determined according to the brightness of the face area in the multi-frame original image shot at the high compensation level, and the face area is subjected to noise reduction processing, so that an image with good face definition and low noise is obtained, and the user experience is improved.
It should be noted that the foregoing explanation of the embodiment of the image noise reduction method is also applicable to the image noise reduction apparatus of this embodiment, and is not repeated here.
In order to implement the above embodiments, the present application also proposes an electronic device, a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the image denoising method as described in the above embodiments.
Referring to fig. 4, another electronic device 200 is also provided. The electronic device 200 comprises a memory 50 and a processor 60. The memory 50 has stored therein computer readable instructions. The computer readable instructions, when executed by the memory 50, cause the processor 60 to perform the image denoising method of any of the above embodiments.
Fig. 4 is a schematic diagram of an internal structure of the electronic device 200 according to an embodiment. The electronic device 200 includes a processor 60, a memory 50 (e.g., a non-volatile storage medium), an internal memory 82, a display screen 83, and an input device 84 connected by a system bus 81. The memory 50 of the electronic device 200 stores, among other things, an operating system and computer-readable instructions. The computer readable instructions can be executed by the processor 60 to implement the image denoising method according to the embodiment of the present application. The processor 60 is used to provide computing and control capabilities that support the operation of the overall electronic device 200. The internal memory 50 of the electronic device 200 provides an environment for the execution of computer readable instructions in the memory 52. The display 83 of the electronic device 200 may be a liquid crystal display or an electronic ink display, and the input device 84 may be a touch layer covered on the display 83, a button, a trackball or a touch pad arranged on a housing of the electronic device 200, or an external keyboard, a touch pad or a mouse. The electronic device 200 may be a mobile phone, a tablet computer, a notebook computer, a personal digital assistant, or a wearable device (e.g., a smart bracelet, a smart watch, a smart helmet, smart glasses), etc. Those skilled in the art will appreciate that the configuration shown in fig. 4 is merely a schematic diagram of a portion of the configuration associated with the present application, and does not constitute a limitation on the electronic device 200 to which the present application is applied, and that a particular electronic device 200 may include more or less components than those shown in the drawings, or combine certain components, or have a different arrangement of components.
Referring to fig. 5, the electronic device 200 according to the embodiment of the present disclosure includes an Image Processing circuit 90, and the Image Processing circuit 90 may be implemented by hardware and/or software components, including various Processing units defining an ISP (Image Signal Processing) pipeline. FIG. 5 is a schematic diagram of image processing circuitry 90 in one embodiment. As shown in fig. 5, for convenience of explanation, only aspects of the image processing technology related to the embodiments of the present application are shown.
As shown in fig. 5, the image processing circuit 90 includes an ISP processor 91 (the ISP processor 91 may be the processor 60) and control logic 92. The image data captured by the camera 93 is first processed by the ISP processor 91, and the ISP processor 91 analyzes the image data to capture image statistics that may be used to determine one or more control parameters of the camera 93. The camera 93 may include one or more lenses 932 and an image sensor 934. Image sensor 934 may include an array of color filters (e.g., Bayer filters), and image sensor 934 may acquire light intensity and wavelength information captured by each imaging pixel and provide a set of raw image data that may be processed by ISP processor 91. The sensor 94 (e.g., a gyroscope) may provide parameters of the acquired image processing (e.g., anti-shake parameters) to the ISP processor 91 based on the type of interface of the sensor 94. The sensor 94 interface may be a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interface, or a combination thereof.
In addition, the image sensor 934 may also send raw image data to the sensor 94, the sensor 94 may provide the raw image data to the ISP processor 91 based on the type of interface of the sensor 94, or the sensor 94 may store the raw image data in the image memory 95.
The ISP processor 91 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 91 may perform one or more image processing operations on the raw image data, gathering statistical information about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.
The ISP processor 91 may also receive image data from the image memory 95. For example, the sensor 94 interface sends raw image data to the image memory 95, and the raw image data in the image memory 95 is then provided to the ISP processor 91 for processing. The image Memory 95 may be the Memory 50, a portion of the Memory 50, a storage device, or a separate dedicated Memory within the electronic device, and may include a DMA (Direct Memory Access) feature.
Upon receiving raw image data from the image sensor 934 interface or from the sensor 94 interface or from the image memory 95, the ISP processor 91 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 95 for additional processing before being displayed. The ISP processor 91 receives the processed data from the image memory 95 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The image data processed by ISP processor 91 may be output to display 97 (display 97 may include display screen 83) for viewing by a user and/or further processed by a Graphics Processing Unit (GPU). Further, the output of the ISP processor 91 may also be sent to an image memory 95, and the display 97 may read image data from the image memory 95. In one embodiment, image memory 95 may be configured to implement one or more frame buffers. Further, the output of the ISP processor 91 may be transmitted to an encoder/decoder 96 for encoding/decoding the image data. The encoded image data may be saved and decompressed before being displayed on the display 97 device. The encoder/decoder 96 may be implemented by a CPU or GPU or coprocessor.
The statistical data determined by the ISP processor 91 may be sent to the control logic 92 unit. For example, the statistical data may include image sensor 934 statistics such as auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 932 shading correction, and the like. The control logic 92 may include a processing element and/or microcontroller that executes one or more routines (e.g., firmware) that determine control parameters of the camera 93 and control parameters of the ISP processor 91 based on the received statistical data. For example, the control parameters of camera 93 may include sensor 94 control parameters (e.g., gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, lens 932 control parameters (e.g., focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), and lens 932 shading correction parameters.
The following steps are implemented for implementing the image denoising method by using the image processing technology in fig. 5:
acquiring a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level;
determining the weight corresponding to the high compensation level and the transition level according to the brightness of a face area in the original image shot at the high compensation level;
and according to the weight corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction synthesized by multiple frames of original images.
To implement the embodiments described above, the embodiments of the present application also provide one or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of:
acquiring a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level;
determining the weight corresponding to the high compensation level and the transition level according to the brightness of a face area in the original image shot at the high compensation level;
and according to the weight corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction synthesized by multiple frames of original images.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. An image noise reduction method, characterized in that it comprises the steps of:
acquiring a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level;
determining weights corresponding to the high compensation level and the transition level according to the brightness of a face area in an original image shot at the high compensation level and the shaking degree of equipment in the shooting process, wherein the original image shot at the high compensation level is at least two frames;
and according to the weight corresponding to the high compensation level and the transition level, fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level to obtain the face region subjected to noise reduction synthesized by multiple frames of original images.
2. The image denoising method according to claim 1, wherein before the fusing the denoising face region in the original image captured at the high compensation level and the denoising face region in the original image captured at the transition level, the method further comprises:
according to the shooting stability when the at least two frames of original images are collected, carrying out multi-frame noise reduction or single-frame airspace noise reduction on the at least two frames of original images shot at the high compensation level in a human face region;
and carrying out single-frame spatial noise reduction processing on the face region of the original image shot by adopting the transition grade.
3. The image noise reduction method according to claim 2, wherein a neighborhood size used for the single-frame spatial noise reduction is determined according to a sensitivity used for capturing a corresponding original image.
4. The image denoising method according to claim 2, wherein the performing multi-frame denoising or single-frame spatial denoising of the face region on the at least two frames of original images shot at the high compensation level according to the shooting stability when the at least two frames of original images are collected comprises:
if the shooting is stable when the at least two frames of original images are collected, carrying out multi-frame noise reduction on the at least two frames of original images shot at the high compensation level in a human face area;
and if the shooting is unstable when the at least two frames of original images are collected, performing single-frame spatial domain noise reduction on a frame of original image with the highest definition in the at least two frames of original images shot at the high compensation level.
5. The image noise reduction method according to claim 1, wherein the determining weights corresponding to the high compensation level and the transition level according to the brightness of the face region in the original image captured at the high compensation level and the shaking degree of the device during the capturing process, wherein the original image captured at the high compensation level is at least two frames before, further comprises:
and determining that the brightness of the face area in the original image shot at the high compensation level is not in a preset brightness interval, and that the brightness of the face area in the original image shot at the high compensation level is not over-exposed.
6. The image noise reduction method according to claim 5, wherein the original image photographed at the high compensation level is at least two frames, the method further comprising:
if the brightness of the face region in the at least two frames of original images shot at the high compensation level accords with a preset brightness interval, carrying out multi-frame noise reduction or single-frame noise reduction on the at least two frames of original images shot at the high compensation level according to the shooting stability when the at least two frames of original images are collected so as to obtain the face region subjected to multi-frame original image synthesis noise reduction.
7. The image noise reduction method according to claim 5, further comprising:
and if the face area in the original image shot by the high compensation level is overexposed, carrying out face area noise reduction processing on the original image shot by the transition level to obtain a face area synthesized by multiple frames of original images and subjected to noise reduction.
8. An image noise reduction apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring a plurality of frames of original images shot by adopting corresponding exposure compensation levels; wherein the exposure compensation levels include a high compensation level, a low compensation level less than the high compensation level, and a transition level between the high compensation level and the low compensation level;
the determining module is used for determining the weight corresponding to the high compensation level and the transition level according to the brightness of a face area in an original image shot at the high compensation level and the shaking degree of equipment in the shooting process, wherein the original image shot at the high compensation level is at least two frames;
and the fusion module is used for fusing the face region subjected to noise reduction in the original image shot at the high compensation level and the face region subjected to noise reduction in the original image shot at the transition level according to the weight corresponding to the high compensation level and the transition level to obtain the face region subjected to noise reduction synthesized by the multi-frame original images.
9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which when executed by the processor implements the image noise reduction method according to any of claims 1 to 7.
10. One or more non-transitory computer-readable storage media containing computer-executable instructions, having stored thereon a computer program, wherein the program, when executed by a processor, implements the image denoising method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811399524.7A CN109348088B (en) | 2018-11-22 | 2018-11-22 | Image noise reduction method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811399524.7A CN109348088B (en) | 2018-11-22 | 2018-11-22 | Image noise reduction method and device, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109348088A CN109348088A (en) | 2019-02-15 |
| CN109348088B true CN109348088B (en) | 2021-05-11 |
Family
ID=65317443
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811399524.7A Active CN109348088B (en) | 2018-11-22 | 2018-11-22 | Image noise reduction method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109348088B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110062159A (en) * | 2019-04-09 | 2019-07-26 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment based on multiple image |
| CN110084016B (en) * | 2019-04-23 | 2024-07-26 | 努比亚技术有限公司 | Face recognition unlocking method and device, mobile terminal and storage medium |
| CN110191291B (en) * | 2019-06-13 | 2021-06-25 | Oppo广东移动通信有限公司 | Image processing method and device based on multi-frame images |
| CN110248107A (en) * | 2019-06-13 | 2019-09-17 | Oppo广东移动通信有限公司 | Image processing method and device |
| CN111091513B (en) * | 2019-12-18 | 2023-07-25 | Oppo广东移动通信有限公司 | Image processing method, device, computer readable storage medium and electronic equipment |
| CN111191606A (en) * | 2019-12-31 | 2020-05-22 | Oppo广东移动通信有限公司 | Image processing method and related product |
| CN111507358B (en) * | 2020-04-01 | 2023-05-16 | 浙江大华技术股份有限公司 | Face image processing method, device, equipment and medium |
| CN113870300A (en) * | 2020-06-29 | 2021-12-31 | 北京迈格威科技有限公司 | Image processing method and device, electronic equipment and readable storage medium |
| CN112866773B (en) * | 2020-08-21 | 2023-09-26 | 海信视像科技股份有限公司 | Display equipment and camera tracking method in multi-person scene |
| WO2022141445A1 (en) * | 2020-12-31 | 2022-07-07 | 华为技术有限公司 | Image processing method and device |
| CN113452924A (en) * | 2021-06-30 | 2021-09-28 | 广州极飞科技股份有限公司 | Camera diaphragm control method, device, equipment and storage medium |
| CN114302026B (en) * | 2021-12-28 | 2024-06-21 | 维沃移动通信有限公司 | Noise reduction method, device, electronic equipment and readable storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9568713B2 (en) * | 2013-01-05 | 2017-02-14 | Light Labs Inc. | Methods and apparatus for using multiple optical chains in parallel to support separate color-capture |
| CN103973958B (en) * | 2013-01-30 | 2018-04-03 | 阿里巴巴集团控股有限公司 | Image processing method and equipment |
| CN104881644A (en) * | 2015-05-25 | 2015-09-02 | 华南理工大学 | Face image acquisition method under uneven lighting condition |
| CN108205796B (en) * | 2016-12-16 | 2021-08-10 | 大唐电信科技股份有限公司 | Multi-exposure image fusion method and device |
| CN107635098B (en) * | 2017-10-30 | 2019-09-10 | Oppo广东移动通信有限公司 | High dynamic range images noise remove method, device and equipment |
| CN107707827B (en) * | 2017-11-14 | 2020-05-01 | 维沃移动通信有限公司 | High-dynamic image shooting method and mobile terminal |
| CN108337449A (en) * | 2018-04-12 | 2018-07-27 | Oppo广东移动通信有限公司 | High-dynamic-range image acquisition method, device based on dual camera and equipment |
| CN108683862B (en) * | 2018-08-13 | 2020-01-10 | Oppo广东移动通信有限公司 | Imaging control method, imaging control device, electronic equipment and computer-readable storage medium |
-
2018
- 2018-11-22 CN CN201811399524.7A patent/CN109348088B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109348088A (en) | 2019-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109348088B (en) | Image noise reduction method and device, electronic equipment and computer readable storage medium | |
| CN108900782B (en) | Exposure control method, exposure control device and electronic equipment | |
| CN110072051B (en) | Image processing method and device based on multi-frame images | |
| AU2019326496B2 (en) | Method for capturing images at night, apparatus, electronic device, and storage medium | |
| CN109068067B (en) | Exposure control method and device and electronic equipment | |
| CN109788207B (en) | Image synthesis method and device, electronic equipment and readable storage medium | |
| CN108322669B (en) | Image acquisition method and apparatus, imaging apparatus, and readable storage medium | |
| CN110072052B (en) | Image processing method and device based on multi-frame image and electronic equipment | |
| CN109068058B (en) | Shooting control method and device in super night scene mode and electronic equipment | |
| CN108683862B (en) | Imaging control method, imaging control device, electronic equipment and computer-readable storage medium | |
| CN110166707B (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| CN110191291B (en) | Image processing method and device based on multi-frame images | |
| CN110166708B (en) | Night scene image processing method and device, electronic equipment and storage medium | |
| CN109089046B (en) | Image noise reduction method and device, computer readable storage medium and electronic equipment | |
| CN110290289B (en) | Image noise reduction method and device, electronic equipment and storage medium | |
| CN110248106B (en) | Image noise reduction method and device, electronic equipment and storage medium | |
| CN109672819B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| WO2020207261A1 (en) | Image processing method and apparatus based on multiple frames of images, and electronic device | |
| CN109005369B (en) | Exposure control method, exposure control device, electronic apparatus, and computer-readable storage medium | |
| CN110166706B (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| CN109151333B (en) | Exposure control method, exposure control device and electronic equipment | |
| CN110264420B (en) | Image processing method and device based on multi-frame images | |
| CN107509044B (en) | Image synthesis method, image synthesis device, computer-readable storage medium and computer equipment | |
| CN110166709B (en) | Night scene image processing method and device, electronic equipment and storage medium | |
| CN107704798B (en) | Image blurring method and device, computer readable storage medium and computer device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |