CN107370958B - Image blurring processing method and device and shooting terminal - Google Patents

Abstract

The invention provides an image blurring processing method, an image blurring processing device and a shooting terminal, wherein the method comprises the following steps: acquiring a depth-of-field image to be processed; determining a foreground region and a background region in the depth image; when blurring processing is carried out on a background region, if it is determined that a target filtering region corresponding to a pixel point to be blurred contains a pixel point located in the foreground region, the weighted value of each pixel point in the target filtering region is adjusted. According to the method, the weighted value of each pixel point in the target filtering area containing the pixel points in the foreground area is adjusted, so that after blurring operation, the boundary between the foreground area and the background area of the image is clear, the blurring edge of the background area is effectively protected, the overall image quality of the image is improved, and the user experience degree is improved.

Description

Image blur processing method, device and shooting terminal

technical field

The present invention relates to the technical field of image processing, and in particular, to an image blurring processing method, device and shooting terminal.

Background technique

At present, when processing a captured image, in order to highlight the main part in the image, the background part of the image can usually be blurred. For example, Gaussian blurring can be used to blur the background part of the image to preserve the main part.

Specifically, Gaussian blur refers to a blur algorithm that uses distance as a weight. Usually, the pixel values of a point to be blurred and the surrounding adjacent points are counted, and the calculation method of weighted average is used to obtain the pixels to be blurred. corresponding pixel value. However, the inventor found that after using the Gaussian blur technology to process the image, the edge of the connection between the main part and the background part in the whole image is not clear, there is ghosting, and the overall image quality of the captured image after processing is not good, as shown in Figure 1. shown.

SUMMARY OF THE INVENTION

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

Therefore, the first object of the present invention is to propose an image blurring processing method, which adjusts the weight value of each pixel point in the target filtering area including the foreground area pixel point, so that the blurring operation , the boundary between the foreground area and the background area of the image is clearer, which effectively protects the blurred edge of the background area, improves the overall image quality of the image, and improves the user experience.

The second object of the present invention is to provide an image blur processing device.

The third object of the present invention is to provide a shooting terminal.

A fourth object of the present invention is to provide a computer-readable storage medium.

In order to achieve the above purpose, an embodiment of the first aspect of the present invention proposes an image blurring processing method, including:

Get the depth of field image to be processed;

determining a foreground area and a background area in the depth-of-field image;

When performing the blurring process on the background area, if it is determined that the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, then the weight of each pixel in the target filtering area is determined. value to adjust.

In the image blurring processing method provided by the embodiment of the present invention, the foreground area and the background area of the obtained depth image are determined by acquiring the depth image to be processed, and then when performing the blurring processing on the determined background area, first determine the foreground area and the background area of the obtained depth image Whether the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, and if so, adjust the weight value of each pixel in the target filtering area. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels in the foreground area, when the background area is blurred, the boundary between the foreground area and the background area of the image is clearer , which effectively protects the blurred edge of the background area, improves the overall quality of the image, and enhances the user experience.

In addition, the image blurring processing method proposed by the above embodiments of the present invention may also have the following additional technical features:

In an embodiment of the present invention, the determining of the foreground area and the background area in the depth-of-field image includes:

According to a preset range, the foreground area and the background area in the depth-of-field image are determined.

In another embodiment of the present invention, the depth-of-field image to be processed includes a portrait;

The determining of the foreground area and the background area in the depth-of-field image includes:

performing face recognition on the depth-of-field image to determine the face area;

The foreground area and the background area in the depth image are determined according to the depth of field value corresponding to the face area.

In another embodiment of the present invention, the target filtering area corresponding to the pixels to be blurred includes pixels located in the foreground area, including:

If the depth of field value corresponding to any pixel included in the target filtering area is within the preset range, then determine that the any pixel is a pixel of the foreground area;

or,

If the difference between the depth values corresponding to two adjacent pixels in the target filtering area is greater than a preset value, it is determined that the target filtering area includes pixels located in the foreground area.

In another embodiment of the present invention, the adjusting the weight value of each pixel in the target filtering area includes:

set the weight value of the pixel in the foreground area to zero;

The weight values of the remaining pixels in the target filtering area are adjusted so that the sum of the weight values of the remaining pixels is equal to one.

In another embodiment of the present invention, the adjusting the weight value of the remaining pixels in the target filtering area includes:

expanding the weight values of the remaining pixels in equal proportions;

or,

The weight value of the remaining pixels is adjusted according to the depth of field values corresponding to the remaining pixels respectively.

In order to achieve the above purpose, the embodiment of the second aspect of the present invention provides an image blurring processing device, including:

an acquisition module, used to acquire the depth-of-field image to be processed;

a determination module for determining the foreground area and the background area in the depth of field image;

The adjustment module is used to perform the blurring process on the background area, if it is determined that the target filtering area corresponding to the pixels to be blurred includes the pixels located in the foreground area, then the target filtering area The weight value of each pixel is adjusted.

The image blurring processing device provided by the embodiment of the present invention determines the foreground area and the background area of the obtained depth-of-field image by acquiring the depth-of-field image to be processed. Whether the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, and if so, adjust the weight value of each pixel in the target filtering area. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels in the foreground area, when the background area is blurred, the boundary between the foreground area and the background area of the image is clearer , which effectively protects the blurred edge of the background area, improves the overall quality of the image, and enhances the user experience.

In addition, the image blur processing apparatus proposed by the above embodiments of the present invention may also have the following additional technical features:

In an embodiment of the present invention, the determining module is specifically used for:

According to a preset range, the foreground area and the background area in the depth-of-field image are determined.

In another embodiment of the present invention, the depth-of-field image to be processed includes a portrait;

The determining module includes:

a first determination subunit, used for performing face recognition on the depth-of-field image to determine a face area;

The second determination subunit is configured to determine the foreground area and the background area in the depth-of-field image according to the depth-of-field value corresponding to the face area.

In another embodiment of the present invention, the adjustment module is specifically used for:

If the depth of field value corresponding to any pixel included in the target filtering area is within the preset range, then determine that the any pixel is a pixel of the foreground area;

or,

If the difference between the depth values corresponding to two adjacent pixels in the target filtering area is greater than a preset value, it is determined that the target filtering area includes pixels located in the foreground area.

In another embodiment of the present invention, the adjustment module further includes:

a first adjustment subunit, used for zeroing the weight value of the pixels located in the foreground area;

The second adjustment subunit is configured to adjust the weight values of the remaining pixels in the target filtering area, so that the sum of the weight values of the remaining pixels is equal to one.

In another embodiment of the present invention, the second adjustment subunit is specifically used for:

expanding the weight values of the remaining pixels in equal proportions;

or,

The weight value of the remaining pixels is adjusted according to the depth of field values corresponding to the remaining pixels respectively.

In order to achieve the above purpose, an embodiment of the third aspect of the present invention provides a shooting terminal, including: a memory, a processor, and an image processing circuit;

the memory for storing executable instructions of the processor;

The image processing circuit is used to process the depth of field image;

The processor is configured to call the program code in the memory, and implement the image blurring processing method according to the embodiment of the first aspect according to the depth image output by the image processing circuit.

The photographing terminal provided by the embodiment of the present invention determines the foreground area and the background area of the acquired depth image by acquiring the depth image to be processed, and then, when performing the blurring process on the determined background area, firstly determines the background area to be blurred Whether the target filtering area corresponding to the pixel points of the target filter area includes pixels located in the foreground area, and if so, adjust the weight value of each pixel point in the target filtering area. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels in the foreground area, when the background area is blurred, the boundary between the foreground area and the background area of the image is clearer , which effectively protects the blurred edge of the background area, improves the overall quality of the image, and enhances the user experience.

In order to achieve the above purpose, the embodiment of the fourth aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the image blurring process described in the embodiment of the first aspect is implemented method.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

Fig. 1 is the schematic diagram after the image is blurred by using Gaussian blur technology at present;

2 is a flowchart of an image blurring processing method according to an embodiment of the present invention;

3 is a schematic diagram of determining a target filtering area of a pixel to be blurred in an embodiment of the present invention;

4 is a flowchart of an image blurring processing method according to another embodiment of the present invention;

5 is a schematic diagram of processing a foreground area pixel that includes a foreground area pixel in a target filtering area according to an embodiment of the present invention;

6 is an effect diagram of the present invention after the image is blurred;

7 is a schematic structural diagram of an image blurring processing apparatus according to an embodiment of the present invention;

8 is a schematic structural diagram of an image blurring processing apparatus according to another embodiment of the present invention;

9 is a schematic structural diagram of an image blurring processing apparatus according to still another embodiment of the present invention;

10 is a schematic structural diagram of a shooting terminal according to an embodiment of the present invention;

FIG. 11 is a schematic structural diagram of an image processing circuit according to an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

The present invention mainly aims at the problem that in the prior art, when the image is blurred by the Gaussian blurring technology, the connection boundary between the foreground area and the background area in the processed image is not clear, and ghosting occurs, and a method is proposed. Image blur processing method.

The image processing method proposed by the present invention first acquires the depth-of-field image to be processed, and then determines the foreground area and the background area in the acquired depth-of-field image. Whether the target filtering area corresponding to the converted pixels includes pixels in the foreground area, if it includes pixels located in the foreground area, adjust the weight value of each pixel in the target filtering area. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels in the foreground area, after the blurring operation, the boundary between the foreground area and the background area of the image is clear, and the background area is effectively protected. The blurred edge of the area improves the overall image quality of the image and improves the user experience.

The image blurring processing method proposed by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 2 is a flowchart of an image blurring processing method according to an embodiment of the present invention.

As shown in Figure 2, the image blurring processing method may include the following steps:

Step 201, acquiring a depth-of-field image to be processed.

The image blurring processing method provided by the embodiment of the present invention may be executed by an image blurring processing apparatus provided by the present invention, and the apparatus may be configured in a shooting terminal to implement image blurring processing on a captured image.

Wherein, the shooting terminal in this application may be any hardware device with a shooting function, such as a camera, a smart phone, a personal digital assistant (Personal Digital Assistant, PDA for short), and so on.

During specific implementation, the depth-of-field image to be processed may be acquired by means of the prior art, which will not be described in detail in this embodiment.

It should be noted that the depth-of-field image obtained in this embodiment may be a landscape image, a person image, or an architectural image, etc., which is not specifically limited in this application.

Step 202: Determine the foreground area and the background area in the depth-of-field image.

During specific implementation, the present application may determine the foreground area and the background area according to the type of the depth image, or may also determine the foreground area and the background area according to the brightness of the object in the depth image.

Specifically, if the foreground area and the background area are determined according to the type of the depth-of-field image in this embodiment, it is first determined whether the depth-of-field image belongs to a person image or a landscape image, and after the specific type of the depth-of-field image to be processed is determined, the following methods can be used. To determine the foreground area and background area of the acquired depth-of-field image:

Manner 1, if the depth-of-field image to be processed is a landscape image or an architectural image, the foreground area and the background area in the depth-of-field image to be processed may be determined according to a preset range.

Specifically, the preset range in this embodiment may be set by the user, or may also be automatically configured by the photographing device according to the depth-of-field image, which is not specifically limited herein.

For example, if the predetermined range is 20 centimeters (cm) to 50 cm. The depth-of-field image currently acquired by the terminal includes a flower, and the depth-of-field range corresponding to the flower is 55cm-60cm. Therefore, it can be determined that the flower is located in the background area in the depth image.

Method 2: If the depth-of-field image to be processed is a human image, face recognition can be performed on the depth-of-field image to be processed to determine the face area in the depth-of-field image, and then the foreground in the depth-of-field image is determined according to the depth-of-field value corresponding to the face area. area and background area.

For example, if the detected face area has a depth of field value of 15, it can be determined that an area with a depth of field value within the range of 12-17 is a foreground area, and other areas are background areas.

It should be noted that the depth of field value of the above-mentioned face area may refer to the mean value of the depth of field values corresponding to all pixels in the face area, or may also be the depth of field value of any pixel in the face area. Not limited.

If in this embodiment, the foreground area and the background area are determined according to the brightness of the object in the depth-of-field image, the brightness of the depth-of-field image is acquired, and the foreground area and the background area are determined according to the brightness.

Specifically, in the actual shooting process, the subject may need to be supplemented with light due to environmental factors. In the process of supplementary light, objects at different distances receive different light intensities. Therefore, the light displayed on the display screen of the shooting device is displayed on the display screen of the shooting device. Brightness is not the same. For example, the brightness of the area close to the photographing device is relatively high, and the brightness of the area far away from the photographing device is relatively low.

Based on the above analysis, the present application can determine the foreground area and the background area of the depth-of-field image according to the brightness of different positions of the object in the image. That is, an area with high brightness in the captured image may be determined as a foreground area, and an area with low brightness may be determined as a background area.

Step 203, when performing the blurring process on the background area, if it is determined that the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, then the weight value of each pixel in the target filtering area is adjusted. .

Specifically, after the foreground area and the background area of the depth-of-field image are determined, blurring processing may be performed on the determined background area. Wherein, when performing the blurring process on the determined background area, it is necessary to determine the target filtering area of the pixels to be blurred in the background area.

In this embodiment, the target filtering area of the pixels to be blurred can be determined according to the determination method in the existing Gaussian blurring technology.

For example, as shown in Figure 3, a 3*3 matrix containing the pixels to be processed can be taken in the image. If the position coordinates of the pixels to be processed are (0, 0), the coordinates of the remaining 8 pixels are (- 1,1),(0,1),(1,1),(-1,0),(1,0),(-1,-1),(0,-1),(1,-1 ).

Further, after the target filtering area of the pixels to be blurred is determined, it can be determined whether the target filtering area includes pixels in the foreground area.

Specifically, whether the target filtering region contains pixels located in the foreground region can be determined in the following manner.

In an implementation situation, if the depth of field value corresponding to any pixel included in the target filtering area is within a preset range, then determine that any pixel is a pixel in the foreground area;

In specific implementation, the depth of field values of all pixels in the target filtering area can be obtained, and then the depth of field values corresponding to all pixels are matched with the preset range one by one. If the depth of field value of the pixel points is within the preset range, It means that the pixel belongs to the foreground area, otherwise, it means that the pixel belongs to the background area.

For example, assuming that the depth of field value of a certain pixel in the target filtering area is 12 and the preset range is 10-20, after matching, it can be concluded that the pixel is within the preset range, that is, the pixel belongs to the foreground area. .

For another example, if the depth of field value of a certain pixel in the target filtering area is 30, after matching, it can be concluded that the pixel belongs to the background area.

Or, in another implementation situation, if the difference between the depth of field values corresponding to two adjacent pixels in the target filtering area is greater than a preset value, it is determined that the target filtering area includes pixels located in the foreground area.

Specifically, because the difference between the depth of field of the foreground area and the depth of field of the background area is generally large, in this embodiment, the area to which each pixel belongs may also be determined according to the change value of the depth of field between adjacent pixels.

First obtain the depth of field values corresponding to all the pixels in the target filtering interval, then make a difference between the depth of field values of adjacent pixels, and compare the difference with the preset value. If the difference is greater than the preset value, It means that the target filtering area includes the pixels of the foreground area.

For example, if the depth of field values of two adjacent pixels in the target filtering area are 24 and 25 respectively, then the difference value is 1 by taking the difference of the depth of field values of these two pixels. If the preset value is 10, it means that the difference between the two pixel points is smaller than the preset value, and the target filtering area does not include pixels in the foreground area.

In this embodiment, if it is determined that the target filtering area corresponding to the pixel to be blurred includes pixels of the foreground area, the weight value of each pixel in the target filtering area is adjusted so that the pixel to be blurred When blurring is performed, the pixels in the foreground area will not affect the blurring effect of the background area, so that the boundaries between the front and background areas after blurring are clear, which not only protects the edges of the blur, but also makes the subject more prominent. .

In the image blurring processing method provided by the embodiment of the present invention, the foreground area and the background area of the obtained depth image are determined by acquiring the depth image to be processed, and then when performing the blurring processing on the determined background area, first determine the foreground area and the background area of the obtained depth image Whether the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, and if so, adjust the weight value of each pixel in the target filtering area. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels of the foreground area, the boundary between the foreground area and the background area in the obtained image after the blurring of the background area is obtained. Clean and clear, it effectively protects the blurred edge of the background area, and makes the subject more prominent, improving the overall image quality of the image and enhancing the user experience.

It can be seen from the above analysis that when blurring the background area, it is necessary to first determine whether the target filtering area corresponding to the pixels to be blurred includes pixels in the foreground area, and if it is determined that the target filtering area contains the foreground When the pixel points of the target filter area are selected, the weight value of each pixel point in the target filter area is adjusted. In a possible implementation scenario of the present invention, the adjustment of the weight value of each pixel in the target filtering area including the pixels of the foreground area can be implemented in various ways, so that the obtained background area is blurred In the processed image, the boundaries between foreground and background regions are clean and clear. The image blurring processing method in the above case will be further described below with reference to FIG. 4 .

FIG. 4 is a flowchart of an image blurring processing method according to another embodiment of the present invention.

As shown in Figure 4, the image blurring processing method may include the following steps:

Step 401, acquiring a depth-of-field image to be processed.

Step 402: Determine the foreground area and the background area in the depth-of-field image.

Step 403: When performing the blurring process on the background area, if it is determined that the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, the weight value of the pixels located in the foreground area is set to zero.

Step 404: Adjust the weight values of the remaining pixels in the target filtering area so that the sum of the weight values of the remaining pixels is equal to one.

Specifically, when it is determined that the target filtering area includes pixels located in the foreground area, the weight value of the pixels in the foreground area can be set to zero, so that when the background area is blurred, The target filtering area of the blurred pixels contains the information of the pixels of the foreground area, so that the boundary between the background area and the foreground area after the blurring process is unclear, as shown in FIG. 5 for details.

Among them, it can be seen from FIG. 5 that the pixel point to be blurred is A, and there is a pixel point B in the foreground area near the pixel point A, then before the blurring process is performed on the pixel point A, the pixel point B will be included first. The weight value of the left area of is set to 0.

Further, after the weight value of the pixels in the foreground area is set to zero, in order to ensure that the standard of the pixels in the blurring process always meets the standard of the preset blurring processing, the present application may The weights of the remaining pixels are adjusted.

During specific implementation, the weight values of the remaining pixels in the target filtering area can be adjusted in the following ways.

In an implementation manner, the weight values of the remaining pixels are proportionally enlarged.

Specifically, assuming that the uniform standard for blurring the pixels in the background area of the image is 1, then after the weight value of the pixels in the foreground area is set to zero, the sum of the weight values of the pixels in the target filtering area is not equal to 1, so after the weight values of the remaining pixels are proportionally enlarged, the sum of the weight values of the remaining pixels is always equal to 1. This ensures that when the pixels to be blurred are blurred, there will be no blurring and distortion, and the boundary between the foreground area and the background area of the image is cleaner and clearer, and no ghosting occurs. Happening.

Or, in another implementation manner, the weight value of the remaining pixels is adjusted according to the depth of field values corresponding to the remaining pixels respectively.

Specifically, since the depth of field values of pixels in different regions in the image are quite different, when adjusting the weight values of the remaining pixels, corresponding adjustments may be made according to the depth values corresponding to the remaining pixels.

For example, the weight value of the pixel point at the same depth level as the pixel point to be blurred may be increased more, and the weight value of the pixel point not at the same depth level as the pixel point to be blurred may be increased less.

It can be understood that by increasing the weight value of the pixels at the same depth of field as the pixels to be blurred, in the image after the blurring of the background area, the foreground area and the background area are realized. The boundary between them is clean and clear to ensure that the background area is not distorted.

To sum up, in the present application, the effect of performing the blurring processing on the image by the above-mentioned image blurring processing method is shown in FIG. 6 .

The image blurring processing method provided by the embodiment of the present invention adjusts the weight value of each pixel point in the target filtering area including the foreground area pixel points, so that in the obtained image after the blurring processing of the background area, the foreground area is The boundary between the background area and the background area is clean and clear, effectively protecting the blurred edge of the background area, making the subject more prominent, improving the overall image quality of the image, and improving the user experience.

In order to realize the above-mentioned embodiments, the present invention also provides an image blurring processing apparatus.

FIG. 7 is a schematic structural diagram of an image blurring processing apparatus according to an embodiment of the present invention.

Referring to FIG. 7 , the image blur processing apparatus includes: an acquisition module 10 , a determination module 20 and an adjustment module 30 .

Wherein, the acquisition module 10 is used to acquire the depth-of-field image to be processed;

The determining module 20 is used for determining the foreground area and the background area in the depth-of-field image;

In a possible implementation situation, the determining module 20 is specifically configured to: determine the foreground area and the background area in the depth-of-field image according to a preset range.

Further, as shown in FIG. 8 , when the depth-of-field image to be processed includes a portrait, the determination module 20 includes: a first determination subunit 21 and a second determination subunit 22 .

Wherein, the first determination subunit 21 is used to perform face recognition on the depth-of-field image, and determine the face area;

The second determination subunit 22 is configured to determine the foreground area and the background area in the depth-of-field image according to the depth-of-field value corresponding to the face area.

The adjustment module 30 is used to perform the blurring process on the background area, if it is determined that the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, then the weight value of each pixel in the target filtering area is determined. make adjustments.

In an implementation form of the present invention, the adjustment module 30 is specifically used for:

If the depth of field value corresponding to any pixel included in the target filtering area is within the preset range, then determine that any pixel is the pixel of the foreground area;

or,

If the difference between the depth values corresponding to two adjacent pixels in the target filtering area is greater than a preset value, it is determined that the target filtering area includes pixels located in the foreground area.

In another possible implementation situation, as shown in FIG. 9 , the adjustment module 30 in the apparatus further includes: a first adjustment subunit 31 and a second adjustment subunit 32 .

Wherein, the first adjustment subunit 31 is used to set the weight value of the pixel in the foreground area to zero;

The second adjustment subunit 32 is configured to adjust the weight values of the remaining pixels in the target filtering area, so that the sum of the weight values of the remaining pixels is equal to one.

Further, the second adjustment subunit 32 is specifically used for:

The weight value of the remaining pixels is proportionally expanded;

or,

The weight value of the remaining pixels is adjusted according to the depth of field values corresponding to the remaining pixels respectively.

It should be noted that, for the implementation process and technical principle of the image blurring processing apparatus in this embodiment, reference may be made to the foregoing explanations of the image blurring processing method embodiments, which will not be repeated here.

The image blurring processing device provided by the embodiment of the present invention determines the foreground area and the background area of the obtained depth-of-field image by acquiring the depth-of-field image to be processed. Whether the target filtering area corresponding to the pixel to be blurred includes pixels located in the foreground area, and if so, adjust the weight value of each pixel in the target filtering area. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels of the foreground area, the boundary between the foreground area and the background area in the obtained image after the blurring of the background area is obtained. Clean and clear, it effectively protects the blurred edge of the background area, and makes the subject more prominent, improving the overall image quality of the image and enhancing the user experience.

In order to realize the above embodiments, the present invention also provides a shooting terminal.

FIG. 10 is a schematic structural diagram of a photographing terminal according to an embodiment of the present invention.

As shown in FIG. 10 , the photographing terminal 100 includes a memory 111 , a processor 112 and an image processing circuit 113 .

Wherein, the memory 111 is used for storing executable instructions of the processor 112;

The processor 112 is configured to call the program code in the memory 111, and implement the image blurring processing method of the embodiment of the first aspect according to the depth image output by the image processing circuit 113.

Specifically, the image processing circuit 113 may be implemented using hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing, image signal processing) pipeline.

FIG. 11 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 11 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present invention are shown.

As shown in FIG. 11 , the image processing circuit 113 includes an imaging device 1140 , an ISP processor 1150 and a control logic 1160 . Imaging device 1140 may include a camera with one or more lenses 1141 , image sensor 1142 and structured light projector 1143 . The structured light projector 1143 projects the structured light to the measured object. Wherein, the structured light pattern may be a laser stripe, a Gray code, a sinusoidal stripe, or a randomly arranged speckle pattern, or the like. The image sensor 1142 captures the structured light image projected onto the measured object, and sends the structured light image to the ISP processor 1150, which demodulates the structured light image to obtain depth information of the measured object. Meanwhile, the image sensor 1142 can also capture the color information of the measured object. Of course, the structured light image and color information of the measured object can also be captured by the two image sensors 1142 respectively.

Wherein, taking the speckle structured light as an example, the ISP processor 1150 demodulates the structured light image, which specifically includes collecting the speckle image of the measured object from the structured light image, and comparing the speckle image of the measured object with the reference The speckle image performs image data calculation according to a predetermined algorithm, and obtains the moving distance of each speckle of the speckle image on the measured object relative to the reference speckle in the reference speckle image. The depth value of each speckle of the speckle image is calculated by triangulation transformation, and the depth information of the measured object is obtained according to the depth value.

Of course, the depth image information can also be obtained by a binocular vision method or a method based on time-of-flight TOF, which is not limited here, as long as the method that can obtain or obtain the depth information of the measured object by calculation belongs to this implementation. the scope of the method.

After the ISP processor 1150 receives the color information of the measured object captured by the image sensor 1142, the image data corresponding to the color information of the measured object can be processed. The ISP processor 1150 analyzes the image data to obtain image statistics that can be used to determine and/or control one or more parameters of the imaging device 1140 . Image sensor 1142 may include an array of color filters (eg, Bayer filters), image sensor 1142 may obtain light intensity and wavelength information captured with each imaging pixel of image sensor 1142 and provide a set of raw materials that may be processed by ISP processor 1150. image data.

The ISP processor 1150 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 1150 may perform one or more image processing operations on the raw image data, collect image statistics about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.

ISP processor 1150 may also receive pixel data from image memory 1170 . The image memory 1170 may be a part of a memory device, a storage device, or an independent dedicated memory within an electronic device, and may include a DMA (Direct Memory Access) feature.

When raw image data is received, the ISP processor 1150 may perform one or more image processing operations.

After the ISP processor 1150 obtains the color information and depth information of the measured object, it can fuse them to obtain a three-dimensional image. The feature of the corresponding measured object can be extracted by at least one of the appearance contour extraction method or the contour feature extraction method. For example, the features of the measured object are extracted by methods such as active shape model method ASM, active appearance model method AAM, principal component analysis method PCA, discrete cosine transform method DCT, etc., which are not limited here. Then, the features of the measured object extracted from the depth information and the features of the measured object extracted from the color information are respectively processed for registration and feature fusion. The fusion processing referred to here can be the direct combination of the features extracted from the depth information and color information, or the combination of the same features in different images after weight setting, or other fusion methods. features to generate a 3D image.

The image data for the three-dimensional image may be sent to image memory 1170 for additional processing before being displayed. The ISP processor 1150 receives processed data from the image memory 1170 and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processed data. The image data of the three-dimensional image may be output to the display 1180 for viewing by the user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the ISP processor 1150 may also be sent to the image memory 1170 , and the display 1180 may read image data from the image memory 1170 . In one embodiment, image memory 1170 may be configured to implement one or more frame buffers. Also, the output of the ISP processor 1150 may be sent to the encoder/decoder 1190 for encoding/decoding the image data. The encoded image data can be saved and decompressed prior to display on the display 1180 device. The encoder/decoder 1190 may be implemented by a CPU or GPU or a coprocessor.

The image statistics determined by the ISP processor 1150 may be sent to the control logic 1160 unit. Control logic 1160 may include a processor and/or microcontroller executing one or more routines (eg, firmware) that may determine control parameters of imaging device 1140 based on received image statistics.

The photographing terminal provided by the embodiment of the present invention determines the foreground area and the background area of the acquired depth image by acquiring the depth image to be processed, and then, when performing the blurring process on the determined background area, firstly determines the background area to be blurred Whether the target filtering area corresponding to the pixel points of the target filter area includes pixels located in the foreground area, and if so, adjust the weight value of each pixel point in the target filtering area. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels of the foreground area, the boundary between the foreground area and the background area in the obtained image after the blurring of the background area is obtained. Clean and clear, it effectively protects the blurred edge of the background area, and makes the subject more prominent, improving the overall image quality of the image and enhancing the user experience.

In order to implement the above-mentioned embodiments, the present invention also provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, implements the image blurring processing method of the embodiment of the first aspect.

The computer-readable storage medium provided by the embodiment of the present invention can be set in a shooting terminal that needs to perform image blurring processing. When the terminal device performs image blurring processing, the acquired depth-of-field image is determined by acquiring the depth-of-field image to be processed. The foreground area and the background area of The weight value of each pixel in the filter area is adjusted. Therefore, by adjusting the weight value of each pixel in the target filtering area including the pixels of the foreground area, the boundary between the foreground area and the background area in the obtained image after the blurring of the background area is obtained. Clean and clear, it effectively protects the blurred edge of the background area, and makes the subject more prominent, improving the overall image quality of the image and enhancing the user experience.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature.

Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the invention includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present invention belong.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (10)

1. An image blurring processing method, comprising:

acquiring a depth-of-field image to be processed;

determining a foreground region and a background region in the depth image;

when blurring processing is carried out on the background area, if it is determined that a target filtering area corresponding to a pixel point to be blurred contains a pixel point located in the foreground area, the weighted value of each pixel point in the target filtering area is adjusted.

2. The method of claim 1, wherein said determining foreground and background regions in said range image comprises:

and determining a foreground region and a background region in the depth image according to a preset range.

3. The method according to claim 1, wherein the depth image to be processed includes a portrait;

the determining a foreground region and a background region in the depth image comprises:

carrying out face recognition on the depth-of-field image to determine a face area;

and determining a foreground region and a background region in the depth-of-field image according to the depth-of-field value corresponding to the face region.

4. The method of claim 2, wherein the determining that the target filtering region corresponding to the pixel point to be blurred includes a pixel point located in the foreground region comprises:

if the depth of field value corresponding to any pixel point contained in the target filtering area is within the preset range, determining that any pixel point is a pixel point of the foreground area;

or,

and if the difference value of the depth of field values respectively corresponding to two adjacent pixel points in the target filtering area is greater than a preset value, determining that the target filtering area contains the pixel points positioned in the foreground area.

5. The method according to any one of claims 1 to 3, wherein the adjusting the weight value of each pixel point in the target filtering region includes:

setting the weighted value of the pixel point positioned in the foreground area to be zero;

and adjusting the weighted values of the residual pixel points in the target filtering area to enable the sum of the weighted values of the residual pixel points to be equal to one.

6. The method of claim 5, wherein the adjusting the weight values of the remaining pixels in the target filtering region comprises:

expanding the weight values of the residual pixel points in equal proportion;

or,

and adjusting the weight values of the residual pixel points according to the depth of field values respectively corresponding to the residual pixel points.

7. An image blurring processing apparatus, comprising:

the acquisition module is used for acquiring a depth-of-field image to be processed;

the determining module is used for determining a foreground region and a background region in the depth image;

and the adjusting module is used for adjusting the weight value of each pixel point in the target filtering area if it is determined that the target filtering area corresponding to the pixel point to be blurred contains the pixel point positioned in the foreground area when blurring the background area.

8. The apparatus of claim 7, wherein the adjustment module is specifically configured to:

if the depth of field value corresponding to any pixel point contained in the target filtering area is within a preset range, determining that any pixel point is a pixel point of the foreground area;

or,

and if the difference value of the depth of field values respectively corresponding to two adjacent pixel points in the target filtering area is greater than a preset value, determining that the target filtering area contains the pixel points positioned in the foreground area.

9. A photographing terminal, comprising: a memory, a processor, and an image processing circuit;

the memory for storing executable instructions of the processor;

the image processing circuit is used for processing the depth image;

the processor is used for calling the executable instructions in the memory and realizing the image blurring processing method according to any one of claims 1 to 6 according to the depth image output by the image processing circuit.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out an image blurring processing method according to any one of claims 1 to 6.