WO2018112764A1

WO2018112764A1 - Image processing method and apparatus, control method and apparatus, imaging apparatus, and electronic apparatus

Info

Publication number: WO2018112764A1
Application number: PCT/CN2016/111149
Authority: WO
Inventors: 孙剑波
Original assignee: 广东欧珀移动通信有限公司
Priority date: 2016-12-20
Filing date: 2016-12-20
Publication date: 2018-06-28

Abstract

Disclosed in the present invention is an image processing method, comprising: (S1) identifying whether an overexposed part in an in-focus state is present in a focused image; (S2) blurring the focused image to obtain a blurred focused image; (S3) when an overexposed part is present, processing an out-of-focus image of the same scene as the focused image to obtain a material part corresponding to the overexposed part; and (S4) substituting the material part for the blurred overexposed part corresponding to the overexposed part in the blurred focused image to obtain a merged image. Also disclosed in the present invention are an image processing apparatus (111), a control method, a control apparatus (11), an imaging apparatus (10), and an electronic apparatus (100).

Description

Image processing method and device, control method and device, imaging and electronic device

Technical field

The present invention relates to image processing technology, and in particular, to an image processing method, a control method, an image processing device, a control device, an imaging device, and an electronic device.

Background technique

Existing blurring algorithms, such as the background blur of portrait photos, generally use a fuzzy algorithm for the software blurring algorithm. The fuzzy algorithm is an average algorithm that averages the pixel brightness values of the light source when blurring the light source in the photo. The light source in the photo appears dim and the effect is not good.

Summary of the invention

Embodiments of the present invention aim to solve at least one of the technical problems existing in the prior art. To this end, embodiments of the present invention are required to provide an image processing method, a control method, an image processing device, a control device, an imaging device, and an electronic device.

The invention provides an image processing method, the image processing method comprising the following steps:

Identifying whether there is an overexposed portion in the in-focus state in the in-focus image;

Bluning the in-focus image to obtain a blurred focus image;

Processing the virtual focus image identical to the focused image scene to obtain a material portion corresponding to the overexposed portion when the overexposed portion is present; and

A blurred overexposed portion corresponding to the overexposed portion of the blurred focus image is replaced with the material portion to obtain a merged image.

In some embodiments, the in-focus image is in a panoramic deep focus state.

In some embodiments, the step of identifying whether there is an overexposed portion in the in-focus state in the in-focus image comprises:

Determining, according to a histogram of the in-focus image, whether the overexposed pixel in the in-focus image is more than or equal to a first predetermined number, and the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value; and

It is determined that the overexposed portion exists when the overexposed pixel is more than or equal to the first predetermined number.

When there is the overexposed portion, determining whether there is an adjacent overexposed pixel in the in-focus image, the number of the adjacent overexposed pixels being more than or equal to a second predetermined number; and

When the adjacent overexposed pixels are present, it is determined that the adjacent overexposed pixels belong to the overexposed portion.

In some embodiments, the predetermined pixel value is an upper limit of a range of pixel values of the expansive image.

In some embodiments, the step of blurring the in-focus image to obtain a blurred focus image is implemented using a Gaussian blur algorithm.

In some embodiments, the degree of blur of the material portion is substantially the same as the degree of blur of the blur exposed portion.

In some embodiments, the image processing method comprises:

Outputting the merged image when the overexposed portion is present; and

The blurred focus image is output when the overexposed portion is absent.

The present invention provides a control method for controlling an imaging device, the control method comprising the following steps:

Controlling the imaging device to output a focused image and a virtual focus image of the same scene; and

The in-focus image and the virtual focus image are processed by the image processing method according to any of the above embodiments.

In some embodiments, the step of controlling the imaging device to output a focused image and a virtual focus image of the same scene comprises the steps of:

Controlling the imaging device to focus to obtain the in-focus image;

Refocusing and processing the cached image output by the imaging device in different focusing states to obtain a corresponding degree of blur; and

The cached image having the same degree of blur as the blurred focus image is saved as the virtual focus image.

The present invention provides an image processing apparatus, the image processing apparatus comprising the following steps:

An identification module, configured to identify whether an overexposed portion in an in-focus state exists in the in-focus image;

a blurring module, configured to blur the in-focus image to obtain a blurred focus image;

a processing module, configured to process a virtual focus image identical to the focused image scene to obtain a material portion corresponding to the overexposed portion when the overexposed portion is present; and

And a merging module, configured to replace the imaginary overexposed portion corresponding to the overexposed portion in the blurred focus image with the material portion to obtain a merged image.

In some embodiments, the in-focus image is in a panoramic deep focus state.

In some embodiments, the identification module comprises:

a first determining sub-module, configured to determine, according to a histogram of the in-focus image, whether the over-exposed pixel in the in-focus image is greater than or equal to a first predetermined number, and the pixel value of the over-exposed pixel is greater than or equal to a predetermined pixel value; with

The first determining submodule is configured to determine that the overexposed portion exists when the overexposed pixel is more than or equal to the first predetermined number.

In some embodiments, the identification module further includes:

a second determining sub-module, configured to determine, in the presence of the over-exposed portion, whether there is an adjacent over-exposed pixel in the in-focus image, the number of the adjacent over-exposed pixels being more than or equal to the second The predetermined number; and

a second determining submodule, configured to determine that the adjacent overexposed pixels belong to the overexposed portion when the adjacent overexposed pixels are present.

In some embodiments, the blurring module is configured to use a Gaussian blurring algorithm to blur the in-focus image to obtain a blurred focus image.

In some embodiments, the image processing apparatus comprises:

a first output module, configured to output the merged image when the overexposed portion is present; and

And a second output module, configured to output the blurred focus image when the overexposed portion is absent.

The present invention provides a control device for controlling an imaging device, the control device comprising:

a control module, configured to control the imaging device to output a focused image and a virtual focus image of the same scene; and

An image processing apparatus as described in any of the above embodiments and electrically connected to the control module.

In some embodiments, the control module comprises:

a focusing sub-module, configured to control the imaging device to focus to obtain the in-focus image;

a focusing sub-module for refocusing and processing a cached image output by the imaging device in different focusing states to obtain a corresponding degree of blur; and

And a saving submodule, configured to save the cached image having a degree of blurring substantially the same as the blurred focus image as the virtual focus image.

The invention provides an imaging device, the imaging device comprising:

camera;

a control module electrically connected to the camera, configured to control the camera to output a focused image and a virtual focus image of the same scene; and

In some embodiments, the control module comprises:

The present invention provides an electronic device comprising the imaging device according to any of the above embodiments.

In some embodiments, the electronic device includes any one of a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart bracelet, a smart helmet, and smart glasses.

In certain embodiments, the imaging device comprises a front camera or/and a rear camera.

The present invention provides an electronic device including a housing, a processor, a memory, a circuit board, and a power supply circuit. The circuit board is disposed inside a space enclosed by the casing, and the processor and the memory are disposed on the circuit board. The power circuit for supplying power to respective circuits or devices of the electronic device; the memory for storing executable program code; the processor running by reading executable program code stored in the memory The program corresponding to the executable program code is used to execute the image processing method according to any of the above embodiments or to execute the control method described in any of the above embodiments.

The present invention provides a computer readable storage medium having instructions stored therein, when the processor of the electronic device executes the instructions, the electronic device executes the image processing method according to any one of the above embodiments or performs the above-described A control method according to an embodiment.

The image processing method, the image processing device, the control method, the control device, the imaging device and the electronic device provided by the invention take two images, one focused image, one virtual focus image, and the virtual focus image and the focused image are taken The material part corresponding to the exposed part is combined into the blurred focus image to form a combined image with a real spot effect, and the spot effect is good.

The additional aspects and advantages of the embodiments of the present invention will be set forth in part in the description which follows.

DRAWINGS

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from

1 is a flow chart showing an image processing method according to a first embodiment of the present invention.

2 is a functional block diagram of an image processing apparatus according to a first embodiment of the present invention.

3 is a flow chart showing a control method of the first embodiment of the present invention.

4 is a schematic diagram of functional blocks of a control device according to a first embodiment of the present invention.

Fig. 5 is a schematic diagram showing the functional blocks of the image forming apparatus of the first embodiment of the present invention.

6 is a schematic diagram of functional blocks of an electronic device according to a first embodiment of the present invention.

FIG. 7 is a schematic diagram of a physical object of the electronic device of FIG. 6. FIG.

Figure 8 is a schematic view showing the operation of the electronic device of the present invention.

9 is a flow chart showing an image processing method according to a second embodiment of the present invention.

Fig. 10 is a functional block diagram of an image processing apparatus according to a second embodiment of the present invention.

Figure 11 is a histogram of a focused image of the present invention.

Fig. 12 is a flow chart showing the control method of the second embodiment of the present invention.

Fig. 13 is a schematic diagram showing the functional blocks of the control device according to the second embodiment of the present invention.

Fig. 14 is a schematic diagram showing the functional blocks of an image forming apparatus according to a second embodiment of the present invention.

15 is a schematic diagram of functional blocks of an electronic device according to a second embodiment of the present invention.

Fig. 16 is a flow chart showing an image processing method according to a third embodiment of the present invention.

Figure 17 is a diagram showing the functional blocks of an image processing apparatus according to a third embodiment of the present invention.

Fig. 18 is a flow chart showing the control method of the third embodiment of the present invention.

Fig. 19 is a schematic diagram showing the functional blocks of the control device according to the third embodiment of the present invention.

20 is a schematic diagram of functional blocks of an image forming apparatus according to a third embodiment of the present invention.

21 is a schematic diagram of functional blocks of an electronic device according to a third embodiment of the present invention.

Fig. 22 is a flow chart showing an image processing method according to a fourth embodiment of the present invention.

Figure 23 is a diagram showing the functional blocks of an image processing apparatus according to a fourth embodiment of the present invention.

Fig. 24 is a flow chart showing the control method of the fourth embodiment of the present invention.

Figure 25 is a diagram showing the functional blocks of a control device according to a fourth embodiment of the present invention.

Fig. 26 is a schematic diagram showing the functional blocks of an image forming apparatus according to a fourth embodiment of the present invention.

Figure 27 is a diagram showing the functional blocks of an electronic device according to a fourth embodiment of the present invention.

28 is a flow diagram of controlling an imaging device to output a focused image and a virtual focus image of the same scene, in accordance with some embodiments of the present invention.

29 is a functional block diagram of a control module of some embodiments of the present invention.

detailed description

The embodiments of the present invention are described in detail below, and the examples of the embodiments are illustrated in the drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the drawings are intended to be illustrative of the invention and are not to be construed as limiting.

In the description of the present invention, it is to be understood that the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" or "second" may include one or more of the described features either explicitly or implicitly. In the description of the present invention, the meaning of "a plurality" is two or more unless specifically and specifically defined otherwise.

In the description of the present invention, it should be noted that the terms "installation", "connected", and "connected" are to be understood broadly, and may be fixed or detachable, for example, unless otherwise explicitly defined and defined. Connected, or connected in one piece; Therefore, they may be mechanically connected, or may be electrically connected or may communicate with each other; they may be directly connected or indirectly connected through an intermediate medium, and may be internal communication of two elements or an interaction relationship of two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood on a case-by-case basis.

The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and arrangements of the specific examples are described below. Of course, they are merely examples and are not intended to limit the invention. In addition, the present invention may be repeated with reference to the numerals and/or reference numerals in the various examples, which are for the purpose of simplicity and clarity, and do not indicate the relationship between the various embodiments and/or arrangements discussed. Moreover, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the use of other processes and/or the use of other materials.

The present invention provides an image processing method according to a plurality of embodiments, a control method of a plurality of embodiments, an image processing apparatus according to a plurality of embodiments, a control apparatus of a plurality of embodiments, an imaging apparatus of a plurality of embodiments, and a plurality of embodiments Electronic device. Hereinafter, the image processing methods of the plurality of embodiments are sequentially numbered, the control methods of the plurality of embodiments are sequentially numbered, the image processing apparatuses of the plurality of embodiments are sequentially numbered, and the control apparatuses of the plurality of embodiments are described. The sequential numbering will be described, the imaging devices of the plurality of embodiments will be sequentially numbered, and the electronic devices of the plurality of embodiments will be sequentially numbered. The image processing method of each embodiment may correspond to a control method of one embodiment, an image processing device corresponding to one embodiment, a control device corresponding to one embodiment, an imaging device corresponding to one embodiment, and a corresponding embodiment. Electronic device. For example, the image processing method of the first embodiment may correspond to the control method of the first embodiment, the image processing device that can correspond to the first embodiment, the control device that can correspond to the first embodiment, and the imaging device that can correspond to the first embodiment. And the electronic device according to the first embodiment; the image processing method according to the second embodiment can correspond to the control method of the second embodiment, the image processing device according to the second embodiment, and the control device according to the second embodiment. The imaging device according to the first embodiment and the electronic device according to the second embodiment can be used.

Referring to FIG. 1, an image processing method according to a first embodiment of the present invention includes the following steps:

S1, identifying whether there is an overexposed portion in the in-focus state in the in-focus image;

S2, blur the in-focus image to obtain a blurred focus image;

S3, when there is an overexposed portion, processing the same virtual focus image as the focused image scene to obtain a material portion corresponding to the overexposed portion; and

S4, replacing the blurred overexposed portion corresponding to the overexposed portion in the blurred focus image with the material portion to obtain a merged image.

The step of blurring the in-focus image to obtain the blurred focus image may be implemented by using a Gaussian blur algorithm. The focus image is in panoramic deep focus. The degree of blurring of the material part is basically the same as the degree of blurring of the blur exposed portion.

Referring to FIG. 2, the image processing method of the first embodiment described above can be realized by the image processing apparatus 111 of the first embodiment of the present invention. Specifically, the image processing apparatus 111 of the first embodiment of the present invention includes an identification module 1111, a blurring module 1112, a processing module 1113, and a merging module 1114, which can be used to execute S1 to S4, respectively. That is, the identification module 1111 is configured to recognize whether there is an overexposed portion in the in-focus state in the in-focus image. The blurring module 1112 is configured to blur the in-focus image to obtain a blurred focus image. The processing module 1113 is configured to process the same virtual focus image as the focused image scene when there is an overexposed portion to obtain a material portion corresponding to the overexposed portion. The merging module 1114 is configured to replace the imaginary overexposed portion corresponding to the overexposed portion in the blurred focus image with the material portion to obtain a merged image.

The blurring module 1112 uses a Gaussian blur algorithm to blur the in-focus image to obtain a blurred focus image. The focus image is in panoramic deep focus. The degree of blurring of the material part is basically the same as the degree of blurring of the blur exposed portion.

Referring to FIG. 3 and FIG. 5-7, the control method of the first embodiment of the present invention is used to control the imaging device 10. The control method includes the following steps:

S6, controlling the imaging device 10 to output the in-focus image and the virtual focus image of the same scene;

S2, blur the in-focus image to obtain a blurred focus image;

Referring to FIG. 4, the control method of the above-described first embodiment can be realized by the control device 11 of the first embodiment of the present invention. Specifically, in the control device 11 of the first embodiment of the present invention, the control device 11 includes the image processing device 111 in the first embodiment and a control module 112 electrically connected to the image processing device 111. The image processing device 111 is configured to perform steps S1 to S4, and the control module 112 is configured to perform step S6. The structure of the image processing apparatus 111 is as described in the first embodiment, and will not be described herein. The control module 112 is for controlling the imaging device 10 (shown in FIGS. 5-7) to output a focused image and a virtual focus image of the same scene.

Referring to Fig. 5, an image forming apparatus 10 according to a first embodiment of the present invention includes a control device 11 of the first embodiment and a camera 12 electrically connected to the control device 11 of the first embodiment. In other words, the imaging device 10 of the present embodiment includes the image processing device 111 of the first embodiment, the control module 112, and the camera 12. Control module 112 is electrically connected to both the camera 12 and the image processing device 111.

Referring to FIGS. 6 and 7, an electronic device 100 according to a first embodiment of the present invention includes the imaging device 10 of the first embodiment. The electronic device 100 may be any one of a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart wristband, a smart helmet, smart glasses, other virtual reality wearable devices, and other augmented reality wearable devices. When the number of imaging devices 10 is one, the imaging device 10 may be a front camera or a rear camera. When the number of imaging devices 10 is two, the two imaging devices 10 may be a front camera and a rear camera, respectively; or both imaging devices 10 are front cameras; or both imaging devices 10 are rear cameras . Of course, when the number of the imaging devices 10 is two or more, the imaging device 10 may be a camera disposed at any position, and may be a top camera, a bottom camera, a side wall camera, or the like in addition to the front camera and the rear camera.

For example, referring to FIG. 8, the camera 12 captures a focused image (shown in the upper left panel of FIG. 8) and a virtual focus image (shown in the upper right panel of FIG. 8) for the same scene. Among them, the in-focus image is in the panoramic focus state. After being identified, it can be judged that there is an overexposed portion in the in-focus image, that is, an elongated strip tube above. Blur the in-focus image to obtain a blurred focus image (shown in the lower left panel of Figure 8), wherein the overexposed portion (long strip light tube) in the blurred focus image is blurred to obtain a blurred overexposed portion (long strip shape) Light tube). When there is an overexposed portion, the virtual focus image is processed to obtain a material portion (long strip light tube) corresponding to the overexposed portion. The blurred overexposed portion corresponding to the overexposed portion in the blurred focus image is replaced with the material portion to obtain a merged image (shown in the lower right panel of FIG. 8).

The image processing method, the image processing device 111, the control method, the control device 11, the imaging device 10, and the electronic device 100 in the first embodiment of the present invention take two images, one focused image, one virtual focus image, and the virtual image is taken out. The material part corresponding to the overexposed part in the focus image in the focus image is combined into the blurred focus image to synthesize a combined image with a real spot effect, and the spot effect is good.

Referring to FIG. 9, the image processing method of the second embodiment of the present invention is substantially the same as the image processing method of the first embodiment. Further, the step of identifying whether there is an overexposed portion in the in-focus state in the focused image includes the following steps. :

S11. Determine, according to a histogram of the in-focus image, whether the overexposed pixel in the in-focus image is greater than or equal to a first predetermined number, and the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value, where the predetermined pixel value is an upper limit of a range of pixel values of the in-focus image. , for example, a predetermined pixel value of 255; and

S12, determining that there is an overexposed portion when the overexposed pixel is more than or equal to the first predetermined number.

Referring to Fig. 10, the image processing method of the second embodiment described above can be realized by the image processing apparatus 111 of the second embodiment of the present invention. The image processing device 111 of the second embodiment of the present invention has substantially the same structure as the image processing device 111 of the first embodiment, except that the identification module 1111 of the image processing device 111 of the second embodiment includes the first determination sub-module 11111. And the first determining sub-module 11112 can be used to perform steps S11 and S12, respectively. That is, the first judging sub-module 11111 is configured to judge according to the histogram of the in-focus image. Whether the overexposed pixel in the in-focus image is more than or equal to a first predetermined number, the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value, and the predetermined pixel value is an upper limit of a range of pixel values of the in-focus image, for example, a predetermined pixel value is 255. The first determining sub-module 11112 is configured to determine that there is an overexposed portion when the overexposed pixel is more than or equal to the first predetermined number.

Referring to FIG. 11, in general, the horizontal axis of the histogram of the in-focus image represents the increment of the pixel value (gray value) from left to right, and the pixel of the pixel value (gray value) from the bottom to the top of the vertical axis. The number is increasing. The pixel value (gray value) ranges from 0 to 255, that is, from black to white, and the higher the peak at a certain point, the more pixels are indicated at the pixel value (gray value).

The first predetermined number should be the number of pixels in the histogram near the right border, that is, the number of overexposed pixels, and the first predetermined number can be set to be one third of the total pixels. It should be noted that one third is only It is a schematic illustration and is not a limitation of the invention.

Referring to FIG. 12, the control method of the second embodiment of the present invention is substantially the same as the control method of the first embodiment. Further, the step of identifying whether there is an overexposed portion in the in-focus state in the focused image includes the following steps:

Referring to Fig. 13, the control method of the second embodiment described above can be realized by the control device 11 of the second embodiment of the present invention. The control device 11 of the second embodiment of the present invention has substantially the same structure as the control device 11 of the first embodiment, except that the identification module 1111 of the control device 11 of the second embodiment includes a first determination sub-module 11111 and a first The determining sub-module 11112 can be used to perform steps S11 and S12, respectively. That is, the first determining sub-module 11111 is configured to determine, according to the histogram of the focused image, whether the overexposed pixel in the focused image is more than or equal to a first predetermined number, and the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value, the predetermined pixel The value is the upper limit of the range of pixel values of the in-focus image, for example, the predetermined pixel value is 255. The first determining sub-module 11112 is configured to determine that there is an overexposed portion when the overexposed pixel is more than or equal to the first predetermined number.

Referring to FIG. 11, in general, the horizontal axis of the histogram of the in-focus image represents the increment of the pixel value (gradation value) from left to right, and the vertical axis represents the pixel at a certain pixel value (gray value) from bottom to top. The number is increasing. The pixel value (gray value) ranges from 0 to 255, that is, from black to white, and the higher the peak at a certain point, the more pixels are indicated at the pixel value (gray value).

Referring to Fig. 14, an image forming apparatus 10 according to a second embodiment of the present invention includes a control device 11 of the second embodiment and a camera 12 electrically connected to the control device 11 of the second embodiment. In other words, the imaging device 10 of the present embodiment includes the image processing device 111, the control module 112, and the camera 12 of the second embodiment. The control module 112 is electrically connected to both the camera 12 and the image processing device 111.

Referring to FIG. 15, an electronic device 100 according to a second embodiment of the present invention includes the imaging device 10 of the second embodiment. The electronic device 100 may be any one of a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart wristband, a smart helmet, smart glasses, other virtual reality wearable devices, and other augmented reality wearable devices. When the number of imaging devices 10 is one, the imaging device 10 may be a front camera or a rear camera. When the number of imaging devices 10 is two, the two imaging devices 10 may be a front camera and a rear camera, respectively; or both imaging devices 10 are front cameras; or both imaging devices 10 are rear cameras . Of course, when the number of the imaging devices 10 is two or more, the imaging device 10 may be a camera disposed at any position, and may be a top camera, a bottom camera, a side wall camera, or the like in addition to the front camera and the rear camera.

The image processing method, the image processing device 111, the control method, the control device 11, the imaging device 10, and the electronic device 100 in the second embodiment of the present invention take two images, one focused image, one virtual focus image, and the virtual image is taken out. The material part corresponding to the overexposed part in the focus image in the focus image is combined into the blurred focus image to synthesize a combined image with a real spot effect, and the spot effect is good.

Referring to FIG. 16, the image processing method according to the third embodiment of the present invention is substantially the same as the image processing method of the first embodiment. Further, the step of identifying whether there is an overexposed portion in the in-focus state in the focused image includes the following steps. :

S11. Determine, according to a histogram of the in-focus image, whether the overexposed pixel in the in-focus image is greater than or equal to a first predetermined number, and the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value, where the predetermined pixel value is an upper limit of a range of pixel values of the in-focus image. , for example, the predetermined pixel value is 255;

S12, determining that there is an overexposed portion when the overexposed pixel is more than or equal to the first predetermined number;

S13, when there is an overexposed portion, determining whether there is an adjacent overexposed pixel in the in-focus image, and the number of adjacent overexposed pixels is greater than or equal to a second predetermined number; and

S14. When there is an adjacent overexposed pixel, it is determined that the adjacent overexposed pixel belongs to the overexposed portion.

Referring to Fig. 17, the image processing method of the third embodiment described above can be realized by the image processing apparatus 111 of the third embodiment of the present invention. The image processing device 111 of the third embodiment of the present invention has substantially the same structure as the image processing device 111 of the first embodiment, except that the recognition module 1111 of the image processing device 111 of the third embodiment includes the first determination sub-module 11111. The first determining sub-module 11112, the second determining sub-module 11113, and the second determining sub-module 11114 are respectively configured to perform steps S11, S12, S13, and S14. That is, the first determining sub-module 11111 is configured to determine, according to the histogram of the focused image, that the overexposed pixel in the focused image is No more than or equal to the first predetermined number, the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value, and the predetermined pixel value is an upper limit of a range of pixel values of the in-focus image, for example, the predetermined pixel value is 255. The first determining sub-module 11112 is configured to determine that there is an overexposed portion when the overexposed pixel is more than or equal to the first predetermined number. The second judging sub-module 11113 is configured to determine whether there is an adjacent over-exposed pixel in the in-focus image when there is an over-exposed portion, and the number of adjacent over-exposed pixels is greater than or equal to a second predetermined number. The second determining sub-module 11114 is configured to determine that the adjacent over-exposed pixels belong to the over-exposed portion when there is an adjacent over-exposed pixel.

Wherein, the second predetermined number should be smaller than the first predetermined number, because the first predetermined number may include noise, and even the in-focus image may include a plurality of adjacent overexposed pixels, for example, there are a plurality of discrete light sources, and the light sources are combined Overexposed part.

The first predetermined number and the second predetermined number should be the number of pixels in the histogram near the right boundary, that is, the number of overexposed pixels, and the first predetermined number may be set to be one third of the total pixels, and the second predetermined number may be It is set to account for a quarter of the total pixels. It should be noted that one-third and one-quarter are only illustrative and are not intended to limit the invention.

Referring to FIG. 18, the control method of the third embodiment of the present invention is substantially the same as the control method of the first embodiment. Further, the step of identifying whether there is an overexposed portion in the in-focus state in the focused image includes the following steps:

Referring to Fig. 19, the control method of the third embodiment described above can be realized by the control device 11 of the third embodiment of the present invention. The control device 11 of the third embodiment of the present invention has substantially the same structure as the control device 11 of the first embodiment, except that the identification module 1111 of the control device 11 of the third embodiment includes a first determination sub-module 11111, and a first The determining sub-module 11112, the second determining sub-module 11113, and the second determining sub-module 11114 are respectively configured to perform steps S11, S12, S13, and S14. That is, the first judgment sub-module 11111 And a method for determining whether the overexposed pixel in the in-focus image is greater than or equal to a first predetermined number according to a histogram of the in-focus image, wherein the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value, and the predetermined pixel value is an upper limit of a pixel value range of the in-focus image. For example, the predetermined pixel value is 255. The first determining sub-module 11112 is configured to determine that there is an overexposed portion when the overexposed pixel is more than or equal to the first predetermined number. The second judging sub-module 11113 is configured to determine whether there is an adjacent over-exposed pixel in the in-focus image when there is an over-exposed portion, and the number of adjacent over-exposed pixels is greater than or equal to a second predetermined number. The second determining sub-module 11114 is configured to determine that the adjacent over-exposed pixels belong to the over-exposed portion when there is an adjacent over-exposed pixel.

Referring to Fig. 20, an image forming apparatus 10 according to a third embodiment of the present invention includes a control device 11 of a third embodiment and a camera 12 electrically connected to the control device 11 of the third embodiment. In other words, the imaging device 10 of the present embodiment includes the image processing device 111, the control module 112, and the camera 12 of the third embodiment. The control module 112 is electrically connected to both the camera 12 and the image processing device 111.

Referring to FIG. 21, an electronic device 100 according to a third embodiment of the present invention includes the imaging device 10 of the third embodiment. The electronic device 100 may be any one of a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart wristband, a smart helmet, smart glasses, other virtual reality wearable devices, and other augmented reality wearable devices. When the number of imaging devices 10 is one, the imaging device 10 may be a front camera or a rear camera. When the number of imaging devices 10 is two, the two imaging devices 10 may be a front camera and a rear camera, respectively; or both imaging devices 10 are front cameras; or both imaging devices 10 are rear cameras . Of course, when the number of the imaging devices 10 is two or more, the imaging device 10 may be a camera disposed at any position, and may be a top camera, a bottom camera, a side wall camera, or the like in addition to the front camera and the rear camera.

The image processing method, the image processing device 111, the control method, the control device 11, the imaging device 10, and the electronic device 100 in the third embodiment of the present invention capture two images, one focused image, and one virtual focus image. The material part corresponding to the overexposed part of the in-focus image in the virtual focus image is taken out, and is merged into the blurred focus image to synthesize a combined image with a real spot effect, and the spot effect is good.

Referring to FIG. 22, the image processing method of the fourth embodiment of the present invention is substantially the same as the image processing method of the first embodiment. Further, the image processing method of the fourth embodiment further includes the following steps:

S7, outputting a merged image when there is an overexposed portion; and

S8, when there is no overexposed portion, the blurred focus image is output.

Referring to Fig. 23, the image processing method of the fourth embodiment described above can be realized by the image processing apparatus 111 of the fourth embodiment of the present invention. The image processing device 111 of the fourth embodiment of the present invention has substantially the same structure as the image processing device 111 of the first embodiment, except that the image processing device 111 of the fourth embodiment further includes a first output module 1117 and a second output. Module 1118 can be used to perform steps S7 and S8, respectively. That is, the first output module 1117 is configured to output a merged image when there is an overexposed portion. The second output module 1118 is configured to output a blurred focus image when there is no overexposed portion.

Referring to FIG. 24, the control method of the fourth embodiment of the present invention is substantially the same as the control method of the first embodiment. The control method of the fourth embodiment further includes the following steps:

S7, outputting a merged image when there is an overexposed portion; and

S8, when there is no overexposed portion, the blurred focus image is output.

Referring to Fig. 25, the control method of the fourth embodiment described above can be realized by the control device 11 of the fourth embodiment of the present invention. The control device 11 of the fourth embodiment of the present invention has basically the same configuration as the control device 11 of the first embodiment, except that the image processing device 111 of the control device 11 of the fourth embodiment further includes a first output module 1117 and a The two output modules 1118 can be used to perform steps S7 and S8, respectively. That is, the first output module 1117 is configured to output a merged image when there is an overexposed portion. The second output module 1118 is configured to output a blurred focus image when there is no overexposed portion.

Referring to Fig. 26, an image forming apparatus 10 according to a fourth embodiment of the present invention includes a control device 11 of a fourth embodiment and a camera 12 electrically connected to the control device 11 of the fourth embodiment. In other words, the imaging device 10 of the present embodiment includes the image processing device 111, the control module 112, and the camera 12 of the fourth embodiment. The control module 112 is electrically connected to both the camera 12 and the image processing device 111.

Referring to FIG. 27, an electronic device 100 according to a fourth embodiment of the present invention includes the imaging device 10 of the fourth embodiment. The electronic device 100 may be any one of a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart wristband, a smart helmet, smart glasses, other virtual reality wearable devices, and other augmented reality wearable devices. When the number of imaging devices 10 is one, the imaging device 10 may be a front camera or a rear camera. When the number of imaging devices 10 is two, the two imaging devices 10 may be a front camera and a rear camera, respectively; or both imaging devices 10 are front cameras; or both imaging devices 10 are rear cameras . when However, when the number of the imaging devices 10 is two or more, the imaging device 10 may be a camera disposed at any position, and may be a top camera, a bottom camera, a side wall camera, or the like in addition to the front camera and the rear camera.

The image processing method, the image processing device 111, the control method, the control device 11, the imaging device 10, and the electronic device 100 in the fourth embodiment of the present invention take two images, one focused image, one virtual focus image, and the virtual image is taken out. The material part corresponding to the overexposed part in the focus image in the focus image is combined into the blurred focus image to synthesize a combined image with a real spot effect, and the spot effect is good.

It can be understood that the steps S7 and S8 can also be applied to the image processing method of the second embodiment and the control method of the second embodiment. Correspondingly, the first output module 1117 and the second output module 1118 can also be applied to the second embodiment. The image processing device, the control device of the second embodiment, the imaging device of the second embodiment, and the electronic device of the second embodiment. Steps S7 and S8 can also be applied to the image processing method of the third embodiment and the control method of the third embodiment. Correspondingly, the first output module 1117 and the second output module 1118 can also be applied to the image processing of the third embodiment. The device, the control device of the third embodiment, the imaging device of the third embodiment, and the electronic device of the third embodiment.

Referring to FIG. 28, in the control methods of the foregoing first to fourth embodiments, further, the step of controlling the imaging device 10 to output the in-focus image and the virtual focus image of the same scene (step S6) may include the following steps:

S61, controlling the imaging device to focus to obtain a focused image;

S62, refocusing and processing the cached image output by the imaging device in different focusing states to obtain a corresponding degree of blur; and

S63, storing a cache image whose blur degree is substantially the same as the blur focus image as a virtual focus image.

Correspondingly, referring to FIG. 29, the control module 112 in the control device 11 of the first to fourth embodiments includes a focus sub-module 1121, a focus sub-module 1122, and a save sub-module 1123, which can be used to perform step S61, respectively. S62 and S63. That is, the focusing sub-module 1121 is for controlling the imaging device to focus to obtain a focused image. The focus sub-module 1122 is used to refocus and process the cached image output by the imaging device in different focus states to obtain a corresponding degree of blur. And the saving sub-module 1123 is configured to store a cache image having substantially the same degree of blur as the blurred focus image as a virtual focus image.

Correspondingly, the imaging device 10 of the first to fourth embodiments and the control module 112 of the electronic device 100 may also include a focusing sub-module 1121, a focusing sub-module 1122, and a saving sub-module 1123. The specific structure and function are the same as before. It will not be expanded in detail here.

Certain embodiments of the present invention also provide an electronic device including a housing, a processor, a memory, a circuit board, and a power supply circuit. The circuit board is disposed inside a space enclosed by the casing, and the processor and the memory are disposed on the circuit board. The power circuit for powering various circuits or devices of the electronic device; the memory for storing executable program code; the processor reading the memory by reading The stored executable program code runs a program corresponding to the executable program code for executing the image processing method of any of the above embodiments or the control method of any of the above embodiments. The electronic device may be any one of a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart bracelet, a smart helmet, and smart glasses.

The embodiment of the present invention further provides a computer readable storage medium having instructions stored therein, when the processor of the electronic device executes the instruction, the electronic device performs the image processing method according to any one of the above embodiments or The control method described in any of the above embodiments is performed.

In the description of the present specification, the terms "some embodiments", "an embodiment", "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", The description of "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the invention. In the present specification, the schematic representation of the above terms does not necessarily mean the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a particular logical function or process. And the scope of the preferred embodiments of the invention includes additional implementations, in which the functions may be performed in a substantially simultaneous manner or in an opposite order depending on the functions involved, in the order shown or discussed. It will be understood by those skilled in the art to which the embodiments of the present invention pertain.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer readable medium, Used in conjunction with, or in conjunction with, an instruction execution system, apparatus, or device (eg, a computer-based system, a system including a processor, or other system that can fetch instructions and execute instructions from an instruction execution system, apparatus, or device) Or use with equipment. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport a program for use in an instruction execution system, apparatus, or device, or in conjunction with the instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as it may be optically scanned, for example by paper or other medium, followed by editing, interpretation or, if appropriate, other suitable The method is processed to obtain the program electronically and then stored in computer memory.

It should be understood that portions of the invention may be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods may be implemented by software stored in a memory and executed by a suitable instruction execution system or Firmware to achieve. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

A person skilled in the art can understand that all or part of the steps carried in implementing the above implementation method can be completed by a program to instruct related hardware, and the program can be stored in a computer readable storage medium, and the program is executed. Including one or a combination of the steps of the method embodiments.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.

The above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like. Although the embodiments of the present invention have been shown and described, it is understood that the above-described embodiments are illustrative and are not to be construed as limiting the scope of the invention. The embodiments are subject to variations, modifications, substitutions and variations.

Claims

An image processing method, characterized in that the image processing method comprises the following steps:

Identifying whether there is an overexposed portion in the in-focus state in the in-focus image;

Bluning the in-focus image to obtain a blurred focus image;

Processing the virtual focus image identical to the focused image scene to obtain a material portion corresponding to the overexposed portion when the overexposed portion is present; and

A blurred overexposed portion corresponding to the overexposed portion of the blurred focus image is replaced with the material portion to obtain a merged image.
The image processing method according to claim 1, wherein the in-focus image is in a panoramic deep focus state.
The image processing method according to claim 1, wherein the step of identifying whether the overexposed portion in the in-focus state exists in the in-focus image comprises:

Determining, according to a histogram of the in-focus image, whether the overexposed pixel in the in-focus image is more than or equal to a first predetermined number, and the pixel value of the overexposed pixel is greater than or equal to a predetermined pixel value; and

It is determined that the overexposed portion exists when the overexposed pixel is more than or equal to the first predetermined number.
The image processing method according to claim 3, wherein the step of identifying whether the overexposed portion in the in-focus state exists in the in-focus image comprises:

When there is the overexposed portion, determining whether there is an adjacent overexposed pixel in the in-focus image, the number of the adjacent overexposed pixels being more than or equal to a second predetermined number; and

When the adjacent overexposed pixels are present, it is determined that the adjacent overexposed pixels belong to the overexposed portion.
The image processing method according to claim 3 or 4, wherein the predetermined pixel value is an upper limit of a range of pixel values of the expansive image.
The image processing method according to claim 1, wherein the step of blurring the in-focus image to obtain a blurred focus image is implemented by a Gaussian blur algorithm.
The image processing method according to claim 1, wherein the degree of blur of the material portion is substantially the same as the degree of blur of the blur exposed portion.
The image processing method according to claim 1, wherein the image processing method comprises:

Outputting the merged image when the overexposed portion is present; and

The blurred focus image is output when the overexposed portion is absent.
A control method for controlling an imaging device, wherein the control method comprises the following steps:

Controlling the imaging device to output a focused image and a virtual focus image of the same scene; and

The in-focus image and the virtual focus image are processed by the image processing method according to any one of claims 1-8.
The control method according to claim 9, wherein said step of controlling said imaging means to output a focused image and a virtual focus image of the same scene comprises the steps of:

Controlling the imaging device to focus to obtain the in-focus image;

Refocusing and processing the cached image output by the imaging device in different focusing states to obtain a corresponding degree of blur; and

The cached image having the same degree of blur as the blurred focus image is saved as the virtual focus image.
An image processing apparatus, characterized in that the image processing apparatus comprises:

An identification module, configured to identify whether an overexposed portion in an in-focus state exists in the in-focus image;

a blurring module, configured to blur the in-focus image to obtain a blurred focus image;

a processing module, configured to process a virtual focus image identical to the focused image scene to obtain a material portion corresponding to the overexposed portion when the overexposed portion is present; and

And a merging module, configured to replace the imaginary overexposed portion corresponding to the overexposed portion in the blurred focus image with the material portion to obtain a merged image.
The image processing device according to claim 11, wherein said in-focus image is in a panoramic deep focus state.
The image processing device according to claim 11, wherein the identification module comprises:

a first determining sub-module, configured to determine, according to a histogram of the in-focus image, whether the over-exposed pixel in the in-focus image is greater than or equal to a first predetermined number, and the pixel value of the over-exposed pixel is greater than or equal to a predetermined pixel value; with

The first determining submodule is configured to determine that the overexposed portion exists when the overexposed pixel is more than or equal to the first predetermined number.
The image processing device according to claim 13, wherein the identification module further comprises:

a second determining sub-module, configured to determine, in the presence of the over-exposed portion, whether there is an adjacent over-exposed pixel in the in-focus image, the number of the adjacent over-exposed pixels being more than or equal to the second The predetermined number; and

a second determining submodule, configured to determine that the adjacent overexposed pixels belong to the overexposed portion when the adjacent overexposed pixels are present.
The image processing device according to claim 13 or 14, wherein said predetermined pixel value is an upper limit of a range of pixel values of said expansive image.
The image processing apparatus according to claim 11, wherein the blurring module is configured to blur the in-focus image by using a Gaussian blurring algorithm to obtain a blurred focus image.
The image processing apparatus according to claim 11, wherein the degree of blur of the material portion is substantially the same as the degree of blur of the blur exposed portion.
The image processing device according to claim 11, wherein said image processing device comprises:

a first output module, configured to output the merged image when the overexposed portion is present; and

And a second output module, configured to output the blurred focus image when the overexposed portion is absent.
A control device for controlling an imaging device, wherein the control device comprises:

a control module, configured to control the imaging device to output a focused image and a virtual focus image of the same scene; and

An image processing apparatus according to any of claims 11-18 and electrically connected to said control module.
The control device of claim 19, wherein the control module comprises:

a focusing sub-module, configured to control the imaging device to focus to obtain the in-focus image;

a focusing sub-module for refocusing and processing a cached image output by the imaging device in different focusing states to obtain a corresponding degree of blur; and

And a saving submodule, configured to save the cached image having a degree of blurring substantially the same as the blurred focus image as the virtual focus image.
An imaging device, characterized in that the imaging device comprises:

camera;

a control module electrically connected to the camera, configured to control the camera to output a focused image and a virtual focus image of the same scene; and

An image processing apparatus according to any of claims 11-18 and electrically connected to said control module.
The image forming apparatus according to claim 21, wherein said control module comprises:

a focusing sub-module, configured to control the imaging device to focus to obtain the in-focus image;

a focusing sub-module for refocusing and processing a cached image output by the imaging device in different focusing states to obtain a corresponding degree of blur; and

And a saving submodule, configured to save the cached image having a degree of blurring substantially the same as the blurred focus image as the virtual focus image.
An electronic device, comprising the image forming device according to any one of claims 21-22.
The electronic device according to claim 23, wherein the electronic device comprises any one of a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart bracelet, a smart helmet, and smart glasses.
The electronic device of claim 23, wherein the imaging device comprises a front camera or/and a rear camera.
An electronic device includes a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside a space enclosed by the housing, and the processor and the memory are disposed at The power circuit is configured to supply power to each circuit or device of the electronic device; the memory is configured to store executable program code; and the processor reads the memory stored in the memory by reading Executing program code to execute a program corresponding to the executable program code for performing the image processing method according to any one of claims 1 to 8 or performing the method of any one of claims 9 to 10. Control method.
A computer readable storage medium having instructions stored therein, the electronic device performing the image processing method or performing according to any one of claims 1 to 8 when the processor of the electronic device executes the instruction The control method according to any one of claims 9 to 10.