JP6433177B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to image processing of contour emphasis processing for enhancing a contour for a subject in an image.

  In recent digital cameras, the evolution of optical elements and image sensors has expanded the range of subjects that can be taken, from shooting in a bright place to shooting in a dark place, and shooting from a close subject to a far subject. .

  However, in the above-described conventional digital camera, image processing is performed corresponding to the subject and the shooting scene, but depending on the type of the subject and the shooting scene, the image processing may not be appropriately performed. For example, among image processing corresponding to a subject and a shooting scene, with respect to the contour enhancement processing for clearly showing the subject, a method of changing the amount of enhancement of contour enhancement according to the color temperature and photographing sensitivity of the subject (Patent Document 1) A method (Patent Document 2) or the like for changing the amount of enhancement for contour enhancement according to the photographing mode has been proposed.

Japanese Patent Laid-Open No. 2004-247872 JP 2002-281348 A

  However, if the degree of enhancement (enhancement amount, intensity) of the contour enhancement is uniformly changed for any subject, for example, if the enhancement amount is increased to sharpen the stars in the night sky, The background subject that does not want to emphasize the outline is also emphasized.

An image processing apparatus according to the present invention is determined to be a night sky region by a subject determination unit that determines a type of a subject in the image, a processing unit that performs a contour enhancement process for enhancing a contour of the subject, and the subject determination unit. The contour enhancement process for strengthening the overshoot against the undershoot is performed on the other areas, and the contour enhancement process is not performed on the other areas not determined as the night sky area, or the overshoot amount and the undershoot are performed. Control means for controlling the processing means so as not to change the ratio of the amount of shoots .

  According to the present invention, it is possible to reduce adverse effects such as excessive edge enhancement and noise increase in an image due to edge enhancement.

It is a structure of the imaging device of 1st Embodiment. 1 is a configuration of an image processing circuit according to a first embodiment. It is the structure of an edge emphasis circuit among the image processing circuits of 1st Embodiment. It is a figure explaining the area | region determination of 1st Embodiment. It is an operation | movement flow of the scene determination of 2nd Embodiment.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, a digital camera will be described as an example of an image processing apparatus, but the image processing apparatus will be described. However, the present invention is not limited to this.

(First embodiment)
An image processing apparatus 100 according to the first embodiment will be described with reference to FIGS. 1 to 4.

  FIG. 1 shows a configuration of an image processing apparatus 100 according to the embodiment.

  The image processing apparatus 100 includes a photographing lens 10 including a focus lens, and a mechanical shutter 12 having a diaphragm function. The image processing apparatus 100 also converts an optical image incident from an optical system such as the photographing lens 10 and the shutter 12 into an electrical signal, and an analog signal output from the imaging element 14 is converted into a digital signal. An (analog / digital) converter 16 is further provided.

  The timing generation circuit 18 that supplies a clock signal and a control signal to the image sensor 14 and the A / D converter 16 is controlled by the memory control circuit 22 and the system control circuit 50. In addition to the mechanical shutter 12, the storage time can be controlled as an electronic shutter by controlling the reset timing of the image sensor 14 by the timing generation circuit 18 and can be used for moving image shooting and the like.

  The image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, an electronic zoom function can be realized by performing image clipping and scaling processing by the image processing circuit 20. Further, the image processing circuit 20 performs predetermined arithmetic processing using the captured image data.

  Based on the calculation result obtained by the image processing circuit 20, the system control circuit 50 performs AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash) on the exposure control means 40 and the distance measurement control means 42. Dimming) processing. Further, the image processing circuit 20 can also perform AWB (auto white balance) processing on the image data.

  Further, the image processing circuit 20 may include a subject detection circuit. As a subject detection method, a method such as pattern matching or feature point extraction can be applied.

  In the present embodiment, the image processing apparatus 20 has been described using an example of detecting the face of a human subject and the night sky of a landscape subject. However, the present invention is not limited to these types of subjects, and other types of subjects can be used. A subject may be detected. The image processing apparatus 20 can function as a face detection circuit that detects the face of a human subject, and the detection result includes the position (coordinates) of the face area, and more specifically, the position (coordinates) of the eyes. It can be output together with area information (coordinates).

  The memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the memory 30, and the compression / decompression circuit 32. The image data output from the A / D converter 16 is written into the memory 30 via the image processing circuit 20 and the memory control circuit 22 or directly from the A / D converter 16 via the memory control circuit 22.

  The image display unit 28 including a TFT, an LCD, etc. displays the display image data written in the memory 20 via the memory control circuit 22. An electronic viewfinder function can be realized by sequentially displaying captured image data using the image display unit 28. The image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50. When the display is turned off, the power consumption of the image processing apparatus 100 can be greatly reduced. I can do it.

  The memory 30 is a memory for storing captured still images and moving image image data, and has a sufficient storage capacity for storing a predetermined number of still images and moving image data for a predetermined time. . This makes it possible to write a large amount of images to the memory 30 at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 30 can also be used as a work area for the system control circuit 50.

  A program code to be executed by the system control circuit 50 is written in the non-volatile memory 31 constituted by a flash ROM or the like, and the system control circuit 50 sequentially reads and executes the stored program code. In addition, an area for storing system information and an area for storing user setting information are provided in the nonvolatile memory, and various information and settings are read and restored at the next startup.

  A compression / decompression circuit 32 that compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like reads the image data stored in the memory 30 and performs compression processing or decompression processing, and the processed image data is stored in the memory 30. Write to.

  The exposure control means 40 for controlling the shutter 12 having an aperture function also has a flash light control function in conjunction with the flash 48.

  Further, the image processing apparatus 100 includes a distance measurement control unit 42 that controls focusing of the photographing lens 10 and a zoom control unit 44 that controls zooming of the photographing lens 10.

  The flash 48 also has an AF auxiliary light projecting function and a flash light control function. The exposure control means 40 and the distance measurement control means 42 are controlled using the TTL method, and the system control circuit 50 uses the exposure control means based on the calculation result obtained by calculating the image data obtained by imaging by the image processing circuit 20. 40, the distance measurement control means 42 is controlled.

  The system control circuit 50 is a control circuit that controls the entire image processing apparatus 100.

  The operation means for inputting various operation instructions of the system control circuit 50 is configured by a single or a plurality of combinations such as a switch, a dial, a touch panel, pointing by visual line detection, and a voice recognition device. In the following, these operating means will be described in detail.

  The mode dial switch 60 can switch and set various function modes such as power-off, automatic shooting mode, shooting mode, HDR shooting mode, panoramic shooting mode, moving image shooting mode, playback mode, and PC connection mode.

  The shutter switch SW1 (62) is turned ON during the operation of the shutter button, and instructs to start operations such as AF (autofocus) processing, AE (automatic exposure) processing, and AWB (auto white balance) processing. The shutter switch SW2 (64) is turned on when the operation of the shutter button is completed.

  In the case of flash photography, after performing EF (flash pre-emission) processing, the image sensor 14 is exposed for the exposure time determined by the AE processing. In the case of flash photography, light is emitted during this exposure period, and light is shielded by the exposure control means 40 simultaneously with the end of the exposure period, thereby completing the exposure to the image sensor 14. In addition, it instructs the start of a series of processes up to a recording process for writing a signal read from the image sensor 14 to the recording medium 200. More specifically, a read process for writing image data to the memory 30 via the A / D converter 16 and the memory control circuit 22, a development process using computations in the image processing circuit 20 and the memory control circuit 22, and the memory 30. The image data is read out from the image data and compressed by the compression / decompression circuit 32.

  The display changeover switch 66 can change the display of the image display unit 28. With this function, when shooting using the optical viewfinder 104, it is possible to save power by cutting off the current supply to the image display unit such as an LCD.

  The operation unit 70 including various buttons, a touch panel, a rotary dial, and the like includes a menu button, a set button, a macro button, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, and the like. Menu move + (plus) button, menu move-(minus) button, playback image move + (plus) button, playback image-(minus) button, shooting quality selection button, exposure compensation button, date / time setting button, etc. is there.

  The zoom switch unit 72 serving as a zoom operation unit for a user to give an instruction to change the magnification of a captured image is also referred to as a zoom switch 72, and includes a tele switch that changes the imaging field angle to the telephoto side and a wide switch that changes the wide angle side. By using the zoom switch 72, the zoom control unit 44 is instructed to change the imaging field angle of the photographing lens 10 and becomes a trigger for performing an optical zoom operation. In addition, it also serves as a trigger for electronic zooming change of the imaging angle of view by image cropping by the image processing circuit 20 or pixel interpolation processing.

  The image processing apparatus 100 includes a power source 86 including a primary battery of an alkaline battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, an AC adapter, or the like.

  Further, an interface 90 with a recording medium such as a memory card or a hard disk, and a connector 92 for connecting to a recording medium such as a memory card or a hard disk are provided.

  Furthermore, the protection unit 102 is a barrier that prevents the imaging unit from being soiled or damaged by covering the imaging unit including the lens 10 of the image processing apparatus 100.

  The optical viewfinder 104 can take an image using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28.

  The communication unit 110 has various communication functions such as USB, IEEE1394, LAN, and wireless communication. Further, the communication unit 110 has a connector 112 or an antenna 112 in the case of wireless communication as an interface for connecting the image processing apparatus 100 to other devices.

  A recording medium 200 such as a memory card or a hard disk includes a recording unit 202 configured by a semiconductor memory, a magnetic disk, or the like, an interface 204 with the image processing apparatus 100, and a connector 206 for connecting to the image processing apparatus 100.

  Each configuration in the image processing apparatus 100 described above may be configured by a hardware module, or a part or all of the configuration may be configured as a software module.

  The configuration of the image processing circuit 20 of this embodiment will be described with reference to FIG.

  The image processing circuit 20 performs luminance signal generation S201, color signal generation S202, and edge enhancement S203 from the Bayer array signal of the image data input from the image sensor 14, and obtains a YUV signal. The YUV signal is subjected to post-processing S204 such as resizing, saturation change, and edge enhancement according to the shooting scene or the like. Also, by performing detection processing S205 such as subject detection such as face detection and scene detection, the image quality parameters in S201 to S204 are changed according to the scene and the like.

  The configuration of the edge enhancement circuit in the image processing circuit 20 of the present embodiment will be described with reference to FIG.

  FIG. 3A shows the configuration of the edge enhancement circuit. A low-pass filter (LPF) S301 is applied to the input image signal, and the difference between the original image and the LPF image is obtained by the subtraction circuit in S302. In this subtraction process, negative signals are not taken into account, thereby obtaining an overshoot that is a convex edge signal on the high luminance side. On the contrary, the subtracting circuit of S303 obtains a signal with inverted undershoot, which is an edge signal convex to the low luminance side, by taking the difference between the LPF image and the original image.

  Further, in the gain circuit (AMP) in S304, a gain is applied to the overshoot obtained in the subtracting circuit in S302 by the gain control circuit (CTRL) in S305. Similarly, in the gain circuit (AMP) in S306, the gain control circuit (CTRL) in S307 applies gain to the inverted signal of the undershoot obtained in the subtracting circuit in S303.

  In addition, the addition circuit in S308 adds the original image and overshoot with gain. The subtracting circuit of S309 subtracts the signal with the undershoot inverted from the original image and the image obtained by adding the overshoot, thereby adding the undershoot as a result.

  Further, the operation of acquiring overshoot and undershoot using the signal waveforms of FIGS. 3B to 3E will be described. FIG. 3B shows a part of the horizontal waveform of the original image, where the vertical axis represents the luminance level and the horizontal axis represents the horizontal coordinate of the image. FIG. 3C shows an LPF image obtained by performing the LPF process on the original image in S301, and has a waveform in which the high frequency component is eliminated from the original image. By subtracting the LPF image from the original image by taking the difference between these signals, the hatched portion S3111 in FIG. 3D is obtained as an overshoot, and conversely, subtracting the original image from the LPF image in FIG. A shaded portion S321 is obtained as an undershoot. Overshoot adjusts the bright side of the contour enhancement, and undershoot adjusts the dark side.

  Note that the low-pass filter used here is, for example, a 1: 2: 1 (/ 4) filter for the target pixel and adjacent pixels, but is not limited thereto. In the present embodiment, the difference between the original pixel and the LPF image is obtained. However, the present invention is not limited to this, and it is only necessary to obtain the difference between the image including the high frequency component and the image not including the high frequency.

  Region determination according to the present embodiment will be described with reference to FIG.

  FIG. 4A is a starry night view that is one of the assumed compositions of the present embodiment, and is composed of a dark distant view S401 such as a mountain, a night sky S402, and a star S403.

  In FIG. 4B, FIG. 4A is divided into a plurality of blocks, the average value of the luminance signal and the color signal is obtained for each block, and blocks having a small difference are combined and set as an area. Area S411 and dark blue area S412.

  In FIG. 4C, the night sky region S421 undershoots the overshoot corresponding to the subject by determining the subject from the average value of the luminance signal and the color signal for each region obtained in FIG. 4B. In contrast, the edge enhancement is applied to S422 other than the night sky region S421 as a normal process, or the edge enhancement without changing the ratio of overshoot and undershoot is performed.

  Further, FIG. 4D is a photograph of a person, which is one of the assumed compositions of the present embodiment, and is composed of a person S431, a distant view S432 such as a mountain, and a sky S433.

  In FIG. 4E, FIG. 4D is divided into a plurality of blocks, the average value of the luminance signal and the color signal is obtained for each block, and the area is set by combining the blocks with a small difference. The skin color area S442, the white area S443, the brown area S444, and the blue area S445 are determined. Furthermore, there is a face region S446 detected by the face detection circuit.

  FIG. 4F shows the face detection region S451 corresponding to the subject by determining the subject from the average value of the luminance signal and color signal for each region and the face detection region obtained in FIG. The included face area S452 is subjected to contour emphasis processing with the undershoot relatively strengthened with respect to overshoot, and S453 other than the face area is not subjected to edge emphasis as a normal process, or the ratio of overshoot to undershoot is not changed. Apply edge enhancement.

  As described above, when the control is performed by varying the ratio of overshoot and undershoot for each region in FIGS. 4C and 4F, the ratio is further increased based on the luminance level of the region. You may adjust. Specifically, the ratio of the undershoot amount is higher in the region where the luminance level is higher than the predetermined value so that the ratio of the overshoot amount is higher in the region where the luminance level is lower than the predetermined value. You may make it enlarge.

  In the present embodiment, the example in which subject determination is performed for the night sky and a person (face) is described, but the subject target is not limited to these. That is, the subject may include a high-luminance region in the low-luminance region instead of the night sky, or may be a subject in which the low-luminance region is included in the high-luminance region instead of a person (face). .

  As described above, in the present embodiment, according to the type of the subject in the image, the ratio of the overshoot amount and the undershoot amount in the contour emphasis process is controlled to perform the image emphasis image processing. Thus, it is possible to perform the contour enhancement. Therefore, it is possible to reduce harmful effects such as excessive edge enhancement and noise increase in the image due to edge enhancement.

(Second Embodiment)
In the first embodiment, the type of the subject is determined, and the ratio of the overshoot amount and the undershoot amount is controlled according to the subject. In the second embodiment, the shooting scene of the image is determined. The ratio is controlled according to the shooting scene.

  The description of the same configuration as that of the first embodiment is omitted, and the characteristic configuration of the present embodiment will be mainly described below.

  FIG. 5 shows an operation flow of scene determination according to the second embodiment. With reference to FIG. 5, the shooting scene determination operation of the present embodiment will be described.

  If a face is detected from the image in S501 and it is determined that there is a face, the illuminance is determined from the photometric result such as exposure setting in S502. For example, brightness is determined by comparing the illuminance of the photometric result with a predetermined value. If it is determined that the image is shot in a bright place, it is determined that the image is a person image (S503), and the undershoot is strengthened against the overshoot to enhance the outline. If it is determined in S502 that the image is taken in a dark place, it is determined that the image is a night scene portrait image (S504), and the edge-segmented edge enhancement is performed, the edge enhancement is not performed as a normal process, or overshoot and undershoot are performed. Apply edge enhancement without changing the ratio.

  In order to perform the above operation, the image processing unit 20 may function as a face detection unit and a brightness detection unit.

  If it is determined that there is no face in S501, the illuminance is determined from the photometric result such as exposure setting in S505. If it is determined that the shooting is performed in a bright place, it is determined that the shooting is a general shooting except for a person and a night view (S506), and edge enhancement is not performed as normal processing or edge enhancement that does not change the ratio of overshoot and undershoot is performed. . If it is determined in S505 that the shooting is in a dark place, it is determined that the shooting is night scene shooting (S507), and the overshoot is strengthened against the undershoot to enhance the outline.

  The present embodiment is characterized in that a shooting scene is determined from an image, and an edge emphasis image process is performed by controlling a ratio of an overshoot amount and an undershoot amount in the edge enhancement process according to the type of the shooting scene. .

  As a modification of the present embodiment, a shooting mode set by the user during shooting may be determined, and the ratio may be controlled according to the determined shooting mode. For example, if the night scene shooting mode is set at the time of shooting, the overshoot is strengthened against the undershoot and the contour is emphasized. If the portrait shooting mode is set, the undershoot is strengthened against the overshoot to enhance the contour.

  Note that the shooting mode determination may be performed by acquiring a shooting mode recorded together with an image in an image file, for example.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  In the above-described embodiment, the image processing device is built in the imaging device and performs image processing for edge enhancement on the image acquired by the imaging device. However, the image processing device is separated from the imaging device. The image may be configured to be acquired by communication from an external device.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus execute the program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 20 Image processing part 50 System control part 100 Image processing apparatus

Claims (4)

An acquisition means for acquiring an image;
Subject determination means for determining the type of subject in the image;
Processing means for performing contour enhancement processing for enhancing the contour of the subject;
The contour enhancement process for strengthening overshoot against the undershoot is performed on the area determined as the night sky area by the subject determination unit, and the outline enhancement process is performed on the other areas not determined as the night sky area. And a control means for controlling the processing means so as not to change the ratio of the overshoot amount and the undershoot amount. Dividing means for dividing the image into regions corresponding to the types of subjects;
The image processing apparatus according to claim 1, wherein the processing unit performs the edge enhancement processing for each region.   The image processing apparatus according to claim 1, further comprising a lens and an image sensor. A control method for an image processing apparatus, comprising:
Get an image,
Determine the type of subject in the image,
An image processing method for performing contour enhancement processing for enhancing the contour of the subject,
The contour enhancement processing is performed so as to enhance overshoot against undershoot with respect to the area determined as the night sky area by the determination of the type of the subject , and the other areas not determined as the night sky area are A control method characterized by performing control so as not to perform contour enhancement processing or to change the ratio of the overshoot amount and the undershoot amount.

Images