JP2006072742A - Catch light composition method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique capable of improving an impression of an eye, and capable of enhancing a value of an image, by putting automatically catch light on the pupil in a natural condition. <P>SOLUTION: This catch light composition method for composing the catch light in the pupil of the image is constituted of a face selection step for selecting a face from the image; a face dividing step for dividing the face into a prescribed number of divided face areas; a feature amount calculation step for calculating a prescribed feature amount, based on a brightness value of each picture element included in the divided face area, in every of the divided face areas; a determination step for determining an incident direction of light into the face, based on the feature amount in the each divided face area; a pupil selection step for selecting the pupil from the face; and a correction step for setting a catch light area in the pupil, based on the incident direction of the light, for conducting correction to change brightness of the catch light area, and for outputting the pupil with the catch light. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a catchlight synthesis technique for synthesizing a catchlight with the pupil of an image.

  In an image such as a portrait, in which a person's face is shown, the influence of the eyes on the person's impression is generally large. For this reason, a technique has been proposed in which the eyes of an image are corrected to improve the impression of a person in the image. The technique for correcting the pupil is to select and correct the red-eye area that needs to be corrected based on the shape, density, position, etc. of the pupil, and when there is a catch light in the selected red-eye area, There is an image processing method for synthesizing a catch light with a pupil (see, for example, Patent Document 1).

JP 2000-76427 A

In general, when a catch light is in the pupil, the facial expression of the person in the image becomes vivid and the impression of the person is improved. However, in the image processing method of Patent Document 1, when a catchlight is included in the pupil of the original image, the catchlight can be left in the corrected image. If it is not, the catchlight cannot be turned on automatically.
For this reason, conventionally, the catchlight was manually synthesized into the pupil. However, when the catchlight was manually inserted into the pupil, the skill of the operator was required, and there was a problem that the correction took time. . In particular, it is difficult to determine in which part of the pupil the catchlight should be placed, and there are cases where appropriate correction cannot be performed.

  The present invention has been made in view of the above problems, and its purpose is to provide a technique for improving the impression of the eyes and enhancing the value of the image by automatically putting the catch light into the pupil in a natural state. The point is to provide.

  In order to achieve this object, the feature configuration of the catchlight synthesis method for synthesizing the catchlight with the pupil of the image according to the present invention includes a face selection step of selecting a face from the image, and a predetermined number of divided face regions. A face dividing step for dividing the divided face area, a feature amount calculating step for calculating a predetermined feature amount based on a luminance value of each pixel included in the divided face area, and the divided face area for each of the divided face areas A determination step for determining an incident direction of light on the face based on a feature amount, a pupil selection step for selecting a pupil from the face, and a catch light area for the pupil based on the incident direction of the light. And a correction step of performing correction for changing the luminance of the catchlight area and outputting a catchlight-equipped pupil.

The catchlight here is a catchlight area 50a and / or a catchlight spot 50b as shown in FIG. 6B. The catchlight area 50a is a portion where the pupil is bright as if the light wraps around the iris of the pupil, and has a certain spread. The catchlight spot 50b is a white portion that enters the pupil by illumination light, and has a uniform shape such as a circle. The pupil here refers to the black eye part of the eye.
Generally, the position where the catch light enters is determined by the incident direction of light, and the catch light area enters a direction symmetrical to the incident direction of light with respect to the center of the pupil. Therefore, by setting the catchlight area based on the incident direction of light as in the above feature configuration, the catchlight area can be placed in the natural position of the pupil, and the value of the image can be improved by improving the impression of the person. Can be increased. Further, according to the above feature configuration, the brightness of the divided face area in the light incident direction becomes brighter. Therefore, by calculating the feature amount based on the luminance value for each divided face area, the light incident direction is appropriately set. Can be determined.

To achieve this object, another feature configuration of the catchlight combining method according to the present invention is to detect a catchlight spot from the pupil selected in the pupil selection step based on a predetermined recognition condition. A detection step, and the correction step is to set the catchlight area around the detected catchlight spot when the catchlight spot is detected in the catchlight spot detection step. .
In other words, according to the above-described feature configuration, when the catchlight spot is already in the pupil, the catchlight area is set so as not to overlap the catchlight spot. Can be put in.

  The present invention also covers a program for causing a computer to execute the catchlight composition method described above and a medium on which the program is recorded. Such a catchlight composition program selects a face from the image. A face selection function for dividing the face into a predetermined number of divided face areas, and a predetermined feature amount for each divided face area based on a luminance value of each pixel included in the divided face area. A feature amount calculation function to be calculated; a determination function that determines the direction of light incident on the face based on the feature amount of each of the divided face regions; a pupil selection function that selects a pupil from the face; A correction function for setting a catch light area on the pupil based on the incident direction, performing a correction for changing the luminance of the catch light area, and outputting a pupil with a catch light. . Naturally, such a catchlight synthesis program can also obtain all the effects of the catchlight synthesis method described above. It is also preferable to add the function according to the dependent claim described in the catchlight synthesis method to the catchlight synthesis program.

In the present invention, a catchlight combining device that performs the above-described catchlight combining method is also subject to rights, and such a catchlight combining device includes a face selection unit that selects a face from the image, and the face. A face dividing unit that divides into a predetermined number of divided face regions; a feature amount calculating unit that calculates a predetermined feature amount for each of the divided face regions based on a luminance value of each pixel included in the divided face region; Based on the feature amount of each of the divided face regions, a determination unit that determines an incident direction of light on the face, a pupil selection unit that selects a pupil from the face, and an entrance direction of the light based on the incident direction of the light. A correction unit that sets a catchlight area, performs correction to change the luminance of the catchlight area, and outputs a pupil with catchlight. Naturally, such a catchlight synthesizing apparatus can also obtain all the effects of the catchlight synthesizing method described above. It is also preferable to add the configuration according to the dependent claim described in the catchlight composition method to the catchlight composition apparatus.
Other features and advantages of the present invention will become apparent from the following description of embodiments using the drawings.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of catchlight synthesis technology according to the present invention will be described with reference to the drawings.
FIG. 1 is an external view of a photographic printing apparatus according to the present invention. This photographic printing apparatus includes a printing station 1B as a photographic printer that performs exposure processing and development processing on photographic paper P, and a developed photographic film. 2A, an operation station 1A that processes a captured image taken from an image input medium such as a digital camera memory card 2b and generates / transfers print data used in the print station 1B.

  This photo printing apparatus is also called a digital minilab. As can be understood from FIG. 2, the printing station 1B pulls out the roll-shaped printing paper P stored in the two printing paper magazines 11 and prints it with the sheet cutter 12. The back print unit 13 prints print processing information such as color correction information and frame number on the back side of the photographic paper P, and the print exposure unit 14 cuts the print paper P into the size. The photographed image is exposed on the surface (photosensitive surface) of the photographic paper P, and the exposed photographic paper P is sent to a processing tank unit 15 having a plurality of development processing tanks for development processing. After drying, the photographic paper P, that is, the photographic prints P, sent to the sorter 17 from the transverse feed conveyor 16 at the upper part of the apparatus is accumulated in a plurality of trays of the sorter 17 in a state of being sorted in order units (see FIG. 1). ).

  A photographic paper transport mechanism 18 is laid to transport the photographic paper P at a transport speed in accordance with various processes for the photographic paper P described above. The photographic paper transport mechanism 18 is composed of a plurality of nipping and transporting roller pairs including a chucker type photographic paper transport unit 18a disposed before and after the print exposure unit 14 in the photographic paper transport direction.

  The print exposure unit 14 applies R (red), G (green), and B (blue) to the printing paper P conveyed in the sub-scanning direction based on print data from the operation station 1A along the main scanning direction. A line exposure head for irradiating laser beams of the three primary colors (1) is provided. The processing tank unit 15 includes a color developing tank 15a for storing a color developing processing liquid, a bleach-fixing tank 15b for storing a bleach-fixing processing liquid, and a stabilizing tank 15c for storing a stable processing liquid.

  At the upper position of the desk-like console of the operation station 1A, there is disposed a film scanner 20 capable of acquiring a photographed image from a photographed image frame of the photographic film 2a with a resolution exceeding 2000 dpi, and is attached to a digital camera or the like. A media reader 21 that acquires captured images from various semiconductor memories, CD-Rs, and the like used as the captured image recording medium 2b is incorporated in a general-purpose personal computer that functions as the controller 3 of the photographic printing apparatus. The general-purpose personal computer is also connected with a monitor 23 for displaying various information, and a keyboard 24 and a mouse 25 as operation input devices used as operation input units used for various settings and adjustments.

  The controller 3 of this photographic printing apparatus has a functional unit for performing various operations of the photographic printing apparatus using hardware and / or software, with the CPU as a core member, as shown in FIG. The function unit particularly related to the present invention includes an image input unit 31 that takes a photographed image read by the film scanner 20 or the media reader 21 and performs pre-processing necessary for the next processing, various windows, and various types. Creation of a graphic operation screen including operation buttons and the like and construction of a graphic user interface (hereinafter abbreviated as GUI) for generating a control command from a user operation input (via the keyboard 24, mouse 25, etc.) through such a graphic operation screen GUI unit 33 and control frames sent from GUI unit 33 A print management unit 32 that performs image processing on the captured image transferred from the image input unit 31 to the memory 30 in order to generate desired print data based on an operation command input directly from the keyboard 24 or the like; A video control unit 35 that generates a video signal for causing the monitor 23 to display graphic data sent from the GUI unit 33, or a simulated image as a print source image, an expected finished print image, or the like during pre-judge printing work such as correction. A print data generation unit 36 that generates print data suitable for the print exposure unit 14 installed in the print station 1B based on the processed captured image that has been subjected to image processing, and a raw capture according to the customer's request Write images and processed shots that have been processed to CD-R Etc. formatter 37 for formatting the like to form for.

  When the photographic image recording medium is the film 2a, the image input unit 31 separately sends the scan data for the pre-scan mode and the main scan mode to the memory 30, and performs preprocessing according to each purpose. When the captured image recording medium is the memory card 2b, when the captured image includes thumbnails (low resolution data), the captured image main data (high data) is used for the purpose of displaying a list on the monitor 23. The thumbnail image is sent to the memory 30 separately from the resolution data. If a thumbnail is not included, a reduced image is created from this data and sent to the memory 30 as a thumbnail.

  The print management unit 32, a print order processing unit 60 that manages the print size, the number of prints, and the like, an image processing unit 70 that performs various image processing on the photographed image developed in the memory 30, and a catch light in the pupil of the image Catch light combining means 80, and means for realizing a photo retouch function such as density correction and resolution conversion. The catchlight combining unit 80 is installed in the print management unit 32 as a program.

  As shown in FIG. 4, the catchlight combining unit 80 includes an input unit 81, a face selection unit 82, a face division unit 84, a feature amount calculation unit 85, a determination unit 86, a pupil selection unit 87, a correction unit 88, and an output unit 89. It has. In this embodiment, the face dividing unit 84, the feature amount calculating unit 85, and the determining unit 86 constitute a light incident direction determining unit 83 that determines the incident direction of light on the face. By combining the catch light based on the light incident direction on the face by the light incident direction determining means 83, the corrected pupil can be made natural, and the light incident direction can be automatically determined. Since it is calculated, labor and time required for catchlight synthesis can be saved.

  The face selection unit 82 is a processing unit that performs a face selection step of selecting a face from an image, and the face division unit 84 is a processing unit that performs a face division step of dividing a face into a predetermined number of divided face regions. The amount calculation unit 85 is a processing unit that performs a feature amount calculation step for calculating a predetermined feature amount based on the luminance value of each pixel included in the divided face region for each divided face region. A processing unit that performs a determination step of determining the incident direction of light on the face based on the feature amount of each face area, and a pupil selection unit 87 is a processing unit that performs a pupil selection step of selecting a pupil from the face, The correction unit 88 is a processing unit that performs a correction step of setting the catch light area 50a in the pupil based on the incident direction of light, changing the luminance of the catch light area 50a, and outputting the catch light pupil. is there.

  Next, the catch light composition processing procedure of the catch light composition means 80 configured as described above will be described with reference to FIGS. The catchlight 50 here is, as shown in FIG. 6B, a crescent-shaped catchlight area 50a and a circular catchlight spot 50b. Prior to synthesizing the catchlight 50 by the catchlight synthesizing unit 80, a photographed image, specifically digital image data of the photographed image, is captured through the film scanner 20 and the media reader 21 and developed in the memory 30. The digital image data here is stored in the memory 30 in a bitmap format for later processing.

  The input unit 81 reads an image for pupil correction from the memory 30 based on an input operation from a pointing device such as the keyboard 24 or the mouse 25 (step # 101). Further, the catchlight combining means 80 displays the read image on the monitor 23 when the input unit 81 reads the image.

  The face selection unit 82 receives an image from the input unit 81 and detects a face from this image (step # 102). Here, the face is detected based on an input operation from the keyboard 24 or the mouse 25. Note that since an average luminance value of skin pixels is obtained by a feature amount calculation unit 85 described later, the face detected here is a skin pixel, and eyes, mouths, and the like including white eyes and pupils are excluded. Specifically, for example, skin pixel candidates are detected in accordance with skin color pixel recognition conditions that define a range of luminance values of RGB three primary colors, the continuity of each detected skin pixel candidate is obtained, and skin that is continuously distributed over a predetermined range The pixel candidate group is certified as a skin pixel.

Subsequently, the catch light combining unit 80 obtains the incident direction of light on the face of the image.
Specifically, first, the face dividing unit 84 divides the face detected by the face selecting unit 82 into a predetermined number of divided face regions (step # 103). Here, since it is unlikely that the incident direction of light is the downward direction of the face, in the present embodiment, it is assumed that the incident direction of light is either the right side or the left side of the face, and the face is Are divided into two divided face regions. Note that the number of divided face regions is not limited to two, and an appropriate number is set in consideration of the size and resolution of the person in the image, the purpose of the image, and the like. Further, the divided face area may be set so that the incident direction of light can be determined, for example, by dividing the face in a radial manner with respect to the center of the face, and a part of the divided face area may be set to overlap.

  The feature amount calculation unit 85 calculates a feature amount for each divided face area divided by the face division unit 84 (step # 104). The feature amount here is an average luminance value of the skin pixels for each divided region. The average luminance value of each skin pixel of the left divided face area and the right divided face area divided by the face dividing unit 84 is calculated. Note that the feature amount is not limited to the average luminance value. It is also a preferred embodiment that the centroid of the skin pixel, the peak of the luminance value, the luminance distribution, etc. are set or added as parameters.

  The determination unit 86 determines the incident direction of light on the face based on the feature amount of each divided face area (step # 105). Specifically, the average luminance values of the left divided face region and the right divided face region are compared, and the side having the divided face region having a large average luminance value is determined as the light incident direction.

  The pupil selection unit 87 selects a pupil from the face (step # 106). Here, the user designates an area including the pupil on the image displayed on the monitor 23 by the keyboard 24 or the mouse 25. A black pixel group that detects black pixels in a specified area based on a predetermined black pixel recognition condition, determines the continuity of the black pixels, and is continuously distributed over a predetermined range, Those that can be considered to be approximately circular are recognized as pupils. This is a case where white pixels are included between the black pixel groups so that the catchlight area 50a can be inserted even when the catchlight spot 50b is already in the pupil of the image before correction. However, it is configured to detect as a pupil.

  Further, the pupil selection unit 87 detects the catch light spot 50b from the pupil (step # 107). Specifically, white pixels are detected from a region surrounded by a black pixel group certified as a pupil based on a predetermined white pixel certification condition, and white pixels continuously distributed over a predetermined range are certified as catchlight spots 50b. To do.

  When the catch light spot 50b is not detected by the pupil selection unit 87 (No branch at step # 108), the correction unit 88 sets the catch light area 50a based on the light incident direction (step # 109). . For example, when the light incident direction is the right side of the face, the catch light area 50a is arranged on the lower left side of the pupil, and when it is on the left side of the face, the catch light area 50a is arranged on the lower right side of the pupil. The catch light area 50a has a crescent shape in advance, and adjusts the size according to the size of the pupil. Further, the direction is set so that the direction of the center of gravity of the catchlight area 50a is opposite to the light incident direction. The shape of the catchlight area 50a is preferably selected from a predetermined shape prepared in advance, directly input, or selected from an external storage device such as a memory.

  When the catch light spot 50b is detected by the pupil selection unit 87 (Yes in step # 108), the correction unit 88 catches the catch light spot 50b so that the catch light area 50a does not overlap the catch light spot 50b. A catch light area 50a is set around 50b (step # 110). Specifically, for example, when the incident direction of light is the right side of the face, the catchlight area 50a is located on the lower left side of the catchlight spot 50b, and when the incident direction is the left side of the face, the catchlight area 50a is located on the lower right side of the catchlight spot 50b. Place. The shape, size and orientation of the catch light area 50a are adjusted in the same manner as in step # 109.

  Subsequently, the correction unit 88 changes the luminance of the set catchlight area 50a (step # 111). Here, in order to brighten the color information slightly remaining in the iris of the pupil, correction is performed to increase the luminance value of the pixels in the catchlight area 50a by 10%.

  The output unit 89 synthesizes the pupil with the catch light corrected in this way into an image and outputs it to the memory 30 (step # 112). Further, a corrected image obtained by synthesizing the catch-lit pupil is displayed on the monitor 23 so that the finish can be confirmed.

  FIG. 6 shows an image A of the pupil before the synthesis process and a corrected image B obtained by synthesizing the catch-lit pupil. Compared with the image A, the corrected image B has a better pupil impression, and the value of the image can be increased by using the catchlight combining technique according to the present invention.

  The catchlight synthesis technique according to the present invention can be applied not only to a person but also to an animal pupil. Furthermore, in the above-described embodiment, the present invention is applied to a photographic printing apparatus. In this case, the face selection unit 82 and the pupil selection unit 87 may be configured to use known means such as an ASP (Application Service Provider) via a network.

External view of photographic printing apparatus employing catchlight synthesis technology according to the present invention Schematic diagram showing the configuration of the print station of the photo printing device Functional block diagram explaining the functional elements built in the controller of the photo printing device Functional block diagram showing the configuration of the catch light combining means according to the present invention The flowchart which shows the process sequence of the catchlight synthetic technique which concerns on this invention The figure which shows the example of the image before and after implementation of the synthetic | combination process by the catchlight synthetic | combination technique which concerns on this invention

Explanation of symbols

50a Catchlight area 50b Catchlight spot 80 Catchlight combining means 82 Face selection section 83 Light incident direction determination means 84 Face segmentation section 85 Feature quantity calculation section 86 Determination section 87 Eye selection section 88 Correction section

Claims (4)

In the catch light synthesis method that synthesizes the catch light with the pupil of the image,
A face selection step of selecting a face from the image;
A face dividing step of dividing the face into a predetermined number of divided face regions;
A feature amount calculating step for calculating a predetermined feature amount based on a luminance value of each pixel included in the divided face region for each of the divided face regions;
A determination step of determining an incident direction of light on the face based on the feature amount of each of the divided face regions;
A pupil selection step of selecting a pupil from the face;
Based on the incident direction of the light, a catch light area is set in the pupil, a correction for changing the brightness of the catch light area is performed, and a correction step for outputting a catch light pupil is configured. Catch light synthesis method. From the pupil selected in the pupil selection step, it has a catch light spot detection step for detecting a catch light spot based on a predetermined certification condition,
The said correction step sets the said catchlight area around the said detected catchlight spot when the said catchlight spot is detected in the said catchlight spot detection step. The catchlight synthesis method described. In the catch light composition program that synthesizes the catch light to the pupil of the image,
A face selection function for selecting a face from the image;
A face dividing function for dividing the face into a predetermined number of divided face regions;
A feature amount calculation function for calculating a predetermined feature amount based on a luminance value of each pixel included in the divided face region for each of the divided face regions;
A determination function for determining a light incident direction on the face based on the feature amount of each of the divided face regions;
A pupil selection function for selecting a pupil from the face;
Based on the incident direction of the light, a catch light composition is set so that a catch light area is set in the pupil, the brightness of the catch light area is changed, and a correction function for outputting the catch light pupil is executed by a computer. program. In the catch light synthesizer that synthesizes the catch light with the pupil of the image
A face selection unit for selecting a face from the image;
A face dividing unit for dividing the face into a predetermined number of divided face regions;
A feature amount calculating unit that calculates a predetermined feature amount based on a luminance value of each pixel included in the divided face region for each of the divided face regions;
A determination unit that determines an incident direction of light on the face based on the feature amount of each of the divided face regions;
A pupil selection unit for selecting a pupil from the face;
A catchlight synthesizing apparatus comprising: a correction unit that sets a catchlight area in the pupil based on the incident direction of the light, performs correction for changing the luminance of the catchlight area, and outputs a pupil with the catchlight .

Images