JP2014048767A - Image generating device, image generating method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a more appropriate portrait image in consideration of a shape of main component of a face.SOLUTION: An imaging device 100 includes: a feature information extracting part 6f which extracts feature information for each predetermined region on main components of a face in a source image acquired by an image acquiring part 6a; an outline specification part 6g which specifies a face configuration part image corresponding to the feature information for each predetermined region extracted by the feature information extracting part from a face configuration part image that shows a configuration part of a face and is recorded in an image recording section 5 in association with feature information for each predetermined region; and a portrait picture generation part 6h which generates a portrait image of the face using the face configuration part image specified by the outline specification part.

Description

  The present invention relates to an image generation apparatus, an image generation method, and a program.

2. Description of the Related Art Conventionally, there is known a portrait creation device that creates a portrait using feature points of facial parts such as eyes, nose, mouth, ears, and face contour (see, for example, Patent Document 1).
There is also known a game device that creates a character by combining part objects prepared in advance for each part (see, for example, Patent Document 2).

JP 2004-145625 A JP 2008-61896 A

  By the way, when specifying a component image corresponding to a main component such as a face outline, which is used to generate a portrait, even if a large number of component images are prepared in advance, the most similar depending on the shape of the component. There is a possibility that the constituent image is not accurately identified. As a result, there is a problem that an appropriate portrait image cannot be generated.

  Accordingly, an object of the present invention is to provide an image generation apparatus, an image generation method, and a program that can generate a more appropriate portrait image in consideration of the shape of the main constituent parts of the face.

In order to solve the above problems, an image generation apparatus according to the present invention provides:
A first acquisition unit that acquires an original image; an extraction unit that extracts feature information for each predetermined region with respect to main constituent parts of the face in the original image acquired by the first acquisition unit; A specification for identifying a face component image corresponding to the feature information for each predetermined region extracted by the extraction unit from a face component image representing a face component recorded in association with feature information for each region And a first generation means for generating a portrait image of the face using the face component image specified by the specifying means.

An image generation method according to the present invention includes:
An image generation method using an image generation apparatus, a process for acquiring an original image, a process for extracting feature information for each predetermined region for main constituent parts of a face in the acquired original image, and a recording unit Processing for identifying a face component image corresponding to the extracted feature information for each predetermined region from the face component image representing the face component recorded in association with the feature information for each predetermined region. And a process of generating a portrait image of the face using the identified face component image.

The program according to the present invention is
A first acquisition unit for acquiring an original image, an extraction unit for extracting feature information for each predetermined region of a main component of a face in the original image acquired by the first acquisition unit; A face component corresponding to the feature information for each predetermined region extracted by the extraction unit from the face component image representing the face component recorded in association with the feature information for each predetermined region on the recording unit It is characterized by functioning as a specifying means for specifying an image and a first generating means for generating a portrait image of the face using the face component image specified by the specifying means.

  According to the present invention, it is possible to generate a more appropriate portrait image in consideration of the shape of the main constituent parts of the face.

It is a block diagram which shows schematic structure of the imaging device of one Embodiment to which this invention is applied. It is a figure which shows typically an example of the face outline image which concerns on the portrait image generation process by the imaging device of FIG. 3 is a flowchart illustrating an example of an operation related to a portrait image generation process performed by the imaging apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to a feature extraction process in the portrait image generation process of FIG. 2. It is a figure which shows typically an example of the image which concerns on the portrait image generation process of FIG. It is a figure for demonstrating the feature extraction process in the portrait image generation process of FIG. It is a figure which shows typically an example of the image which concerns on the portrait image generation process of FIG.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment to which the present invention is applied.
As shown in FIG. 1, the imaging device 100 according to the present embodiment specifically includes an imaging unit 1, an imaging control unit 2, an image data generation unit 3, a memory 4, an image recording unit 5, and an image. A processing unit 6, a display control unit 7, a display unit 8, an operation input unit 9, and a central control unit 10 are provided.
The imaging unit 1, the imaging control unit 2, the image data generation unit 3, the memory 4, the image recording unit 5, the image processing unit 6, the display control unit 7, and the central control unit 10 are connected via a bus line 11. Yes.

The imaging unit 1 captures a predetermined subject and generates a frame image as an imaging unit.
Specifically, the imaging unit 1 includes a lens unit 1a, an electronic imaging unit 1b, and a lens driving unit 1c.

The lens unit 1a includes a plurality of lenses such as a zoom lens and a focus lens, for example.
The electronic imaging unit 1b includes, for example, an image sensor (imaging device) such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). Then, the electronic imaging unit 1b converts the optical image that has passed through the various lenses of the lens unit 1a into a two-dimensional image signal.
For example, although not shown, the lens driving unit 1c includes a zoom driving unit that moves the zoom lens in the optical axis direction, a focusing driving unit that moves the focus lens in the optical axis direction, and the like.
The imaging unit 1 may include a diaphragm (not shown) that adjusts the amount of light passing through the lens unit 1a in addition to the lens unit 1a, the electronic imaging unit 1b, and the lens driving unit 1c.

  The imaging control unit 2 controls the imaging of the subject by the imaging unit 1. That is, the imaging control unit 2 includes a timing generator, a driver, and the like, although not illustrated. Then, the imaging control unit 2 scans and drives the electronic imaging unit 1b with a timing generator and a driver, and converts the optical image that has passed through the lens unit 1a into a two-dimensional image signal for each predetermined period by the electronic imaging unit 1b. Then, the frame image is read out from the imaging region of the electronic imaging unit 1b for each screen and is output to the image data generation unit 3.

The imaging control unit 2 may adjust the in-focus position of the lens unit 1a by moving the electronic imaging unit 1b in the optical axis direction instead of the focus lens of the lens unit 1a.
Further, the imaging control unit 2 may perform adjustment control of conditions when imaging an object such as AF (automatic focusing process), AE (automatic exposure process), AWB (automatic white balance), and the like.

The image data generation unit 3 appropriately adjusts the gain for each RGB color component with respect to the analog value signal of the frame image transferred from the electronic imaging unit 1b, and then performs sample holding by a sample hold circuit (not shown). The digital signal is converted into digital data by a / D converter (not shown), color processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y and color difference signal Cb , Cr (YUV data).
The luminance signal Y and the color difference signals Cb and Cr output from the color process circuit are DMA-transferred to a memory 4 used as a buffer memory via a DMA controller (not shown).

  The memory 4 is composed of, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by the image processing unit 6 or the central control unit 10.

  The image recording unit 5 is configured by, for example, a non-volatile memory (flash memory) or the like, and is recorded by a coding unit (not shown) of the image processing unit 6 in a predetermined compression format (for example, JPEG format). Record image data for use.

The image recording unit (recording unit) 5 records image data of a predetermined number of face contour images P1 (see FIG. 2).
The image data of the face contour image P1 is, for example, an image that schematically represents the shape of the face contour W corresponding to the front face of a human, and is associated with the feature information of the face contour W.
Here, the image data of the face contour image P1 is obtained by, for example, performing a process (details will be described later) using an AAM (Active Appearance Model) on the face area F1 detected by the face detection process (described later). After erasing the facial components (for example, eyes, nose, mouth, eyebrows, etc.) and hairstyle that are inside the jaw contour, draw a line along the facial contour, and the skin portion inside the facial contour The method of forming by painting with a predetermined color. The line drawing along the contour of the face may be performed manually based on a predetermined operation of the operation input unit 9 by the user, or automatically performed under the control of the CPU of the central control unit 10. Also good.
The feature information of the face contour W includes, for example, a direction code corresponding to a predetermined angle range (for example, every 45 °) specified based on the coordinates of each pixel constituting the face contour W ( Details will be described later).
Each of the image data of the face contour image P1 includes, for example, an ear image of the image that is the generation source of the face contour image P1, and is further associated with the hairstyle image P7 of the image of the generation source. May be.

  The image recording unit 5 may be configured, for example, such that a recording medium (not shown) is detachable, and controls reading of data from the loaded recording medium and writing of data to the recording medium.

The image processing unit 6 includes an image acquisition unit 6a, a face detection unit 6b, a configuration unit image generation unit 6c, a binary image generation unit 6d, a coordinate acquisition unit 6e, a feature information extraction unit 6f, and an outline specifying unit. 6g and a portrait image generating unit 6h.
Note that each unit of the image processing unit 6 is configured by, for example, a predetermined logic circuit, but the configuration is an example and is not limited thereto.

The image acquisition unit 6a acquires an image to be processed in the portrait image generation process.
That is, the image acquisition unit (acquisition means) 6a acquires image data of the original image (for example, a photographic image) P2. Specifically, the image acquisition unit 6a stores a copy of the image data (RGB data or YUV data) of the original image P2 generated by the image data generation unit 3 by imaging the subject by the imaging unit 1 and the imaging control unit 2. 4 or a copy of the image data of the original image P2 recorded in the image recording unit 5 (see FIG. 5A).
Each process by the image processing unit 6 to be described later may be performed on the image data itself of the original image P2, or a predetermined size (reduced by a predetermined ratio) of the image data of the original image P2 as necessary. For example, it may be performed on reduced image data of VGA size or the like.

The face detection unit 6b detects a face area F1 (see FIG. 5A) from the original image P2 to be processed.
That is, the face detection unit 6b detects a face area F1 including a face from the original image P2 acquired by the image acquisition unit 6a. Specifically, the face detection unit 6b acquires the image data of the original image P2 acquired as the processing target image of the portrait image generation process by the image acquisition unit 6a, and performs predetermined face detection on the image data. Processing is performed to detect the face area F1. Then, the face detection unit 6b cuts out a region A (see FIG. 5A) having a predetermined size surrounding the face region F1 to obtain a face region image.
Since the face detection process is a known technique, detailed description thereof is omitted here.

The component image generation unit 6c generates a face component image P4 (see FIG. 5C) representing the main face component of the face.
That is, the component image generation unit 6c generates a face component image P4 related to the main face component of the face in the original image P2 (see FIG. 5A) acquired by the image acquisition unit 6a. Specifically, the component image generation unit 6c performs a fine part extraction process on the face area image including the face area F1 of the original image P2, for example, eyes, nose, mouth, eyebrows, hair, face, etc. A fine face portion image P3 (see FIG. 5B) in which a face component such as the contour W is represented by a line is generated. For example, the constituent part image generation unit 6c generates the face fine part image P3 by the process using AAM as the fine part extraction process. Further, the component image generation unit 6c performs a fine portion extraction process on the face region F1 detected from the image data of the original image P2 by the face detection unit 6b.
Here, AAM is a technique for modeling a visual event, and is a process for modeling an image of an arbitrary face region F1. For example, the component image generation unit 6c obtains statistical analysis results of positions and pixel values (for example, luminance values) of predetermined feature parts (for example, eyes, nasal head, face line, etc.) in a plurality of sample face images. Register in the registration means. Then, the component image generation unit 6c sets a shape model that represents the shape of the face and a texture model that represents “Appearance” in the average shape with reference to the position of the feature part, and uses these models. An image of the face area F1 (face area image) is modeled. As a result, the component image generation unit 6c extracts the main components in the original image P2 and generates a fine face image P3 represented by lines.

Further, the component image generation unit 6c identifies the face contour W in the face region F1 by the fine portion extraction process, and the face component existing in the face contour W and the face component in contact with the face contour W with lines. The represented face component image P4 (see FIG. 5C) is generated.
Specifically, the component image generation unit 6c identifies a pixel in contact with the face contour W in the face fine portion image P3, and erases pixels existing outside the face contour W among pixels continuous with the pixel. To do. That is, the component image generation unit 6c erases a portion existing outside the face contour W in the face fine portion image P3, exists inside the face contour W, and is in contact with the face contour W. For example, a face component image P4 including part images of main face components other than the face contour W such as eyes, nose, mouth, and eyebrows is generated while leaving the portion.
Here, the component image generation unit 6c may extract and acquire information related to the relative positional relationship of the part images of each face component in the XY plane space and information related to the coordinate position.

In addition, although the process using AAM was illustrated as a fine part extraction process, it is an example and it is not restricted to this, It can change arbitrarily arbitrarily.
For example, as the fine part extraction process, an edge extraction process or an anisotropic diffusion process may be performed to generate the face constituent part image P4 including the part image of the face constituent part. Specifically, the component image generation unit 6c performs, for example, a differentiation operation on the image data of the original image P2 using a predetermined differential filter (for example, a high-pass filter) to obtain a luminance value, a color, or a density. Edge detection processing may be performed in which a sharp change is detected as an edge. In addition, the component image generation unit 6c uses a predetermined anisotropic diffusion filter for the image data of the original image P2, for example, to change the weight between the tangential direction of the linear edge and the vertical direction of the edge. An anisotropic diffusion process for smoothing may be performed.

The binary image generation unit 6d generates a binary image P5 related to the face outline W.
That is, the binary image generation unit (second generation unit) 6d generates a binary image P5 (see FIG. 6A) related to the face outline W in the original image P2 acquired by the image acquisition unit 6a. Specifically, the binary image generation unit 6d acquires a copy of the face fine part image P3 generated by the component part image generation unit 6c, and exists outside the face outline W of the face fine part image P3. After erasing the pixels, the minimum and maximum values in the x-axis direction (horizontal direction) and the y-axis direction (vertical direction) are calculated. The area surrounding the face contour W is defined by the minimum value and the maximum value.
Then, the binary image generation unit 6d normalizes the face fine portion image P3 so as to have a predetermined size (for example, the number of pixels in the x-axis direction and the y-axis direction is 256 pixels). Subsequently, the binary image generation unit 6d connects the portions corresponding to the left and right temples of the face outline W, and then sets the pixels inside the face outline W to pixel values corresponding to one color (for example, white Each pixel value of RGB is “255”), and a binary image (luminance image) P5 in which the outer pixel is a pixel value corresponding to another color (for example, black; each pixel value of RGB is “0”). Image data is generated. The generated image data of the binary image P5 is temporarily stored in a predetermined storage unit (for example, a memory) after the black and white pixel values are inverted.
Note that the method of generating the binary image P5 is an example and is not limited to this, and can be arbitrarily changed as appropriate.

The coordinate acquisition unit 6e acquires the coordinates of each pixel of the face outline W.
That is, the coordinate acquisition unit (second acquisition unit) 6e acquires the coordinates of each pixel of the face contour W based on the binary image P5 generated by the binary image generation unit 6d. Specifically, the coordinate acquisition unit 6e acquires the image data of the binary image P5 generated by the binary image generation unit 6d and reversed in black and white from a predetermined storage unit, and the white pixels constituting the face contour W Using a predetermined position of the boundary with the black pixel as a starting point (for example, a part corresponding to the temple on the right side in FIG. 6A), each adjacent pixel is traced until returning to the starting point, and all pixels are Get the coordinates. The acquired coordinates of each pixel of the face contour W are temporarily stored in a predetermined storage means (for example, a memory).
Note that the method of acquiring the coordinates of each pixel is an example and is not limited to this, and can be arbitrarily changed as appropriate.

The feature information extraction unit 6f extracts feature information from the face outline of the original image P2.
In other words, the feature information extraction unit (extraction means) 6f obtains feature information for each predetermined region with respect to main constituent parts (for example, the face outline W) of the face in the original image P2 acquired by the image acquisition unit 6a. Extract (see FIG. 6C). Specifically, the feature information extraction unit 6f acquires the coordinates of each pixel of the face outline W acquired by the coordinate acquisition unit 6e from a predetermined storage unit, and for each pixel based on the coordinates of each pixel. A direction code corresponding to a predetermined angle range is extracted as feature information. At this time, the feature information extraction unit 6f may extract a direction code corresponding to a pixel other than a pixel connecting the left and right temples among the pixels of the face outline W as the feature information.
For example, the feature information extraction unit 6f converts the transition of the movement of the coordinates of each pixel of the face outline W into an 8-direction code (see FIG. 6B) every 45 °. At this time, since each pixel of the straight line portion connecting the portions corresponding to the left and right temples is not related to the face contour W, the feature information extraction unit 6f does not calculate the direction code of the pixel or the direction Delete code after calculation. For example, the feature information extraction unit 6f deletes the direction codes represented by “1” to “3” corresponding to the pixels of the straight line portion connecting the portions corresponding to the left and right temples. After that, the feature information extraction unit 6f thins out the direction code (for example, the direction code represented by “0”, “4” to “7”) extracted as the feature information using, for example, a nearest neighbor. Is normalized to a predetermined number (for example, 64 points) and extracted as a feature amount (feature information).
Here, for example, among the direction codes normalized to 64 points, the feature information extraction unit 6f considers the pixel shift at the time of comparison and collation for the direction codes corresponding to the start point and end point pixels. The direction code of 66 points as a whole may be obtained by duplicating by shifting pixels.
The normalized direction code is temporarily stored in a predetermined storage means (for example, a memory).

  Note that the feature extraction processing for calculating the direction code is a known technique, and thus detailed description thereof is omitted here. Further, the method of calculating the direction code is an example, and is not limited to this, and can be arbitrarily changed as appropriate.

The contour specifying unit 6g specifies the face contour image P1 corresponding to the face contour W in the original image P2.
That is, the contour specifying unit (specifying unit) 6g represents a face contour image P1 (face configuration) representing the face contour W recorded in the image recording unit 5 in association with feature information (for example, a direction code) for each predetermined region. A facial contour image P1 (face component image) corresponding to the feature information for each predetermined area extracted by the feature information extraction unit 6f. Specifically, the contour specifying unit 6g specifies the face contour image P1 in the image recording unit 5 using the direction code corresponding to each pixel of the face contour W normalized by the feature information extracting unit 6f.
For example, the contour specifying unit 6g compares the direction code corresponding to each of the plurality of face contour images P1, ... recorded in the image recording unit 5 with the direction code extracted by the feature information extracting unit 6f, The amount of deviation is calculated. As the amount of deviation between the direction codes, for example, for all the pixels, the total sum of the squares of the distances between the corresponding pixels can be cited. Then, the contour specifying unit 6g uses the shift amount corresponding to each face contour image P1 recorded in the image recording unit 5 as a reference, in order from a smaller shift amount to a predetermined rank (for example, 7th place). The face contour image P1 is specified.
Here, the contour specifying unit 6g may automatically specify the one face contour image P1 (see FIG. 7A) having the highest degree of coincidence, or among the face contour images P1 up to a predetermined order. Thus, the user-desired face contour image P1 may be specified based on a predetermined operation of the operation input unit 9 by the user.

The portrait image generating unit 6h generates a portrait image P6 based on the face outline image P1.
That is, the portrait image generation unit (first generation unit) 6h generates the portrait image P6 using the image data of the face outline image P1 specified by the outline specifying unit 6g. Specifically, the caricature image generation unit 6h specifies the position where the part images of the respective facial component parts such as eyes, nose, mouth, and eyebrows are superimposed on the inner side of the face outline W of the face outline image P1. Then, the part image of each face constituting unit is superimposed on the position. Further, the portrait image generation unit 6h acquires a predetermined hairstyle image P7, specifies a position where the hairstyle image P7 is superimposed outside the face outline W, and then places the hairstyle image P7 at the specified position. Then, image data of a portrait image P6 representing the original image P2 as a portrait is generated (see FIG. 7B).
As the hairstyle image P7, the hairstyle image P7 designated based on a predetermined operation of the operation input unit 9 by the user may be acquired, or the hairstyle image P7 associated with the face contour image P1 is automatically obtained. May be acquired. Further, the image data of the hairstyle image P7 is preferably one in which the size of the hairstyle image P7 is adjusted in advance according to the size of the face outline image P1.
In addition, the portrait image generation unit 6h may generate an image represented by adding a predetermined color to a predetermined portion of the portrait image P6 (for example, a face component such as eyes, mouth, and eyebrows).

The display control unit 7 performs control to read display image data temporarily stored in the memory 4 and display it on the display unit 8.
Specifically, the display control unit 7 includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. The digital video encoder reads the luminance signal Y and the color difference signals Cb and Cr read from the memory 4 and stored in the VRAM (not shown) under the control of the central control unit 10 through the VRAM controller. Are periodically read out, and a video signal is generated based on these data and output to the display unit 8.

  The display unit 8 is, for example, a liquid crystal display panel, and displays an image captured by the imaging unit 1 based on a video signal from the display control unit 7 on a display screen. Specifically, the display unit 8 sequentially updates a plurality of frame images generated by imaging the subject by the imaging unit 1 and the imaging control unit 2 at a predetermined frame rate in the still image capturing mode and the moving image capturing mode. While displaying the live view image. The display unit 8 displays an image (rec view image) recorded as a still image or an image being recorded as a moving image.

  The operation input unit 9 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 9 includes a shutter button related to an imaging instruction of a subject, a selection determination button related to an instruction to select an imaging mode and a function, a zoom button related to an instruction to adjust the zoom amount, and the like (all not shown). And outputs a predetermined operation signal to the central control unit 10 in accordance with the operation of each button of the operation unit.

  The central control unit 10 controls each unit of the imaging device 100. Specifically, although not shown, the central control unit 10 includes a CPU (Central Processing Unit) and the like, and performs various control operations according to various processing programs (not shown) for the imaging apparatus 100.

Next, a portrait image generation process performed by the imaging apparatus 100 will be described with reference to FIGS.
FIG. 3 is a flowchart illustrating an example of an operation related to the portrait image generation process.

The portrait image generation processing is performed when the portrait image generation mode is instructed to be selected from a plurality of operation modes displayed on the menu screen based on a predetermined operation of the selection determination button of the operation input unit 9 by the user. This process is executed by each unit of the imaging apparatus 100, particularly the image processing unit 6, under the control of the unit 10.
In addition, it is assumed that the image data of the original image P2 and the image data of the face contour image P1 used to generate the portrait image P6 that are the processing target of the portrait image generation process are recorded in the image recording unit 5 in advance.

  As shown in FIG. 3, first, the image recording unit 5 selects the original image P2 designated based on a predetermined operation of the operation input unit 9 by the user in the recorded image data (see FIG. 5A). ) And the image acquisition unit 6a of the image processing unit 6 acquires the read image data as a processing target of the portrait image generation process (step S1).

Next, the face detection unit 6b performs a predetermined face detection process on the image data of the original image P2 acquired by the image acquisition unit 6a as a processing target to detect the face region F1 (step S2).
Subsequently, the component image generation unit 6c performs a fine part extraction process (for example, a process using AAM) on the face area image including the detected face area F1, for example, eyes, nose, mouth, and the like. Then, a fine face portion image P3 (see FIG. 5B) in which the face constituting portions such as the eyebrows, the hair, and the face contour W are represented by lines is generated (step S3). Then, the component image generation unit 6c identifies the face contour W in the face region F1 by the fine portion extraction process, and the face component existing in the face contour and the face component in contact with the contour, that is, for example, A face component image P4 including part images of main face components other than the face contour W such as eyes, nose, mouth, and eyebrows is generated (step S4; see FIG. 5C).

Next, the image processing unit 6 executes feature extraction processing (see FIG. 4) (step S5).
The feature extraction process will be described in detail below with reference to FIG. FIG. 4 is a flowchart illustrating an example of an operation related to the feature extraction process.

  As shown in FIG. 4, the binary image generation unit 6d acquires a copy of the image data of the face fine part image P3 and erases pixels existing outside the face outline W of the face fine part image P3. Then, normalization is performed so that the number of pixels in the x-axis direction and the y-axis direction of the face fine portion image P3 is 256 pixels (step S11). Subsequently, the binary image generation unit 6d connects the portions corresponding to the left and right temples of the face contour W, then sets the pixels inside the face contour W as pixel values corresponding to white, and sets the outer pixels as pixels. Image data of a binary image P5 having a pixel value corresponding to black is generated (step S12).

Next, the coordinate acquisition unit 6e acquires the image data of the binary image P5 generated by the binary image generation unit 6d and reversed in black and white, tracks each pixel of the binary image P5, and coordinates each pixel. Is acquired (step S13). For example, the coordinate acquisition unit 6e corresponds to one temple (for example, the temple on the right side in FIG. 6) at the boundary between the white pixel and the black pixel constituting the face contour W of the binary image P5 that is reversed in black and white. The coordinates of all the pixels are obtained by tracking each adjacent pixel until the portion returns to the start point, starting from the portion.
Subsequently, the feature information extraction unit 6f acquires the coordinates of each pixel of the face outline W acquired by the coordinate acquisition unit 6e, and extracts the direction code based on the coordinates of each pixel (step S14). For example, based on the coordinates of each pixel of the face outline W, the feature information extraction unit 6f converts the movement of the coordinate of each pixel into an 8-direction code (see FIG. 6B) every 45 °.
Then, the feature information extraction unit 6f is represented by direction codes (for example, “1” to “3”) corresponding to the pixels of the straight line portion connecting the portions corresponding to the left and right temples among the pixels of the face outline W. Are deleted, and the remaining direction codes (for example, direction codes represented by “0”, “4” to “7”) are reduced to a predetermined number (for example, 64 points) by a predetermined thinning process. It is normalized and extracted as a feature amount (feature information) (step S15). At this time, for example, among the direction codes normalized to 64 points, the feature information extraction unit 6f duplicates the direction codes corresponding to the start point and end point pixels to obtain a total of 66 point direction codes.
Thereby, the feature extraction process is terminated.

  Returning to FIG. 3, the contour specifying unit 6g uses the direction code corresponding to each pixel of the face contour W extracted by the feature information extracting unit 6f to use the face contour image P1 corresponding to the face contour W in the original image P2. Is specified in the image recording unit 5 (step S6). For example, the contour specifying unit 6g acquires the direction code of the face contour W corresponding to each of the plurality of face contour images P1,... From the image recording unit 5, and the original image P2 extracted by the feature information extracting unit 6f. After obtaining the direction code of the face contour W, these are compared to calculate the shift amount between corresponding pixels. Then, the contour specifying unit 6g automatically specifies the face contour image P1 (see FIG. 7A) having the smallest calculated shift amount. At this time, the specifying unit g2 rearranges the face contour images P1 in order from the smallest deviation amount to a predetermined order (for example, seventh), and then inputs an operation input by the user in the face contour images P1. The face contour image P1 desired by the user may be specified based on a predetermined operation of the unit 9.

Thereafter, the portrait image generation unit 6h generates a portrait image P6 (see FIG. 7B) using the face outline image P1, the face component image P4, and the hairstyle image P7 (step S7). Specifically, the caricature image generation unit 6h is arranged on the inner side of the face contour W of the face contour image P1 specified by the contour specifying unit 6g, for example, each face constituent unit such as eyes, nose, mouth, and eyebrows. The position where the part image is superimposed is specified, and the part image of each face constituting unit is superimposed at the position. Furthermore, the portrait image generation unit 6h acquires, for example, a hairstyle image P7 associated with the face contour image P1, places the hairstyle image P7 at a predetermined position outside the face contour W, and renders the original image P2 as a portrait. The image data of the portrait image P6 expressed by
This completes the portrait image generation process.

As described above, according to the imaging apparatus 100 of the present embodiment, the face component image (for example, the face) representing the face component recorded in the image recording unit 5 in association with the feature information for each predetermined area. Since the face component image corresponding to the feature information for each predetermined region is specified from the face contour image P1 representing the contour W and the like and the facial portrait image P6 is generated using the face component image, the original image Considering the feature of the shape of each predetermined region of the main constituent part of the face of the image P2, the face constituent part image corresponding to the constituent part of the face can be specified, and a more appropriate portrait image P6 is generated Can do.
That is, even if a large number of face component images are prepared in advance, there is a possibility that the most similar face component image cannot be accurately identified depending on the shape of the main component of the face such as the face contour W. However, in the present embodiment, a face component image (for example, face contour image P1 or the like) that uniquely represents the face component is specified in consideration of the shape characteristics of each predetermined region of the face component. By using the face component image, a more appropriate portrait image P6 can be generated.

Specifically, the coordinates of each pixel of the face contour W are acquired based on the binary image P5 related to the face contour W in the original image P2, and each pixel is determined based on the acquired coordinates of each pixel. Since a direction code corresponding to a predetermined angle range is extracted as feature information every time, for example, the face contour image P1 is specified using an ellipse that defines the approximate shape of the face contour W associated with the face contour W. Compared with the case where it does, specification of the face outline image P1 which characteristically represented the shape for every predetermined area | region of the face outline W can be performed more accurately.
At this time, the direction code corresponding to the pixels other than the pixels connecting the left and right temples among the pixels of the face outline W is extracted as the feature information, so that the face outline W, that is, the left and right temples are passed through the jaws. A direction code can be extracted for each pixel having a substantially “U” shape. Further, by using the direction code extracted and normalized as the feature information, the direction code associated with the face outline image P1 recorded in the image recording unit 5 and the face outline W of the original image P2 are extracted. The comparison with the direction code thus performed can be performed easily and appropriately, and the face contour image P1 can be specified more accurately.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in the above-described embodiment, the direction code corresponding to a predetermined angle range is extracted for each pixel of the face contour W as the feature information. However, this is an example, and the present invention is not limited to this. Any feature information that can be extracted for each predetermined region can be arbitrarily changed.

  In the above embodiment, the feature extraction process is performed after the generation of the face component image P4. However, the order of these processes is only an example, and the order is not limited to this. That is, the generation process of the face component image P4 may be performed after the feature extraction process.

  Furthermore, in the above-described embodiment, the image that is the generation source of the face contour image P1 and the face component image P4 does not have to be an image representing the front face. For example, the image is slanted around a predetermined axis. In the case of the original image P2 in which the face is inclined so as to face, the face outline image P1 may be prepared according to the angle of the inclination, or the face of the original image P2 faces the front. A deformed image may be generated and used.

  In the above embodiment, the image recording unit 5 that records the face contour image P1 is provided. However, the present invention is not limited to this. For example, a predetermined connection that can be connected to the apparatus main body via a predetermined communication network. The face contour image P1 is recorded in the server, and a predetermined server is accessed via a communication network from a communication processing unit (not shown), and the image acquisition unit 6a acquires the face contour image P1 from the server. It is good also as a structure.

  Furthermore, the configuration of the imaging apparatus 100 is merely an example illustrated in the above embodiment, and is not limited thereto. Further, although the imaging apparatus 100 is illustrated as an image generation apparatus, the present invention is not limited to this, and any configuration may be used as long as the image generation process according to the present invention can be performed.

In addition, in the above-described embodiment, the functions of the first acquisition unit, the extraction unit, the specifying unit, and the first generation unit are controlled under the control of the central control unit 10, and the image acquisition unit 6a, the feature information extraction unit 6f, the contour specifying unit 6g, and the portrait image generating unit 6h are configured to be driven. However, the configuration is not limited to this, and is realized by executing a predetermined program or the like by the central control unit 10. It is good also as a structure.
That is, a program memory (not shown) that stores a program stores a program including an acquisition processing routine, an extraction processing routine, a specific processing routine, and a generation processing routine. Then, the CPU of the central control unit 10 may function as means for acquiring the original image P2 by the acquisition processing routine. Further, the CPU of the central control unit 10 may function as a means for extracting feature information for each predetermined region with respect to the main constituent parts of the face in the acquired original image P2 by the extraction processing routine. In addition, the CPU of the central control unit 10 by the extraction processing routine makes each predetermined area extracted from the face component image P4 representing the face component recorded in the recording means in association with the feature information for each predetermined area. You may make it function as a means to specify the face structure part image P4 corresponding to the feature information. Further, the CPU of the central control unit 10 may function as a means for generating a facial portrait image P6 using the specified face component image P4 by the extraction processing routine.

  Similarly, the second generation unit and the second acquisition unit may be realized by executing a predetermined program or the like by the CPU of the central control unit 10.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
First acquisition means for acquiring an original image;
Extraction means for extracting feature information for each predetermined region with respect to main constituent parts of the face in the original image acquired by the first acquisition means;
A face component corresponding to the feature information for each predetermined region extracted by the extraction unit from the face component image representing the face component recorded in association with the feature information for each predetermined region on the recording unit An identification means for identifying an image;
First generating means for generating a portrait image of the face using the face component image specified by the specifying means;
An image generation apparatus comprising:
<Claim 2>
The main components of the face include the outline of the face,
Second generation means for generating a binary image related to the contour of the face in the original image acquired by the first acquisition means;
Second acquisition means for acquiring the coordinates of each pixel of the contour of the face based on the binary image generated by the second generation means;
The extraction means extracts, as the feature information, a direction code corresponding to a predetermined angle range for each pixel based on the coordinates of each pixel acquired by the second acquisition means. Item 2. The image generation device according to Item 1.
<Claim 3>
The extraction means further extracts a direction code corresponding to a pixel other than a pixel connecting the left and right temples among the pixels of the facial contour as the feature information. Image generation device.
<Claim 4>
The extraction means further normalizes the direction code extracted as the feature information,
The image generation apparatus according to claim 3, wherein the specifying unit further specifies the face component image using the direction code normalized by the extracting unit.
<Claim 5>
An image generation method using an image generation apparatus,
Processing to acquire the original image;
A process of extracting feature information for each predetermined area for the main constituent parts of the face in the acquired original image;
A face component image corresponding to the extracted feature information for each predetermined region is identified from the face component image representing the face component recorded in association with the feature information for each predetermined region on the recording means Processing,
A process of generating a portrait image of the face using the identified face component image;
An image generation method comprising:
<Claim 6>
The computer of the image generation device
First acquisition means for acquiring an original image;
Extraction means for extracting feature information for each predetermined region with respect to main constituent parts of the face in the original image acquired by the first acquisition means;
A face component corresponding to the feature information for each predetermined region extracted by the extraction unit from the face component image representing the face component recorded in association with the feature information for each predetermined region on the recording unit Identification means for identifying an image,
First generating means for generating a portrait image of the face using the face component image specified by the specifying means;
A program characterized by functioning as

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Imaging part 5 Image recording part 6 Image processing part 6a Image acquisition part 6b Face detection part 6c Structure part Image generation part 6d Binary image generation part 6e Coordinate acquisition part 6f Feature information extraction part 6g Contour specification part 6h Caricature image Generation unit 10 Central control unit

Claims (6)

First acquisition means for acquiring an original image;
Extraction means for extracting feature information for each predetermined region with respect to main constituent parts of the face in the original image acquired by the first acquisition means;
A face component corresponding to the feature information for each predetermined region extracted by the extraction unit from the face component image representing the face component recorded in association with the feature information for each predetermined region on the recording unit An identification means for identifying an image;
First generating means for generating a portrait image of the face using the face component image specified by the specifying means;
An image generation apparatus comprising: The main components of the face include the outline of the face,
Second generation means for generating a binary image related to the contour of the face in the original image acquired by the first acquisition means;
Second acquisition means for acquiring the coordinates of each pixel of the contour of the face based on the binary image generated by the second generation means;
The extraction means extracts, as the feature information, a direction code corresponding to a predetermined angle range for each pixel based on the coordinates of each pixel acquired by the second acquisition means. Item 2. The image generation device according to Item 1.   The extraction means further extracts a direction code corresponding to a pixel other than a pixel connecting the left and right temples among the pixels of the facial contour as the feature information. Image generation device. The extraction means further normalizes the direction code extracted as the feature information,
The image generation apparatus according to claim 3, wherein the specifying unit further specifies the face component image using the direction code normalized by the extracting unit. An image generation method using an image generation apparatus,
Processing to acquire the original image;
A process of extracting feature information for each predetermined area for the main constituent parts of the face in the acquired original image;
A face component image corresponding to the extracted feature information for each predetermined region is identified from the face component image representing the face component recorded in association with the feature information for each predetermined region on the recording means Processing,
A process of generating a portrait image of the face using the identified face component image;
An image generation method comprising: The computer of the image generation device
First acquisition means for acquiring an original image;
Extraction means for extracting feature information for each predetermined region with respect to main constituent parts of the face in the original image acquired by the first acquisition means;
A face component corresponding to the feature information for each predetermined region extracted by the extraction unit from the face component image representing the face component recorded in association with the feature information for each predetermined region on the recording unit Identification means for identifying an image,
First generating means for generating a portrait image of the face using the face component image specified by the specifying means;
A program characterized by functioning as