JP5650012B2 - Facial image processing method, beauty counseling method, and facial image processing apparatus - Google Patents

Description

  The present invention relates to a face image processing method, a beauty counseling method, and a face image processing apparatus.

  Warping and morphing have been proposed as image processing methods for changing face images in various ways. Warping is a technique in which an image is transformed geometrically to change some parts, and morphing is a technique in which an intermediate image between an original image and another image is generated and the original image is gradually changed. is there.

  In Patent Document 1, in evaluating the skin color distribution in the face image of the subject, the subject's face is transformed into a standard face such as an average face (warping), and the influence of facial shape characteristics is excluded. A skin color evaluation method for principal component analysis of the hue of a face is described. In this skin color evaluation method, a face image is divided into several tens of regions, an average value of hues is obtained for each region, and the tendency of the distribution of hues is evaluated based on statistical processing by principal component analysis.

  Also, morphing that continuously changes an original face image into another person's face image is often used for creating video content (see Patent Document 2).

  Non-Patent Document 1 improves the attractiveness of the face by moving feature points in the face image of the subject by statistically processing the face image of the subject using a face image database including a large number of sample images. A warping technique is described. In this method, the beauty score estimated in advance using the sensory evaluation and Support Vector Regression (SVR) algorithm is higher than the face image database, and it is close to the tendency of the facial feature point position of the subject. Sample images are ranked and a plurality of images are extracted from the top, and this is used as a population. According to Non-Patent Document 1, the attractiveness of the subject's face is improved by moving the facial feature point of the subject to the average facial feature point of the population.

JP 2009-169758 A JP 2007-300562 A

Tommer Leyvand, Daniel Cohen-Or, Gideon Dror and Dani Lischinski, `` Data-Driven Enhancement of Facial Attractiveness '', Proceedings of ACM SIGGRAPH 2008, vol.27, no.3, 2008

  An image processing technique that virtually changes the appearance of a face (hereinafter referred to as “face impression”) provides various industrial uses. As an example, the effects of cosmetics and beauty treatments can be quantitatively evaluated and used as a tool for designing and developing products and services. In addition, it is expected to be used as a research tool in the interdisciplinary field of makeup and cosmetic surgery, and also as a technical tool for the realization of new entertainment with the theme of “face”.

  However, the above-mentioned conventional methods simply transform the subject's face image toward the target face or face shape, such as the average face of the population or the face of another person, so research on the determinants of face impression It did not provide technology for deepening the face or changing the facial impression freely.

  The present invention has been made in view of the above-described problems, and provides a face image processing technique capable of obtaining a determinant of face impression, and a beauty counseling method using the face image processing technique. is there.

  According to the present invention, at least a first change axis and a second change axis selected (including overlapping selection) from the shape, texture, or color of the face with respect to a face image obtained by capturing at least a part of the face, respectively. A principal component analysis is performed to calculate a weight coefficient of a plurality of basis vectors, the first change image in which the weight coefficient applied to the first change axis is changed, and the weight coefficient applied to the second change axis are changed. A face image processing method is provided, wherein the second change image is generated and the first change image and the second change image are output in comparison with each other.

  Further, according to the present invention, there is provided a beauty counseling method using the face image processing method described above, and a change in the shape, texture or color of the face due to the application of cosmetics or the application of cosmetic treatment, and the first change image or A beauty counseling method is provided, wherein the second change image is output in association with the second change image.

  Further, according to the present invention, at least a first change axis and a second change axis selected (including overlapping selection) from the shape, texture, or color of the face with respect to a face image obtained by capturing at least a part of the face. Principal component analysis means for calculating a weight coefficient of a plurality of basis vectors by performing principal component analysis for each, a first change image in which the weight coefficient applied to the first change axis is changed, and the second change axis A change image generation means for generating a second change image in which the weighting coefficient is changed, and an image output means for outputting the first change image and the second change image in comparison with each other. An image processing apparatus is provided.

Here, the face shape includes the face outline and the position, size, and range of characteristic parts such as eyes, nose, and mouth, as well as the presence and shape of a wearing object such as glasses. Also, the texture of the face image refers to an image pattern that represents the texture of the face surface. The face color means at least one of the average brightness, saturation, or hue of the skin in the entire face or a partial area of the face.
The change axis means the directionality of changing the shape, texture, or color of the face (hereinafter collectively referred to as “determinant”) in the face image (original image). The change axis is determined by selection of a determinant and a population used for principal component analysis of face images.

  According to the above invention, the images in which the weighting coefficients (eigenvalues) of the basis vectors are individually changed for the plurality of change axes selected by the determinant are compared. Thereby, the degree of influence of the selected change axis on the face impression can be individually visualized. For this reason, when changing a test subject's facial impression as desired, for example by cosmetics or cosmetic treatment, the determinant can be appropriately selected.

  ADVANTAGE OF THE INVENTION According to this invention, the face image processing technique which can acquire the determinant of face impression, and the beauty counseling method using this face image processing technique are provided.

It is a flowchart of the face image processing method concerning a first embodiment. It is a block diagram of the face image processing device of the first embodiment. It is a figure which shows the example of a face shape table. It is a figure which shows the state which compares and outputs a 1st change image, a 2nd change image, and a superimposition change image. It is a figure which shows the example of a texture table. It is a figure which shows the example of a color table. It is a figure which shows the example of a makeup | decoration texture table. It is a block diagram of the face image processing apparatus of 3rd embodiment. It is a flowchart which shows the detail of the change image production | generation process which produces | generates a 1st change image in 3rd embodiment. It is a flowchart which shows the detail of the change image production | generation process which produces | generates a 2nd change image in 3rd embodiment. It is a flowchart which shows the detail of the modification of the change image production | generation process which produces | generates a 1st change image in 3rd embodiment. It is a figure which shows the example of a beauty means selection table.

<First embodiment>
First, the outline of the face image processing method according to the first embodiment of the present invention will be described.
FIG. 1 is a flowchart of a face image processing method (hereinafter also referred to as the present method) of the present embodiment. FIG. 2 is a block diagram of the face image processing apparatus 100 of the present embodiment that implements the method.

  The method includes a face image acquisition step S10, a principal component analysis step S40, a change image generation step S50, and an output step S70.

  The face image processing apparatus 100 according to the present embodiment is constructed by connecting various devices to a computer apparatus in which a computer program is installed as necessary. The various constituent elements of the face image processing apparatus 100 only have to be formed so as to realize their functions. For example, dedicated hardware that exhibits a predetermined function, data provided with a predetermined function by a computer program It can be realized as a processing device, a predetermined function realized in the data processing device by a computer program, an arbitrary combination thereof, or the like.

  The various components of the face image processing apparatus 100 do not have to be independent of each other, but a plurality of components are formed as a single element, and a single component is formed of a plurality of elements. That a certain component is a part of another component, a part of a certain component overlaps a part of another component, and the like.

  The face image processing apparatus 100 of the present embodiment includes an imaging unit 10, a principal component analysis unit 40, a change image generation unit 50, and an output unit 70. These are connected to each other by a bus 95.

In the face image acquisition step S10, the imaging unit 10 acquires a face image (original image OIM) obtained by capturing at least a part of a human face. The face image may be an image of the face of the user who operates the face image processing apparatus 100, or may be an image of the face of another person viewed from the user including a celebrity.
In the present embodiment, a digital camera is illustrated as the imaging unit 10. The imaging unit 10 acquires face image data obtained by capturing the face as digital data with the user of the face image processing apparatus 100 as a subject. The subject may be a natural face or a face with makeup (makeup face). Hereinafter, in this method, a case where a face image of a subject's face is used as face image data will be exemplified.

  Instead of the present embodiment, the imaging unit 10 may be a receiving device that acquires face image data related to an arbitrary person including a celebrity via a storage medium or a network.

In the principal component analysis step S40, the principal component analysis unit 40 selects at least the first change axis AX ₁ and the second change axis selected from the face shape, texture, or color (including overlapping selection) with respect to the original image OIM. AX ₂ is subjected to principal component analysis _, and weight coefficients b _{1, i} and b _{2, j of} multi-order basis vectors e _{1, i} and e _{2, j} (where i and j are natural numbers representing degrees, respectively). Is calculated.
Hereinafter, the basis vectors e _{1, i} and e _{2, j} may be collectively referred to as a basis vector e. Similarly, the weighting factors b _{1, i} and b _{2, j} may be collectively expressed as a weighting factor b.

As shown in FIG. 1, in this method, a plurality of change axes can be set by conditional branching in step S60. Then, the weighting factor b for each base vector e applied to the change axis set in each change axis setting step S30 is calculated in the principal component analysis step S40.
In other words, in the first principal component analysis step S40, the first change determined by the determinant (for example, face shape) selected in the change axis setting step S30 and the population generated in the population generation step S34. A principal component analysis is performed on the original image OIM according to the axis AX ₁ to calculate a weight coefficient b _{1, i} for each base vector e _{1, i} . In the change image generation step S50, the first change image IM ₁ is generated by reconstructing the image by changing the weight coefficient b _{1, i} .

If it is selected to further change the image based on another change axis (step S60: YES), a determinant is selected in the second change axis setting step S30. The determinant selected here may be the same as that for the first time (for example, face shape). In the population generation step S34, a population for principal component analysis is generated. As a result, the second change axis AX ₂ is set. When the determinant for the second time is the same as that for the first time, at least a part of the sample image SIM constituting the population is different from the first time population. When the determinants for the first time and the second time are different, the sample images SIM constituting the population may be completely the same. As a result, the weight coefficients b _{1 and i} obtained in the first and second principal component analyzes are different from each other.
In the second principal component analysis step S40, principal component analysis is performed on the original image OIM in accordance with the set second change axis AX ₂ to calculate weighting coefficients b _{2, j} relating to the plural-order basis vectors e _{2, j.} .

In the change image generation step S50, the change image generation unit 50 (see FIG. 2) uses the first change image IM ₁ in which the weighting factor b _{1, i} applied to the first change axis AX ₁ is changed, and the second change a second change image IM ₂ changing the weighting coefficients b _{2, j} according to the axis AX _2, to produce a. Here, changing the weight coefficients b _{1, i} , b _{2, j} means increasing or decreasing part or all of the plurality of weight coefficients. Decreasing the weight coefficients b _{1, i} and b _{2, j} means reducing the absolute value of the weight coefficient, and includes making the weight coefficient zero. In the change image generation step S50, the first change image IM ₁ reconstructed by changing the weight coefficient b _{1, i,} and the second change image IM ₂ reconstructed by changing the weight coefficient b _{2, j} Is generated.

In the output step S70, the output unit 70 outputs the first change image IM ₁ and the second change image IM ₂ in comparison with each other. Here, outputting a plurality of images in contrast to each other means outputting the plurality of images simultaneously or individually by the same or different image output means. Specifically, a mode in which a plurality of images are displayed in close proximity to the same display device, a mode in which images are alternately displayed on a same display device or a display device arranged adjacent thereto, a plurality of images As an example, a mode of printing and printing on the same or different paper can be given.

  Hereinafter, the method and the face image processing apparatus 100 of the present embodiment will be described more specifically.

  As illustrated in FIG. 2, the face image processing apparatus 100 includes a normalization unit 12 that performs a normalization process on the face image data of the original image OIM to generate a standardized image. The normalization unit 12 acquires about tens of feature points such as eyes, nose, mouth, and ear from the original image OIM, and normalizes the face shape (FIG. 1: image normalization step S20).

  In the image normalization step S20, the acquired original image OIM is subjected to normalization processing such as normalization, and the shape characteristics of the subject's face (hair contours of hair, cheeks or jaws, eyebrows, eyes, nose, etc.) Etc.) is generated. The normalization process can be performed using a commercially available face image synthesis system (for example, International Telecommunications Research Institute, Futon).

The face image processing apparatus 100 includes a face image database 90 that stores a plurality of sample images SIM, a condition input unit 20 that receives input of conditions for extracting the sample images SIM, and the sample image SIM from the face image database 90 according to the received conditions. And an extraction unit 60 for generating a population by extracting.
The principal component analysis unit 40 calculates a multi-order basis vector e from the population, and calculates a weighting factor b applied to the calculated basis vector from the face image (original image OIM).

  Note that the normalization process of the original image OIM is also performed on the sample image SIM stored in the face image database 90. That is, the positions of the eyes, nose and mouth of the plurality of sample images SIM stored in the face image database 90 are standardized. The subject original image OIM and the sample image SIM are standardized in the same manner.

  In this method, a change axis setting step S30 for setting a determinant for changing the original image OIM is performed. In this method, after setting a determinant, conditions for changing the determinant are input (condition input step S32), and an appropriate sample image SIM is extracted from the face image database 90 to generate a population (mother). Group generation step S34). Here, the conditions for changing the determinant (face shape) can be variously set as will be described later. As an example, “the average of people who were sensorially evaluated as attractive faces among Japanese women” "Close to the face".

In this way, in the first round of change axis setting step S30, it will be described a case of selecting a face shape (face shape) as a determinant according to the first variation axis AX ₁ an example. However instead of this method, it may be selected texture and color as the first change axis AX _1.

  In the condition input step S32, attribute data for extracting the population of the sample image SIM used for the principal component analysis is input using the condition input unit 20. Such input may be performed by the subject or may be performed by an operator of the face image processing apparatus 100. However, a part of the attribute data (for example, attribute data f3 relating to the attractiveness shown in FIG. 3) may be prefixed to the face image processing apparatus 100 so that it is always selected.

In the population generation step S34, the extraction unit 60 generates a population by extracting some sample images SIM from the face image database 90 including a plurality of sample images SIM. In the principal component analysis step S40, the principal component analysis unit 40 calculates a multi-order basis vector e from the population, and calculates the weighting factor b _{1, i} applied to the calculated basis vector e from the original image OIM.

In the principal component analysis step S40, a determinant (face shape, texture, or color) of the original image OIM (hereinafter simply referred to as the original image OIM) of the subject obtained as the normalized image obtained above is common to many sample images. This is a calculation step for decomposing the basis vector (eigenface) into a linear sum.
The basis vectors are obtained by eigenvector analysis of a covariance matrix obtained by raster scanning each of a large number of sample images SIM standardized in advance, and are arranged in descending order of eigenvalues.

  Each pixel value of the original image OIM is decomposed as shown in the following equations (1) and (2) by principal component analysis (PCA), where n is the highest order of the base vector e. The first average face image and the second average face image are face images obtained by averaging the sample image SIM of the population with respect to each selected determinant. By using a large number of sample images SIM as a population for obtaining the basis vector e, any standardized face image (original image OIM) can be decomposed into the following equation with a predetermined accuracy. In addition, the original image OIM can be reconstructed using the average face image, the base vector e, and the weight coefficient b.

(Equation 1)
Original image OIM = first average face image + b _1,1 * first basis vector e _1,1 + b _1,2 * second basis vector e _1,2 + b _1,3 * third basis vector e _1,3 +. ... + B _{1, k} * kth basis vector e _{1, k} +... + B _{1, n-1} * (n-1) basis vector e _{1, n-1} + b _{1, n} * nth basis vector e _{1, n} (1)

(Equation 2)
Original image OIM = second average face image + b _2,1 * first basis vector e _2,1 + b _2,2 * second basis vector e _2,2 + b _2,3 * third basis vector e _2,3 +. .. + b _{2, k} * kth basis vector e _{2, k} +... + B _{2, n-1} * (n-1) basis vector e _{2, n-1} + b _{2, n} * nth basis vector e _{2, n} (2)

  Note that the face shape, texture, and color, which are determinants, can be handled independently of each other. When the principal component analysis is performed on the population of the sample image SIM with respect to any determinant, the respective basis vectors are orthogonal to each other, and the basis vectors in which the characteristics of other determinants are discarded are calculated. .

The basis vectors e _{1, i} and e _{2, j} and the weight coefficients b _{1, i} and b _{2, j} relating to the original image OIM are stored in the basis storage unit 92 in correspondence with each order.
Note that it is preferable to perform a principal component analysis in advance on all populations that may be extracted from the face image database 90 to calculate a basis vector e and store the basis vector e in the basis storage unit 92. Thereby, it is not necessary to obtain the basis vector e from the population each time in the principal component analysis step S40, and the weighting coefficient b _{1, i} relating to the original image OIM can be quickly calculated.

  FIG. 3 is an example of a face shape table showing conditions for changing the face shape (decision factor). The face shape table is a table in which a large number of sample images SIM stored in the face image database 90 are classified according to attributes that are considered to contribute to facial shape characteristics.

  The sample image SIM is provided with a plurality of attribute data indicating the attributes of the sample provider corresponding to the subject. By searching the face image database 90 by selecting one or a plurality of such attribute data, a population of sample images SIM having the attribute is generated.

In the change image generation step S50, the face image (original image OIM) is warped toward the average face shape obtained by averaging the face shapes of the sample images SIM belonging to the population extracted from the face image database 90, and the first change is performed. An image IM ₁ is generated. Here, the average face shape of the sample image SIM is obtained by averaging the positions of feature points such as eyes, nose, and mouth in a plurality of standardized sample images SIM.

Specifically, the weighting coefficients of higher order than the predetermined threshold order k of the first weighting factor b ₁ according to the change axis AX _{1, i} in the above formula _{(1) b 1, m (} m = k + 1, k + 2 , K + 3,...), And warping processing is performed. The threshold order k can be set based on the degree of change in face shape. Details will be described in the third embodiment.

More specifically, the position of the feature point of the original image OIM is moved on the higher order side in the original image OIM. Then, coordinates of a minute element having a feature point as a vertex are transformed by an image processing method such as a piece-wise affine method or a B-Spline approximation method. As a result, an image after the warping process in which the feature points of the original image OIM are moved (that is, the first change image IM ₁ ) is generated.

Note that the piecewise affine method is an image processing method in which a triangular element having a feature point as a vertex is coordinate-transformed by affine transformation. By using this method, when the processing speed is fast, and generates a superimposed change image IM _SP to be described later, it is possible to change the texture extracted in each partial area of the face.
The B-spline approximation method is an image processing method in which square elements having feature points as lattice points are subjected to coordinate conversion, and is characterized by performing coordinate conversion after giving a weight to each lattice point. By using this method, smooth warping processing from the original image OIM to the first change image IM ₁ is possible.

Returning to FIG. 1, in this method, YES is selected in the first conditional branch of step S60, and the determinant concerning the _second change axis AX2 is set (change axis setting step S30). Hereinafter, a case where a facial texture is selected as a determinant related to the second change axis AX ₂ will be described as an example.

Also for the second change axis AX ₂ , a condition for setting the directionality for changing the determinant is input (condition input step S32), and an appropriate sample image SIM is extracted from the face image database 90 to generate the mother. A group is generated (population generation step S34). Then, the base vector e2 _{, j} is obtained in advance from the texture of the sample image SIM of the population, and the original image OIM is principally analyzed to calculate the weighting factor b2 _{, j} from the original image OIM.

And in this way, it is assumed to generate a sample image SIM morphing the original image OIM towards the average face texture texture averaged face treatment to the second variation image IM ₂ belonging to the population. Here, the average face texture of the sample image SIM is an average of skin textures in a plurality of standardized sample images SIM.

In this method, the face image (original image OIM) is subjected to principal component analysis for each of multiple colors, and the weight coefficient b _{2, j of the} primary or multi-dimensional basis vector e _{2, j} applied to the second change axis AX _2. Are calculated respectively. Thus, it is possible to change the texture while maintaining the color of the original image OIM, a difference in texture of the original image OIM and the second change image IM ₂ can be easily visually confirmed.

Specifically, higher weight factor than the predetermined threshold degree n ₂ of the second change-axis weighting factor b ₂ according to AX _{2, j} in the above formula _{(2) b 2, k (} k = n 2 +1, n ₂ +2, n ₂ +3,.

  In the second conditional branch in step S60, either YES or NO may be selected. The original image OIM may be changed according to the third change axis to generate a third change image (step S60: YES), or the process may proceed to a change image output step S70 (step S60: NO).

In the output step S70, the output unit 70 outputs the first change image IM ₁ and the second change image IM ₂ in comparison with each other. FIG. 4 is a diagram illustrating an example of the first change image IM ₁ and the second change image IM ₂ displayed and output by the output unit 70.

In the figure, the vertical axis indicates the degree of change in face shape, and the horizontal axis indicates the degree of change in texture. Specifically, the lower right image is the original image OIM. Its three images arranged in the upper portion is a first change image IM _1, it is arranged downward in proximity to the original image OIM sequentially change degree from the original image OIM is small. Details of the degree of change will be described later. Of the first change image IM ₁ shown in the upper right of the figure, the image having the largest degree of change from the original image OIM is the average face shape image IM _1av . The average face shape image IM _1av is _obtained by using only the face shape as the average face shape of the population sample image SIM while retaining the texture and color of the original image OIM.

On the other hand, the lowermost three images arranged on the left side of the lower right original image OIM in the same figure are the second change image IM ₂ , and are adjacent to the original image OIM in ascending order of change from the original image OIM. It is arranged in the direction. Of the second change image IM ₂ shown in the lower left of the figure, the image having the largest degree of change from the original image OIM is the average face texture image IM _2av . The average face texture image IM _2av is _obtained by using only the texture as the average face texture of the population sample image SIM while retaining the face shape and color of the original image OIM.

The first change image IM ₁ of the present method is an image obtained by changing only the weighting factor b _{1, i} applied to the first change axis AX ₁ with respect to the original image OIM. It also includes an image in which the weight coefficient b _{2, j} applied to the second change axis AX ₂ is further slightly changed. The same applies to the second change image IM ₂ , in addition to an image in which only the weighting factor b _{2, j} applied to the second change axis AX ₂ is changed with respect to the original image OIM, It includes an image in which the weighting coefficient b _{1, i} applied to one change axis AX ₁ is further slightly changed.
That is, the images IM ₁₁ and IM ₂₁ (see FIG. 4) having a large change degree with respect to one of the first change axis AX _{1 and} the second change axis AX _{2 and} a sufficiently small change degree with respect to the other. , Included in the first change image IM ₁ and the second change image IM ₂ , respectively. In the example shown in FIG. 4, four first change images IM ₁ and second change images IM ₂ are shown on the upper right side and the lower left side.

Further, in the output step S70 of the present method, the superposition change in which the weight coefficient b _{1, i} applied to the first change axis AX ₁ and the weight coefficient b _{2, j} applied to the second change axis AX ₂ are respectively changed in multiple stages. An image IM _SP is generated. Then, in the output step S70, further outputs a first change image IM ₁ or the second change image IM ₂ and superimposed change image IM _SP versus.

Here, superimposed change image IM _SP is an image obtained by changing the equivalent of about both weighting coefficients b _{1, i} and b _{2, j} with respect to the original image OIM, are shown seven on the upper left side in FIG. 4. The upper left face image in FIG. 4 is an average face IM _av having an average face shape and an average face texture.

Here, in the conventional warping process, a changed image is generated by changing only the shape while maintaining the texture and color of the original image. In the conventional morphing process, a change image is generated by changing all of the face shape, texture, and color of the original image. A white arrow MRP shown in FIG. 4 indicates a locus of morphing processing in which both the face shape and the texture are changed from the original image OIM toward the average face IM _av .
On the other hand, the first change image IM ₁ and the second change image IM _{2 in} which the original image OIM is changed according to different change axes as in the present method are output in comparison with each other. The influence can be confirmed visually. For this reason, unlike the conventional morphing process in which the first change axis AX ₁ and the second change axis AX ₂ are mixed to change the image, according to this method, the change for each determinant is made. The correspondence between the amount and the change in face impression can be quantified.

Returning to FIG. 3, in this method, a sample image SIM can be extracted according to the attributes (gender, race, age, etc.) of the subject to generate a population (FIG. 1: Population generation step S34). . In this way, the basis vector calculated from the population having the same attributes as the subject is a good reflection of the facial features of the subject. The low-order basis represents a common point of appearance of the subjects and the population, and the high-order basis represents a local appearance impression of each subject. Therefore, the face image is gradually changed according to the first change axis AX ₁ or the second change axis AX ₂ while leaving the characteristics of the subject by sequentially reducing from the original image OIM to the higher-order weight coefficient. Can continue.

  In the face shape table illustrated in FIG. 3, as attribute data, the sex (female: f1 or male: f2) of the sample provider, whether multiple evaluators felt sensory evaluation of the sample image SIM in advance was attractive. No (attractive: f3, unattractive: f4), sample provider race (Japanese: f5, other race 1: f6, other race 2: f7) in the sample image SIM Has been granted. Here, the high degree of attractiveness of the face image is that the rate at which the evaluator feels attractive when the face image is subjected to sensory evaluation is said to be high.

  Further, in the example shown in FIG. 3, attribute data is also prepared for sensual evaluation items other than attractiveness, and the appearance impression of the sample image SIM determined by sensory evaluation in advance by a plurality of evaluators is shown. Attribute data is given to the sample image SIM. Specifically, the face (f8) that received the impression that the eyes are large, the face (f9) that received the impression that it was a small face, the face (f10) that received the impression that the nose was passing, The face (f11) that received the impression that it is a round face and the face (f12) that received the impression that it is a square face are illustrated.

  Further, as attribute data that can be objectively evaluated, a face wearing glasses (f13), a face with a strong symmetrical tendency (f14), and the like can be given. The symmetry of the face can be quantitatively obtained by calculating the coordinate position of the feature point of the face in the sample image.

  These attribute data are factors that affect the face shape. In other words, the average face shapes (average face shapes) of the sample images SIM belonging to each population are different from each other. Here, since the average face shape of the human face is generally a balanced face, the population extracted from the face image database 90 based on the sex (f1, f2) and race (f5-f7) of the subject. The average face shape is often attractive and sensory evaluated. Specifically, when the subject is a Japanese woman, when the attribute data f1 and f5 are selected by the condition input unit 20 in the face shape table of FIG. 3, the extraction unit 60 determines the face image according to the logical product (AND condition). A sample image SIM is extracted from the database 90. Specifically, NO1, 2, 5, 8, 14,... Of the sample image SIM are extracted to generate a population.

Then, the principal component analysis unit 40 obtains the basis vector e regarding this population and calculates the weight coefficient b (eigenvalue) of the original image OIM. The change image generation unit 50 sets the face shape of the original image OIM of the subject gradually closer to the average face shape of the population (Japanese women) by sequentially setting the weighting coefficient b to zero from the higher order basis. A change image IM ₁ is generated. By displaying and outputting the first change image IM ₁ with the output unit 70, the subject can visually confirm that the appearance is improved by leveling his / her face. The subject can quantitatively grasp how much the portion of the face is changed and how much the beauty is improved.

  Then, by extracting a sample image SIM whose attribute matches that of the subject and generating a population, the face shape can be leveled while maintaining the subject likeness in the original image OIM.

In the population generation step S34, in addition to the attribute data f3 representing the set of sample images SIM that the evaluator has answered by sensory evaluation and attractive, a set of sample images SIM (attribute data) that has been sensory evaluated that the eyes are large f8), a set of sample images SIM that has been sensory-evaluated to be a small face (attribute data f9), a set of sample images SIM that have been sensory-evaluated if the nose is passing (attribute data f10), etc. . Alternatively, a set (f11) of sample images SIM that are sensory-evaluated to be a round face and a set (f12) of sample images SIM that are sensory-evaluated to be an angular face may be selected.
By developing the original image OIM with basis vectors common to these populations and reducing the weighting coefficient b from the higher order side (to zero), the average face shape and attractive eyes of the attractive face Towards the face shape desired by the subject, such as the average face shape of a large face, the average face shape of a small face, the average face shape of a face that passes through the nose, the average face shape of a round face, and the average face shape of an angular face The original image OIM can be deformed (warping process). That is, according to this method, it is possible to set a sensory attribute that has been difficult to quantify as a change axis in the past, and the original image OIM is subjected to image processing so as to gradually strengthen the tendency of the change axis. can do.

  When the subject wears glasses, a set of sample images SIM (attribute data f13) wearing glasses may be selected. Further, when the subject's face is close to left-right symmetry, a set of sample images SIM (attribute data f14) having a strong left-right symmetry tendency may be selected.

  The face image database 90 only needs to have a function of storing image data, and it is not always necessary that the sample image SIM is actually stored in the face image database 90 when the present method is performed. The basis storage unit 92 only needs to have a function of storing vector data of the basis vector e, and it is not always necessary that these data are actually stored when the face image processing apparatus 100 is constructed. Absent.

FIG. 5 is an example of a texture table representing conditions for changing the facial texture (determinant). The texture table is obtained by classifying a large number of sample images SIM stored in the face image database 90 according to attributes that are considered to contribute to facial texture characteristics.
FIG. 6 is an example of a color table representing conditions for changing the face color (determinant). The color table is obtained by classifying a large number of sample images SIM stored in the face image database 90 according to attributes that are considered to contribute to facial color characteristics.

  Each sample image SIM is assigned one or more attribute data for at least one of the determinants.

  In the texture table illustrated in FIG. 5, as attribute data, the sex (female: t1 or male: t2) of the sample provider, attractiveness (attractive: t3, unattractive: t4), race (Japan) Person: t11, other race 1: t12, other race 2: t13) are given to the sample image SIM. These are the same as the face shape table shown in FIG. In the texture table, other attributes contributing to the texture of the face image include the age of the sample provider (10s: t5, 20s: t6, 30s: t7, 40s: t8, 50s: t9, 60s). Thereafter, t10) is further added to the sample image SIM. This is because the texture has a significant influence on the age compared to the face shape.

  Furthermore, in the example shown in FIG. 5, attribute data is also prepared for sensory evaluation items other than the attractiveness. Specifically, the face (t14) that received the impression that the skin is whitened, the face (t15) that received the impression that there was little unevenness in the color of the skin, and the face (t16) that received the impression that the texture was fine ). These are aesthetically good evaluation items. In the example illustrated in FIG. 5, for example, a face (t17) that has received an impression that freckles are conspicuous may be prepared as sensual attribute data that represents an aesthetically poor evaluation item.

  Among the attribute data in the color table shown in FIG. 6, the sex (female: c1 or male: c2), attractiveness (attractive: c3, unattractive: c4), race (Japanese) of the sample provider : C5, other race 1: c6, other race 2: c7) are common to other tables. The attribute data related to the sensual evaluation items other than the attractiveness level includes a face (c8) that received an impression of fairness, a face (c9) that received an impression of strong redness, and a healthy appearance. The face (c10) which received the impression of is illustrated.

  These attribute data can be set by selecting one or more of the tables shown in FIGS. 3, 5 and 6 as change axes in the set determinants. As a result, a population for principal component analysis of the original image OIM is generated.

  As described above, the condition input unit 20 inputs the attribute data to which the subject belongs or the attribute data indicating the facial tendency desired by the subject when generating the population of the sample image SIM in the population generation step S34. However, the present invention is not limited to this.

Instead of this method, for example, after the original image OIM is image-analyzed by the standardization unit 12 and the tendency for each determinant is quantified, the extraction unit 60 extracts the sample image SIM having high commonality with the original image OIM. You may extract automatically from the image database 90. FIG. Such extraction can be performed using a support vector machine (SVM) that has learned the tendency of the position of the feature points of the sample image SIM.
In addition, instead of extracting a sample image SIM for each change axis and generating a population, all sample images SIM accumulated in the face image database 90 may always be used as the population. That is, the condition input step S32 and the population generation step S34 are optional.

<Second embodiment>
A face image processing method (hereinafter, this method) according to the second embodiment of the present invention will be described. A duplicate description of matters common to the first embodiment is omitted.

This method is characterized in that both the first change axis AX ₁ and the second change axis AX _{2 have} a determinant as texture (overlapping selection). The flowchart of this method and the block diagram of the face image processing apparatus 100 of this embodiment are the same as those of the first embodiment (see FIGS. 1 and 2).

When the common determinant is selected repeatedly, the population for which the principal component analysis is performed is different. Thereby, the average face texture image of the first change axis AX _{1 and} the average face texture image of the second change axis AX ₂ are different from each other. For this reason, even when the common determinant is selected repeatedly, the first change image IM ₁ and the second change image IM ₂ are different images. For example, two different populations can be generated by making one or more attribute data selected from the texture table shown in FIG. 5 different. In addition, attribute data may be selected from different tables.

In this method, the first change axis AX ₁ is selected from the texture table of FIG. 5, and the second change axis AX ₂ is selected from the makeup texture table of FIG. 7 (FIG. 1: change axis setting step S30). ).

In the case of this method, the first change axis AX ₁ is a first texture representing the general appearance of the face, and the second change axis AX ₂ is a second texture representing the texture of the face. In the change image generation step S50, the face image (original image OIM) is morphed to the average face texture obtained by averaging the face textures of the sample images SIM belonging to the population generated in the population generation step S34. A first change image IM ₁ and a second change image IM ₂ are generated.

  The makeup texture table shown in FIG. 7 is obtained by classifying the sample images SIM according to the texture of makeup applied to the sample provider. By changing the texture of makeup, the facial texture generally changes greatly. In this method, m1: Sharp (stereoscopically slender impression), m2: Wet (wet skin-like), m3: Soft (soft impression) and m4: Heavy (appropriate) for each sample provider's face A face image in the case where makeup with four different textures (impression) is applied individually is stored in the face image database 90 as a sample image SIM. These four types of makeup with different textures are preferably those made by a makeup specialist.

  That is, the face image database 90 according to the present embodiment includes sample images obtained by capturing faces that have been subjected to a plurality of makeup methods with different textures. The extraction unit 60 selects a makeup method from the plurality of methods and generates a population.

  In this method, as an example, a case will be described in which the subject is a typical Japanese woman in her 30s who wants the texture to look fine and attractive by applying soft makeup. In this case, attribute data t1, t7, and t11 to which the subject belongs and attribute data t3 and t14 as the evaluation items desired by the subject are selected from the texture table in FIG. Further, the attribute data m3 is selected from the makeup texture table of FIG. 7 (FIG. 1: condition input step S32).

  The face image database 90 is searched based on the attribute data selected from each table, sample images SIM are extracted, and two populations are generated (FIG. 1: population generation step S34).

When the population is generated, the basis vector e is obtained in advance, and the original image OIM is subjected to principal component analysis to calculate weight coefficients b _{1, i} and b _{2, j} (FIG. 1: principal component analysis step S40). .
Then, in the change image generation step S50, by gradually reducing the weighting factors b _{1 and i} from the higher order side, while maintaining the face shape and color of the subject's original image OIM, “the texture is finely attractive. It is possible to generate the first change image IM _{1 in} which the original image OIM is brought close to the average texture of the “Japanese woman in his 30s”. In the change image generation step S50, the weight coefficients b _{2 and j} are gradually reduced from the higher order side, so that the face shape and color of the original image OIM of the subject are maintained and the “soft texture makeup” is maintained. It is possible to generate the second change image IM _{2 in} which the original image OIM is brought close to the average texture of “the face subjected to.

<Third embodiment>
A face image processing method (hereinafter, this method) according to a third embodiment of the present invention will be described.
FIG. 8 is a block diagram of the face image processing apparatus 100 of the present embodiment. The face image processing apparatus 100 of the present embodiment is different from the first embodiment (see FIG. 2) in that it includes a change degree calculation unit 52, a change degree determination unit 54, a beauty means storage unit 80, and a beauty means extraction unit 82. To do.
A duplicate description of matters common to the first embodiment or the second embodiment is omitted. The beauty means storage unit 80 and the beauty means extraction unit 82 will be described later in a modification of the present embodiment.

The method is characterized in that the first change image IM ₁ and the second change image IM ₂ are generated in a range that looks natural without losing the characteristics of the original image OIM.

Here, in the first embodiment, depending on the selection of the population of the sample image SIM, the average face shape image IM _1av and the average face texture image IM _2av (and the average face IM _av ) may be _separated from the original image OIM. is there. On the other hand, in this embodiment, it is possible to know whether or not the changed image looks natural by determining the degree of change from the original image OIM.

In this method, the threshold order k for reducing the weighting factor b _{1, m} (m = k + 1, k + 2, k + 3,...) Applied to the first change axis AX ₁ is set based on the degree of change in face shape. . In the change image generation step S50 of this method, the face shape of the original image OIM is changed by setting the higher-order weight coefficients b _{1, m} higher than the threshold order k to zero.

Figure 9 is a flow chart showing the details of the modification image generation step S50 that generates a first change image IM _1. In this method, a cumulative contribution ratio (CCR) is used as the change degree of the face shape.

The degree-of-change calculating unit 52 calculates the cumulative contribution ratio (CCR) of the weighting factors b _{1, i} applied to the first change axis AX ₁ by the following equation (3).

Here, the first change image IM ₁ reconstructed only from the low-order base having a low cumulative contribution CCR is strongly influenced by the average face shape image, and the local appearance of the subject is discarded. It becomes an image. In other words, by setting an appropriate threshold order k and removing a higher-order base from the original image OIM, the original image OIM is an average face shape image while leaving a general appearance impression of the subject. It becomes a face shape with good balance. If the threshold order k is too high, the change from the original image OIM is too small. If the threshold order k is too low, the change from the original image OIM is excessive and the naturalness is lost. For this reason, the critical order (threshold order k) at which the cumulative contribution rate becomes an appropriate threshold value (for example, 0.7 to 0.9) is calculated using Expression (3).

In other words, the cumulative contribution rate CCR and the degree of change have a negative correlation, and when the cumulative contribution rate CCR is low, it can be determined that the degree of change of the original image OIM is high. Thus, it is possible to adjust the rate of change of the face shape according to the first change image IM ₁ as desired.

The threshold value is stored in the change degree determination unit 54. The degree-of-change determination unit 54 determines the magnitude of the cumulative contribution rate CCR and the threshold value. When the change image generation unit 50 according to the present embodiment determines that the cumulative contribution rate CCR has reached a value corresponding to the threshold, that is, the threshold degree k has been found by the change degree determination unit 54, the threshold order k is set to the highest order. The first change image IM ₁ is reconstructed.

More specifically, the degree-of-change calculating unit 52 of the present embodiment subtracts the order by 1 from the highest order (n-th order) of the base in the above equation (1) (step S51), and from the first order to the n-th order. The cumulative contribution rate CCR up to the first order is calculated (step S521). When the degree-of-change determination unit 54 determines that the cumulative contribution rate CCR exceeds a threshold value (for example, 0.9) (step S531: YES), the degree-of-change calculation unit 52 determines the n−1th order and the nth order. Higher-order bases are removed by setting the weight coefficients b _{1, n-1} and b _{1, n} to zero (step S54).
Subsequently, the degree-of-change calculating unit 52 further reduces the base order by 1 (step S51), and calculates the cumulative contribution rate CCR from the first order to the n-2nd order (step S521). The degree of change determination unit 54 determines the cumulative contribution rate CCR as a threshold value.
Thus, by repeating the update of the cumulative contribution rate CCR and determining the magnitude of the threshold value, the order (kth order) that causes the cumulative contribution rate CCR to fall below the threshold value is obtained (step S531: NO). The change image generation unit 50 generates the first change image IM ₁ by performing image reconstruction using the highest order without removing the k-th order basis.

In this method, a plurality of first change images IM _{1 in} which the degree of change from the original image OIM gradually increases can be generated by reducing the threshold value stepwise.

Specifically, when the continuation of generation of the first change image IM ₁ is selected by the operation of the condition input unit 20 (see FIG. 2) (step S56: NO), the degree of change determination unit 54 reduces the threshold ( For example, it is reduced from 0.9 to 0.8) (step S57).
The degree-of-change calculator 52 reduces the order of the base until the new threshold is reached (step S51), updates the cumulative contribution rate CCR (step S521), and removes the higher-order base (step S54). When the threshold order k the cumulative contribution ratio CCR is a new threshold value or less is found by the change determination unit 54 (step S531: NO), change image generating unit 50 generates a new first change image IM ₁ (step S55).

By repeating such processing, it is possible to generate a plurality of first change images IM _{1 in} which the degree of change of the original image OIM gradually increases (see FIG. 4).

For the second variation axis AX _2, based on the second weighting factor b ₂ according to the change axis AX _{2, j,} determining the degree of change in texture due morphing process. Thereby, the second changed image IM _{2 in} which the texture of the original image OIM is changed within a natural range can be generated without losing the texture characteristics of the original image OIM.

The degree of texture change in the second change image IM ₂ can also be determined using the cumulative contribution rate CCR. In the modification of the present embodiment, the degree of texture change is determined using the cumulative contribution rate CCR (see FIG. 11).
In the present embodiment, a method of calculating the degree of change between the original image OIM and the second change image IM ₂ instead of the cumulative contribution rate CCR will be described below.

Two methods of calculating the degree of change will be described. In both methods, a distance score representing the distance between the second change image IM ₂ and the average face texture image IM _2av (see FIG. 4) is obtained. The smaller the distance score is, the larger the degree of change is because the second change image IM ₂ is greatly deviated from the original image OIM and close to the average face texture image IM _2av . That is, there is a negative correlation between the distance score and the degree of change.

In the first method, the sum of the weighting factors b _{2 and j} of the second change image IM ₂ is set as the distance score s ₁ . This distance score s ₁ can be calculated based on the following equation (4).

In the second method for calculating the distance score, the product sum of the weight coefficients b _{2 and j} considering the contribution ratio of the base vector e of the second changed image IM ₂ after the texture change is set as the distance score s ₂ . The distance score s ₂ can be calculated based on the following expressions (5) and (6). However, in Formula (5), t and u are natural numbers, respectively.

W _i obtained by the above equation (6) represents a contribution rate for each order of the basis vector e. The distance score s ₂ can be calculated by a product-sum operation of this contribution rate (which may be multiplied by an arbitrary weight factor t) and the square of the weighting factor b (which may also be multiplied by an arbitrary weight factor u). it can.

That is, in this method, a distance score s ₂ representing the distance between the average face texture and the face texture in the second change image IM ₂ is calculated based on the product-sum operation of the contribution ratio of the base vector e and the weighting factor b. based on the calculated distance score s ₂ may determine the change degree.

By using the distance score s ₂ , the contribution rate of the base vector e is emphasized, and the degree of change in texture is quantified. Therefore, it is possible to accurately quantify the impression of the second change image IM ₂ looks.

Figure 10 is a flowchart showing the details of the modification image generation step S50 that generates a second change image IM _2.
In the present embodiment, the degree-of-change calculator 52 calculates the distance score s ₂ as the degree of change each time the order is subtracted by 1 from the highest order of the base (step S51) (step S522).

The degree-of-change determination unit 54 stores a natural visual limit that is a sensory evaluation value indicating a limit at which the change from the original image OIM is viewed as natural. When the distance score s ₂ is smaller than the natural viewing limit (step S532: YES), the degree of change determination unit 54 removes the higher order basis (step S54).

Then, when the higher order basis is gradually removed and the basis approaches the average face texture image IM _2av from the original image OIM and the distance score s ₂ reaches the natural viewing limit (step S532: NO), the degree-of-change determination unit 54 The second change image IM ₂ is reconstructed using the base order at that time as the threshold order k (step S55).

That is, in this method, weighting factors b _{1, m} (m = k + 1, k + 2, k + 3,...) Higher than a predetermined threshold order k among the weighting factors b _{2, j} applied to the second change axis AX _2. ) To reduce).

In this way, the change image generation unit 50 generates the second change image IM ₂ in which the texture is changed to the maximum extent within the range that is naturally visible from the original image OIM. The second change image IM ₂ is displayed and output by the output unit 70 (FIG. 1: output step S70).

In other words, in this method, at least the second change image IM ₂ having the lowest threshold order k among the second change images IM ₂ determined not to exceed the predetermined upper limit (natural viewing limit) is output. To do.

As a result, the second changed image IM ₂ in which the texture is most changed within the range where the characteristics of the subject are not lost is output. Then, according to this method, the first change image IM ₁ is the second variation image IM _2, natural appearance of the original image OIM respectively, are generated in a range maintained. Therefore, with regard weighting coefficients b _{1, i} and b _2, superimposed changes are both changing the _j image IM _SP, it is possible to produce what the natural appearance without losing the characteristics of the subject.

FIG. 11 is a flowchart showing details of a modification of the change image generation step S50 for generating the first change image IM ₁ in the third embodiment. The change image generation step S50 of this modification is different from the third embodiment shown in FIG. 9 in that it includes a beauty means extraction step S58.

The face image processing apparatus 100 of this modification includes a beauty means storage unit 80 and a beauty means extraction unit 82. The beauty means storage unit 80 is a beauty means selection table in which the degree of change for each determinant in the change image (first change image IM ₁ or second change image IM ₂ ) is associated with the beauty means (see FIG. 12). Is a means for storing. The beauty means extraction unit 82 refers to the beauty means storage unit 80 and is a means for extracting beauty means for realizing the face indicated by the change image on the subject. The beauty means extracted by the beauty means extraction unit 82 is output by the output unit 70 together with the corresponding change image. Thereby, the beauty means for becoming the face shape, texture or color desired by the subject is presented.

That is, the present modification provides a beauty counseling method using the face image processing method (FIG. 1: face image acquisition step S10 to output step S70). In this beauty counseling method, the change in the shape, texture or color of the face due to the application of cosmetics or the application of cosmetic treatment and the first change image IM ₁ or the second change image IM ₂ are output in association with each other. Features.

More specifically, in this modification, a plurality of change images (first change image IM ₁ or second change image IM ₂ ) obtained by changing the texture of the original image OIM are generated, and the face of the subject is changed. Counseling for face shape, texture, or color.

  FIG. 12 is a diagram illustrating an example of a beauty means selection table. This beauty means selection table is for each population of sample images SIM extracted from the face image database 90 based on attribute data selected from the face shape table, texture table, or color table, and for each determinant change amount. , Which is associated with beauty means.

Examples of the beauty means include selection of one or more cosmetics, usage amount or usage of cosmetics, selection of cosmetic treatment, and the like.
For example, as a beauty means for changing the face shape, makeup that makes the eyes look larger, liquid glue (double glue) for making double wrinkles, small face massage, etc. are exemplified.
Examples of beauty means for changing the texture or color of the face include base makeups such as foundations and concealers, and various cosmetics including point makeup.

For example, (in the case of the attribute data f1 and f5 indicates a Japanese woman, the population 1) sample image SIM population to produce a first change image IM ₁ of changing the face shape for each face shape The beauty means A, B, C,... Are stored in the beauty means selection table according to the degree of change. Each time the change image generation unit 50 generates a plurality of first change images IM ₁ , the beauty means extraction unit 82 extracts the beauty means corresponding to the degree of change calculated by the change degree calculation unit 52 and outputs the beauty means. Present at 70. For this reason, the subject can not only simply change the original image OIM as the first change image IM ₁ or the second change image IM ₂ , but can specifically know the beauty means that realizes this.

Further, the face image processing method of the present invention describes a plurality of steps in order, but the order of description does not limit the order or timing of executing the plurality of steps. For this reason, when carrying out the face image processing method of the present invention, the order of the plurality of steps can be changed within a range that does not hinder the contents, and some or all of the execution timings of the plurality of steps are mutually connected. It may be duplicated. For example, in the above embodiment, the case where the change axis setting step S30 and the population generation step S34 are performed after the face image acquisition step S10 is exemplified, but instead, after the change axis setting step S30 and the population generation step S34. You may perform face image acquisition process S10. In FIG. 1, the population generation step S34 is performed every time the change image generation step S50 is repeated, but the present invention is not limited to this. The first change image IM ₁ and the second change image IM ₂ may be generated after performing principal component analysis by setting a plurality of change axes in advance.

DESCRIPTION OF SYMBOLS 10 Imaging part 12 Normalization part 20 Condition input part 40 Principal component analysis part 50 Change image generation part 52 Change degree calculation part 54 Change degree determination part 60 Extraction part 70 Output part 80 Beauty means memory | storage part 82 Beauty means extraction part 90 Face image Database 92 Base storage unit 95 Bus 100 Face image processing device

Claims (14)

Generate a population by extracting some sample images from a face image database containing multiple sample images,
Calculating a multi-order basis vector from the population;
Regarding at least a first change axis selected from the face shape, texture or color (including overlapping selection) and a second change axis indicating the texture of the face with respect to a face image in which at least a part of the face is captured each principal component analysis to calculate the weighting coefficients of the plurality next basis vector,
Generating a first change image in which the weighting coefficient applied to the first change axis is changed;
The weighting coefficient higher than a threshold order among the weighting coefficients applied to the second change axis is reduced, and the face image is moved toward an average face texture obtained by averaging the face textures of the sample images belonging to the population. It generates a second change picture image by morphing process,
Outputting the first change image and the second change image in comparison with each other ;
Including
The generation of the second change image is as follows:
Determining the degree of texture change due to the morphing process based on the calculated weighting factor;
Changing the threshold order based on the degree of change;
Reconstructing the second change image based on the changed threshold order,
A face image processing method characterized by the above.   A superimposed change image is generated by changing the weighting coefficients applied to the first change axis and the second change axis in multiple stages, and the overlap change is compared with the first change image or the second change image. The face image processing method according to claim 1, further comprising outputting an image. The first change axis is the face shape, and the face image is warped toward an average face shape obtained by averaging the face shapes of the sample images belonging to the population to generate the first change image. The face image processing method according to claim 1 or 2 . The face image processing method according to claim 3 , wherein the warping process is performed by reducing the weighting coefficient higher than a predetermined threshold order among the weighting coefficients applied to the first change axis. The first change axis is a first texture representing a global appearance impression of the face,
Ri second texture der that the second change-axis represents the texture of the face,
The first change image is generated by
Average than said threshold value order of the first change-axis of the first said weighting coefficient according to the change axis of reducing the weighting coefficients of higher order, the average texture of the face of Rusa sample images that belong to the population the facial image toward the face texture and morphing process to generate the first changing picture image,
Determining the degree of texture change due to the morphing process based on the weighting factor applied to the first change axis;
Changing the threshold order of the first change axis based on the degree of change;
Reconstructing the first change image based on the changed threshold order;
Including that,
The face image processing method according to claim 1 or 2 . The face image database includes the sample images obtained by imaging faces each subjected to a plurality of makeup methods with different textures,
Selecting the makeup method from the plurality of ways to generate the population ;
The face image processing method according to any one of claims 1 to 5 . Said change degree of the determination by said second change image does not exceed a predetermined upper limit value, according to any one of 6 claim 1 wherein the threshold degree is at least outputs the lowest said second change image Face image processing method. Calculating a distance score representing a distance between the average face texture and the facial texture in the second change image based on a product-sum operation of the contribution ratio of the basis vector and the weighting factor;
Determining the degree of change based on the calculated distance score ;
The face image processing method according to claim 1, wherein A beauty counseling method using the face image processing method according to any one of claims 1 to 8 ,
A beauty counseling method comprising: outputting a change in shape, texture or color of a face by applying cosmetics or applying a cosmetic treatment, and the first change image or the second change image in association with each other. A face image database for storing a plurality of sample images;
Condition input means for receiving input of conditions for extracting sample images;
Extracting means for generating a population by extracting the sample image from the face image database according to the received condition;
A plurality of basis vectors are calculated from the population, and at least a first change axis selected (including overlapping selection) from the shape, texture, or color of the face with respect to a face image obtained by capturing at least a part of the face. a principal component analysis means for calculating the weighting coefficients of the plurality next basis vectors and each principal component analysis for the second change-axis indicating the texture of the face,
Wherein said weighting coefficient according to the first change-axis to generate a first change image changing, to reduce the weight coefficients of higher order than the threshold degree of the weighting coefficients according to the second variation axis, said base and it changes the image generating means for generating a second change picture image by morphing processing the facial image toward the average face texture obtained by averaging the texture of the face of the sample images that belong to the group,
A degree-of-change calculating means for calculating a degree of change in texture due to the morphing process based on the calculated weighting factor;
A degree-of-change determination means for changing the threshold order based on the degree of change;
Image output means for comparing and outputting the first change image and the second change image;
Equipped with a,
The change image generation means reconstructs the second change image based on the changed threshold order;
Face image processing device. The face image processing device according to claim 10 , wherein the plurality of sample images stored in the face image database have standardized positions of eyes, nose and mouth.   The first axis of change is the shape of the face;
  The change image generation means generates the first change image by warping the face image toward an average face shape obtained by averaging the face shapes of the sample images belonging to the population.
  The face image processing apparatus according to claim 10 or 11.   The change image generation means performs the warping process by reducing the weighting coefficient higher than a threshold order among the weighting coefficients applied to the first change axis.
  The face image processing apparatus according to claim 12.   The first change axis is a first texture representing a global appearance impression of the face,
  The second change axis is a second texture representing the texture of the face;
  The change image generation unit reduces a weight coefficient higher than a threshold order of the first change axis among the weight coefficients applied to the first change axis, and the face of the sample image belonging to the population The first change image is generated by morphing the face image toward an average face texture obtained by averaging the textures of
  The degree-of-change calculating means calculates a degree of change in texture due to the morphing process based on the weighting factor applied to the first change axis,
  The change degree determination means changes the threshold order of the first change axis based on the change degree,
  The change image generation means reconstructs the first change image based on the changed threshold order;
  The face image processing apparatus according to claim 10 or 11.

Images