JP2011150595A - Apparatus, method and program for evaluating face shape - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly accurately evaluate a face based on a variation between two face images by three-dimensionally capturing the face. <P>SOLUTION: A face shape evaluation apparatus for obtaining three-dimensional (3D) data of a face shape of an object to be evaluated and evaluating the face shape has: a 3D face shape data acquisition means for acquiring the 3D data of the face shape on many points; a face shape evaluation means for evaluating the face shape of the object to be evaluated on the basis of the 3D face shape data and 3D average face data extracted based on a preset condition; a differential color region generation means for generating color regions on the face shape based on differential data between the 3D face shape data obtained by the face shape evaluation means and the 3D average face data; a display picture generation means for generating a picture for displaying the color regions generated by the differential color region generation means; and an output means for outputting the picture obtained by the display picture generation means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a face shape evaluation device, a face shape evaluation method, and a face shape evaluation program, and in particular for capturing a face three-dimensionally and performing face evaluation with high accuracy based on the amount of change between two face images. The present invention relates to a face shape evaluation device, a face shape evaluation method, and a face shape evaluation program.

  In recent years, it has been diagnosed from the viewpoint of skin care, makeup counseling, cosmetic surgery, etc. by evaluating the degree of aging and anti-aging due to facial slack, etc. from the human face image, and evaluating the beauty of the face. Or a technique for identifying a face (see, for example, Patent Documents 1 and 2).

  For example, in the technique disclosed in Patent Document 1, a face image for determination represented by a plurality of regions distinguished with respect to brightness by performing image enhancement processing on a face image obtained by imaging a face to be determined. Is created, and a face is determined based on the shape of the contour of the region in the determination face image.

  Further, in the technique disclosed in Patent Document 2, the aging degree of the face of the person whose face image is captured is determined by performing image enhancement processing on the face image obtained by capturing the face image of the target person. A technique is disclosed.

Japanese Patent Laid-Open No. 11-169358 JP 2006-334427 A

  By the way, in the above-described conventional technology, evaluation is performed based on a two-dimensional image of the face of the photographed subject, but the shape of the face is uneven due to the contours of the face and features such as eyes, nose and mouth. For this reason, when the evaluation is performed based on the two-dimensional image, the three-dimensional brightness and darkness due to the unevenness is not reflected in the image, and high-precision evaluation cannot be performed.

  Further, in the face evaluation, for example, in the case of facial sag, which is one index for evaluating the degree of aging, it is difficult to judge only by contrast. Furthermore, despite the fact that the looseness of the face is a phenomenon that can be grasped visually, it has not been possible to grasp the shape specifically and quantify and evaluate it.

  In addition, a comparison method based on a photographed two-dimensional image (photograph) may be used to evaluate the improvement of slackness, but this is also easily influenced by lighting, angle, etc. at the time of photography, and is objective. In order to perform the evaluation, since a very high imaging technique is required, it is difficult to evaluate the face with high accuracy.

  The present invention has been made in view of the above-described problems, and is a face shape evaluation device and a face for capturing a face three-dimensionally and performing face evaluation with high accuracy based on a change amount of two face images. An object is to provide a shape evaluation method and a face shape evaluation program.

  In order to solve the above-described problems, the present invention employs means for solving the problems having the following characteristics.

  According to the first aspect of the present invention, in the face shape evaluation apparatus that acquires the three-dimensional data of the face shape of the person to be evaluated and evaluates the face shape, the three-dimensional data of the face shape is acquired at multiple points. The evaluation object based on three-dimensional face shape data acquisition means, three-dimensional face shape data obtained by the three-dimensional face shape data acquisition means, and three-dimensional average face data extracted based on preset conditions Generating a color region in the face shape based on difference data between the three-dimensional face shape data and the three-dimensional average face data obtained by the face shape evaluation means Difference color area generation means, display screen generation means for generating a screen for displaying the color area generated by the difference color area generation means, and a screen obtained by the display screen generation means And an outputting means for outputting.

  According to the first aspect of the present invention, it is possible to grasp the face three-dimensionally and perform the face evaluation with high accuracy based on the amount of change of the two face shape data.

  The invention described in claim 2 is characterized in that the three-dimensional average face data is obtained from a plurality of three-dimensional face shape data extracted based on preset conditions from three-dimensional face shape data for the plurality of evaluation subjects. 3D average face data generating means for generating

  According to the invention described in claim 2, it is possible to perform evaluation with high objectivity by evaluating with the average face instead of the representative example data of the individual, and information for using the three-dimensional average face data for other evaluations and the like. When provided, since the average face is not personal information, the data itself can be handled easily.

  According to a third aspect of the present invention, the three-dimensional average face data generating unit synthesizes two three-dimensional face shape data selected from the plurality of three-dimensional face shape data by a morphing process using a predetermined ratio. It is characterized by doing.

  According to the third aspect of the present invention, various types of three-dimensional average face data can be easily generated by morphing processing.

  The invention described in claim 4 is characterized in that the face shape evaluating means evaluates a predetermined part of the face of the person to be evaluated.

  According to the fourth aspect of the present invention, it is possible to perform evaluation not only for the entire face but also for each part such as the jaw part, the mouth part, the eye part, and the face line.

  According to a fifth aspect of the present invention, the face shape evaluation means displays a plurality of the three-dimensional face shape data or the three-dimensional average face data on the same screen in three dimensions, and uses one of the data as reference data. In other areas where the thickness of the nose, forehead, earlobe region, and soft tissue is thin and difficult to change, at least one location was overlaid with other data so that there was no difference on the 3D coordinates. Alignment data is generated, and evaluation is performed based on the generated alignment data.

  According to the fifth aspect of the present invention, the two face shape data can be compared with high accuracy by aligning with reference to the portion where the face hardly changes.

  In the invention described in claim 6, the face shape evaluation unit may perform evaluation at each point of the 3D face shape data or the 3D average face data to be evaluated with respect to the reference data based on the alignment data. Difference data on three-dimensional coordinates is generated.

  According to invention of Claim 6, the three-dimensional change of a face can be acquired with high precision using a three-dimensional coordinate.

  In the seventh aspect of the present invention, the face shape evaluating means may be configured such that the evaluation portion of the three-dimensional face shape data to be evaluated or the three-dimensional average face data is located outside the face of the reference data. In this case, it is evaluated that the evaluation portion is expanded.

  According to the seventh aspect of the present invention, the face shape can be evaluated with high accuracy based on the amount of change between the two face shape data.

  According to an eighth aspect of the present invention, the face shape evaluating means has the evaluation portion of the three-dimensional face shape data or the three-dimensional average face data to be evaluated at a place corresponding to the inside of the face of the reference data. The evaluation location is evaluated as being tightened.

  According to the eighth aspect of the present invention, the face shape can be evaluated with high accuracy based on the amount of change between the two face shape data.

  The invention described in claim 9 is a face shape evaluation method for evaluating the face shape by acquiring the three-dimensional data of the face shape of the person to be evaluated. Based on the three-dimensional face shape data acquisition step to be acquired, the three-dimensional face shape data obtained by the three-dimensional face shape data acquisition step, and the three-dimensional average face data extracted based on preset conditions A face shape evaluation step for evaluating the face shape of the evaluation subject, and a color region in the face shape based on difference data between the three-dimensional face shape data and the three-dimensional average face data obtained by the face shape evaluation step A difference color area generation step for generating a display screen, a display screen generation step for generating a screen for displaying the color area generated by the difference color area generation step, And an outputting step of outputting by the output device a screen obtained by 示画 plane generation step.

  According to the ninth aspect of the present invention, it is possible to grasp the face three-dimensionally and perform the face evaluation with high accuracy based on the amount of change of the two face shape data.

  The invention described in claim 10 is characterized in that the three-dimensional average face data is obtained from a plurality of three-dimensional face shape data extracted based on preset conditions from three-dimensional face shape data for the plurality of evaluation subjects. And a three-dimensional average face data generation step for generating.

  According to the invention described in claim 10, it is possible to perform evaluation with high objectivity by evaluating with an average face instead of individual representative example data, and information for using the three-dimensional average face data for other evaluations and the like. When provided, since the average face is not personal information, the data itself can be handled easily.

  In the invention described in claim 11, in the three-dimensional average face data generation step, two three-dimensional face shape data selected from the plurality of three-dimensional face shape data are synthesized by morphing processing using a predetermined ratio. It is characterized by doing.

  According to the eleventh aspect of the present invention, various types of three-dimensional average face data can be easily generated by morphing processing.

  The invention described in claim 12 is characterized in that the face shape evaluation step means evaluates a predetermined part of the face of the person to be evaluated.

  According to the twelfth aspect of the present invention, it is possible to perform evaluation not only for the entire face but also for each part such as a chin part, a mouth part, an eye part, and a face line.

  In the invention described in claim 13, in the face shape evaluation step, a plurality of the three-dimensional face shape data or the three-dimensional average face data are displayed three-dimensionally on the same screen, and one of the data is used as reference data. In other areas where the thickness of the nose, forehead, earlobe region, and soft tissue is thin and difficult to change, superimpose other data so that there is no difference in three-dimensional coordinates with respect to the reference data in at least one location. Then, alignment data is generated by aligning the positions, and evaluation is performed based on the generated alignment data.

  According to the thirteenth aspect of the present invention, the two face shape data can be evaluated with high accuracy by aligning with reference to the portion where the face hardly changes.

  In the invention described in claim 14, in the face shape evaluation step, at each point of the 3D face shape data or the 3D average face data to be evaluated with respect to the reference data based on the alignment data. Difference data on three-dimensional coordinates is generated.

  According to invention of Claim 14, the three-dimensional change of a face can be acquired with high precision using a three-dimensional coordinate.

  In the invention described in claim 15, in the face shape evaluation step, the evaluation portion of the 3D face shape data to be evaluated or the 3D average face data is outside the face of the face shape data as the reference data. When it exists in a corresponding place, it is evaluated that the said evaluation part is expanding.

  According to the fifteenth aspect of the present invention, the face shape can be evaluated with high accuracy based on the amount of change between the two face shape data.

  In the invention described in claim 16, in the face shape evaluation step, the evaluation portion of the three-dimensional face shape data to be evaluated or the three-dimensional average face data is inside the face of the face shape data as the reference data. It is characterized by evaluating that the evaluation part is tightened when it is in a corresponding place.

  According to the sixteenth aspect of the present invention, the face shape can be evaluated with high accuracy based on the change amount of the two face shape data.

  The invention described in claim 17 is characterized in that the computer is operated so as to execute the face shape evaluation method according to any one of claims 9 to 16.

  According to the seventeenth aspect of the present invention, it is possible to grasp the face three-dimensionally and perform the face evaluation with high accuracy based on the amount of change of the two face shape data. Further, by installing the program, the face shape evaluation process in the present invention can be easily realized by a general-purpose personal computer or the like.

  According to the present invention, it is possible to grasp a face in three dimensions and perform face evaluation with high accuracy based on the amount of change between two face images.

It is a figure which shows an example of the face shape evaluation system in this embodiment. It is a figure which shows an example of the hardware constitutions which can implement | achieve the face shape evaluation process in this embodiment. It is a flowchart which shows an example of the face shape evaluation process sequence in this embodiment. It is a figure for demonstrating the outline | summary of imaging | photography using a three-dimensional face shape measuring apparatus. It is a figure which shows the example of a production | generation of three-dimensional average face data. It is a figure which shows the mesh area | region for morphing implementation. It is a figure for demonstrating the mode of the change of the morphing result by the setting of a mixing ratio. It is a figure which shows an example of the production | generation result of 3D average face data. It is a figure which shows an example of the face shape evaluation method in this embodiment. It is a figure which shows the example of a production | generation of a difference color area. It is a figure which shows an example of a display result. It is a figure which shows an example of the evaluation method of a volume and a surface area evaluation function. It is a figure which shows an example of the morphing implementation procedure in the production | generation of three-dimensional average face data. It is a figure for demonstrating the mode of the image processing in this embodiment. It is a figure which shows the result evaluated corresponding to each condition mentioned above. It is a figure which shows an example of a volume and a surface area evaluation result. It is a figure which shows an example of an analysis and evaluation result and a questionnaire result.

<About the present invention>
The present invention captures a photographed face of an evaluation subject in a three-dimensional manner, and performs face evaluation with high accuracy based on the amount of change between two face images. Specifically, the present invention creates a three-dimensional average face, for example, by comparing the three-dimensional average face before and after continuous use of skin external preparations, cosmetics, health foods, pharmaceuticals, massage, etc. Evaluate how much the effect is occurring in which part.

  In addition, the present invention compares the three-dimensional face shape data, performs coloring according to the amount of change, and displays the change amount so as to be visually distinguishable, thereby using skin care products, cosmetics, etc. As a result, it is possible to accurately grasp what effect has been achieved, and it is also possible to evaluate cosmetics and the like according to the content of the result.

  Hereinafter, embodiments in which a face shape evaluation apparatus, a face shape evaluation method, and a face shape evaluation program according to the present invention are suitably implemented will be described with reference to the drawings.

<Facial shape evaluation system: system configuration example>
First, a system configuration example of a face shape evaluation system that performs face shape evaluation in the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a face shape evaluation system in the present embodiment. The face shape evaluation system 10 shown in FIG. 1 is configured to include a three-dimensional face shape measurement device 11 and a face shape evaluation device 12. Note that the three-dimensional face shape measuring device 11 and the face shape evaluating device 12 are connected in a state where data can be transmitted and received by a communication network 13 formed by wire or wireless.

  The three-dimensional face shape measuring apparatus 11 is an apparatus for photographing a face image of an evaluation subject or a sampler (subject) for generating a three-dimensional average image. As the three-dimensional face shape measuring apparatus 11, for example, a photographing method using a lattice pattern projection method (for example, Danae (manufactured by NEC Engineering)) or a photographing method using a laser light cutting method (for example, vivid 910 (Konica Minolta)). 3D image obtained by a conventional photographing method such as a stereo photographing method (for example, QM-3000 (manufactured by Topcon Techno House Co., Ltd.)).

  Here, the images photographed by the three-dimensional face shape measuring apparatus 11 are images before and after application of a skin external preparation or the like, for example, 6 hours, 1 day, 1 week, or 1 month. Acquire images taken at different times and periods later. In addition, the three-dimensional face shape measurement apparatus 11 outputs a three-dimensional image of the face of the subject to be photographed obtained by measurement to the face shape evaluation apparatus 12 via the communication network 13.

  In addition, when the image image | photographed by the three-dimensional face shape measuring device 11 is used for the production | generation of a three-dimensional average face, the age of a photographing subject, the color and texture of skin, the presence or absence of freckles, and the skin are rough. The target person profile information of the location and other information about the skin, time information (season, humidity, temperature, climate, etc.) at the time of shooting, shooting information such as how to illuminate, etc. are transmitted together, and the face shape evaluation device 12 Are stored and managed.

  The face shape evaluation device 12 acquires the 3D face shape data acquired from the 3D face shape measurement device 11 via the communication network, and evaluates the face shape by comparing with the previously stored 3D average face data. . Specifically, the face shape evaluation apparatus 12 compares two three-dimensional average face data to be compared, compares the three-dimensional coordinates of each point, and measures the amount of unevenness from the difference data of each point. Further, the face shape evaluation device 12 performs coloring or the like on the three-dimensional face shape data according to the difference value (change amount) of the unevenness from the measurement result, and a display provided in advance in the face shape evaluation device 12 Etc.

  Further, the face shape evaluation device 12 can store the three-dimensional image from the three-dimensional face shape measurement device 11 in advance, and collectively perform the evaluation process on the stored face image at a predetermined timing. .

  In FIG. 1, the face shape evaluation system 10 has a one-to-one configuration of the three-dimensional face shape measurement device 11 and the face shape evaluation device 12. However, the present invention is not limited to this. N, N: 1, N: N (N: arbitrary multiple values) may be configured, and the three-dimensional face shape measurement device 11 and the face shape evaluation device 12 may be configured to be integrated. .

<Face shape evaluation device 12: functional configuration example>
Here, a functional configuration example of the face shape evaluation apparatus 12 in the present embodiment will be described. As shown in FIG. 1, the face shape evaluation apparatus 12 includes an input means 21, an output means 22, a storage means 23, a 3D face shape data acquisition means 24, a 3D average face data generation means 25, a face shape. The evaluation unit 26, the difference color area generation unit 27, the display screen generation unit 28, the transmission / reception unit 29, and the control unit 30 are configured.

  The input means 21 inputs each instruction information from a user or the like for evaluating the face shape in the present embodiment. Specifically, the input unit 21 inputs various instructions such as a 3D face shape acquisition instruction, a 3D average face generation instruction, a face shape evaluation instruction, a difference color area generation instruction, and a display screen generation instruction from a user or the like. Accept. Note that the input unit 21 includes, for example, a keyboard and a pointing device such as a mouse.

  Also, the input means 21 is an instruction input for acquiring the 3D face shape data already photographed by the 3D face shape measuring device 11 connected to the communication network 13 or other external devices by the transmission / reception means 29 or the like. Also, it has a function of inputting various data such as user information of the person to be evaluated for the photographed image.

  The output unit 22 displays / outputs the content input by the input unit 21 and the content executed based on the input content. Note that the output means 22 includes a display, a speaker, and the like. Furthermore, the output unit 22 may have a function of a printer or the like. In this case, the face shape evaluation result or the like can be printed on a print medium such as paper and presented to the user or the like.

  The accumulating unit 23 includes 3D face shape data obtained by the 3D face shape data obtaining unit 24, an average face image obtained by the 3D average face data generating unit 25, an evaluation result obtained by the face shape evaluating unit 26, and a difference. Necessary for executing the face shape evaluation in the present invention, such as difference color allocation information set by the color area generation means 27, various information of the display screen generated by the display screen generation means 28, information obtained by the transmission / reception means 29, etc. Various kinds of data. Further, the storage means 23 can read out various data stored as required.

  The three-dimensional face shape data acquisition means 24 is an evaluation subject or a subject for generating an average face from the three-dimensional face shape measurement device 11 via the communication network 13 in response to an image acquisition instruction from the input means 21 by a user or the like. 3D face shape data is acquired. The three-dimensional face shape data acquisition unit 24 may store the three-dimensional face shape data obtained by the transmission / reception unit 29 using the storage unit 23, and the three-dimensional average face data generation unit 25 may store the three-dimensional average face data. Can be generated, or face shape evaluation means 26 can perform face shape evaluation.

  The three-dimensional average face data generating unit 25 generates three-dimensional average face data using three-dimensional face shape data obtained from a plurality of users acquired from the storage unit 23 and the like. The three-dimensional average face data generating means 25 generates three-dimensional average face data for each age, such as teens, twentieth, thirties,. 3D before and after application of facial preparations such as laughing face, angry face, crying face, etc. by facial expression, gender, country, region, face application Average face data may be generated. Furthermore, the three-dimensional average face data generation means 25 can also generate three-dimensional average face data that combines the above-described contents. A specific generation method will be described later. The three-dimensional average face data generating unit 25 stores the generated one or more three-dimensional average face data in the storage unit 23 and the like.

  The face shape evaluation means 26 includes the three-dimensional average face data extracted by the preset condition among the various three-dimensional average face data generated by the three-dimensional average face data generation means 25, and the third order of the evaluation target person. The original face shape data is compared, and the face shape is evaluated based on the comparison result. The preset conditions are, for example, three-dimensional average face data generation means such as average face data before and after application of the skin external preparation on the face, by age, country, etc. corresponding to the evaluation subject, etc. This is a condition set by the user or the like in order to extract a specific average face image from the various types of average face images generated in Step 25.

  Further, the face shape evaluation means 26 digitizes the comparison result between the two images described above. The value to be quantified is a value obtained by quantifying the relationship between the unevenness obtained by superimposing the three-dimensional average face data and the three-dimensional face shape data of the evaluation subject. Further, when digitizing, the digitization is performed based on the relative positional relationship based on one of the coordinates based on coordinates (x, y, z) set in advance in the three-dimensional space. The face shape evaluation unit 26 outputs the digitized information to the difference color area generation unit 27.

  Furthermore, when a plurality of three-dimensional face shape data for the same evaluation subject is accumulated over time, the face shape evaluation means 26 uses the three-dimensional face shape data of the same evaluation subject having different times. The evaluation as described above can also be performed.

The difference color area generation unit 27 generates the change amount as a difference value based on the information digitized by the face shape evaluation unit 26. Further, the difference color area generation unit 27 generates a color area corresponding to the difference value. Specifically, the difference color area generation means 27 assigns color information based on the difference value in advance, and displays the assigned difference color assignment information on the screen based on the information digitized by the face shape evaluation means 26. Color information to be generated. Further, the difference color area generation means 27 determines which is appearing on the surface by comparing the unevenness between the three-dimensional average face data and the three-dimensional image of the evaluation subject, and color information is also set based on the determination result. The That is, of the two images, different color information is generated depending on the difference between the image on the surface and the color information for the screen display generated by the difference color area generation means 27 This is information about the color and the position coordinates of the 3D face shape data in the 3D space. Thereby, color information can be displayed on the three-dimensional face shape data.

  The display screen generation unit 28 generates a display screen for displaying the result evaluated by the face shape evaluation unit 26 on the output unit 22 such as a display. Note that the display screen generation means 28 can also perform processing for extracting audio data to be output by the output means 22 such as a speaker from a plurality of audio data stored in advance in the storage means 23 based on the displayed contents. .

  The transmission / reception unit 29 and the above-described three-dimensional face shape measurement apparatus 11 are caused to photograph the three-dimensional face shape data of the evaluation subject, and various control signals for acquiring the photographed image and related information are transmitted. . The transmission / reception means 29 is acquired from the three-dimensional face shape measurement device 11 by three-dimensional face shape data, time information, position information obtained by photographing the evaluation subject, and an external device connected by the communication network 13. Images and the like can be received. The transmission / reception means 29 can output the three-dimensional average face data, the three-dimensional image, the face shape evaluation result, and the like to an external device or the like via the communication network 13.

  The control means 30 controls the entire components of the face shape evaluation apparatus 12. Specifically, the control unit 30 is configured to generate a 3D face shape acquisition process, a 3D average face generation process, a face shape evaluation process, a difference color area generation, based on various instruction information from the input unit 21 by a user, for example. Each control such as processing and display screen generation processing is performed.

  In the present embodiment, by having the configuration as described above, the face can be captured in three dimensions and the face can be evaluated with high accuracy based on the amount of change between the two face images.

<Face shape evaluation device 12: hardware configuration>
Here, in the face shape evaluation apparatus 12 described above, an execution program (face shape evaluation program) capable of causing a computer to execute each function is generated, and the execution program is installed in, for example, a general-purpose personal computer or server. Thus, the face shape evaluation process and the like in the present invention can be realized. Here, a hardware configuration example of a computer capable of realizing the face shape evaluation process in the present embodiment will be described with reference to the drawings.

  FIG. 2 is a diagram illustrating an example of a hardware configuration capable of realizing the face shape evaluation process according to the present embodiment. 2 includes an input device 31, an output device 32, a drive device 33, an auxiliary storage device 34, a memory device 35, a CPU (Central Processing Unit) 36 that performs various controls, and a network connection device. 37, and these are connected to each other by a system bus B.

  The input device 31 has a pointing device such as a keyboard and a mouse operated by a user or the like, and inputs various operation signals such as execution of a program from the user or the like. The input device 31 is a sampler (subject) for generating an evaluation target person or an average face image measured from the three-dimensional face shape measurement device 11 connected to the network connection device 37 or the like via the communication network 13. It is also possible to input various data such as an instruction for acquiring the three-dimensional face shape data and image related information for the acquired image.

  The output device 32 has a display for displaying various windows and data necessary for operating the computer main body for performing the processing according to the present invention, and displays the program execution progress and results by the control program of the CPU 36. can do. Further, the output device 32 can print the above processing result or the like on a print medium such as paper and present it to the user or the like.

  Here, the execution program installed in the computer main body in the present invention is provided by, for example, a portable recording medium 38 such as a USB (Universal Serial Bus) memory, a CD-ROM, or a DVD. The recording medium 38 on which the program is recorded can be set in the drive device 33, and the execution program included in the recording medium 38 is installed in the auxiliary storage device 34 from the recording medium 38 via the drive device 33.

  The auxiliary storage device 34 is a storage means such as a hard disk, and can store an execution program in the present invention, a control program provided in a computer, and the like, and can perform input / output as necessary.

  The memory device 35 stores an execution program read from the auxiliary storage device 34 by the CPU 36. The memory device 35 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

  The CPU 36 controls processing of the entire computer, such as various operations and input / output of data with each hardware component, based on a control program such as an OS (Operating System) and an execution program stored in the memory device 35. Thus, each processing can be realized. Various information necessary during the execution of the program can be acquired from the auxiliary storage device 34, and the execution result and the like can also be stored.

  The network connection device 37 acquires an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program or an execution in the present invention The program itself can be provided to other terminals.

  The network connection device 37 can also acquire various types of data such as the three-dimensional face shape data of the evaluation subject using an external device such as the three-dimensional face shape measurement device 11 connected to the communication network.

  With the above-described hardware configuration, the face shape evaluation process according to the present invention can be executed. Further, by installing the program, the face shape evaluation process in the present invention can be easily realized by a general-purpose personal computer or the like.

<Face shape evaluation processing procedure>
Next, the face shape evaluation processing procedure in this embodiment will be described. FIG. 3 is a flowchart showing an example of a face shape evaluation processing procedure in the present embodiment. In the following description, it is assumed that three-dimensional average face data is generated in advance and stored in the storage unit 23 and the like.

  In the face shape evaluation process shown in FIG. 3, first, the three-dimensional face shape data of the evaluation subject is acquired (S01), and the acquired three-dimensional face shape data is compared with the three-dimensional average face data under preset conditions. The face shape is evaluated by (S02).

  Next, as a result of the face shape evaluation, it is determined whether there is a difference value based on the coordinate value of each point of each point cloud data forming both face images (S03). Whether or not there is a difference value is determined based on whether or not the difference value in a preset region is equal to or greater than a preset threshold value.

  If there is a difference value (YES in S03), a difference color area is generated (S04). Further, after the process of S04 is completed or when there is no difference value in the process of S03 (NO in S03), a display screen is generated (S05), and the generated screen is output to a display or the like (S06).

  Next, for example, using three-dimensional average face data extracted under other conditions, it is determined whether or not to perform face shape evaluation under other conditions (S07), and when face shape evaluation is performed (in S07, YES), the processing after S02 is performed under other conditions. If face shape evaluation is not performed (NO in S07), the process ends.

<Example 1>
Next, a specific example of the face shape evaluation method in the present embodiment using the above-described configuration will be described. In the following description of the embodiment, an example of performing an improvement effect evaluation on the looseness of the face will be described as an example of the face shape evaluation. However, the content that can be evaluated in the present invention is not limited to this, for example, other It can be suitably applied to aging degree evaluation, anti-achieving degree evaluation, beauty degree evaluation, and the like based on the above index.

  In addition, as a reference for evaluating the change in looseness of the face with high accuracy, “change in thickness (color difference)”, “volume”, “area”, and the like in the “part” are important. Therefore, in the following embodiment, a method for obtaining the above contents and performing evaluation with high accuracy will be described.

<Specific Example of Measuring Method in Three-dimensional Face Shape Measuring Device 11>
First, a specific example of the measurement method in the three-dimensional face shape measurement apparatus 11 will be described. FIG. 4 is a diagram for explaining an outline of photographing using the three-dimensional face shape measuring apparatus. In the example of FIG. 4, an imaging method based on the lattice pattern projection method will be described. However, the example of generating the three-dimensional face shape data applied to the present invention is not limited to this, and the above-described laser light cutting is performed. A three-dimensional image obtained by a photographing method using a method, a photographing method using a stereo method, or the like can be used.

  In the example shown in FIG. 4, at the time of measurement, the evaluation target person P seems to photograph a face from the opening 41 in correspondence with the position of the camera of the three-dimensional face shape measuring apparatus 11 whose outer shape is formed in a box shape. To. At this time, the three-dimensional face shape measuring apparatus 11 is provided with photographing units 42-1 and 42-2 for photographing images from the left and right of the evaluation subject P, respectively. The photographing units 42-1 and 42-2 are cameras as imaging means for photographing either the left or right face of the evaluation subject P from the opening 41, as shown in the state where the upper box in FIG. 4 is removed. 43, and a light source 44 and a pattern grid 45 as a projector for projecting a grid pattern are provided.

  Further, when acquiring the three-dimensional face shape data of the evaluation target person P, a grid pattern by the pattern grid 45 is projected onto the face of the evaluation target person P using the light source 44 of the grid pattern projection projector as shown in FIG. Then, this lattice pattern is shifted and photographed by the camera 43. By analyzing this captured image by the phase shift method, three-dimensional face shape data can be measured in a short time.

  In the present embodiment, the evaluation target person P is about 400 mm away from the three-dimensional face shape measurement apparatus 11, the measurement range is, for example, about 400 × 300 [mm], and the apparatus accuracy is the sampling interval (x −in-plane): 0.33 mm, z-direction noise (2σ): 0.046 mm.

  Also, in measurement, because slight differences in facial expressions before and after continuous use of cosmetics, etc. have an effect on the face shape, data obtained by measuring multiple times after obtaining a predetermined facial expression with a preset voice guide is acquired. Then, data having a high degree of coincidence between facial expressions before and after continuous use is acquired based on a predetermined condition such as performing alignment before and after continuous use and selecting a combination of data having the smallest difference in facial expressions.

  Also, the three-dimensional face shape measuring apparatus 11 using the grid pattern projection method shown in FIG. 4 is a twin-lens apparatus, which measures the left and right half faces, and then joins the data to the entire face. Data is being acquired. The measurement range is such that the vertical direction is from the neck to the head, the horizontal direction is such that the left and right ears enter, and the measurement time is about 1 second. The interval at which data is acquired is about 0.33 mm, and the amount of noise in the depth direction is about 0.046 mm (2σ).

  Further, the 3D face shape data acquired by the 3D face shape measuring apparatus 11 in the present embodiment is point cloud data having xyz coordinates. In FIG. 4, the coordinates are converted by CG (Computer Graphics) shading. Although it is displayed, it can also be output in a general three-dimensional shape format such as stl or dxf, or a file format unique to the apparatus.

<Generation example of 3D average face data>
Next, a generation example of the three-dimensional average face image in the above-described embodiment will be described with reference to the drawings. FIG. 5 is a diagram illustrating an example of generating the three-dimensional average face data. In this embodiment, among the plurality of three-dimensional face images, two face images are combined by a morphing function to generate one composite image, and the composite image is further applied to another three-dimensional face image to obtain a tertiary image. Generate original average face data. In the example of FIG. 5, three-dimensional average face data 47 is generated from eight three-dimensional face images 46-1 to 46-8.

  Here, in this embodiment, as an advantage of evaluation using an average face, for example, evaluation with an average face rather than individual representative example data enables evaluation with high objectivity, and three-dimensional When the average face image is provided for use in other evaluations, the average face is not personal information, so that the data itself can be handled easily.

  Further, in the morphing in the present embodiment, for example, one intermediate data corresponding to the mixing ratio set from two data is created by a three-dimensional morphing function using existing analysis software (for example, DTwin2 etc.), and similarly Of the created intermediate data, two are combined to further generate an average image. A three-dimensional average face can be created by setting the mixing ratio appropriately and performing morphing many times so that the contribution of each personal data is equal.

  Here, as preparation for performing morphing, information on the same location (correspondence as a position) is required between data. Here, FIG. 6 is a diagram illustrating a mesh region for performing morphing. In the present embodiment, a mesh region 48 having a predetermined number of points and a predetermined number of triangular meshes as shown in FIG. And the facial feature points.

  Note that each triangle of the mesh is placed on the same part on each face data. Basically, the finer the mesh, the more accurate the average face can be acquired. However, since the effort and time required for setting increases, it can be arbitrarily set according to the processing speed, the target accuracy, etc. .

<3D average face creation procedure>
Here, the 3D average face creation procedure will be described. The following processing can be performed by analysis software such as DTwin2, for example. First, the face three-dimensional shape data is imported, and in each three-dimensional face shape data, each point of the mesh having the triangular mesh number of 74 and the point number of 44 is set at a position where it is determined as the same position as in FIG. Note that mesh data is automatically set in the data created by morphing, so the above-described operation is not necessary.

  Next, for example, tertiary for a predetermined period (for example, before continuous use, after 4 weeks of continuous use, after 6 weeks of continuous use, etc.) for each predetermined group (classification) of each group of cosmetic use group and cosmetic + food use group Generate original average face.

  In the present embodiment, different morphing data can be generated by setting a mixing ratio of two data to be morphed. FIG. 7 is a diagram for explaining how the morphing result is changed by setting the mixing ratio.

  In the example of FIG. 7, the calculation result of the morphing created with the predetermined | prescribed mixing ratio is shown using the original data of the 20's woman and the 60's woman. A plurality of morphing data based on a predetermined condition (for example, age group or sex) can be easily generated based on the evaluation contents set in advance.

  Here, in the morphing process, only one data is obtained from two data. Therefore, the creation of the three-dimensional average face is performed by repeatedly setting the mixing ratio so that the remaining ratio of each personal data becomes equal and morphing is repeated. For example, for an average face of two people, the mixing ratio of 50:50 is completed once. If an average face of three people A, B, and C is to be created, morph processing is performed at a 50:50 mixing ratio of A and B in the first processing, and then the data is acquired. To do. Assuming that the morphing result of the data of A and B is AB, the average face of these three people is obtained by morphing AB and C at a mixing ratio of 66.7: 33.3 in the second processing. Can do. That is, it is preferable to perform the processing so that the ratio of each individual in the average face is equalized by the processing described above. In addition, various average faces can be acquired from the same original image by changing the ratio as necessary.

<3D average face generation result>
Here, FIG. 8 is a diagram illustrating an example of a generation result of the three-dimensional average face data. The example of FIG. 8 shows an example of an average face generated using three-dimensional face shape data of eight women of Japanese nationality, German nationality, and each nationality in their 20s and 50s. . In this way, various kinds of three-dimensional average face data can be generated by appropriately setting the above-described conditions such as the mixing ratio.

<Facial shape evaluation method>
Next, the face shape evaluation method in this embodiment will be described with reference to the drawings. FIG. 9 is a diagram illustrating an example of a face shape evaluation method according to the present embodiment. FIG. 9A shows a state of comparison in each point cloud data with the three-dimensional average face data, and FIG. 9B shows a flow of calculating a difference value.

  In the example of FIG. 9A, the 3D face shape data 51 before use such as cosmetics and massage and the 3D face shape data 52 after use are used, and the 3D face shape data before use and after use. The improvement is evaluated by obtaining the difference in the shape of the 3D face shape data. For example, the position of the data after use (pink) is matched with the data before use (green) based on preset facial feature points. Further, after the alignment, the aligned pre-data is output as point cloud data (csv format), and the data after continuous use is output in stl format.

  Here, in order to evaluate the shape change in the present embodiment, it is necessary to exactly match the position of the shape data after use with the shape data before use such as cosmetics and massage. Therefore, in the alignment function shown in FIG. 9A, translation or rotation is applied to the data after use with reference to the data before use to match the position and orientation of the data before use. To confirm that they match, the two data to be aligned are displayed three-dimensionally on the same screen, and the thickness of the soft tissue part such as the nose, forehead, and earlobe parts that are preset feature points The alignment data 53 is generated by aligning the two data so that there is no difference between the two data in a thin region where the change hardly occurs.

  Next, as shown in FIG. 9B, the output data is read into VX (CAD software) or the like, and a curved surface is created from the pre-use point group data. Further, the post-use point cloud data is obtained from the used stl data, and the distance from each point to the curved surface before use is obtained to obtain the difference value.

  Conventionally, when evaluating the facial shape change before and after use from the measured face shape data, there have been examples using the contour lines and the difference in cross-sectional shape so far, but in that case the contour shape before and after use If they are different, it can be recognized that the shape has changed, but a specific amount of change cannot be evaluated. In comparison of cross-sectional shapes, the amount of change in a cross-section can be evaluated, but the region that can be evaluated by one cross-section is only a linear range. However, the improvement change due to loosening or swelling causes a shape change in a wide range, and the distribution information of the position and the change amount is important, so the evaluation by the cross section is insufficient.

  Therefore, the error evaluation function shown in FIG. 9B is a method for evaluating the shape change of a wide area, and after matching two three-dimensional shapes to be compared at the same position, the difference between the two data for each place is calculated. Evaluate and color display according to the amount of difference. Here, as the direction for evaluating the difference in shape, the face has an egg-shaped curved shape, and the direction of shape change such as loosening or swelling (tightening or expansion) is close to the normal direction of the curved surface. If the direction of change for each location of the face is different, the area to be evaluated is somewhat wide, and the normal direction of the face shape is different for each location, the change in the normal direction of the curved surface of the face shape is evaluated It is good.

  This function can be analyzed using, for example, VX (manufactured by Machineware) of 3D CAD software. As a procedure, first, point cloud data of reference data (for example, data before using cosmetics or massage) is converted into curved surface data. Then, in order to compare this curved surface data with the point cloud data after use, using the error evaluation function, the normal direction difference (distance) from the curved surface data before use for each point of the point cloud data after use. Coloring according to).

  Note that the coloring shown in FIG. 9 is an example, and according to the present embodiment, by arbitrarily changing the coloring, blinking, brightness, etc. set in advance, it is arbitrarily emphasized corresponding to the degree of unevenness of the uneven portions. Can be displayed.

<Difference color area generation method>
Next, the difference color area generation method in the present embodiment will be described with reference to the drawings. FIG. 10 is a diagram illustrating a generation example of the difference color area. As shown in FIG. 10, visual data is acquired based on preset coloring conditions. Specifically, as shown in FIG. 10, for example, the screen can be set using a color bar or the like. In the present embodiment, the present invention is not limited to this, and RGB numerical values and names such as red, blue, and green can be directly input.

  In the difference color area generation in the present embodiment, each point of the continuous data is colored according to the distance from the curved surface before continuous use. As a specific procedure, first, point cloud data of reference data (for example, data before using cosmetics or massage) is converted into curved surface data. Then, in order to compare this curved surface data with the point cloud data after use, using the error evaluation function, the normal direction difference (distance) from the curved surface data before use for each point of the point cloud data after use. ) To obtain visual data. Next, as shown in FIG. 10, in the image processing, based on a preset difference value (change amount) threshold, a green area that does not change is deleted to generate a difference color. Note that the color setting content is not limited to the coloring method shown in FIG. 10 and can be arbitrarily set in the present embodiment.

<Evaluation by difference vector display>
Here, the evaluation based on the difference vector display will be described. When evaluating the change between two face shape data using differential color display, the distance between the faces of the two face shape data is evaluated, so the face surface such as a cheek being pulled up or hanging down Inward changes cannot be assessed. In order to evaluate these changes, it is necessary to identify the same position (corresponding point) between the face shape data to be compared, and obtain the difference as a difference vector for each direction of x, y, z.

  Here, what is important is a method for identifying corresponding points. Many of the three-dimensional face shape measurement apparatuses 11 used for face shape measurement are equipped with a camera, acquire an image obtained by scanning a laser beam or an image obtained by projecting a pattern on the face, and each pixel of this image Alternatively, three-dimensional coordinate data (x, y, z) is acquired at intervals of several pixels in order to reduce the data amount. Therefore, the 3D face shape measurement apparatus 11 can acquire face shape data and a face image composed of point clouds, but it is known which pixel of the face image corresponds to each point of the face shape data. is there.

  In this embodiment, for example, when searching for a corresponding point between two face shape data, when it is difficult to obtain a corresponding point directly from the face shape data, it is searched which pixel is the corresponding point using the face image data. Then, it is possible to know which point of the face shape data is the corresponding point from the pixel.

  Here, since there are various image matching methods for searching for corresponding points using a face image, for example, any one of them may be selected and used. For example, an image matching method based on feature points of a face is used. Can be used. By using the image matching method, it is possible to evaluate many points in the evaluation area at once.

  In this way, in the present embodiment, it is possible to obtain which point corresponds to which point on the face shape data, and obtain the difference as a difference vector for each direction of x, y, z. In this way, an evaluation that considers the direction of the change is performed, and the change is displayed with an arrow, so that the display can be easily understood visually.

<Differences from other methods>
As a method of displaying the amount of change with an arrow, there is one that uses an optical flow in an image (two-dimensional) to perform facial expression analysis, etc., and evaluates the direction and direction of the change with an arrow, but the change in the depth direction of the image Cannot be assessed accurately, so changes cannot be assessed accurately.

  In addition, as a method for evaluating the amount of change three-dimensionally, there is a method for attaching a marker by motion analysis or the like and evaluating the change in the position three-dimensionally, but only the marker installation point is evaluated. In addition, it is possible to increase the evaluation score by adding a lot of markers, but when using it for product information (pamphlets) etc., how many or tens of unnatural markers are attached to the face Visual data does not look good.

  Here, a procedure for implementing the above-described contents and a specific example will be described. As an example, a face shape is measured by a three-dimensional face shape measuring device in the daytime and evening and the change is evaluated.

  First, the measurement measures the face shape using a three-dimensional face shape measuring device. At this time, the image acquired by the camera of the apparatus and the face shape data are acquired in advance. In the above-described alignment, for example, the alignment function of analysis software such as DTwin2 is used to align the position of the evening face shape data with the daytime face shape data. Note that the measurement and alignment are the same as those described above.

Also, the setting of the movement amount analysis area designates an area to be evaluated on the face image, for example. For the area, a predetermined area such as a square or a circle is designated. Also, in setting the calculation conditions,
The number of pixels at which the evaluation interval (interval for displaying the arrow) is set is set by a preset calculation interval. Also, the window size is set as a setting at the time of image fitting execution.

  Next, the execution of the calculation and the result will be described. FIG. 11 is a diagram illustrating an example of the display result. 11A shows the evaluation subject and the movement amount analysis region, FIG. 11B shows a view of the evaluation subject viewed from the side, and FIG. 11C shows the evaluation subject. The figure which looked at from the front diagonal direction is shown.

  As shown in FIGS. 11B and 11C, in setting the window size for the movement amount analysis region shown in FIG. 11A, the calculation interval is analyzed every 15 pixels and the window size of the image fitting is 128 pixels. The execution result can be clearly displayed by an arrow or the like. In addition, the magnitude | size of the arrow shown to FIG.11 (b), (c) can be changed by changing a magnification. Note that the amount of change can be easily shown to the user or the like by changing the size or length of the arrow according to the amount of change.

<Volume and surface area evaluation function>
Next, the volume and surface area evaluation function in the present embodiment will be described with reference to the drawings. FIG. 12 is a diagram illustrating an example of an evaluation method for the volume and surface area evaluation function.

  The function in this embodiment can evaluate the volume change amount of the designated area | region before and behind use of a massage or cosmetics, for example, and can evaluate a surface area, for example using VX (CAD software). First, as shown in FIG. 12A, the procedure for evaluating the volume change amount designates a region 49 by surrounding a region or part to be evaluated with a straight line or a curve. Next, as shown in FIG. 12B, a roughly parallel plane is set for the enclosed region, a projection view for the specified region is created for this surface, and the normal direction of this surface is set. Create a three-dimensional shape with thickness. A copy of this solid is created for the number of face data (the positions are exactly the same), and the volume of the solid inside the face (FIG. 12 (c)) is calculated by cutting at a location that intersects the curved surface of each face data. Further, a value obtained by subtracting the volume value before use from the obtained volume value after use is the volume change amount in the designated region.

  The same procedure can be used for the evaluation of the surface area of the designated site. Specifically, first, as shown in FIG. 12A, an area to be evaluated is specified by surrounding it with a straight line, a curve or the like. Next, as shown in FIG. 12 (b), a roughly parallel plane is set for the enclosed region, a projection view of the specified region is created for this surface, and the normal method of this surface is created. Create a three-dimensional shape with thickness.

  Thereafter, this solid is cut at a location where it intersects the curved surface of the face data, and a solid inside the face (FIG. 12C) is obtained. Further, only the curved surface having the acquired three-dimensional face shape curved surface is left, all other surfaces (bottom surface, side surface, etc.) are deleted, and the surface area is calculated.

<Example 2: Example of generation of average face image by slack grade>
Here, an example of generating an average face image by slack grade will be described as another embodiment of the present invention. For example, when generating an average face image for evaluation related to facial slackness, a user or the like sets a slack grade by visual inspection or the like for a plurality of three-dimensional face shape data in advance, and sets the slack grade to the set slack grade. Based on this, three-dimensional average face data is generated.

  In this embodiment, 128 females in their 20s to 60s are contrasted, three-dimensional face shape data is obtained, and six determiners of the two-dimensional texture image of the left and right oblique sides collected at the same time The degree of slackness was scored with “not feeling” as “0” and “feeling clearly” as “6”.

  Also, five grades are set from the slack scores, eight persons are selected from each grade, and three-dimensional average face data is generated. By doing so, it is possible to create a grade-specific slack average face.

  It should be noted that the contour line change of the average facial image by slack grate created using the three-dimensional face shape data by applying this example, the central part of the cheek, face line, nose which is a part where slack due to aging is likely to appear Checked by focusing on the lip groove, corner of the mouth, and upper and lower eyelids, it was confirmed by the individual three-dimensional shape before averaging, sagging of the cheek center, face line droop, nose lip groove, wrinkle shape of each part It was possible to confirm the change in the contour line shape caused by.

<Evaluation results>
Next, specific evaluation results to which the above-described embodiment is applied will be described. Here, the creation content of the three-dimensional average face data used for the evaluation result will be described. FIG. 13 is a diagram illustrating an example of a morphing execution procedure in generating the three-dimensional average face data.

  Here, as the three-dimensional average face creation procedure, as described above, the face three-dimensional shape data is imported, and in each three-dimensional face shape data, the points of the meshes having the triangular mesh number of 74 and the score of 44 are described above. It is installed in a place where it is determined that it is the same position as shown in FIG. For each group of cosmetic use group and cosmetic + food use group, a three-dimensional average face is created before continuous use, after 4 weeks of continuous use, and after 6 weeks of continuous use.

<Monitor (subject)>
The monitor is a woman in their late 30s to 40s who is worried about slackness in the face line and submandibular, and has 33 relatively simple subjects who usually have 1 to 3 items such as lotions and emulsions to clean after washing their face. They were grouped as follows. In addition, the “cosmetics use group (hereinafter abbreviated as a cosmetics group as required)” performs “essence application + mask application” for 10 persons, and the “cosmetics + food use group (hereinafter referred to as necessary). "Abbreviated as cosmetics + food group.)" Was performed "essence application + mask application + drink intake" for 23 persons.

  The essence was used every day, and the mask and drink were used every day from the start of the test to the third day, and then the first week after the start of the test.

  Next, data used when creating a three-dimensional average face will be described. As for the use data at the time of creating the 3D average face, out of a total of 10 “cosmetics group”, 1 person who was absent after 4 weeks of continuous use and 1 person who had a large difference in facial expressions before and after continuous use were not used. An average face is created using data from a total of eight people. For “Cosmetics + Food Group”, out of a total of 23 people, 1 person absent from the measurement after 4 weeks of continuous use and 2 people who had a large difference in facial expressions before and after continuous use were not used. Use average face to create.

  As for the morphing mixture ratio and allocation at the time of creating the 3D average face, as shown in FIG. 13, select “Cosmetic Group” and “Cosmetic + Food Group” as two predetermined images like a tournament. Repeated morphing to create an average face. At this time, the mixing ratio at the time of each morphing process is set so that the contribution of each personal data becomes equal. As the average face, three average faces are created in each group before continuous use, after 4 weeks of continuous use, and after 6 weeks of continuous use.

<About measurement>
Measurements were taken before using the test product, 4 weeks after use, 6 weeks after use, and subject to daily fluctuations such as swelling in the morning. Measurement was performed. Regarding data analysis, analysis was performed using only data on the right half face side among data acquired from the apparatus.

<About analysis and evaluation>
The evaluation was performed in the order of the evaluation procedure in the present embodiment described above. However, with regard to the details of the analysis, first, when evaluating facial shape changes, the data with a large difference in facial expression between the data to be compared includes the shape difference caused by the difference in facial expressions, resulting in data with a large error. For this reason, when creating an average face for each group, those with a large difference in facial expression between the face shape data before and after using the test product were not adopted.

  Also, in the actual measurement, out of a total of 10 “cosmetics group”, 1 person who was absent after 4 weeks of use, and 1 person who had a large difference in facial expressions at the time of measurement before and after use were rejected, and data for a total of 8 people The average face used was created. Also, out of a total of 23 "cosmetics + food group", 1 person who was absent after 4 weeks of use and 2 persons who had a large difference in facial expression before and after use were not used. It was created.

  After the three-dimensional average face data created under the above-mentioned conditions are used in each group with reference to the data before use, the position of the data after use is adjusted, and then the error evaluation function in VX described above is used, and 4 weeks after use The analysis was conducted 6 weeks after use.

  After this, image processing was performed using a photoshop to make the data easier to understand visually. Here, FIG. 14 is a diagram for explaining a state of image processing in the present embodiment.

  First, an image displaying the difference color analysis result and the analyzed three-dimensional face shape is acquired (FIG. 14A). On this image, error evaluation analysis was performed only on the colored area. Next, the “green” area where there was little shape change was deleted. Furthermore, the “red” portion in FIG. 14 (a) only means a simple difference amount, and reflects the direction of whether the direction has been tightened after use or has changed to the expanded direction. Absent. Therefore, as shown in FIG. 14B, the data before use is “green”, the data after use is “pink”, the alignment is performed, and the superimposed result is displayed on the screen. Here, the area where “pink” is visible after use means that there is an expanded change, and the place where “green” before use means that there is a tightened change. ing.

  The result of identifying the tightening and expansion directions as shown in FIG. 14C is obtained by changing “yellow to red” to “light blue to blue” in the area where the tightening direction has changed. it can. Note that the above-described method of changing “yellow to red” to “light blue to blue” is performed by selecting an area to be changed using image processing software such as photoshop, and performing tone inversion after performing tone inversion. This can be done by inverting the green component in the curve. In addition, with a typical software rapidform, it is possible to obtain a result in consideration of directionality without performing image processing.

  Here, FIG. 15 is a figure which shows the result evaluated corresponding to each above-mentioned conditions. As shown in FIG. 15, by the above-described processing, the average face after 4 weeks of use and 6 weeks after use in each of the “cosmetics group” and “cosmetics + food group” based on the three-dimensional average face before use. We acquired visual data that shows what kind of changes have occurred.

  As shown in FIG. 15, in the difference color display by the average face of the “cosmetics group”, it can be seen that there is a tightening effect at the face line part. In addition, in the difference color display by the average face of “cosmetics + food group”, it can be seen that there is a tightening effect not only in the face line part but also in a wide area of the cheek.

  Next, an example of the volume and surface area evaluation results will be described with reference to the drawings. FIG. 16 is a diagram illustrating an example of a volume and surface area evaluation result. As shown in FIG. 16 (a), in the white region (the horizontal line from the nose to the vicinity of the rising line, the point where the perpendicular line and the face line intersect, and the face line and Mario from the point along the face line) The volume change of the area around the intersection of the extension line of the net line and the area surrounding the ridge line of the convex part on the ear side of the marionette line and the convex part on the outer side of the nasal lip groove) For example, it evaluated using the volume calculation function of VX mentioned above. Moreover, the area | region which changed was designated as shown in FIG.16 (b), and the surface area was calculated | required. Finally, the largest amount of change was determined.

Here, FIG. 17 is a diagram illustrating an example of analysis / evaluation results and questionnaire results. In the evaluation result shown in FIG. 17A, the “cosmetics group” had a tightening effect on the face line part. In addition, the “cosmetics + food group” had a tightening effect not only on the face line that was effective in the “cosmetic group” but also in a wide area of the cheeks, and the effect appeared in a wide range by the combined use of food. Furthermore, the effect after 6 weeks of continuous use was greater for both the “cosmetics group” and “cosmetics + food group” with the same trend as after 4 weeks of continuous use. Also, the questionnaire results shown in FIG. The subject was interviewed after 6 weeks of use, and the improvement rate was calculated for each site that had improved from the comments, but it was found that the rate of recognition of any improvement was high. Can be recognized as meaningful.

  As described above, according to the present invention, it is possible to capture a face in three dimensions and perform face evaluation with high accuracy based on the amount of change between two face images. Specifically, according to the present invention, the three-dimensional facial shape data before and after continuous use is compared, and analysis is performed to color a place where there has been tightening or expansion change. Can be generated and displayed.

  In addition, according to the present invention, a morphing technique is used to create a three-dimensional average face by superimposing data of a plurality of persons, and by comparing before and after continuous use, an average effect is generated in which part and how much. Can be evaluated.

  In addition, according to the present invention, since evaluation using a three-dimensional average face is possible, it is possible to acquire visual data with high objectivity, not a remarkable example of an individual. Further, since this data is not personal information, there is an advantage that handling of information is easier than a remarkable example of an individual.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10 Face shape evaluation system 11 3D face shape measuring device 12 Face shape evaluation device 21 Input means 22 Output means 23 Accumulation means 24 3D face shape data acquisition means 25 3D average face data generation means 26 Face shape evaluation means 27 Difference color Area generation means 28 Display screen generation means 29 Transmission / reception means 30 Control means 31 Input device 32 Output device 33 Drive device 34 Auxiliary storage device 35 Memory device 36 CPU
37 Network connection device 38 Recording medium 41 Opening portion 42 Imaging unit 43 Camera 44 Light source 45 Pattern grid 46 Three-dimensional face image 47 Three-dimensional average face data 48 Mesh region 49 region 51 Three-dimensional face shape data (pre-use image)
52 3D average face data (image after use)
53 Alignment data

Claims (17)

In the face shape evaluation apparatus that acquires the three-dimensional data of the face shape of the person to be evaluated and evaluates the face shape,
3D face shape data acquisition means for acquiring the 3D data of the face shape at multiple points;
A face for evaluating the face shape of the evaluation subject based on the three-dimensional face shape data obtained by the three-dimensional face shape data acquisition means and the three-dimensional average face data extracted based on preset conditions. Shape evaluation means;
Difference color area generation means for generating a color area in the face shape based on difference data between the 3D face shape data obtained by the face shape evaluation means and the 3D average face data;
Display screen generating means for generating a screen for displaying the color area generated by the difference color area generating means;
An output means for outputting a screen obtained by the display screen generating means.   Three-dimensional average face data generating means for generating the three-dimensional average face data from a plurality of three-dimensional face shape data extracted based on preset conditions from a plurality of face shape three-dimensional data for the plurality of evaluation subjects The face shape evaluation apparatus according to claim 1, comprising: The three-dimensional average face data generating means
The face shape evaluation apparatus according to claim 2, wherein two 3D face shape data selected from the plurality of 3D face shape data are synthesized by morphing processing using a predetermined ratio. The face shape evaluation means includes
The face shape evaluation apparatus according to claim 1, wherein an evaluation is performed on a predetermined part of the face of the evaluation target person. The face shape evaluation means includes
The three-dimensional face shape data or the three-dimensional average face data is displayed three-dimensionally on the same screen, and one of the data is used as reference data, and the thickness of the nose, forehead, earlobe region, and soft tissue portion changes thinly. Among the regions that are unlikely to occur, at least one position is generated so that there is no difference on the three-dimensional coordinate, and alignment data is generated by aligning the other data, and based on the generated alignment data 5. The face shape evaluation apparatus according to claim 1, wherein the evaluation is performed. The face shape evaluation means includes
The difference data on three-dimensional coordinates is generated at each point of the three-dimensional face shape data or the three-dimensional average face data to be evaluated with respect to the reference data based on the alignment data. Item 6. The face shape evaluation apparatus according to Item 5. The face shape evaluation means includes
When the evaluation part of the three-dimensional face shape data to be evaluated or the three-dimensional average face data is in a place corresponding to the outside of the face of the reference data, it is evaluated that the evaluation part is expanded. The face shape evaluation apparatus according to claim 6. The face shape evaluation means includes
When the evaluation part of the three-dimensional face shape data or the three-dimensional average face data to be evaluated is in a place corresponding to the inside of the face of the reference data, it is evaluated that the evaluation part is tightened The face shape evaluation apparatus according to claim 6. In the face shape evaluation method for acquiring the three-dimensional data of the face shape of the person to be evaluated and evaluating the face shape,
3D face shape data acquisition step for acquiring the face shape 3D data at multiple points;
The face that evaluates the face shape of the evaluation subject based on the three-dimensional face shape data obtained by the three-dimensional face shape data acquisition step and the three-dimensional average face data extracted based on preset conditions A shape evaluation step;
A difference color area generation step for generating a color area in the face shape based on difference data between the three-dimensional face shape data and the three-dimensional average face data obtained by the face shape evaluation step;
A display screen generation step for generating a screen for displaying the color area generated by the difference color area generation step;
An output step of outputting the screen obtained by the display screen generation step to an output means.   Three-dimensional average face data generation step for generating the three-dimensional average face data from a plurality of three-dimensional face shape data extracted based on preset conditions from a plurality of face shape three-dimensional data for the plurality of evaluation subjects The face shape evaluation method according to claim 9, further comprising: The three-dimensional average face data generation step includes
The face shape evaluation method according to claim 10, wherein two 3D face shape data selected from the plurality of 3D face shape data are synthesized by morphing processing using a predetermined ratio. The face shape evaluation step means includes
The face shape evaluation method according to any one of claims 9 to 11, wherein an evaluation is performed on a predetermined portion of the face of the evaluation subject. The face shape evaluation step includes
The three-dimensional face shape data or the three-dimensional average face data is displayed three-dimensionally on the same screen, and one of the data is used as reference data, and the thickness of the nose, forehead, earlobe region, and soft tissue portion changes thinly. In the region where the occurrence of the occurrence is difficult to occur, at least one position is generated by aligning other data so that there is no difference on the three-dimensional coordinate with respect to the reference data, and the generated position is generated. The face shape evaluation method according to claim 9, wherein the evaluation is performed based on the alignment data. The face shape evaluation step includes
The difference data on three-dimensional coordinates is generated at each point of the three-dimensional face shape data or the three-dimensional average face data to be evaluated with respect to the reference data based on the alignment data. Item 14. The face shape evaluation method according to Item 13. The face shape evaluation step includes
When the evaluation part of the three-dimensional face shape data or the three-dimensional average face data to be evaluated is in a place corresponding to the outside of the face of the face shape data as the reference data, the evaluation part is expanded. The face shape evaluation method according to claim 14, wherein: The face shape evaluation step includes
When the evaluation part of the three-dimensional face shape data to be evaluated or the three-dimensional average face data is in a place corresponding to the inside of the face of the face shape data as the reference data, the evaluation part is tightened The face shape evaluation method according to claim 14, wherein evaluation is performed.   A face shape evaluation program for operating a computer to execute the face shape evaluation method according to any one of claims 9 to 16.