JP2014199536A - Face model generating device, method for controlling face model generating device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a face model generating device which can reduce the working time required for generating a three dimensional model of a face based on a photographed face image.SOLUTION: A face model generating device 10 generates a face model which is a three dimensional model of a face based on a photographed face image. A feature point acquisition unit 34 acquires positions of a plurality of feature points corresponding to a prescribed facial part which is set in a face image 20. A face model generation unit 36 generates the face model based on the positions of the plurality of feature points.

Description

  The present invention relates to a face model generation device, a control method for a face model generation device, and a program.

  A three-dimensional model of a face is created based on a face image obtained by photographing a real person's face with a photographing apparatus. For example, when creating the face of a character object based on the face image as described above, the creator pastes the face image on the face portion of the character object and deforms the face portion of the character object to match the face image. Thus, the face of the character object was created.

JP 2012-101024 A

  However, conventionally, it takes time to perform the deformation work after the face image is pasted on the face portion of the character object.

  The present invention has been made in view of the above problems, and its purpose is a face model capable of reducing the work time required to create a three-dimensional model of a face based on a captured face image. A generation apparatus, a control method of a face model generation apparatus, and a program are provided.

  In order to solve the above-described problems, a face model generation device according to the present invention includes a face model generation device (10) that generates a face model that is a three-dimensional model of a face based on a captured face image. Feature point acquisition means (34) for acquiring the positions of a plurality of feature points corresponding to a predetermined part in the face set in the image, and generating the face model based on the positions of the plurality of feature points And a face model generation means (36).

  The face model device control method according to the present invention is the face model generation device (10) control method for generating a face model, which is a three-dimensional model of a face, based on a captured face image. A feature point acquiring step for acquiring the positions of a plurality of feature points corresponding to a predetermined part in the face, and a face model generating step for generating the face model based on the positions of the plurality of feature points It is characterized by including these.

  A program according to the present invention is a program for causing a computer to function as a face model generation device (10) that generates a face model, which is a three-dimensional model of a face, based on a captured face image. Feature point acquisition means (34) for acquiring the positions of a plurality of feature points corresponding to a predetermined part in the face set in the face image, and the face model based on the positions of the plurality of feature points It is a program for causing the computer to function as a face model generation means (36) to be generated.

  An information storage medium according to the present invention is a computer-readable information storage medium recording the above program.

  In the aspect of the invention, a plurality of bones may be set in the face model, and the face model generation unit (36) may determine the position of the plurality of bones based on the positional relationship between the plurality of feature points. A means for setting the size may be included.

  Moreover, in one aspect of the present invention, the plurality of feature points include a feature point corresponding to the center of the forehead or the eyebrows and a feature point corresponding to the chin, and the face model generation means (36) includes: Means for setting the positions or sizes of the plurality of bones based on the distance obtained based on the feature point corresponding to the central part of the forehead or between the eyebrows and the feature point corresponding to the jaw. Good.

  In the aspect of the invention, the plurality of feature points include a feature point corresponding to the left cheekbone and a feature point corresponding to the right cheekbone, and the face model generation means (36) includes the left cheekbone. Means for setting positions or sizes of the plurality of bones based on distances obtained based on the feature points corresponding to the right cheekbone and the feature points corresponding to the right cheekbone.

  In the aspect of the invention, the plurality of feature points correspond to a feature point corresponding to the left cheekbone, a feature point corresponding to the right cheekbone, a feature point corresponding to the left gillbone, and a right gillbone. The face model generation means (36) includes a distance obtained based on a feature point corresponding to the left cheekbone and a feature point corresponding to the right cheekbone and the left gill bone. Means for setting the positions or sizes of the plurality of bones based on a ratio between a feature point and a distance obtained based on the feature point corresponding to the right gill bone may be included.

  In the aspect of the invention, the face model includes an eyeball object representing an eyeball, and the plurality of feature points correspond to a feature point corresponding to the eye's head, a feature point corresponding to the corner of the eye, and an upper eyelid. Including the feature point and the feature point corresponding to the lower eyelid, the face model generation means (36), the position of the feature point corresponding to the eye corner, the position of the feature point corresponding to the outer corner of the eye, and the upper eyelid And a means for setting the position or size of the eyeball object based on the position of the feature point corresponding to and the position of the feature point corresponding to the lower eyelid.

  In the aspect of the invention, a bone for opening and closing the eyelid is set in the face model, the eyelid is opened and closed by rotating the bone, and the face model generating means (36) An upper limit of the rotation angle of the bone for opening and closing the eyelid is set based on the position of the feature point corresponding to the upper eyelid, the position of the feature point corresponding to the lower eyelid, and the position of the eyeball object Means may be included.

  In one aspect of the present invention, a bone corresponding to a cheekbone is set in the face model, and the plurality of feature points include a feature point corresponding to a left mouth corner, a feature point corresponding to a right mouth corner, and a lip The face model generation means (36) includes a feature point position corresponding to the left mouth corner, a feature point position corresponding to the right mouth corner, Means for setting the position of the bone corresponding to the cheekbone based on the position of the feature point corresponding to the central portion may be included.

  Further, in one aspect of the present invention, the image processing apparatus includes a face texture image generation unit (38) that generates a face texture image to be mapped to the face model based on the face image, and the face texture image generation unit (38) Means for adjusting the color of the face texture image based on the difference between the color of any one of the plurality of feature points and a predetermined color may be included.

  In the aspect of the invention, the face model generation means (36) adjusts the face model so that at least two of the plurality of feature points are located on a median line of the face model. It may be included.

  In the aspect of the invention, the plurality of feature points include a feature point corresponding to a central part of the forehead or the eyebrows, and a feature point corresponding to a central part of the lips or the nasal head, and the face model generating unit (36) the face model is such that a feature point corresponding to the central part of the forehead or between the eyebrows and a feature point corresponding to the central part of the lip or the nasal head are located on the median line of the face model. A means for adjusting may be included.

  In the aspect of the invention, the plurality of feature points include a feature point corresponding to a left mouth corner and a feature point corresponding to a right mouth corner, and the face model generating unit ( 36) means for adjusting the face model so that an intermediate point between the feature point corresponding to the left mouth corner and the feature point corresponding to the right mouth corner is located on the median line of the face model; It may be included.

  In the aspect of the invention, the plurality of feature points include a feature point corresponding to the left eye and a feature point corresponding to the right eye, and the face model generation unit (36) corresponds to the left eye. A means for adjusting the face model may be included so that an intermediate point between the feature point and the feature point corresponding to the right eye is located on the median line of the face model.

  In one aspect of the present invention, as a setting screen for setting the positions of the plurality of feature points by the user, a display in which a screen in which the plurality of feature points are displayed on the face image is displayed on a display unit. Control means (30) and change means (32) for changing the position of at least one of the plurality of feature points represented on the face image based on the user's operation may be included. . On the setting screen, the feature points may be connected by a straight line.

  In addition, in the above, in order to make an understanding of this invention easy, the code | symbol described in drawing is described in the bracket | parenthesis, However, This invention is not limited to the aspect of illustration by this.

  According to the present invention, it is possible to reduce the work time required for creating a three-dimensional model of a face based on a photographed face image.

It is a figure which shows an example of the hardware constitutions of the face model production | generation apparatus which concerns on embodiment of this invention. It is a figure which shows an example of a face image. It is a functional block diagram of a face model generation device. It is a figure which shows an example of a setting screen. It is a flowchart which shows an example of the process performed with a face model production | generation apparatus. It is a figure which shows an example of a basic face model. It is a flowchart which shows an example of the production | generation process of a face model. It is a flowchart which shows an example of the production | generation process of a face model. It is a figure for demonstrating the production | generation process of a face model. It is a figure for demonstrating the production | generation process of a face model. It is a figure for demonstrating the production | generation process of a face model. It is a figure for demonstrating the production | generation process of a face model. It is a figure for demonstrating the production | generation process of a face model. It is a figure for demonstrating the production | generation process of a face model. It is a figure for demonstrating the production | generation process of a face texture image. It is a figure for demonstrating the production | generation process of a face texture image.

  Hereinafter, examples of embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a hardware configuration of a face model generation apparatus according to an embodiment of the present invention. The face model generation apparatus 10 according to the present embodiment is realized by a computer such as a personal computer. As illustrated in FIG. 1, the face model generation device 10 according to the present embodiment includes a control unit 11, a storage unit 12, a communication unit 13, an operation unit 14, and a display unit 15.

  The control unit 11 includes, for example, one or a plurality of microprocessors, and executes processing according to an operating system and other programs. The storage unit 12 includes a main storage unit (for example, RAM) and an auxiliary storage unit (for example, a hard disk drive or a solid state drive).

  The communication unit 13 is an interface for connecting to a communication network. Programs and data are supplied to the face model generation apparatus 10 via a communication network. Note that the face model generation device 10 may include a component for reading a program or data stored in an information storage medium such as an optical disk or a memory card. Then, a program and data may be supplied to the face model generation device 10 via an information storage medium such as an optical disk or a memory card.

  The operation unit 14 is for a user to perform an operation. For example, when the face model generation device 10 is a personal computer, a mouse or keyboard provided in the personal computer corresponds to the operation unit 14. The display unit 15 is, for example, a liquid crystal display panel or an organic EL display, and displays a screen according to instructions from the control unit 11.

  The face model generation device 10 has a function of generating a face model, which is a three-dimensional face model, based on a face image photographed by a photographing device such as a digital camera. That is, the storage unit 12 of the face model generation device 10 stores a program for generating a face model based on a face image photographed by the photographing device. The above function is realized by executing this program.

  FIG. 2 shows an example of a face image used for generating a face model. As shown in FIG. 2, a square (or substantially square) face image 20 is used to generate a face model. In addition, the face model 20 is generated by using a face image 20 that is an image obtained by photographing a face from the front and whose straight line is straight (or substantially straight). The face image 20 may not be an image obtained by photographing the face from the front. For example, even an image obtained by photographing a face obliquely from the front can be used to generate a face model as long as the outlines of both eyes are reflected.

  FIG. 3 is a functional block diagram illustrating an example of functional blocks realized by the face model generation device 10. As illustrated in FIG. 3, the face model generation device 10 includes a setting screen display control unit 30, a feature point setting unit 32, a feature point acquisition unit 34, a face model generation unit 36, and a face texture image generation unit 38. These functional blocks are realized by the control unit 11. That is, the control unit 11 functions as these functional blocks when the control unit 11 executes processing according to a program.

  The setting screen display control unit 30 displays a setting screen for the user to set a plurality of feature points on the face image 20 on the display unit 15.

  FIG. 4 shows an example of the setting screen. As shown in FIG. 4, the face image 20 is displayed on the setting screen 40. The face image 20 displayed on the setting screen 40 is set to a predetermined size (for example, 256 × 256 pixels).

  On the setting screen 40, 23 feature points 42 </ b> A to 42 </ b> W are displayed in the face image 20. Hereinafter, the feature points 42A to 42W may be collectively referred to as “feature points 42”.

  The feature point 42A is a feature point to be set between the eyebrows. The feature point 42A may be a feature point to be set at the center of the forehead.

  Further, the feature points 42B to 42F are feature points corresponding to the left eye, and should be set so as to surround the left eye. Among these, the feature points 42B and 42C are feature points corresponding to the upper eyelid, and should be set so that the upper eyelid is divided into three. The feature point 42D is a feature point to be set at the top of the eye, and the feature point 42F is a feature point to be set at the corner of the eye. The feature point 42E is a feature point corresponding to the lower eyelid, and should be set so as to bisect the lower eyelid.

  Similarly, the feature points 42G to 42K are feature points corresponding to the right eye, and should be set so as to surround the right eye. Among these, the feature points 42G and 42H are feature points corresponding to the upper eyelid, and should be set so that the upper eyelid is divided into three. The feature point 42I is a feature point to be set at the top of the eye, and the feature point 42K is a feature point to be set at the corner of the eye. The feature point 42J is a feature point corresponding to the lower eyelid, and should be set so as to bisect the lower eyelid.

  The feature points 42L to 42N are feature points corresponding to the nose. Among these, the feature point 42M is a feature point to be set on the nasal head (nasal tip). The feature point 42L is a feature point to be set on the left nose wing, and the feature point 42N is a feature point to be set on the right nose wing.

  The feature points 42O to 42Q are feature points corresponding to the lips. Of these, the feature point 42P is a feature point to be set at the center of the lips. The feature point 42O is a feature point to be set at the left mouth corner (left end of the lips), and the feature point 42Q is a feature point to be set at the right mouth corner (right end of the lips).

  The feature point 42R is a feature point corresponding to the left cheekbone and should be set at a position where the left cheekbone protrudes. The feature point 42W is a feature point corresponding to the right cheekbone and should be set at a position where the right cheekbone protrudes. The feature points 42T and 42U are feature points to be set on the jaw. The feature point 42S is a feature point corresponding to the left gill bone and should be set between the left cheekbone and the jawbone. The feature point 42V is a feature point corresponding to the right gill bone, and should be set between the right cheekbone and the jawbone.

  In the setting screen 40, the feature points 42 are connected by a connecting line 44 (straight line). Specifically, the feature points 42 are connected to each other by the connecting line 44 for each same part or for each predetermined part group.

  In the setting screen 40 shown in FIG. 4, feature points 42 </ b> B to 42 </ b> F corresponding to the left eye are connected by a connecting line 44. In addition, feature points 42 </ b> G to 42 </ b> K corresponding to the right eye are connected by a connecting line 44. Furthermore, feature points 42 </ b> L to 42 </ b> N corresponding to the nose are connected by a connecting line 44. In addition, feature points 42 </ b> O to 42 </ b> Q corresponding to the lips are connected by a connecting line 44. Further, feature points 42 </ b> R to 42 </ b> W corresponding to the cheekbone, gillbone, and jaw are connected by a connecting line 44.

  By displaying such a connection line 44, the user can intuitively understand which part each feature point 42 corresponds to.

  The feature point setting unit 32 (changing unit) changes the position of at least one of the plurality of feature points 42 based on an operation performed by the user while the setting screen 40 is displayed.

  For example, when the initial position of the feature point 42A is not set between the eyebrows, the user moves the feature point 42A to the position between the eyebrows of the face image 20 by dragging and dropping the feature point 42A using a mouse or the like. The same applies to the other feature points 42B to 42W.

  The feature point setting unit 32 changes the position of at least one of the feature points 42A to 42W based on the above operation performed by the user.

  The feature point acquisition unit 34 acquires the positions of a plurality of feature points 42 set in the face image 20.

  The face model generation unit 36 generates a face model based on the positions of the plurality of feature points 42 acquired by the feature point acquisition unit 34. For example, the face model generation unit 36 sets the positions or sizes of a plurality of bones set in the face model based on the positional relationship between the plurality of feature points 42. Details of the function of the face model generation unit 36 will be described later (see FIGS. 7 and 8).

  The face texture image generation unit 38 generates a face texture image to be mapped to the face model based on the face image 20. Details of the function of the face texture image generation unit 38 will be described later (see step S215 in FIG. 8).

  Next, a process executed by the face model generation device 10 to generate a face model based on the face image 20 will be described. FIG. 5 is a flowchart showing an example of this processing. When the control unit 11 executes the process shown in FIG. 5 according to the program, the control unit 11 functions as the functional block shown in FIG.

  As shown in FIG. 5, first, the control unit 11 (setting screen display control unit 30) displays the setting screen 40 on the display unit 15 (S101). When the setting screen 40 is displayed, the control unit 11 (feature point setting unit 32) changes the position of the feature point 42 based on an operation performed by the user.

  When the setting of the feature point 42 is completed, the control unit 11 closes the setting screen 40 and displays the skin color selection screen on the display unit 15 (S102). The skin color selection screen is a screen for selecting a skin color. For example, in the face model generation device 10, a plurality of types of colors are prepared in advance as skin colors, and the user can select any one of the plurality of types of colors as the skin color.

  When the selection of the skin color is completed, the control unit 11 (the feature point acquisition unit 34 and the face model generation unit 36) closes the skin color selection screen and executes a face model generation process (S103).

  The face model generation apparatus 10 stores a basic face model. FIG. 6 shows an example of a basic face model. The face model 50 is a three-dimensional model of a face and includes a large number of polygons.

  In FIG. 6, reference numeral “52A” indicates the space between the eyebrows and corresponds to the feature point 42A in FIG. 4. Symbols “52B” and “52C” indicate the upper eyelid of the left eye, and correspond to the feature points 42B and 42C in FIG. 4, respectively. Reference numeral “52E” indicates the lower eyelid of the left eye and corresponds to the feature point 42E of FIG. Reference numerals “52D” and “52F” respectively indicate the eyes and corners of the left eye, and correspond to the feature points 42D and 42F in FIG. 4, respectively.

  Reference numerals “52G” and “52H” indicate the upper eyelid of the right eye, and correspond to the feature points 42G and 42H in FIG. 4, respectively. Reference numeral “52J” indicates the lower eyelid of the right eye and corresponds to the feature point 42J in FIG. Symbols “52I” and “52K” indicate the right eye corner and the corner of the eye, respectively, and correspond to the feature points 42I and 42K in FIG.

  Reference numeral “52M” indicates the nasal head and corresponds to the feature point 42M in FIG. Reference numeral “52L” indicates the left nose wing and corresponds to the feature point 42L in FIG. Although not shown in FIG. 6, the face model 50 also includes the right nose wing corresponding to the feature point 42N of FIG.

  Reference numeral 52P denotes the center of the lips, and corresponds to the feature point 42P in FIG. Reference numeral 52O denotes the left mouth corner (left end of the lips), which corresponds to the feature point 42O in FIG. Reference numeral 52Q indicates the right mouth corner (the right end of the lips) and corresponds to the feature point 42Q in FIG.

  Further, reference numeral “52R” indicates the left cheekbone and corresponds to the feature point 42R in FIG. Reference numeral “52S” indicates the left gill bone, which corresponds to the feature point 42S in FIG. Reference numerals “52T” and “52T” denote jaws and correspond to the feature points 42T and 42U in FIG. Although not shown in FIG. 6, the face model 50 includes the right cheekbone corresponding to the feature point 42W in FIG. 4 and the right gill bone corresponding to the feature point 42V in FIG.

  The symbol “54L” indicates the left eyeball, and the symbol “54R” indicates the right eyeball. Reference numeral “56” indicates the middle bone of the nose, and reference numeral “58” indicates the eyelid bone.

  A plurality of bones are set in the face model 50. Each bone is associated with at least a part of the vertices of the polygon constituting the face model 50. When the state of the bone (position and rotation angle) changes, the position of the vertex associated with the bone changes based on the change in the state of the bone. That is, the shape of the face model changes.

  In the following description, it is assumed that the X axis, the Y axis, and the Z axis as shown in FIG. 6 are set. The X axis is an axis corresponding to the horizontal direction, and the Y axis is an axis corresponding to the vertical direction. The Z-axis is an axis corresponding to the depth direction (the direction from the front face to the back face). In the following description, it is assumed that the X coordinate of the eyebrow space 51 of the face model 50 is zero.

  In the face model generation process (step S103), the position of the basic face model 50, the position and size of the bone, and the like are changed based on the feature points 42A to 42W set in the face image 20. A face model 50 corresponding to the face image 20 is generated.

  7 and 8 are flowcharts showing an example of face model generation processing (step S103). As shown in FIG. 7, first, the control unit 11 adjusts the scale of the feature point 42 (S201). In the face image 20 photographed by the photographing device, the face may appear large or may be small. In order to remove this influence, the process of step S201 is executed.

  FIG. 9 is a diagram for explaining the processing in step S201. In step S <b> 201, the control unit 11 adjusts the scale of the feature point 42 so as to match the basic face model 50.

  Specifically, the control unit 11 adjusts the scale of the feature point 42 based on the distance obtained based on the feature point 42A corresponding to the eyebrows and the feature points 42T and 42U corresponding to the chin. For example, the control unit 11 causes the difference (H) in the Y coordinate between the feature point 42A corresponding to the eyebrows and the midpoint 60 of the feature points 42T and 42U corresponding to the chin to be the basic value (HB). The scale of the feature point 42 is adjusted by enlarging or reducing the face image 20.

  Here, the basic value (HB) is a value corresponding to the length from the jaws 52T and 52U of the basic face model 50 to the eyebrows 52A. That is, this is a value corresponding to the length between the midpoint of the jaws 52T and 52U of the basic face model 50 and the eyebrows 52A. Therefore, in step S201, the scale of the feature point 42 is adjusted by matching the length between the feature point 42A and the midpoint 60 to the basic face model 50.

  In a step after step S201, the position of the face model 50, the position and size of the bone, and the like are set based on the feature points 42A to 42W whose scale has been adjusted. In the step after step S201, the process is executed on the assumption that the feature point 42A corresponds to the eyebrow space 52A of the face model 50. In the face image 20, the face may appear to be biased to either the left side or the right side. Has been removed.

  After step S201 is executed, the control unit 11 adjusts the positions and sizes of all bones included in the face model 50 based on the face type (S202).

  FIG. 10 is a diagram for explaining step S202. In step S202, the control unit 11 determines the positions of all bones included in the face model 50 based on the distance obtained based on the feature point 42R corresponding to the left cheekbone and the feature point 42W corresponding to the right cheekbone. Adjust the size.

  Specifically, the control unit 11 calculates the X coordinate difference (W) between the feature point 42R corresponding to the left cheekbone and the feature point 42W corresponding to the right cheekbone. And the control part 11 calculates the value (S) of a face type parameter by following formula (1).

  S = W / WB (1)

  In the above formula (1), “WB” represents a basic value. This basic value (WB) is a value corresponding to the width of the basic face model 50 (the length from the left cheekbone 52R to the right cheekbone). For this reason, the face type parameter (S) indicates whether or not the width is wider than that of the basic face model 50. That is, the case where the face type parameter (S) is greater than 1 indicates that the face has a wider width than the basic face model 50. The case where the face type parameter (S) is smaller than 1 indicates that the face has a narrower width than the basic face model 50 (in other words, a vertically long face).

  The control unit 11 adjusts the positions and sizes of all bones set in the face model 50 based on the face type parameter (S) calculated as described above. For example, when the face type parameter (S) is larger than 1, the positions and sizes of all bones are adjusted to adjust the face model 50 so that the face model has a wider width than the basic face model 50. Is done. On the other hand, when the face type parameter (S) is smaller than 1, all the face models 50 are adjusted so that the face model 50 has a narrower width than the basic face model 50 (in other words, a vertically long face). The position and size of the bone are adjusted.

  After step S202 is executed, the control unit 11 sets a bone position corresponding to the gill bone (the left gill bone 52S and the right gill bone) of the face model 50 (S203).

  FIG. 11 is a diagram for explaining step S203. In step S203, the control unit 11 corresponds to the distance obtained based on the feature point 42R corresponding to the left cheekbone and the feature point 42W corresponding to the right cheekbone, the feature point 42S corresponding to the left gillbone, and the right gillbone. The position of the bone corresponding to the gill bone (the left gill bone 52S and the right gill bone) is set based on the ratio to the distance obtained based on the feature point 42V.

  Specifically, first, the control unit 11 calculates the difference (W1) in the X coordinate between the feature point 42R corresponding to the left cheekbone and the feature point 42W corresponding to the right cheekbone. Further, the control unit 11 calculates an X coordinate difference (W2) between the feature point 42S corresponding to the left gill bone and the feature point 42V corresponding to the right gill bone. Furthermore, the control part 11 calculates those ratios (r) by following formula (2).

  Thereafter, the control unit 11 determines the X coordinate (x1) of the bone corresponding to the left gill bone 52S. The X coordinate (x1) is calculated by the following equation (3). In the following formula (3), “Xr” is the X coordinate of the feature point 42R corresponding to the left cheekbone.

  Further, the control unit 11 determines the X coordinate (x2) of the bone corresponding to the right gill bone. The X coordinate (x2) is calculated by the following equation (4). In the following formula (4), “Xw” is the X coordinate of the feature point 42W corresponding to the right cheekbone.

  Further, the control unit 11 determines the Y coordinate (y) of the bone corresponding to each of the left gill bone 52S and the right gill bone. The Y coordinate (y) is calculated by the following equation (5). In the following formula (5), “Ys” is the Y coordinate of the feature point 42S corresponding to the left gill bone, and “Yv” is the Y coordinate of the feature point 42V corresponding to the right gill bone.

r = W2 / W1 (2)
x1 = Xr * r (3)
x2 = Xw * r (4)
y = (Ys + Yv) / 2 (5)

  After step S203 is executed, the control unit 11 sets the positions of the left eye, right eye, nose, lips, and chin of the face model 50 (S204). Since these parts are parts in the center of the face, even if the face appears in the face image 20 with a slight inclination, it is not so much affected. For this reason, the control part 11 sets the position of the feature point 42 corresponding to these parts as the position of these parts.

  For example, the control unit 11 determines the positions of the feature points 42D and 42F corresponding to the left eye corner and the corner of the eye as the positions of the left eye corner 52D and the corner 52F. Further, the control unit 11 determines the positions of the feature points 42B and 42C corresponding to the upper eyelid of the left eye as the positions of the upper eyelids 52B and 52C of the left eye, and sets the position of the feature point 42E corresponding to the lower eyelid of the left eye to the lower eyelid 52E of the left eye. Determine as the position.

  In addition, the control unit 11 determines the positions of the feature points 42I and 42K corresponding to the right eye corner and the corner of the eye as the positions of the right eye corner 52I and the eye corner 52K. Further, the control unit 11 determines the positions of the feature points 42G and 42H corresponding to the upper eyelid of the right eye as the positions of the upper eyelids 52G and 52H of the left eye, and sets the position of the feature point 42J corresponding to the lower eyelid of the right eye to the lower eyelid 52J of the right eye. Determine as the position.

  In addition, the control unit 11 determines the position of the feature point 42M corresponding to the nasal head as the position of the nasal head 52M. Further, the control unit 11 determines the position of the feature point 42L corresponding to the left nose wing as the position of the left nose wing 52L, and determines the position of the feature point 42N corresponding to the right nose wing as the position of the right nose wing.

  The control unit 11 determines the position of the feature point 42P corresponding to the center part of the lips as the position of the center part 52P of the lips. Further, the control unit 11 determines the position of the feature point 42O corresponding to the left mouth corner as the position of the left mouth corner 52O, and determines the position of the feature point 42Q corresponding to the right mouth corner as the position of the right mouth corner 52Q.

  Further, the control unit 11 determines the positions of the feature points 42T and 42U corresponding to the jaws as the positions of the jaws 52T and 52U.

  After step S204 is executed, the control unit 11 sets bone positions corresponding to the eyelid 58, cheekbone (left cheekbone 52R and right cheekbone), and nasal middle bone 56A of the face model 50 (S205).

  First, the control unit 11 sets a bone position corresponding to the eyelid bone 58. FIG. 12 is a diagram for explaining a method for determining the position of the bone corresponding to the eyelid bone 58. As shown in FIG. 12, the control unit 11 calculates the position of the bone corresponding to the left eyelid bone 58 based on the feature point 42F corresponding to the left eye corner and the feature point 42R corresponding to the left cheekbone. To do.

  That is, the X coordinate (x) of the bone corresponding to the left eye heel bone 58 is calculated by the following equation (6). In the following equation (6), “Xf” is the X coordinate of the feature point 42F corresponding to the corner of the left eye, and “Xr” is the X coordinate of the feature point 42R corresponding to the left cheekbone. According to the following equation (6), the position of the bone corresponding to the left eye heel bone 58 is a position between the feature point 42F corresponding to the left eye corner and the feature point 42R corresponding to the left cheekbone, The position is set closer to the feature point 42R.

  Further, the Y coordinate (y) of the bone corresponding to the left eye heel bone 58 is calculated by the following equation (7). In the following equation (7), “Yf” is the Y coordinate of the feature point 42F corresponding to the corner of the left eye. According to the following equation (7), the position of the bone corresponding to the left eyelid bone 58 is set to a position slightly below the feature point 42F corresponding to the eye corner of the left eye.

x = Xf + (Xr−Xf * 0.6) (6)
y = Yf−0.2 (7)

  Note that the position of the bone corresponding to the right eye heel bone 58 is also calculated in the same manner as the bone corresponding to the left eye heel bone 58.

  In step S205, the control unit 11 sets a bone position corresponding to the cheekbone (the left cheekbone 52R and the right cheekbone). For example, the control unit 11 sets the position of the feature point 42R corresponding to the left cheekbone as the position of the bone corresponding to the left cheekbone 52R, and sets the position of the feature point 42W corresponding to the right cheekbone to the position of the bone corresponding to the right cheekbone Set as.

  Further, in step S205, the control unit 11 sets a bone position corresponding to the nasal bone 56. FIG. 13 is a diagram for explaining a bone position setting method corresponding to the nasal middle bone 56. As illustrated in FIG. 13, the control unit 11 calculates the position of the bone corresponding to the nasal bone 56 based on the feature point 42A corresponding to the eyebrow and the feature point 42M corresponding to the nasal head.

  That is, the X coordinate of the bone corresponding to the nasal bone 56 is calculated by the following equation (8). In the following formula (8), “Xa” is the X coordinate of the feature point 42A, and “Xm” is the X coordinate of the feature point 42M.

  Further, the Y coordinate of the bone corresponding to the nasal bone 56 is calculated by the following equation (9). In the following formula (9), “Ya” is the Y coordinate of the feature point 42A, and “Ym” is the Y coordinate of the feature point 42M. According to the following equation (9), the position of the bone corresponding to the nasal bone 56 is a position between the feature point 42A corresponding to the eyebrow and the feature point 42M corresponding to the nasal head, and slightly closer to the feature point 42M. The position is set.

x = (Xa + Xm) / 2 (8)
y = ((Ya + Ym) / 2) −0.3 (9)

  After step S205 is executed, the control unit 11 sets the positions and sizes of the eyeballs 54L and 54R (eyeball objects) of the face model 50 (S206).

  For example, the control unit 11 calculates the position of the eyeball 54L of the left eye based on the feature points 42B to 42F corresponding to the left eye.

  That is, the X coordinate (x) of the eyeball 54L is calculated by the following equation (10). In the following formula (10), “Xd” is the X coordinate of the feature point 42D corresponding to the top of the eye, and “Xf” is the X coordinate of the feature point 42F corresponding to the corner of the eye.

  Further, the Y coordinate (y) of the eyeball 54L is calculated by the following equation (11). In the following formula (11), “Yb” and “Yc” are the Y coordinates of the feature points 42B and 42C corresponding to the upper eyelid, and “Ye” is the Y coordinate of the feature point 42E corresponding to the lower eyelid. .

x = ((Xd + Xf) / 2) −0.1 (10)
y = ((((Yb + Yc) / 2) + Ye) /2+0.2 (11)

  Similarly, the control unit 11 calculates the position of the right eyeball 54R based on the feature points 42G to 42K corresponding to the right eye.

  For example, the X coordinate (x) of the eyeball 54R is calculated by the following equation (12). In the following formula (12), “Xi” is the X coordinate of the feature point 42I corresponding to the top of the eye, and “Xk” is the X coordinate of the feature point 42K corresponding to the corner of the eye.

  For example, the Y coordinate (y) of the eyeball 54R is calculated by the following equation (13). In the following equation (13), “Yg” and “Yh” are the Y coordinates of the feature points 42G and 42H corresponding to the upper eyelid, and “Yj” is the Y coordinate of the feature point 42J corresponding to the lower eyelid. .

x = ((Xi + Xk) / 2) +0.1 (12)
y = (((Yg + Yh) / 2) + Yj) /2+0.2 (13)

  After step S206 is executed, the control unit 11 corrects the positions of bones corresponding to the jaws 52T and 52U of the face model 50 (S207). Specifically, the Y coordinate (y) of the bone corresponding to the jaws 52T and 52U is set as in the following formula (14). In the following formula (14), “Yt” and “Yu” are the Y coordinates of the feature points 42T and 42U corresponding to the jaws.

  y = (Yt + Yu) / 2 (14)

  After step S207 is executed, as shown in FIG. 8, the control unit 11 sets the width of the nose of the face model 50 (S208). For example, the control unit 11 calculates the width (w) of the nose according to the following formula (15). In the following equation (15), “Xl” is the X coordinate of the feature point 42L corresponding to the left nose wing, and “Xn” is the X coordinate of the feature point 42N corresponding to the right nose wing.

  w = (X1-Xn) /3.3 (15)

  After step S208 is executed, the control unit 11 sets the lip width of the face model 50 (S209). For example, the control unit 11 calculates the lip width (w) by the following equation (16). In the following equation (16), “Xo” is the X coordinate of the feature point 42O corresponding to the left mouth corner, and “Xq” is the X coordinate of the feature point 42Q corresponding to the right mouth corner.

  w = (Xo−Xq) /4.26 (16)

  After step S209 is executed, the control unit 11 adjusts the position of the bone corresponding to the cheekbone (left cheekbone 52R or right cheekbone) of the face model 50 in consideration of the height of the mouth corner (S210). In step S210, the control unit 11 adjusts the position of the bone corresponding to the cheekbone (the left cheekbone 52R or the right cheekbone) based on the positions of the feature points 42O to 42Q corresponding to the lips.

  First, the control unit 11 calculates the correction value (Δy) of the Y coordinate of the bone corresponding to the cheekbone by the following equation (17). In the following equation (17), “Yo” is the Y coordinate of the feature point 42O corresponding to the left mouth corner, and “Yq” is the Y coordinate of the feature point 42Q corresponding to the right mouth corner. “Yp” is the Y coordinate of the feature point 42P corresponding to the center of the lips. Then, as shown in the following formula (18), the control unit 11 adds the correction value (Δy) to the Y coordinate (y) of the bone corresponding to the cheekbone, thereby obtaining the Y coordinate (y) of the bone corresponding to the cheekbone. Correct.

Δy = (((Yt + Yq) / 2) −Yp) * 2 (17)
y = y + Δy (18)

  After step S210 is executed, the control unit 11 sets the position of the bone corresponding to the rib of the face model 50 (S211). That is, the control unit 11 sets the position of the bone corresponding to the rib on the left eye side based on the position and size of the eyeball 54L of the left eye, and sets the position of the bone corresponding to the rib on the right eye side of the right eyeball 54R. Set based on position and size. In this case, the Z coordinate of the bone corresponding to the rib on the left eye side is adjusted along the eyelid based on the size of the eyeball 54L. Similarly, the Z coordinate of the bone corresponding to the rib on the right eye side is adjusted along the eyelid based on the size of the eyeball 54R.

  After step S211 is executed, the control unit 11 sets the Z coordinate of each part or each bone of the face model 50 (S212).

  Since the face image 20 is a two-dimensional image, the Z coordinate of each part or each bone cannot be set based on the feature points 42A to 42W. In this regard, in the face model generation apparatus 10, a plurality of types of sample face models with different facial irregularities and occipital shape are prepared in advance. In step S212, one of the plurality of types of sample face models is selected, and the Z coordinate of each part or each bone of the selected sample face model is set as the Z coordinate of each part or each bone of the face model 50. Set as

  After step S212 is executed, the control unit 11 sets the bone position and rotation angle for opening and closing the eyelid of the face model 50 (S213).

  FIG. 14 is a diagram for explaining the processing in step S213. FIG. 14 shows a bone 70 for opening and closing the heel. As shown in FIG. 14, a bone 70 for opening and closing the eyelid is set in the face model 50. The upper collar 72 sticks to the lower collar 74 by rotating the bone 70 by a predetermined angle (θ).

  The control unit 11 sets the position and rotation angle (θ) of the bone 70 for opening and closing the eyelid of the left eye based on the positions of the eyeball 54L, the upper eyelids 52B and 52C, and the lower eyelid 52E of the face model 50. To do. That is, the control unit 11 determines the position and rotation angle (θ) of the bone 70 so that the upper collar 72 sticks to the lower collar 74 when the bone 70 rotates by a predetermined rotation angle (θ). Set.

  A bone for opening and closing the eyelid of the right eye is also set in the face model 50, and the position and rotation angle of this bone are set in the same manner as the bone 70 for opening and closing the eyelid of the left eye.

  After step S213 is executed, the control unit 11 corrects the bone positions corresponding to the jaws 52T and 52U of the face model 50 (S214). That is, the control unit 11 moves the bones corresponding to the jaws 52T and 52U according to the feature point 42P corresponding to the central part of the lips.

  After step S214 is executed, the control unit 11 (face texture image generation unit 38) generates a face texture image (S215). 15 and 16 are diagrams for explaining generation of a face texture image.

  First, the control unit 11 arranges the face image 20 on the face model 50 so that the feature points 42 </ b> A of the face image 20 coincide with the eyebrow space 52 </ b> A of the face model 50. In this state, the control unit 11 generates a UV map. As shown in FIG. 15, the control unit 11 generates an image 80 formed by parallel projection of the face image 20 arranged on the face model 50.

  Thereafter, as shown in FIG. 16, the control unit 11 generates a face texture image 84 by superimposing the skin frame 82 on the image 80. Note that a distorted face is drawn in the actual image 80, but in FIG. 16, a non-distorted face is drawn in the image 80 for the sake of simplicity of explanation.

  The skin frame 82 serves as a mask to be superimposed on the image 80. The skin frame 82 includes a skin region 90, a transparent region 92 (shaded portion), a heel region 94, a nostril region 96, and a lip region 98. The skin region 90 is a region indicating skin, and the color of the skin region 90 is set to the color selected on the skin color selection screen. The transparent area 92 is an area set to be transparent. The transparent region 92 has a shape corresponding to the face portion.

  The wrinkle region 94 is set to the skin color in the same manner as the skin region 90. Due to the presence of the wrinkle region 94, a skin color is added to the wrinkle portion in the face texture image 84 formed by superimposing the skin frame 82 on the image 80. The face image 20 showing the face with open eyes shows almost no wrinkles, but the presence of the wrinkle region 94 allows the face texture image with the wrinkles (in other words, the face with the eyes slightly closed). Texture image) can be generated.

  The nostril region 96 is set to the skin color in the same manner as the skin region 90. Due to the presence of the nostril region 96, a skin color is given to the nostril portion in the face texture image 84 formed by superimposing the skin frame 82 on the image 80.

  The lip area 98 is set to the color of the lips. In the face image 20, the mouth may be slightly open, but in the face texture image 84 formed by superimposing the skin frame 82 on the image 80 due to the presence of the lip region 98, the slightly open mouth portion is the lip region. 98 will be overwritten.

  Note that the control unit 11 adjusts the color of the image 80. That is, the control unit 11 changes the color of the image 80 based on the difference between any one of the feature points 42A to 42W of the face image 20 and the predetermined color set in the skin region 90 of the skin frame 82. adjust. For example, the control unit 11 calculates the average value of the colors of the feature point 42A and one or more points in the vicinity thereof. Further, the control unit 11 calculates a difference value between the color and a predetermined color set in the skin region 90 of the skin frame 82. Then, the control unit 11 adds the difference value to the color set in the image 80.

  The face image 20 may be too bright, too dark, too reddish, or greenish depending on the shooting environment. By performing the above adjustment, the color of the image 80 is changed to the skin area of the skin frame 82. It will be adjusted to 90 colors. As a result, the color of the face texture image 84 matches the color of the skin region 90 of the skin frame 82.

  After step S215 is executed, the control unit 11 adjusts the median line of the face model 50 (S216). In step S216, for example, the control unit 11 adjusts the median line of the face model 50 based on at least two of the feature points 42A to 42W. Specifically, the face model 50 is adjusted such that at least two of the feature points 42 </ b> A to 42 </ b> W are positioned on the median line of the face model 50.

  For example, the control unit 11 matches the X coordinates of the feature point 42P corresponding to the center of the lips and the feature point 42M corresponding to the nasal head with the X coordinate of the feature point 42A corresponding to the eyebrow, thereby causing the feature points 42A, The face model 50 is adjusted so that 42M and 42P are positioned at the midline of the face model 50. In other words, the control unit 11 matches the X coordinate of the nasal head 52M, the nasal bone 56, and the center part 52P of the lip with the X coordinate of the glazing 52A, thereby causing the eyebrow 52A, the nasal head 52M, the nasal bone 56, And the face model 50 is adjusted so that the center part 52P of the lips is located at the median line of the face model 50.

  For example, the control unit 11 matches the X coordinate of the midpoint between the feature point 42O corresponding to the left mouth corner and the feature point 42Q corresponding to the right mouth corner with the X coordinate of the feature point 42A corresponding to the eyebrow. The face model 50 is adjusted so that the midpoint of the feature points 42O and 42Q and the feature point 42A are located on the median line of the face model 50. In other words, the control unit 11 matches the X coordinate of the left mouth corner 52O and the right mouth corner 52Q of the lips (the midpoint between the bones set at both ends of the lips) with the X coordinate of the lip 52A, thereby The face model 50 is adjusted so that the midpoint of the left mouth corner 52O and the right mouth corner 52Q and the eyebrow space 52A are positioned on the median line of the face model 50.

  Furthermore, the control unit 11 matches the X coordinate of the midpoint between the feature point 42D corresponding to the left eye and the feature point 42I corresponding to the right eye with the X coordinate of the feature point 42A corresponding to the eyebrow. Thus, the face model 50 is adjusted so that the midpoint of the feature points 42D and 42I and the feature point 42A are located on the median line of the face model 50. In other words, the control unit 11 determines the midpoint between the left eye's eye 52D and the right eye's eye 52I (the midpoint between the bone set at the left eye's eye 52D and the bone set at the right eye's eye 52I). By matching the coordinates with the X coordinate of the eyebrows 52 </ b> A, the face model 50 is adjusted so that the midpoint between the left eye eye 52 </ b> D and the right eye eye 52 </ b> I and the eyebrows 52 </ b> A are located at the midline of the face model 50. By executing step S216, the median line of the face model 50 is adjusted.

  Thus, the processing shown in FIGS. 7 and 8 ends. That is, the process of step S103 in FIG. After step S103 (FIGS. 7 and 8) is executed, the control unit 11 stores the data of the face model 50, the face texture image 84, and the UV map in the storage unit 12 (S104).

  According to the face model generation device 10 described above, the face model 50 can be created based on the face image 20 photographed by the photographing device. In particular, according to the face model generation device 10, the user can create a face model 50 corresponding to the face image 20 by setting a plurality of feature points 42 on the face image 20. That is, according to the face model generation device 10, it is possible to reduce the work time necessary for creating the face model 50 corresponding to the face image 20.

  The present invention is not limited to the embodiment described above.

  DESCRIPTION OF SYMBOLS 10 Face model production | generation apparatus, 11 Control part, 12 Storage part, 13 Communication part, 14 Operation part, 15 Display part, 20 Face image, 30 Setting screen display control part, 32 Feature point setting part. 34 feature point acquisition unit, 36 face model generation unit. 38 face texture image generation unit, 40 setting screen, 42A to 42W feature point, 44 connecting line, 50 face model, 52A eyebrow, 52B, 52C, 52G, 52H upper eyelid, 52D, 52I eyelid, 52E, 52J lower eyelid, 52F , 52K Eye corner, 52L Left nose wing, 52M Nasal head, 52O Left mouth corner, 52P Center, 52Q Right mouth corner, 52R Left cheekbone, 52S Left gill bone, 52T, 52U Jaw, 54L, 54R Eyeball, 56 Nasal bone, 58 eyes Shoebone, 70 bones, 72 upper eyelids, 74 lower eyelids, 80 images, 82 skin frames, 84 facial texture images, 90 skin regions, 92 transparent regions, 92 heel regions, 94 nostril regions, 96 lip regions.

Claims (16)

In a face model generation device that generates a face model that is a three-dimensional model of a face based on a photographed face image,
Feature point acquisition means for acquiring the positions of a plurality of feature points corresponding to a predetermined part in the face set in the face image;
Face model generation means for generating the face model based on the positions of the plurality of feature points;
A face model generation apparatus characterized by comprising: The face model generation device according to claim 1,
A plurality of bones are set in the face model,
The face model generation means includes means for setting positions or sizes of the plurality of bones based on a positional relationship between the plurality of feature points.
A face model generation device characterized by that. The face model generation device according to claim 2,
The plurality of feature points include a feature point corresponding to the center of the forehead or the eyebrows, and a feature point corresponding to the chin,
The face model generation means determines the positions or sizes of the plurality of bones based on distances obtained based on a feature point corresponding to a central portion of the forehead or between the eyebrows and a feature point corresponding to the jaw. Including means for setting,
A face model generation device characterized by that. The face model generation device according to claim 2 or 3,
The plurality of feature points include a feature point corresponding to the left cheekbone and a feature point corresponding to the right cheekbone,
The face model generating means is means for setting positions or sizes of the plurality of bones based on a distance obtained based on a feature point corresponding to the left cheekbone and a feature point corresponding to the right cheekbone. including,
A face model generation device characterized by that. The face model generation device according to claim 4,
The plurality of feature points include a feature point corresponding to the left cheekbone, a feature point corresponding to the right cheekbone, a feature point corresponding to the left gillbone, and a feature point corresponding to the right gillbone,
The face model generation means corresponds to the distance obtained based on the feature point corresponding to the left cheekbone and the feature point corresponding to the right cheekbone, the feature point corresponding to the left gillbone, and the right gillbone Means for setting the position or size of the plurality of bones based on a ratio of a distance obtained based on the feature points;
A face model generation device characterized by that. The face model generation device according to any one of claims 1 to 5,
The face model includes an eyeball object representing an eyeball,
The plurality of feature points include a feature point corresponding to the eye corner, a feature point corresponding to the corner of the eye, a feature point corresponding to the upper eyelid, and a feature point corresponding to the lower eyelid,
The face model generation means includes a position of a feature point corresponding to the eye corner, a position of a feature point corresponding to the corner of the eye, a position of a feature point corresponding to the upper eyelid, and a feature point corresponding to the lower eyelid. And means for setting the position or size of the eyeball object based on the position,
A face model generation device characterized by that. The face model generation device according to claim 6,
In the face model, a bone for opening and closing the eyelid is set, and when the bone rotates, the eyelid opens and closes,
The face model generation means sets the upper limit of the rotation angle of the bone for opening and closing the eyelid, the position of the feature point corresponding to the upper eyelid, the position of the feature point corresponding to the lower eyelid, and the eyeball object And means for setting based on the position,
A face model generation device characterized by that. The face model generation device according to any one of claims 1 to 7,
In the face model, a bone corresponding to the cheekbone is set,
The plurality of feature points include a feature point corresponding to the left mouth corner, a feature point corresponding to the right mouth corner, and a feature point corresponding to the center of the lips,
The face model generation means, based on the position of the feature point corresponding to the left mouth corner, the position of the feature point corresponding to the right mouth corner, and the position of the feature point corresponding to the center of the lips, Including means for setting the position of the bone corresponding to the cheekbone,
A face model generation device characterized by that. The face model generation device according to any one of claims 1 to 8,
A face texture image generating means for generating a face texture image to be mapped to the face model based on the face image;
The face texture image generation means includes means for adjusting the color of the face texture image based on a difference between a color of any one of the plurality of feature points and a predetermined color.
A face model generation device characterized by that. The face model generation device according to any one of claims 1 to 9,
The face model generation means includes means for adjusting the face model so that at least two of the plurality of feature points are located on a median line of the face model.
A face model generation device characterized by that. The face model generation device according to claim 10,
The plurality of feature points include a feature point corresponding to the center part of the forehead or the eyebrows, and a feature point corresponding to the center part of the lips or the nasal head,
The face model generation means is configured to make the face point such that a feature point corresponding to the center part of the forehead or the eyebrow and a feature point corresponding to the center part of the lip or the nasal head are located on the median line of the face model. Including means to adjust the model,
A face model generation device characterized by that. The face model generation device according to claim 10,
The plurality of feature points include a feature point corresponding to a left mouth corner and a feature point corresponding to a right mouth corner,
The face model generation unit adjusts the face model so that an intermediate point between the feature point corresponding to the left mouth corner and the feature point corresponding to the right mouth corner is located on the median line of the face model. Including means to
A face model generation device characterized by that. The face model generation device according to claim 10,
The plurality of feature points include a feature point corresponding to the left eye and a feature point corresponding to the right eye,
The face model generation means adjusts the face model so that an intermediate point between the feature point corresponding to the left eye and the feature point corresponding to the right eye is located on the median line of the face model. including,
A face model generation device characterized by that. The face model generation device according to any one of claims 1 to 13,
Display control means for displaying on the display means a screen in which the plurality of feature points are displayed on the face image as a setting screen for the user to set the positions of the plurality of feature points;
Changing means for changing the position of at least one of the plurality of feature points represented on the face image based on the operation of the user,
In the setting screen, feature points are connected with a straight line,
A face model generation device characterized by that. In a control method of a face model generation device that generates a face model which is a three-dimensional face model based on a photographed face image
A feature point obtaining step for obtaining positions of a plurality of feature points corresponding to a predetermined part in the face set in the face image;
A face model generation step for generating the face model based on the positions of the plurality of feature points;
A control method for a face model generation device, comprising: A program for causing a computer to function as a face model generation device that generates a face model that is a three-dimensional model of a face based on a photographed face image,
Feature point acquisition means for acquiring the positions of a plurality of feature points corresponding to a predetermined part in the face set in the face image; and
Face model generation means for generating the face model based on the positions of the plurality of feature points;
A program for causing the computer to function as

Images