JP2021049082A - Image processing device, image processing method, and program - Google Patents

Abstract

To provide a method for wearing makeup for appropriately expressing an impression desired by a user.SOLUTION: A display unit 104 displays a captured image in which a user's face is imaged. A feature specification unit 105 specifies a feature of the user's face on the basis of the captured image. A makeup content determination unit 110 determines makeup contents for expressing a target impression on the basis of a target impression which is an impression of a target expressed by the makeup of the face and the feature of the user's face. A display control unit 103 causes the display unit 104 to display a makeup image obtained by processing the captured image according to the makeup contents.SELECTED DRAWING: Figure 4

Description

The present invention relates to an image processing apparatus, an image processing method and a program.

Currently, there is known a technique for simulating the state of the face when cosmetics are applied to the user's face. For example, Patent Document 1 describes a technique of superimposing a make-up part indicating a make-up state when a make-up is applied to a face part on a user's face image to generate a simulation image and displaying the generated simulation image. Is described.

Here, the user generally makes up while being aware of the impression that he / she wants to express with the make-up of the face (for example, a cool impression, a bright impression, a calm impression, etc.). However, in order to properly express the desired impression with makeup, it is necessary to consider the shape and arrangement of the user's facial parts. Therefore, it is not easy for the user to properly express the desired impression with make-up. Therefore, in the make-up simulation, it is convenient to present a make-up method for appropriately expressing the impression desired by the user.

International Publication No. 2017/115453

However, Patent Document 1 does not describe or suggest a method of making up for expressing an impression desired by a user in the first place. Therefore, there is a demand for a technique for presenting a makeup method for appropriately expressing the impression desired by the user.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing device, an image processing method, and a program that present a makeup method for appropriately expressing an impression desired by a user. ..

In order to achieve the above object, the image processing apparatus according to the present invention is
A display means for displaying an captured image of the user's face, and
A feature identifying means for identifying the facial features of the user based on the captured image, and
Makeup that determines the makeup content for expressing the target impression based on the target impression set as the target impression expressed by the facial makeup and the facial features of the user specified by the feature specifying means. Content determination means and
The display means includes a display control means for displaying a make-up image in which the captured image is processed according to the make-up content determined by the make-up content determining means.

According to the present invention, it is possible to present a make-up method for appropriately expressing the impression desired by the user.

External view of the make-up simulation apparatus according to the first embodiment of the present invention. A block diagram showing a hardware configuration of the image processing apparatus according to the first embodiment of the present invention. A block diagram showing a hardware configuration of an instruction device according to a first embodiment of the present invention. A block diagram showing a functional configuration of the image processing apparatus according to the first embodiment of the present invention. The figure which shows the state that the user is using the image processing apparatus which concerns on Embodiment 1 of this invention. The figure which shows a mode that the image processing apparatus which concerns on Embodiment 1 of this invention displays a make-up image. First explanatory view of face parameters Second explanatory diagram of face parameters Explanatory diagram of model value information Explanatory diagram of basic operation amount information Explanatory drawing of makeup content information It is a figure which shows the make-up method to make a user's face close to a model face, (A) is a figure which shows the make-up method to make a drooping eye look like a standard eye, and (B) is a figure which shows a make-up method to make a slanted eye look like a standard eye. Diagram showing how to make up to show A flowchart showing an image display process executed by the image processing apparatus according to the first embodiment of the present invention. A block diagram showing a functional configuration of the image processing apparatus according to the second embodiment of the present invention. Explanatory drawing of conformity impression information

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals.

(Embodiment 1)
First, the configuration of the make-up simulation device 100 according to the first embodiment of the present invention will be described with reference to FIG. The make-up simulation device 100 is a device that simulates a state after the user has made up with cosmetics on a computer. By using the make-up simulation device 100, the user can easily practice make-up and check the state after make-up in advance without using real cosmetics.

As shown in FIG. 1, the make-up simulation device 100 includes an image processing device 10 and an instruction device 30. The image processing device 10 is a device that generates and displays an image showing the state after make-up by simulation using the captured image of the user's face. The image processing device 10 has, for example, a shape that imitates a palette for makeup.

FIG. 2 shows the hardware configuration of the image processing device 10. As shown in FIG. 2, the image processing device 10 includes a processor 11, a flash memory 12, a camera 13, a first touch screen 14, a second touch screen 15, and a communication interface 16. These elements included in the image processing apparatus 10 are connected to each other via, for example, a system bus.

The processor 11 controls the overall operation of the image processing device 10. The processor 11 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and an RTC (Real Time Clock). The flash memory 12 stores various types of information.

The camera 13 takes an image of the subject. The camera 13 includes a lens that collects light emitted from the subject, an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) that receives the collected light and acquires an image of the subject. It includes an A / D (Analog / Digital) converter that converts the image pickup data sent as an electric signal from the image pickup device into digital data. As shown in FIG. 1, the camera 13 is arranged in the vicinity of the first touch screen 14 and images the face of a user who is using the image processing device 10 while looking at the first touch screen 14.

The first touch screen 14 and the second touch screen 15 are user interfaces, receive various information from the user, and present various information to the user. As shown in FIG. 1, the first touch screen 14 is arranged on the upper surface of the image processing device 10 and displays the captured image obtained by the imaging of the camera 13. The second touch screen 15 is arranged on the lower surface of the image processing device 10 and displays an operation reception image or a cosmetics image.

The operation reception image is a screen for receiving various operations from the user. The cosmetic image is an image representing cosmetics that are virtually applied to the user's face in a makeup simulation. A cosmetic image is an image that represents each of a plurality of cosmetics in different regions. Each of the plurality of cosmetics is a cosmetic having at least one difference in color, luster, texture and the like. Further, the second touch screen 15 accepts a user operation involving contact with the surface of the second touch screen 15.

The communication interface 16 is a module that communicates with an external device including an instruction device 30 under the control of the processor 11. The communication interface 16 communicates with an external device by, for example, wireless communication such as a wireless LAN (Local Area Network) or Bluetooth (registered trademark), or wired communication such as a USB (Universal Serial Bus) cable.

The instruction device 30 is a device for selecting cosmetics in the make-up simulation and instructing the position of the face to which the selected cosmetics are virtually applied. The instruction device 30 has a stick-like shape and is a member that imitates an application tool used for applying cosmetics to the face, such as a so-called makeup brush (makeup brush, makeup brush). The user holds the instruction device 30 by hand and uses the cosmetic image displayed on the image processing device 10 and the user's actual face so as to point to the end of the instruction device 30.

FIG. 3 shows the hardware configuration of the instruction device 30. The instruction device 30 includes a processor 31, a flash memory 32, a pressure sensor 33, and a communication interface 34. These elements included in the indicator 30 are interconnected, for example, via a system bus.

The processor 31 controls the overall operation of the instruction device 30. The processor 31 includes, for example, a CPU, ROM, RAM, and RTC. The flash memory 32 stores various types of information. The pressure sensor 33 detects the pressure applied to the end of the indicator device 30. That is, the pressure sensor 33 detects contact with the end of the indicator device 30.

The pressure sensor 33 detects the pressure by a method such as a capacitance type or a strain gauge type. The communication interface 34 is a module that communicates with the image processing device 10 under the control of the processor 31. The communication interface 34 transmits pressure information indicating the pressure detected by the pressure sensor 33 to the image processing device 10. The communication interface 34 communicates with the image processing device 10 by, for example, wireless communication such as wireless LAN and Bluetooth (registered trademark), or wired communication such as a USB cable.

The make-up simulation device 100 has a virtual make-up function and a make-up method presentation function. The virtual make-up function is a function of superimposing a make-up part according to a make-up operation by the user on a face image of the user. The make-up method presentation function is a function of superimposing a make-up part that matches the impression that the user wants to express with the make-up of the face on the user's face image. That is, the make-up method presenting function is a function of presenting a make-up sample for bringing the user's face closer to the face having a desired impression. The virtual make-up function and the make-up method presenting function are switched according to, for example, a user operation on the second touch screen 15.

Hereinafter, the process executed by the make-up simulation device 100 in order to realize the virtual make-up function will be briefly described. First, the make-up simulation device 100 displays the captured image obtained by the imaging of the camera 13 on the first touch screen 14, and displays the cosmetic image on the second touch screen 15. Then, the make-up simulation device 100 superimposes and displays the make-up parts indicating the cosmetics virtually applied to the user's face on the captured image according to the make-up operation of the user.

This make-up operation includes a first make-up operation and a second make-up operation. The first make-up operation is an operation that imitates the operation of attaching the cosmetics stored in the cosmetic palette to the tip of the makeup brush. Specifically, the first make-up operation is an operation in which the tip of the instruction device 30 is brought into contact with a region representing a desired cosmetic product in the cosmetic image displayed on the second touch screen 15. The makeup simulation device 100 can identify the cosmetics selected by the user from the contact position on the second touch screen 15.

The second make-up operation is an operation that imitates the operation of applying the cosmetics adhering to the tip of the makeup brush to the user's face. Specifically, the second operation is an operation of bringing the tip of the instruction device 30 into contact with a desired region of the user's face. The make-up simulation device 100 determines the positional relationship between the pressure information transmitted from the instruction device 30 to the image processing device 10 and the user's face and the tip of the instruction device 30 in the image captured by the image captured by the camera 13. Based on this, the contact position of the tip of the instruction device 30 on the user's face can be specified.

The make-up simulation device 100 simulates an operation of applying the cosmetics selected by the user in the first make-up operation to the facial area designated by the user in the second make-up operation. That is, the make-up simulation device 100 displays a make-up image in which make-up parts representing cosmetics selected by the user are superimposed on a face area designated by the user on the captured image. According to the virtual make-up function, the state after make-up can be simulated. It is preferable that the user grasps an appropriate make-up method by using the make-up method presenting function before using the virtual make-up function. Hereinafter, in the present embodiment, the makeup method presentation function will be basically described.

As shown in FIG. 4, the image processing device 10 functionally receives an image pickup unit 101, an image acquisition unit 102, a display control unit 103, a display unit 104, a feature identification unit 105, and an impression designation operation reception. A unit 106, an impression setting unit 107, an adjustment instruction operation reception unit 108, a make-up content determination unit 110, and a storage unit 120 are provided. The feature specifying unit 105 includes a user value acquisition unit 115. The make-up content determination unit 110 includes a basic operation amount determination unit 111, an additional operation amount determination unit 112, a make-up area determination unit 113, and a make-up color determination unit 114. The storage unit 120 includes a model value storage unit 121, a basic operation amount storage unit 122, and a make-up content storage unit 123.

The image acquisition unit 102 is an example of the image acquisition means in the present invention. The display control unit 103 is an example of the display control means in the present invention. The display unit 104 is an example of the display means and the display device in the present invention. The feature specifying unit 105 is an example of the feature specifying means in the present invention. The impression setting unit 107 is an example of the impression setting means in the present invention. The adjustment instruction operation reception unit 108 is an example of the adjustment instruction operation reception means in the present invention. The make-up content determining unit 110 is an example of the make-up content determining means in the present invention. The basic operation amount determination unit 111 is an example of the basic operation amount determination means in the present invention. The additional manipulated variable determination unit 112 is an example of the additional manipulated variable determining means in the present invention. The user value acquisition unit 115 is an example of the user value acquisition means in the present invention. The model value storage unit 121 is an example of the model value storage means in the present invention. The basic manipulated variable storage unit 122 is an example of the basic manipulated variable storage means in the present invention. The make-up content storage unit 123 is an example of the make-up content storage means in the present invention.

The imaging unit 101 images a range including the user's face and generates an captured image representing this range. FIG. 5 shows a state in which a user faces a first touch screen 14 included in the image processing device 10 in order to simulate make-up using the make-up simulation device 100. The imaging unit 101 captures an image of the area inside the field of view shown by the broken line in FIG. 5, specifically, the area including the user's face facing the first touch screen 14.

Therefore, the installation position of the camera 13 and the orientation of the optical axis for realizing the image pickup unit 101 are adjusted so that the user's face facing the first touch screen 14 is included in the field of view of the camera 13. The image pickup unit 101 continuously executes a process of capturing the user's face and generating an captured image while the image processing device 10 is being used by the user. The function of the imaging unit 101 is realized, for example, by the cooperation of the processor 11 and the camera 13.

The image acquisition unit 102 acquires the captured image generated by the image capturing unit 101. The function of the image acquisition unit 102 is realized by, for example, the function of the processor 11.

The display control unit 103 controls the display of the image by the display unit 104. Specifically, the display control unit 103 displays the captured image acquired by the image acquisition unit 102 in the first display area included in the display unit 104. The first display area is, for example, a display area corresponding to the first touch screen 14. In this way, the display control unit 103 displays the state of the user's face in real time.

In addition, the display control unit 103 displays an operation acceptance image for accepting a user operation in a second display area included in the display unit 104. The second display area is a display area different from the first display area, and is, for example, a display area corresponding to the second touch screen 15. The operation reception image is an image including various buttons for receiving user operations. For example, in the operation reception image, a button for determining a desired impression from a plurality of impressions by the user, a process executed by the make-up simulation device 100 is changed from a process for a make-up method presentation function to a process for a virtual make-up function. It is an image including a button for instructing to switch. The function of the display control unit 103 is realized by, for example, the function of the processor 11.

The display unit 104 displays an image according to the control of the display control unit 103. Specifically, the display unit 104 displays the captured image acquired by the image acquisition unit 102 in the first display area, and displays the operation reception image in the second display area. The display unit 104 displays a make-up image in which make-up parts are superimposed on the captured image in the first display area under the control of the display control unit 103. FIG. 6 shows how the first touch screen 14 displays a make-up image in which the image 51, which is a make-up part, is superimposed on the captured image including the image 50, which is a face image. The function of the display unit 104 is realized, for example, by the cooperation of the processor 11, the first touch screen 14, and the second touch screen 15.

The feature specifying unit 105 identifies the features of the user's face based on the captured image acquired by the image acquisition unit 102. The features of the user's face are, for example, features related to the size, shape, placement position, placement angle of the user's face parts, or the placement relationship between the user's face parts. The features of the user's face are represented, for example, by the values of the user's face parameters. As described above, the feature specifying unit 105 includes a user value acquisition unit 115. The function of the feature specifying unit 105 is realized by, for example, the function of the processor 11.

The user value acquisition unit 115 acquires the user value, which is the value of the user's face parameter, based on the captured image acquired by the image acquisition unit 102. The face parameter is a parameter related to a face part, which is a part of the face. Face parts include the right eye, left eye, right eyebrow, left eyebrow, nose, glabellar, mouth, right ear, and left ear. Face parameters are, for example, parameters such as face part size, face part shape, face part placement position, face part placement angle, size ratio between multiple face parts, and placement balance between multiple face parts. is there. The type of face parameter, not the value of the face parameter, is appropriately and simply referred to as a face parameter. In this embodiment, there are four face parameters: eye angle, eye spacing, eye size, and eye opening. Hereinafter, these face parameters will be described with reference to FIGS. 7 and 8.

FIG. 7 shows a part of the user's face displayed on the first touch screen 14. The right eye in FIG. 7 is the user's right eye, and the left eye in FIG. 7 is the user's left eye. Further, FIG. 8 shows an enlarged view of the vicinity of the user's right eye. In FIG. 7, W1 represents the width of the right eye, W2 represents the width of the left eye, W3 represents the length from the inner corner of the right eye to the inner corner of the left eye, and W4 represents the length from the inner corner of the right eye to the contour on the right side of the face. Represents

In FIG. 8, W1 represents the width of the right eye, H1 represents the height of the right eye, W5 represents the length from the inside of the right eyebrow to the end of the right eyebrow, and W6 represents the length from the inside of the right eyebrow to the end of the right eyebrow. Represents the length to Mt. Bizan. Further, in FIG. 8, the straight line L1 is a straight line that passes through the inner corner of the right eye and extends in the width direction of the face, the straight line L2 is a straight line connecting the inner corner of the right eye and the outer corner of the right eye, and D1 is a straight line L1. Represents the angle formed by the straight line L2. The width direction of the face is a direction orthogonal to the vertical direction of the face and the front-back direction of the face, and is generally a direction connecting both ears. The vertical direction of the face is generally the direction connecting the brain and the chin. The anteroposterior direction of the face is generally orthogonal to the face.

The eye angle is a concept that collectively refers to the angle of the right eye and the angle of the left eye. The angle of the eye is represented by the angle formed by the straight line connecting the inner and outer corners of the eye and the straight line passing through the inner corner of the eye and extending in the width direction of the face. Therefore, the angle of the right eye is represented by D1 representing the angle formed by the straight line L1 and the straight line L2. The distance between the eyes is expressed by the ratio of the length between the inner corners of both eyes to the width of the eyes. As the width of the eyes, the width of the right eye, the width of the left eye, or the average value of the width of the right eye and the width of the left eye can be adopted. The eye spacing is represented by W3 / W1, W3 / W2, or W3 × 2 / (W1 + W2).

Eye size is a concept that collectively refers to the size of the right eye and the size of the left eye. The size of the eye is expressed by the ratio of the width of the eye to the length from the inner corner of the eye to the contour of the face. Therefore, the size of the right eye is represented by W1 / W4. Eye opening is a concept that collectively refers to the opening of the right eye and the opening of the left eye. The opening of the eyes is represented by the ratio of the vertical width of the eyes to the horizontal width of the eyes. Therefore, the opening of the right eye is represented by H1 / W1.

The face parameter is not limited to the face parameter related to the eyes, and may be, for example, a face parameter related to the eyebrows. For example, as a face parameter, there is the position of Mt. Bizan. The position of Mt. Bizan is a concept that collectively refers to the position of Mt. Bizan on the right eyebrow and the position of Mt. Bizan on the left eyebrow. The position of the ridge of the eyebrow is represented by the ratio of the length from the inside of the eyebrow to the ridge of the eyebrow to the length from the inside of the eyebrow to the end of the eyebrow. Therefore, the position of the ridge of the right eyebrow is represented by W6 / W5.

The user value acquisition unit 115 specifies the position of the user's face from the captured image acquired by the image acquisition unit 102, and specifies the position and size of each face part from the specified face position. Then, the user value acquisition unit 115 obtains the user value, which is the value of the user's face parameter, from the position and size of each face part for all the face parameters. The user value acquisition unit 115 can specify the position of the face, the position of each face part, and the size of each face part by using well-known techniques such as edge detection and pattern matching. The function of the user value acquisition unit 115 is realized by, for example, the function of the processor 11.

The impression designation operation reception unit 106 receives an impression designation operation from the user. The impression designation operation is an operation of designating an impression that the user wants to express with facial makeup. That is, the impression designation operation reception unit 106 acquires information indicating the impression of the face that the user wants to realize by makeup from the user. Impressions specified by the impression specification operation include intellectual, easy-going, modest, gorgeous, cool, bright, cute, and sharp. The impression designation operation reception unit 106, for example, presents an image representing these impressions to the user, and then accepts an operation of selecting an impression from the user. The function of the impression designation operation receiving unit 106 is realized, for example, by the cooperation of the processor 11 and the second touch screen 15.

The impression setting unit 107 sets a target impression, which is an impression of the target expressed by facial makeup, based on the information acquired from the user. In the present embodiment, the impression specified by the user is designated by the impression designation operation received from the user by the impression designation operation receiving unit 106. Therefore, the impression setting unit 107 sets the impression designated by the impression designation operation as the target impression. The function of the impression setting unit 107 is realized by, for example, the function of the processor 11.

The adjustment instruction operation reception unit 108 receives an adjustment instruction operation from the user. The adjustment instruction operation is an operation for instructing the adjustment of the make-up content determined by the make-up content determination unit 110. The adjustment instruction operation receiving unit 108, for example, presents an object for adjusting the make-up color and the make-up area to the user, and then receives an operation for adjusting the make-up color and the make-up area from the user using this object. Examples of such an object include a scroll bar and the like. The make-up color is the color of the make-up parts superimposed on the captured image. The make-up area is an area in which make-up parts are superimposed on the captured image. The function of the adjustment instruction operation receiving unit 108 is realized, for example, by the cooperation of the processor 11 and the second touch screen 15.

The make-up content determination unit 110 determines the make-up content for expressing the target impression based on the user value acquired by the user value acquisition unit 115 and the target impression set by the impression setting unit 107. The make-up content includes a make-up color and a make-up area. In the present embodiment, the cosmetic to be virtually applied is an eye shadow, the make-up color is the color of the eye shadow to be applied virtually, and the make-up area is an area to which the eye shadow is virtually applied. The details of the make-up content determination unit 110 will be described later. The function of the make-up content determination unit 110 is realized by, for example, the function of the processor 11.

Here, the display control unit 103 causes the display unit 104 to display a make-up image in which make-up parts corresponding to the make-up contents determined by the make-up content determination unit 110 are superimposed on the captured image. For example, as shown in FIG. 6, the display control unit 103 displays a make-up image obtained by superimposing an image 51 representing a make-up part according to the make-up content on an captured image including an image 50 which is a face image.

The storage unit 120 stores various information used when the make-up content determination unit 110 determines the make-up content. As described above, the storage unit 120 includes a model value storage unit 121, a basic operation amount storage unit 122, and a make-up content storage unit 123. The functions of the storage unit 120, the model value storage unit 121, the basic operation amount storage unit 122, and the make-up content storage unit 123 are realized by, for example, the functions of the flash memory 12.

The model value storage unit 121 stores model value information. The model value information is information indicating a model value which is a predetermined value of the face parameter of the model face for each of the plurality of face parameters. The model face is a model face in which the size of each face part and the arrangement balance of each face part are appropriate. That is, the model value is a model value of the value of the face parameter. The model value information is, for example, as shown in FIG. 9, information indicating a model value for each face parameter. In this model value information, a face having an eye angle of +5 degrees, an eye spacing of 1, an eye size of 0.7, and an eye opening of 0.33 is a model face. It is shown that.

The basic operation amount storage unit 122 stores the basic operation amount information. The basic manipulated variable information is information that can specify the basic manipulated variable, which is the manipulated variable for the model value for giving the model face a target impression. Specifically, the basic manipulated variable information is information indicating the manipulated variable for each model value for giving each impression to the model face for each of the plurality of impressions. The model face basically does not match the target face, which is a face that can give a target impression. In other words, the model value, which is the value of the face parameter of the model face, does not match the target value, which is the value of the face parameter of the target face. Therefore, the manipulated variable for setting the model value to the target value, that is, the difference value between the model value and the target value is set as the manipulated variable for the model value.

The basic operation amount information is, for example, as shown in FIG. 10, information indicating a face parameter and a basic operation amount for each impression. The basic operation amount information shown in FIG. 10 indicates that, for example, in order to bring the model face closer to the face with an intellectual impression, it is necessary to narrow the eye spacing by 0.2. In other words, this basic manipulated variable information shows that the facial eye spacing of the intellectual impression is 0.8, which is 0.2 less than the exemplary facial eye spacing of 1.

Further, considering the content of the basic operation amount information, it can be considered as follows. In other words, in order to bring the model face closer to a face with a laid back impression, it is advisable to widen the eye spacing by 0.2. Also, in order to bring the model face closer to a face with a modest impression, the size of the eyes should be reduced by 0.1. Further, in order to bring the model face closer to a face with a gorgeous impression, it is advisable to increase the size of the eyes by 0.1. Also, in order to bring the model face closer to a face with a cool impression, it is advisable to reduce the eye opening by 0.05. Further, in order to bring the model face closer to the face with a bright impression, the eye opening should be increased by 0.05. In addition, in order to bring the model face closer to the face with a cute impression, the angle of the eyes should be reduced by 5 degrees. Further, in order to bring the model face closer to a face with a sharp impression, it is advisable to increase the angle of the eyes by 5 degrees.

The make-up content storage unit 123 stores the make-up content information. The make-up content information is information indicating the make-up content corresponding to the operation amount for the user value or the model value for each of the plurality of face parameters. This manipulated variable corresponds to a basic manipulated variable or an additional manipulated variable. The basic operation amount is an operation amount with respect to the model value for giving a target impression to the model face. Further, the additional operation amount is an operation amount for the user value for bringing the user's face closer to the model face. The make-up content information is information indicating the make-up content that can be expected to have the same effect as when the face parameter is changed, in other words, the make-up content that can be expected to have the same effect as the operation on the face parameter.

The make-up content information is, for example, as shown in FIG. 11, information indicating the make-up content including the make-up area and the make-up color for each combination of the face parameter and the operation amount. The makeup content information shown in FIG. 11 indicates that, for example, in order to obtain the same effect as when the angle of the eyes is lowered by 10 degrees, a makeup in which a wide purple eye shadow is applied around the eyes should be applied. There is.

Hereinafter, makeup for changing the appearance impression will be described with reference to FIGS. 12 (A) and 12 (B). First, with reference to FIG. 12 (A), a make-up for bringing the drooping eyes closer to the exemplary eyes will be described. It is assumed that the proper value of the eye angle is +5 degrees and the eye angle of the model face is +5 degrees. Further, it is assumed that the user's right eye is a drooping eye and the angle of the user's right eye is −5 degrees. In this case, the amount of operation for setting the angle of the user's right eye to the angle of the right eye of the model face is +10 degrees. Therefore, as a make-up that can obtain the same effect as when the angle of the right eye is increased by 10 degrees, the make-up content of applying purple eye shadow to the upper part of the outside of the outer corner of the eye is determined. FIG. 12A shows the area E1 as a make-up area to which the eye shadow is virtually applied. The user's face to which such makeup is applied can be expected to give an impression similar to that of a model face in which the angle of the right eye is +5 degrees.

Next, with reference to FIG. 12B, a make-up for bringing the slanted eyes closer to the model eyes will be described. Here, it is assumed that the user's right eye is a chipping eye and the angle of the user's right eye is +10 degrees. In this case, the amount of operation for setting the angle of the user's right eye to the angle of the right eye of the model face is −5 degrees. Therefore, as a make-up that can obtain the same effect as when the angle of the right eye is lowered by 5 degrees, the make-up content of applying a wide range of lilac eye shadow around the eyes is determined. FIG. 12B shows the area E2 as a make-up area to which the eye shadow is virtually applied. The user's face to which such makeup is applied can be expected to give an impression similar to that of a model face in which the angle of the right eye is +5 degrees.

The make-up that brings the drooping eyes closer to the model eyes is also the make-up that brings the model eyes closer to the eyes that give a sharp impression. In other words, make-up that makes the eyes look like they are raised is a make-up that gives a sharper impression. Similarly, make-up that brings the model eyes closer to the model eyes is also make-up that brings the model eyes closer to the eyes that give a cute impression. In other words, make-up that makes the eyes look like they are lowered is a make-up that gives a more cute impression.

In addition, make-up that makes the eyes look narrower is a make-up that gives a more intellectual impression. In addition, make-up that makes the eyes look wider is a make-up that gives a more laid back impression. Also, make-up that makes the eyes look smaller is a make-up that gives a more modest impression. In addition, the make-up that makes the eyes look larger is a make-up that gives a more gorgeous impression. In addition, the make-up that makes the eyes look narrower is a make-up that gives a cooler impression. In addition, the make-up that makes the eyes look wider is a make-up that gives a brighter impression.

Next, the make-up content determination unit 110 will be described in detail. As described above, the make-up content determination unit 110 includes a basic operation amount determination unit 111, an additional operation amount determination unit 112, a make-up area determination unit 113, and a make-up color determination unit 114.

The basic operation amount determination unit 111 determines the basic operation amount for each of the plurality of face parameters based on the operation amount information stored in the basic operation amount storage unit 122 and the target impression. For example, when the target impression is sharp, +5 degrees is determined as the basic operation amount for the eye angle. For parameters other than the eye angle, 0 is determined as the basic operation amount. The function of the basic operation amount determination unit 111 is realized by, for example, the function of the processor 11.

The additional operation amount determination unit 112 determines the additional operation amount for each of the plurality of face parameters based on the user value acquired for each of the plurality of face parameters and the model value information stored in the model value storage unit 121. decide. For example, it is assumed that the face parameter is the eye angle, the eye angle of the user's face is 0 degrees, and the eye angle of the model face is +5 degrees. In this case, +5 degrees is determined as an additional operation amount for the eye angle. For parameters other than the eye angle, 0 is determined as the additional operation amount. The function of the additional operation amount determination unit 112 is realized by, for example, the function of the processor 11.

The make-up content determination unit 110 includes make-up content information stored in the make-up content storage unit 123, a basic operation amount determined by the basic operation amount determination unit 111, and an additional operation amount determined by the additional operation amount determination unit 112. Determine the makeup content based on. Here, the make-up content determination unit 110 may determine the make-up content in consideration of the specific face parameter, or may determine the make-up content without considering the specific face parameter. The specific face parameter is a face parameter in which, among the basic operation amounts determined by the basic operation amount determination unit 111, an operation amount equal to or greater than a threshold value predetermined as the basic operation amount is determined. The specific face parameter is a face parameter that should be manipulated to give a target impression to the model face.

Here, the make-up content determination unit 110 considers the basic operation amount determined for the specific face parameter when determining the make-up content. On the other hand, when determining the makeup content, the makeup content determination unit 110 does not have to consider the basic operation amount determined for the face parameters other than the specific face parameters. This is because such a face parameter has a small basic operation amount in the first place, and it is not preferable to increase the number of makeup points due to such a face parameter.

In addition, the make-up content determination unit 110 considers the additional operation amount determined for the specific face parameter when determining the make-up content. This is because such an additional operation amount is related to the part to be made, and therefore it is desirable to consider it in order to absorb individual differences in the user's face. On the other hand, the make-up content determination unit 110 does not have to consider the additional operation amount determined for the face parameter other than the specific face parameter when determining the make-up content. Since such an additional operation amount does not affect the part to be made, even if this part of the user's face has a feature, it is considered desirable not to consider it in order to suppress the increase of the make part. is there.

Hereinafter, a specific example of determining the makeup content in consideration of specific face parameters will be described. For example, as a first specific example, it is assumed that the target impression is sharp, the basic operation amount determined for the eye angle is +5 degrees, and the additional operation amount determined for the eye angle is +5 degrees. To do. In this case, only the eye angle is the specific face parameter. Therefore, the make-up content determination unit 110 makes up corresponding to +10 degrees, which is the total amount of +5 degrees, which is the basic operation amount determined for the eye angle, and +5 degrees, which is the additional operation amount determined for the eye angle. Determine the content.

That is, the make-up content determination unit 110 applies a make-up to obtain the same effect as when the eye angle is raised by 10 degrees, that is, a purple eye shadow is applied to the outer upper part of the outer corner of the eye based on the make-up content information. Determine the makeup content. In this way, when the user with drooping eyes desires a sharp impression, the make-up content determination unit 110 determines the make-up content that is visible to the extent that it gives a sharp impression. As described above, it is preferable to consider the additional operation amount determined for the specific face parameter in order to suppress excessive make-up or make-up deficiency.

Next, as a second specific example, when the target impression is gorgeous, the basic operation amount determined for the eye size is +5 degrees, and the additional operation amount determined for the eye angle is +5 degrees. Is assumed. In this case, only the size of the eyes is a specific face parameter, and the angle of the eyes is not a specific face parameter. Therefore, the make-up content determination unit 110 determines the make-up content corresponding to +5 degrees, which is the basic operation amount determined for the size of the eyes. That is, the make-up content determination unit 110 applies a make-up for obtaining the same effect as when the eye size is increased by 0.1, that is, a lilac eye shadow in the vicinity of the outer corner of the eye, based on the make-up content information. Decide what makeup to do. As described above, it is preferable not to consider the additional operation amount determined for the face parameter other than the specific face parameter in order to suppress the increase of the make-up portion.

The makeup content may be determined in consideration of face parameters other than the specific face parameters. For example, as a third specific example, a case where the target impression is gorgeous, the basic operation amount determined for the eye size is +5 degrees, and the additional operation amount determined for the eye angle is +5 degrees. Suppose. In this case, first, the make-up content determination unit 110 determines the make-up content corresponding to +5 degrees, which is the basic operation amount determined for the size of the eyes. That is, the make-up content determination unit 110 applies a make-up for obtaining the same effect as when the eye size is increased by 0.1, that is, a lilac eye shadow in the vicinity of the outer corner of the eye, based on the make-up content information. Decide what makeup to do.

Further, the make-up content determination unit 110 determines the make-up content corresponding to the additional operation amount determined for the eye angle. That is, the make-up content determination unit 110 applies make-up to obtain the same effect as when raising the eye angle by 5 degrees, that is, a lilac eye shadow on the outer upper part of the outer corner of the eye based on the make-up content information. Determine the makeup content. As described above, the additional operation amount determined for the face parameter other than the specific face parameter may be considered in order to bring the user's face closer to the model face.

The make-up area determination unit 113 determines the make-up area based on the basic operation amount determined for each of the plurality of face parameters, the additional operation amount determined for each of the plurality of face parameters, and the make-up content information. .. The function of the make area determination unit 113 is realized by, for example, the function of the processor 11.

The make-up color determination unit 114 determines the make-up color based on the basic operation amount determined for each of the plurality of face parameters, the additional operation amount determined for each of the plurality of face parameters, and the make-up content information. .. Determining the make-up color means determining the cosmetics and the amount to be applied. The make-up color is, for example, a color obtained by synthesizing the color of the make-up area in the face image and the determined color of the cosmetic product at a ratio corresponding to the determined application amount. For example, the makeup color approaches the color of cosmetics as the amount of application increases, and approaches the color of the makeup area in the face image as the amount of application decreases. The color of the make-up parts may be a color corresponding to the make-up color as a whole. That is, the entire make-up part does not have to be unified with the make-up color, and the color of each part of the make-up part may be different. The function of the make-up color determining unit 114 is realized by, for example, the function of the processor 11.

Further, when the adjustment instruction operation is received by the adjustment instruction operation reception unit 108, the make-up content determination unit 110 selects at least one of the make-up color and the make-up area included in the make-up content based on the adjustment instruction operation. adjust. For example, the make-up content determination unit 110 darkens the color of the make-up parts or lightens the color of the make-up parts according to the adjustment instruction operation. Further, the make-up content determination unit 110 widens the make-up area as a whole or in a specific direction, or narrows the make-up area as a whole or in a specific direction according to the adjustment instruction operation.

Next, the image display process executed by the image processing apparatus 10 will be described with reference to the flowchart shown in FIG.

First, the processor 11 included in the image processing device 10 displays the operation reception image (step S101). Specifically, the processor 11 displays the operation reception image on the second touch screen 15. When the processor 11 completes the process of step S101, the processor 11 starts imaging (step S102). Specifically, the processor 11 causes the camera 13 to start imaging, and thereafter continuously acquires the captured image from the camera 13.

When the processor 11 completes the process of step S102, the processor 11 displays the captured image (step S103). Specifically, the processor 11 displays the captured image acquired from the camera 13 on the first touch screen 14. When the processor 11 completes the process of step S103, it determines whether or not there is an impression designation operation (step S104). For example, the processor 11 determines whether or not an impression designation operation has been performed on the second touch screen 15.

When the processor 11 determines that there is an impression designation operation (step S104: YES), the processor 11 sets a target impression (step S105). The processor 11 sets the impression instructed by the impression designation operation as the target impression. When the processor 11 completes the process of step S105, the processor 11 identifies the facial features of the user from the captured image (step S106). That is, the processor 11 specifies each face part from the face image extracted from the captured image, and specifies the user value for each face parameter from the shape, size, positional relationship, and the like of each face part.

When the processor 11 completes the process of step S106, the processor 11 determines the basic operation amount (step S107). Specifically, the processor 11 determines the basic operation amount for each face parameter based on the basic operation amount information and the target impression. When the processor 11 completes the process of step S107, the processor 11 determines the additional operation amount (step S108). Specifically, the processor 11 determines the additional operation amount of each face parameter by obtaining the difference between the user value and the model value for each face parameter.

When the processor 11 completes the process of step S108, the processor 11 specifies a specific face parameter (step S109). Specifically, the processor 11 specifies as a specific face parameter a face parameter in which an operation amount equal to or larger than a predetermined threshold value as a basic operation amount is determined among the basic operation amounts determined for each of a plurality of face parameters. To do. When the processor 11 completes the process of step S109, the processor 11 determines the make area (step S110). Specifically, the processor 11 obtains the total operation amount, which is the sum of the basic operation amount and the additional operation amount, for each specific face parameter, and responds to the total operation amount for each specific face parameter based on the makeup content information. Determine the make-up area.

When the processor 11 completes the process of step S110, the processor 11 determines the make-up color (step S111). Specifically, the processor 11 determines the make-up color according to the total operation amount for each specific face parameter based on the make-up content information. When the processor 11 completes the process of step S111, the processor 11 displays a make-up image (step S112). Specifically, the processor 11 controls the display unit 104 to display the make-up image obtained by superimposing the make-up parts corresponding to the make-up colors on the make-up area in the captured image. When the processor 11 completes the process of step S112, the processor 11 returns the process to step S103.

Here, when the processor 11 determines that there is no impression designation operation (step S104: NO), the processor 11 determines whether or not there is an adjustment instruction operation (step S113). Specifically, the processor 11 determines whether or not an adjustment instruction operation has been performed on the second touch screen 15. When the processor 11 determines that there is an adjustment instruction operation (step S113: YES), the processor 11 adjusts the make parts (step S114). Specifically, the processor 11 adjusts at least one of the make-up area and the make-up color according to the adjustment instruction operation.

When the processor 11 completes the process of step S114, the processor 11 displays the adjusted make-up image (step S115). Specifically, the processor 11 controls the display unit 104 to generate and generate an adjusted make-up image by superimposing a make-up part corresponding to the adjusted make-up color on the adjusted make-up area in the captured image. Display the adjusted make-up image. When the processor 11 determines that there is no adjustment instruction operation (step S113: NO), or when the process of step S115 is completed, the processor 11 returns the process to step S103.

In the present embodiment, the make-up content for expressing the target impression is determined according to the facial features of the user, and the captured image is displayed as the make-up image processed according to the make-up content. Therefore, according to the present embodiment, it is possible to present a make-up method for appropriately expressing the impression desired by the user.

Further, in the present embodiment, the makeup content is determined based on the basic operation amount for giving the model face a target impression and the additional operation amount for bringing the user's face closer to the model face. Therefore, according to the present embodiment, it is possible to present an appropriate makeup method according to the impression desired by the user and the facial features of the user.

Further, in the present embodiment, the makeup content is determined based on the basic operation amount determined for the specific face parameter and the additional operation amount determined for the specific face parameter. Therefore, according to the present embodiment, it is possible to present a make-up in which the make-up on the face portion, which has little influence on the impression desired by the user, is excluded.

Further, in the present embodiment, the makeup content is adjusted based on the adjustment instruction operation. Therefore, according to the present embodiment, it is possible to present a make-up method for more appropriately expressing the impression desired by the user.

(Embodiment 2)
In the first embodiment, an example in which the impression desired by the user is directly specified by the user has been described. The impression desired by the user does not have to be directly specified by the user. Hereinafter, in the present embodiment, an example of specifying an impression desired by the user from a scene specified by the user will be described. Hereinafter, the function of the image processing apparatus 20 according to the present embodiment will be described with reference to FIG.

Functionally, the image processing device 20 according to the present embodiment does not include the impression designation operation reception unit 106 but includes the scene designation operation reception unit 109 and stores as compared with the image processing device 10 according to the first embodiment. The unit 120 further includes a conforming impression storage unit 124. The scene designation operation reception unit 109 is an example of the scene designation operation reception means in the present invention. The conforming impression storage unit 124 is an example of the conforming impression storage means in the present invention.

The conforming impression storage unit 124 stores conforming impression information indicating an impression conforming to each of a plurality of scenes. FIG. 15 shows conformity impression information. This conforming impression information is information indicating the correspondence relationship between the scene and the impression, and is information indicating the impression considered to be suitable for each scene for each scene. The conforming impression information shown in FIG. 15 indicates that, for example, in an interview scene, a face that gives an intellectual impression is desirable. The function of the conforming impression storage unit 124 is realized by, for example, the function of the flash memory 12.

The scene designation operation reception unit 109 receives a scene designation operation from the user. The scene designation operation is an operation for designating one of a plurality of scenes. Multiple scenes include interviews, customer service, attendance, parties, presentations, drinking parties, sightseeing, meetings, and so on. The scene designation operation reception unit 109, for example, presents an image representing these scenes to the user, and then accepts an operation of selecting a scene from the user. The function of the scene designation operation reception unit 109 is realized, for example, by the cooperation of the processor 11 and the second touch screen 15.

Here, when the scene designation operation is received by the scene designation operation reception unit 109, the impression setting unit 107 performs the scene designation operation among a plurality of scenes based on the conformity impression information stored in the conformity impression storage unit 124. An impression that matches the specified scene is set as the target impression.

In this embodiment, the target impression is automatically set based on the scene specified by the user. Therefore, according to the present embodiment, it is possible to present a make-up method for appropriately expressing an impression suitable for a scene specified by the user.

(Modification example)
Although the embodiments of the present invention have been described above, various modifications and applications are possible in carrying out the present invention.

In the present invention, which part of the configuration, function, and operation described in the above embodiment is adopted is arbitrary. Further, in the present invention, in addition to the above-described configurations, functions, and operations, further configurations, functions, and operations may be adopted. In addition, the above-described embodiments can be freely combined as appropriate. In addition, the number of components described in the above-described embodiment can be adjusted as appropriate. Further, it goes without saying that the materials, sizes, electrical characteristics, and the like that can be adopted in the present invention are not limited to those shown in the above embodiments.

In the first embodiment, an example in which the process for the make-up method presentation function and the process for the virtual make-up function are executed separately has been described. These processes may be executed in combination. For example, the make-up simulation device 100 displays the make-up image generated by the make-up method presenting function in the left half area of the first touch screen 14, and displays the make-up image generated by the virtual make-up function on the right side of the first touch screen 14. It can be displayed in half of the area, and the cosmetic image generated by the virtual make-up function can be displayed on the second touch screen 15.

The make-up image generated by the make-up method presentation function is an image in which make-up parts corresponding to the make-up contents for expressing a target impression are superimposed on the captured image. The make-up image generated by the virtual make-up function is an image in which make-up parts according to the make-up operation by the user are superimposed on the captured image. According to such a configuration, the user can simulate make-up with reference to a make-up image showing a sample make-up method displayed in the left half area of the first touch screen 14.

Alternatively, an image obtained by superimposing the contour image showing the contour of the make-up area determined by the make-up method presentation function on the make-up image generated by the virtual make-up function is displayed on the first touch screen 14, and the cosmetic image generated by the virtual make-up function is displayed. May be displayed on the second touch screen 15. According to such a configuration, the user can simulate the make-up while referring to the contour image displayed on the first touch screen 14.

In the first and second embodiments, the impression setting unit 107 has described an example of setting a target impression based on the information acquired from the user. The impression setting unit 107 may automatically set a target impression according to a predetermined standard. For example, the target impression may be automatically set to an impression based on the user's taste. In this case, the information indicating the correspondence between the preference and the impression may be stored in the flash memory 12. The user's preference is automatically specified from, for example, information acquired from another device via the communication interface 16. Further, the target impression may be automatically set to an impression that makes the impression of the user's current face more conspicuous. Alternatively, the target impression may be automatically set to an impression that weakens the impression of the user's current face. For example, if the user's eyes are slanted, the target impression may be automatically set to an impression that softens the slanted eyes.

In the first embodiment, an example in which the makeup content is a makeup in which eye shadow is applied around the eyes has been described. Of course, the makeup content is not limited to this example. For example, the makeup content may be a makeup in which an eyeliner is applied around the eyes. Further, the makeup content may be a makeup in which eyebrow is applied around the eyebrows.

In the above embodiment, an example in which the CPU executes various processes by executing a program stored in the ROM in the processor 11 has been described. In the present invention, instead of the processor 11, for example, ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), various control circuits, and other dedicated hardware are provided, and the dedicated hardware performs various processes. You may do it. In this case, the functions of each part may be realized by individual hardware, or the functions of each part may be collectively realized by a single hardware. Further, some of the functions of each part may be realized by dedicated hardware, and the other part may be realized by software or firmware.

Not only can it be provided as an image processing device provided with a configuration for realizing the function according to the present invention, but also an existing information processing device or the like can be made to function as an image processing device according to the present invention by applying a program. You can also. That is, the image according to the present invention is applied by applying the program for realizing each functional configuration by the image processing device 10 exemplified in the above embodiment so that the CPU or the like that controls the existing information processing device or the like can execute the program. It can function as a processing device.

Moreover, the method of applying such a program is arbitrary. The program can be stored and applied in a computer-readable storage medium such as a flexible disc, a CD (Compact Disc) -ROM, a DVD (Digital Versatile Disc) -ROM, or a memory card. Further, the program can be superimposed on the carrier wave and applied via a communication medium such as the Internet. For example, the program may be posted and distributed on a bulletin board system (BBS: Bulletin Board System) on a communication network. Then, this program may be started and executed in the same manner as other application programs under the control of the OS (Operating System) so that the above processing can be executed.

Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the specific embodiment, and the present invention includes the invention described in the claims and the equivalent range thereof. included. The inventions described in the claims of the original application of the present application are described below.
(Appendix 1)
A display means for displaying an captured image of the user's face, and
A feature identifying means for identifying the facial features of the user based on the captured image, and
Makeup that determines the makeup content for expressing the target impression based on the target impression set as the target impression expressed by the facial makeup and the facial features of the user specified by the feature specifying means. Content determination means and
The display means includes a display control means for displaying a make-up image in which the captured image is processed according to the make-up content determined by the make-up content determining means.
Image processing device.
(Appendix 2)
The feature specifying means includes a user value acquiring means for acquiring a user value which is a value of the face parameter of the user based on the captured image.
The make-up content determining means determines the make-up content based on the target impression and the user value acquired by the user value acquisition means.
The image processing apparatus according to Appendix 1.
(Appendix 3)
A model value storage means for storing model value information indicating a model value which is a value of a predetermined model face parameter, and a model value storage means.
A basic operation amount storage means for storing basic operation amount information indicating a basic operation amount which is an operation amount with respect to the model value for giving the model face the target impression.
Further provided with a make-up content storage means for storing make-up content information indicating the make-up content corresponding to the operation amount with respect to the user value or the model value.
The means for determining the content of makeup is
A basic operation amount determining means for determining the basic operation amount based on the basic operation amount information stored in the basic operation amount storage means, and a basic operation amount determining means.
Based on the user value acquired by the user value acquisition means and the model value information stored in the model value storage means, the amount of operation on the user value for bringing the user's face closer to the model face. It is provided with an additional operation amount determining means for determining a certain additional operation amount.
The make-up content determining means includes the basic operation amount determined by the basic operation amount determining means, the additional operation amount determined by the additional operation amount determining means, and the make-up stored in the make-up content storage means. The makeup content is determined based on the content information.
The image processing apparatus according to Appendix 2.
(Appendix 4)
The model value information is information indicating the model value for each of a plurality of face parameters.
The basic manipulated variable information is information indicating the manipulated variable for each model value for giving each impression to the model face for each of the plurality of impressions.
The make-up content information is information indicating the make-up content corresponding to the operation amount with respect to the user value or the model value for each of the plurality of face parameters.
The user value acquisition means acquires the user value for each of the plurality of face parameters based on the captured image.
The basic operation amount determining means determines the basic operation amount for each of the plurality of face parameters based on the basic operation amount information and the target impression.
The additional operation amount determining means performs the additional operation for each of the plurality of face parameters based on the user value and the model value information acquired for each of the plurality of face parameters by the user value acquisition means. Determine the amount,
The make-up content determining means is the basic operation amount determined for each of the plurality of face parameters by the basic operation amount determining means, and the additional operation amount determining means determined for each of the plurality of face parameters. The make-up content is determined based on the additional operation amount and the make-up content information.
The image processing apparatus according to Appendix 3.
(Appendix 5)
Among the basic operation amounts determined for each of the plurality of face parameters by the basic operation amount determining means, the make content determining means determines an operation amount equal to or greater than a predetermined threshold value as the basic operation amount. Of the basic operation amount determined for the specific face parameter which is a face parameter and the additional operation amount determined for each of the plurality of face parameters by the additional operation amount determining means, the addition determined for the specific face parameter. The makeup content is determined based on the operation amount and the makeup content information.
The image processing apparatus according to Appendix 4.
(Appendix 6)
Further provided with an adjustment instruction operation receiving means for receiving an adjustment instruction operation which is an operation for instructing the adjustment of the make-up content determined by the make-up content determining means.
The make-up content determining means is a region in which the make-up color, which is the color of the make-up image, and the make-up image, which are the colors of the make-up image, are superimposed on the make-up content when the adjustment instruction operation is received by the adjustment instruction operation receiving means. Adjust at least one of the make-up areas,
The image processing apparatus according to any one of Appendix 1 to 5.
(Appendix 7)
Further provided with an impression setting means for setting the target impression.
The image processing apparatus according to any one of Appendix 1 to 6.
(Appendix 8)
A conforming impression storage means for storing conforming impression information indicating an impression conforming to each of a plurality of scenes,
Further provided with a scene designation operation receiving means for receiving a scene designation operation which is an operation of designating one of the plurality of scenes from the user.
When the scene designation operation is received by the scene designation operation receiving means, the impression setting means performs the scene designation operation among the plurality of scenes based on the matching impression information stored in the conforming impression storage means. An impression that matches the scene specified by is set as the target impression.
The image processing apparatus according to Appendix 7.
(Appendix 9)
The make-up image is an image in which make-up parts representing a state in which cosmetics according to the make-up content are applied to the face of the user are superimposed on the make-up area corresponding to the make-up content in the captured image.
The image processing apparatus according to any one of Appendix 1 to 8.
(Appendix 10)
Further provided with an image acquisition means for acquiring the captured image,
The display means displays the captured image acquired by the image acquisition means.
The image processing apparatus according to any one of Appendix 1 to 9.
(Appendix 11)
Display the captured image of the user's face,
Based on the captured image, the facial features of the user are identified.
Based on the target impression set as the target impression to be expressed by the facial makeup and the facial features of the user, the makeup content for expressing the target impression is determined.
The captured image displays a make-up image processed according to the make-up content.
Image processing method.
(Appendix 12)
Computer,
A display control means for displaying an captured image of a user's face on a display device,
A feature identifying means for identifying the facial features of the user based on the captured image,
Makeup that determines the makeup content for expressing the target impression based on the target impression set as the target impression expressed by the facial makeup and the facial features of the user specified by the feature specifying means. A program that functions as a means of determining content
The display control means causes the display device to display a make-up image in which the captured image is processed according to the make-up content determined by the make-up content determining means.
program.

10, 20 ... Image processing device, 11 ... Processor, 12 ... Flash memory, 13 ... Camera, 14 ... First touch screen, 15 ... Second touch screen, 16 ... Communication interface, 30 ... Indicator device, 31 ... Processor , 32 ... Flash memory, 33 ... Pressure sensor, 34 ... Communication interface, 50, 51 ... Image, 100 ... Make simulation device, 101 ... Imaging unit, 102 ... Image acquisition unit, 103 ... Display control unit, 104 ... Display unit, 105 ... Feature identification unit, 106 ... Impression designation operation reception unit, 107 ... Impression setting unit, 108 ... Adjustment instruction operation reception unit, 109 ... Scene designation operation reception unit, 110 ... Make content determination unit, 111 ... Basic operation amount determination unit , 112 ... Additional operation amount determination unit, 113 ... Make area determination unit, 114 ... Make color determination unit, 115 ... User value acquisition unit, 120 ... Storage unit, 121 ... Model value storage unit, 122 ... Basic operation amount storage unit, 123 ... Make content storage unit, 124 ... Conformity impression storage unit, E1, E2 ... Area, L1, L2 ... Straight line

Claims (12)

A display means for displaying an captured image of the user's face, and
A feature identifying means for identifying the facial features of the user based on the captured image, and
Makeup that determines the makeup content for expressing the target impression based on the target impression set as the target impression expressed by the facial makeup and the facial features of the user specified by the feature specifying means. Content determination means and
The display means includes a display control means for displaying a make-up image in which the captured image is processed according to the make-up content determined by the make-up content determining means.
Image processing device. The feature specifying means includes a user value acquiring means for acquiring a user value which is a value of the face parameter of the user based on the captured image.
The make-up content determining means determines the make-up content based on the target impression and the user value acquired by the user value acquisition means.
The image processing apparatus according to claim 1. A model value storage means for storing model value information indicating a model value which is a value of a predetermined model face parameter, and a model value storage means.
A basic operation amount storage means for storing basic operation amount information indicating a basic operation amount which is an operation amount with respect to the model value for giving the model face the target impression.
Further provided with a make-up content storage means for storing make-up content information indicating the make-up content corresponding to the operation amount with respect to the user value or the model value.
The means for determining the content of makeup is
A basic operation amount determining means for determining the basic operation amount based on the basic operation amount information stored in the basic operation amount storage means, and a basic operation amount determining means.
Based on the user value acquired by the user value acquisition means and the model value information stored in the model value storage means, the amount of operation on the user value for bringing the user's face closer to the model face. It is provided with an additional operation amount determining means for determining a certain additional operation amount.
The make-up content determining means includes the basic operation amount determined by the basic operation amount determining means, the additional operation amount determined by the additional operation amount determining means, and the make-up stored in the make-up content storage means. The makeup content is determined based on the content information.
The image processing apparatus according to claim 2. The model value information is information indicating the model value for each of a plurality of face parameters.
The basic manipulated variable information is information indicating the manipulated variable for each model value for giving each impression to the model face for each of the plurality of impressions.
The make-up content information is information indicating the make-up content corresponding to the operation amount with respect to the user value or the model value for each of the plurality of face parameters.
The user value acquisition means acquires the user value for each of the plurality of face parameters based on the captured image.
The basic operation amount determining means determines the basic operation amount for each of the plurality of face parameters based on the basic operation amount information and the target impression.
The additional operation amount determining means performs the additional operation for each of the plurality of face parameters based on the user value and the model value information acquired for each of the plurality of face parameters by the user value acquisition means. Determine the amount,
The make-up content determining means is the basic operation amount determined for each of the plurality of face parameters by the basic operation amount determining means, and the additional operation amount determining means determined for each of the plurality of face parameters. The make-up content is determined based on the additional operation amount and the make-up content information.
The image processing apparatus according to claim 3. Among the basic operation amounts determined for each of the plurality of face parameters by the basic operation amount determining means, the make content determining means determines an operation amount equal to or greater than a predetermined threshold value as the basic operation amount. Of the basic operation amount determined for the specific face parameter which is a face parameter and the additional operation amount determined for each of the plurality of face parameters by the additional operation amount determining means, the addition determined for the specific face parameter. The makeup content is determined based on the operation amount and the makeup content information.
The image processing apparatus according to claim 4. Further provided with an adjustment instruction operation receiving means for receiving an adjustment instruction operation which is an operation for instructing the adjustment of the make-up content determined by the make-up content determining means.
The make-up content determining means is a region in which the make-up color, which is the color of the make-up image, and the make-up image, which are the colors of the make-up image, are superimposed on the make-up content when the adjustment instruction operation is received by the adjustment instruction operation receiving means. Adjust at least one of the make-up areas,
The image processing apparatus according to any one of claims 1 to 5. Further provided with an impression setting means for setting the target impression.
The image processing apparatus according to any one of claims 1 to 6. A conforming impression storage means for storing conforming impression information indicating an impression conforming to each of a plurality of scenes,
Further provided with a scene designation operation receiving means for receiving a scene designation operation which is an operation of designating one of the plurality of scenes from the user.
When the scene designation operation is received by the scene designation operation receiving means, the impression setting means performs the scene designation operation among the plurality of scenes based on the matching impression information stored in the conforming impression storage means. An impression that matches the scene specified by is set as the target impression.
The image processing apparatus according to claim 7. The make-up image is an image in which make-up parts representing a state in which cosmetics according to the make-up content are applied to the face of the user are superimposed on the make-up area corresponding to the make-up content in the captured image.
The image processing apparatus according to any one of claims 1 to 8. Further provided with an image acquisition means for acquiring the captured image,
The display means displays the captured image acquired by the image acquisition means.
The image processing apparatus according to any one of claims 1 to 9. Display the captured image of the user's face,
Based on the captured image, the facial features of the user are identified.
Based on the target impression set as the target impression to be expressed by the facial makeup and the facial features of the user, the makeup content for expressing the target impression is determined.
The captured image displays a make-up image processed according to the make-up content.
Image processing method. Computer,
A display control means for displaying an captured image of a user's face on a display device,
A feature identifying means for identifying the facial features of the user based on the captured image,
Makeup that determines the makeup content for expressing the target impression based on the target impression set as the target impression expressed by the facial makeup and the facial features of the user specified by the feature specifying means. A program that functions as a means of determining content
The display control means causes the display device to display a make-up image in which the captured image is processed according to the make-up content determined by the make-up content determining means.
program.

Images