JP7356927B2 - Skin analysis method, skin analysis system and skin analysis program - Google Patents

Description

The present invention relates to a skin analysis method, a skin analysis system, and a skin analysis program.

In recent years, research has been conducted on technology that uses videos to analyze human skin. For example, in Patent Document 1, a plurality of tracking points are placed in advance in an analysis area such as the outer corner of the subject's eyes, a video is shot, and the amount of change in the tracking points due to changes in facial expression is tracked to calculate the compression ratio of the skin. A technique for analyzing a subject's skin condition by acquiring the information has been disclosed.

Patent Document 2 discloses a technique related to a process of evaluating clinical signs such as wrinkles and fine lines from a sequence of images of a person's face taken while emitting sound.

Additionally, Patent Document 3 discloses a method for extracting factors that have a high correlation with facial appearance impressions using dynamic information, and a technique for discriminating facial appearance impressions based on the factors. There is. Patent Document 3 discloses that the areas where age impression is determined are the cheeks and the areas around the eyes.

Japanese Patent Application Publication No. 2014-193197 Japanese Patent Application Publication No. 2017-502732 Japanese Patent Application Publication No. 2016-194901

As mentioned earlier, technology for non-invasively evaluating skin by analyzing videos containing people's faces has been widely researched. However, conventional techniques only evaluate signs that appear on the skin surface, such as wrinkles, and no specific method has been disclosed for non-invasively analyzing physical properties such as skin viscoelasticity and collagen structure. Not yet. Therefore, in order to evaluate the physical properties of the skin, methods such as collecting skin tissue, applying force to the skin, and measuring its mechanical properties have been necessary. Therefore, an object of the present invention is to provide a novel method for non-invasively analyzing the physical properties of the skin.

In order to solve the above problems, the skin analysis method according to the present invention is based on the correlation between the physical quantities of skin changes caused by changes in facial expressions and the viscoelasticity of the skin. It is characterized by calculating the viscoelasticity of the skin.

In this way, by calculating the skin viscoelasticity from the measured value of the physical amount of skin change based on the correlation between the physical amount of skin change caused by changes in facial expressions and the skin viscoelasticity, it is possible to calculate the skin viscoelasticity. Skin analysis including calculations can be performed non-invasively.

In a preferred embodiment of the present invention, the viscoelasticity includes at least one of dermal viscoelasticity and subcutaneous viscoelasticity.

In a preferred embodiment of the present invention, a viscoelasticity calculation model created by inputting a plurality of sets of measured values of the physical quantity and viscoelasticity is used to calculate the viscoelasticity of the subject's skin from the measured values of the physical quantity about the subject. It is characterized by calculating.
In this way, by calculating viscoelasticity using a viscoelasticity calculation model created using a set of physical quantities and viscoelasticity measurement data, it is possible to calculate viscoelasticity more accurately based on actual examples. can.

A preferred embodiment of the present invention is characterized in that the collagen structure of the subject's skin is estimated from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin.
In this way, by estimating the subcutaneous collagen structure based on skin viscoelasticity calculated from the measured values of physical quantities of skin changes caused by changes in facial expressions, skin analysis including estimation of the subcutaneous collagen structure can be performed. can be performed non-invasively.

In a preferred embodiment of the present invention, a collagen structure estimation model created by inputting a plurality of sets of measured values of the viscoelasticity and collagen structure of the skin is used to estimate the skin of the subject based on the viscoelasticity calculated for the subject. It is characterized by estimating collagen structure.

A preferred embodiment of the present invention is characterized in that one or more skin evaluation values are calculated based on the viscoelasticity.
In this way, by calculating the skin evaluation value based on the viscoelasticity of the skin, it is possible to show results that are easy for the test subject to understand.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the viscoelasticity calculation result is output.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the measured value of the physical quantity is output together with information based on the calculated result of the viscoelasticity.
In this way, by outputting information based on the calculated viscoelasticity and the measured physical quantity together, it becomes easier for the user to understand the correspondence between the measured quantity and the analysis result.

The present invention is characterized in that, based on the correlation between the physical quantities of skin changes caused by changes in facial expressions and the collagen structure of the skin, the collagen structure of the subject's skin is estimated from the measured values of the physical quantities for the subject. .
In this way, the collagen structure of the skin can be estimated by estimating the collagen structure of the skin from the measured value of the physical amount of skin change based on the correlation between the physical amount of skin change caused by changes in facial expressions and the collagen structure of the skin. Skin analysis including estimation can be performed non-invasively.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the estimation result of the collagen structure is output.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the estimated result of the collagen structure as well as information based on the measured value of the physical quantity is output.

In a preferred embodiment of the present invention, the physical quantity is measured based on a moving image that includes the process of facial expression change.
In this way, by measuring a physical quantity using a moving image, it is possible to more easily calculate the viscoelasticity of the skin without contacting the subject's skin. This can reduce the burden on both the subject and the person performing the analysis.

A preferred embodiment of the present invention is characterized in that a guide for aligning facial parts on the subject's face is displayed on the screen for photographing the moving image.
In this way, by displaying a guide for aligning facial parts on the video shooting screen and shooting according to this guide, images can be taken with the subject at the appropriate distance and in the appropriate position. It can be performed.

In a preferred embodiment of the present invention, the video is shot from a direction rotated within a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face.
In this way, by using an image of the face taken from an angle rotated within a range of 20 to 30 degrees from the front, the impression of the face, such as the cheeks and the corners of the eyes, can be improved from side to side, compared to when using an image taken from the front. By acquiring an image that includes a wide range of skin areas that are easily affected, it is possible to perform more accurate skin analysis of the skin areas that tend to affect the impression of the face.

In a preferred embodiment of the present invention, the video is a video shot from a direction rotated clockwise in a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face; It is characterized by using both a moving image taken from a direction rotated counterclockwise around a vertical axis passing through the face within a range of 20 to 30 degrees with the front as 0 degrees.
In this way, by using videos taken from different directions, it is possible to analyze the physical properties of the skin from physical quantities that comprehensively consider changes in the skin obtained from changes in left and right facial expressions.

In a preferred form of the present invention, the video is:
An initial position is a state in which the subject is captured from the front, and a rotation angle of the imaging device from the initial position is obtained;
permitting recording of the video when the rotation angle is within a predetermined range;
It is characterized in that the photograph is taken based on the permission.

In a preferred embodiment of the present invention, the physical quantity is measured based on at least one of the following: speed, direction, acceleration, and direction of movement of the feature points included in the face. It is characterized by

In a preferred embodiment of the present invention, the physical quantity is selected from skin followability, elasticity, and deformability.

A preferred embodiment of the present invention is characterized in that the moisture content of the epidermis of the subject is calculated from the measured value of the physical quantity for the subject, based on the correlation between the physical quantity and the moisture content of the epidermis.
In this way, by calculating the moisture content of the epidermis from the measured value of the physical quantity based on the correlation between the physical quantity of skin changes caused by changes in facial expressions and the moisture content of the epidermis, it is possible to calculate the moisture content of the epidermis. Skin analysis can be performed non-invasively.

In a preferred embodiment of the present invention, the water content of the subject is calculated from the measured values of the physical quantity for the subject using a water content calculation model created by inputting a plurality of sets of measured values of the physical quantity and water content. It is characterized by

The present invention is a skin analysis system that analyzes a subject's skin based on skin movements when facial expressions change,
a physical quantity measuring means for measuring a physical quantity of skin change caused by the facial expression change;
Storage means for storing the correlation between the physical quantity and the viscoelasticity of the skin;
The present invention is characterized by comprising a viscoelasticity calculation means for calculating the viscoelasticity of the subject's skin from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the viscoelasticity of the skin.

In a preferred embodiment of the present invention, the viscoelasticity calculation means calculates at least one of dermal viscoelasticity and subcutaneous viscoelasticity.

In a preferred embodiment of the present invention, the storage means stores a viscoelasticity calculation model created by inputting a plurality of sets of measured values of the physical quantity and viscoelasticity,
The viscoelasticity calculation means is characterized in that the viscoelasticity is calculated from the physical quantity measured for the subject using the viscoelasticity calculation model.

In a preferred form of the present invention, the storage means stores a correlation between the viscoelasticity and the collagen structure of the skin,
The method further comprises collagen structure estimating means for estimating the collagen structure of the subject's skin from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin.

In a preferred form of the present invention, the storage means stores a collagen structure estimation model created by inputting a plurality of sets of measured values of the viscoelasticity and collagen structure,
The collagen structure estimating means is characterized in that, using the collagen structure estimation model, the collagen structure of the subject's skin is estimated from the viscoelasticity calculated for the subject.

A preferred embodiment of the present invention is characterized by further comprising an evaluation means for calculating one or more skin evaluation values based on the viscoelasticity of the skin of the subject.

A preferred embodiment of the present invention is characterized by further comprising an output means for outputting an analysis result including information based on the viscoelasticity calculation result.

In a preferred embodiment of the present invention, the output means outputs an analysis result including information based on the measured value of the physical quantity as well as information based on the calculated result of the viscoelasticity.

The present invention is a skin analysis system that analyzes a subject's skin based on skin movements when facial expressions change,
a physical quantity measuring means for measuring a physical quantity of skin change caused by the facial expression change;
storage means for storing the correlation between the physical quantity and the collagen structure of the skin;
The present invention is characterized by comprising a collagen structure estimating means for estimating the collagen structure of the subject's skin from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the collagen structure of the skin.

A preferred embodiment of the present invention is characterized by further comprising an output means for outputting information based on the estimation result of the collagen structure.

The output means is characterized in that it outputs an analysis result including information based on the estimated result of the collagen structure as well as information based on the measured value of the physical quantity.

In a preferred embodiment of the present invention, the invention further includes a video acquisition means for acquiring a video including the process of facial expression change,
The physical quantity measuring means is characterized in that it measures the physical quantity based on the moving image.

A preferred embodiment of the present invention is characterized by further comprising means for displaying a guide for aligning the positions of facial parts on the subject's face on the screen for photographing the moving image.

In a preferred embodiment of the present invention, the video acquisition means captures a video of the subject's face from a direction rotated within a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face. It is characterized by obtaining.

In a preferred embodiment of the present invention, the video is a video shot from a direction rotated clockwise in a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face; It is characterized by using both a moving image taken from a direction rotated counterclockwise around a vertical axis passing through the face within a range of 20 to 30 degrees with the front as 0 degrees.

In a preferred form of the present invention, the video acquisition means acquires the video taken by a terminal device for photographing the subject under predetermined conditions;
The terminal device is
means for obtaining a rotation angle of the terminal device from the initial position, with an initial position taken as the subject is viewed from the front;
means for permitting recording of the video when the rotation angle is within a range of 20 to 30 degrees;
and means for photographing the moving image based on the permission.

In a preferred embodiment of the present invention, the physical quantity measuring means determines the physical quantity based on at least one of the magnitude of the velocity, the direction of velocity, the magnitude of acceleration, and the direction of acceleration of the movement of the feature points included in the face. It is characterized by measuring physical quantities.

In a preferred embodiment of the present invention, the physical quantity measuring means measures any one of skin followability, elasticity, and deformability as the physical quantity.

In a preferred embodiment of the present invention, the storage means stores a correlation between the physical quantity and the moisture content of the epidermis,
The present invention is characterized by further comprising a moisture content calculation means for calculating the moisture content of the epidermis of the subject from the measured value of the physical quantity for the subject, based on the correlation between the physical quantity and the moisture content of the epidermis.

In a preferred embodiment of the present invention, the storage means stores a moisture content calculation model created by inputting a plurality of sets of measured values of the physical quantity and moisture content;
The water content calculating means is characterized in that the water content calculation means calculates the water content from the physical quantity measured for the subject using the water content calculation model.

The present invention is a skin analysis program for analyzing the skin of a subject based on skin movements when facial expressions change,
a physical quantity measuring means for measuring a physical quantity of skin change caused by the facial expression change;
Storage means for storing the correlation between the physical quantity and the viscoelasticity of the skin;
The present invention is characterized in that it functions as a viscoelasticity calculating means for calculating the viscoelasticity of the subject's skin from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the viscoelasticity of the skin.

The present invention is a skin analysis program for analyzing the skin of a subject based on skin movements when facial expressions change,
a physical quantity measuring means for measuring a physical quantity of skin change caused by the facial expression change;
storage means for storing the correlation between the physical quantity and the collagen structure of the skin;
The present invention is characterized in that it functions as a collagen structure estimating means for estimating the collagen structure of the skin of a subject from the measured values of the physical quantities for the subject, based on the correlation between the physical quantities and the collagen structure of the skin.

The physical properties of the skin can be non-invasively analyzed based on changes in the skin when facial expressions change.

FIG. 2 is a schematic diagram of facial expressions reproduced by a subject in an embodiment of the present invention. FIG. 1 is a functional block diagram of a skin analysis system according to an embodiment of the present invention. 1 is a flowchart showing a skin analysis method according to an embodiment of the present invention. It is a flowchart which shows processing of a video acquisition process in one embodiment of the present invention. FIG. 3 is a diagram showing a photographing direction of a subject in an embodiment of the present invention. FIG. 3 is a diagram showing facial parts coordinates in an embodiment of the present invention. It is a figure showing an example of reference distance in one embodiment of the present invention. It is a flowchart which shows processing of an analysis object extraction process in one embodiment of the present invention. It is a flowchart which shows processing of a movement amount measuring process in one embodiment of the present invention. FIG. 3 is a diagram showing a fractionation pattern of an analysis region in an embodiment of the present invention. It is a flowchart which shows the flow of processing of a viscoelasticity calculation process in one embodiment of the present invention. It is a flowchart which shows the processing flow of the water content calculation process in one embodiment of this invention. FIG. 2 is a flowchart showing a process flow for calculating a subcutaneous collagen structure based on subcutaneous viscoelasticity in an embodiment of the present invention. FIG. 1 is a diagram showing the appearance of a terminal device according to an embodiment of the present invention. 3 is a flowchart showing the flow of video shooting in an embodiment of the present invention. FIG. 3 is a diagram showing the shooting angle of a moving image in an embodiment of the present invention. FIG. 3 is a diagram showing an example of a video shooting preparation screen according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of a video shooting screen according to an embodiment of the present invention.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment, a video showing the process of facial expression change is acquired, the frame where the facial expression change starts and the frame where the facial expression change ends are identified, and the frames captured between the two are analyzed. conduct.

In the present invention, "calculating viscoelasticity" refers to calculating values for quantitatively evaluating viscosity and elasticity. In this embodiment, the viscoelasticity of the skin refers to the viscoelasticity of the dermis and the viscoelasticity of the subcutaneous layer. Such viscoelasticity of the skin causes changes in appearance such as wrinkles and sagging, and therefore serves as an index for confirming skin aging phenomena associated with aging.

Furthermore, the collagen structure of the skin refers to the collagen structure of the subcutaneous or dermal skin, including the fibrosis level of the fibrous structure surrounding subcutaneous fat cells and the degree of cohesion of collagen fibers in the dermis.

The skin is made up of three layers: the epidermis, dermis, and subcutaneous tissue. Most of the subcutaneous tissue that supports the epidermis and dermis is subcutaneous fat, which is composed of adipose lobules in which fat cells form clusters, and has functions such as heat retention and buffering against external forces. Fat lobules are surrounded by connective tissues such as collagen fibers and elastin fibers in a mesh pattern, forming a fibrous structure that envelops fat cells.

As a result of intensive research by the present inventors, it has become clear that collagen fibers surrounding subcutaneous fat cells become fibrotic with age. In the present invention, an index representing the fibrosis level of such a subcutaneous fibrous structure is estimated as a "subcutaneous collagen structure."

Furthermore, it is known that the connective tissue of the dermis gradually loses flexibility and elasticity and becomes hard with age. This is thought to be due to the fact that collagen fibers, which are the main component of connective tissue, become crosslinked and bundled with age. Therefore, in the present invention, an index representing the degree of cohesion of collagen fibers in the dermis is estimated as "dermal collagen structure." In addition, any other index for evaluating the structure of collagen may be used.

In the present embodiment, a value (hereinafter referred to as (travel amount). The amount of movement may be determined as a value for each frame, or may be determined as an average value over one facial expression change. Alternatively, areas such as cheeks and jaws may be set, and the integrated value, average value, etc. of the values within the area may be determined. Furthermore, in the present embodiment, the physical amount of skin change caused by a change in facial expression refers to a value measured based on the amount of movement.

Note that in the following, facial parts coordinates are coordinates indicating the positions of parts on the face for tracking facial movements and evaluating changes in facial expressions. Facial parts refer to any features such as eyes, nose, facial contours, etc. For example, points at the upper and lower ends, points at the left and right ends of the lip contour, etc. can be detected and used as facial parts coordinates.

Furthermore, the analysis target is a set of images that are targeted for skin analysis. In this embodiment, skin analysis is performed by specifying a frame where a facial expression change starts and a frame where a facial expression change ends, and analyzing movement using an image during the facial expression change included between them.

In this embodiment, a video showing the process of vertical facial expression change in which the skin of the cheek area is expanded and contracted in the vertical direction and a process of horizontal facial expression change in which the skin of the cheek area is expanded and contracted in the horizontal direction are used. A skin analysis is performed by acquiring a video and measuring the physical amount of skin change caused by each facial expression change based on the video. In this way, by using a moving image showing the process of facial expression change in which the skin is stretched or contracted in a specific direction, it becomes possible to analyze the movement characteristics of the skin when the skin is stretched or contracted in a specific direction. In particular, by analyzing both vertical and horizontal facial expression changes, a wider range of skin movement characteristics can be investigated.

FIG. 1 is a diagram showing facial expressions reproduced by a subject in this embodiment. In this embodiment, as shown here, the subjects showed a neutral expression (Figure 1(a)), an expression with the mouth open vertically (hereinafter referred to as an "a" expression, Figure 1(b)), and an expression with the mouth open horizontally (Figure 1(b)). There are four facial expressions: a facial expression that opens in response (hereinafter referred to as the expression ``I'', Fig. 1(c)), and an expression in which the mouth is pursed (hereinafter referred to as the expression ``u''; Figure 1(d)).

In this embodiment, the subject reproduces facial expressions in the order of vertical facial expression changes: no expression, "a" expression, "u" expression, "a" expression, "u" expression, and no expression. . In other words, vertical facial expression changes from a neutral expression to an “a” expression, vertical facial expression changes from an “a” expression to an “u” expression, and vertical facial expression changes from an “u” expression to an “a” expression. Skin analysis is performed by acquiring videos containing four types of vertical facial expression changes: a vertical facial expression change, and a vertical facial expression change from a "U" expression to a neutral expression. Of the four types of facial expressions used for skin analysis, only two are used: a vertical facial change from a neutral expression to an ``a'' expression, and a vertical facial change from a ``u'' expression to a neutral expression. It's okay.

In addition, as for lateral facial expression changes, the subject reproduces facial expressions in the following order: no expression, "I" expression, "U" expression, "I" expression, "U" expression, and no expression. In other words, the lateral facial expression change from a neutral expression to an ``i'' expression, the lateral facial expression change from an ``i'' expression to an ``u'' expression, and the lateral facial expression change from an ``u'' expression to an ``i'' expression. Skin analysis is performed by acquiring videos containing four types of lateral facial expression changes: a lateral facial expression change, and a lateral facial expression change from a "U" expression to a neutral expression. Of the four types of facial expressions used for skin analysis, only two types are used: lateral facial expression change from neutral expression to ``I'' expression and lateral facial expression change from ``U'' expression to neutral expression. It's okay. Here, when the facial expression changes from a neutral expression to a "U" expression, the skin in the cheek area expands and contracts both vertically and horizontally, so it is treated as both a vertical facial expression change and a horizontal facial facial change.

In this way, in this embodiment, a video containing multiple types of facial expression changes that expand and contract the skin of the cheek area in the same direction is acquired, the frames where each type of facial expression change starts and ends are identified, and various types of facial expression changes are determined. The images from the frame where the facial expression change starts to the frame where it ends are targeted for analysis. In this way, by specifying and analyzing multiple types of facial expression changes that cause the skin to expand and contract in the same direction, it is possible to analyze in more detail the movement characteristics of the skin as the skin expands and contracts in a specific direction. I can do it.

In this embodiment, analysis targets are extracted for each of the four types of vertical facial expression changes and four types of horizontal facial expression changes, and the amount of movement is measured for each analysis target. As the physical quantity, any value derived from the amount of movement can be used, such as an average value of a plurality of amounts of movement measured for each analysis target, a total value, a value calculated by weighting each amount of movement, etc.

FIG. 2 is a functional block diagram of the skin analysis system according to this embodiment. As shown here, the skin analysis system according to the present embodiment is configured such that the skin analysis device 10 and the terminal device 20 can communicate with each other. The skin analysis device 10 includes an arithmetic unit such as a CPU (Central Processing Unit), a main storage device such as a RAM (Random Access Memory), an auxiliary device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), and a flash memory. memory A general computer device such as a server device, which is equipped with various input/output devices including communication means with the terminal device 20, can be used. More specifically, programs for operating the computer are stored in the auxiliary storage device in advance or by copying from a recording medium, etc., and these programs are expanded onto the main storage device. The computer device can be used as the skin analysis device 10 in this embodiment by performing calculations using the calculation device and controlling the input/output means.

The terminal device 20 includes a photographing means 201 and a display section 202. The terminal device 20 is a general computer device equipped with an arithmetic unit, a main storage device, an auxiliary storage device, a communication means with the skin analysis device 10, a photographing means such as a digital camera, various input/output devices, etc. can be used. For example, a tablet terminal or the like having a camera function may be used. Specifically, for example, by installing dedicated application software for operating the computer device as each means described below, the computer device can be used as the terminal device 20 in the skin analysis system according to the present embodiment. .

The skin analysis device 10 in this embodiment includes a video acquisition unit 1 that acquires a video of the process of facial expression change, and a video acquisition unit 1 that identifies the timing of the facial expression change and extracts an analysis target from the video acquired by the video acquisition unit 1. an analysis target extraction means 2 that measures the amount of movement in the analysis object extracted by the analysis object extraction means 2, and a physical quantity measurement means 3 that measures the physical amount of skin change based on the amount of movement, and stores correlation data. A storage means 4, an analysis section 5 that calculates the viscoelasticity of the skin, the moisture content of the epidermis, and estimates the subcutaneous collagen structure, and calculates the evaluation value of the subject's skin based on the analysis results of the analysis section 5. Evaluation means 6.

In this embodiment, the physical quantity measuring means 3 includes a feature point extracting means 31 that extracts feature points of a face from a video, the magnitude of the speed of movement of the feature points, the direction of the velocity, the magnitude of the acceleration, the direction of the acceleration, The moving amount measuring means 32 includes a moving amount measuring means 32 that identifies a motion vector indicating a motion vector, and measures a moving amount from the motion vector, and a moving amount analyzing means 33 that measures a physical quantity based on the moving amount.

The storage means 4 stores the correlation between physical quantities and viscoelasticity. In this embodiment, the correlation between the physical quantity and viscoelasticity stored in the storage means 4 includes the correlation between the physical quantity and the viscoelasticity of the subcutaneous skin, and the correlation between the physical quantity and the viscoelasticity of the dermis. In this embodiment, the storage means 4 also stores information such as the correlation between physical quantities and epidermal water content, the correlation between dermal viscoelasticity and dermal collagen structure, and the correlation between subcutaneous viscoelasticity and subcutaneous collagen structure. Data, information on the video acquired by the video acquisition means 1, information on the analysis target extracted by the analysis target extraction means 2, analysis results, etc. are stored.

Here, various types of correlation data in this embodiment include, for example, a function representing the mutual relationship between various values, a model learned by giving various data as a set, etc., which is stored as a calculation model. You can keep it. In addition to this, information representing the correlation may be stored using any method and used for calculation.

In this embodiment, the analysis unit 5 includes a viscoelasticity calculation means 51 that calculates viscoelasticity from measured values of physical quantities based on the correlation between physical quantities and viscoelasticity, and a viscoelasticity calculation means 51 that calculates viscoelasticity from measured values of physical quantities based on the correlation between viscoelasticity and subcutaneous collagen structure. Collagen structure estimating means 52 estimates the collagen structure of the skin from the calculated viscoelasticity, and moisture content calculating means calculates the moisture content of the epidermis from the measured value of the physical quantity based on the correlation between the physical quantity and the moisture content of the epidermis. 53. The viscoelasticity calculation means 51 includes a subcutaneous viscoelasticity calculation means 511 that calculates subcutaneous viscoelasticity, and a dermal viscoelasticity calculation means 512 that calculates the viscoelasticity of the dermis. Further, the collagen structure estimating means 52 includes a subcutaneous collagen structure estimating means 521 for estimating a subcutaneous collagen structure, and a dermal collagen structure estimating means 522 for estimating a dermal collagen structure.

In the present embodiment, the evaluation means 6 includes a dermal viscoelasticity evaluation means 61 that calculates an evaluation value regarding the viscoelasticity of the dermis calculated by the dermal viscoelasticity calculation means 512, and a subcutaneous viscoelasticity calculated by the subcutaneous viscoelasticity calculation means 511. subcutaneous viscoelasticity evaluation means 62 that calculates an evaluation value regarding the dermal collagen structure estimated by the dermal collagen structure estimation means 522; The subcutaneous collagen structure evaluation means 64 calculates an evaluation value regarding the collagen structure, and the moisture content evaluation means 65 calculates an evaluation value regarding the moisture content of the epidermis calculated by the moisture content calculation means 53.

FIG. 3 is a flowchart showing the skin analysis method in this embodiment. In this embodiment, steps S11 and S12 in FIG. 3 are performed for each of the videos including a vertical facial expression change and the horizontal facial expression change, and in steps S13 to S16, two types of videos are performed. Each step is processed based on multiple movement amounts measured from the video. In this way, we can analyze the skin in more detail by measuring the amount of movement for multiple expression changes that cause the skin to expand and contract in different directions, and by calculating the viscoelasticity of the skin based on the physical quantities measured from the multiple amounts of movement. It can be performed.

<Extraction of analysis target>
First, in the video acquisition step of step S11, a video is acquired and an analysis target is extracted. FIG. 4 shows a flowchart of processing in the moving image acquisition step. In this embodiment, a video including a vertical facial expression change and a video including a horizontal facial expression change are acquired, and the processing shown in FIG. 4 is executed for each facial expression change included in each to extract the analysis target. do. That is, the processing shown in FIG. 4 is executed for each of the two types of moving images. Here, in each of the steps shown in FIG. 4, the process moves to the next step after the processing of one entire video is completed. That is, for example, the facial expression motion evaluation step in step S24 is started after the calculation of distances between facial part coordinates for all frames constituting the moving image is completed in the distance calculation step in step S23.

In step S21, the video acquisition means 1 causes the display unit 202 of the terminal device 20 to display information prompting video shooting, and the photographing means 201 shoots the video. The moving image acquisition means 1 acquires information on the moving image photographed by the photographing means 201 and records it in the storage means 4.

FIG. 5 is a diagram showing the photographing direction of the subject in this embodiment. In this embodiment, the video acquisition means 1 acquires information on a video showing the process of facial expression change, which is photographed from a direction rotated 25 degrees with the front as 0 degrees around a vertical axis passing through the subject's face. do. At this time, a configuration may be adopted in which the photographing means 201 obtains the rotation angle from the front and permits photographing when the rotation angle is within a predetermined range. The direction of rotation may be clockwise or counterclockwise toward the subject, or both may be analyzed.

In this way, it is desirable to take the image from a direction rotated around the vertical axis passing through the subject's face. More specifically, the rotation angle at this time is preferably in the range of 20 to 30 degrees with the front as 0 degrees. This makes it possible to obtain images that include more of the skin around the cheeks and eyes, which tend to affect the impression of the face, making it possible to more accurately analyze the skin in these areas. .

At this time, a guide for aligning facial parts such as eyes and contours may be displayed on the photographing screen. For example, a configuration may be adopted in which a guide for adjusting the position of the face is displayed first, and when the alignment of the face is completed, a guide is displayed to set the photographing angle to 20 to 30 degrees based on the angle of the photographing means 201. .

Furthermore, in the present embodiment, the skin analysis device 10 executes a process of correcting a shift in the position of the face in the acquired video. Specifically, it is possible to correct movement of the face position due to facial movements and camera shake during photographing, using facial parts such as the nose as a reference. At this time, it is preferable that the facial part used as a reference is a part whose position changes little relative to the analysis area.

Note that processing for displaying information and guides that prompt video shooting, correction of face position deviation, etc. may be performed in either the skin analysis device 10 or the terminal device 20. For example, the video capturing process may be completed in the terminal device 20, and the video acquiring means 1 may function as a means for simply receiving the video captured and temporarily stored in the terminal device 20.

Next, in the facial part coordinate detection step of step S22, the analysis target extraction means 2 detects a plurality of facial part coordinates of the face included in the video acquired in step S21 for each frame. Here, at least one set of two points whose distance changes in conjunction with a change in facial expression included in the video is detected. In this way, by detecting two points whose distance changes in conjunction with a change in facial expression as a pair, it is possible to better identify images where a facial expression change starts and images where a facial expression change ends based on changes in the distance between the two points. can be accurately identified.

When the detection of facial parts coordinates for the entire video is completed in the facial parts coordinate detection step of step S22, the analysis target extraction means 2 calculates the distance between the facial parts coordinates in step S23. The distance calculation is performed for each frame of the entire moving image, and the calculation result is stored in the storage means 4.

FIG. 6 is a diagram showing facial parts coordinates used in this embodiment. Figure 6(a) shows the facial parts coordinates used when extracting the analysis target for vertical facial expression changes, and Figure 6(b) shows the facial parts coordinates used when extracting the analytical target for horizontal facial expression changes. show. As shown here, in this embodiment, when extracting the analysis target for vertical facial expression changes, in addition to the two points between the upper and lower ends of the lips, multiple points on each of the left and right cheeks and the point on the chin are extracted. Detect the coordinates as facial parts coordinates. Then, the distance calculation means 13 calculates the distance between the two points at the upper and lower ends of the lip contour and the distance between each point on the left and right cheeks and the point on the chin, and based on the calculation results, the analysis target is determined. Extracted.

In addition, when extracting an analysis target in lateral facial expression changes, two points at the top and bottom edges of the lip contour and two points at the left and right edges of the lip contour are detected as facial parts coordinates, and the distance calculation means 13 uses these as a pair of lips. The distance between two points at the upper and lower ends of the contour and the distance between two points at the left and right ends of the lip contour are calculated. In this way, by setting the facial part coordinates to be used according to a series of facial expression changes to be analyzed, it is possible to accurately specify the start and end of a facial expression change with a small amount of information.

Here, the distance between the two points on the upper and lower edges of the lips and the two points on the cheek and chin increases when the skin of the cheek area is stretched vertically, and when the skin of the cheek area is shortened vertically, The distance becomes smaller when This makes it possible to specify the timing of various vertical facial expression changes. Furthermore, the distance between the two points at the left and right ends of the lip contour becomes smaller when the cheek skin is stretched laterally, and the distance becomes larger when the cheek skin is shrunk laterally. Furthermore, by using the distance between the two points at the top and bottom edges of the lip contour, for example, when changing the facial expression in the lateral direction, the change in distance between the two points at the left and right edges of the lip contour is small. Regarding changes, etc., it is possible to specify the image where the facial expression change starts and the image where the facial expression change ends, and the timing of various lateral facial expression changes can be specified.

In this way, in the facial parts coordinate detection step, it is preferable to detect a plurality of sets of two points whose distances change in conjunction with the expansion and contraction of the skin of the cheek area. As a result, for videos containing multiple types of facial expression changes, the frame where each type of facial expression change starts and the frame where the facial expression change ends can be identified through a series of processes, without having to perform analysis target extraction processing for each type. be able to.

Note that any generally known method can be used to detect facial parts coordinates. For example, application software developed using a commonly available development kit may be used. As a development kit, for example, use Face Sensing Engine (registered trademark) (https://www.oki.com/jp/fse/function/#fp, last accessed: December 19, 2018), etc. be able to.

Further, in the present embodiment, in the distance calculation step S23, a reference distance calculation step is executed to calculate a reference distance for standardizing the distance between facial parts coordinates. Then, in facial motion evaluation step S24 to analysis target extraction step S26, which will be described later, an analysis target is extracted using a value obtained by standardizing the distance between the facial part coordinates by a reference distance. By doing so, it is possible to reduce the effects of individual differences in faces, magnification at the time of photographing, etc., and to extract the analysis target with higher accuracy.

FIG. 7 is a diagram showing an example of a reference distance that can be used in this embodiment. In this embodiment, the distance between the left and right eyes (distance "A" in the illustrated example) is used as a reference distance to standardize the distance between facial part coordinates. Specifically, for example, the distance between the facial part coordinates can be standardized by calculating the distance of A and dividing all the distances between the facial part coordinates by A.

In addition, any distance that is considered to be able to correct the effects of individual differences, magnification at the time of photography, etc., such as the length of a perpendicular line drawn from the line connecting the left and right eyes to the chin (distance "B" in the illustrated example) It is preferable to use as the reference distance. Alternatively, a configuration may be adopted in which a plurality of reference distances are combined to standardize distances between facial part coordinates. Note that the reference distance may be a distance related to a point indicating an arbitrary feature on the face, and is not particularly limited.

In the facial expression motion evaluation step of step S24, the magnitude of the facial expression change is evaluated based on the change in the distance between the facial component coordinates calculated in step S23. Specifically, for example, the distance between the coordinates of facial parts when the subject reproduces a neutral expression, and the reproduction of facial expressions other than neutral expression (``a'' expression, ``i'' expression, and ``u'' expression). The ratio between the distance between the coordinates of facial parts when By using the ratio in this way, it is possible to reduce the influence of individual differences such as face size and magnification during photographing.

Next, in step S25, the evaluation value calculated in step S24 is compared with a reference value, and if it exceeds the reference value, the process proceeds to step S26, an analysis target extraction step. If the reference value is not met, the process proceeds to step S27, and the video acquisition means 1 transmits information to the terminal device 20 for prompting the user to shoot a video again. When step S27 ends, the process returns to step S21 and the moving image is acquired again. Regarding the re-acquired video, in steps S22 to S25, a facial parts coordinate detection step, a distance calculation step, and a facial motion evaluation step are performed.

In this embodiment, since the video to be acquired includes multiple changes in facial expressions, in the facial motion evaluation process, all facial changes included in the video are compared with reference values, and all facial changes are If the evaluation value exceeds the reference value, proceed to the analysis target extraction step.

In this way, by evaluating the magnitude of facial expression changes and re-shooting and re-acquiring the video if it does not meet the predetermined criteria, we can ensure that the magnitude of facial expression changes is sufficient for skin analysis. Skin analysis can be performed on videos. Thereby, the accuracy of calculating the viscoelasticity of the skin can be improved.

In step S26, the analysis target extraction means 2 extracts the analysis target. FIG. 8 is a flowchart showing an example of processing in the analysis target extraction step. Here, in step S26, the processing shown in FIG. 8 is executed for each of the analysis targets for various facial expression changes included in the video, and the analysis targets are extracted. In this embodiment, a total of eight types of facial expression changes are analyzed: four types of vertical facial expression changes in videos that include vertical facial expression changes, and four types of horizontal facial expression changes in videos that include horizontal facial expression changes. Extract.

Note that the process shown in FIG. 8 is a process performed for each distance between the facial part coordinates of two points detected as a set. In this embodiment, in order to extract the analysis target using a plurality of distances between the facial part coordinates, this process is performed for each distance between the facial part coordinates of two points detected as a set, and the combination of the results Extract the analysis target. Specifically, the analysis target is extracted using the distance between the facial part coordinates of two points detected as a pair, and the distance between the facial part coordinates of the other two points is calculated for facial expression changes that could not be extracted at that time. An analysis target can be extracted by performing the process shown in FIG. 8 using the distance.

First, in step S31, the analysis target extraction means 2 calculates the absolute value of the difference (amount of change) in the distance between facial part coordinates between consecutive frames. Next, in step S32, a vertex (maximum point) where the difference in distance between the facial parts coordinates is larger than a reference value is detected. That is, in each process of a plurality of facial expression changes included in the moving image, the point where the distance between facial parts coordinates changes the most is detected.

In step S33, frames in which the difference in distance between facial parts coordinates before and after the vertex detected in step S32 are less than a reference value are identified. As a result, a frame in which the difference in distance between the facial parts coordinates is less than the reference value immediately before the vertex detected in step S32 is identified as a frame in which a facial expression change has started, and similarly, immediately after the vertex, the frame between the facial parts coordinates is A frame in which the difference in distance is less than a reference value is identified as a frame in which the facial expression change has been completed, and is recorded in the storage means 4. Thereby, the analysis target extracting means 2 extracts frames included between the frame where the facial expression change starts and the frame where it ends as the analysis target.

Note that when performing the processing shown in FIG. 8, filtering or the like may be performed as appropriate. For example, in step S31, a low pass filter (LPF) is applied to the temporal change in the distance between the facial part coordinates (change in the distance between the facial part coordinates between each frame) to remove noise. After that, the difference between frames may be calculated. In addition, in order to remove noise, it is possible to perform processing such as setting the absolute value of the difference in the distance between facial part coordinates between frames calculated in step S31 to 0 for a portion that is less than a predetermined value. can.

As described above, facial part coordinates are detected from multiple images, images where facial expression changes start and images where facial expression changes end are identified based on the distance between facial part coordinates, and images included in between are analyzed. By extracting the frame as a target, a frame including a predetermined facial expression change can be specified with high accuracy. Thereby, the movement characteristics of the skin during a predetermined change in facial expression can be analyzed more accurately, and the accuracy of skin analysis can be improved.

<Measurement of physical quantities>
When step S11 in FIG. 3 ends as described above, the movement amount measuring process of step S12 starts. FIG. 9 is a flowchart showing the flow of processing in the movement amount measuring step. Here, in step S12, the process shown in FIG. 9 is executed for each of the analysis targets in the various facial expression changes extracted in step S11, and the amount of movement is measured. In this embodiment, the amount of movement is measured for each of the analysis targets for four types of vertical facial expression changes and four types of horizontal facial expression changes, that is, eight types of facial expression changes.

First, in step S41, feature points are extracted from each frame in the analysis target. Here, we will introduce a method of setting feature points by determining the unevenness of the skin from the brightness value of each pixel in a frame, and a method of setting feature points in a grid pattern in a video. Any method may be used, such as the method of

In this embodiment, the amount of movement is measured using an optical flow method. Specifically, one side of the analysis area of the cheek is divided into a predetermined number of areas, the divided square areas are used as the basic analysis unit, the center point of each basic analysis unit is regarded as the feature point, and the amount of movement is calculated. Measure. FIG. 10 is a diagram showing an example of a fractionation pattern of an analysis region in this embodiment. The shaded area in FIG. 10 is one basic analysis unit in each fractionation pattern. In this embodiment, one side is first divided into 12 parts, and motion vectors in analysis units of various sizes are identified (step S42). The motion vector can be determined based on changes in brightness between frames.

Thereafter, in step S43, the motion vector is analyzed and the amount of movement is measured. Examples of the amount of movement here include the amount of displacement of the coordinates of the feature points mentioned earlier, movement speed, acceleration, movement direction, etc., wave parameters of plane movement, such as period, circumference, and movement of facial skin. Examples include quantities related to.

Note that each of these movement amounts may be obtained as a value for each frame, or may be obtained as an average value over one facial expression change. Alternatively, a region may be set, such as a cheek region or a jaw region, and the integrated value or average value of the values within the region may be determined.

In step S43, in this embodiment, the analysis area is first divided into 144 basic analysis units of 12×12. Next, by measuring the amount of movement of each basic analysis unit and taking the average value of adjacent basic analysis units, each of 6 x 6, 4 x 4, 3 x 3, 2 x 2, 1 x 1, etc. The amount of movement of each basic analysis unit in the fractionation pattern can be determined. As a result, in this embodiment, a total of 210 pieces of vector data can be used as the movement amount for each frame.

Furthermore, in this embodiment, a value based on a comparison of the amount of movement between basic analysis units within each fractionation pattern, and a comparison of the amount of movement in the same region between different fractionation patterns (basic analysis of different sizes) A value based on a comparison between units) is calculated for each frame, and these are used as the movement amount in addition to the above vector data. More specifically, the difference in the amount of movement between basic analysis units within each fractionation pattern is calculated by brute force, and the movement of the corresponding region (basic analysis unit indicating the same position) between different fractionation patterns is calculated using brute force. It is possible to perform brute force arithmetic on the difference in amounts and use these as the amount of movement for each frame.

In this way, by analyzing the amount of movement from various viewpoints for each position within the analysis region (basic analysis unit), effects such as observing changes in motion characteristics for each region and reducing individual differences can be expected.

As described above, by executing the process shown in FIG. 9 in step S12 for each of the analysis targets for the various facial expression changes extracted in step S11, four types of vertical facial expression changes and four types of horizontal facial expressions can be analyzed. It is possible to obtain the amount of movement for each change, that is, the analysis target for eight types of facial changes.

In the physical quantity measuring step of step S13, physical quantities are measured based on the movement amounts corresponding to these eight facial expression changes. In this embodiment, various amounts of movement including the direction and speed of movement "for each frame" as described above, that is, a two-dimensional waveform having a time dimension, are converted into scalar amounts using an arbitrary method. For example, conversion is possible by using analysis methods such as frequency analysis, autoregressive model analysis, and wavelet analysis. Thereby, the converted scalar quantity can be used as a physical quantity. Although it is expected that a huge amount of physical quantities will be obtained at this time, the physical quantities may be reduced by any method such as principal component analysis.

Further, any value derived from the amount of movement may be used as the physical amount, such as an average value, a total, or a value calculated by weighting each amount of movement for each facial expression change.

<Calculation of viscoelasticity>
In the viscoelasticity calculation step of step S14, the viscoelasticity of the subject's skin is calculated from the physical quantity about the subject measured in step S13, based on the correlation between the physical quantity and the viscoelasticity of the skin.

FIG. 11 is a flowchart showing the flow of processing in the viscoelasticity calculation step. In this embodiment, in step S14, the viscoelasticity and evaluation value of the dermis and the viscoelasticity and evaluation value of the subcutaneous skin are calculated, respectively.

First, in step S51, the physical quantity-viscoelasticity correlation data stored in the storage means 4 is acquired. In this embodiment, the physical quantity-viscoelasticity correlation data includes a correlation between a physical quantity and subcutaneous viscoelasticity, and a correlation between a physical quantity and dermal viscoelasticity.

Next, in step S52, the viscoelasticity of the skin is calculated based on the correlation obtained in step S51 and the physical quantity measured by the physical quantity measuring means 3. Specifically, for example, a function representing the relationship between a physical quantity and the viscoelasticity of the skin is registered in advance in the storage means 4 as a viscoelasticity calculation model, and the physical quantity measured by the physical quantity measuring means 3 is added to the function. There are methods such as calculating the viscoelasticity of the skin by substitution.

A function representing the relationship between the physical quantity and the viscoelasticity of the skin can be obtained by a statistical method from data of a set of measured values of the physical quantity and the viscoelasticity. More specifically, multiple sets of physical quantities measured on a subject using the method described above and skin viscoelasticity measured on the same subject are input into a computer, and the results are processed using, for example, linear regression or decision tree regression. A function (regression model) can be obtained by regression analysis such as support vector regression, Gaussian process regression, etc. Furthermore, functions representing the relationship between physical quantities and water content, and functions representing the relationship between viscoelasticity and collagen structure, which will be described later, can also be determined in the same manner. In this embodiment, such a viscoelasticity calculation model is created for each of the viscoelasticity of the dermis and the viscoelasticity of the subcutaneous layer, and the viscoelasticity of each is calculated in step S52.

When the calculation of viscoelasticity is completed, an evaluation value of viscoelasticity is calculated in step S53. In calculating the evaluation value, the evaluation value of the viscoelasticity of each of the dermis and the subcutaneous layer is calculated using the viscoelasticity calculated in step S52. For example, the calculated viscoelasticity can be subjected to predetermined processing and a standardized value that can be easily understood intuitively by the subject can be used as the evaluation value.

Here, step S51 and step S52 are performed by the subcutaneous viscoelasticity calculating means 511 when calculating the viscoelasticity of the subcutaneous layer, and by the dermal viscoelasticity calculating means 512 when calculating the viscoelasticity of the dermis. Further, step S53 is performed by the subcutaneous viscoelasticity evaluation means 62 when calculating the evaluation value of subcutaneous viscoelasticity, and by the dermal viscoelasticity evaluation means 61 when calculating the evaluation value of the viscoelasticity of the dermis.

<Calculation of water content>
When step S14 ends, a moisture content calculation process starts in step S15. In step S15, the moisture content of the epidermis of the subject is calculated from the physical quantity about the subject measured in step S13, based on the correlation between the physical quantity and the moisture content of the epidermis.

FIG. 12 is a flowchart showing the flow of processing in the moisture content calculation step. First, in step S61, the moisture content calculation means 53 acquires the physical quantity-moisture content correlation data stored in the storage means 4. Next, in step S62, the moisture content calculating means 53 calculates the moisture content of the epidermis based on the correlation acquired in step S61 and the physical quantity measured in step S13. Specifically, for example, a function representing the relationship between a physical quantity and the moisture content of the epidermis is registered in the storage means 4 as a moisture content calculation model, and the physical quantity measured by the physical quantity measuring means 3 is substituted into the function. There are methods such as calculating the moisture content of the epidermis. Similar to the function representing the relationship between physical quantities and viscoelasticity, the function expressing the relationship between the physical quantity and the moisture content of the epidermis is obtained from the data of the set of measured values of the physical quantity and moisture content using a statistical method. be able to.

When the calculation of the moisture content is completed, in step S63, the moisture content evaluation means 65 calculates an evaluation value of the moisture content of the epidermis using the moisture content calculated in step S62. As the evaluation value, for example, a value obtained by subjecting the moisture content calculated in step S62 to a predetermined process and standardizing it can be used.

<Estimation of collagen structure>
In the collagen structure estimation step of step S16, the collagen structure of the subject's skin is estimated from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin. In this embodiment, in step S16, the dermal collagen structure and evaluation value and the subcutaneous collagen structure and evaluation value are calculated, respectively. FIG. 13 is a flowchart showing the process flow in the collagen structure estimation step.

In the present embodiment, in the collagen structure estimation step, the subcutaneous collagen structure estimating means 521 estimates the skewness of the histogram derived by analyzing the ultrasound image of subcutaneous fat cells as the subcutaneous collagen structure. , the subcutaneous collagen structure evaluation means 64 estimates the collagen structure by dividing the degree of distortion into a plurality of stages and determining evaluation values. That is, the evaluation value of the degree of distortion is estimated as the collagen structure.

The histogram used here is obtained by cutting out the subcutaneous fat part from the ultrasound image of the subject and using image analysis software to use this as an analysis image. When we analyze this histogram and calculate the skewness, we find that the lower the degree of fibrosis, the higher the skewness, and the higher the degree of fibrosis, the lower the skewness. Able to evaluate the level of improvement. Therefore, in this embodiment, the skewness of the histogram obtained as described above is used as an index of the subcutaneous collagen structure.

Regarding the dermal collagen structure, image processing is performed on microscopic images of dermal collagen fibers, and a value obtained by quantifying the degree of cohesion of collagen fibers is estimated as the "dermal collagen structure."

First, in step S71, the subcutaneous viscoelasticity calculated by the subcutaneous viscoelasticity calculating means 511 is acquired. Next, in step S72, viscoelasticity-collagen structure correlation data stored in the storage means 4 is acquired. In this embodiment, the viscoelasticity-collagen structure correlation data includes the correlation between dermal viscoelasticity and dermal collagen structure, and the correlation between subcutaneous viscoelasticity and subcutaneous collagen structure.

In this embodiment, a function representing the relationship between subcutaneous viscoelasticity and the skewness of a histogram obtained by analyzing an ultrasound image of a subcutaneous fat layer is registered in the storage means 4 as a subcutaneous collagen structure estimation model. Then, the skewness is determined by substituting the viscoelasticity calculated by the subcutaneous viscoelasticity calculating means 511 into the function. A function representing the relationship between subcutaneous viscoelasticity and skewness can be determined by a statistical method from data of a set of measured values of viscoelasticity and skewness.

In addition, the function representing the relationship between the viscoelasticity of the dermis and the value quantifying the degree of cohesion of collagen fibers obtained by performing image processing on microscopic images of dermal collagen fibers was calculated as follows: It is registered in the storage means 4 as an estimated model, and the degree of cohesion of the dermal collagen fibers is determined by substituting the viscoelasticity calculated by the dermal viscoelasticity calculation means 512 into the function. A function representing the relationship between dermal viscoelasticity and cohesion can be determined by a statistical method from data of a set of measured values of viscoelasticity and cohesion.

Note that the order of steps S71 and S72 may be changed arbitrarily. Further, as the correlation data, a calculation model trained by a set of measured values of viscoelasticity and collagen structure (strainness or degree of cohesion) may be used.

In step S73, the collagen structure estimating means 52 estimates the dermal collagen structure and subcutaneous collagen structure of the subject based on the skin viscoelasticity and viscoelasticity-collagen structure correlation data acquired in steps S71 and S72. At this time, the storage means 4 may store the analysis results.

When the estimation of the collagen structure is completed, an evaluation value of the collagen structure is calculated in step S74. In calculating the evaluation value, the collagen structure calculated in step S73 is used to calculate the evaluation value of each of the collagen structures in the dermis and subcutaneous tissue. For example, a value obtained by subjecting the calculated collagen structure (distortion degree or cohesion degree) to a standardized value by subjecting it to a predetermined process, a value obtained by dividing the collagen structure value into multiple stages and evaluating it, etc. can be used as the evaluation value.

As described above, according to the present embodiment, skin can be easily and non-invasively analyzed by having a subject reproduce specific facial expressions in sequence and analyzing a video of the facial expressions captured. Specifically, the physical quantities of skin changes caused by changes in facial expressions are measured from videos, and the viscoelasticity of the subject's skin is calculated from the measured values of the physical quantities for the subject, based on the correlation between the physical quantities and the viscoelasticity of the skin. be able to.

Furthermore, as the viscoelasticity of the skin, the viscoelasticity of the dermis and the viscoelasticity of the subcutaneous layer can be calculated, and their evaluation values can be calculated. Furthermore, the moisture content of the epidermis and its evaluation value can be calculated. Furthermore, the collagen structure of the skin can be estimated based on the viscoelasticity of the skin, and its evaluation value can be calculated. Thereby, detailed results can be presented by combining these evaluation values.

In the present embodiment, the storage means 4 stores functions representing the relationships obtained from data of various sets of measured values by statistical methods, and according to this function, the viscoelasticity, moisture content, etc. of the skin are determined. The morphology of the estimated collagen structure is shown. However, in addition to this, for example, by giving a set of measured values of physical quantity and viscoelasticity (or water content) as training data and letting the neural network learn, you can create a viscoelasticity calculation model, a water content calculation model, and a collagen structure estimation model. It is also possible to create a configuration. In addition, the configuration may be arbitrarily changed without departing from the spirit of the present invention, such as changing the order of various processes.

Hereinafter, the results of an experiment conducted by the present inventors to calculate skin viscoelasticity and the like based on measured values of physical quantities, and the results of estimating collagen structure based on viscoelasticity will be described. Although the present invention was made based on the following experimental results, the values and correlations used in the present invention are not limited to the following.

<Experiment results 1>
(1) Measurement of physical quantities Using the same method as in the above embodiment, moving images showing the process of changes in facial expressions were obtained for 100 people in their 20s to 60s, and physical quantities were measured. Specifically, the cheek analysis region was divided into basic analysis units using multiple fractionation patterns, and the amount of movement of each region was measured. Then, this amount of movement is further analyzed frame by frame by comparing the amount of movement of the basic analysis unit within each fractionation pattern and the amount of movement of the corresponding region between the fractionation patterns. The physical quantity was measured by converting the amount of movement into a scalar quantity. Measurements of such physical quantities were performed for each facial expression change.

(2) Measurement of viscoelasticity and water content The viscoelasticity and water content were measured for the same subjects as in (1). The viscoelasticity of the dermis was measured using a device capable of measuring viscoelasticity by suctioning the skin surface with negative pressure and monitoring the displacement of the skin when released. Furthermore, subcutaneous viscoelasticity was measured by acquiring an elastography image of the inside of the skin using elastography. The moisture content of the epidermis was measured using a device that measures the moisture content of the stratum corneum on the skin surface using a high-frequency sinusoidal current.

(3) Construction of various models A statistical analysis was performed on the results of (1) and (2) for each subject, and a viscoelasticity calculation model for the dermis and subcutaneous skin, and a model for calculating the moisture content of the epidermis were created. . Specifically, regression analysis was performed using an arbitrary method to obtain a function that describes the relationship between a physical quantity, viscoelasticity, and water content, and used as a viscoelasticity calculation model and a water content calculation model.

(4) Verification of accuracy Physical quantities, viscoelasticity, and water content were similarly measured for a different subject than in (1) to (3), and the accuracy of the model was verified. Specifically, using the model (3), viscoelasticity and water content were calculated from physical quantities and compared with actual measurement results. For viscoelasticity and water content, evaluation values were determined in multiple stages based on the values, and it was confirmed whether the evaluation values matched. As a result, the estimation accuracy when allowing a one-step deviation in the evaluation value was as follows.
Epidermal moisture content: 95%
Dermal viscoelasticity: 90%
Subcutaneous viscoelasticity: 90%

The results of (4) indicate that viscoelasticity and water content can be estimated with high accuracy using the measured values of physical quantities of skin changes obtained based on videos showing the process of facial expression changes.

<Experiment results 2>
(1) Measurement of viscoelasticity and evaluation of collagen structure Subcutaneous viscoelasticity was measured in the same manner as (2) in <Experimental Results 1>. Then, the subcutaneous fat portion was cut out from the ultrasound image of the same subject, and this was used as an analysis image to create a histogram using image analysis software. When this histogram was analyzed and the skewness was calculated, the lower the degree of fibrosis, the higher the skewness, and the higher the degree of fibrosis, the lower the skewness. That is, the fibrosis level of the collagen fiber structure surrounding subcutaneous fat can be evaluated based on the degree of skewness.

(2) Regression analysis Regression analysis was performed on the viscoelasticity measurement results in (1) and the skewness representing the fibrosis level of subcutaneous fat cells. As a result, it was found that a positive correlation exists between the viscoelasticity of the subcutaneous tissue and the skewness of the histogram of the ultrasound image of the subcutaneous fat layer.

The results of (2) showed that the fibrosis level of subcutaneous fat cells (subcutaneous collagen structure) can be estimated from the subcutaneous viscoelasticity. Furthermore, since the subcutaneous viscoelasticity can be estimated by the measured values of physical quantities of skin changes obtained based on videos showing the process of facial expression change, the subcutaneous collagen structure can be estimated indirectly based on the measured values of physical quantities. It was shown that it is possible to estimate the

An example of a detailed embodiment related to video shooting and output of analysis results will be described below.

<Video shooting>
As described above, when shooting a video, a guide for aligning facial parts such as eyes and contours can be displayed on the shooting screen. That is, the skin analysis device 10 or the terminal device 20 includes means for displaying a guide for aligning facial parts on the subject's face. Here, a more suitable method of photographing a moving image using the terminal device 20 as a photographing device will be described in detail. Note that although a case will be described below in which a tablet terminal is used as the terminal device 20, any other computer device equipped with a camera, a gyro sensor, an acceleration sensor, etc. can be used as the terminal device.

FIG. 14 is a diagram showing the external appearance of the terminal device 20. As shown in FIG. FIG. 14(a) is a diagram showing the back side of the terminal device 20. As shown here, a camera C is provided on the back side of the terminal device 20, and the photographing means 201 uses this to photograph a moving image. FIG. 14(b) is a diagram showing the front side of the terminal device 20. As shown here, the front side of the terminal device 20 is provided with a touch panel T, etc. that displays various displays for the photographer, receives operations from the operator, and the like.

In this embodiment, as shown in FIG. 14, the x-axis is along the long side, the y-axis is along the short side, and The z-axis is taken in the direction from the front side to the back side, and represents the angle at which the terminal device 20 is held and the angle at which it is rotated. However, the present invention is not limited to this, and may have a configuration in which the axis is taken in another manner, or a configuration in which the axis is taken based on the subject instead of the terminal device 20.

FIG. 15 is a flowchart showing the flow of video shooting in this embodiment. This shows an example in which a photographer uses the terminal device 20 to photograph a moving image that includes changes in facial expressions of a subject. It is preferable to explain to the subject in advance how his/her facial expression will change during imaging before performing each of the processes shown here. Note that this explanation may be performed by a method in which the terminal device 20 is configured to have a means for displaying a moving image for explaining the photographed content on the touch panel T, and this is shown to the photographer or the subject.

First, in step S81, the angle is reset. This is a process for determining the initial position of the terminal device 20. In this embodiment, the initial state of the terminal device 20 is such that the y-axis is parallel to the vertical direction, the x-axis is substantially parallel to the subject's face, and the z-axis is substantially perpendicular to the subject's face. position. That is, when viewed from above, the state is as shown in FIG. 16(a). More specifically, on a screen as shown in FIG. 17, guidelines for aligning the subject's eye height, chin height, and nose position are displayed on the touch panel T of the terminal device 20 as alignment guides. , is displayed superimposed on the subject's face photographed by camera C, thereby prompting the photographer to align the terminal device 20. Furthermore, by displaying a pointer P that moves according to the inclination of the z-axis, alignment becomes easier. Then, the initial position of the terminal device 20 is determined by resetting the angle by accepting an operation using the touch panel T (selecting the "reference value set" button in FIG. 17).

Note that it is preferable that the terminal device 20 be configured to be configured so that its rotation around the x-axis, that is, the tilt in the front-back direction, and the rotation around the z-axis, that is, the tilt in the left-right direction, can also be reduced. . For this purpose, for example, the gyro sensor, acceleration sensor, etc. included in the terminal device 20 identify the vertical direction and the horizontal direction perpendicular to the vertical direction based on gravitational acceleration, etc., and calculate the angles of the x-axis and the z-axis with respect to the horizontal direction. The differences may be displayed as inclinations around the respective axes, and the photographer may be prompted to adjust the angle of the terminal device 20 so that they approach 0 and fall within a predetermined allowable range.

After resetting the angle in this way and determining the initial position of the terminal device 20, the terminal device 20 is moved in step S82. In this embodiment, the terminal device 20 is rotated 25 degrees and the subject's face is photographed from an angle of 25 degrees diagonally forward to the left. Here, the terminal device 20 is moved to the right side toward the subject so that the subject's face is photographed diagonally from the front left, so that the state shown in FIG. , adjust the position. In other words, while keeping the camera C of the terminal device 20 facing the subject, the terminal device 20 may be moved so as to draw an arc with the center angle of 25 degrees centered on the subject. When such a movement is performed, the terminal device 20 is rotated 25 degrees counterclockwise around the y-axis, as shown in FIG. 16(b). Note that the shooting direction may be either left or right, or may be shot from both left and right.

A more specific adjustment method here is to display guidelines for adjusting the subject's eye height, chin height, and left eye position on the touch panel T, as shown in Figure 18(a), and then This is to encourage alignment. At this time, a pointer P that moves along with the rotation of the terminal device 20 around the y-axis and overlaps the center guideline when rotated 25 degrees is further displayed on the touch panel T. If it is determined that the guideline and pointer are appropriate, the display format, such as the color of the guideline and pointer, is changed.

Note that since the terminal device 20 is held by the photographer, the configuration here does not allow only the case where the angle is exactly 25 degrees, but allows it in advance, for example, 25 degrees ± 5 degrees. It is preferable to adopt a configuration in which an angular range is determined.

In this way, by displaying the guideline and aligning the terminal device 20 according to the guideline, even if the terminal device 20 does not move accurately in the circular arc described above, as long as the subject does not change direction, , it is possible to reliably take pictures from a predetermined angle. This makes it possible to easily and reliably adjust the shooting angle even in a configuration in which the photographer holding the terminal device 20 moves, rather than in a configuration in which the terminal device 20 (or the subject) is fixed and moved using an instrument. .

Note that in this embodiment, as described above, a configuration is shown in which three guidelines are displayed to match the subject's eye level, chin height, and left eye position, but the present invention is not limited to this. isn't it. By displaying points and lines as guides for aligning the positions of the subject's facial parts and adjusting the position of the terminal device 20 accordingly, images can be taken from an appropriate distance with the subject in an appropriate position. It becomes possible.

Note that at this point, even if the rotation angle is determined to be within the allowable range and appropriate, recording cannot be started because the focus and exposure adjustments have not been completed. Therefore, it is preferable to indicate to the photographer that recording cannot be started yet, such as by displaying a button that cannot be pressed or not displaying the button for starting recording.

As a result of the movement of the terminal device 20 in step S82, if it is determined in step S83 that the rotation angle of the terminal device 20 is within the predetermined range, the process proceeds to step S84, where automatic focus and exposure adjustment is performed. These adjustments may be made using any method, and especially when using a tablet terminal like the terminal device 20 according to the present embodiment, adjustments may be made using any method provided by the vendor of the OS (Operating System) running on the terminal. The configuration may be such that it is performed using an API (Application Programming Interface) related to camera control. Further, a configuration may be adopted in which not only automatic adjustment but also manual adjustment is possible. Here, the configuration includes a means to automatically adjust not only the focus and exposure but also other parameters related to image shooting such as ISO sensitivity, and the illuminance of the shooting environment is detected and when the illuminance is low, an LED (Light Emitting Diode) is installed. ) or the like may be provided with a means for illuminating the subject.

After the automatic focus and exposure adjustment is completed in step S84, the focus and exposure are locked (AFAE lock). If it is determined in step S85 that the focus and exposure adjustments have been completed, the process advances to step S86, and a state is set in which the press of the recording start button by the photographer is accepted.

After receiving an instruction to start recording from the photographer, recording by camera C is started in step S87. Note that even after the start of recording, it is preferable to continue measuring the rotation angle of the terminal device 20 and display it to the photographer as shown in FIG. 18(c). If the rotation angle falls outside the allowable range adjusted in step S86 during recording, the recording may be stopped immediately, or the recording may be continued after prompting the photographer to adjust the rotation angle. You can also use it as

In addition, in this embodiment, in order to capture a moving image that includes changes in the subject's facial expression, for example, as shown in FIG. There may be a case where the position is deviated from the state aligned in step S82, such as the position being below the guideline. Even in such a case, there is no problem as long as the subject's position and the position of the terminal device 20 do not move because the AFAE lock is being performed. Alternatively, the photographer may adjust the position of the terminal device 20 so as to maintain the position of the left eye in accordance with the guidelines, which is considered to be less likely to occur due to changes in facial expressions, even during recording. You may also use a method that continues to match.

Furthermore, the rotation angle of the terminal device 20 during recording is measured not only for the rotation around the y-axis, but also for the rotation around the x-axis and the z-axis, that is, the tilt of the terminal device 20 in the front-rear direction and left-right direction. It is preferable. Here, too, the configuration may be such that recording is immediately stopped when the tilt of the terminal device 20 exceeds a predetermined allowable range, or the configuration may be configured to prompt the photographer to adjust the rotation angle and continue recording. Good too. Alternatively, the terminal device 20 may be configured to perform correction according to the inclination of the terminal device 20, or the inclination of the terminal device 20 at each point in time during recording may be recorded, and the correction may be made during subsequent analysis by the skin analysis device 10. The configuration may be such that it can be performed. For example, if the correction is for horizontal tilt, it is sufficient to rotate each frame image in the recorded video, or if the correction is for longitudinal tilt, it is necessary to rotate each frame image. One method is to apply keystone correction.

If it is determined in step S88 that the recording should be ended, the recording is ended and recorded as a moving image file in step S89. Here, the determination of the end of recording may be based on the operation of the photographer, or the recording may be ended after a predetermined period of time has elapsed. The video file saved in this way is sent to the skin analysis device 10.

As described above, the terminal device 20 according to the present embodiment can be programmed with a dedicated program for a general-purpose tablet terminal equipped with sensors such as a gyro sensor and an acceleration sensor, and a camera C, without the need for special equipment or equipment. This can be achieved by deploying. Thereby, for analysis, it is possible to easily shoot a moving image while specifying conditions such as the shooting angle.

In the above embodiment, an example was explained in which the subject was photographed from diagonally in front of the left, but for example, the physical quantity could be calculated by recording videos from the same rotation angle on both the left and right sides and combining the videos. It's okay. When photographing a subject from both the left and right directions, in this embodiment, the amount of movement can be obtained for the analysis target for eight types of facial expression changes in both left and right directions, that is, a total of 16 types of facial expression changes. In step S13 of FIG. 3, by calculating the physical quantity using the movement amount obtained from the analysis target in these 16 types of facial expression changes, the physical quantity is calculated by comprehensively considering the skin changes obtained from the left and right facial expression changes. , the physical properties of the skin can be analyzed.

<Specific examples of physical quantities>
Examples of physical quantities used in the analysis include skin followability, elasticity, deformability, and the like. Here, "followability of the facial skin to a change in facial expression" refers to the degree of delay in the movement of the facial skin following a change in facial expression. When facial expressions change, the skin on the face changes with a delay in response to the movement, and the smaller the degree of delay, the better the followability.

The followability can be quantitatively evaluated by observing two arbitrary points on the face when the facial expression changes and measuring the degree of timing difference between the movements of these two points. For example, a method can be considered in which the speed of movement in a specific analysis region is specified based on the amount of movement, and quantitatively measured as the difference in time at which the speed of movement reaches its maximum between the analysis regions. At this time, an analysis area corresponding to the position of the face that moves most noticeably during a change in facial expression and an analysis area corresponding to other face positions are used to calculate the difference in the time at which the movement speed reaches its maximum between them. It is preferable to measure.

Furthermore, "stretchability of facial skin when facial expression changes" refers to the ease with which the skin stretches and contracts when facial expression changes. For example, when there is an expression change in which the facial skin stretches, the higher the ratio of the increase in the distance in the stretching direction in a given region to the increase in the overall distance in the stretching direction, the better the stretchability. can be evaluated. For example, from the amount of movement such as the displacement of the coordinates of feature points along the direction of skin extension when facial expressions change, it is possible to evaluate the rate of increase in distance in the extension direction in a specific analysis region as elasticity. can.

As for deformability, it is possible to measure how an analysis region corresponding to an arbitrary position on the face is deformed (distorted). Although the method for measuring deformability is not limited, it may be calculated using, for example, the amount of displacement of the feature point, the amount of movement such as the direction of movement.

In particular, it is known that trackability and elasticity each have a negative correlation with age, and both tend to decrease with age. It is also known that there is a positive correlation between conformability and subcutaneous viscoelasticity, and it is possible to calculate viscoelasticity based on the correlation between conformability and viscoelasticity. Conceivable.

<Output of analysis results>
The skin analysis device 10 may further include an output means for outputting the analysis results. The output means outputs, as analysis results, information based on calculation results of dermal and subcutaneous viscoelasticity, epidermal moisture content, dermal and subcutaneous collagen structures, evaluation values thereof, and the like. The form of output is not limited, and may be displayed as a static or dynamic image on the terminal device 20, or may be output as sound, vibration, or text.

Furthermore, in addition to the calculation results of viscoelasticity and water content and the estimation results of collagen structure, it also outputs information based on physical quantities used in analysis, that is, physical quantities that have a correlation with any of viscoelasticity, water content, and collagen structure. You may. This makes it possible to clearly show how measured values and values obtained from captured videos affect the analysis results.

<Modifications related to estimation of collagen structure>
In the above embodiment, the collagen structure estimating means 52 estimates the collagen structure using the calculation result by the viscoelasticity calculating means 51 and the viscoelasticity-collagen structure correlation data. Not limited. As mentioned above, a correlation is observed between physical quantities and viscoelasticity, and a correlation is also observed between viscoelasticity and collagen structure, so there is also a direct correlation between physical quantities and collagen structure. High probability.

Therefore, for example, the collagen structure estimating means 52 may directly estimate the collagen structure of the subject's skin from the measured value of the physical quantity for the subject, based on the correlation between the physical quantity and the collagen structure of the skin.

In this case, the storage means 4 stores a function representing the relationship between the physical quantity and an index related to the collagen structure as physical quantity-collagen structure correlation data, and the physical quantity measured by the physical quantity measuring means 3 is added to this function. By substituting , an index of collagen structure can be calculated.

Note that even when estimating the collagen structure directly from physical quantities without calculating viscoelasticity, the subcutaneous collagen structure, the dermal collagen structure, and their evaluation values can be estimated in the same manner as in the previously described embodiments. is preferred. That is, the storage means 4 stores the correlation between the physical quantity and the subcutaneous collagen structure (for example, the skewness of a histogram obtained by analyzing an ultrasound image of the subcutaneous fat layer), and the correlation between the physical quantity and the dermal collagen structure (for example, the dermal collagen structure). The correlation with the value obtained by quantifying the degree of cohesion of collagen fibers obtained from microscopic images of the fibers is memorized.

In this way, by estimating the collagen structure based on the correlation between the physical quantity and the collagen structure, an improvement in estimation accuracy can be expected compared to the case where the collagen structure is estimated via viscoelasticity.

10 Skin analysis device 1 Video acquisition means 2 Analysis target extraction means 3 Physical quantity measurement means 31 Feature point extraction means 32 Movement amount measurement means 33 Movement amount analysis means 4 Storage means 5 Analysis section 51 Viscoelasticity calculation means 511 Subcutaneous viscoelasticity calculation means 512 Dermal viscoelasticity calculation means 52 Collagen structure estimation means 521 Subcutaneous collagen structure estimation means 522 Dermal collagen structure estimation means 53 Water content calculation means 6 Evaluation means 61 Dermal viscoelasticity evaluation means 62 Subcutaneous viscoelasticity evaluation means 63 Dermal collagen structure evaluation means 64 Subcutaneous Collagen structure evaluation means 65 Water content evaluation means 20 Terminal device 201 Photographing means 202 Display section C Camera T Touch panel P Pointer

Claims (34)

A skin analysis method, comprising calculating the viscoelasticity of the subject's skin from the measured values of the physical quantity for the subject, based on the correlation between the physical quantity of skin changes caused by changes in facial expression and the viscoelasticity of the skin. And,
The physical quantity is measured based on a video including the process of facial expression change,
The video is characterized in that it is shot from a direction rotated within a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face,
The video is captured with the subject viewed from the front as an initial position, the rotation angle of the imaging device from the initial position is obtained, and the video is captured when the rotation angle is within a range of 20 to 30 degrees. A skin analysis method, characterized in that the skin analysis method is characterized in that the skin is photographed based on the permission. The skin analysis method according to claim 1, wherein the viscoelasticity includes at least one of dermal viscoelasticity and subcutaneous viscoelasticity. The method is characterized in that the viscoelasticity of the subject's skin is calculated from the measured values of the physical quantity about the subject using a viscoelasticity calculation model created by inputting a plurality of sets of the measured values of the physical quantity and viscoelasticity. , the skin analysis method according to claim 1 or claim 2. 4. The method according to claim 1, wherein the collagen structure of the subject's skin is estimated from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin. skin analysis method. Estimating the collagen structure of the subject's skin from the viscoelasticity calculated for the subject using a collagen structure estimation model created by inputting a plurality of sets of measured values of the viscoelasticity and the collagen structure of the skin. The skin analysis method according to claim 4, characterized by: The skin analysis method according to any one of claims 1 to 5, characterized in that one or more skin evaluation values are calculated based on the viscoelasticity. 7. The skin analysis method according to claim 1, further comprising outputting an analysis result including information based on the viscoelasticity calculation result. The skin analysis method according to claim 7, characterized in that an analysis result including information based on the measured value of the physical quantity is outputted together with information based on the calculation result of the viscoelasticity. A skin analysis method, comprising estimating the collagen structure of the skin of a subject from the measured values of the physical quantity for the subject, based on the correlation between the physical quantity of skin changes caused by changes in facial expression and the collagen structure of the skin. And,
The physical quantity is measured based on a video including the process of facial expression change,
The video is characterized in that it is shot from a direction rotated within a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face,
The video is captured with the subject viewed from the front as an initial position, the rotation angle of the imaging device from the initial position is obtained, and the video is captured when the rotation angle is within a range of 20 to 30 degrees. A skin analysis method, characterized in that the skin analysis method is characterized in that the skin is photographed based on the permission. The skin analysis method according to claim 9, further comprising outputting an analysis result including information based on the estimation result of the collagen structure. 11. The skin analysis method according to claim 10, further comprising outputting an analysis result including information based on the measurement value of the physical quantity as well as information based on the estimation result of the collagen structure. The skin analysis method according to any one of claims 1 to 11 , characterized in that a guide for aligning facial parts on the subject's face is displayed on the screen for shooting the video. The videos include a video taken from a direction rotated clockwise within a range of 20 to 30 degrees with the front as 0 degrees around the vertical axis passing through the subject's face, and a video taken from a direction rotated clockwise within a range of 20 to 30 degrees around the vertical axis passing through the subject's face. 13. The method according to any one of claims 1 to 12 , characterized in that both a moving image taken from a direction rotated counterclockwise within a range of 20 to 30 degrees with the front as 0 degrees is used. Skin analysis method. The physical quantity is measured based on at least one of the magnitude of the velocity, the direction of velocity, the magnitude of acceleration, and the direction of acceleration of the movement of the feature points included in the face. The skin analysis method according to any one of items 1 to 13 . 15. The skin analysis method according to claim 1, wherein the physical quantity is selected from skin followability, stretchability, and deformability. According to any one of claims 1 to 15 , the method calculates the moisture content of the epidermis of the subject from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the moisture content of the epidermis. skin analysis method. Calculating the water content of the subject from the measured values of the physical quantity for the subject using a water content calculation model created by inputting a plurality of sets of measured values of the physical quantity and water content, The skin analysis method according to claim 16 . A skin analysis system that analyzes a subject's skin based on skin movements when facial expressions change,
a physical quantity measuring means for measuring a physical quantity of skin change caused by the facial expression change;
Storage means for storing the correlation between the physical quantity and the viscoelasticity of the skin;
a viscoelasticity calculation means for calculating the viscoelasticity of the subject's skin from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the viscoelasticity of the skin ,
Further comprising a video acquisition means for acquiring a video including the process of facial expression change,
The physical quantity measuring means measures the physical quantity based on the moving image,
The video acquisition means is characterized in that it acquires a video of the subject's face from a direction rotated in a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face,
The video acquisition means acquires the video taken by a terminal device for photographing the subject under predetermined conditions,
The terminal device is
means for obtaining a rotation angle of the terminal device from the initial position, with a state in which the subject is viewed from the front as an initial position;
means for permitting recording of the video when the rotation angle is within a range of 20 to 30 degrees;
A skin analysis system comprising: means for photographing the video based on the permission. 19. The skin analysis system according to claim 18 , wherein the viscoelasticity calculation means calculates at least one of dermal viscoelasticity and subcutaneous viscoelasticity. The storage means stores a viscoelasticity calculation model created by inputting a plurality of sets of measured values of the physical quantity and viscoelasticity,
20. The skin analysis system according to claim 18, wherein the viscoelasticity calculation means uses the viscoelasticity calculation model to calculate the viscoelasticity from the physical quantity measured for the subject. the storage means stores a correlation between the viscoelasticity and the collagen structure of the skin;
18. The method further comprises collagen structure estimating means for estimating the collagen structure of the subject's skin from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin. ~20 . The skin analysis system according to any one of 20. The storage means stores a collagen structure estimation model created by inputting a plurality of sets of measured values of the viscoelasticity and collagen structure of the skin,
22. The skin analysis system according to claim 21 , wherein the collagen structure estimating means estimates the collagen structure of the subject's skin from the viscoelasticity calculated for the subject using the collagen structure estimation model. The skin analysis system according to any one of claims 18 to 22 , further comprising evaluation means for calculating one or more skin evaluation values based on the viscoelasticity of the subject's skin. The skin analysis system according to any one of claims 18 to 23 , further comprising an output means for outputting an analysis result including information based on the viscoelasticity calculation result. 25. The skin analysis system according to claim 24 , wherein the output means outputs an analysis result including information based on the measured value of the physical quantity as well as information based on the calculated result of the viscoelasticity. A skin analysis system that analyzes a subject's skin based on skin movements when facial expressions change,
a physical quantity measuring means for measuring a physical quantity of skin change caused by the facial expression change;
storage means for storing the correlation between the physical quantity and the collagen structure of the skin;
Collagen structure estimating means for estimating the collagen structure of the subject's skin from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the collagen structure of the skin ,
Further comprising a video acquisition means for acquiring a video including the process of facial expression change,
The physical quantity measuring means measures the physical quantity based on the moving image,
The video acquisition means is characterized in that it acquires a video of the subject's face from a direction rotated in a range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face,
The video acquisition means acquires the video taken by a terminal device for photographing the subject under predetermined conditions,
The terminal device is
means for obtaining a rotation angle of the terminal device from the initial position, with a state in which the subject is viewed from the front as an initial position;
means for permitting recording of the video when the rotation angle is within a range of 20 to 30 degrees;
A skin analysis system comprising: means for photographing the video based on the permission. The skin analysis system according to claim 26 , further comprising an output means for outputting information based on the estimation result of the collagen structure. 28. The skin analysis system according to claim 27 , wherein the output means outputs an analysis result including information based on the estimated result of the collagen structure as well as information based on the measured value of the physical quantity. 29. The skin analysis system according to claim 18 , further comprising means for displaying a guide for aligning facial parts on the subject's face on the screen for photographing the moving image. The video acquisition means includes a video taken from a direction rotated clockwise around a vertical axis passing through the subject's face in a range of 20 to 30 degrees with the front as 0 degrees, and a video taken in a vertical direction passing through the subject's face. 30. A moving image taken from a direction rotated counterclockwise around an axis of 0 degrees in a range of 20 to 30 degrees counterclockwise. skin analysis system. The physical quantity measuring means measures the physical quantity based on at least one of the following: the magnitude of the velocity, the direction of the velocity, the magnitude of the acceleration, and the direction of the acceleration of the movement of the feature points included in the face. The skin analysis system according to any one of claims 18 to 30 . 32. The skin analysis system according to claim 18, wherein the physical quantity measuring means measures any one of followability, elasticity, and deformability of the skin as the physical quantity. the storage means stores a correlation between the physical quantity and the moisture content of the epidermis;
18. The method further comprises a moisture content calculation means for calculating the moisture content of the epidermis of the subject from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the moisture content of the epidermis. 32. The skin analysis system according to any one of 32 . The storage means stores a moisture content calculation model created by inputting a plurality of sets of measured values of the physical quantity and moisture content,
34. The skin analysis system according to claim 33 , wherein the moisture content calculation means uses the moisture content calculation model to calculate the moisture content from the physical quantity measured for the subject.