JP2008146318A - Emotion estimation apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an emotion estimation apparatus that recognizes unspecified persons' expressions and estimates the persons' emotions. <P>SOLUTION: The emotion estimation apparatus 100 comprises an expression map generation means 11 for learning expression images of specific persons associated with predetermined emotions to generate expression maps based on a neural network, a region division means 12 for dividing the expression maps for the respective specific persons generated by the expression map generation means 11 into a plurality of regions according to the predetermined emotions, and an emotion estimation means 13 for estimating an unspecified person's emotion according to an expression image U of the unspecified person and the plurality of expression maps with the regions divided by the region division means 12. A region standardization means 14 is further provided for standardizing corresponding regions of the plurality of expression maps, and the emotion estimation means 13 estimates the unspecified person's emotion according to the expression image U of the unspecified person and the plurality of expression maps with the regions standardized. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an emotion estimation apparatus using a self-organizing map based on a Kohonen type neural network, and more particularly to an emotion estimation apparatus that recognizes the expression of an unspecified person and estimates the emotion of the person.

  Conventionally, emotional information indicating an emotion generally thought to be held by a user who has watched a scene of a movie is stored in association with the facial expression image of the user who has viewed the scene, and the relationship between the emotional information and the facial expression image is stored. There has been known an information processing apparatus that estimates a specific user's emotion from a specific user's facial expression image using a learned neural network emotion model (see, for example, Patent Document 1).

Since this information processing apparatus creates an emotion model for each user and estimates the emotion for each user based on the emotion model for each user, it is not affected by the difference in the reaction of each user to the movie scene. It is possible to estimate the emotion of each user more accurately.
JP 2005-346471 A

  However, the information processing apparatus described in Patent Document 1 is for estimating the emotion of a specific person who has learned an expression image from an emotion model, and the expression of an unspecified person who has not learned an expression image from the emotion model. The emotions of the unspecified person cannot be estimated based on the image.

  In view of the above-described points, an object of the present invention is to provide an emotion estimation apparatus that uses a Kohonen-type neural network to recognize the facial expression of an unspecified person and estimate the emotion of the unspecified person.

  In order to achieve the above-mentioned object, an emotion estimation device according to a first invention comprises an expression map generation means for generating an expression map by a neural network by learning an expression image of a specific person associated with a predetermined emotion. The expression map corresponding to each of the specific persons generated by the expression map generation means is divided into a plurality of areas based on the predetermined emotion, the expression image of the unspecified person, and the area classification means Emotion estimation means for estimating an emotion of an unspecified person based on a plurality of facial expression maps having divided areas.

  Further, the second invention is the emotion estimation device according to the first invention, wherein the region segmenting means predetermines at least one of the center position, shape or area of the segmented region. To do.

  Further, the third invention is the emotion estimation device according to the first invention, wherein the region is divided by the region dividing means, and the region is shared by the corresponding region in each of the plurality of facial expression maps. Means for estimating the emotion of the unspecified person based on the facial expression image of the unspecified person and a plurality of facial expression maps having areas shared by the area sharing means. Features.

  The fourth invention is the emotion estimation device according to the third invention, wherein the region sharing means shares at least one of the shape, center position or area of the corresponding region. And

  With the above-described means, the present invention can provide an emotion estimation device that recognizes the facial expression of an unspecified person and estimates the emotion of the unspecified person using a Kohonen type neural network.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example of an emotion estimation apparatus 100 according to the present invention. The emotion estimation device 100 includes a control unit 1, an imaging unit 2, and a display unit 3. The emotion estimation apparatus 100 estimates the emotion of the person based on the facial expression image of the person captured by the imaging unit 2, and displays the estimation result. 3 is output.

  The control unit 1 is a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc., and includes a facial expression image processing means 10, a facial expression map generation means 11, an area classification means 12, Programs corresponding to the emotion estimation means 13 and the area sharing means 14 are stored in the ROM, and the programs are expanded on the RAM and the corresponding processing is executed by the CPU.

  The imaging unit 2 is a means for capturing a facial expression image of a person, for example, a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal-Oxide Semiconductor) camera.

  The display unit 3 is a means for outputting the estimation result of the emotion estimation apparatus 100. For example, the display unit 3 is a liquid crystal display, and includes “anger”, “joy”, “sadness”, “surprise”, “disgust”, “ Of emotions such as “fear” and “praise”, emotions that match the facial expression image captured by the imaging unit 2 are expressed and output as characters, video, illustrations, a self-organizing map described later, or the like. The emotion estimation apparatus 100 may output the estimation result by voice.

  Next, the facial expression image processing means 10 included in the control unit 1 will be described.

  The facial expression image processing means 10 is a means for processing so that an image captured by the imaging unit 2 can be input to a neural network. Here, the flow of processing for processing a facial expression image so that the facial expression image processing means 10 can input to the neural network will be described with reference to FIGS. 2 is a flowchart showing the flow of facial expression image processing, and FIG. 3 is a facial expression image for explaining each step of facial expression image processing.

  First, the facial expression image processing means 10 performs edge detection, filtering processing, etc. on the facial expression image (for example, a 512 × 512 pixel bitmap file) as shown in FIG. Image processing is performed to specify the position of the human eye as shown in FIG. 3B (step S1).

  Thereafter, the facial expression image processing means 10 similarly specifies the position of the person's nose as shown in FIG. 3C (step S2), and converts the facial expression image to gray scale by binarization processing (step S3). . The facial expression image processing means 10 may specify the positions of the eyes and nose of the person after converting the facial expression image to gray scale.

  After that, the facial expression image processing means 10 recognizes the inclination of the face reflected in the facial expression image based on the specified eye and nose positions, and translates or rotates the facial expression image by affine transformation, thereby correcting the facial inclination. Is corrected (step S4), and the facial expression image is normalized so that the nose position is at the center of the facial expression image as shown in FIG. 3D (step S5).

  After that, the facial expression image processing means 10 is a facial expression part of a predetermined size (for example, 256 × 256 pixels) from the facial expression image (for example, an image part including eyes, nose, mouth, and cheeks expressing a facial expression of a person, The image compression technique such as JPEG (Joint Photographic Expert Group) method is used so that the image of the facial expression part cut out can be used as an input image to the neural network (step S6). To a predetermined size (for example, 64 × 64 pixels) (step S7).

  Next, the expression map generating means 11 included in the control unit 1 will be described.

  The expression map generation means 11 is a means for generating an expression map which is a self-organizing map using a Kohonen type neural network. Here, a flow of processing in which the expression map generation unit 11 generates an expression map will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of facial expression map generation processing.

  "Self-organizing map" is a neural network technique that reflects multidimensional data on a two-dimensional plane, placing data with similar characteristics close to the map, and other things that are separated on the map This is a technique for visualizing high-dimensional information by placing it at a position and making it easy to intuitively understand the relationship between data.

  First, the facial expression map generation means 11 reads values such as a learning coefficient α, a learning radius r, and a set learning count t stored in a storage device such as a ROM (step S11), and has a predetermined size (32 × 32 nodes). The value of each node in the facial expression map is initialized with a random number (step S12).

  The facial expression map is composed of two-dimensional coordinates, and has one neuron at each coordinate. The neuron is represented as a weight vector w having the number of elements equal to the number of pixels of the input image (for example, 4096 in the case of 64 × 64 pixels). Note that the input image is handled as a vector having the number of elements of the number of pixels, and each element is, for example, a value of luminance of each pixel.

  The expression map initialization is a process of initializing each element of the weight vector w with a random number, and the “set learning count t” is a learning image that is an input image (the weight vector w satisfying a predetermined condition). This is a setting value indicating the number of times each element is updated.

  The learning image is a group of facial expression images obtained by capturing information on a specific person while presenting information that gives a predetermined emotion such as a movie or a photograph to the specific person. Each learning image is associated with a predetermined emotion. .

  The “learning coefficient α” is a coefficient used to update the weight of the neuron, and the following formula (1)

Used in. Here, i is a number indicating the position on the facial expression map, and x is a vector indicating the learning image.

  “Learning radius r” is a set value for specifying a range on the expression map, and the expression map generation means 11 is centered on a neuron having the maximum similarity, which will be described later, and has a radius r (unit is a neuron). The weight vector w of the neuron included in the circle “Yes” is updated using the above equation (1).

  Referring to FIG. 4 again, the description of the flow of processing in which the expression map generation unit 11 generates the expression map will be continued.

  Thereafter, when the learning image processed by the facial expression image processing means 10 is acquired (step S13), the facial expression map generation means 11 calculates the similarity between the learning image and each neuron (step S14).

  The similarity is represented by, for example, the Mahalanobis distance or the Euclidean distance between the learning image vector x and the weight vector w of each neuron. Preferably, the Mahalanobis distance is used.

  After the similarity between the input learning image and all the neurons is calculated, the facial expression map generation means 11 identifies the neuron with the maximum similarity (step S15), and the similarity on the facial expression map. Each element of the weight vector w possessed by all the neurons within the range indicated by the circle whose center is the position of the neuron where is the maximum and whose radius is the learning radius r is updated by Equation (1) (step S16).

  Thereafter, the facial expression map generation means 11 determines whether learning has been performed using all of the learning images prepared in advance (for example, one for each emotion) (step S17). If all the learning images have not been used yet (NO in step S17), the next learning image is read from a storage device such as a hard disk (step S13), and steps S14 to S16 are executed. This series of processes is repeated until learning is performed using the learning images.

  When learning is performed using all the learning images (YES in step S17), the facial expression map generation unit 11 compares the preset learning number t with the number of learnings performed so far ( Step S18).

  When the number of learnings performed is equal to or less than the set learning number t (NO in step S18), the facial expression map generation unit 11 increments the number of learnings (step S19) and uses all the learning images again to perform steps S13 to S13. Step S17 is executed.

  The facial expression map generation means 11 repeats Steps S13 to S17 until the number of learnings performed exceeds the set learning number t, and when the number of performed learnings exceeds the set learning number t (YES in Step S18), The expression map is divided for each emotion by the area dividing means 12 described later.

  Next, the area sorting means 12 included in the control unit 1 will be described.

  The area classifying means 12 is a means for classifying the facial expression map generated by the facial expression map generating means 11 into emotional areas (hereinafter referred to as “emotional areas”).

  FIG. 5 is a diagram showing an example of a facial expression map segmented by the region segmenting means 12, and each of the regions partitioned by the grid corresponds to a neuron. The area classification unit 12 labels each emotion area classified by the boundary based on the emotion information associated with each learning image, and, for example, “angry” is added to the area R1 of the facial expression map shown in FIG. The labels are associated with each other so that the “joy” label is associated with the region R2. In addition, labels representing emotions are also associated with the regions R3 to R11.

  The area segmentation means 12 classifies a neuron most similar to each of the plurality of learning images representing each emotion (a neuron representing each emotion, hereinafter referred to as “central neuron”) and any one of the areas. The Mahalanobis distance or Euclidean distance between the target neuron (hereinafter referred to as “target neuron”) is calculated, and a boundary is set so that the central neuron with the smallest calculated value and the target neuron are in the same area on the expression map. To pull.

  In addition, the region classification unit 12 may calculate the Mahalanobis distance or the Euclidean distance between the weight vectors in two adjacent neurons on the expression map, and draw a boundary between neurons where the calculated value is a predetermined value or more. .

  Alternatively, the region classifying unit 12 determines in advance the position of the central neuron representing each emotion on the facial expression map regardless of each of the plurality of learning images (in this case, the central neuron in this case is particularly “fixed center”). Neuron ”)), the Mahalanobis distance or Euclidean distance between the fixed central neuron and the target neuron is calculated, and the boundary is set so that the fixed central neuron and the target neuron with the smallest calculated value are in the same area on the facial expression map. (Hereinafter, this classification method is referred to as “central neuron fixed clustering”). In this case, the position of the fixed central neuron on the facial expression map is common to a plurality of facial expression maps.

  Thereby, the area | region classification means 12 can form each emotion area | region in the common position on several facial expression maps, without being influenced too much by the specific facial expression image of a specific person.

Alternatively, the region segmenting means 12 may select the M1 point (k, l) and the M2 point (m, The distance L to n) is represented by the square root of (km) ² + (1−n) ² , and the predetermined distance is the same distance (number of neurons) as the learning radius r). The target neuron and its fixed central neuron may be restricted so as not to be in the same region on the expression map (hereinafter, this classification method is referred to as “conditional clustering”).

  This is because a target neuron whose distance on a facial expression map from a specific fixed center neuron is equal to or larger than the learning radius is not affected by the fixed center neuron during learning. This is because only the target neuron affected by the fixed central neuron is divided into the same area as the fixed central neuron.

  Thereby, the area | region classification means 12 can prevent the enclave of each emotion area | region, a protrusion land, etc. being produced | generated on an expression map.

  FIG. 6 is a diagram for explaining conditional clustering by the region segmenting means 12, FIG. 6 (A) shows segmentation by central neuron fixed clustering, and FIG. 6 (B) is improved by conditional clustering. FIG. 6A shows division by central neuron fixed clustering. In addition, the gray part of FIG. 6 (A) shows the part improved by conditional clustering.

  Alternatively, the area segmentation means 12 has the same distance between fixed central neurons and the area of each emotional area (the area on the expression map, specifically the number of neurons included in the area) in each expression map. The expression map may be divided by setting the arrangement, shape, and size of each emotion area in advance.

  The “distance between fixed central neurons” refers to the distance when transitioning from one facial expression to another, and “transition” refers to the process of transition from one facial expression to another. Also, when transitioning from one facial expression to another, facial expression changes through a path that guarantees the minimum change in energy, and the facial expression image (neuron) on the path is an intermediate image (neuron) between facial expressions Are treated as This route is determined by DP (Dynamic Programming) matching or the like.

  FIG. 7 is a schematic diagram showing an example of an expression map in which the arrangement, shape, and size of each emotion area are equal. E1 to E7 show the fixed central neurons of each of the seven emotion areas, and have the same shape and size. Each hexagon (indicated by a dotted line) indicates the shape and size of each emotional area.

  The position of the fixed central neuron is arranged at the center of each hexagon. Each emotion region is cyclically arranged on the two-dimensional plane in the same order. For example, an emotion region having a fixed central neuron E1 always has an emotion region having a fixed central neuron E7 adjacent to the left side.

  Thus, equalizing the distance between the fixed central neurons equalizes the number of intermediate neurons between each facial expression, and the emotion estimation apparatus 100 is not affected excessively by a specific facial expression, and the facial expression image Emotion can be estimated while treating

  Next, the emotion estimation means 13 included in the control unit 1 will be described.

  The emotion estimation unit 13 is a unit for estimating the person's emotion from the facial expression image captured by the imaging unit 2 using the facial expression map segmented by the region segmenting unit 12. Here, the flow of processing in which the emotion estimation means 13 estimates an emotion will be described with reference to FIG. FIG. 8 is a flowchart showing the flow of emotion estimation processing.

  Initially, the emotion estimation means 13 acquires the facial expression image P used as the object of emotion estimation via the imaging part 2 (step S21).

  Thereafter, the emotion estimation means 13 processes the expression image P so that it can be input to the neural network by the expression image processing means 10 (step S22), and the processed expression image P (handled as a vector) and expression map (expression map generation). It is a facial expression map that has already been learned by the means 11 and has already been classified by the area classification means 12. The similarity between each neuron on the top is calculated (step S23).

  The emotion estimation means 13 uses the Mahalanobis distance or the Euclidean distance as the similarity between the expression image P and each neuron, and the similarity between the expression image P and the expression image P decreases as the value of the Mahalanobis distance or the Euclidean distance decreases. High.

Thereafter, the emotion estimation means 13 compares the calculated maximum value of similarity with a predetermined threshold (step S24), and if the maximum value of similarity is equal to or greater than the predetermined threshold (YES in step S24),
The region to which the neuron having the maximum similarity belongs is specified, the label that the specified region has is estimated to be the emotion that the person of the facial expression image P has, and that effect is output to the display unit 3 (step S25). .

  On the other hand, when the maximum value of the similarity is lower than the predetermined threshold (NO in step S24), the emotion estimation unit 13 cannot estimate the emotion that the person of the expression image P has, or the person of the expression image P is an unspecified person (It means a person who has not learned the neural network.) This is output to the display unit 3 (step S26).

  FIG. 9 is a diagram showing the distribution of similarity between the facial expression image P and each neuron on the facial expression map calculated by the emotion estimation means 13, and the darker the color, the higher the similarity. As shown in FIG. 9, when the neuron with the maximum similarity (the maximum value of the similarity is equal to or greater than a predetermined threshold) is located in the upper right part of the region R5, the emotion estimation means 13 It is estimated that the emotion of the person in the facial expression image P is the emotion indicated by the label in the region R5 (for example, “surprise”).

  Also, even if the emotional region is near the boundary, even if it belongs to a different emotional region, adjacent neurons have a relatively high similarity, so there may be a distribution of similarity across the boundary. The means 13 not only estimates the emotion according to which emotion region the neuron with the maximum similarity belongs, but also can estimate the emotion according to which emotion region the entire similarity distribution belongs to. Accurate emotion estimation is possible.

  Next, the area sharing means 14 included in the control unit 1 will be described.

  The area sharing means 14 is a means for sharing each emotion area in a plurality of facial expression maps corresponding to each of the specific persons generated by the facial expression map generation means 11. This is a means for generating a facial expression map used to estimate the emotion of an unspecified person.

  For example, the region sharing unit 14 acquires a plurality of facial expression maps generated for each specific person by the facial expression map generation unit 11, and each emotion region (for example, a plurality of facial expression maps) in each of the plurality of facial expression maps. Adjust the positions of the centroid vectors (central neurons) of the “angry” area of each to be equal.

  Further, the area sharing means 14 may adjust the position of the central neuron of each corresponding emotion area so that the shape or area of each corresponding emotion area becomes equal in each of the plurality of facial expression maps.

  In this case, the region sharing means 14 may determine the position of the central neuron of each emotion region while adjusting the shape or area of each emotion region using, for example, an LBG (Linde-Buzo-Gray) algorithm. Good.

  FIG. 10 shows three facial expression maps generated using learning images of three specific persons A, B, and C, and each of the central neurons CV1 to CV9 in each emotional region by the region sharing means 14 is shown. Indicates that the position on the facial expression map is shared.

  Thereafter, the emotion estimation unit 13 estimates the emotion of the person from the facial expression image of the unspecified person captured by the imaging unit 2 using a plurality of facial expression maps shared by the area sharing unit 14.

  The emotion estimation apparatus 100 can obtain the same effect as the sharing by the area sharing means 14 by using the emotion area divided by the area sorting means 12 using central neuron fixed clustering or conditional clustering. In this case, sharing by the area sharing unit 14 is not necessary.

  Here, the flow of processing in which the emotion estimation means 13 estimates an emotion will be described with reference to FIG. FIG. 11 is a flowchart showing the flow of emotion estimation processing.

  First, the emotion estimation means 13 acquires the facial expression image U of an unspecified person who is an emotion estimation target via the imaging unit 2 (step S31).

  Thereafter, the emotion estimation means 13 processes the expression image U so that it can be input to the neural network by the expression image processing means 10 (step S32), and the processed expression image U (handled as a vector) and a plurality of expression maps (regions). A plurality of facial expression maps in which the position of the central neuron is determined in advance by the classifying means 12, or a plurality of facial expression maps in which the position of the central neuron is adjusted by the area sharing means 14). The similarity is calculated (step S33).

  The emotion estimation means 13 uses the Mahalanobis distance or the Euclidean distance as the similarity between the expression image U and each neuron, and the similarity between the expression image U and the expression image U decreases as the value of the Mahalanobis distance or the Euclidean distance decreases. High.

  Thereafter, the emotion estimation means 13 compares the calculated maximum similarity value with a predetermined threshold value in each facial expression map (step S34), and if the maximum similarity value is equal to or greater than the predetermined threshold value (step S34). (YES in S34), the emotion region to which the neuron having the maximum similarity belongs is specified (step S35), and the value (for example, a number indicating the type of emotion) indicated by the specified emotion region is the value of the facial expression map. (Hereinafter referred to as “expression map value”) and stored in a storage device such as a RAM.

  On the other hand, when the maximum value of the similarity is below the predetermined threshold (NO in step S34), the emotion estimation means 13 invalidates the facial expression map value (for example, sets the value to zero) (step S36).

  Thereafter, the emotion estimation means 13 determines whether or not the similarity is calculated for all the facial expression maps of the emotion estimation device 100 (step S37), and the similarity is calculated for all the facial expression maps. If not (NO in step S37), another facial expression map is obtained (step S38), and the similarity between the facial expression image U and each neuron in the newly obtained facial expression map is calculated and newly obtained. The performed facial expression map value is stored (steps S33 to S36).

  When the similarity is calculated for all the facial expression maps (YES in step S37), the emotion estimation means 13 estimates the emotion of the unspecified person based on the facial expression map values stored in the RAM so far ( Step S39).

  The emotion estimation means 13 may estimate (output) the emotion indicated by the most frequent facial expression map value among all facial expression map values as the emotion of an unspecified person, and includes a neuron having a high similarity. A predetermined number of facial expression maps may be extracted in descending order of similarity, and the emotion represented by the facial expression map value having the highest frequency among facial expression map values included in those facial expression maps may be output as the emotion of an unspecified person. .

  With the above configuration, the emotion estimation apparatus 100 can more accurately estimate the emotion of the specific person by learning the neural network using the facial expression image of the specific person.

  Moreover, since the emotion estimation device 100 estimates the emotion of the specific person based on the distribution of the similarity between the facial expression image of the specific person and each neuron, the facial expression obtained by capturing a facial expression different from the facial expression in the learning image Even when an image is input, the emotion of the specific person can be estimated more accurately.

  In addition, emotion estimation apparatus 100 generates a plurality of facial expression maps based on learning images of a plurality of specific persons, and adjusts the shape, area, or centroid vector position of the emotion region of each of the generated facial expression maps. Therefore, it is generalized (meaning that the position, shape, area, etc. of each emotional region that varies from person to person is shared so that the facial expression features of a specific person do not have an excessive influence on the emotion estimation process). It is possible to estimate the emotion of an unspecified person more accurately.

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-described embodiment, when the emotion of an unspecified person is estimated using a plurality of facial expressions maps of the specified person, the emotion estimation device 100 uses the area classification unit 12 to determine the shape, area, or centroid vector of each emotion area. Is determined in advance, or the shape, area or centroid vector position of each emotional region is adjusted afterwards by the region sharing means 14, but without such determination or adjustment. Emotion estimation by the estimation means 13 may be executed.

  A predetermined number of facial expression maps including neurons with high similarity are extracted in descending order of similarity, and the emotions indicated by the most frequent facial expression map value among those facial expression map values are unspecified. Can be used as learning images of a plurality of specific persons whose facial features are common to unspecified persons without using central neuron fixed clustering, conditional clustering, or area sharing means 14. This is because the emotion of an unspecified person can be estimated using the expression map generated based on the expression map.

  The emotion estimation apparatus 100 may be applied to a face authentication system, a facial neuralgia symptom determination system, or the like, or may be used as a driving support apparatus.

  The driving support apparatus using the emotion estimation apparatus 100 learns a neural network using a driver's facial expression image associated with a predetermined emotion, and estimates the driver's emotion from the driving driver's facial expression image.

  This driving support device performs predetermined driving support based on the estimated emotion. For example, when “anger” is estimated (detected), the accelerator opening is limited to a predetermined value or less.

It is a block diagram which shows the structural example of the emotion estimation apparatus which concerns on this invention. It is a flowchart which shows the flow of a facial expression image process. It is a facial expression image for demonstrating each step of facial expression image processing. It is a flowchart which shows the flow of a facial expression map production | generation process. It is a figure which shows the example of the facial expression map produced | generated by the facial expression map production | generation means. It is a figure for demonstrating conditional clustering by an area | region classification means. It is a schematic diagram which shows the example of the expression map which made arrangement | positioning of each emotion area | region, shape, and size equal. It is a flowchart which shows the flow of an emotion estimation process. It is a figure which shows distribution of the similarity degree between a facial expression image and each neuron on a facial expression map. 3 shows three facial expression maps generated using learning images of three specific persons. It is a flowchart which shows the flow of an emotion estimation process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Control part 2 Image pick-up part 3 Display part 10 Facial expression image processing means 11 Facial expression map production | generation means 12 Area | region classification means 13 Emotion estimation means 14 Area commonality means 100 Emotion estimation apparatus E1-E7 Fixed central neuron CV1-CV9 Central neuron R1-R11 region

Claims (4)

A facial expression map generating means for generating a facial expression map by a neural network by learning a facial expression image of a specific person associated with a predetermined emotion;
Area dividing means for dividing an expression map corresponding to each of the specific persons generated by the expression map generating means into a plurality of areas based on the predetermined emotion;
An emotion estimation means for estimating an emotion of an unspecified person based on a facial expression image of the unspecified person and a plurality of facial expression maps having areas divided by the area classification means;
An emotion estimation apparatus comprising: The region dividing means predetermines at least one of the center position, shape or area of the region to be divided.
The emotion estimation apparatus according to claim 1, wherein: An area sharing means for sharing the corresponding area in each of the plurality of facial expression maps, which is an area divided by the area dividing means;
The emotion estimation means estimates the emotion of the unspecified person based on the expression image of the unspecified person and a plurality of expression maps having a region shared by the area sharing means;
The emotion estimation apparatus according to claim 1, wherein: The region sharing means shares at least one of the shape, center position, or area of the corresponding region,
The emotion estimation apparatus according to claim 3, wherein: