JP2011227710A - Data processor, data processing system, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processor for generating a correct expression map using natural expression images by acquiring the various expression images of a specific person without imposing a mental load on the specific person.SOLUTION: A data processor 3 comprises: an acquisition part 30 for acquiring image data obtained by photographing the expression of a person; a generation part 37 for generating an expression map 40 including data concerning the image of the expressions of a specific person, based on the image data acquired by the acquisition part 30; a determination part 33 for determining whether the image data acquired by the acquisition part 30 is the image data of a new expression of the specific person; and an updating part 38 for updating the expression map 40 by additionally registering data concerning the image of the new expression to the expression map 40 when it is determined that the image data acquired by the acquisition part 30 is the image data of the new expression of the specific person.

Description

  The present invention relates to a data processing device, a data processing system, and a program, and more particularly, to a data processing device, a data processing system, and a program using a self-organizing map by a Kohonen type neural network.

  For example, Patent Document 1 below discloses an emotion estimation apparatus using a self-organizing map using a Kohonen type neural network. The emotion estimation apparatus includes an expression map generation unit, a region segmentation unit, and an emotion estimation unit. The expression map generation means generates an expression map by a neural network by learning an expression image of a specific person associated with a predetermined emotion such as “anger”, “joy”, “surprise”, and the like. The area classifying means classifies the facial expression map generated by the facial expression map generating means into a plurality of areas based on the predetermined emotion. The emotion estimation means estimates the emotion of the unspecified person based on the expression image of the unspecified person and the expression map divided into the plurality of regions.

JP 2008-146318 A

  However, according to the emotion estimation apparatus disclosed in Patent Document 1, when the facial expression map generating means generates the facial expression map, the specific person corresponding to each emotion such as “anger”, “joy”, “surprise”, etc. It is necessary to prepare a facial expression image in advance. Therefore, there is a possibility that the specific person has a great mental burden at the time of photographing a facial expression image, and the facial expression becomes unnatural due to tension at the time of photographing, and an accurate facial expression image may not be acquired.

  The present invention has been made in view of such circumstances, and an accurate facial expression map using a natural facial expression image is obtained by acquiring various facial expression images of a specific person without placing a mental burden on the specific person. An object of the present invention is to obtain a data processing device, a data processing system, and a program that can generate a program.

  According to a first aspect of the present invention, there is provided a data processing apparatus for acquiring image data obtained by photographing a facial expression of a person, and data relating to an image of a specific person's facial expression based on the image data acquired by the acquisition means. A generating means for generating a facial expression map including the image data acquired by the acquiring means is different from data relating to the facial expression image of the specific person already registered in the facial expression map. A determination unit that determines whether the image data is the image data; and the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person. Update means for updating the facial expression map by additionally registering data relating to the facial image in the facial expression map.

  According to the data processing device according to the first aspect, the determination unit is configured to determine whether the image data acquired by the acquisition unit is different from the data related to the facial expression image of the specific person already registered in the facial expression map. It is determined whether or not the image data is a simple expression. When the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person, the update unit additionally registers data related to the new facial expression image in the facial expression map. As a result, the facial expression map is updated. Therefore, the specific person is continuously photographed, and when the specific person has a new facial expression different from the facial expression image already registered in the facial expression map, the new facial expression image is added to the facial expression map. be able to. As a result, since various facial expression images of a specific person can be acquired without applying a mental burden to the specific person, an accurate facial expression map using a natural facial expression image can be generated.

  In the data processing device according to the second aspect of the present invention, in particular, in the data processing device according to the first aspect, the update unit adds the data related to the new facial expression image to the facial expression map. Data relating to an image of a facial expression intermediate between the facial expression of the specific person already registered in the facial expression map and the new facial expression is additionally registered in the facial expression map.

  According to the data processing device of the second aspect, when the data relating to the new facial expression image is additionally registered in the facial expression map, the updating means adds the facial expression of the specific person already registered in the facial expression map and the new facial expression. Data related to images of facial expressions in the middle of facial expressions are additionally registered in the facial expression map. Therefore, since the facial expression images of a specific person registered in the facial expression map are enriched, it is possible to improve the accuracy of authentication when performing personal authentication using the facial expression map.

  In the data processing device according to the third aspect of the present invention, in particular, in the data processing device according to the first or second aspect, the acquisition unit acquires image data based on the facial expression map updated by the update unit. It is further characterized by further comprising a determining means for determining whether or not the selected person matches the specific person.

  According to the data processing apparatus according to the third aspect, the determining means determines whether or not the person whose image data has been acquired by the acquiring means matches the specific person based on the facial expression map updated by the updating means. In the facial expression map updated by the updating means, an intermediate facial expression image between the registered facial expression image and the new facial expression image is registered, and the facial expression image of the specific person is enriched. Therefore, by referring to the updated facial expression map, it is possible to improve the accuracy of determination when the determination unit determines match / mismatch of a person.

  In the data processing device according to the fourth aspect of the present invention, in the data processing device according to any one of the first to third aspects, in particular, the estimation for estimating the emotion of the specific person when the specific person is photographed. And classification means for classifying data relating to images of a plurality of facial expressions of the specific person registered in the facial expression map for each emotion estimated by the estimation means. .

  According to the data processing device of the fourth aspect, the classifying unit classifies the data related to the plurality of facial expression images of the specific person registered in the facial expression map for each emotion estimated by the estimating unit. Therefore, it is possible to specify the emotion of the specific person depending on which emotional area in the facial expression map the facial expression of the specific person in the image data acquired by the acquisition means matches or is most similar.

  A data processing device according to a fifth aspect of the present invention is the data processing device according to the fourth aspect, in particular, the specific person is a video game player, and the estimating means captures the specific person at a time The emotion of the specific person is estimated based on the progress or result of the video game.

  According to the data processing device of the fifth aspect, the estimating means estimates the emotion of the specific person based on the progress or result of the video game at the time when the specific person was photographed. Therefore, the emotion of a specific person can be easily estimated by a simple method of referring to the progress of the video game or the result situation.

  The data processing device according to the sixth aspect of the present invention, in the data processing device according to the fifth aspect, particularly corresponds to the progress or result of the video game at the time of shooting the specific person and the time. Setting means for setting conditions for the progress of the video game based on the emotion of the specific person estimated by the estimation means and the emotion of the specific person at the time specified based on the facial expression map It is characterized by comprising.

  According to the data processing device of the sixth aspect, the setting means includes the progress or result of the video game at the time when the specific person was photographed, and the emotion of the specific person estimated by the estimating means corresponding to the time. Based on the emotion of the specific person at the time specified based on the facial expression map, a condition for progressing the video game is set. Therefore, the scenario development and difficulty level of the video game can be set by considering the player's emotion. As a result, it is possible to obtain a video game in which scenario development and difficulty level can be automatically set according to the skill of the player.

  A data processing system according to a seventh aspect of the present invention includes a photographing device and a data processing device, and the data processing device obtains image data obtained by photographing a facial expression of a person from the photographing device; Generating means for generating an expression map including data relating to an expression image of a specific person based on the image data acquired by the acquiring means; and the image data acquired by the acquiring means is already registered in the expression map. Determining means for determining whether or not the image data of a new facial expression of the specific person is different from data relating to the facial expression image of the specific person, and the image data acquired by the acquisition means is determined by the determining means. When it is determined that it is image data of a new facial expression of the specific person, by additionally registering data related to the image of the new facial expression in the facial expression map, And updating means for updating the serial expression maps, it is characterized in that it has a.

  According to the data processing system of the seventh aspect, the determining unit is configured to determine whether the image data acquired by the acquiring unit is different from the data related to the facial expression image of the specific person already registered in the facial expression map. It is determined whether or not the image data is a simple expression. When the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person, the update unit additionally registers data related to the new facial expression image in the facial expression map. As a result, the facial expression map is updated. Therefore, the specific person is continuously photographed, and when the specific person has a new facial expression different from the facial expression image already registered in the facial expression map, the new facial expression image is added to the facial expression map. be able to. As a result, since various facial expression images of a specific person can be acquired without applying a mental burden to the specific person, an accurate facial expression map using a natural facial expression image can be generated.

  A program according to an eighth aspect of the present invention specifies a computer mounted on a data processing device based on acquisition means for acquiring image data obtained by photographing a facial expression of a person, and image data acquired by the acquisition means. Generating means for generating a facial expression map including data relating to a facial expression image of a person, and image data acquired by the acquiring means is different from data relating to the facial expression image of the specific person already registered in the facial expression map; A determination unit that determines whether or not the image data is a new facial expression of the specific person, and the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person And updating means for updating the facial expression map by additionally registering data related to the new facial expression image in the facial expression map. It is characterized in that to ability.

  According to the program according to the eighth aspect, the determining means has the new facial expression of the specific person whose image data acquired by the acquiring means is different from the data related to the facial expression image of the specific person already registered in the facial expression map. It is determined whether or not the image data. When the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person, the update unit additionally registers data related to the new facial expression image in the facial expression map. As a result, the facial expression map is updated. Therefore, the specific person is continuously photographed, and when the specific person has a new facial expression different from the facial expression image already registered in the facial expression map, the new facial expression image is added to the facial expression map. be able to. As a result, since various facial expression images of a specific person can be acquired without applying a mental burden to the specific person, an accurate facial expression map using a natural facial expression image can be generated.

  According to the present invention, it is possible to generate an accurate facial expression map using a natural facial expression image by acquiring various facial expression images of the specific person without placing a mental burden on the specific person. A processing device, a data processing system, and a program can be obtained.

It is a figure showing typically the whole data processing system composition concerning an embodiment of the invention. It is a block diagram which shows the structure of a data processor. It is a block diagram which shows the function structure of an authentication process part. It is a figure which shows the outline | summary of the process which an authentication process part performs. It is a figure which shows the structure of a Kohonen type neural network. It is a figure which shows the production | generation process of the facial expression map by the production | generation part shown in FIG. It is a figure which shows the determination process by the determination part shown in FIG. It is a figure which shows the update process of the facial expression map by the update part shown in FIG. It is a figure which shows the facial expression map classified according to emotion.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, the element which attached | subjected the same code | symbol in different drawing shall show the same or corresponding element.

  FIG. 1 is a diagram schematically showing an overall configuration of a data processing system 1 according to an embodiment of the present invention. The data processing system 1 includes a photographing device 2 such as a CCD camera and a data processing device 3 such as a main body of a video game machine. The photographing device 2 is attached on a monitor 4 such as a liquid crystal television, and continuously shoots a video game player 100. The data processing device 3 can perform data communication with the imaging device 2, the monitor 4, and the controller 5 of the video game machine by wire or wireless.

  FIG. 2 is a block diagram showing the configuration of the data processing device 3. The data processing device 3 includes a communication unit 10, a processing unit 11 such as a CPU, and a storage unit 12 having an arbitrary storage medium such as a semiconductor memory or a hard disk. The communication unit 10 performs data communication with the imaging device 2, the monitor 4, and the controller 5. The processing unit 11 includes a game processing unit 20 that performs processing related to the progress of a video game and an authentication processing unit 21 that performs processing related to person authentication. The storage unit 12 stores a game program 22 and an authentication program 23. When the processing unit 11 reads out and executes the game program 22 from the storage unit 12, the processing unit 11 functions as the game processing unit 20. In addition, the processing unit 11 functions as the authentication processing unit 21 by the processing unit 11 reading the authentication program 23 from the storage unit 12 and executing it.

  FIG. 3 is a block diagram illustrating a functional configuration of the authentication processing unit 21. As shown in the connection relationship of FIG. 3, the authentication processing unit 21 includes an acquisition unit 30, a processing unit 31, a facial expression map processing unit 32, a determination unit 33, a determination unit 34, an estimation unit 35, and a setting unit 36. Has been. The facial expression map processing unit 32 includes a generation unit 37, an update unit 38, and a classification unit 39. The storage unit 12 stores a facial expression map 40.

  FIG. 4 is a diagram showing an outline of the processing performed by the authentication processing unit 21. With reference to FIGS. 3 and 4, the acquisition unit 30 acquires image data 50 including the player 100 captured by the imaging device 2 from the communication unit 10. Since the imaging device 2 continuously captures the player 100, the acquisition unit 30 continuously acquires the image data 50 from the communication unit 10. The image data 50 is input from the acquisition unit 30 to the processing unit 31 as a signal S1.

  The processing unit 31 extracts a human face portion from the image data 50 by using a known method such as binarization, edge extraction, and affine transformation. Further, the processing unit 31 generates image data 51 relating to a facial expression image of a person by compressing the face part image to a predetermined size so that the extracted face part image can be used as an input image to the neural network. . The image data 51 is input from the processing unit 31 to the facial expression map processing unit 32, the determination unit 33, and the determination unit 34 as a signal S2.

  The expression map processing unit 32 is configured using a Kohonen type neural network, and generates, updates, and updates the expression map 40 that is a self-organizing map based on the image data 51 input to the input unit of the neural network. And processing including classification. Details of the processing contents executed by the facial expression map processing unit 32 will be described later. The expression map 40 is configured as a set of a plurality of output units 62 of the neural network, and one expression image of the player 100 appears in each output unit 62. In FIG. 4, the expression of the player 100 is shown in a unified manner for simplification of the drawing, but actually, the expression map 40 includes image data 52 corresponding to various expression images of the player 100. It becomes.

FIG. 5 is a diagram illustrating a configuration of a Kohonen type neural network. The Kohonen type neural network includes an input layer including a plurality of input units 60 and an output layer including a plurality of output units 62. In FIG. 5, the horizontal and vertical coordinates of the input unit 60 are represented by “k” and “l”, respectively, and the horizontal and vertical coordinates of the output unit 62 are represented by “i” and “j”, respectively. Yes. In the Kohonen type neural network, first, a small random value is given as the weighting value W _ijkl between each input unit and each output unit. When the input image x _kl is input, each pixel value (for example, luminance value) of the image data 51 is input to the corresponding input unit 60. Based on the following formula (1), the similarity u _ij with the input image is obtained for all output units, and the output unit with the highest similarity u _ij is set as the maximum similarity unit. Then, the pixel values of the plurality of output units included in the predetermined learning radius d _s centered on the maximum similarity unit are approximated to the input image x _kl based on the following formula (2), and The weight value W _ijkl is modified. Here, the following formula (3) is used for updating the weighting value W _ijkl , and the following formula (4) is used for updating the learning coefficient α (t). Here, d _i and d _j are distances in the i direction and j direction from the maximum similarity unit to the weight to be updated (d _s > d _i , d _s > d _j ), respectively, and T is the total number of learning times , T is the current number of learning times, and X is an arbitrary variable.

  FIG. 6 is a diagram showing a generation process of the facial expression map 40 by the generation unit 37 shown in FIG. In an initial stage where the player 100 is not registered, each node (that is, each output unit 62) of the facial expression map 40 is initialized using a random number. When the registration of the player 100 is instructed, the image data 51 generated based on the image data 50 photographed by the photographing device 2 is input from the processing unit 31 to the generation unit 37. The image data 51 includes an arbitrary expression image (for example, an expressionless image) of the player 100. The generation unit 37 registers the same image data 52A as the input image data 51 in a predetermined node of the expression map 40. At this time, the same image data 52A is registered in the surrounding nodes according to a preset learning radius. In the example shown in FIG. 6, the image data 52 </ b> A is registered in a node included in the area A <b> 1 of the expression map 40. In other words, the weight value W of the relevant part of the neural network is set so that the same image data 52A as the image data 51 appears in the nodes included in the region A1.

  While the player 100 continues the video game, the photographing device 2 continuously photographs the player 100. Therefore, the processing unit 31 continuously outputs image data 51 including various facial expression images of the player 100, and these image data 51 are input from the processing unit 31 to the determination unit 33.

  FIG. 7 is a diagram illustrating a determination process performed by the determination unit 33 illustrated in FIG. The determination unit 33 receives the image data 51 from the processing unit 31 as the signal S2 and the facial expression map 40 from the storage unit 12 as the signal S4. The determination unit 33 specifies image data 52M having the highest similarity to the image data 51 input from the processing unit 31 among the image data 52 already registered in the facial expression map 40. Then, when the similarity N between the image data 51 and the image data 52M is equal to or greater than the threshold Th1, the determination unit 33 regards the image data 51 and the image data 52M as substantially the same, and determines the expression map. 40 is not updated (details will be described later). Further, when the similarity N is equal to or greater than the threshold Th2 and less than the threshold Th1, the determination unit 33 regards that the image data 51 includes a new facial expression image of the player 100, and updates the facial expression map 40. An update command signal S5 is input to the update unit 38. Further, when the similarity N is less than the threshold Th2, the determination unit 33 regards that the image data 51 includes a facial expression image of a person different from the player 100, and does not update the facial expression map 40. .

  FIG. 8 is a diagram showing an update process of the facial expression map 40 by the update unit 38 shown in FIG. When the update command signal S5 is input from the determination unit 33, the update unit 38 additionally registers the same image data 52B as the image data 51 input from the processing unit 31 in the facial expression map 40. At that time, in the same manner as described above, the same image data 52B is registered in the surrounding nodes in accordance with a preset learning radius. In the example illustrated in FIG. 8, the image data 52 </ b> B is registered in a node included in the area A <b> 2 of the expression map 40. In the overlapping area where the area A1 and the area A2 overlap, image data in which the image data 52A and the image data 52B are fused (mixed) with each other is registered. For example, when the expression of the player 100 in the image data 52A is no expression and the expression of the player 100 in the image data 52B is a smile, the image data of the expression between the expressionless expression and the smile is the overlap area. Registered in

  Each time image data 51 including a new facial expression image of the player 100 is input, the determination unit 33 inputs an update command signal S5 to the update unit 38, and the update unit 38 includes the new facial expression image data, and Image data of an intermediate facial expression between a new facial expression and a registered facial expression is additionally registered in the facial expression map 40. Thus, whenever the player 100 shows a new facial expression while the video game continues, the image data 52 including various facial expression images of the player 100 is automatically accumulated in the facial expression map 40, and the facial expression map 40 is enriched. .

  When the video game is started after the next time, the player 100 is authenticated using the latest facial expression map 40 stored at that time stored in the storage unit 12. With reference to FIG. 3, the image data 50 of the player 100 imaged by the imaging device 2 at the time of activation is input from the acquisition unit 30 to the processing unit 31 as a signal S1. Image data 51 generated based on the image data 50 is input from the processing unit 31 to the determination unit 34 as the signal S2. Further, the expression map 40 is input from the storage unit 12 to the determination unit 34 as the signal S4. The determination unit 34 specifies image data 52M having the highest similarity to the image data 51 input from the processing unit 31 among the image data 52 of various facial expression images registered in the facial expression map 40. When the similarity between the image data 51 and the image data 52M is equal to or greater than a predetermined threshold, the determining unit 34 determines that the person photographed by the photographing device 2 is the player 100 registered in the facial expression map 40. Considering the same person, the authentication result is “success”. In this case, various parameter values such as a score history and a progress situation regarding the progress of the video game are automatically set to the setting values of the player 100 stored in the storage unit 12. On the other hand, when the similarity between the image data 51 and the image data 52M is less than the threshold, the determination unit 34 determines that the person photographed by the photographing device 2 is the player 100 registered in the facial expression map 40. Assuming that the person is another person, the authentication result is “failure”.

  Referring to FIG. 3, a signal S <b> 6 indicating the progress or result of the video game is input to the estimation unit 35 from the game processing unit 20 illustrated in FIG. 2. The estimation unit 35 estimates the emotion of the player 100 according to the progress of the video game or the situation of the result based on the signal S6. For example, when the high score is updated or when the opponent is won in the competitive game, it is estimated that the emotion of the player 100 is “joy”. In addition, when an effect that surprises the player in a horror game or the like is performed, it is estimated that the emotion of the player 100 is “surprise”. The estimation unit 35 inputs a signal S7 indicating the estimated emotion of the player 100 to the facial expression map processing unit 32.

  The classification unit 39 is included in the image data 52 based on the emotion estimation result of the player 100 at the moment when the player 100 is photographed when the generation unit 37 or the update unit 38 registers the image data 52 in the facial expression map 40. The facial expression image of the player 100 is associated with the emotion of the player 100. Thereby, the plurality of image data 52 in the facial expression map 40 is classified for each emotion of the player 100. FIG. 9 is a diagram showing facial expression maps 40 classified for each emotion. For example, image data of a plurality of facial expression images associated with the emotion of “joy” is registered in the area B1, and image data of a plurality of facial expression images associated with the emotion of “sadness” is registered in the area B2. Has been. Similarly, for the areas B3 to B6, the image data of the facial expression image of the player 100 is classified for each emotion of the player 100 and registered in each area.

  Referring to FIG. 3, a signal S6 indicating the progress or result of the video game is input from the game processing unit 20 to the setting unit 36 and a signal S7 indicating the estimation result of the emotion of the player 100 is input to the estimation unit 35. Further, a signal S8 indicating the emotion of the player 100 specified based on the facial expression map 40 shown in FIG. Here, the facial expression map processing unit 32 specifies image data 52M having the highest similarity to the image data 51 input from the processing unit 31 among the image data 52 of various facial expression images registered in the facial expression map 40. Then, the emotion of the player 100 is specified depending on which region B1 to B6 the image data 52M belongs to. For example, when the image data 52M belongs to the area B1, the emotion of the player 100 is specified as “joy”.

  The setting unit 36 variably sets the conditions for progressing the video game based on these input signals S6 to S8. For example, in a situation where the player 100 is defeated in a battle-type game and the emotion of the player 100 specified at that time is extreme “sadness”, the computer's strength level is set to the level of the computer in the next battle. Set lower than this time. In addition, if the emotion of the identified player 100 is not “surprise” in spite of performing an effect that surprises the player in a horror game or the like, the player will be surprised at the scenario of the next scene that the player proceeds to. Set the scenario to be more radical than the current one. This can be realized by preparing a plurality of scenarios for the same scene and selecting one scenario from the plurality of scenarios by the setting unit 36.

  In the above description, the content of the present invention has been described by taking a video game for displaying a game video on the monitor 4 as an example. However, the present invention is not limited to this example. The present invention can also be applied to a case where a game video is displayed on the screen of a mobile terminal, as in the case of a video game.

  In the above description, the example in which image data is registered in the facial expression map 40 has been described. However, in order to reduce the amount of stored data, instead of the image data, the distance between the eyes and the positional relationship between the eyes and the nose Data obtained by digitizing the characteristics of a person's face may be registered in the facial expression map 40. Alternatively, in order to improve the accuracy of authentication, an expression map using image data and an expression map using digitized data may be used in combination.

  According to the data processing device 3 according to the present embodiment, the determination unit 33 determines whether the image data acquired by the acquisition unit 30 is different from the facial expression image of the player 100 already registered in the facial expression map 40. It is determined whether or not it is a new facial expression image. Then, when the determination unit 33 determines that the image data acquired by the acquisition unit 30 is a new facial expression image of the player 100, the update unit 38 additionally registers the new facial expression image in the facial expression map 40. Thus, the facial expression map 40 is updated. Therefore, the player 100 is continuously photographed, and when the player 100 makes a new facial expression different from the facial expression image already registered in the facial expression map 40, the new facial expression image is added to the facial expression map 40. can do. As a result, since various facial expression images of the player 100 can be acquired without placing a mental burden on the player 100, it is possible to generate an accurate facial expression map 40 using a natural facial expression image.

  In addition, according to the data processing device 3 according to the present embodiment, when the update unit 38 additionally registers a new facial expression image in the facial expression map 40, the update unit 38 and the facial expression of the player 100 already registered in the facial expression map 40 The facial expression image intermediate to the new facial expression is additionally registered in the facial expression map 40. Therefore, since the facial expression images of the player 100 registered in the facial expression map 40 are enriched, it is possible to improve the accuracy of authentication when authenticating a person using the facial expression map 40.

  Further, according to the data processing device 3 according to the present embodiment, the determination unit 34 determines that the person whose image data is acquired by the acquisition unit 30 matches the player 100 based on the facial expression map 40 updated by the update unit 38. Determine whether or not. In the facial expression map 40 updated by the updating unit 38, an intermediate facial expression image between the registered facial expression image and the new facial expression image is registered, and the facial expression image of the player 100 is enriched. Therefore, by referring to the updated facial expression map 40, it is possible to improve the accuracy of determination when the determination unit 34 determines match / mismatch of a person.

  Further, according to the data processing device 3 according to the present embodiment, the classification unit 39 classifies the plurality of facial expression images of the player 100 registered in the facial expression map 40 for each emotion estimated by the estimation unit 35. Therefore, it is possible to specify the emotion of the player 100 according to which emotional region in the facial expression map 40 the facial expression of the player 100 in the image data acquired by the acquisition unit 30 matches or is most similar to. .

  Further, according to the data processing device 3 according to the present embodiment, the estimation unit 35 estimates the emotion of the player 100 based on the progress of the video game or the situation of the result at the time when the player 100 is photographed. Therefore, it is possible to easily estimate the emotion of the player 100 by a simple method of referring to the progress or result of the video game.

  Further, according to the data processing device 3 according to the present embodiment, the setting unit 36 estimates the progress or result of the video game at the time when the player 100 is photographed and the estimation unit 35 corresponding to the time. Based on the emotion of the player 100 and the emotion of the player 100 at the time point specified by the facial expression map processing unit 32 based on the facial expression map 40, conditions for progressing the video game are set. Therefore, by considering the emotion of the player 100, it is possible to set the subsequent scenario development and difficulty level of the video game. As a result, it is possible to obtain a video game in which scenario development and difficulty level can be automatically set according to the skill of the player.

DESCRIPTION OF SYMBOLS 1 Data processing system 2 Imaging device 3 Data processing device 11 Processing part 12 Storage part 20 Game processing part 21 Authentication processing part 22 Game program 23 Authentication program 30 Acquisition part 31 Processing part 32 Expression map processing part 33 Determination part 34 Determination part 35 Estimation Unit 36 setting unit 37 generation unit 38 update unit 39 classification unit 40 facial expression map 51, 52 image data

Claims (8)

An acquisition means for acquiring image data obtained by photographing a facial expression of a person;
Generating means for generating a facial expression map including data relating to the facial expression image of the specific person based on the image data acquired by the acquiring means;
It is determined whether the image data acquired by the acquisition unit is image data of a new facial expression of the specific person that is different from data related to the facial expression image of the specific person already registered in the facial expression map. A determination means;
When the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person, by additionally registering data related to the new facial expression image in the facial expression map Updating means for updating the facial expression map;
A data processing apparatus.   When the data relating to the new facial expression image is additionally registered in the facial expression map, the updating unit is configured to display an intermediate facial expression between the facial expression of the specific person already registered in the facial expression map and the new facial expression. The data processing apparatus according to claim 1, wherein data related to an image is additionally registered in the facial expression map. The determination unit according to claim 1, further comprising: a determination unit that determines whether or not a person whose image data has been acquired by the acquisition unit matches the specific person based on the facial expression map updated by the update unit. Data processing device. Estimating means for estimating an emotion of the specific person when the specific person is photographed;
Classification means for classifying data relating to images of a plurality of facial expressions of the specific person registered in the facial expression map for each emotion estimated by the estimation means;
The data processing device according to claim 1, further comprising: The specific person is a video game player,
The data processing apparatus according to claim 4, wherein the estimation unit estimates an emotion of the specific person based on a progress or result of the video game at the time when the specific person is photographed. The progress or result of the video game at the time of shooting the specific person, the emotion of the specific person estimated by the estimating means corresponding to the time, and the time specified based on the facial expression map The data processing apparatus according to claim 5, further comprising setting means for setting a condition for progress of the video game based on the emotion of the specific person. A photographing device;
A data processing device;
With
The data processing device includes:
Obtaining means for obtaining image data obtained by photographing a facial expression of a person from the photographing device;
Generating means for generating a facial expression map including data relating to the facial expression image of the specific person based on the image data acquired by the acquiring means;
It is determined whether the image data acquired by the acquisition unit is image data of a new facial expression of the specific person that is different from data related to the facial expression image of the specific person already registered in the facial expression map. A determination means;
When the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person, by additionally registering data related to the new facial expression image in the facial expression map Updating means for updating the facial expression map;
A data processing system. A computer installed in a data processing device
An acquisition means for acquiring image data obtained by photographing a facial expression of a person;
Generating means for generating a facial expression map including data relating to the facial expression image of the specific person based on the image data acquired by the acquiring means;
It is determined whether the image data acquired by the acquisition unit is image data of a new facial expression of the specific person that is different from data related to the facial expression image of the specific person already registered in the facial expression map. A determination means;
When the determination unit determines that the image data acquired by the acquisition unit is image data of a new facial expression of the specific person, by additionally registering data related to the new facial expression image in the facial expression map Updating means for updating the facial expression map;
A program that functions as

Images