JP5294315B2 - Dialogue activation robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interaction activating system and an interaction activating robot, which can activate direct communication between people much more than before. <P>SOLUTION: In a game activation system 15, conditions of attendant A, B and C are always observed based on a total activation degree. When the total activation degree is a predetermined threshold or less, it is presumed that there is possibility that the attendant does not participate in a difficult reading game. By spontaneously outputting a content of speech optimal for the site from a loudspeaker 24, direct interaction of a chairman and the attendants A, B and C is triggered. Thus, the direct communication of the chairman MC and the attendants A, B and C is activated much more than before. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

The present invention relates to a dialogue activated robot, and is preferably applied to the obfuscated game shed kana kanji performed by the example presenter and participants.

  2. Description of the Related Art In recent years, day services (home care facilities) that provide care for daily life such as health check, bathing, meals, rehabilitation, etc. are known in facilities where elderly people go to. Such facilities are one of the few places of communication for the elderly, and active exchange at the facilities is very important for improving the quality of life (QOL) of the elderly.

  Conventionally, various robots have been developed for elderly care. For example, a robot that registers tens of thousands of conversations and can enjoy one-on-one conversations (for example, see Non-Patent Document 1), and content that activates the brain for elderly people are also provided. Yes. Users using such robots have reported that the presence of robots has changed the relationship between families, and the air that tends to be heavy at the end of life has been relieved.

  Also, as such a robot, a seal-type robot for healing purposes is also considered. This seal-type robot increases communication between patients and nurses in addition to psychological effects such as relaxing users or increasing motivation for communication, and physiological effects such as improving vital signs. It has been reported that a social effect such as making it appear has been seen (for example, see Non-Patent Document 2).

And these robots are excellent in terms of indirectly improving communication between humans and humans through the interaction between humans and robots, taking advantage of the presence and event characteristics of robots. Has been successful.
Seita Ibuki, Kenji Kimura, Natsuyoshi Takeda: Life support system for the elderly using communication robots, Journal of the Japan Society of Mechanical Engineers, Vol.108, No.1038, pp.392-395 (2005) Kazuyoshi Wada, Takanori Shibata, Kazuo Tanie: Robot therapy in a health care facility for the elderly-Evaluation of effects in the first year of experiments, The Society of Instrument and Control Engineers, Proceedings of the Society of Instrument and Control Engineers, Vol.42, No.4, pp.386- 392 (2006)

  By the way, in the day service mentioned above, various ideas are made about communication opportunities between elderly people and communication opportunities between elderly people and caregivers, and many such communication opportunities are provided. A lot of time is spent all day in the recreation of games, gymnastics, handicrafts, crafts, music, etc. In fact, we conducted continuous field surveys on day services, and found that in such care settings, communication itself was the goal, and active communication was also a prevention of dementia and depression. However, active communication is not necessarily performed just by providing such a place, and the above-mentioned various robots improve indirect communication between people. There is a problem that it is difficult to activate direct communication with people.

  The present invention has been made in consideration of the above points, and an object thereof is to propose a dialogue activation system and a dialogue activation robot that can further activate direct communication between people than ever before. .

In order to solve such a problem, claim 1 of the present invention is based on an image capturing unit that acquires images of a plurality of participants who interact with a moderator on a predetermined subject, and an image acquired by the image capturing unit. and activity estimation means for estimating the activity of indicating the degree of participation in the dialogue on the subject of the participants, and storing means for storing a plurality of progress keyword in advance, speech detection for detecting the participant who is speaking A voice recognition unit that recognizes words included in the voice signal by performing predetermined voice recognition processing on the voice signal obtained from the microphone, and sends the recognized word as character string data; , Receiving the character string data obtained by the voice recognition unit, reading the progress keyword stored in advance in the storage means, Determining whether the said character string data Zureka match, the character string data matches any progress keyword, according to the estimated activity by the activity estimating means, to dialogue on the subject A selection unit that selects speech information that encourages participation, a speaker that utters speech content of the speech information selected by the selection unit, and a head resembling an animal shape, and serves as an eye of the head The imaging means is installed at a position, and the utterance detecting means is a lip motion recognition process for recognizing lip movement recognized from a video obtained by the imaging means, or a microphone volume for directing each speaker. The participant who is speaking is identified using either the speaker recognition process based on the sound source or the sound source localization process based on the sound collected by the microphone And a control means for placing the participant detected by the utterance detection means at the center of the angle of view of the imaging means, the selection means not recognizing the progress status keyword for a certain period of time, and When the sum total activity is less than a predetermined threshold value, the predetermined utterance content is read from the storage means and the utterance content is uttered from the speaker as a voice, and the participant's voice break is Recognizing and performing at least one of a nodding operation for moving the head in accordance with the cut or a colloquial speech operation for producing a companion voice in accordance with the cut .

Further, a second aspect of the present invention, when emitting the voice utterance of the speech information from the speaker, the participants or Equipment of interest to emit emitting story contents, arranged in the center of the angle of view of the imaging means It is characterized by providing the control means to make.

According to a third aspect of the present invention, there is provided recognition means for recognizing the participant's face direction or line-of-sight direction based on an image obtained by the imaging means, and the control means is configured such that the participant speaks. Then, the angle of view center of the imaging unit is arranged in the face direction or line-of-sight direction of the participant recognized by the recognition unit.

Further, Claim 4 of the present invention, collected and moderator microphone that collects sound moderator to interact for a given subject with the participants, the voice of the moderator by the moderator microphone The progress status recognition means for recognizing the progress status of the dialogue with the participant on the subject based on the audio signal obtained in the above manner, and the understanding status regarding the progress status of the dialog in the progress status recognition means to the moderator And display means for presenting.

According to the present invention, in accordance with the situation of dialogue between a plurality of participants and the presenter, the speaker can spontaneously utter the optimum utterance content for the occasion, thereby allowing direct dialogue between the presenter and the participant. Thus, direct communication between the presenter and a plurality of participants can be more activated than before.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(1) Outline of the Present Invention In FIG. 1, reference numeral 1 denotes a game activation robot 1 as a dialog activation robot according to the present invention. This game activation robot 1 is, for example, a kana reading kana presented on a whiteboard WB. Can be used when playing an obfuscated game. And this game activation robot 1 selects the kanji which becomes a problem, the MC MC who advances an obfuscated game, and a plurality of participants A who answer the kana reading kana presented by participating in the obfuscated game. , B and C can be activated in a direct dialogue. Incidentally, in the case of this embodiment, the participants A, B, and C play the obfuscated game, but the present invention is not limited to this, and various other numbers such as two or four people. Participants may play an obfuscated game.

  In practice, the game activation robot 1 is wirelessly connected to the monitoring server 2 so that various data can be transmitted to and received from the monitoring server 2. The monitoring server 2 is wirelessly connected to the portable terminal 3 operated by the MC MC. When a problem is arbitrarily selected by the MC MC from among kanji classified by genre and level displayed on the display unit 4, the portable terminal 3 transmits this to the monitoring server 2 as a problem selection command. Has been made to get. When the monitoring server 2 receives the problem selection command from the mobile terminal 3, the monitoring server 2 selects individual behavior pattern information corresponding to the problem selection command from a plurality of pieces of pre-stored individual behavior pattern information (described later). It can be transmitted to the game activation robot 1. As a result, the game activation robot 1 executes various kinds of optimum behavior patterns according to the problem in accordance with the individual behavior pattern information, so that the direct dialogue between the moderator MC and the participants A, B, and C is performed. Can be activated.

  In addition, the game activation robot 1 is connected to a headset microphone 5 attached to the head of the MC MC, and the voice of the MC MC collected by the headset microphone 5 can be transmitted. It is made like that. As a result, the game activation robot 1 uses the voice of the MC MC acquired from the headset microphone 5 to indicate a keyword indicating the progress status in the utterance content of the MC MC (hereinafter referred to as a progress status keyword). A speaker (not shown in FIG. 1) that recognizes whether or not it is included, grasps the progress of the obfuscated game on the basis of the recognized progress status keyword, and makes the utterance content best suited to the situation according to the progress ), The direct dialogue between the MC MC and the participants A, B, and C can be activated based on the content of the utterance.

  Here, the game activation robot 1 displays the progress of the obfuscated game before the problem of the obfuscated game is presented based on the progress keyword obtained based on the contents of the utterance of the MC MC (hereinafter referred to as questions). And before the correct answer is answered after the problem of the obfuscated game is presented (hereinafter called the game in progress), and after the answer of the obfuscated game is answered It can be divided into three situations, until now (hereinafter referred to as “under explanation”), and the situation (stage) of the obfuscated game at the present time can be stored in a database (described later) It is made like that. In practice, for example, when the game activation robot 1 recognizes the progress status keyword (“starts an obfuscated game”) based on the utterance content of the MC, for example, the game activation robot 1 grasps that it is before the application, Recognizing that the progress keyword ("Is a problem") based on the utterance content, the game is in progress, and explaining the progress keyword ("correct answer") based on the utterance content of the MC MC It is made to be able to grasp that it is. In addition, the game activation robot 1 indicates how close the answer of the participants A, B, and C is to the correct answer as the progress of the game in progress, and the utterance contents of the MC MC Ascertained by the progress status keyword ("Oishi") obtained by the above, a predetermined utterance content corresponding to the proximity to the correct answer is uttered as speech.

  Actually, the game activation robot 1 is a humanoid robot that is made to resemble a human, and a head 8 and an arm 9 that resemble a human head have a torso via a drive unit (not shown). The unit 10 is provided so as to be drivable, and a speaker for uttering the utterance content as a voice is provided at a predetermined position. In addition, the game activation robot 1 performs a nodding operation for turning the head 8 up and down around the neck, a negative operation for turning the head 8 left and right around the neck, and the tip of the arm 9 with a predetermined tip. Each operation such as a pointing operation directed in the direction can be executed at a predetermined timing during the game or during the explanation.

  A camera 11 is installed on the head 8 of the game activation robot 1 at a position corresponding to a human eye. The game activation robot 1 is configured to take in a video of a participant imaged by a camera 11 as an imaging means as a video signal. As a result, the game activation robot 1 analyzes the obtained video to thereby determine the angle at which the face is directed from the contours of the faces of the participants A, B, and C (hereinafter referred to as the face direction). ) And the facial expression, and based on these, it is possible to estimate the degree of activity (described later) indicating whether each participant A, B and C is actively participating in the obfuscated game. Has been made. In addition, the game activation robot 1 is provided with a microphone (not shown in FIG. 1) at a predetermined position of the body unit 10, for example, and the voices of the participants A, B, and C collected by the microphone are received. Can be incorporated. As a result, the game activation robot 1 analyzes the obtained voice to grasp the presence or absence of the voice of each participant A, B, and C, and based on this, the activity of each participant A, B, and C is determined. The degree can be estimated.

  And the game activation robot 1 will estimate the activity of each participant A, B, and C, will calculate the total activity by adding the activity of each participant A, B, and C, and this total activity It is possible to constantly monitor whether or not the current value is equal to or less than a predetermined threshold value set in advance. When the game activation robot 1 determines that the total activity is equal to or less than a predetermined threshold, the game activation robot 1 predicts an action pattern that will maximize the total activity and can execute an action pattern that is a prediction result thereof. Has been made. Thus, the game activation robot 1 increases the activity of each participant A, B, and C by executing an action pattern such as uttering the optimal utterance content according to the prediction result while the game is in progress or during explanation, A direct dialogue performed between the MC MC and the participants A, B, and C can be activated by using the utterance content.

(2) Configuration of Game Activation System In practice, the game activation robot 1 incorporates a game activation system 15 as shown in FIG. The game activation system 15 includes an utterance situation understanding unit 16, and the utterance situation understanding unit 16 can constantly monitor the video signals of the participants A, B, and C given from the camera 11. . In practice, the utterance state understanding unit 16 performs a predetermined lip motion recognition process on the video signal, identifies the mouth movements of the participants A, B, and C based on the video signal, and speaks. Participants A, B, or C (hereinafter referred to as “speakers”) can be detected and sent to the situation management unit 17 as a speaker detection result. The situation management unit 17 sends the speaker detection result to the behavior playback unit 19 via the behavior selection unit 18, and the speaker plays the center of the angle of view of the camera 11 based on the speaker detection result in the behavior playback unit 19. A control command for driving the head 8 to be arranged at the position is generated. Thus, the head 8 is driven based on the control command, so that the speaker can be placed at the center position of the angle of view of the camera 11. Thus, the game activation robot 1 can give the participants A, B, and C the impression as if they are facing the speaker.

  Further, the utterance situation understanding unit 16 can detect the face direction of the utterer based on the video signal given from the camera 11, and the MC or the MC is placed ahead of the face direction of the utterer. An utterance target such as another participant A, B, or C or a whiteboard WB staring while speaking is specified from the video obtained from the video signal, and this is detected as the utterance target detection result in the situation management unit 17. Send it out. Further, the utterance state understanding unit 16 constantly monitors the audio signal given from the microphone 20, and determines whether or not the speaker has finished speaking based on whether or not the audio signal was obtained for a certain period of time. Has been made to get. When the utterance situation understanding unit 16 determines that the utterer has finished speaking based on the presence or absence of a voice signal, the utterance situation understanding unit 16 can send this to the situation management unit 17 as a speech end detection result.

  The situation management unit 17 sends the utterance target detection result and the utterance end detection result to the behavior reproduction unit 19 via the behavior selection unit 18. As a result, the behavior reproducing unit 19 is configured so that the utterance target ahead of the speaker's face direction is arranged at the center position of the angle of view of the camera 11 based on the utterance target detection result and the utterance end detection result. A control command for driving 8 is generated. Thereby, the head 8 is driven based on the control command after the utterance of the speaker is completed, so that the utterance target can be arranged at the center position of the angle of view of the camera 11. Thus, the game activation robot 1 gives the participants A, B, and C the impression that they are facing the direction of the utterance after the utterance of the utterer ends and actively participating in the obfuscated game. be able to.

  Further, when the behavior reproduction unit 19 receives a speech break detection result indicating that an audio signal has not been obtained for a predetermined time from the situation management unit 17, the behavior reproduction unit 19 performs a nodding operation to rotate the head 8 up and down. Control instruction to be generated. As a result, the game activation robot 1 makes an impression as if it was nodded by listening to the utterer's utterance by the head's nodding action based on the control command at the break of the speaker's utterance. And C. At the same time, the behavior reproducing unit 19 can execute a companion speech operation that emits a companion voice such as “Yes” via the speaker 24. Incidentally, in this case, the behavior reproducing unit 19 may execute only one of the nodding operation and the competing speech operation.

  In addition to this configuration, the game activation system 15 includes a database 21 and receives from the monitoring server 2 individual action pattern information (described later) in which utterance contents and actions are set for each question to be asked. Then, the individual behavior pattern information can be stored in the database 21. Further, in this database 21, common action pattern information (described later) in which utterance contents and actions common to all problems are set, and each participant A, B predicted by the action pattern of the game activation robot 1 And an expected effect table (to be described later) in which actions of C and C are defined, and a progress status keyword (to be described later) are stored in advance.

(2-1) Individual Action Pattern Information As shown in FIG. 3, the individual action pattern information 30 includes a problem table 31, a genre table 32, an information table 33, a script table 34, and an action table 35. The ID of the problem table 31 (“ID” column in the figure), the genre ID of the genre table 32 (“Genre ID” column in the figure), and the genre ID of the information table 33 (“Genre” in the figure) ID ”field), the Kanji ID in the script table 34 (“ Kanji ID ”field in the figure), and the script ID in the operation table 35 (“ Script ID ”field in the figure) are assigned the same ID number. Are associated with each other. For example, ID number “1” includes ID “1” of problem table 31, genre ID “1” of genre table 32, genre ID “1” of information table 33, and kanji ID “1” of script table 34. And the script ID “1” in the operation table 35 are associated with each other.

  Incidentally, in the case of this embodiment, the ID “1” of the problem table 31 includes a kanji “kai pork” in the “Kanji” column and a reading kana as an answer to this problem “kai pork” in the “Yomi” column. A genre ID “1” indicating the genre of the problem in the “Genre ID” field and a level “3 (normal)” indicating the difficulty level of the problem in the “Level” field are preset. . The level can be set from 1 to 5, for example, where 1 is “easy”, 2 is “slightly simple”, 3 is “normal”, 4 is “somewhat difficult”, and 5 is “difficult”.

  In addition, in the genre ID “1” of the genre table 32, “animal” indicating the genre of the ID “1” of the problem table 31 is set in advance in the “Genre” column. In the genre ID “1” of the information table 33, information related to the ID “1” of the problem table is preset in the “information” column. In the script table 34, the Chinese character ID “1” associated with the ID “1” in the question table 31 indicates the degree of accuracy “10” indicating whether the answers of the participants A, B, and C are close to the correct answer. (Details will be described later) is set in the “Correctness” field, and the script type “answer” (details will be described later) indicating the script type is set in the “Scriot type” field. The contents are set in the “Script” field.

  In addition, in the action table 35, “look” that acts as a gaze action is set in the “Action” column for the script ID “1” associated with the Chinese character ID “1” in the script table 34, and the head 8 The angle indicating the left / right direction (in this case 0 °) for turning the head 8 is set in the “Target x” field, and the angle indicating the up / down direction for turning the head 8 (in this case, moving the head upward by about 30 °) Is set in the “Target y” field.

  Here, the script type roughly categorizes the utterance contents that the game activation robot 1 utters as a voice. For example, as shown in FIG. 4, for example, the action “Answer” that the game activation robot 1 autonomously answers. The action "Hesitate" that the game activation robot 1 answers when prompted by the MC MC, the action "Mutter" that the game activation robot 1 speaks, and problems such as impressions and tips It is composed of an action “SpeakImpression” that utters related information related to and an action “SpeakLevelImpression” that utters the difficulty level of the problem at the time of the question. In other words, the action “Answer”, the action “Hesitate”, the action “Mutter”, the action “SpeakImpression”, and the action “SpeakLevelImpression” are stored as script information (utterance information) in the script table 34 for each problem. For each action “Answer”, action “Hesitate”, action “Mutter”, action “SpeakImpression”, and action “SpeakLevelImpression”, predetermined utterance contents suitable for the title are set.

  In practice, the script type set for each problem can be set in the monitoring server 2. The monitoring server 2 can set the script information on the setting screen 40 as shown in FIG. 5, and the contents set via the setting screen 40 are transmitted to the game activation robot 1, and the network interface 26 And stored in the database 21 as individual action pattern information. For example, in the case where the Chinese character “鯔 鯔” is set as the problem of the ID number “81”, basic information 41 and script information 42 regarding this “鯔 鯔” can be set. As the basic information 41, an ID number (in this case, “81”) is set in the “ID” field, and the kanji character “Kashiwa” in question is set in the “Kanji” field, and the reading kana “Inase” is the answer. Is set in the “Reading” column. In addition, as basic information 41, “verb, adjective, adverb name” indicating the genre of the part of speech is set in the “genre” column, and the relevant information such as the meaning of the problem “sore” and its history (Topics) is set in the “Other information” column. The script information 42 of this problem includes three actions “Answer” (“answer” in the figure), three actions “Hesitate” (“hesitate” in the figure), and four actions “ “Mutter” (“mutter” in the figure) and three actions “SpeakImpression” (“speak # impression” in the figure) are set.

  As described above, the contents of the action “Answer”, action “Hesitate”, action “Mutter”, and action “SpeakImpression” indicating the script type indicate the correctness (“closeness to the correct answer”) set in the correctness column 44. For example, it is 0 to 100, and a larger value indicates that the answer is closer to the correct answer). In this case, every time the MC MC utters the word “good” (a word indicating that it is close to the correct answer), the game activation system 15 determines that it is approaching the correct answer, and the correct answer indicating that it is closer to the correct answer. It is designed to output utterance contents with a high degree. In the script type field 43 in which each script type is input, in addition to the correctness degree field 44, an utterance content field 45 in which optimum utterance content is set and an operation content of the game activation robot 1 are set. An action column 46 is provided. In this case, for the action “Answer”, for example, the utterance content “Nekoze!” Is set to “10”, which has a low accuracy, and the utterance content “Semushi?” Is set to the accuracy “20”. In addition, the content of the utterance “Boraze” is set to the correct answer level “30” (ie, the word “Boraze” is closer to the correct answer than the word “Nekoze!”).

  In addition, the action “Hesitate” has an utterance content of “Yes? I can only read the back of the second letter” in the accuracy level “10”, and “N? The content of the utterance “Is it related to the back?” Is set. The action “Mutter” is set to the correct answer level “10” and the utterance content “What is the first letter read?” And the correct answer level “20” is “I think the first letter is the name of the fish.” The correct utterance content is set to the correct answer level “30”. In the action “SpeakImpression”, the correct answer level “10” is set with the utterance content “Inase Daenee”, and the correct answer level “20” is set with the utterance content “Edo Kotoba, Good”. , The content of the utterance “pure and miserable” is set for the correct answer “30”.

  Furthermore, the action contents of the game activation robot 1 are set in advance in these actions “Answer”, action “Hesitate”, action “Mutter”, and action “SpeakImpression”, and the game activation is performed when each utterance is issued. The robot 1 can move the head 8 and the arm 9 to perform an operation suitable for the utterance content. For example, the action “lookP 0 0, happy” is set in association with the utterance content “Nekoze!” With the correctness level “10” in the action “Answer”. Here, “lookP” indicates that the whole body of the game activation robot 1 is directed in a specific direction. “0 0” rotates the head 8 in the direction of the target (ie, participant A or the like). Left and right direction x, and up and down direction y (in this case, x is 0, y is 0 and the head 8 does not rotate), and “happy” is a sound tone that utters the utterance content happily. Indicates.

The action in the "Answer" is, for the speech content of the correct answer of "20", "hunchback?", Operating tape b le 35 action, which is set to "lookP 0 0, happy, handup" is set in association with ing. Here, “handup” indicates that the arm portion 9 of the game activation robot 1 is raised. In addition, in this action column 46, “sad” indicating a voice tone that utters the utterance content sadly, “shake” that causes the arm part 9 to swing, and the arm part 9 on the head 8 “Handthink” for making an action indicating that the person is thinking and “nod” for turning the head 8 up and down to make a nodding action are set. Such setting contents can be freely set for each script type column 43 by selecting and determining the correction button 47 provided on the setting screen 40. In the setting screen 40, a script type can be added as new script information 44 by selecting and determining the new addition button 48.

(2-2) About common action pattern information Next, the common action pattern information memorize | stored in the database 21 is demonstrated. As shown in FIG. 4, the common action pattern information that is “an action whose utterance content does not change” includes, for example, an action “AskHint” that requests a hint, and an action “React TO” that reacts to the utterance “Oishi” of the MC MC. Oshii ”, the action“ React TO Seikai ”that responds to the utterance of“ right ”by the MC, the action“ Say Something ”that speaks something, and the action“ Let Answer ”that prompts someone to answer It consists of the action “Look Someone”, these actions “AskHint”, action “React TO Oshii”, action “React TO Seikai”, action “Say Something”, action “Let Answer” and action “Look Someone” A pattern is preset.

  For example, the action “AskHint” that requests a hint is associated with an action pattern that issues a hint to the presenter, for example, “What is a hint?”. In addition, in the action “React TO Oshii” that reacts to the utterance of “Oshii” by the MC MC, the utterance of the moderator is confirmed and other participants A, B and An action pattern of speaking a language for recognizing that C is close to the correct answer is associated. The action “React TO Seikai” that responds to the MC's utterance of “correct answer” is a language for allowing other participants A, B, and C to recognize the correct answer, such as “Is that so!” Is associated with an action pattern of uttering. An action pattern of uttering a single word such as “What is it?” Is associated with an action “Say Something” that speaks something. In the action “Let Answer” that prompts someone to answer, the direction of participant A, B, or C while speaking, for example, “How is A (B or C)?” To prompt participant A, B, or C to answer Is associated with an action pattern of rotating the head 8. In the action “Look Someone” to see someone, the head 8 is randomly rotated in the direction of the presenter, participant A, B or C without speaking, and the game activation robot 1 is the presenter and participant. Action patterns as if facing the direction of A, B or C are associated.

(2-3) Progress Status Keywords In addition to this, the database 21 stores a keyword table in which a plurality of progress status keywords are set. These progress status keywords are used to estimate the approximate progress status of the obfuscated game from the utterance contents of the MC MC controlling the progress of the obfuscated game, and trigger various action patterns of the game activation robot 1. Can be. In this embodiment, as shown in FIG. 6, the keyword table 50 includes a start keyword “HAJIME” for recognizing “start an obfuscated game” uttered by the MC as a progress status keyword. And the correct keyword “SEIKAI” for recognizing the “correct answer” spoken by the MC in response to someone ’s answer, and the determination for recognizing “good” spoken by the MC in response to someone ’s answer. The keyword “OSHII”, the incorrect keyword “CHIGAU” for recognizing “different” spoken by MC MC in response to someone ’s answer, and the “next” spoken by MC MC to start the next question The problem change keyword “TSUGI” for recognizing In addition, although not shown in the figure, the keyword presentation keyword “mondai” for recognizing “Is a problem” spoken by the MC is also set in order to recognize that the game is in progress.

  Further, in the keyword table 50, as a progress status keyword, an advisory keyword “HINT” for recognizing “hint is” spoken by the MC to provide a hint, and the MC MC activates the game activation robot 1. The robot answer keyword “KAITO ROBOTTO” for recognizing the utterance that prompted the answer, the A answer keyword “KAITO A” for recognizing the utterance that the MC invited the participant A to answer, and the MC MC Answer keyword “KAITO B” for recognizing the utterance that urged participant B to answer, and C answer keyword “KAITO C” for recognizing the utterance that MC invited participant C to answer Is set.

(2-4) Action Pattern Based on Progress Status Keyword Here, as shown in FIG. 2, in the voice recognition unit 22 for recognizing the progress status keyword described above, the voice obtained from the headset type microphone 5 It has a function of recognizing words contained in the speech signal in units of words by executing predetermined speech recognition processing on the signal, and sends the recognized words to the situation management unit 17 as character string data. Has been made. The action selection unit 18 receives the character string data obtained by the voice recognition unit from the situation management unit 17, reads out the progress keyword stored in advance in the database 21, and reads any of the progress keywords and the character string data. It is judged whether or not. When the character string data matches any of the progress status keywords, the action selection unit 18 calculates the expected activity increase amount ΔA based on the following formula, and displays the individual action pattern information (“the utterance content changes” shown in FIG. The behavior pattern that maximizes the activity can be selected from the common behavior pattern information (“the behavior in which the utterance content does not change”).

  In this case, “Behavior i” is the behavior “Answer”, behavior “Hesitate”, behavior “Mutter”, behavior “SpeakImpression”, behavior “SpeakLevelImpression”, behavior “AskHint”, behavior “React TO Oshii”, behavior in FIG. “React TO Seikai”, action “Say Something”, action “Let Answer”, or action “Look Someone”. “Person j” is a participant A, B whose activity increases with each of these action patterns Or C. “Ftask” indicates an activity evaluation function based on the behavior pattern and the participants A, B, or C whose activity increases. That is, “ftask” expresses that the activity is calculated from the action pattern and the target person (participant A, B, or C) that is the cause. As “ftask”, for example, based on an expected effect table (FIG. 7 to be described later), an expected activity increase amount ΔA due to the game activation robot 1 executing a predetermined action pattern is estimated in advance. Some of them indicate that the game activation robot 1 determines an action pattern. In this way, using ftask, the game activation robot 1 selects an action pattern that maximizes the predicted activity increase amount ΔA and a participant A, B, or C as shown in the following equation.

  That is, when the speech recognition unit 22 recognizes that the MC MC has prompted the game activation robot 1 to answer based on the robot answer keyword “KAITO ROBOTTO” (“KAITO ROBOTTO” in FIG. 7), the above number Based on the equation 1, a predicted activity increase amount ΔA for each of these behavior patterns shown in FIG. 4 is calculated. Among these, an action “Hesitate” (game activation robot 1 Is an action to be answered by being prompted by the MC MC. The action selection unit 18 reads out the predetermined utterance content associated with the action pattern that is a selection candidate from the database 21 and sends it to the speech synthesis unit 23. The voice synthesizer 23 has a function of converting the utterance content given from the action selection unit 18 into a voice signal, and sends the voice signal thus obtained to the speaker 24. As a result, sound based on the sound signal can be output from the speaker 24.

  On the other hand, when the voice recognition unit 22 recognizes the start keyword “HAJIME”, the behavior selection unit 18 performs the behavior “Say Something” in which the expected activity increase amount ΔA is relatively increased accordingly. When the correct keyword “SEIKAI” is recognized by the voice recognition unit 22 as a selection candidate, an action “React TO Seikai” in which the expected activity increase amount ΔA is relatively increased accordingly is selected as a selection candidate. When the speech recognition unit 22 recognizes the determination keyword “OSHII”, an action “React TO Oshii” having a relatively increased expected activity increase amount ΔA according to the determination keyword “OSHII” can be selected as a selection candidate.

  Similarly, when the incorrect keyword “CHIGAU” is recognized by the voice recognition unit 22, the behavior selection unit 18 selects the behavior “Say Something” whose predicted activity increase amount ΔA has increased relatively accordingly. When the problem-changing keyword “TSUGI” is recognized by the voice recognition unit 22, the action “Say Something” whose expected activity increase ΔA is relatively increased in response to this is selected as a selection candidate. When the recognition keyword “HINT” is recognized by the recognition unit 22, an action “Mutter” in which the expected activity increase amount ΔA is relatively increased in response thereto can be selected as a selection candidate. In addition, when the robot recognition keyword “KAITO ROBOTTO” is recognized by the voice recognition unit 22, the behavior selection unit 18 selects the behavior “Hesitate” in which the predicted activity increase amount ΔA is relatively increased in accordance with this as a selection candidate. In addition, when the A answer keyword “KAITO A”, the B answer keyword “KAITO B”, or the C answer keyword “KAITO C” is recognized in the voice recognition unit 22, the expected activity increase amount ΔA increases accordingly. The action “Look Someone” may be selected.

  In addition to this, during the progress of the game, the MC MC spoke about how close the dialogue between the MC MC and the participants A, B, and C in the obfuscated game is until the correct answer. Judgment can be made on the basis of the voice of “Oh!”. In practice, when the character string data “Oshii” of the MC MC received from the situation management unit 17 matches the progress situation keyword (determination keyword “OSHII”), the action selection unit 18, for example, “ The correctness level set to “0” is incremented by “10” units, and the current correctness level is stored.

  Here, when the plurality of behavior patterns become selection candidates based on the predicted activity increase amount ΔA among the plurality of behavior patterns stored in the database 21, the plurality of behavior patterns that have become these selection candidates. It can be used as a guide when selecting an action pattern suitable for the progress situation from the list. That is, as a plurality of actions “Mutter” (behavior that the game activation robot 1 speaks to itself) as a selection candidate because the expected activity increase amount ΔA has relatively increased, “fish?” (Correctness: 10) and “Oh, isn't it a big and elegant fish?” (Accuracy level: 50), the utterance of MC “Ooi” When the degree of correctness rises to 50 or more at present, the utterance content of “Oh, isn't it a large and elegantly swimming fish?” (Correctness degree: 50) is a candidate for selection. On the other hand, when the correct answer level is 50 or less at the present time, the utterance content of “fish?” (Correct answer: 10) with a low correct answer is a selection candidate.

  For example, when an action pattern having the same degree of correctness is set in parallel for the action “Mutter”, the action selecting unit 18 randomly selects one of these action patterns. In this way, the game activation system 15 grasps the progress of whether or not the correct answer is approaching based on the correct answer varying based on the voice of the MC MC “Ooi”. By uttering the utterance content previously associated with the utterance content, it is possible to activate the direct dialogue between the MC and the participant based on the utterance content optimal for the progress situation.

(2-5) Action Pattern Based on Activity Next, the case where the game activation robot 1 spontaneously acts according to the activity will be described below. As shown in FIG. 2, the activity detection unit 25 constantly monitors the video signals of the participants A, B, and C given from the camera 11 and executes face recognition processing on the video signals, so that the video Based on the signal, the number of participants A, B, and C, the postures of the participants A, B, and C, the face direction, and facial expressions such as smiles can be detected.

  In practice, when the obfuscation game is started, the activity detection unit 25 first determines the direction of the face from the contours of the faces of the participants A, B and C based on the video signal given from the camera 11 first. Detect direction. Then, the activity detection unit 25 determines whether each participant A, B and C stares at the whiteboard WB or the MC MC presenting the problem from these face direction directions, and each participant A, The initial activity of B and C can be specified.

  Incidentally, in the case of this embodiment, the activity detection unit 25 is based on the video signal given from the camera 11 and the whiteboard WB or the MC MC specified in advance in the direction of the detected speaker's face direction. It can be determined whether or not each participant A, B, and C stares at the whiteboard WB or the MC MC depending on whether or not it is displayed. As for whether each participant A, B and C stares at the whiteboard WB or the MC MC, the positional relationship between the whiteboard WB, the MC MC and each participant A, B and C is determined in advance. It may be inputted as position data, and it may be determined whether or not each participant A, B and C stares at the whiteboard WB or the MC MC based on the position data.

  Here, the activity is estimated by observing the states of the participants A, B, and C, and how actively the participants A, B, and C participate in the obfuscated game. This is a digitized version. In the case of this embodiment, when the activity detection unit 25 determines that the face direction of the participants A, B, and C is the direction of the whiteboard WB or the MC MC based on the video signal, the participant A , B or C can be determined to actively participate in the obfuscated game, and the activity level of the participant A, B or C can be set to a predetermined numerical value. On the other hand, when the activity detection unit 25 determines that the face orientation directions of the participants A, B, and C are directions other than the whiteboard WB or the MC MC based on the video signal, the participants A, B Alternatively, it may be determined that C is not actively participating in the obfuscated game, and the activity level of the participant A, B, or C may be set to zero.

  Further, the activity detection unit 25 constantly monitors the video signal given from the camera 11, and based on the video signal, the facial expressions (smiles) from the eyes and mouths of the participants A, B and C are obtained. It can detect and change the activity of each participant A, B, and C from these facial expressions. In this embodiment, the activity detection unit 25 compares the eyes and mouths of the participants in the video with the pre-stored face patterns based on the video signal, and the participants A, B, and C smile. It is determined whether or not. As a result, when the activity detection unit 25 determines that the participant A, B, or C is smiling, the activity detection unit 25 determines that the participant A, B, or C is actively participating in the obfuscated game, and the participation When the activity level of the participant A is increased and the facial expression of the participant A, B or C is determined not to be a smile, the participant A, B or C is not actively participating in the obfuscation game. Judging, the activity of the participant A, B or C may be reduced.

  The activity detection unit 25 constantly monitors a voice signal obtained by collecting the voices of the participants A, B, and C with the microphone 20, and executes a predetermined voice identification process on the voice signal. By doing so, it is possible to estimate the amount of speech (whether or not there is a speech signal) as to whether or not each participant A, B and C is speaking based on the speech signal. In the case of this embodiment, when the activity detection unit 25 determines that a predetermined participant A, B or C is speaking based on the audio signal, the participant A, B or C The activity of the participant A, B or C can be increased. On the other hand, when the activity detection unit 25 determines that the predetermined participant A, B, or C is not speaking based on the audio signal, the participant A, B, or C actively participates in the obfuscated game. It may be determined that the activity of the participant A, B or C is reduced.

  In this way, the activity detection unit 25 calculates the activity for each of the participants A, B, and C based on the video signal from the camera 11 and the audio signal from the microphone 20, and each participant A, B, and The activity of C can be sent to the status management unit 17. The situation management unit 17 can calculate the total activity obtained by adding up the activities of the participants A, B, and C, and send the activity and the total activity to the action selection unit 18. Yes. The action selection unit 18 constantly monitors the total activity sent from the activity detection unit 25 via the situation management unit 17, and determines whether the total activity is lower than a predetermined threshold value set in advance. It is made to be able to judge. When the action selection unit 18 determines that the total activity is lower than the predetermined threshold, the activity selection unit 18 uses the expected effect table 60 as shown in FIG. The increase amount ΔA is calculated, and a behavior pattern that maximizes the activity can be selected from the plurality of behavior patterns shown in FIG.

  Here, first, the expected effect table 60 will be described below. As shown in FIG. 7, the expected effect table 60 has a high effect of increasing the activities of the participants A, B, and C from currently selectable action patterns among the individual action pattern information and the common action pattern information. Can be used in selecting the expected behavior pattern. This expected effect table 60 indicates in advance what actions the participants A, B, and C will take as a result of the game activation robot 1 executing each action pattern in the action pattern field 61 (“Behavior Pattern”). It is defined in anticipation.

  Actually, the expected effect table 60 includes a state column 62 (“State” in the figure) indicating the progress state of the obfuscated game and an action pattern column 61 (in the figure) indicating the action pattern executed by the game activation robot 1. "Behavior Pattern"). The action pattern column 61 includes an action “Answer” that the game activation robot 1 answers by itself and an action “Ask that requests an answer from the participant A for the game in progress (“ GAME ”in the figure) in the state column 62. #A ”, an action“ Ask # B ”requesting an answer from the participant B, an action“ Ask # C ”requesting an answer from the participant C, and an action“ Hint ”requesting an hint from the MC MC In other words, “AskHint”) and an action “Mutter” in which the game activation robot 1 speaks to itself are provided. In addition, the behavior pattern column 61 requests the participant “AskImpression # A” for requesting an impression from the participant A and the participant B for an impression while explaining the state column 62 (“CORRECT” in the figure). The action “AskImpression # B”, the action “AskImpression # C” requesting the impression from the participant C, and the action “where the game activating robot 1 utters the related contents (related information) such as the impression of the problem and the knowledge” "SpeakImpression".

  The expected effect table 60 also includes an A gaze direction column 63 (“a # looking # at” in the figure) indicating the direction in which the participant A gazes for each behavior pattern in the behavior pattern column 61, and participation. B staring direction column 64 (“b # looking # at” in the figure) indicating the direction in which person B stares, and C staring direction column 65 (“c # looking #” in the figure) indicating the direction in which participant C stares at "), an A utterance column 66 (" a # speaking "in the figure) indicating whether or not the participant A speaks, and a B utterance column 67 (in the figure, indicating whether or not the participant B speaks) “B # speaking”), a C utterance column 68 indicating whether or not the participant C speaks (“c # speaking” in the figure), and an A facial expression column 69 indicating whether or not A laughs (see FIG. "A # laughing"), B expression field 70 indicating whether B laughs ("b # laughing" in the figure), and C expression field 71 indicating whether C laughs (in the figure) "C # laughing") and the A thought field 72 (Figure "A # thinking"), a B thinking column 73 indicating whether B is thinking ("b # thinking" in the figure), and a C thinking column 74 indicating whether C is thinking (in the figure) "C # thinking").

  Actually, in the expected effect table 60, the A gaze direction column 63, the B gaze direction column 64, and the C gaze direction column 65 associated with the action “Answer” are defined as “robot”, and the game activation robot If 1 performs the action “Answer”, it is expected that all participants A, B and C will stare at the game activated robot 1. In the expected effect table 60, the A utterance field 66, the B utterance field 67, and the C utterance field 68 associated with the action “Answer” are defined as “FALSE”. It is expected that when executing “Answer”, all participants A, B and C will not speak. Further, in the expected effect table 60, the A expression field 69, the B expression field 70, and the C expression field 71 associated with the action “Answer” are defined as “FALSE”, and the game activation robot 1 has the action “ It is expected that when executing “Answer”, all participants A, B and C will not laugh. In addition, in the expected effect table 60, the A thinking column 72, the B thinking column 73, and the C thinking column 74 associated with the action “Answer” are defined as “*”, and the game activation robot 1 It is expected that executing the action “Answer” will not change as to whether all participants A, B and C are thinking.

  Similarly, in the expected effect table 60, the A gaze direction column 63 associated with the action “Ask # A” is defined as “whiteboard”, and the B gaze direction column 64 and the C gaze direction column 65 are “A”. When the game activation robot 1 executes the action “Ask # A”, it is expected that the participant A will stare at the whiteboard WB, and B and C stare at the participant A. ing. In the expected effect table 60, the A utterance column 66 associated with the action “Ask # A” is defined as “TRUE”, and the B utterance column 67 and the C utterance column 68 are defined as “FALSE”. When the game activation robot 1 executes the action “Ask # A”, it is expected that the participant A will speak and B and C will not speak. Further, in the expected effect table 60, the A expression field 69, the B expression field 70, and the C expression field 71 associated with the action “Ask # A” are defined as “FALSE”. It is expected that when the action “Ask # A” is performed, all participants A, B and C will not laugh. In addition, in the expected effect table 60, the A thinking column 72 associated with the action “Ask # A” is defined as “”, and the B thinking column 73 and the C thinking column 74 are defined as “*”. When the game activation robot 1 executes the action “Ask # A”, it is expected that the participant A will think and will not change whether B and C are thinking.

  In the expected effect table 60, the A gaze direction column 63, the B gaze direction column 64, and the C gaze direction column 65 associated with the action “Hint” are defined as “mc”, and the game activation robot 1 It is expected that participants A, B and C will stare at MC MC when he performs the action “Hint”. In the expected effect table 60, the A thinking column 72, the B thinking column 73, and the C thinking column 74 associated with the action “Hint” are defined as “TRUE”. It is expected that all participants A, B, and C will think when performing “Hint”.

  Further, in the expected effect table 60, the A gaze direction column 63, the B gaze direction column 64, and the C gaze direction column 65 associated with the action “Mutter” are defined as “robot”, and the game activation robot 1 It is expected that all participants A, B, and C will stare at the game activation robot 1 when performing the action “Mutter”. In the expected effect table 60, the A expression field 69, the B expression field 70, and the C expression field 71 associated with the action “Mutter” are defined as “TRUE”. It is expected that all participants A, B, and C will laugh when executing “Mutter”.

  In the expected effect table 60, the A gaze direction column 63 associated with the action “AskImpression # A” requesting the impression from the participant A is defined as “robot”, and the B gaze direction column 64 and the C gaze direction column 65 Are defined as “A”, and when the game activation robot 1 executes the action “AskImpression # A”, the participant A stares at the game activation robot 1 and B and C stare at the participant A. It is expected to be. In the expected effect table 60, the A utterance column 66 associated with the action “AskImpression # A” is defined as “TRUE”, and the B utterance column 67 and the C utterance column 68 are defined as “FALSE”. When the game activation robot 1 executes the action “AskImpression # A”, it is expected that the participant A will speak and B and C will not speak. Further, in the expected effect table 60, the A expression field 69 associated with the action “AskImpression # A” is defined as “TRUE”, and the B expression field 70 and the C expression field 71 are defined as “*”. When the game activation robot 1 executes the action “AskImpression # A”, it is expected that only the participant A will laugh. In addition, in the expected effect table 60, the A thinking column 72 associated with the action “AskImpression # A” is defined as “TRUE”, and the B thinking column 73 and the C thinking column 74 are defined as “*”. If the game activation robot 1 executes the action “AskImpression # A”, it is expected that only the participant A will think.

  In the expected effect table 60, the A gaze direction column 63, the B gaze direction column 64, and the C gaze direction column 65 associated with the action “SpeakImpression” requesting the impression from the participant A are defined as “robot”. When the game activation robot 1 executes the action “SpeakImpression”, it is expected that all the participants A, B, and C will stare at the game activation robot 1. In the expected effect table 60, the A utterance field 66, the B utterance field 67, and the C utterance field 68 associated with the action “SpeakImpression” are defined as “FALSE”. It is expected that participants “A”, “B” and “C” will not speak when “SpeakImpression” is performed. Further, in the expected effect table 60, the A expression field 69, the B expression field 70, and the C expression field 71 associated with the action “SpeakImpression” are defined as “TRUE”, and the game activation robot 1 has the action “ It is expected that when performing “SpeakImpression”, all participants A, B and C will laugh.

  When the action selection unit 18 determines that the total activity is lower than the predetermined threshold, the behavior selection unit 18 calculates the expected activity increase amount ΔA based on the above-described equation (1) using the expected effect table 60. It is made to be able to do. Here, when the spontaneous action pattern of the game activation robot 1 is selected using the above equation (1), “Behavior i” in the above equation (1) is the action “Answer”, “action“ Ask # A, action 'Ask # B', action 'Ask # C', action 'Hint', action 'Mutter', action 'AskImpression # A', action 'AskImpression # B', action 'AskImpression # C', Or “SpeakImpression”, and “Person j” indicates a participant A, B, or C whose activity increases according to each of these behavior patterns. Then, the game activation robot 1 selects a behavior pattern that maximizes the predicted activity increase amount ΔA and a participant A, B, or C as shown in the above equation (2), and performs a predetermined behavior spontaneously. Can do.

  In this way, the action selection unit 18 determines that the game activation robot 1 of the game activation robot 1 is in the expected effect table 60 when the voice recognition unit 22 does not recognize the progress status keyword for a certain period of time and the total activity becomes equal to or less than a predetermined threshold. Based on the predicted behavior of each participant A, B, and C associated with each behavior pattern, the expected activity increase amount ΔA is calculated for all selectable behavior patterns using the above equation (1). The behavior selection unit 18 selects a behavior pattern that is predicted to have the highest activity effect in which the predicted activity increase amount ΔA is the largest among the predicted activity increase amounts ΔA calculated for each behavior pattern.

  Thereby, for example, when the behavior pattern in which the predicted activity increase amount ΔA has increased most during the game is the behavior “Mutter” (the behavior in which the game activation robot 1 speaks to itself), the behavior selection unit 18 The predetermined utterance content “Mutter” is read from the database 21 and transmitted to the speech synthesizer 23. The speech synthesizer 23 converts the utterance content (for example, “fish?”) Into an audio signal, and the obtained audio signal Is sent to the speaker 24. Thus, when the game activation robot 1 estimates that the participant A, B, or C is not actively participating in the obfuscated game based on the total activity, the game activation robot 1 selects based on the expected activity increase ΔA. By uttering the predetermined utterance content as sound from the speaker 24, the direct dialogue performed between the MC MC and the participant can be activated again based on the utterance content.

  By the way, in this game activation system 15, for example, when executing an action “AskImpression # A” that requests an impression from the participant A or an action “Ask # A” that requests an answer from the participant A, the camera 11 The head 8 can be rotated so that the participant A is arranged at the center of the angle of view. Actually, in the case of this embodiment, before starting the obfuscated game, position data (for turning the head 8 at the position of the game activation robot 1 and directing the front of the head 8 toward the participant A ( Rotation direction, angle, etc.) are set in advance for the action “Ask # A” and the action “AskImpression # A”, and the action “Ask # A” or action “AskImpression # A” is executed. When uttering the utterance content to the participant A, the impression is as if the game activating robot 1 is uttering by visually observing the participant A by rotating the head 8 based on the set position data. Has been made to give.

  In addition, the action “Ask # B” and the action “AskImpression # B” include position data for turning the head 8 at the position of the game activation robot 1 and directing the front of the head 8 toward the participant B. When the action “Ask # B” or the action “AskImpression # B” is executed and the utterance content is uttered to the participant B, the head 8 is rotated based on the set position data. Thus, an impression is given as if the game activation robot 1 is speaking while visually observing the participant B.

  Further, the action “Ask # C” and the action “AskImpression # C” also have position data for turning the head 8 at the position of the game activation robot 1 and directing the front of the head 8 toward the participant C. When the action “Ask # C” or the action “AskImpression # C” is executed and the utterance content is uttered to the participant C, the head 8 is rotated based on the set position data. Thus, an impression is given as if the game activation robot 1 is speaking while visually observing the participant C.

  In addition, even when the utterance contents uttered toward the MC and the utterance contents about the kanji presented on the whiteboard WB are issued, the position data of the MC and the whiteboard WB are previously stored for these behavior patterns. When the utterance content is set and the head 8 is rotated based on the position data, it is as if the game activating robot 1 is speaking by visually observing the MC MC and the whiteboard WB. It is made to give an impression.

  In addition to this, the situation management unit 17 reads the current state of progress stored in the database 21 before the question, in progress of the game or during the explanation as the understanding state data, and reads this understanding state data through the network interface 26. It can be transmitted to the monitoring server 2 via the network. The monitoring server 2 can transmit the understanding state data received from the game activation robot 1 to the portable terminal 3. As a result, the mobile terminal 3 uses the understanding state data received from the game activation system 15 via the monitoring server 2 to indicate that the game activation robot 1 is currently in progress as the obfuscated game progress before the question is being played and the game is in progress. Alternatively, the display unit 4 displays which one of the explanations is recognized. As a result, the mobile terminal 3 can present to the MC MC the progress that the game activation robot 1 is currently grasping at the present time. Thus, the MC MC can confirm the progress status currently understood by the game activation robot 1 based on the understanding state data, and the progress status grasped by the game activation robot 1 can be determined based on the actual obfuscated game. It is possible to advance the obfuscated game while confirming whether or not it is consistent with the progress status.

(3) Operation and Effect In the above configuration, the game activation system 15 collects the voices of the participants A, B, and C who talk about the problem answer of the obfuscated game with the MC by the microphone 20. By acquiring the audio signals of the participants A, B, and C, the video signals of the participants A, B, and C are acquired by capturing the participants A, B, and C with the camera 11. Based on the audio signal and the video signal, the total activity indicating the degree of participation of all the participants A, B and C in the obfuscated game is calculated.

  The game activation system 15 calculates the predicted activity increase ΔA for all selectable behavior patterns when the progress status keyword is not recognized for a certain period of time and the total activity is below a predetermined threshold. The action pattern of the predicted activity increase amount ΔA that has increased most among the plurality of predicted activity increase amounts ΔA is selected as a candidate for selection. In the game activation system 15, predetermined utterance content that is an action pattern selected as a selection candidate is read from the database 21, and the utterance content is uttered from the speaker 24 as sound.

  Thus, in the game activation system 15, the states of the participants A, B, and C are constantly observed based on the total activity, and when the total activity is equal to or less than a predetermined threshold, the participant participates in the obfuscated game. Estimate that there is a possibility that the speaker has not been informed, and spontaneously utter the optimum utterance content from the speaker 24, to give a direct dialogue between the MC MC and the participants A, B and C Thus, direct communication between the MC MC and the participants A, B, and C can be further activated than before.

In addition, the game activation system 15 recognizes the progress of the obfuscated game based on the audio signal obtained by collecting the voice of the MC MC by the headset type microphone 5, and the utterance stored in the database 21. The utterance content corresponding to the progress is selected from the contents, and the utterance content is uttered from the speaker 24.

  In this way, in the game activation system 15, for example, the moderator MC questions the problem and the participant A, B, or C answers the game, and the utterances that are suitable for the commentary explaining the answer after the correct answer are given. The content can be issued, thus giving a chance for direct dialogue between the MC and the participants, and more direct communication between the MC and the participants A, B and C than before. obtain. Also, in the game activation system 15, while the game is in progress, based on the degree of accuracy, the progress of how much the correct answer is approaching is also grasped, and the optimal utterance content according to the progress of the obfuscated game is issued Thus, it is possible to provide an opportunity for direct dialogue between the MC MC and the participant, and to further activate direct communication between the MC MC and the participants A, B, and C than before.

  In addition, in the game activation system 15, for each problem, the history and origin of the kanji that are the problem, the meaning of the kanji, and related information related to the reading kana (answer) are stored in the database 21 in advance. Depending on the total activity and the progress of the obfuscated game, the utterance content of the related information is appropriately issued. As a result, the game activation system 15 provides not only the answer to the problem but also various utterances that are different from the answer, thereby providing a topic related to the problem and the answer. It is possible to activate the direct communication between the MC MC and the participants A, B, and C more than before by providing an appropriate dialogue opportunity.

  Further, in the game activation system 15, the face direction of each participant A, B and C, the facial expression of each participant A, B and C, each participant A based on the video signal given from the camera 11. The degree of whether each participant A, B, and C is actively participating in the obfuscated game by calculating the total activity based on the presence or absence (speech volume) of B, C, and C Can be recognized more accurately.

  In addition to this, in the game activation robot 1 incorporating the game activation system 15, a head 8 similar to a human head is provided, and a camera 11 is provided at a position of the head 8 that becomes a human eye. When the participants A, B, and C visually recognize the game activation robot 1, the eyes of the game activation robot 1 are attracted attention, and the participants A, B, and C are surely secured by the camera 11 provided on the eyes. The total activity indicating the degree of participation of each participant A, B, and C in the obfuscated game can be calculated using this video signal.

  Thus, in this way, the game activation robot 1 constantly observes the states of the participants A, B, and C based on the total activity, and when the total activity is equal to or less than the predetermined threshold, It is estimated that there is a possibility of not participating in the event, and by spontaneously uttering the content of the utterance best suited to the situation from the speaker 24, a direct dialogue between the MC MC and the participants A, B and C Thus, direct communication between the MC MC and the participants A, B, and C can be activated more than before.

  Further, in the game activation robot 1, when a predetermined utterance content is uttered based on the total activity, the MC MC, the whiteboard WB, and the participants A, B, or C to be uttered utterance content are By turning the head 8 so as to be arranged at the center position of the angle of view of 11, the game activating robot 1 is as if the MC MC, the whiteboard WB, the participants A, B or C You can give the impression that you are speaking.

  In the game activation robot 1, the utterance situation understanding unit 16 recognizes the utterer from the participants A, B, and C, and the action reproducing unit 19 places the utterer at the center of the angle of view of the camera 11. As described above, by adjusting the angle of the head 8 provided with the camera 11, an impression is given to the participant as if he or she is facing the speaker and listening to the speaker's speech. Can do.

  Furthermore, in the game activation robot 1, the utterance state understanding unit 16 shows the utterance target detection result indicating the utterance target spoken by the speaker from the face direction and the utterance end detection result indicating that the speaker has finished speaking. Based on these utterance target detection results and utterance end detection results, the utterance target in the direction that the speaker is facing is placed at the center of the angle of view of the camera 11 after the utterance ends. As a result, the impression that the game activation robot 1 is actively participating in an obfuscated game is directed to the direction of the utterance target after the utterance of the speaker ends. Can be given for A, B and C.

  Further, the game activation robot 1 recognizes the break of the speaker based on the detection result of the end of the speech, and performs a nodding operation for rotating the head 8 in the vertical direction at that timing, so that it is as if the speaker is speaking. The participants A, B, and C can be given an impression as if they were listening to

  Further, the game activation robot 1 transmits the activity and total activity of each participant A, B, and C calculated by the activity detection unit 25 to the monitoring server 2 through the network interface 26 as activity information, The activity information is transmitted from the monitoring server 2 to the mobile terminal 3. Thereby, the game activation robot 1 presents the activity level information to the presenter MC via the display unit 4 of the mobile terminal 3, so that how actively each participant A, B, and C participates in the obfuscated game at the present time. It is possible to objectively determine whether or not he / she is participating in the activity based on the activity information.

(4) Other Embodiments The present invention is not limited to this embodiment, and various modifications can be made within the scope of the gist of the present invention. For example, the present invention resembles an animal such as a dog. A game activation robot may be applied. In the above-described embodiment, the case where the game activating robot 1 is applied when performing an obfuscated game that answers a kana reading kana as the dialogue activating robot has been described. However, the present invention is not limited to this. Applied to other games, such as games that apply translation languages to Japanese and English and English-Japanese languages, etc., and not only games, but also between a MC and participants A, B, and C on a predetermined subject A dialog activation robot may be applied to a simple dialog. In this case, the individual action pattern information, the common action pattern information, and the expected effect table 60 corresponding to the other game and the subject may be prepared in advance and stored in the database 21.

  In the above-described embodiment, a predetermined threshold is set for the total activity, and when the total activity falls below the threshold, the game activation robot 1 executes an optimal action pattern that increases the total activity. However, the present invention is not limited to this, and a predetermined threshold is provided for each activity calculated for each participant A, B, and C, and any one of the activities is equal to or lower than the predetermined threshold. When it becomes, you may make it the game activation robot 1 perform the optimal action pattern which increases the said activity.

  Further, in the above-described embodiment, when the progress status keyword is recognized, an action pattern having the highest expected activity increase amount ΔA is selected as a selection candidate, and the action pattern that is the selection candidate is executed. Although the case has been described, the present invention is not limited to this, and when a progress situation keyword is recognized, an action pattern previously associated with the progress situation keyword may be executed.

  Further, in the above-described embodiment, the lip motion recognition process for detecting the speaker by identifying the movements of the mouths of the participants A, B, and C based on the video signal for the detection of the speaker in the utterance state understanding unit 16 However, the present invention is not limited to this. For example, speaker recognition processing for detecting participants A, B, and C speaking based on the sound collected by the microphone 20, Voice recognition processing for registering voice prints of each participant A, B, and C and detecting the utterances of each participant A, B, and C based on this, and providing microphones for each participant A, B, and C individually , By voice classification processing that identifies participants' utterances based on the microphones assigned to each participant, or by directional microphones fixed to the participants and registering the orientation of each participant The direction of the sound source Various other speaker specifying processes such as a sound source localization process for identifying a participant's utterance may be used.

  Furthermore, in the above-described embodiment, the microphone is used as the activity level specifying unit that specifies the activity level indicating the degree of participation in the conversation related to the subject of the participant based on the audio and / or video acquired by the acquiring unit. In the case of applying the activity detection unit 25 that detects the activity indicating the degree of participation of the participants A, B, and C in the obfuscated game based on the audio acquired by 20 and the video acquired by the camera 11 As described above, the present invention is not limited to this. Based on only the sound acquired by the microphone 20, the activity indicating the degree of participation of the participants A, B, and C in the obfuscated game is detected, or the camera 11 Based on only the video acquired by the above, an activity detection unit that detects the activity indicating the degree of participation of the participants A, B, and C in the obfuscated game may be applied.

  Furthermore, in the above-described embodiment, facial expressions obtained from the images of the participants A, B, and C, the orientation directions of the faces obtained from the images of the participants A, B, and C, and the participants A, B Although the case where the activity level is detected based on the presence or absence (utterance amount) of the voices of C and C is described, the present invention is not limited to this, and the face obtained from the images of the participants A, B, and C is not limited thereto. The activity is determined based on any one of the expression, the direction of the face obtained from the images of the participants A, B, and C, and the presence or absence (utterance amount) of the voices of the participants A, B, and C. You may make it detect.

  Further, in the above-described embodiment, the face direction of the speaker A, B, or C is recognized based on the video obtained by the camera 11, and the face direction of the speaker A, B, or C is determined after the end of the utterance. Although the case where the head of the game activation robot 1 is rotated has been described, the present invention is not limited to this, and recognizes the line-of-sight direction of the speaker A, B, or C, and the speaker A, The head of the game activation robot 1 may be rotated in the B or C line-of-sight direction. In this case, as a technique for recognizing the line-of-sight direction, a line-of-sight recognition technique based on appearance information using a partial image of the eye region may be applied.

  Furthermore, in the above-described embodiment, the case where the determination keyword “OSHII” for recognizing the utterance content of “Oshii” of the presenter is applied as the progress status keyword for increasing the correctness degree is described. However, as a progress status keyword that increases the degree of accuracy, a determination keyword that recognizes various utterance contents that provide advice such as “close” to the presenter may be applied. The correct answer may be reduced based on the recognition of the incorrect keyword “CHIGAU” that recognizes the content of the utterance “,” and the correct answer may be varied according to the progress.

  Furthermore, in the above-described embodiment, the headset microphone 5 used exclusively by the MC is used as a microphone for collecting the voice of the MC, but the present invention is not limited to this, and the MC However, various microphones such as a microphone for a participant who is not used exclusively may be applied.

  Further, in the above-described embodiment, the obfuscated game is based on the action of uttering the utterance content according to the situation from the speaker 24 based on the total activity and the sound signal obtained by collecting the sound of the MC MC. However, the present invention is not limited to this, and the utterance content according to the situation based on the total activity is described. Executes only the action uttered from the speaker 24, or recognizes the progress of the obfuscated game based on the audio signal obtained by collecting the voice of the MC MC, and the utterance content corresponding to the progress Only actions originating from 24 may be executed.

  Furthermore, in the above-described embodiment, the head 8 is driven so that the speaker is placed at the center position of the angle of view of the camera 11 based on the speaker detection result, or the utterance target detection result and the utterance end Based on the detection result, the case where the head 8 is driven so that the utterance target ahead in the direction of the face of the speaker is arranged at the center of the angle of view of the camera 11 has been described. Is not limited to this, it is very good not to drive the head 8 so that the speaker and the utterance target are arranged at the center of the angle of view of the camera 11, and only the optimum utterance content according to the situation of the conversation is uttered. But you can.

It is the schematic where it uses for description of the obfuscated game using a game activation robot. It is a block diagram which shows the circuit structure of a game activation system. It is the schematic which shows the structure of individual action pattern information. It is the schematic which shows an action pattern. It is the schematic which shows the structure of a setting screen. It is the schematic which shows a progress condition keyword. It is the schematic which shows the structure of an anticipation effect table.

1 Game activation robot (dialog activation robot)
4 Display section (display means)
5 Headset microphone ( moderator microphone)
8 head
11 Camera (imaging means, acquisition means)
15 Game activation system (dialog activation system)
16 Utterance situation understanding department
20 Microphone (Acquisition means)
21 Database (storage means)
24 Speaker
25 Activity detector (activity estimation means)

Claims (4)

Imaging means for acquiring images of a plurality of participants interacting with a moderator on a predetermined subject;
An activity estimation means for estimating an activity indicating a degree of participation in the dialogue on the subject of the participant based on the video acquired by the imaging means;
A pre-stored memory means a plurality of progress keyword,
Utterance detection means for detecting the participant who is speaking;
A speech recognition unit that recognizes words included in the speech signal by performing predetermined speech recognition processing on the speech signal obtained from the microphone, and sends the recognized word as character string data;
The character string data obtained by the voice recognition unit is received, the progress keyword stored in the storage means is read out, and whether any of the progress keywords matches the character string data And selecting means for selecting utterance information that prompts participation in the dialogue on the subject according to the activity estimated by the activity estimation means when the character string data matches any of the progress status keywords ; ,
A speaker that utters the utterance content of the utterance information selected by the selection means;
With a head resembling an animal shape,
The imaging means is installed at a position to be the eye of the head ,
The utterance detection means is a lip motion recognition process for recognizing lip movement recognized from the video obtained by the imaging means, or a speaker recognition process based on the volume of a microphone that directs directivity to each speaker, or Using any of sound source localization processing based on the sound collected by the microphone, the participant who is speaking is identified,
Control means for placing the participant detected by the utterance detection means at the center of the angle of view of the imaging means;
The selection means does not recognize the progress status keyword for a certain period of time, and when the total activity total activity obtained by adding up the activities of a plurality of participants falls below a predetermined threshold, the predetermined utterance content is Read from the storage means and utter the utterance content as a voice from the speaker,
Recognizing a break in the participant's voice and performing at least one of a nodding operation for moving the head in accordance with the break or a collaborative speech operation in which a conflicting voice is generated in accordance with the break. Dialogue activation robot. When the utterance content of the utterance information is uttered from the speaker as speech, the control unit is configured to arrange the participant or related equipment to be the utterance content at the center of the angle of view of the imaging unit. The dialog activation robot according to claim 1 . Recognizing means for recognizing the participant's face direction or line-of-sight direction based on the video obtained by the imaging means,
The control means includes
Wherein after the participant has finished speaking, the face orientation direction or line-of-sight direction of the participants recognized by the recognition means, interactive activity according to claim 1, wherein the placement of the center of the angle of view of the imaging means Robot. A microphone for the moderator who collects the voice of the moderator who talks with the participants about a predetermined subject,
Progress status recognition means for recognizing the progress status of the dialogue with the participant on the subject based on an audio signal obtained by collecting the voice of the moderator by the microphone for the moderator ;
2. The dialog activation robot according to claim 1 , further comprising display means for presenting an understanding state regarding the progress status of the dialog in the progress status recognition means to the presenter.