JP2018132624A - Voice interaction apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate emotion while suppressing the influence due to a difference depending on interaction scenes.SOLUTION: A voice interaction apparatus 1 which recognizes voice of a user on the other side of interaction and outputs voice to the user includes: a feature quantity acquisition unit 101 for acquiring feature quantity to be used for estimating emotion from the user; an interaction scene determination unit 104 which determines whether the user or the device is a listener and whether the user or the device is a speaker, on the basis of interaction history of the user and the device; an emotion threshold setting unit 106 which sets a threshold for estimating emotion, in accordance with a result determined by the interaction scene determination unit 104; and an emotion estimation unit 105 which calculates an index value of emotion of the user, on the basis of the feature quantity, and estimates emotion of the user, in accordance with a result of comparing the calculated index value with the threshold set by the emotion threshold setting unit 106.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a voice interaction apparatus, and more particularly to a voice interaction apparatus that estimates emotions.

  2. Description of the Related Art A voice interactive device that has a conversation with a user is known. With regard to such a technique, for example, Patent Document 1 discloses a dialog processing apparatus that performs a response in accordance with the user's emotion. In this interactive processing device, based on the content of the user's utterance, it is estimated according to a certain criterion whether the user's emotion is positive, negative, or neutral, and a response is made according to the estimation result.

  As described above, there is a technique for indexing a user's emotion based on a feature amount such as language information, phonological information, or image information, and estimating the emotion based on a certain criterion regardless of the user.

JP 2006-178063 A

  However, the degree of reflection of the user's emotion in the feature quantity used for emotion estimation may differ depending on the conversation scene. For example, when the user is mainly interacting as a speaker, the degree of reflection of emotion on the feature amount is greater than when the user is mainly interacting as a listener. In the case of the technique described in Patent Document 1, in the case of a conversation scene in which the user's emotion is difficult to be reflected in the feature amount, the estimation result is often neutral for the user, and the user's emotion is characteristic. In the case of an interactive scene that is easily reflected in the amount, positive or negative increases as an estimation result. In this way, there are many variations depending on the conversation scene, and appropriate emotion estimation cannot be performed.

  The present invention has been made against the background of the above-described circumstances, and an object thereof is to provide a voice interaction apparatus capable of performing emotion estimation while suppressing the influence of a difference in a conversation scene.

One aspect of the present invention for achieving the above object is a speech dialogue apparatus that recognizes a speech voice of a user who is a conversation partner and outputs the voice to the user, wherein the feature amount used for emotion estimation is the user. Based on the feature amount acquisition unit acquired from the user and the conversation history of the user and the own apparatus, the conversation scene determination is performed to determine which of the user and the own apparatus is interacting as the listener side and which is interacting as the speaker side. And an emotion threshold setting unit for setting a threshold for estimating emotion according to a determination result by the dialog scene determination unit, and calculating an index value of the user's emotion based on the feature amount It is a voice interactive apparatus having an emotion estimation unit that estimates the user's emotion according to a comparison result between the index value and the threshold set by the emotion threshold setting unit.
According to this voice interactive apparatus, it is determined whether the user is interacting as a listener or a speaker, and a threshold for emotion estimation is set according to the determination result. For this reason, according to this speech dialogue apparatus, it is possible to perform emotion estimation while suppressing the influence due to the difference in dialogue scene.

  ADVANTAGE OF THE INVENTION According to this invention, the audio | voice dialog apparatus which can perform the emotion estimation which suppressed the influence by the difference in a dialog scene can be provided.

FIG. 2 is a diagram illustrating a hardware configuration of the voice interaction apparatus according to the first embodiment. 1 is a block diagram showing a configuration of a control device of a voice interaction device according to a first exemplary embodiment; 3 is a flowchart illustrating an example of an operation of the voice interaction apparatus according to the first exemplary embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. Note that, in each drawing, the same element is denoted by the same reference numeral, and redundant description is omitted as necessary.

  FIG. 1 is a diagram illustrating a hardware configuration of the voice interactive apparatus 1 according to the first embodiment. The voice interactive apparatus 1 performs a conversation with a user using voice. Specifically, the voice interaction apparatus 1 interacts with the user by outputting a voice to the user in response to an utterance from the user. That is, the voice dialogue apparatus 1 recognizes the voice of the user who is the dialogue partner and outputs the voice to the user. The voice interactive apparatus 1 can be mounted on, for example, a robot such as a life support robot and a small robot, a cloud system, a smartphone, and the like.

  The voice interaction device 1 includes a microphone 2 that collects surrounding sounds, a speaker 3 that emits sound, and a control device 10. The control device 10 has a function as a computer, for example. The control device 10 is connected to the microphone 2 and the speaker 3 by wire or wirelessly. Note that the voice interactive apparatus 1 may further include a camera (not shown) that acquires surrounding images, and the control apparatus 10 may be further connected to the camera by wire or wirelessly.

  The control device 10 includes a CPU (Central Processing Unit) 12, a ROM (Read Only Memory) 14, and a RAM (Random Access Memory) 16 as main hardware configurations. The CPU 12 has a function as an arithmetic device that performs control processing, arithmetic processing, and the like. The ROM 14 has a function for storing a control program executed by the CPU 12, an arithmetic program, and the like. The RAM 16 has a function for temporarily storing processing data and the like.

  The control device 10 analyzes the user's utterance collected by the microphone 2 and generates a response to the user according to the user's utterance. And the control apparatus 10 outputs the audio | voice (response audio | voice) corresponding to the produced | generated response via the speaker 3. FIG.

  FIG. 2 is a block diagram of the configuration of the control device 10 of the voice interactive device 1 according to the first embodiment. The control device 10 includes a feature amount acquisition unit 101, a speech act type determination unit 102, a dialogue history storage unit 103, a dialogue scene determination unit 104, an emotion estimation unit 105, an emotion threshold setting unit 106, and a response generation unit. 107 and a speech synthesizer 108. Note that the feature quantity acquisition unit 101, the speech act type determination unit 102, the conversation scene determination unit 104, the emotion estimation unit 105, the emotion threshold setting unit 106, the response generation unit 107, and the speech synthesis unit 108 illustrated in FIG. This can be realized by the CPU 12 executing a program stored in the ROM 14. In addition, a necessary program may be recorded in an arbitrary nonvolatile recording medium and installed as necessary. Further, the dialogue history storage unit 103 is realized by a storage device such as the ROM 14, for example. Each component is not limited to being realized by software as described above, and may be realized by hardware such as some circuit element.

  The feature amount acquisition unit 101 acquires a feature amount used for emotion estimation from a user who is a conversation partner. In the present embodiment, the feature amount acquisition unit 101 generates text data as a feature amount by performing speech recognition processing on the user's speech data acquired by the microphone 2. Therefore, the feature quantity acquisition unit 101 may be referred to as a voice recognition unit. In the present embodiment, the feature quantity acquired by the feature quantity acquisition unit 101, that is, text data, is used not only for emotion estimation but also for generating a response. Further, the feature amount acquisition unit 101 identifies the user's utterance time based on the voice recognition processing result, and stores the identified utterance time in the dialogue history storage unit 103 described later.

  The speech act type discriminating unit 102 discriminates the user's speech act type. The speech act type discriminating unit 102 discriminates, for example, five types of speech act types (question, response, greeting, information provision, and nickname). Here, the speech act type “question” is an utterance for asking a question to the conversation partner. Also, the speech act type “response” is a speech that responds to a question from the conversation partner. Further, the speech act type “greeting” is an utterance that greets the conversation partner. Also, the utterance action type “information provision” is an utterance that provides a spontaneous topic, such as explanation of an event or the like, provision of information about itself, and the like. Further, the speech act type “Aichichi” is an utterance that indicates the identity of the conversation partner. Note that various known methods can be applied as a specific determination method by the speech act type determination unit 102. For example, the speech act type discrimination unit 102 may perform discrimination using machine learning, but the discrimination method is not limited to this. Further, the speech act type determination unit 102 stores the determined speech act type in the dialogue history storage unit 103.

  The dialogue history storage unit 103 stores a dialogue history. In the present embodiment, the dialogue history storage unit 103 stores the dialogue history between the user and the voice dialogue apparatus 1 in the last N turns (where N is a predetermined positive integer). That is, the user's utterance action type and utterance time, and the utterance action type and utterance time of the voice interaction apparatus 1 in the last N turns are stored.

  Based on the conversation history between the user and the voice conversation apparatus 1 stored in the conversation history storage section 103, the conversation scene determination unit 104 currently has a conversation with either the user or the voice conversation apparatus 1 as the listener side, and which is the speaker side. Determine if you are interacting with. That is, the dialogue scene determination unit 104 determines the current dialogue scene using the utterance time or the utterance action type stored in the dialogue history storage unit 103.

  For example, the conversation scene determination unit 104 calculates the utterance ratio of both the latest N turns from the utterance time of the latest N turns user and the voice interaction device 1 stored in the conversation history storage unit 103, and the calculation result Based on the above, the conversation scene is determined as follows. Here, it is assumed that the conversation scene determination unit 104 calculates the utterance ratio of the voice interaction apparatus 1 with a value of 0 to 100%. That is, the utterance ratio of the user is expressed as 100% − (utterance ratio of the voice interactive apparatus 1).

(1) When the speech rate of the voice interaction apparatus 1 is not less than the threshold value T1 (for example, 60%) and not more than 100%, that is, the user's rate of speech is not less than 0% and not more than the threshold value T2 (for example, 40%):
The conversation scene determination unit 104 determines that the conversation scene is an information provision scene by the voice interaction device 1. That is, the conversation scene determination unit 104 determines that the current conversation scene is a scene in which the voice conversation apparatus 1 is interacting as the speaker side and the user is interacting as the listener side.
(2) When the utterance ratio of the voice interactive apparatus 1 is equal to or higher than the threshold T3 (for example, 40%) and lower than the threshold T1 (for example, 60%), that is, the user's utterance ratio is higher than the threshold T2 (for example, 40%). 60%) if:
The conversation scene determination unit 104 determines that the conversation scene is a chat scene between the voice interaction device 1 and the user. That is, the conversation scene determination unit 104 determines that the current conversation scene is a scene in which it is not possible to specify which is mainly talking as the speaker side and which is mainly talking as the listener side.
(3) When the utterance ratio of the voice interaction apparatus 1 is 0% or more and less than the threshold T3 (for example, 40%), that is, when the user's utterance ratio is higher than the threshold T4 (for example, 60%) and 100% or less:
The conversation scene determination unit 104 determines that the conversation scene is a listening scene by the voice interaction device 1. That is, the conversation scene determination unit 104 determines that the current conversation scene is a scene in which the voice conversation apparatus 1 has a conversation as a listener and a user is a conversation as a speaker.

  In other words, when such a determination is made, the conversation scene determination unit 104 determines that the speech conversation apparatus 1 is the speaker when the speech ratio by the speech conversation apparatus 1 during a predetermined number of conversations is greater than the user's speech ratio by a predetermined value or more. It is determined that the scene is a scene in which the user talks as the listener and the user talks as the listener. In addition, when the utterance ratio by the voice interaction apparatus 1 during a predetermined number of conversations is smaller than the user's utterance ratio by a predetermined value or more, the conversation scene determination unit 104 interacts as the listener side, and the user performs the conversation side. It is determined that the scene is a dialogue.

  The conversation scene determination unit 104 may determine a conversation scene based on the conversation history, and the determination method is an example. Therefore, for example, the conversation scene determination unit 104 may determine a conversation scene based on the utterance action types of both of the latest N turns. Specifically, for example, the dialogue scene determination unit 104 is a listening scene by the voice dialogue device 1 when the ratio of “information provision” is equal to or greater than a threshold in the user's utterance action type in the last N-turn dialogue. If the ratio of “Aichichi” is equal to or greater than the threshold value, it may be determined that the scene is an information provision scene by the voice interactive device 1. The dialog scene determination unit 104 may determine the dialog scene using both the utterance time and the utterance action type stored in the dialog history storage unit 103. For example, the conversation scene determination unit 104 may determine a conversation scene based on the utterance ratio between the voice interaction device 1 and the user for a predetermined utterance action type.

  The emotion estimation unit 105 calculates an index value of the user's emotion based on the feature amount acquired by the feature amount acquisition unit 101. Specifically, the emotion estimation unit 105 analyzes the text data generated by the feature amount acquisition unit 101, and calculates an index value indicating the user's emotion according to a predetermined calculation rule. Moreover, the emotion estimation part 105 estimates a user's emotion according to the comparison result of the calculated index value and a threshold value. Here, the threshold value is a value set by an emotion threshold setting unit 106 described later, and is a value corresponding to the conversation scene.

  The emotion estimation unit 105 may perform emotion estimation based on the calculation of the emotion index value and the comparison between the index value and the threshold value, and any known method can be applied as a method for such emotion estimation. is there. For example, as one of the emotion estimation methods, the technique described in “Emotion estimation based on the emotion-causing factor corpus acquired from the Web” (Ryoko Tokuhisa et al., Proc. Of the 14th Annual Conference of the Language Processing Society, March 2008) May be used.

  In the present embodiment, emotion estimation unit 105 calculates a numerical value within a range of −1.0 to +1.0 as an index value from an analysis result of text data indicating the user's utterance content. Here, when the analysis result indicates a negative emotion, the index value is a negative value, and when the analysis result indicates a positive emotion, the index value is a positive value.

  In the present embodiment, emotion estimation section 105 determines a positive, negative, or neutral emotion using the calculated index value and threshold value. Neutral is an emotion that is neither positive nor negative. For example, the threshold for estimating a positive emotion is +0.5, and the threshold for estimating a negative emotion is -0.5. In this case, the emotion estimation unit 105 determines that the user's emotion is negative when the index value calculated based on the feature amount acquired by the feature amount acquisition unit 101 is −0.5 or less. The emotion estimation unit 105 determines that the user's emotion is positive when the index value calculated based on the feature amount acquired by the feature amount acquisition unit 101 is +0.5 or more. Then, the emotion estimation unit 105 determines that the user's emotion is neutral when the index value calculated based on the feature amount acquired by the feature amount acquisition unit 101 is greater than −0.5 and less than +0.5. To do.

  The emotion threshold setting unit 106 sets a threshold for estimating the emotion according to the determination result by the conversation scene determination unit 104. For example, when the dialog scene determination unit 104 determines that the dialog scene is an information providing scene by the voice dialog device 1, that is, the current dialog scene is dialoged by the voice dialog device 1 as the speaker side, and the user When it is determined that the scene is a dialogue with the listener, it is considered that the emotion of the user is not easily reflected in the feature amount. For this reason, in this case, the emotion threshold setting unit 106 sets a threshold that is looser than a predetermined basic threshold so that a positive or negative emotion is easily estimated.

  On the other hand, when the dialog scene determination unit 104 determines that the dialog scene is a listening scene by the voice dialog device 1, that is, the current dialog scene is dialoged as the listener side by the voice dialog device 1, and the user If it is determined that the scene is a conversation with the speaker, the user's emotion is likely to be reflected in the feature amount. For this reason, in this case, the emotion threshold setting unit 106 sets a threshold that is harder than the basic threshold so that a positive or negative emotion is less likely to be estimated.

  Therefore, specifically, when the conversation scene determination unit 104 determines that the conversation scene is an information providing scene by the voice interaction apparatus 1, for example, the emotion threshold setting unit 106 estimates a positive emotion. Is lowered by 0.2 from its basic threshold of +0.5 to +0.3, and the threshold for estimating negative emotion is raised by 0.2 from its basic threshold of -0.5. , −0.3. In addition, when the conversation scene determination unit 104 determines that the conversation scene is a listening scene by the voice interaction device 1, for example, the emotion threshold setting unit 106 sets a threshold for estimating a positive emotion as the basic threshold. Raise the threshold by +0.2 from +0.5 to +0.7, and lower the threshold for estimating negative emotion by 0.2 from the basic threshold of −0.5 to −0.7. To do. Further, when the conversation scene determination unit 104 determines that the conversation scene is a chat scene between the voice interaction apparatus 1 and the user, for example, the emotion threshold setting unit 106 sets a threshold for estimating a positive emotion as The basic threshold value is set to +0.5, and the threshold value for estimating a negative emotion is set to the basic threshold value −0.5.

  The response generation unit 107 generates a response to the utterance of the user who is the conversation partner. The response is typically text data. In the present embodiment, the response generation unit 107 generates an appropriate response according to the emotion estimated by the emotion estimation unit 105. For example, the response generation unit 107 may generate a response sentence by selecting an appropriate response sentence from the response sentence table with reference to a response sentence table including a response sentence associated with the type of emotion. . Note that the response generation unit 107 stores the utterance action type and the utterance time for the generated response in the dialogue history storage unit 103. The utterance action type and the utterance time for the response generated by the response generation unit 107 may be stored in advance in a storage device such as the ROM 14 in association with the response sentence in the response sentence table, or the utterance action type determination unit. You may obtain by analysis by 102 grade | etc.,.

  The voice synthesis unit 108 converts the response generated by the response generation unit 107 into voice data. That is, the speech synthesizer 108 converts the text data of the response sentence generated by the response generator 107 into speech data. Generation of voice data from text data can be realized by various known voice synthesis techniques. Thereafter, typically, a D / A converter (not shown) converts the audio data into an analog audio signal, and the speaker 3 outputs the analog audio signal as audio.

  Next, the operation of the voice interaction apparatus 1 will be described. FIG. 3 is a flowchart showing an example of the operation of the voice interaction apparatus 1. Hereinafter, an operation example of the voice interactive apparatus 1 will be described with reference to FIG.

  In step 100 (S100), the feature quantity acquisition unit 101 performs voice recognition processing on the user voice data acquired by the microphone 2 to generate text data. For example, in step 100, text data “It is sunny today” is generated from the user's utterance. Also, the feature amount acquisition unit 101 specifies the user's utterance time, and stores the specified utterance time in the dialogue history storage unit 103.

  Next, in step 101 (S101), the speech act type discriminating unit 102 discriminates the speech act type for the user's voice acquired in step 100. For example, the utterance action type determination unit 102 determines that the utterance action type of the utterance “It is sunny today” is “information provision”, and stores the determined utterance action type in the conversation history storage unit 103. .

  Next, in step 102 (S102), the dialog scene determination unit 104 determines the current dialog scene based on the dialog history of the latest N turns. For example, the conversation scene determination unit 104 determines that the current conversation scene is a listening scene by the voice interaction apparatus 1.

  Next, in step 103 (S103), the emotion threshold setting unit 106 sets a threshold corresponding to the conversation scene determined in step 102. For example, when the conversation scene is a listening scene, the emotion threshold setting unit 106 sets a threshold for estimating a positive emotion as +0.7, which is 0.2 larger than the basic threshold, and estimates a negative emotion. The threshold for this is −0.7, which is smaller by 0.2 than the basic threshold.

  Next, in step 104 (S104), the emotion estimation unit 105 calculates an index value of the user's emotion based on the text data obtained in step 100, and the calculated index value and the threshold value set in step 103 are calculated. The user's emotion is estimated according to the comparison result. For example, when the index value of the user's emotion is +0.6, the index value is larger than the basic threshold value +0.5, but is less than +0.7 which is the threshold value corrected in Step 103. The emotion is not determined as positive but is determined as neutral.

  Next, in step 105 (S105), the response generation unit 107 generates a response according to the emotion estimated in step 104. For example, if the emotion estimated in step 104 is neutral, a response “Yes” is generated in step 105.

  Next, in step 106 (S106), the speech synthesizer 108 converts the response text data generated in step 105 into speech data. Thereby, the response sound generated in Step 105 is output from the speaker 3. Thereafter, the process returns to step 100 and the dialogue is repeated.

  Heretofore, the voice interactive apparatus 1 according to the embodiment has been described. As described above, the voice interaction apparatus 1 determines a conversation scene, and dynamically changes the threshold according to the determination result. Therefore, according to the voice interaction device 1, it is possible to perform emotion estimation while suppressing the influence due to the difference in the conversation scene. That is, for example, even in a scene where the user's emotion is difficult to be reflected in the feature amount and the user is mainly a listener, the emotion can be appropriately estimated. As described above, since the response can be generated after accurately capturing the emotion, the voice interaction apparatus 1 can communicate with the user more smoothly.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention. For example, in the above embodiment, the content of the user's utterance is used as the feature amount. However, the present invention is not limited to this, and based on other feature amounts such as the user's facial expression and prosody, the calculation of the index value and the emotional amount are performed. An estimation may be made. That is, as another example, the feature amount acquisition unit 101 may perform image processing on an image acquired by a camera (not shown) to acquire a user's facial expression as a feature amount. Moreover, each value regarding an index value and a threshold value is an example, and is not limited to the above values.

DESCRIPTION OF SYMBOLS 1 Voice conversation apparatus 101 Feature-value acquisition part 102 Speech action type discrimination | determination part 103 Dialog history memory | storage part 104 Dialogue scene determination part 105 Emotion estimation part 106 Emotion threshold value setting part 107 Response generation part 108 Speech synthesizer

Claims (1)

A speech dialogue apparatus for recognizing a speech voice of a user who is a conversation partner and outputting a voice to the user,
A feature amount acquisition unit for acquiring a feature amount used for emotion estimation from the user;
Based on the interaction history of the user and the own device, an interaction scene determination unit that determines which of the user and the own device interacts as a listener side and which is interacting as a speaker side;
An emotion threshold setting unit for setting a threshold for estimating emotion according to a determination result by the dialog scene determination unit;
Emotion estimation that calculates an index value of the user's emotion based on the feature amount, and estimates the user's emotion according to a comparison result between the calculated index value and a threshold set by the emotion threshold setting unit And a voice interaction device.

Images