JP2006030282A - Interaction understanding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interaction understanding device which has practical recognition capability and also has language understanding capability applicable to a speech interaction system capable of acquiring information that a user requires in a short period of time by greatly reducing influence of background noise of, especially, utterance. <P>SOLUTION: Context information of a conversation is also taken into consideration to select more likely words and when integrated arithmetic operation for language recognition is performed, the operation is carried out by using numeric parameters corresponding to the kind of noise to greatly reduce the influence of the background noise, thereby performing the language recognition with high precision. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to device control by a voice dialogue system, and in particular, a technique for improving recognition accuracy in a dialogue understanding device that requires control without bothering an operator's "hand" or "eye". About.

In conventional voice dialogue systems, the sound source is a naturally generated voice, and in vehicles, etc., there is an influence of noise during driving, etc., so it is not possible to correctly understand the user's utterance. In some cases, the response was different from the person's intention. As a result, the dialogue between the system and the user does not proceed smoothly, which may cause discomfort to the user. As countermeasures, various studies have been made to improve the recognition accuracy of speech recognition, and various research results have been reported (see, for example, Non-Patent Document 1 and Non-Patent Document 2). .
Kai, Ishimaru, Ito, Konishi, Ito, “Characteristic analysis of corrected utterance in destination setting task and application to detection”, Acoustical Society of Japan Annual Conference Proceedings 2-1-8, 2001, pp. 63-64 Komatani, Kawahara, “How to use the reliability of speech recognition results in a spoken dialogue system”, Proc. 73-74

  The non-patent document 1 is a study on misrecognition in speech recognition, and the non-patent document 2 is a study on dialogue control using reliability in the speech recognition result, and a method adopted in these studies. Are basically based on the recognition of input speech signals acoustically in units of words, and do not perform language recognition including contextual information that is being performed by humans. For this reason, there is a limit to reducing the influence of the background noise and the like on the speaking condition of the speaker and on both the transmitting side and the receiving side.

  The present invention has a recognition ability that can be practically used and exceeds the above-mentioned performance limitations, and can be applied to a speech dialogue system that can acquire information required by a user in a short time. An object of the present invention is to provide a dialogue understanding device having

  As one of the methods for achieving the above object, a method of interactively understanding speech information from a context flow in the utterance content of the speaker can be considered. According to this method, it is expected that a more accurate recognition result can be obtained as compared with the conventional method that simply focuses on improving the intelligibility or intelligibility of the speech. Therefore, in the present invention, it is fundamental to perform language understanding and response generation by combining the reliability of speech recognition with speech information processing using context information. In other words, not only using the reliability of conventional speech recognition, but also using the results learned using the type of utterance and information of the conversation history (recognition history), more probable language understanding interactively Was made to run. Further, while performing this language understanding, the influence of background noise and the like can be reduced more appropriately by performing calculations using parameters corresponding to the background noise of the utterance.

  The dialogue understanding device according to the present invention employs the above-described method to improve the recognition accuracy. That is, the dialog understanding device according to claim 1 of the present invention classifies the utterances included in the dialog into a plurality of categories and classes formed by subdividing the categories in the order of the size of the utterances. , Calculate a class score that gives the certainty of which class of words was uttered and a word score that gives the certainty of the words included in the utterance. A dialogue understanding apparatus for understanding a dialogue content by integrating a plurality of class scores and a plurality of word scores calculated for each utterance, wherein the parameters used in the integration calculation include utterance data mixed with a plurality of types of noise. It is characterized by the fact that the dialogue content is understood using parameters corresponding to the type of detected noise.

  According to a second aspect of the present invention, there is provided a dialog understanding device according to the first aspect, in which the voice input means including a microphone and a voice amplifier is digitally output. Using speech recognition means for performing speech recognition, reliability generation means for calculating reliability of results recognized by the speech recognition means, and results obtained by the speech recognition means and the reliability generation means A class score generator for classifying the plurality of categories set in advance and a hierarchical structure composed of the classes into which the categories are subdivided, and determining the likelihood of the utterances classified into the classes, and the results obtained thereby A category understanding unit for obtaining each category from the above, a word score generating unit for obtaining the probability of the recognized word, and generating an understanding content as a result processed by each processing unit. Language understanding means comprising an understanding content generation unit; storage means for storing past recognition history used for executing processing in the language understanding means; and response information from results obtained from the language understanding means. A response generation means for creating and an output means for outputting the response information, and when there is a new utterance, the language understanding means performs a parameter operation on a class score and a word score of a past recognition history. A new class score and a word score are generated using the value obtained in this way and the new reliability based on the latest recognition result, the recognition history is updated with the new class score and the word score, and the new class score is updated. The content of the dialogue is understood using the category understanding obtained from the above and a new word score, and the parameter is an utterance data in which plural kinds of noises are mixed. Are statistically estimated by motor, it is characterized by generating a new class score and the word score using the parameters corresponding to the type of the detected noise.

  Further, in the dialogue understanding device according to claim 3 of the present invention, in the dialogue understanding device according to claim 1 or 2, the parameter is statistically estimated from utterance data classified for each noise level, The parameter is changed based on the noise level during speech detected by the noise detection means.

  A dialogue understanding device according to claim 4 of the present invention is the dialogue understanding device according to claim 1 or 2, wherein the parameter is set for each traveling speed of the vehicle. The parameter is statistically estimated from the classified speech data, and the parameter is changed based on the speed information detected by the traveling speed detecting means.

  A dialogue understanding device according to claim 5 of the present invention is the dialogue understanding device according to claim 1 or 2, wherein the parameter is set for each traveling route of the vehicle. The parameter is statistically estimated from the classified speech data, and the parameter is changed based on the route information detected by the traveling route detection means.

  According to the present invention, not only speech recognition of words but also a method of further classifying recognized words into categories and classes and selecting more likely words in consideration of the relationship with the context. Therefore, the recognition accuracy can be further improved efficiently. In addition, since the calculation is performed using parameters corresponding to the background noise of the utterance so as to understand the content of the dialogue, for example, when used in a noisy environment such as voice input in a vehicle navigation system Even so, good recognition accuracy can be obtained.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 shows a basic configuration of a dialogue understanding device according to a first embodiment to which the present invention is applied. In the dialogue understanding device shown in FIG. 1, an analog voice input signal input by a user is converted into a digital signal by the voice input unit 101. Here, the voice input unit 101 includes a microphone, an input amplifier, and an A / D converter. This digitized voice signal is input to the voice recognition unit 102. The speech recognition unit 102 performs matching processing between the speech signal input from the user and the recognition target sentence stored in the speech signal recognition unit 102, and a plurality of recognition result candidate sentences and their likelihoods (details). Is output later). The reliability generation unit 103 uses the output information, and from the plurality of recognition result candidate sentences input along with a single utterance from the user, the words included in the recognition result candidate sentence, and these words The reliability indicating the likelihood of the class indicating the classification of is output.

  Here, the class is defined as shown in FIG. 2 according to the expression format of the utterance. That is, for example, when the utterance is an expression format indicating the destination, it is classified hierarchically as shown in FIG. 2, and categories are arranged in order from the widest range to the narrowest range and included in each category. A word is a class that is classified by content. For example, in the example of FIG. 2, each word is classified into three categories of upper (PR), middle (HR), and lower (LM), and further classified into a plurality of classes in each category. For example, in the case of FIG. 2, the upper category has only one class of “prefecture”, but the lower category has three classes of “interchange”, “city”, and “station”.

  The reliability in units of words is obtained as follows. That is, first, word strings arranged in descending order of likelihood from the first place to the N-th place of candidate word strings (for example, sentences formed of a plurality of words) obtained from word recognition results (hereinafter referred to as words). The column is referred to as an N-best candidate.) And the log likelihood for each word is obtained. Here, the likelihood is a value defined by the posterior probability that the speech signal sequence uttered by the user is W when the speech signal sequence obtained from the recognition result is Y. On the other hand, it is the maximum probability among the ratios of the product of the “prior probability that the speech signal sequence Y is observed” and the “probability that the speech signal sequence W is uttered” and the probability that the speech signal sequence Y is observed.

Thereby, the reliability Conf (w) of the word w included in the first candidate is obtained from the following equation (1).

The probability pi that the word w is included in the i-th candidate among the N-best candidates in the formula (1) is obtained from the following formula (2). Here, Li is the log likelihood for each N-best candidate.

Further, as in the case of the above word unit, the reliability in class units can be obtained from the following equation (3) by using the class Cw of each word w included in the first candidate. .

Here, as in the case of the above-mentioned word unit, pi can be obtained from the following equation (4).

  The recognition data (recognition result candidate sentence, likelihood and reliability) obtained as described above is input to the language understanding unit 104.

  In addition, the dialogue understanding device according to the present embodiment detects noise type and level (noise level) mixed in a section other than a user's utterance from an analog voice input signal input to the voice input unit 101. Unit 113, and the output of the noise detection unit 113 is also input to the language understanding unit 104.

  The language understanding unit 104 includes a class score generation unit 105, a category understanding unit 106, a word score generation unit 107, and an understanding content generation unit 108. The language understanding unit 104 is input with a plurality of utterances from the user. It has a function of generating an understanding result from a word and the reliability of the class to which the word belongs.

  Here, the class score generation unit 105 calculates a score indicating which class has been uttered from the class reliability of the word input in association with a plurality of utterances from the user, and the category understanding unit 106. Is to output an understanding result of a category indicating the classification of the class from the class score inputted with a plurality of utterances from the user, that is, which category is uttered. Further, the word score generation unit 107 calculates a score indicating which word is uttered from the reliability of the word input with a plurality of utterances from the user, and the understanding content generation unit 108 2 has a function of generating an understanding content from the category understanding result (output of the category understanding unit 106) and the word score (output of the word score generating unit 107) obtained in the above.

  In the process of generating the understanding content in the language understanding unit 104, parameter calculation using various numerical parameters is performed as will be described in detail later. In the dialog understanding device of the present embodiment, the numerical parameter used for this calculation is It is changed according to the type and degree of noise detected by the noise detection unit 113. That is, the dialogue understanding device of the present embodiment includes a parameter set holding unit 114 that holds a plurality of numerical parameters classified according to the type and degree of noise, and the numerical parameters held in the parameter set holding unit 114. Among these, numerical parameters corresponding to the type and degree of noise are read out and used in the calculation. The plurality of numerical parameters held in the parameter set holding unit 114 are collected according to the type and degree of noise by collecting a large number of utterance data mixed with a plurality of types of noise in advance and statistically processing them. It is estimated as a parameter.

  The output information of the language understanding unit 104 obtained as described above is input to the response generation unit 109. The response generation unit 109 generates a response sentence from the understanding content obtained by the language understanding unit 104. The response sentence generated by the response generator 109 is synthesized as a digital signal by the voice synthesizer 110, and is output as a voice output through a D / A converter and an output amplifier (not shown) built in the voice synthesizer 110. The output response sentence generated by the response generation unit 109 is sent to a display device (not shown) via the GUI display unit 111 and displayed as character information on the display device. The recognition history 112 is a storage device such as a hard disk storage device that stores past recognition status as history data.

  Next, the operation of the dialogue understanding device of the present embodiment configured as described above will be specifically described by taking as an example the case where the destination of the vehicle is set by dialogue with the user.

  First, the destination expression format handled by the dialogue understanding device of the present embodiment will be described. Here, interchanges, stations, and city names can be set as destinations, and prefectures, expressways, and railway lines can be added to each destination. As described above, FIG. 2 shows these representation formats in a hierarchical structure. That is, in the present embodiment, the destination can be uttered by a combination of partial utterances of upper, middle, and lower three stages, and in the present embodiment, each of the three stages is called a category. In the upper category PR, you can utter prefectures (prefectures), in the middle category HR you can utter automobile roads or railway lines, and in the lower category LM you can speak interchanges, municipalities, and stations. Can do.

  In the dialogue understanding device of this embodiment, the destination setting in the dialogue format can be performed by more flexible speech. That is, the user can speak all categories at once, for example, “Hamamatsu Nishi Interchange on Tomei Expressway in Shizuoka Prefecture”. It is also possible to speak in multiple sessions, such as “Shizuoka Prefecture” in the first utterance and “Hamamatsu Nishi Interchange at Tomei Expressway” in the second utterance.

  Further, when the user utters a plurality of times, a detailed utterance in which more detailed information is added to the past utterance is enabled. For example, it is possible to speak “Shizuoka Prefecture” in the first utterance and “Hamamatsu City” in the second utterance. Further, when the user utters a plurality of times, the utterance for correcting the response result of the dialogue understanding device is enabled. For example, when the first response “Hamamatsu City in Shizuoka Prefecture” is answered incorrectly as the first response “Is Hamamatsu Nishi Interchange in Shizuoka Prefecture”, the second utterance is “No Hamamatsu City” Can be spoken. "

In addition, when the user utters a plurality of times, when the response from the dialogue understanding device is a question, it is also possible to utter the answer. For example, when the first response is “How many interchanges in Shizuoka Prefecture”, it is possible to say “This is Hamamatsu Nishi Interchange” in the second utterance.
In addition, when the user utters a plurality of times, when the response from the dialogue understanding device is an utterance prompting re-input, the utterance can be answered. For example, when the first response is “Please speak again”, it is possible to perform the same utterance as the first utterance in the second utterance.
The recognition target words in this embodiment are as exemplified in FIG. Moreover, the example of a conversation in this embodiment is as shown in FIG. 4, for example. In FIG. 4, U is the user's utterance, S is the response from the dialogue understanding device, and the numbers are the utterance order.

  Here, the operation of the dialogue understanding device of the present embodiment will be described with reference to a specific example using the flowchart of FIG.

  First, when the process is started in step S301, it is determined whether or not the user has operated an unillustrated voice input switch (speech switch) in order to instruct the start of utterance. In step S302, the voice input switch is When it is detected that the signal is in the on state, the process proceeds to the step of starting the capturing of the audio signal / noise (step S303). Here, if an operation to turn on the voice input switch is not detected, the process waits in step S302 until this operation is detected.

  In step S303, an utterance (for example, the word illustrated in FIG. 3) included in the sentence to be recognized is performed from the user, and the voice input unit 101 receives the utterance from the user and receives the signal from the microphone as A. The digital signal is converted by the / D converter and output to the voice recognition unit 102. At the same time, the noise detection unit 113 detects the type and degree of noise mixed in a section other than the user's utterance and outputs it to the language understanding unit 104. The voice recognition unit 102 continues to calculate the average power of the digital signal until the utterance switch is operated. After the utterance switch is operated, the instantaneous power of the digital signal is larger than the average power. When it becomes larger than a predetermined value, it is determined that the user speaks, and the capturing of the audio signal is started.

  Then, the voice recognition unit 102 sets a plurality of candidates by comparing the stored recognition target sentence with the digitized voice signal and calculating the likelihood (step S304). Note that while executing step S304, the audio signal capturing in step S303 is continued by parallel processing.

  Thereafter, at the stage where the instantaneous power of the digitized audio signal continues for a predetermined time or more and a predetermined value or less, the dialogue understanding device determines that the user's utterance has ended and ends the audio signal input processing. (Step S305). Thereby, the speech recognition unit 102 outputs the top N candidates obtained by arranging a plurality of recognition result candidate sentences in order of likelihood together with the likelihood data. FIG. 6 shows an example of an output result by the voice recognition unit 102. In FIG. 6, the part marked XXX indicates the likelihood calculated for each word.

  Next, based on the recognition result made up of a plurality of candidates ranked by the acoustic likelihood called the N-Best candidate, the acoustic likelihood and N -From the appearance frequency in the best candidate, the reliability is calculated as a measure based on the post-establishment (step S306). This calculation is executed in the reliability generation unit 103, and an example of the calculation result is shown in FIG. In FIG. 7, the left table is the output of the speech recognition unit shown in FIG. 6, the word reliability in the right table indicates the possibility that a certain word has been uttered, and the class reliability is a word of a certain class. Indicates the possibility of being uttered. Regarding this calculation, Non-Patent Document 2 (Komatani, Kawahara, “Usage of reliability of speech recognition results in a spoken dialogue system”, Proceedings of the Acoustical Society of Japan National Conference 3-5-2, 2000, pp. 73-74).

  As described above, the reliability of the spoken word is obtained to estimate the probable word. In the dialogue understanding device of the present embodiment, the accuracy of the word estimation is further improved by the dialogue with the user. It is improving. For this reason, in the next step S307, the class score is calculated by the class score generation unit 105. Prior to this class score calculation, the user's utterance type is determined.

  That is, the first utterance type is an utterance type that serves to add new information to the previous information. For example, details and response processing correspond to this. The second utterance type has a function of correcting previous information. For example, correction and re-input processing correspond to this. Whether or not the utterance type is selected is determined based on the situation described in the determination material column, as shown in FIG. 8, for example. There are also other determination methods. For example, there is a method of detecting the occurrence of partial rephrasing often used in place name input using the word spotting method by DP matching. Regarding this, Kakutani, Kitaoka, Nakagawa, “ Analysis and detection of corrected utterances ", Information Processing Society of Japan Research Report, Spoken Language Information Processing 37-11, 2001.

  After the utterance type is determined as described above, a class score is generated by the class score generation unit 105 in step S307. Here, the class score is a value indicating the likelihood of the class during the conversation, that is, during the user's multiple utterances. In this case, a score can be generated more appropriately by adding new information while leaving previously understood information. The class score is generated using a different generation formula for each utterance type. Therefore, in step S307 in FIG. 5, a process divided into two as shown in FIG. 9 is performed. That is, in step S315, it is determined whether or not it corresponds to the utterance type of the answer according to the situation described in the column of determination material in FIG. 8, and if so, it is processed in step S316. In the case of the input utterance type, after processing in step S317, in any case, the processing proceeds to step S308.

  In the case of the refinement and answer utterance type, that is, in the case of step S316 in FIG. 9, the class score is obtained by the following equation (5).

Score (c) = Score (c) x weight s, n + Conf (c) (Equation 5)
However, Score is a class score, the left side of the formula (5) is a newly obtained class score, and the right side of the formula (5) is a process for a past class score (read from the recognition history 112). is there. Conf is the class reliability obtained from the latest recognition result. Further, weight s, n is a weighting parameter that takes a value of 0.0 to 1.0. C is a class for generating a score. Here, the weight of the past class score is weighted using the parameter weight s, n, and the class score is lowered at a certain rate. The policy is that “the reliability decreases as information becomes older”. This is because it is applied. In the case of the refinement and answer utterance type, the past class score read from the recognition history 112 is updated with the class score newly obtained by the equation (5) and written to the recognition history 112. .

  The parameter weight s, n for weighting the past class score can be obtained experimentally using speech data mixed with noise. The feature of the present embodiment is that the language understanding unit 104 The numerical parameter corresponding to the actually detected noise type and degree is read out from the parameter set holding unit 114 and used from among a plurality of numerical parameters classified for each noise type and degree. Here, n indicates the type and degree of the mixed noise. For example, the weight is experimentally determined from the speech data classified for each noise level [n = level 0, level 1, level 2, level 3 ...]. When s, n is obtained, a plurality of weights s, n corresponding to the noise level are obtained. In the dialogue understanding device according to the present embodiment, when a new class score is obtained by the equation (5), weight s, n corresponding to the noise level actually detected by the noise detection unit 113 is selected and used. I have to.

  FIG. 10 shows how the class score is generated for the refinement / response utterance type. In this example, the user utters, for example, “JR Hamamatsu Station in Shizuoka Prefecture” in the past utterance, the old class score of “prefecture” and “railway line” is 1.00, and the old class score of “station” is 0.84. It has become. Next, in response to the question “How many stations of JR in Shizuoka Prefecture?” From the dialogue understanding device, the user replied, for example, “Hamamatsu Station” as the latest utterance, and the new class trust of “Station” The degree is 0.81. As a result, a new class score is generated based on the formula (5), a new class score for “prefecture” is 0.90, a new class score for “railway” is 0.90, and a new class for “station” The score is 1.65, and the old class score is updated with these class scores and written in the recognition history 112.

  In the case of the utterance type of correction and re-input, that is, in the case of step S317 in FIG. 9, the class score is obtained by the following (Equation 6).

Score (ca) = Score (ca) × weight t, n−Conf (cb) + Conf (ca) (Formula 6)
However, Score is a class score, the left side of the formula (6) is a newly obtained class score, and the right side of the formula (6) is a past class score (read from the recognition history 112). Conf is the class reliability obtained from the latest recognition result. Further, weight t, n is a weighting parameter that takes a value of 0.0 to 1.0. Also, ca is a class that generates a score, and cb is all different classes in the same category as ca. This equation (6) differs from the equation (5) in that the reliability of the same category and different class is subtracted. This makes it easier to correct the score when the class is wrong. In the case of the refinement / answer utterance type, the past class score read from the recognition history 112 is updated with the class score newly obtained by the equation (6) and written to the recognition history 112.

  The parameter weight t, n for weighting the past class score can be obtained experimentally using speech data mixed with noise. The feature of the present embodiment is that the language understanding unit 104 Among the plurality of numerical parameters classified for each noise type and degree, the numerical parameter corresponding to the actually detected noise type and degree is read from the parameter set holding unit 114 and used. In the dialogue understanding device according to the embodiment, when the new class score is obtained by the equation (6), the noise detection unit 113 actually detects the new class score similarly to the weight s, n used by the equation (5). The weight t, n corresponding to the noise level is selected and used.

  FIG. 11 shows how the correction / re-input utterance type class score is generated. In this example, the user utters, for example, “Shizuoka Prefecture, Hamamatsu Station” in the past utterance, and the old class score of “prefecture” is 0.39 and the old class score of “station” is 0.63. For this reason, the class score value is insufficient and the category cannot be specified, and the dialogue understanding device outputs a response “Please speak again”. In response to this response, the user then speaks again with the same "Shizuoka Prefecture, Hamamatsu Station", obtaining a new class reliability of 0.54 for "prefecture" and a new class reliability of 0.52 for "station". Is obtained. As a result, a new class score is generated based on the above equation (6), the new class score of “prefecture” is 0.89, the new class score of “station” is 0.86, and the old class score is these classes. It is updated with the score and written in the recognition history 112.

  Subsequently, the process proceeds to step S308 of the category understanding process. This process is performed by the category understanding unit 106 for both the past class score (read from the recognition history 112) and the class reliability in the latest recognition result. This is done by calculating a score. The state of this processing is shown in FIG. The category score is obtained by adding all the class scores or reliability belonging to the same category, as can be seen from the numbers in the respective columns in the part indicated by a and the part indicated by B in FIG. Each category score is determined by a threshold value, and the logical sum of the determination results is calculated for the three categories of PR (upper), HR (middle), and LM (lower). The results obtained there show combinations of categories spoken so far. When the class score is obtained as shown in FIG. 12, the subsequent category understanding is shown in FIG. That is, each category is determined from the old and new scores, and as a result, category understanding is obtained.

  Next, a word score is generated in step S309. The processing in step S309 is executed by the word score generation unit 107, 1) a past word (which already exists in the recognition history 112), and 2) a new one. A score is generated for each of the two words (words in the latest recognition result) appearing in, using different policies. In the case of the latter 2), all words included in the N-Best candidate of the latest recognition result are targeted. The word score is generated in the order of 1) → 2) every time the language understanding unit 104 acquires the latest recognition rate.

  For words that exist in the recognition history of 1) above, the existing word score is increased or decreased from the newness of the word, the response content of the dialogue understanding device and the user's utterance type (detailed, corrected, answered, re-input). To generate a new word score. For this, the following five policies are adopted.

  Policy 1: Under the assumption that the old information has low reliability, every time a new recognition result is input, the scores of all the words existing in the recognition history 112 are lowered.

  Policy 2: When the word A in the recognition history 112 and the word B newly obtained as a recognition result are in a detailed relationship, the score of the word A is increased.

  Policy 3: When the word A in the recognition history 112 and the word B in the recognition history 112 are in a correction relationship, the score of the word A is lowered.

  Policy 4: If the recognition result includes affirmation (yes, yes, etc.), increase the score of the word included in the response.

Policy 5: If the recognition result includes a negative result (No, wrong, etc.), the score of the word included in the response is lowered.
Generation of the word score in the recognition history 112 is based on the following equation (7).

Score (Wd) = Score (Wd) −p1n + p2n × Conf (Ws) −p3n × Conf (Wt) + i × (p4n × Conf (yes) −p5n × Conf (no) −p6n × Conf (rej)) )
However, Score is the score of the word in the recognition history 112, the right side is before update, and the left side is after update. Wd is a word in the recognition history 112 to be calculated. As a term corresponding to the policy 1, there is a term (first term on the left side) that lowers the word score using the parameter p1n. The next two terms on the left are the terms corresponding to policy 2 and policy 3, p2n and p3n are parameters for weighting, Conf is the reliability obtained from the latest recognition result, and Ws is the latest recognition result. Are all the words that are in a detailed relationship with Wd, and Wt is all the words that are included in the latest recognition result and that have a correction relationship with Wd. The next term on the left side is a term corresponding to policy 4 and policy 5, p4n, p5n, and p6n are parameters, and i is i = if a word is included in the response from the previous dialog understanding device. 1 and i = 0 if not included. Further, (yes) indicates an affirmative word included in the latest recognition result, (no) indicates after negation included in the current recognition result, and (rej) indicates a sentence end negative word included in the current recognition result.

  The parameters p1n to p6n used when generating the word score in the above equation (7) can be experimentally obtained using speech data mixed with noise. The understanding unit 104 reads out from the parameter set holding unit 114 and uses the numerical parameter corresponding to the type and degree of the actually detected noise from a plurality of numerical parameters classified for each noise type and degree. In the dialogue understanding device according to the present embodiment, when the word score is obtained by the equation (7), the weight s, n used in the equation (5) or the weight t, used in the equation (6) is used. Similarly to n, p1n to p6n corresponding to the noise level actually detected by the noise detection unit 113 are selected and used.

  For the words in the latest recognition result in the above 2) that are not yet registered in the recognition history 112, that is, newly appearing words, the response content and the user utterance type (detailed, corrected, answered, replayed) Input), N-Best ranking, and utterance length (the number of spoken words), the word score is generated by raising and lowering the reliability of speech recognition. For this, the following four policies are adopted.

  Policy 6: When the word A of the recognition result and the word B included in the response are in a refinement relationship, the score of the word A is increased.

  Policy 7: When the response of the dialogue understanding device is a question (for example, how many interchanges?) And the content of the recognition result is an answer, the score of the word of the recognition result is increased.

  Policy 8: Since many correct words are included at the top of the recognition result, the score of the words included at the top is increased.

  Policy 9: Since an utterance with a long utterance length (short utterance) is easily recognized (not easily recognized), the result of one category lowers the score of the word, and the score of two or more categories raises the score.

  Generation of a score of a word in the latest recognition result that has not yet been registered in the recognition history 112 is based on the following equation (8).

Score (Wd) = Conf (Wd) + p7n × Score (Ws) + p8n × Conf (Wa) + Conf (Wd) × (p9n + p10n × len2-p11n × len1) (Equation 8) Where, Score is the word in the recognition history It is a score, and Conf is the reliability obtained from the latest recognition result. Wd is a word in the recognition result to be calculated. As a term corresponding to the policy 6, there is a term (first term on the left side) that increases the score of the word using the parameter p7n, and Ws is all words that have a detailed relationship with Wd included in the recognition result. The next term on the left side is a term corresponding to policy 7, p8n is a parameter for weighting, and Wa is a word included in the recognition result when the recognition result is an answer to the question. The next term on the left side is a term corresponding to the policies 8 and 9, and p9n is a parameter for weighting corresponding to the policy 8 and the height of the N-Best ranking. P10n and p11n are parameters for weighting corresponding to the policy 9, len2 is set to len2 = 1 when the recognition category is 2 or more, and len1 is set to len1 = 1 when the recognition category is 1. Is the value.

  The parameters p7n to p11n used when generating the word score by the above equation (8) can be experimentally obtained using utterance data mixed with noise. The understanding unit 104 reads out from the parameter set holding unit 114 and uses the numerical parameter corresponding to the type and degree of the actually detected noise from a plurality of numerical parameters classified for each noise type and degree. In the dialogue understanding device according to the present embodiment, when the word score is obtained by the equation (8), weight s, n used by the equation (5) or weight t, used by the equation (6). n, similarly to p1n to p6n used in the equation (7), p7n to p11n corresponding to the noise level actually detected by the noise detector 113 are selected and used.

  The score of the word updated in 1), the word added in 2), and the score are written in the recognition history 112 as an integrated recognition history. An example of the integrated recognition result is shown in FIG. 14 using actual prefecture names, railway names, and the like as actual examples. There are cases where there are a plurality of the same names in the figure (Atsugi, Tanashi, etc.), which are the names of stations included in the plurality of routes.

  A plurality of candidates are generated as an appropriate combination from the category understanding result obtained as described above and the integrated recognition result. That is, a plurality of likely candidates are generated as contents understood by the present apparatus based on the information obtained as described above (step S310). The processing in step S310 is executed in the understanding content generation unit 108. Specifically, the understanding content generation unit 108 recognizes that, for example, three categories of PR, HR, and LM are uttered from the result shown in FIG. 13, and from the recognition result illustrated in FIG. A combination that actually exists is extracted as a candidate. Then, the one with the largest sum of scores of each category is selected. The result is shown in FIG. In the example shown in FIG. 15, <PR category = Aichi, score = 1.47>, <HR category = Nagoya Railway, score = 1.17>, <LM category = Toyohashi, score = 0.62> are selected as the understanding results. .

  Heretofore, the processing process up to generation of the understanding content that is the result of processing in each step of the language understanding unit 104 has been described. The output information obtained in this way is input to the response generation unit 109, and in step S311, a response sentence is generated by the response generation unit 109 using a response flag corresponding to the understanding result. The types of response flags processed by the response generation unit 109 are shown in FIG. FIG. 17 shows the contents indicated by the bits (bits a to F) in FIG. From the understanding result, when a word corresponding to the category exists, a corresponding flag is set. In this case, for example, a flag of a value (number of bits) obtained by evaluating the score in four stages is set. That is, the score from the maximum to the minimum is set from evaluation 1 to evaluation 4, and the flags are set to 1000, 0100, 0010, and 0001.

  The response generation unit 109 makes a response according to the following policy in the dialog using the response flag.

・ Response policy 1: Approval (consideration)
When there is no lower category and the score evaluation of the upper category or the middle category is evaluation 1, a response for smoothly proceeding with the dialogue is performed.

<Example>User's utterance "Shizuoka Prefecture"
Response of the dialogue understanding device… Yes
-Response policy 2: Repetition When the score evaluation is evaluation 2 or when a negative word comes after the beginning of the user's utterance, a recitation is performed with the meaning of confirmation.

<Example>User's utterance "Shizuoka Prefecture"
Response of dialogue understanding device ... "Shizuoka"
Response policy 3: Final confirmation If the lower category is spoken and reliable (score evaluation is evaluation 1 or evaluation 2), final confirmation is performed.

<Example>User's utterance… “Ride from Hamamatsu Inter”
Response of dialogue understanding device… “Are you sure you want to set up Hamamatsu Inter”
Response policy 4: Destination setting If there is a lower category in the previous response and the affirmative utterance is reliable (score evaluation is evaluation 1 or evaluation 2), it is set as the destination.

<Example> Response of the dialogue understanding device ... "Are you sure you want to set up Hamamatsu Inter?"
User's utterance “Yes”
Response of dialogue understanding device… “Set as destination”
・ Response policy 5: Ask only unknown information Ask the user about only unknown information.

<Example>User's utterance: "It is Tomei Expressway in Shizuoka Prefecture" (when the score evaluation of "Tomei" is low)
Response of dialogue understanding device… “How many expressways in Shizuoka?”
Response policy 6: Do not respond to unconfident information If only one of the combinations of the higher category (PR) and middle category (HR) is unreliable (score rating is 4), respond only to those with higher scores To promote dialogue.

<Example>User's utterance ... "Tomei Expressway in Shizuoka Prefecture" (when score evaluation of "Shizuoka Prefecture" is low)
Response of dialogue understanding device ... "Tomei Expressway"
-Response policy 7: Ask also the upper category When there is little additional information of other information and the score evaluation is bad, the recognition rate is improved by listening to the upper category.

<Example>User's utterance: "Riding from Hamamatsu interchange" (when score evaluation of "Hamamatsu" is low)
Response of the dialogue understanding device ... "How many prefectures are you?"
Response policy 8: Encourage the next utterance If an affirmative utterance comes after the upper category and it is reliable (when the score evaluation is evaluation 1 or evaluation 2), the next utterance is prompted.

<Example> Response of the dialogue understanding device ... "Tomei Expressway"
User's utterance “Yes”
Response of dialogue understanding device… “Where is Tomei Expressway”
Response policy 9: Return another candidate When the negative utterance is reliable (when the score evaluation is evaluation 1 or evaluation 2), another candidate not used in the previous response is returned.

<Example> Response from the device for understanding the dialogue ... "Do you want to set up Hamamatsu Inter"
User's utterance “No”
Response of dialogue understanding device… “Do you want to set up Hamamatsu Nishi Inter”
Response policy 10: Repeating previous response When a positive utterance or negative utterance is unreliable (when the score evaluation is evaluation 4), the previous response is repeated.

<Example> Response of the dialogue understanding device ... "Are you sure you want to set up Hamamatsu Inter?"
User's utterance… “Yes” (when “Yes” score is low)
Response of dialogue understanding device… “Are you sure you want to set up Hamamatsu Inter”
Response policy 11: Listening back If all information is unreliable (score evaluation is 4), all information is returned.

<Example>User's utterance: "Shizuoka Prefecture" (when the score evaluation of "Shizuoka Prefecture" is low)
Dialogue understanding device response… “Please speak again”
In order to implement the above dialogue policy, the response generation unit 109 compares the response flag with the flag table of FIG. 16 and returns a response with a response pattern in which the flag first matches.

  Specifically, for example, when the response flag generated from the understanding result by the response generation unit 109 is “1 111000 1000 1000 0100 0000 0000 0000 0”, the response pattern as a result of referring to the flag table of FIG. As a result, the following pattern is selected.

“Are you sure you want to set“ PR category word ”,“ PR category class ”,“ HR category word ”,“ HR category class ”,“ LM category word ”,“ LM category class ”?
As a result, “Are you sure you want to set up Toyohashi Station on the Nagoya Railway in Aichi Prefecture?” Is generated as a response sentence.

  In step S312, the response sentence generated as described above is synthesized as a digital signal by the voice synthesizer 110, and is output as a voice output through a D / A converter and an output amplifier (not shown) built in the voice synthesizer 110. The The response text is sent to a display device (not shown) via the GUI display unit 111 and displayed as character information on the display device.

  At this stage, it is confirmed whether or not all the input processes have been completed (step S313). When all the input processes have been completed, the process proceeds to step S314 to end all the input processes. That is, it is determined whether or not the words of the lower category (LM) are confirmed. If not confirmed (in the case of no in step S313), the processing is continued, and the words of the lower category (LM) are confirmed. If yes (YES in step S313), the process proceeds to step S314, and all input processes are terminated. In this example, “Are you sure you want to set up Toyohashi Station on the Nagoya Railroad in Aichi Prefecture” is being answered, and then the user will say “Yes” to “Set as the destination. The process is terminated after the response “Yes” is made.

  As described above, according to the dialogue understanding device of the present embodiment, the utterances included in the dialogue are hierarchically divided into a plurality of categories and in a class configured by subdividing the categories in the order of the size of the utterances. When classifying the class score that gives the certainty of which class of words has been uttered and the word score giving the certainty of the words included in the utterance, and when there are multiple utterances in the dialogue Since the conversation contents are understood by integrating the plurality of class scores and the plurality of word scores calculated for each utterance, the voice information can be understood interactively from the context flow in the utterance contents. Therefore, a more accurate recognition result can be obtained as compared with the conventional method that simply focuses on improving the intelligibility or intelligibility of the speech.

  In addition, the parameters used in the integrated calculation are statistically estimated from utterance data mixed with a plurality of types of noise, and the conversation contents are understood using the parameters corresponding to the detected noise level. Therefore, it is possible to greatly reduce the influence of the background noise of the utterance, and to obtain good recognition accuracy even when used in a noisy environment such as voice input in a vehicle navigation system, for example. Can do.

(Second Embodiment)
Next, a dialogue understanding device according to a second embodiment to which the present invention is applied will be described. The dialogue understanding device according to the present embodiment is an example of a dialogue understanding device mounted on a vehicle. As shown in FIG. 18, the noise detection unit 113 in the dialogue understanding device according to the first embodiment described above is used. Instead, a travel speed detector 115 is provided. In addition, the parameters used in the integrated calculation for understanding the conversation contents are statistically estimated from the utterance data classified for each traveling speed of the vehicle, and based on the speed information detected by the traveling speed detecting unit 115. The above parameters are changed. Since the other configuration and the basic processing flow are the same as those in the first embodiment described above, the same description as in the first embodiment will be omitted below, and the present embodiment will be omitted. Only the characteristic part will be described.

  In the dialogue understanding device of this embodiment, a vehicle speed pulse detected by the vehicle wheel speed sensor is sent to the travel speed detection unit 115, and the travel speed detection unit 115 sets the pulse interval of the vehicle speed pulse detected by the wheel speed sensor. By measuring, the traveling speed of the vehicle in which the present apparatus is mounted is detected. Then, speed information indicating the traveling speed of the vehicle is input to the language understanding unit 104.

  In the language understanding unit 104, various numerical parameters are generated in the process of generating an understanding result from a word input with a plurality of utterances from the user and the reliability of the class to which the language understanding unit 104 belongs, as in the first embodiment. In the dialog understanding device of this embodiment, the numerical parameter used for this calculation is changed according to the traveling speed of the vehicle detected by the traveling speed detector 115. That is, in the dialogue understanding device according to the present embodiment, the parameter set holding unit 114 holds a plurality of numerical parameters classified for each traveling speed of the vehicle, and among the numerical parameters held in the parameter set holding unit 114, Thus, a numerical parameter corresponding to the actually detected traveling speed of the vehicle is read out and used in the calculation. The plurality of numerical parameters held in the parameter set holding unit 114 collects a large number of utterance data mixed with a plurality of types of noises associated with vehicle running in advance, and statistically processes them to thereby calculate the vehicle running speed. It is estimated as a parameter according to.

  More specifically, in the dialogue understanding device according to the present embodiment, the class score is obtained using the above (Equation 5) and (Equation 6), as in the first embodiment described above. 7) The word score is calculated using equation (8) and the parameters weight s, n, weight t, n and p1n to p11n used in these calculations are utterance data mixed with noise. However, the feature of the present embodiment is that the language understanding unit 104 actually detects the travel of the vehicle from a plurality of numerical parameters classified for each travel speed of the vehicle. A numerical parameter corresponding to the speed is read from the parameter set holding unit 114 and used. Here, n indicates the vehicle traveling speed at which mixed noise is observed, and is experimental from speech data classified by traveling speed [n = speed 0, speed 1, speed 2, speed 3...]. When the numerical parameters are obtained, a plurality of numerical parameters are obtained in accordance with the traveling speed. In the dialogue understanding device according to the present embodiment, when the class score is obtained using the formula (5) or the formula (6) and the word score is obtained using the formula (7) or the formula (8), respectively, Numerical parameters (weight s, n, weight t, n, and p1n to p11n) corresponding to the traveling speed of the vehicle actually detected by the speed detector 115 are selected and used.

  As described above, according to the dialog understanding device of the present embodiment, the parameters used in the integrated calculation for interactively understanding the utterance contents are based on the utterance data in which plural types of noise are mixed for each vehicle traveling speed. Since it is statistically estimated and the conversation contents are understood using parameters corresponding to the actually detected vehicle traveling speed, the influence of background noise of the utterance can be greatly reduced, For example, even when used in a noisy environment such as voice input in a vehicle navigation system, good recognition accuracy can be obtained.

(Third embodiment)
Next, a dialogue understanding device according to a third embodiment to which the present invention is applied will be described. The dialog understanding device of the present embodiment is an example of a dialog understanding device realized as a function of the vehicle navigation system. As shown in FIG. 19, the dialog understanding device of the first embodiment described above. A travel route detection unit 116 is provided instead of the noise detection unit 113 in FIG. In addition, the parameters used in the integrated calculation for understanding the dialogue contents are statistically estimated from the utterance data classified for each travel route of the vehicle, and based on the route information detected by the travel route detection unit 116. The above parameters are changed. Since the other configuration and the basic processing flow are the same as those in the first embodiment described above, the same description as in the first embodiment will be omitted below, and the present embodiment will be omitted. Only the characteristic part will be described.

  In the dialog understanding device of this embodiment, the travel route information is acquired from the navigation device by the travel route detection unit 116, the travel route on which the vehicle on which the device is mounted is currently traveling is detected, and the information is understood in language. Input to the unit 104. Here, the travel route information acquired from the navigation device is information of contents such as a road surface state, the presence of a tunnel, and the presence of an elevated route. Such travel route information is stored in the navigation device, and the travel route detection unit 116 acquires information on the route on which the vehicle is currently traveling from the navigation device and outputs the information to the language understanding unit 104.

  In the language understanding unit 104, various numerical parameters are generated in the process of generating an understanding result from a word input with a plurality of utterances from the user and the reliability of the class to which the language understanding unit 104 belongs, as in the first embodiment. In the dialog understanding device of this embodiment, the numerical parameter used for this calculation is changed according to the route information of the travel route detected by the travel route detection unit 116. That is, in the dialogue understanding device of the present embodiment, the parameter set holding unit 114 holds a plurality of numerical parameters classified for each travel route, and among the numerical parameters held in the parameter set holding unit 114, Numerical parameters corresponding to the actually detected route information of the traveling route of the vehicle are read out and used in the calculation. The plurality of numerical parameters held in the parameter set holding unit 114 collects a large number of utterance data mixed with a plurality of types of noises associated with vehicle traveling in advance, and statistically processes them to thereby calculate the vehicle travel route. It is estimated as a parameter according to.

  More specifically, in the dialogue understanding device according to the present embodiment, the class score is obtained using the above (Equation 5) and (Equation 6), as in the first embodiment described above. 7) The word score is calculated using equation (8) and the parameters weight s, n, weight t, n and p1n to p11n used in these calculations are utterance data mixed with noise. However, the feature of the present embodiment is that the language understanding unit 104 actually detects the travel of the vehicle from a plurality of numerical parameters classified for each travel route of the vehicle. The numerical parameter corresponding to the route information of the route is read from the parameter set holding unit 114 and used. Here, n indicates a travel route in which mixed noise is observed, and is experimentally determined from speech data classified into each travel route [n = route 0, route 1, route 2, route 3 ...]. When the numerical parameters are obtained, a plurality of numerical parameters are obtained according to the route information of the traveling route. In the dialogue understanding device of the present embodiment, when the class score is obtained using the above (Equation 5) and (Equation 6), and the word score is obtained using the (Equation 7) and (Equation 8), respectively, Numerical parameters (weight s, n, weight t, n, and p1n to p11n) corresponding to the route information of the travel route of the vehicle actually detected by the route detection unit 116 are selected and used.

  As described above, according to the dialogue understanding device of the present embodiment, the parameters used in the integrated calculation for interactively understanding the utterance contents are utterance data in which plural types of noise are mixed for each traveling route of the vehicle. Since the dialogue contents are understood using parameters corresponding to the route information of the actually detected vehicle travel route, the influence of background noise of the utterance is greatly reduced. For example, even when used in a noisy environment such as voice input in a vehicle navigation system, good recognition accuracy can be obtained.

It is a block diagram which shows the basic composition of the dialogue understanding apparatus of 1st Embodiment to which this invention is applied. It is a figure explaining the hierarchical structure classification method of the uttered word. It is a response | compatibility figure which shows the relationship between a recognition object word and an utterance type. It is a figure which shows the example of the dialogue between a dialogue understanding device and a user. It is a flowchart which shows operation | movement of a dialog understanding apparatus. It is a figure which shows the relationship between the recognition result candidate sentence as an output of a speech recognition part, and likelihood. It is a figure explaining the method of calculating | requiring reliability from the relationship between a recognition result candidate sentence and likelihood. It is a correspondence figure which shows the relationship between an utterance type and an utterance type determination material. It is a flowchart which shows the proper use of the process by speech type. It is a figure which shows the production | generation process of the class score after the update in a refinement | miniaturization and an answer speech type. It is a figure which shows the production | generation process of the class score after the update in correction and re-input utterance type. It is a figure explaining the procedure which calculates a score for every category. It is a figure explaining a category comprehension process. It is a figure which shows the example of the integrated recognition result. It is a figure which compares and shows the final score in language understanding for every item. It is a correspondence figure which shows the relationship between a response flag and a response pattern. It is a correspondence figure which shows the relationship between a response flag and its content. It is a block diagram which shows the basic composition of the dialogue understanding apparatus of 2nd Embodiment to which this invention is applied. It is a block diagram which shows the basic composition of the dialogue understanding apparatus of 3rd Embodiment to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Speech input part 102 Speech recognition part 103 Reliability generation part 104 Language understanding part 105 Class score generation part 106 Category understanding part 107 Word score generation part 108 Understanding content generation part 109 Response generation part 110 Speech synthesis part 111 GUI display part 112 Recognition History 113 Noise detection unit 114 Parameter set holding unit 115 Traveling speed detection unit 116 Traveling route detection unit

Claims (5)

Class that classifies the utterances included in the dialogue into a class composed of multiple categories and subcategories of the categories hierarchically in the order of the width of the utterances, and gives a certainty of which class of words were uttered The score and the word score that gives the certainty of the words included in the utterance are calculated. When there are multiple utterances in the dialogue, multiple class scores and multiple word scores are calculated for each utterance. A dialogue understanding device that understands the dialogue contents by integrating
The parameters used in the integrated calculation are statistically estimated from utterance data mixed with a plurality of types of noise, and the contents of the dialogue are understood using parameters corresponding to the types of detected noise. Dialogue understanding device. A voice input means comprising a microphone and a voice amplifier;
Voice recognition means for digitizing the output of the voice input means to perform voice recognition;
Reliability generation means for calculating the reliability of the result recognized by the voice recognition means;
Using the results obtained by the speech recognition unit and the reliability generation unit, the plurality of categories set in advance and a hierarchical structure composed of the classes obtained by subdividing the categories are classified into the classes. Processed by a class score generation unit for determining the probability of utterance, a category understanding unit for determining each category from the results obtained thereby, a word score generation unit for determining the probability of a recognized word, and the above processing units A language understanding means comprising an understanding content generation unit for generating an understanding content as a result of
Storage means for storing past recognition history used for executing the processing in the language understanding means;
Response generation means for creating response information from the result obtained from the language understanding means;
Output means for outputting the response information,
When there is a new utterance, the language comprehension means uses the value obtained by parameter calculation of the class score and word score of the past recognition history, and a new confidence score based on the latest recognition result. Generating a word score, updating the recognition history with the new class score and word score, and using the category understanding obtained from the new class score and the new word score to understand the dialogue content,
The parameter is statistically estimated from utterance data mixed with a plurality of types of noise, and a new class score and word score are generated using a parameter corresponding to the type of detected noise. The dialogue understanding device according to claim 1.   The parameter is statistically estimated from speech data classified for each noise level, and the parameter is changed based on a noise level during speech detected by a noise detection means. The dialogue understanding device according to 1 or 2. A dialogue understanding device mounted on a vehicle,
The parameter is statistically estimated from utterance data classified for each traveling speed of the vehicle, and the parameter is changed based on speed information detected by a traveling speed detecting means. The dialogue understanding device according to 1 or 2. A dialogue understanding device mounted on a vehicle,
The parameter is statistically estimated from speech data classified for each travel route of the vehicle, and the parameter is changed based on route information detected by the travel route detection means. The dialogue understanding device according to 1 or 2.

Images