JP2008250236A - Speech recognition device and speech recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve speech recognition precision by recognizing an utterance object with high precision and selectively performing speech recognition for an utterance to an onboard device. <P>SOLUTION: When a passenger detector 26 detects the presence of a passenger using the output of a pressure sensor 45 installed in a seat, a speech, an image, and vibrations of a driver's seat in a state where a driver is conversing with the passenger are obtained and stored as profile data in a database 24. After a speech input detector 25 detects the driver operating a talk switch 44, the state of the driver is compared with the profile data to determine whether the driver gives utterance to the passenger or an onboard system, and selectively performs speech recognition of speech input directed to the onboard system. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a voice recognition apparatus and a voice recognition method for performing input processing using voice recognition, and more particularly, to an in-vehicle voice recognition apparatus and a voice recognition method.

  In recent years, various ideas have been made for realizing a technology for recognizing a user's voice. If the user's voice can be recognized, it is possible for the user to perform various device operations by voice. Especially, in-vehicle devices are concerned about the influence of manual operation by the driver on the driving. The practical application of operation technology is eagerly desired.

  In speech recognition, it may be necessary to recognize who is speaking to where. Thus, for example, Patent Document 1 discloses a technique for specifying a sound partner by using a microphone array and detecting a face direction to specify a speaking partner.

  Japanese Patent Laid-Open No. 2004-228688 measures a silent time between utterances in an in-vehicle device that accepts voice input, and whether or not the voice is a voice input to the in-vehicle device based on the measured silent time. The technique which discriminates is disclosed.

JP 2006-2111156 A JP 2003-308079 A

  However, in the method using the silent time between utterances as in the above-described prior art, there is a problem that speech recognition cannot be performed until the silent time is counted. In addition, there has been a problem that it is difficult to determine whether or not the speech is uttered when a sufficient amount of silent time cannot be obtained due to music data reproduction or the like.

  Therefore, when the user's utterance is performed, the speech recognition apparatus that can determine whether the utterance is a voice input to the in-vehicle device or a conversation with a passenger, that is, a speech utterance target with high accuracy. The realization of speech recognition methods has become an important issue.

  The present invention has been made to solve the above-described problems in the prior art and to solve the problems. The present invention recognizes an utterance target with high accuracy and selectively performs speech recognition on an utterance to an in-vehicle device. It is an object of the present invention to provide a speech recognition apparatus and speech recognition method that improve speech recognition accuracy.

  In order to solve the above-described problems and achieve the object, the speech recognition processing device and speech recognition method according to the present invention creates profile data from a state in which the driver is talking to a passenger, and profile data during speech. , It is determined whether the utterance is a conversation for a passenger or a voice operation input.

  According to the present invention, the voice recognition device and the voice recognition method discriminate the difference between the case where the driver talks with the passenger and the case where voice input is performed on the in-vehicle device, and only when the voice input is performed. Since voice recognition is performed, there is an effect that a voice recognition device and a voice recognition device with improved voice recognition accuracy can be obtained.

  Exemplary embodiments of a speech recognition apparatus and a speech recognition method according to the present invention will be explained below in detail with reference to the accompanying drawings.

  FIG. 1 is a schematic configuration diagram showing a schematic configuration of an in-vehicle system equipped with a voice recognition device 20 according to an embodiment of the present invention. The in-vehicle system 10 shown in FIG. 1 includes a voice recognition device 20, an in-vehicle integrated machine 30, a microphone 41, a camera 42, a vibration sensor 43, a talk switch 44, a pressure sensor 45, a display 51, and a speaker 52.

  The in-vehicle integrated machine 30 is a device having a navigation function for setting a route to a destination and guiding a route, and an audio / visual function for receiving radio and television and reproducing music and images stored in a recording medium. And includes a device operation unit 31, a recording medium 32, a navigation processing unit 33, an AV processing unit 34, and an output processing unit 35 therein.

  The device operation unit 31 is a processing unit that receives an operation input from a user, and the received operation content is used for operation control of the navigation processing unit 33 and the AV processing unit 34. The recording medium 32 is an HDD, DVD, CD, MD, or the like, on which map data, music data, and video data are recorded. The navigation processing unit 33 is a processing unit that implements a navigation function, and the AV processing unit 34 is a processing unit that implements an audio / visual function. Then, the output processing unit 35 performs processing for controlling the display output of the display 51 and the audio output of the speaker 52 based on the outputs of the navigation processing unit 33 and the AV processing unit 34.

  The voice recognition device 20 functions as an operation unit that controls the operation of the in-vehicle integrated machine 30 by recognizing the user's voice and inputting it to the device operation unit 31.

  Specifically, the voice recognition device 20 is connected to the microphone 11, the camera 42, the vibration sensor 43, the talk switch 44, and the pressure sensor 45. The voice recognition device 20 includes a buffer memory 21, a voice recognition unit 22, a profile creation unit 23, profile data 24, a voice input detection unit 25, a passenger detection unit 26, and a main control unit 27 therein.

  The microphone 41 is sound collection means for acquiring ambient sounds, particularly the driver's voice. The camera 42 is an imaging unit that captures a driver's face image, and the vibration sensor 43 is a detection unit that detects vibration of the driver's seat. The audio data acquired by the microphone 41, the image data captured by the camera 42, and the vibration data acquired by the vibration sensor 43 are stored in the buffer memory 21.

  The talk switch 44 is an operation means operated by a user, and is used for a request for starting speech recognition. The voice recognition device 20 recognizes a voice by cutting out an utterance from voice data collected by the microphone 41 after the talk switch 44 is operated.

  Here, even the voice data acquired after the talk switch 44 is operated is not necessarily a voice operation instruction for the in-vehicle system 10. For example, a conversation with a passenger may occur after the driver presses a talk switch. Therefore, if the utterance after the talk switch operation is simply regarded as a voice input to the in-vehicle system 10 and is subject to voice recognition, the words originally spoken to the passenger will be handled as the operation input, which is erroneous. Increase recognition.

  Therefore, in the voice recognition device 20, the state when the driver speaks to the passenger is acquired in advance and profile data is created. Then, by comparing the driver's state after the talk switch operation and the profile data, it is determined whether the utterance is a conversation to the passenger or a voice input to the in-vehicle system 20.

  The state of the driver used for the creation of profile data and comparison with the profile data includes the volume and change of the driver's voice, the height of the voice, the difference in the volume of the voice from the noise section, the utterance start part or It is possible to use the voice characteristics at the end of the utterance, the utterance speed, the change of the face in the image data, the vibration of the driver's seat, and the like.

  If there is no passenger, it can be estimated that all of the driver's speech is voice input. Therefore, the voice recognition device 20 collects the output of the pressure sensor 45 built in the seat surface and backrest of each seat by the passenger detection unit 26 and detects whether or not the passenger is in the vehicle.

  The main control unit 27 is a control unit that controls the entire speech recognition apparatus 20. If the profile data has not yet been created, the main control unit 27 connects the buffer memory 21 to the profile creation unit 23 to acquire the state of the driver during the conversation with the passenger. The profile creation unit 23 creates profile data using the acquired information and stores it in the database 24. When the voice input detection unit 25 detects that the talk switch 44 has been operated, the main control unit 27 connects the buffer memory 21 to the voice recognition unit 22, and the state of the driver during conversation with the passenger To get.

  The voice recognition unit 22 detects the start and end of an utterance from the voice data and image data stored in the buffer memory 21, recognizes the content of the utterance by voice recognition, and inputs it to the device operation unit 31.

  At this time, the voice recognition unit 22a determines whether the utterance in the voice data is a conversation for the passenger or a voice input to the in-vehicle system 10 by the utterance target determination unit 27a therein, and targets the in-vehicle system 10 Speech recognition is performed only for the utterance. Specifically, the utterance target determination unit 22a determines the utterance target by comparing the driver's state and profile data during voice recognition.

  FIG. 2 is a flowchart showing the processing operation of the speech recognition apparatus 20. The processing operation shown in the figure is an example of a main flow that the speech recognition apparatus 20 repeatedly executes.

  First, the speech recognition apparatus 20 determines whether or not profile data needs to be created (step S101). Specifically, it is determined that the profile data needs to be created when the profile data has not been created or when the occupant configuration of the vehicle has changed after the creation of the profile data. The change in the vehicle occupant configuration can be made by detecting the output change of the pressure sensor 45 or the opening / closing of the door.

  If it is necessary to create profile data (step S101, Yes), the profile creation unit 23 creates profile data and registers it in the database 24 (step S102). After completion of profile learning or when it is not necessary to create profile data (No in step S101), the main control unit 27 monitors whether or not the talk switch 44 has been operated (step S103). If not operated (step S103, No), the process is terminated as it is.

  On the other hand, when the talk switch 44 is operated (step S103, Yes), the voice recognition device 20 performs voice recognition by the voice recognition unit (step S104), and operates the in-vehicle integrated machine 30 using the voice recognition result. The input is executed (step S105), and the process ends.

  Next, details of the profile learning process shown in FIG. 2 will be further described with reference to FIG. As shown in the figure, in the profile learning process, first, the passenger detection unit 26 determines whether or not a passenger is on the basis of the output of the pressure sensor 45 (step S201). If not (No at Step S201), the process is terminated as it is.

  On the other hand, when there is a passenger (step S201, Yes), the profile creation unit 23 acquires audio data, image data, and the like from the buffer memory (step S202), and registers them as profile data (step S203). Exit.

  Next, a specific example of the speech recognition process shown in FIG. 2 will be described. As described above, in the present invention, the volume and change of the driver's voice, the pitch of the driver, the difference in the volume of the voice from the noise section, the characteristics of the voice at the utterance start part or the utterance end part, the utterance speed Then, the face change in the image data, the vibration of the driver's seat, etc. are compared with the profile data to determine the utterance target.

  For example, when using the loudness of the voice, as shown in FIG. 4, first, the utterance section is detected for the input speech, and then the input level average of the utterance section is calculated. Then, the calculated profile data value is compared with the obtained calculation result, and when it is larger than the profile data value by a threshold value or more, it is determined that the voice is directed to the in-vehicle system. That is, in this method, the driver can determine the utterance target based on the viewpoint that the driver utters a louder voice than when speaking with the passenger when performing voice input.

  FIG. 5 shows an utterance target determination method when voice volume inflection is used. In the figure, first, an utterance section is detected for the input speech, and an average value is calculated for each section by dividing the input level of the utterance section into unit times. Then, the fluctuation width (dispersion value) of the calculation result is calculated, and compared with the profile value calculated in the same manner, if it is greater than the threshold value compared to the value of the profile data, the voice is directed to the in-vehicle system judge. That is, in this method, the driver can determine the utterance target based on the viewpoint that the driver speaks in a clearer tone than when speaking with the passenger.

  FIG. 6 shows an utterance target determination method when using voice pitch inflection. In this method, first, an utterance interval is detected for input speech, the utterance interval is Fourier-transformed in the frequency domain, and a fluctuation width on the frequency axis is calculated for the strongest frequency band. Then, the calculation result is compared with the value of the profile calculated in the same manner, and when it is larger than the threshold value by the threshold value, it is determined that the voice is directed to the in-vehicle system. Also in this method, the driver can determine the utterance target based on the viewpoint that the driver speaks in clearer tone than when talking with the passenger.

  FIG. 7 shows an utterance target determination method based on a difference in sound input level between the utterance section and the other sections. In the figure, first, an utterance section is detected for input speech, and an average of input levels is calculated for the utterance section and other sections. Then, a difference is calculated for the average of these two input levels, and if the difference value is larger than the threshold value calculated in the same manner by the threshold value, it is determined that the voice is directed to the in-vehicle system. In other words, in this method, the driver determines his / her utterance volume according to the volume of the environmental sound (the sound around the non-speaking section), and the volume level relative to the environmental volume is the same as when talking with the passenger. It is possible to determine the utterance target based on the viewpoint of uttering in a clearer tone.

  When image data is used, as shown in FIG. 8, first, a speech section is detected for the input voice, and then the position of the lower lip in the same time zone as the speech section is detected from the camera image. Then, the deflection width of the position of the lower lip is calculated. The obtained calculation result is compared with the value of the profile data calculated in the same manner, and when the value is larger than the threshold value by the threshold value or more, it is determined that the voice is directed to the in-vehicle system. That is, in this method, the driver can determine the utterance target based on the viewpoint that the driver utters a louder voice than when speaking with the passenger when performing voice input.

  Further, when using vibration of the driver's seat, as shown in FIG. 9, first, the utterance section is detected for the input voice, and then the vibration value of the driver's seat in the same time zone as the utterance section is detected, and the fluctuation width is detected. Calculate Then, the obtained calculation result is compared with the value of the profile data calculated in the same manner, and when the value is larger than the threshold value by the threshold value or more, it is determined that the voice is directed to the in-vehicle system. In other words, this method is based on the viewpoint that the driver speaks with a louder voice and a clear tone than when talking with the passenger, and the difference in utterance appears in the vibration of the driver's seat. To determine the utterance target.

  In addition, when using the voice features of the utterance start portion and the utterance end portion as shown in FIG. 10, first, the utterance interval is detected for the input speech, and the input level of the utterance interval is divided into unit times and is divided for each interval. Calculate the average value. Then, the transition of the average input level per unit time is calculated and compared with the profile value calculated in the same manner. As a result, when the decrease level of the input level (in the case of the start portion) is greater than the threshold value by the threshold value, it is determined that the voice is directed to the in-vehicle system. In other words, with this method, the driver speaks in clearer tone than when speaking with the passenger, and the utterance is based on the point that the difference appears at the beginning and end of the utterance. The target can be determined.

  Furthermore, the speech rate can be used as shown in FIG. In this case, first, the utterance interval is detected for the input speech, the utterance interval is Fourier-transformed for the frequency domain, and the number of syllables (vowel number) per unit time is obtained from the characteristics of the input speech. Then, the calculation result is compared with the calculated profile value, and when the utterance speed is slower than the threshold value by the threshold value, it is determined that the utterance is directed to the in-vehicle system. Even in this method, the driver can determine the utterance target based on the viewpoint that when the voice is input, the utterance is more polite and slower than when speaking with the passenger.

  As described above, the voice recognition device 20 according to the present invention acquires the driver's state as profile data when the driver is talking to the passenger, and sets the driver's state as profile data when voice is input. To determine whether the utterance of the driver is a conversation directed to the passenger or a voice input directed to the in-vehicle system.

  Therefore, it is possible to suppress the occurrence of misrecognition that erroneously recognizes a conversation with a passenger as a voice input, and to realize a highly accurate voice input.

  Note that the configuration and operation shown in this embodiment are merely examples, and do not limit the present invention. The configuration and operation of the present invention can be implemented with appropriate modifications. For example, if there is a configuration in which microphones having directivity are provided in the driver direction and the passenger seat direction, it is possible to determine who speaks from the difference in the input level of each microphone, and in which direction the driver faces. Can be estimated. Furthermore, by monitoring whether or not there is a voice input from the driver, the talk switch can be omitted.

  As described above, the speech recognition apparatus and speech recognition method according to the present invention are useful for improving speech recognition accuracy, and are particularly suitable for improving recognition accuracy by determining an utterance target in an in-vehicle device.

BRIEF DESCRIPTION OF THE DRAWINGS It is a schematic block diagram explaining the schematic structure of the vehicle-mounted system carrying the speech recognition concerning this invention. It is a flowchart explaining the processing operation of a speech recognition apparatus. It is a flowchart explaining a profile learning process. It is explanatory drawing explaining the speech object determination by a loudness. It is explanatory drawing explaining the speech object determination by the inflection of a voice volume. It is explanatory drawing explaining the speech object determination by the inflection of a voice pitch. It is explanatory drawing explaining the determination using the level difference of an utterance area and another area. It is explanatory drawing explaining the speech object determination by the change of the opening width of a mouth. It is explanatory drawing explaining the speech object determination by the vibration detection of a driver's seat. It is explanatory drawing explaining the utterance target determination by the feature of the talk start and the talk end. It is explanatory drawing explaining the speech object determination by the speed of a talk.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 In-vehicle system 20 Speech recognition apparatus 21 Buffer memory 22 Speech recognition part 22a Speech object determination part 23 Profile creation part 24 Database 25 Voice input detection part 26 Passenger detection part 27 Main control part 30 In-vehicle integrated machine 31 Device operation part 32 Recording medium 33 Navigation processing unit 34 AV processing unit 35 Output processing unit 41 Microphone 42 Camera 43 Vibration sensor 44 Talk switch 45 Pressure sensor 51 Display 52 Speaker

Claims (7)

Profile creation means for obtaining information on the state in which the driver is talking to the passenger and creating profile data from the obtained information;
An utterance target determination unit that compares the state of the driver with the profile data during the driver's utterance and determines whether the utterance is a conversation with a passenger or a voice operation input;
Voice recognition means for performing voice recognition on an utterance determined to be a voice operation input by the utterance target determination means;
A speech recognition apparatus comprising:   The voice recognition apparatus according to claim 1, wherein the profile creating unit creates the profile data by acquiring at least one of the driver's voice, an image, and a driver's seat vibration.   The profile creating means is configured such that the driver's voice volume when the driver is talking to a passenger, the change in the volume, the voice height, the difference in the voice volume from the noise interval, The speech recognition apparatus according to claim 2, wherein the profile data is created by using at least one of a voice feature and a speech rate at a speech start part or a speech end part.   The voice recognition device according to claim 2, wherein the profile creating unit creates the profile data by obtaining a fluctuation range of mouth opening from the driver's face image.   A passenger detection means for detecting the presence or absence of a passenger is further provided, wherein the profile creation means creates the profile data when the presence of a passenger is detected by the passenger detection means. The speech recognition device according to any one of 1 to 4.   Operation means for instructing start of voice operation input is further provided, and the voice recognition means is determined to be voice operation input by the utterance target determination means among voice data input after the operation means is operated. The speech recognition apparatus according to claim 1, wherein speech recognition is performed on the uttered speech. A profile creation step of obtaining information on a state in which the driver is talking to the passenger and creating profile data from the obtained information;
An utterance target determination step of comparing the driver state with the profile data during the driver's utterance, and determining whether the utterance is a conversation with a passenger or a voice operation input;
A voice recognition step for performing voice recognition on the utterance determined to be a voice operation input by the utterance target determination step;
A speech recognition method comprising:

Images