JP7002822B2 - Voice analysis system and voice analysis method - Google Patents

Description

The present invention relates to a voice analysis system and a voice analysis method for analyzing input voice.

In recent years, devices called smart speakers have become widespread. Generally, smart speakers are used in the following embodiments. That is, first, the user utters a specific word called a wake word to the smart speaker, and then utters some question or a word for making a request (hereinafter referred to as "request"). Voice data including wake words and requests are transmitted to the cloud server, and the cloud server recognizes the request by voice, understands the content of the request, and executes processing according to the content of the request. For example, when the content of the request is a question, the cloud server generates voice data for outputting the answer to the question by voice and sends it to the smart speaker, and causes the smart speaker to output the answer to the question by voice.

Regarding speech recognition, the following techniques are described in Cited Document 1. That is, when the volume level of the voice data (sound data 4) exceeds the threshold value, the voice portion is identified from the recorded voice data retroactively from the timing when the threshold value is exceeded to a predetermined time (time Tw2) before. A technique for performing voice recognition for the voice part is described. According to the technique of Cited Document 1, it is possible to recognize the entire voice portion without missing the head portion of the voice portion necessary for voice recognition.

Japanese Unexamined Patent Publication No. 2009-122598

By the way, a service related to voice recognition is provided as a cloud service by a predetermined company. Vendors who provide in-vehicle devices have a need to make the above-mentioned cloud services available in the products they provide, to make the in-vehicle devices function as smart speakers, and to add value to the products.

To use the cloud service, it is necessary to send voice data including wake words and requests (however, it may not be necessary to include the request) to the cloud server. Then, regarding the transmission of voice data to the cloud server, the following may be requested by a predetermined company that provides a cloud service. That is, in the voice data, it may be requested to transmit information specifying the start point and the end point of the voice corresponding to the wake word. In this case, it is necessary to detect the start point and the end point.

Here, some in-vehicle devices execute triggerless voice recognition to execute a process corresponding to a command word when a user boarding the vehicle utters a voice corresponding to the command word. The command word is, for example, a "nearby convenience store". In this case, the in-vehicle device searches a convenience store around the vehicle in which the in-vehicle device is mounted and presents the search result as a process corresponding to the command word. Execute the processing to be performed. In triggerless speech recognition, the speech pattern of the command word is registered in the speech recognition dictionary in advance, and the similarity is calculated based on the comparison result between the speech waveform of the input speech and the registered speech pattern, and this similarity is the threshold value. When the above is exceeded, it is detected that the voice corresponding to the command word is spoken.

By using this triggerless voice recognition, by registering the voice pattern of the wake word in the voice recognition dictionary, it is possible to detect with high accuracy when the user utters the voice corresponding to the wake word. However, as described above, the triggerless speech recognition does not perform processing such as specifying the range of speech corresponding to the instruction word from the input speech, but calculates the similarity with the pre-registered speech pattern and uses the calculated similarity as the calculated similarity. Since it is detected that the voice corresponding to the instruction word is uttered based on the triggerless voice recognition, the start point and the end point cannot be detected.

The present invention has been made to solve such a problem, and an object of the present invention is to enable detection of a wake word start point and end point with high accuracy.

In order to solve the above-mentioned problems, the present invention recognizes a specific word by comparing the input voice with the voice pattern of the pre-registered specific word, and is continuous at the timing when the specific word is recognized. Among the frequencies constituting the input voice input during the period, the frequency corresponding to the frequency having the highest sound pressure level is specified as the target frequency. Then, the present invention detects the start point and the end point of the specific word in the input voice based on the transition of the sound pressure level of the target frequency in the period before and after the timing when the specific word is recognized.

According to the present invention configured as described above, since the voice recognition for recognizing a specific word is performed by comparing the input voice and the voice pattern of the pre-registered specific word (wake word), the user can perform the voice recognition. When a specific word is spoken as voice, it can be detected with high accuracy. On top of that, according to the present invention, the characteristic of the sound pressure level is targeted at the period before and after the timing when the specific word is recognized, in other words, the period during which the specific word is very likely to be spoken. It is possible to detect the start point and the end point based on the change, and the start point and the end point can be detected with high accuracy at that point. In particular, according to the present invention, among the frequencies constituting the voice input when the user speaks a specific word as voice, the frequency corresponding to the frequency having the highest sound pressure level (the highest frequency and the highest frequency). The transition of the sound pressure level is analyzed with the target frequency (including frequencies close to the frequency) as the target frequency. For this reason, it is possible to analyze the frequency peculiar to the spoken voice of the user who spoke a specific word, and analyze the transition of the sound pressure level under the condition that it is not easily affected by noise by the voice of other frequencies. In this respect as well, the start point and end point of a specific word in the input voice can be detected with high accuracy.

It is a block diagram which shows the functional structure example of the voice analysis system which concerns on one Embodiment of this invention. It is a figure used for the explanation of the recognition target voice data. It is a figure which shows an example of the transition of a distance value. It is a figure which shows an example of the transition of the sound pressure level of a target frequency. It is a flowchart which shows the operation example of the in-vehicle device which concerns on one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a functional configuration example of the voice analysis system 1 according to the present embodiment. As shown in FIG. 1, the voice analysis system 1 includes an in-vehicle device 2. The in-vehicle device 2 can access the network N configured to include a communication network such as the Internet and a telephone network. The service providing server 3 is connected to the network N.

The in-vehicle device 2 is a so-called car navigation system mounted on a vehicle, and has a function of detecting the current position of the vehicle, a function of searching for a route to a destination set by a user boarding the vehicle, and a function of guiding the vehicle. Have. A detailed description of these functions will be omitted. The in-vehicle device 2 does not have to be a device fixedly attached to the vehicle. For example, the in-vehicle device 2 may be a mobile terminal brought into the vehicle by the user. As shown in FIG. 1, a microphone 4 and a speaker 5 are connected to the vehicle-mounted device 2. Hereinafter, the vehicle on which the in-vehicle device 2 is mounted is referred to as "own vehicle". The microphone 4 is a sound collecting device provided at a position capable of collecting sound spoken by a user boarding the own vehicle. The speaker 5 is a sound emitting device that outputs sound into the vehicle of the own vehicle.

The service providing server 3 is a cloud server that provides a service related to voice recognition of voice collected by a client terminal. Hereinafter, the service provided by the service providing server 3 is referred to as a "voice recognition service". A smart speaker, a mobile terminal equipped with a virtual assistant, an in-vehicle device 2 according to the present embodiment, and the like function as a client terminal for the service providing server 3. One of the services related to voice recognition is to recognize voice collected by a client terminal, understand the content of the voice, and execute a process corresponding to the content of the voice. As an example, when the user utters a voice containing a question to the client terminal, the service providing server 3 recognizes the voice, understands the content of the voice, and generates an answer to the question. , The client terminal is made to output as voice, thereby realizing a voice dialogue between the user and the client terminal.

The vehicle-mounted device 2 according to the present embodiment functions as a client for the service providing server 3, and the user can use the voice recognition service via the vehicle-mounted device 2. When a user uses a voice recognition service, he / she speaks a specific predetermined word called a wake word, and following the utterance of the wake word, a word for making a question or request (hereinafter referred to as "request"). ). The in-vehicle device 2 generates processing request data and transmits the processing request data to the service providing server 3 in response to the wake word and the utterance of the request by the user. The processing request data includes recognition target voice data and control information data.

The recognition target voice data includes voice corresponding to the wake word spoken by the user (hereinafter referred to as "wake word compatible voice") and voice corresponding to the request spoken by the user (hereinafter referred to as "request compatible voice"). Recorded voice data. This recognition target voice data must meet the functional requirements required for using the voice recognition service.

FIG. 2 is a diagram schematically showing the structure of the recognition target voice data in a mode suitable for explanation in order to explain the structure. In FIG. 2, the recognition target voice data is represented by a band-shaped object starting from the left end in the figure. In the present embodiment, the recognition target voice data is voice data sampled in a sample cycle of 16 kHz. In FIG. 2, the range H1 indicates the range of the wakeword-compatible voice recorded in the recognition target voice data (a set of sampling cycle data in which the voice waveform of the wakeword-compatible voice is recorded), and the range H2 is the recognition target. Shows the range of request-responsive voice recorded in the voice data.

As shown in FIG. 2, in the recognition target voice data, it is required as a functional requirement to include a preroll for 8000 samples (500 milliseconds) before the wake word-compatible voice. Therefore, in the voice data to be authenticated, 8000 samples (500 milliseconds) of voice are recorded as a pre-roll before the range in which the wake word-compatible voice is recorded. become.

As shown in FIG. 2, in the recognition target voice data, the wake word-compatible voice is in the range H1 for the α sample from the Q0 (Q0 = 8000 in this embodiment) sampling cycle to the Q1 sampling cycle at the end of the preroll. Recorded. The value of α varies depending on the time length of the wake word-compatible voice spoken by the user (the length of the period from the start of uttering the wake word voice to the end of utterance), and the value of Q1 is the value of α. It fluctuates according to. In the recognition target voice data, the point at the Q0 sampling cycle is the timing at which the wakeword-compatible voice is started, and corresponds to the start point (sometimes called the start index). Further, in the recognition target voice data, the point at the Q1 sampling cycle is the timing at which the wakeword-compatible voice ends, and corresponds to the end point (sometimes called an end index). As shown in FIG. 2, in the recognition target voice data, the request-corresponding voice is recorded in the range H2 following the range H1 in which the wake word-corresponding voice is recorded at intervals.

The control information data is data in which necessary reference information regarding the recognition target voice data is described according to a predetermined format (for example, JSON format data). In the present embodiment, the control information data includes information used for identifying the processing request data, information indicating the format of the recognition target voice data, and the start point and end point of the wake word-compatible voice in the recognition target voice data. It is required to include information that specifies the location of. In the control information data, both the information that specifies the position of the start point and the information that specifies the position of the end point are represented by the sampling period starting from the beginning of the data (0th cycle), and in particular, the start point. The information that clearly indicates the position is fixedly set to the 8000th cycle.

As described above, the control information data describes the information that clearly indicates the position of the start point and the information that clearly indicates the position of the end point, but in the present embodiment, the information described in the control information data is described. The error between the indicated start point (8000th cycle in this example) and the point at which the wake word-compatible voice actually starts in the recognition target voice data (position of the start end of the voice waveform of the voice) is 50 milliseconds (800 samples). ) Is required as a functional requirement. Further, in the present embodiment, there is an error between the end point indicated by the information described in the control information data and the point at which the wakeword-compatible voice actually ends in the voice data to be authenticated (the position of the end of the voice waveform of the voice). Is required to be within 150 milliseconds (2400 samples) as a functional requirement. On the premise of the above, the configuration and processing of the in-vehicle device 2 will be described in detail below.

As shown in FIG. 1, the in-vehicle device 2 has a communication unit 10, a voice processing unit 11, a registered word detection unit 12, a command word corresponding processing execution unit 13, a user frequency registration unit 14, a frequency identification unit 15, and so on. It includes a point detection unit 16 and a linkage process execution unit 17. Each of the above functional blocks 10 to 17 can be configured by any of hardware, DSP (Digital Signal Processor), and software. For example, when configured by software, each of the above functional blocks 10 to 17 is actually configured to include a computer CPU, RAM, ROM, etc., and is a program stored in a recording medium such as RAM, ROM, hard disk, or semiconductor memory. Is realized by the operation of.

As shown in FIG. 1, the in-vehicle device 2 includes a voice buffer 20, a dictionary storage unit 21, and a user frequency storage unit 22 as storage means. The voice buffer 20 is a temporary storage area in which input voice data described later is buffered. A voice recognition dictionary is stored in the dictionary storage unit 21. The voice recognition dictionary is a dictionary for voice recognition in which voice patterns are registered for each of the registered words. The registration word includes a command word and a wake word. That is, in the present embodiment, a wake word is registered as a registered word in the speech recognition dictionary in addition to the command word.

The command word is a word that instructs the in-vehicle device 2 (which may be another device under the control of the in-vehicle device 2) to execute a specific process. By uttering a voice corresponding to one command word, the user performs a specific process corresponding to the one command word on the vehicle-mounted device 2 without performing a manual instruction operation on the vehicle-mounted device 2. Can be executed.

The communication unit 10 accesses the network N according to a predetermined communication standard, and communicates with an external device (including the service providing server 3) connected to the network N. Any communication method may be used when the communication unit 10 accesses the network N, and any communication standard may be used for communication with the external device. For example, the communication unit 10 communicates with a mobile terminal brought into its own vehicle by wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), and accesses network N by using the tethering function of the mobile terminal. do. Further, the communication unit 10 accesses the network N by directly accessing the mobile communication network.

The voice processing unit 11 performs analog / digital conversion processing including sampling, quantization, and coding processing on the voice picked up by the microphone 4 to generate voice data, and buffers it in the voice buffer 20. As a result, the voice data based on the voice picked up by the microphone 4 is stored in the voice buffer 20 during a predetermined period retroactively from the present time. Hereinafter, a set of voice data stored in the voice buffer 20 is referred to as "input voice data". Further, the sound processing unit 11 includes a D / A converter, a volume circuit, an amplifier circuit, and the like, digitally / analog-converts the input audio data by the D / A converter, adjusts the volume level by the volume circuit, and performs the amplifier circuit. It is amplified by the speaker 5 and output as audio from the speaker 5.

The registered word detection unit 12 detects when the input voice input to the voice processing unit 11 via the microphone 4 contains the registered word. More specifically, the registered word detection unit 12 continuously analyzes the input voice data cumulatively stored in the voice buffer 20 by the voice processing unit 11, and the voice waveform of the voice recorded in the input voice data (hereinafter referred to as “the voice waveform”). , It may be simply expressed as "voice waveform of input voice"), and the similarity of the voice pattern of the registered word registered in the voice recognition dictionary is continuously calculated. The registered word detection unit 12 continuously calculates the similarity for each of the registered words registered in the speech recognition dictionary. In the present embodiment, the registered word detection unit 12 uses a “distance value” as a degree of similarity, which takes a smaller value as the voice waveform of the input voice and the voice pattern of the registered word are similar in the range of 0 to 1000. calculate.

Then, the registered word detection unit 12 has a threshold TH in which the distance value between the voice pattern of any of the registered words and the voice waveform of the input voice among the voice patterns of the registered words registered in the voice recognition dictionary is predetermined. If it is less than, it is detected that the input voice contains the voice corresponding to the registered word (= the user has spoken the voice corresponding to the registered word). Hereinafter, detecting that the voice corresponding to the registered word (command word, wake word) is included in the input voice is expressed as simply "detecting the registered word (command word, wake word)". ..

FIG. 3 is a graph showing an example of the transition of the distance value calculated for a certain registered word by a graph in which the horizontal axis is time and the vertical axis is the distance value. In the example of FIG. 3, the distance value that has changed around 1000 during the period from timing TM0 to timing TM1 gradually decreases from around timing TM1 and falls below the threshold value TH in timing TM2. In the case of the example of FIG. 3, the registered word detection unit 12 detects the registered word at the timing TM2.

When the registered word detection unit 12 detects an instruction word among the registered words, the registered word detection unit 12 notifies the instruction word corresponding processing execution unit 13 of the identification information assigned to the detected instruction word. On the other hand, when the registered word detection unit 12 detects a wake word among the registered words, the registered word detection unit 12 notifies the frequency specifying unit 15 of information indicating that the wake word has been detected. As described above, the registered word detection unit 12 has a function of comparing the input voice and the voice pattern of the pre-registered wake word to detect the wake word (specific word), and in this respect. It corresponds to the "specific word detector" in the claims.

When the instruction word identification information is input from the registered word detection unit 12, the instruction word correspondence processing execution unit 13 executes the processing associated with the identification information. As a result, when the user utters a voice corresponding to the instruction word, the processing corresponding to the instruction word is executed by the instruction word correspondence processing execution unit 13. For example, the instruction word is a "map display" instructing to display a map around the own vehicle. In this case, the instruction word correspondence processing execution unit 13 displays a map around the own vehicle on a display means (not shown). indicate. Further, for example, the command word is "going home" instructing the search for the guided route to the registered home and the guidance of the searched guide route. In this case, the command word correspondence processing execution unit 13 is self. The guidance route from the current position of the vehicle to the pre-registered home is searched, and the guidance of the guidance route is executed.

The function of automatically detecting the instruction word on the vehicle-mounted device 2 side without giving a trigger by the user is an existing function called triggerless speech recognition, and has been conventionally implemented in the device. Therefore, the wake word can be detected with high accuracy only by registering the wake word voice pattern in the voice recognition dictionary related to this existing function.

The user frequency registration unit 14 registers user frequency-related information in the user frequency table stored in the user frequency storage unit 22. More specifically, a user who may board the vehicle and may receive the service of the service providing server 3 via the in-vehicle device 2 shall perform the following frequency registration work in advance. ing. The following work is performed in a quiet environment in the own vehicle. In the frequency registration work, the user first inputs a predetermined instruction to notify the user frequency registration unit 14 that the frequency registration work has started, and further, the wake word is in the same manner as when the wake word is usually spoken. To speak.

Upon receiving the notification that the frequency registration work has started, the user frequency registration unit 14 analyzes the voice data stored in the voice buffer 20 for a certain period from that point, and identifies the following items. The length of time for this period of time is sufficient to include the timing at which the user ends the wakeward utterance. That is, first, the user frequency registration unit 14 analyzes the input voice in the fixed period and identifies the frequency having the highest sound pressure level among the frequencies constituting the input input voice. The frequency specifying unit 15 specifies the frequency by, for example, the spectrum analyzer function of the DSP mounted on the microcomputer of the in-vehicle device 2. Hereinafter, the frequency specified here is referred to as a "user-specific frequency". The frequency specified here is a frequency peculiar to the user's uttered voice, and is a dominant frequency in which the characteristics of the user's uttered voice are most likely to appear.

Further, the user frequency registration unit 14 analyzes the input voice in the fixed period, and the time length of the voice corresponding to the wake word spoken by the user (the period from the start of uttering the wake word voice to the end of the utterance). (Length) is specified. Here, the input voice input during a certain period after receiving the notification that the frequency registration work has started is the voice corresponding to the wake word spoken by the user without fail, and is therefore recorded in the input voice data. The characteristic voice waveform is definitely a waveform derived from Wakeward. Based on this, the user frequency registration unit 14 corresponds to the wake word spoken by the user regarding the time length between the position where the voice waveform of the specified frequency rises and the position where the voice waveform falls in the input voice data. Specify as the time length of the voice. Hereinafter, the length specified here is referred to as "user-specific time length".

After specifying the user-specific frequency and the user-specific time length, the user frequency registration unit 14 generates a unique user ID, and the user frequency including the user ID, information indicating the user-specific frequency, and information indicating the user-specific time length. Register related information in the user frequency table. As a result of the above processing performed by the user frequency registration unit 14, the user frequency table stored in the user frequency storage unit 22 is displayed for each user who may board the own vehicle and use the voice recognition service. The user frequency-related information including the ID, the information indicating the user-specific frequency, and the information indicating the user-specific time length is registered. In the above example, the "user-specific time length" is measured based on one utterance of the user, but the user is made to utter a wake word-compatible voice a plurality of times, and the average is obtained from the measured values of each sample. It may be configured to measure the user-specific time length by another statistical method.

When the frequency specifying unit 15 inputs information indicating that a wake word has been detected from the registered word detecting unit 12, the frequency specifying unit 15 refers to the voice buffer 20 and refers to the current time (= the distance value is below the threshold value TH, and the registered word detecting unit 12 causes it. The input voice data for a period retroactively by a predetermined time starting from the timing when the wake word is detected) is analyzed, and the following items are specified. That is, the frequency specifying unit 15 specifies the frequency corresponding to the frequency having the highest sound pressure level among the frequencies constituting the input voice input during the period as the target frequency. The frequency corresponding to the frequency having the highest sound pressure level includes not only the frequency having the highest sound pressure level but also the frequency close to the highest frequency.

The period is a period continuous with the timing when the registered word detection unit 12 detects the wake word. Therefore, it is assumed that the input voice during the period includes the wake word-compatible voice spoken by one person in the vehicle (may include noise and other voices), and the target frequency. It can be said that the frequency specified as includes a frequency peculiar to the spoken voice of the person.

The point detection unit 16 wakes in the input voice based on the "transition of the sound pressure level of the target frequency specified by the frequency specifying unit 15" in the period before and after the timing when the wake word is recognized by the registered word detection unit 12. Detects the start and end points of a word. Hereinafter, the processing of the point detection unit 16 will be described in detail.

First, the point detection unit 16 extracts the transition of the sound pressure level of the target frequency (waveform of the target frequency) in the first period from the input voice data stored in the voice buffer 20, and uses the working area as the transition data in the first period. Expand to a predetermined storage area that functions as. The first period includes the timing when the wake word is detected by the registered word detection unit 12 (hereinafter referred to as “wake word detection timing”), and the period for a predetermined time and the wake word detection are retroactive in time from the wake word detection timing. It is a period that includes a predetermined time from the timing. If the current time has not reached the end of the first period, the point detection unit 16 expands the first period transition data into a predetermined storage area after reaching the end of the first period.

FIG. 4 is a graph showing an example of the transition of the sound pressure level of the target frequency during the period including the wake word detection timing and including the start point and the end point. In the graph of FIG. 4, the horizontal axis represents the passage of time, and the vertical axis represents the sound pressure level of the target frequency. In FIG. 4, the timing TX1 is the wake word detection timing. Further, in FIG. 4, the period KX1 is the first period, and the first period is a period of 200 milliseconds before and after the wake word detection timing. When the sound pressure level of the target frequency changes in the embodiment illustrated in FIG. 4, and the wake word detection timing is the timing TX1, the point detection unit 16 performs the sound of the target frequency in the period KX1 which is the first period. The first period transition data indicating the transition of the pressure level is stored in a predetermined storage area.

Next, the point detection unit 16 analyzes the transition data in the first period, and in the first period, the point where the sound pressure level of the target frequency changes from a state in which the sound pressure level of the target frequency decreases to a state in which the sound pressure level changes stably with the passage of time. Is detected as the end point. In the case of the example of FIG. 4, the point detection unit 16 detects the point PX1 as the end point. As an example, the point detection unit 16 calculates an approximate curve of the transition of the sound pressure level of the target frequency in the first period, and the slope of the tangent line of the approximate curve with the passage of time converges within a range of a predetermined value or less. The position of the starting point is detected as the ending point. In FIG. 4, reference numeral R1 represents an approximate curve. Hereinafter, the superiority of such an end point detection method will be described.

As described above, in the present embodiment, the wake word is detected when the distance value indicating the degree of similarity between the voice waveform of the input voice and the voice pattern of the wake word registered in advance falls below the threshold value TH. The distance value calculation method and the threshold value TH are designed so that the distance value gradually decreases from the timing when the wakeword-compatible voice starts to be spoken, and falls below the threshold value around the timing when the voice is finished. (See also Fig. 2). By designing the distance value calculation method and the threshold value TH in this way, it is possible to detect the wake word only when the user is likely to have spoken the entire wake word-compatible voice, for example. , It is possible to prevent the wake word from being erroneously detected when only a part of the wording of the wake word is unintentionally and accidentally spoken.

The value of the first period belongs to the timing when the wakeword-compatible voice is finished to be spoken within this first period, and the result of a preliminary test or simulation so that the first period does not become unnecessarily long. It is set properly based on. Therefore, when the user utters the wake word-compatible voice, it can be considered that the wake-word-compatible voice has been uttered within the first period including the wake word detection timing. Based on the above, according to the present embodiment, the "transition of the sound pressure level of the target frequency" is analyzed for a limited period (first period) in which the end point can be considered to belong. Since the end point is detected, the end point can be detected efficiently and with high accuracy, the processing load is small, and the time required for processing can be shortened. The reason why the first period is not unnecessarily long is that the longer the first period, the larger the period (data amount) to be analyzed, the larger the processing load and the processing time, and the more. This is because the possibility that noise is included in the first period transition data increases.

Further, in the present embodiment, the transition of the sound pressure level of the target frequency is the target of the analysis for detecting the end point. As a result, the frequency peculiar to the person who actually uttered the wake word-compatible voice (the dominant frequency in which the characteristics of the uttered voice of the person are most likely to appear) becomes the subject of analysis, so that other voices (for example, the own vehicle) are analyzed. It is possible to analyze the transition data for the first period in a situation where the noise-like influences (noise such as engine noise and environmental noise) are eliminated as much as possible, and the end point can be detected with high accuracy in this respect.

Further, in the present embodiment, a point at which the sound pressure level of the target frequency changes from a state in which the sound pressure level decreases to a state in which the sound pressure level changes stably is detected as an end point. When a series of wakeword-compatible voices is completed, the sound input to the microphone 4 is basically only the environmental sound, and the sound pressure level of the target frequency derived from the person who uttered the wakeword-compatible voice drops rapidly. It becomes a stable state at a low value. Therefore, the sound pressure level of the target frequency changes from a decreasing state to a stable state at the timing when the wakeword-compatible voice utterance ends. Based on this, according to the present embodiment, the end point can be detected with high accuracy by utilizing the characteristics of the mode of changing the sound pressure level of the target frequency.

The end point is a specific position in the waveform of the target frequency indicated by the first period transition data, but it is also used as information indicating a specific position in the audio waveform of the input audio data that is the source of the waveform of the target frequency. It is possible. This point is the same for the start point described later.

After detecting the end point, the point detection unit 16 refers to the user frequency table stored in the user frequency storage unit 22, and whether or not a user-specific frequency having the same value as or close to the value of the target frequency is registered. Is determined. Approximating the values of the two frequencies means that the difference between the two frequencies is less than or equal to a predetermined value. When a user-specific frequency having the same value as or close to the value of the target frequency is registered, the point detection unit 16 recognizes the user-specific time length included in the user frequency-related information corresponding to the user-specific frequency. The user-specific time length is specified as "detection time".

Here, when the value of the target frequency and the value of the user-specific frequency corresponding to one user frequency-related information are the same or similar, the person who uttered the wakeword-compatible voice this time and the one user frequency-related information are used. It is strongly presumed that the corresponding person is the same. Therefore, the user-specific time length specified as the detection time can be used as the time required from the start of uttering the wakeword-compatible voice to the end of the utterance by the person who has spoken the wakeword-compatible voice this time.

On the other hand, when such user frequency-related information is not registered, the point detection unit 16 specifies a predetermined time as a "detection time". The predetermined time is the average time required to speak the wakeword-compatible voice, and is calculated in advance using an average or other statistical method for a plurality of samples. In this case, a user whose user frequency-related information is not registered in the user frequency table has spoken a wake word-compatible voice, or a user whose user frequency-related information is registered in the user frequency table has spoken. It is probable that the target frequency that is the same as or close to the user-specific frequency corresponding to the user was not specified due to the detection error or other causes.

After specifying the detection time, the point detection unit 16 specifies the detection time and the retroactive timing (hereinafter referred to as "second period correspondence timing") from the end point of the input voice data stored in the voice buffer 20. do. In FIG. 4, the timing TX2 is the timing corresponding to the second period. Next, the point detection unit 16 includes a second period correspondence timing, a period of a predetermined time retroactively from the second period correspondence timing, and a period including a predetermined time ahead of the second period correspondence timing. Specified as 2 periods. In FIG. 4, the period KX2 is the second period, and the second period is a period of 200 milliseconds before and after the second period correspondence timing. Next, the point detection unit 16 extracts the transition of the sound pressure level of the target frequency (waveform of the target frequency) in the second period, and develops it in a predetermined storage area as the transition data of the second period.

Next, the point detection unit 16 analyzes the second period transition data, and in the second period, the point where the sound pressure level of the target frequency changes from a stable state to an increasing state with the passage of time. Is detected as the starting point. In the case of the example of FIG. 4, the point detection unit 16 detects the point PX2 as a start point. As an example, the point detection unit 16 calculates an approximate curve of the transition of the sound pressure level of the target frequency in the second period, and the slope of the tangent line of the approximate curve with the passage of time converges within a range of a predetermined value or less. The position of the starting point in which the value exceeds a predetermined value and becomes larger is detected as the starting point. Hereinafter, the superiority of the method for detecting such a start point will be described.

As described above, the detection time is set to the user-specific time length of the user frequency-related information registered in the user frequency table, or is set to a predetermined value. When the detection time is the user-specific time length, the length of the detection time represents the length of time required for the person who utters the wakeword-compatible voice this time to normally utter the wakeword-compatible voice. Therefore, when the detection time is the user-specific time length, the timing at which the wakeword-compatible voice starts to be spoken and the timing at which the detection time is advanced from the end point (second period correspondence timing) are close to each other. Can be regarded.

Based on the above, when the detection time is a user-specific time length, the "change in sound pressure level of the target frequency" is targeted for a limited period (second period) in which the start point can be considered to belong. Is analyzed and the start point is detected, so that the start point can be detected efficiently and with high accuracy. Further, the processing load is small and the time required for processing can be shortened. Furthermore, the possibility that the target data contains noise can be reduced, and the influence of noise can be eliminated as much as possible for analysis. On the other hand, when the detection time is a predetermined value, the length of the detection time represents the average time required to utter the wake word-compatible voice. In this case as well, the timing at which the wakeword-compatible voice starts to be spoken and the timing at which the detection time is traced back from the end point are not extremely separated, and the difference is assumed to be within a predetermined range. Therefore, the start point can be detected efficiently and with high accuracy.

The value of the second period belongs to the timing when the wake word-compatible voice is started to be spoken in this second period, and is the result of a preliminary test or simulation so that the second period does not become unnecessarily long. It is set appropriately based on. Then, the second period is changed so that the detection time is shorter when the detection time is the user-specific time length and when the detection time is a predetermined value. It may be configured to be used. When the detection time is a user-specific time length, the second period can be shortened because the difference between the timing when the wake word-compatible voice starts to be spoken and the timing when the detection time goes back from the end point is small. Further, by shortening the second period, it is possible to further reduce the processing load and the processing time, and further reduce the influence of noise. Further, by lengthening the second period when the detection time is a predetermined value, the start point can be more reliably set to belong to the second period.

The point to be analyzed to detect the start point of the transition of the sound pressure level of the target frequency, and the point where the transition of the sound pressure level of the target frequency changes from a stable transition state to an increasing state. The effect of detecting as the start point is the same as in the case of the end point.

The cooperation processing execution unit 17 generates processing request data by using the start point and the end point detected by the point detection unit 16, and transmits the processing request data to the service providing server 3. Further, when the service providing server 3 receives a response, the cooperation processing execution unit 17 executes the processing corresponding to the response. Hereinafter, the processing of the cooperation processing execution unit 17 will be described in detail.

When the start point and the end point are detected by the point detection unit 16, the cooperation processing execution unit 17 analyzes the input voice data stored in the voice buffer 20, and inputs following the end point detected by the point detection unit 16. For voice (voice assumed to be a request), if the sound pressure level of the input voice is below the predetermined value and then the state below the predetermined value continues for a certain period of time or more, that is detected. Hereinafter, the timing at which this is detected is referred to as "end detection timing". In this case, it is assumed that the user has finished speaking the request-corresponding voice.

When it is detected that the sound pressure level of the input voice assumed to be a request is equal to or lower than a predetermined value for a certain period of time or longer, the cooperation processing execution unit 17 uses the point detection unit 16 for the input voice data of the voice buffer 20. Data from the point 8000 samples before the detected start point to the point corresponding to the end detection timing is extracted, and this data is used as the recognition target voice data to be included in the processing request data. As described with reference to FIG. 2, in the recognition target voice data, a wake word-compatible voice, a wake-word-compatible voice followed by a request-compatible voice are recorded, and 8000 samples of pre-roll are recorded before the start point. Is a requirement for the data structure, but since the voice data to be authenticated is generated by the above method, it is possible to generate the voice data to be authenticated that satisfies this requirement.

Further, the cooperation processing execution unit 17 generates control information data according to the format. As described above, the linkage process execution unit 17 sets the “information specifying the start point” described in the control information data to the 8000th cycle of a fixed value. Here, in the present embodiment, since the start point is detected with high accuracy by the point detection unit 16, the point at which the wakeword-compatible voice actually starts in the recognition target voice data (that is, the point detection unit 16 detects it). It is assumed that the difference between the start point (start point) and the start point indicated by the information described in the control information data (8000th cycle in this example) is very small, and the error is basically 50 milliseconds (800 samples). ) Will meet the functional requirements within. Further, the linkage process execution unit 17 sets the "information specifying the end point" described in the control information data as the end point detected by the point detection unit 16. As described above, since the end point is also detected with high accuracy by the point detection unit 16, the functional requirement that the error is within 150 milliseconds (2400 samples) is basically satisfied.

After generating the recognition target voice data and the control information data, the cooperation processing execution unit 17 generates the processing request data including these data, and transmits the generated processing request data to the service providing server 3. Information necessary for transmitting the processing request data to the service providing server 3 (address of the service providing server 3, information necessary for authentication, putrocol to be used, etc.) is registered in advance.

When the service providing server 3 receives the processing request data, the service providing server 3 performs an analysis including voice recognition on the recognition target voice data based on the content of the control information data, understands the content of the request, and processes corresponding to the content of the request. To execute. In the following, the content of the request is a question, and the service providing server 3 shall execute a process of outputting the answer to the question to the in-vehicle device 2 as voice.

If the content of the request is a question, a sentence (hereinafter referred to as "response") that answers the request is generated. Response generation is properly performed based on existing technology. For example, the response is "Tomorrow's weather is sunny" in response to the request "Tell me the weather for tomorrow's XX" (where XX represents the place). In this case, the service providing server 3 recognizes the content of the request through voice recognition processing by artificial intelligence, natural language processing, information summarization processing, etc., and the information necessary for generating the request (in this example, tomorrow's information). XX weather) is collected and a response is generated based on the collected information.

After generating the response, the service providing server 3 generates voice data (hereinafter referred to as "response voice data") in which the voice corresponding to the response is recorded. Next, the service providing server 3 responds to the vehicle-mounted device 2 with the response voice data.

The cooperation processing execution unit 17 of the in-vehicle device 2 receives the response voice data transmitted by the service providing server 3. The cooperative processing execution unit 17 outputs the received response voice data to the voice processing unit 11, and causes the speaker 5 to output the voice corresponding to the response recorded in the response voice data.

As described above, in the present embodiment, the in-vehicle device 2 recognizes the wake word by comparing the input input voice with the voice pattern of the wake word (specific word) registered in advance, and the wake word recognizes the wake word. Among the frequencies constituting the input voice input in the continuous period at the specified timing, the frequency corresponding to the frequency having the highest sound pressure level is specified as the target frequency. Then, the in-vehicle device 2 detects the start point and the end point of the wake word in the input voice based on the transition of the sound pressure level of the target frequency in the period including the timing when the wake word is recognized. More specifically, the in-vehicle device 2 detects the end point based on the transition of the sound pressure level of the target frequency in the first period including the wake word detection timing, and also includes the second period corresponding timing. The start point is detected based on the transition of the sound pressure level of the target frequency in.

According to the present embodiment, voice recognition (triggerless voice recognition) is performed in which the input voice is compared with the voice pattern of the wake word registered in advance to detect the wake word, so that the wake word is voiced by the user. When it is spoken as, it can be detected with high accuracy. On top of that, according to the present embodiment, the characteristics of the sound pressure level are targeted for the period before and after the timing when the wake word is recognized, in other words, the period when the wake word is very likely to be spoken. It is possible to detect the start point and the end point based on the change, and the start point and the end point can be appropriately detected with high efficiency and high accuracy.

In particular, according to the present embodiment, among the frequencies constituting the voice input when the user speaks the wake word as voice, the frequency corresponding to the frequency having the highest sound pressure level (in addition to the highest frequency, the highest frequency). The transition of the sound pressure level is analyzed with the target frequency (including frequencies close to high frequencies) as the target frequency. Therefore, since the frequency peculiar to the spoken voice of the user who uttered the wake word can be analyzed, the transition of the sound pressure level is analyzed under the condition that the sound is not easily affected by the voice of other frequencies. can do.

Further, according to the present embodiment, the start point and the end point are appropriately detected by effectively utilizing the existing resources used for triggerless speech recognition without implementing a dedicated speech recognition engine for wake word detection. be able to. As a result, the manufacturing cost and the product price can be reduced.

Next, an operation example of the in-vehicle device 2 when the user utters a wake word-compatible voice will be described with reference to the flowchart of FIG. As shown in the flowchart of FIG. 5, when the user utters a wake word-compatible voice, the registered word detection unit 12 sets the threshold value TH as the distance value between the voice waveform of the input voice and the voice pattern of the registered wake word. A wake word is detected when the value falls below the limit (step SA1). Next, the registered word detection unit 12 notifies the frequency specifying unit 15 of the information indicating that the wake word has been detected (step SA2).

When the frequency specifying unit 15 inputs information indicating that a wake word has been detected from the registered word detection unit 12, the frequency specifying unit 15 refers to the voice buffer 20 and inputs voice data for a period retroactively by a predetermined time starting from the wake word detection timing. Is analyzed and the target frequency is specified (step SA3).

When the target frequency is specified by the frequency specifying unit 15, the point detection unit 16 extracts the transition of the sound pressure level of the target frequency in the first period from the input voice data stored in the voice buffer 20, and the first period. It is expanded into a predetermined storage area as transition data (step SA4). Next, the point detection unit 16 analyzes the transition data for the first period, and sets the point at which the sound pressure level of the target frequency changes from the state where the sound pressure level of the target frequency decreases to the state where the sound pressure level changes stably as the end point. Detect (step SA5).

Next, the point detection unit 16 refers to the user frequency table stored in the user frequency storage unit 22, and registers user frequency-related information corresponding to the user-specific frequency having a value equal to or close to the value of the target frequency. Whether or not it is determined (step SA6).

When the user frequency-related information corresponding to the user-specific frequency having the same value as or close to the value of the target frequency is registered (step SA6: YES), the point detection unit 16 is the user included in the user frequency-related information. The unique time length is recognized, and the user-specific time length is specified as the "detection time" (step SA7). After the processing of step SA7, the point detection unit 16 shifts the processing procedure to step SA9. On the other hand, when such user frequency-related information is not registered (step SA6: NO), the point detection unit 16 specifies a predetermined time as a “detection time” (step SA8). After the processing of step SA8, the point detection unit 16 shifts the processing procedure to step SA9.

In step SA9, the point detection unit 16 specifies the second period correspondence timing for the input voice data stored in the voice buffer 20. Next, the point detection unit 16 identifies the second period, extracts the transition of the sound pressure level of the target frequency in the second period, and develops it in a predetermined storage area as the second period transition data (step SA10). Next, the point detection unit 16 analyzes the transition data for the second period, and sets the point at which the sound pressure level of the target frequency changes from a stable transition state to an ascending state in the second period as a start point. Detect (step SA11).

When the start point and the end point are detected by the point detection unit 16, the cooperation processing execution unit 17 determines that the sound pressure of the input voice is equal to or less than a predetermined value for the input voice following the end point detected by the point detection unit 16. After that, it is monitored whether or not the state of the predetermined value or less continues for a certain period of time or more (step SA12). If it continues (step SA12: YES), the linkage process execution unit 17 extracts data from the point 8000 samples before the start point detected by the point detection unit 16 to the point corresponding to the end detection timing. By doing so, the recognition target voice data is generated (step SA13). Next, the cooperation processing execution unit 17 generates control information data (step SA14). Next, the cooperation processing execution unit 17 generates the processing request data to be included, and transmits the generated processing request data to the service providing server 3 (step SA15). As described above, if there is a response to the processing request data, the cooperation processing execution unit 17 executes the processing corresponding to the response.

<Modification example>
Next, a modified example of the above embodiment will be described with reference to FIG. In the above embodiment, the point detection unit 16 specifies the first period, detects the end point based on the transition of the sound pressure level of the target frequency in the first period, and specifies the second period. The start point was detected based on the transition of the sound pressure level of the target frequency in the second period. On the other hand, in this modification, the point detection unit 16 includes the wake word detection timing as in the period indicated by the reference numeral Y in FIG. 4, and the wake word-corresponding voice is uttered in any mode. Even if there is (however, except when the utterance is made in an extremely abnormal manner), the period in which the end point and the start point are included (hereinafter referred to as "target period") is specified. The target period is, for example, a period sandwiched between a timing that is advanced by a predetermined time from the wake word detection timing and a timing that is advanced by a predetermined time from the timing.

The point detection unit 16 extracts the transition of the sound pressure level of the target frequency (waveform of the target frequency) in the target period from the input audio data stored in the audio buffer 20, and functions as a working area as the target period transition data. Expand to the storage area of. Next, the point detection unit 16 analyzes the target period transition data, and the point where the sound pressure level of the target frequency changes from a state in which the sound pressure level of the target frequency decreases with the passage of time in the target period to a state in which the sound pressure level changes stably. Is detected as the end point. Further, the point detection unit 16 detects as a start point a point at which the sound pressure level of the target frequency changes from a stable state to an increasing state according to the passage of time in the target period.

For example, the point detection unit 16 analyzes the transition (mode of change) of the sound pressure level of the target frequency while tracing back in time from the end of the target period, and rises from the state where the sound pressure level of the target frequency changes stably. The point where the sound pressure level changes to the end point is detected as the end point, and the point is a point before the end point in time, and the sound pressure level of the target frequency is stably changed from the state where the sound pressure level is decreasing. The point that changes to is detected as the starting point. In this case, since the transition of the sound pressure level is analyzed while going back in time, the end point and the start point are specified by the above method.

Further, for example, the point detection unit 16 analyzes the transition (mode of change) of the sound pressure level of the target frequency according to the passage of time from the beginning of the target period, and from the state where the sound pressure level of the target frequency changes stably. A point that changes to an ascending state is detected as a start point, and a point that is later in time than the start point and stably transitions from a state in which the sound pressure level of the target frequency decreases. The point that changes to the state is detected as the end point.

In the case of this modification as well, it is determined that there is a high possibility that the wake word-compatible voice was uttered as a result of comparison between the voice waveform of the input voice and the registered voice pattern during the period including the wake word detection timing. Since the analysis is performed for the possible period and the start point and end point are detected, each point can be detected with high accuracy. Furthermore, among the frequencies that make up the input voice, the transition of the sound pressure level in the target frequency band is analyzed as the target frequency for analyzing the target frequency band, which is the dominant frequency in which the characteristics of the user's spoken voice are most likely to appear. Since the start point and the end point are detected, each point can be detected with high accuracy. However, in the case of this modification, since the range to be detected for the input voice data is larger than that in the case of the above embodiment, the processing efficiency and the processing time are inferior in that respect.

Although the embodiments (including modifications) of the present invention have been described above, the above-described embodiments are merely examples of the embodiment of the present invention, thereby the technical scope of the present invention. Should not be interpreted in a limited way. That is, the present invention can be implemented in various forms without departing from its gist or its main features.

For example, in the above embodiment, the in-vehicle device 2 may execute a part or all of the processing executed by the service providing server 3. Further, the service providing server 3 (which may be an external device other than the service providing server 3) may execute a part or all of the processing executed by the in-vehicle device 2.

Further, in the above embodiment, the distance value is used as an index indicating the degree of similarity between the voice waveform of the input voice and the registered voice pattern, but it is needless to say that an index other than the distance value may be used as the index. ..

Further, the contents of the recognition target data and the control information data transmitted from the in-vehicle device 2 to the service providing server 3 are not limited to the contents exemplified in the above embodiment, and should be changed according to the required functional requirements. As an example, when it is required to include 1000 milliseconds of voice data as a preroll, the content of the recognition target data is so. Also, if it is necessary to specify the bit number of the start point or end point instead of the sampling cycle, or if the data is time-stamped and it is necessary to use the time stamp, control information data. The content of is done that way.

Further, in the above embodiment, when the user-specific time length corresponding to the target frequency is registered, the point detection unit 16 sets the detection time as the user-specific time length. Regarding this point, the point detection unit 16 may be configured to set the detection time as a fixed value. In this case, it is not necessary to register the user-specific time length in advance. Further, the length of the second period is appropriately set from the viewpoint that the length of the second period does not become unnecessarily long and the start point is included in the second period. Will be done.

Further, in the above embodiment, the vehicle-mounted device 2 executes triggerless voice recognition that automatically detects a command word on the vehicle-mounted device 2 side without giving a trigger by the user. In this regard, it does not necessarily have to be triggerless, and voice recognition may be performed using a predetermined operation on the touch panel or the operator, a gesture, or the like as a trigger.

1 Voice analysis system 12 Registered word detection unit (specific word detection unit)
15 frequency specific part 16 point detection part

Claims (8)

A specific word detection unit that detects the specific word by comparing the input voice with the voice pattern of the specific word registered in advance, and
Of the frequencies constituting the input voice input during a period continuous to the timing when the specific word is detected by the specific word detection unit, the frequency corresponding to the frequency having the highest sound pressure level is specified as the target frequency. With a specific part
The start point of the specific word in the input voice based on the transition of the sound pressure level of the target frequency specified by the frequency specific unit in the period before and after the timing when the specific word is detected by the specific word detection unit. And a point detector that detects the end point,
A voice analysis system characterized by being equipped with. The point detection unit is
The point at which the sound pressure level of the target frequency changes in a predetermined manner in the first period including the timing when the specific word is detected by the specific word detection unit is detected as the end point.
In the second period, which is a period before the first period and includes the detection time and the timing retroactive from the detected end point, the sound pressure level of the target frequency is in a predetermined embodiment. The voice analysis system according to claim 1, wherein a changing point is detected as the start point. When the value of the target frequency is a user-specific frequency registered in advance, the point detection unit is characterized in that the value of the detection time is a user-specific time length previously associated with the user-specific frequency. The voice analysis system according to claim 2. The point detection unit is
In the first period, a point at which the sound pressure level of the target frequency changes from a state in which the sound pressure level decreases to a state in which the sound pressure level changes stably is detected as the end point.
The second or third aspect of the present invention, wherein a point at which the sound pressure level of the target frequency changes from a stable state to an increasing state is detected as the start point in the second period. Voice analysis system. The point detection unit is
A target period including the timing at which the specific word is detected by the specific word detection unit and expected to include the start point and the end point is specified, and the sound pressure level of the target frequency in the target period is determined. The voice analysis system according to claim 1, wherein the start point and the end point are detected based on the transition. The point detection unit is
The transition of the sound pressure level of the target frequency is analyzed while going back in time from the end of the target period, and the point at which the sound pressure level of the target frequency changes from a stable transition state to an ascending state is determined. The start point is detected as the end point and is a point before the end point and changes from a state in which the sound pressure level of the target frequency decreases to a state in which the sound pressure level changes stably. The voice analysis system according to claim 5, wherein the sound is detected as a point. The point detection unit is
The transition of the sound pressure level of the target frequency is analyzed according to the passage of time from the beginning of the target period, and the point at which the sound pressure level of the target frequency changes from a stable transition state to an increasing state is described above. The end point is a point that is detected as a start point and is later in time than the start point and changes from a state in which the sound pressure level of the target frequency decreases to a state in which the sound pressure level changes stably. The voice analysis system according to claim 5, wherein the sound is detected as. A step in which the specific word detection unit of the voice analysis system detects the specific word by comparing the input input voice with the voice pattern of the pre-registered specific word.
Among the frequencies constituting the input voice input during a period continuous with the timing when the specific word is detected by the specific word detection unit, the frequency specifying unit of the voice analysis system has the highest sound pressure level. Steps to identify the corresponding frequency as the target frequency,
The point detection unit of the voice analysis system is based on the transition of the sound pressure level of the target frequency specified by the frequency identification unit in the period before and after the timing when the specific word is detected by the specific word detection unit. A step of detecting the start point and the end point of the specific word in the input voice, and
A speech analysis method characterized by including.

Images