JP7432879B2 - speech training system - Google Patents

Description

The present invention analyzes the voice (words) uttered by the user, determines whether the user's voice is easy to hear for elderly and middle-aged people, and then outputs the voice with simulated hearing loss. The present invention relates to a training system for the user to learn a speaking method that is easier for the user to hear.

Japan's aging rate continues to be extremely high, and it has become an extremely important issue to be able to communicate smoothly with the elderly, not only in daily life but also in the business field.

It is known that a decline in the ability to understand speech due to a decline in the frequency resolution of hearing occurs not only in the elderly, but also in the middle-aged generation. , has become a major problem in social life and business settings.

In order to solve these problems, it is necessary to improve the way we speak during conversations and to make our voices easier to hear even for elderly and middle-aged people. In particular, it is known from psychoacoustic theory that consonants need to be uttered longer and more clearly than vowels included in speech, especially for elderly and middle-aged people whose ability to hear speech has declined.

In recent years, the use of simulating hearing loss technology that converts the voice spoken by a user into the voice that an elderly person would likely hear has been used to allow the user to experience how the other person is hearing them, and to increase the awareness of elderly and middle-aged people. There are speech training applications and the like that are used to utter sounds that are easy for people to hear. There is also a movement to divert speech training applications used for language learning to speech training for elderly people.

Patent Document 1 discloses a first audio data storage unit that stores first audio data representing audio, a sound collection unit that outputs second audio data representing collected audio, and a first audio data storage unit that stores first audio data that represents audio. Comparing the characteristics of the first audio data stored in the data storage means and the characteristics of the second audio data output from the sound collecting means, and identifying the pointed out section based on the comparison result. means, in the first audio data, an aspect of the audio represented by the audio data corresponding to the pointed out section specified by the pointed out section specifying means, and an aspect of the audio represented by the audio data corresponding to the section other than the pointed out section; a sound data processing means for processing the first sound data such that the first sound data is different; and a sound emission control means for causing the sound emitting means to emit the sound represented by the first sound data processed by the sound data processing means. Disclosed is a voice evaluation device characterized by comprising:

Patent Document 2 discloses a signal processing device that generates a time-varying filter whose filter characteristics change at each point in time, and uses the generated time-varying filter to process an input signal that is a temporally changing sound signal. The processing unit includes a processing unit that obtains an output signal, and the processing unit outputs a first auditory spectrogram obtained by passing the input signal through a first auditory filter bank that reflects compression characteristics of a first listener, and a second auditory spectrogram. and a second auditory spectrogram obtained by passing the input signal through a second auditory filter bank that reflects the compression characteristics of A signal processing device configured with the following is disclosed.

Patent application 2006-217300 Patent application 2015-27305

It is common knowledge to speak loudly when words cannot be conveyed to an elderly person, etc., but it is common sense to speak loudly when words cannot be conveyed to an elderly person. It is known from psychoacoustic theory that speaking loudly has almost no effect on improving mishearing, but rather increases discomfort due to loud voices.

Additionally, it is known from psychoacoustics that it is consonants rather than vowels that are more likely to be misheard in such cases, but many people feel that their words are not being understood. There is a tendency to vocalize vowels strongly.

The hearing loss simulation technology, which converts the voice spoken by the user into the voice that an elderly person would probably hear, allows users to simulate how their voice is heard by an elderly person, so it is difficult to shout out loud. It has the characteristic that it makes it easy for the user to realize that there is no meaning even if the consonant is pronounced, and that the pronunciation of the consonant is important.

However, even if an elderly person can realize that it is difficult to hear by simply listening to a simulated hearing-impaired voice, it may be difficult to understand which phonemes in one's own speech are causing the difficulty in hearing, and specifically how to understand the speech. There was a problem in that users themselves were unable to know how to improve their methods.

The sound evaluation device described in Patent Document 1 includes: a first sound data storage means for storing first sound data representing a sound; a sound collection means for outputting second sound data representing the collected sound; Comparing the characteristics of the first audio data stored in the first audio data storage means and the characteristics of the second audio data output from the sound collecting means, and determining the indicated section based on the comparison result. a pointed-out section specifying means for specifying, and in the first audio data, the mode of sound expressed by the audio data corresponding to the pointed-out section specified by the pointed-out section specifying means, and the audio data corresponding to sections other than the pointed out section. a sound data processing means for processing the first sound data, and a sound represented by the first sound data processed by the sound data processing means, emitted to a sound emitting means so that the form of the sound to be expressed is different from that of the sound expressed by the sound data processing means; and a sound emission control means. This makes it easier for the user to understand the scoring results for language learning and the like.

However, users cannot experience how their voices are heard by elderly people, and this does not motivate them to perform speech training. Furthermore, it is necessary to specifically inform the user which phoneme in his or her speech is bad and how to improve the speech, but there is no suggestion or disclosure of a method for doing so.

The signal processing device described in Patent Document 2 generates a time-varying filter whose filter characteristics change at each point in time, and uses the generated time-varying filter to convert an input signal, which is a sound signal that changes over time, into an output signal. a first auditory spectrogram obtained by passing the input signal through a first auditory filter bank in which compression characteristics of a first listener are reflected, and a compression characteristic of a second listener. and a second auditory spectrogram obtained by passing the input signal through a second auditory filter bank whose characteristics are reflected, and is configured to generate the time-varying filter at each time point. ing. This makes it possible to generate simulated hearing-impaired speech that more appropriately reflects the auditory characteristics of a hearing-impaired person or the like.

However, users cannot know specifically which phonemes in their own speech are bad and how to improve their speech, and there are no suggestions or disclosures regarding how to do so. None.

As a means for solving the above-mentioned problems, the speech training system of the present invention includes a sound collection unit for collecting the user's voice, and a duration timer for extracting the duration of each phoneme of the collected user's voice. an extraction unit, a duration holding unit that extracts and/or holds the duration of each phoneme of a model voice recorded in advance, a simulating hearing loss converting unit that converts the collected user's voice into a simulating hearing loss; The device is configured to include a duration comparison unit that compares the duration of each phoneme of the user's voice with the duration of each phoneme of the model voice, and a comparison result presentation unit that presents the comparison result of the duration comparison unit to the user. .

This makes it possible for users to experience how their voices are heard by elderly people, giving them a high level of motivation to perform speech training, as well as learning which phonemes are difficult to pronounce and how to improve their speech. You can know what needs to be improved.

Moreover, the speech training system of the present invention is configured to include a comparison result reproduction section that reproduces only the specific phoneme presented by the comparison result presentation section. This allows the user to learn in more detail which phoneme is poorly uttered and how to improve the utterance while actually listening to the phoneme.

According to the speech training system according to the present invention, it is possible to improve the method of speech during a conversation and to produce a voice that is easy to hear even for elderly people and middle-aged people.

While listening to the user's own voice converted into a simulated hearing loss, the user can understand the difference in duration of each phoneme from the model uttered voice. It is possible to improve the ability to speak the consonants contained in the words.

Furthermore, since it is possible to playback and listen to only the phonemes that are particularly difficult to convey in the voice that the user has uttered, it is possible to pay particular attention to the utterance of the phonemes in question, making it possible to further improve speaking ability. .

Block diagram in the first embodiment of the present invention Block diagram in the second embodiment of the present invention An example of a user screen in the first embodiment An example of a screen of the comparison result presentation section in the first embodiment An example of a screen for listening to simulated hearing loss converted speech in the first embodiment An example of a user screen in the second embodiment

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the best mode for carrying out the present invention will be described in detail based on the drawings. In the following description, parts having the same function are designated by the same reference numerals, and repeated description thereof will be omitted.

FIG. 1 is a block diagram of a system according to a first embodiment of the present invention, including a sound collection unit 2 that collects voices uttered by a user, and a duration of each phoneme of the collected user's voice. a duration extraction unit 3 that extracts the duration of each phoneme of a pre-recorded model voice, a duration holding unit 4 that extracts and/or holds the duration of each phoneme of a model voice recorded in advance, and converts the collected user's voice into a simulated hearing loss. A simulated hearing loss conversion unit 5, a duration comparison unit 6 that compares the duration of each phoneme of the user's voice with a duration of each phoneme of the model voice, and a comparison result of the duration comparison unit is presented to the user. It is composed of a comparison result presentation section 7.

The user 1 utters predetermined task audio content to be presented on a user screen or the like to the present system. Here, this system may be dedicated hardware, or may be a smartphone terminal, a personal computer, or the like.

The sound uttered by the user 1 is collected by the sound collection unit 2. The sound collection unit 2 may be the dedicated hardware, a smartphone terminal, a microphone built into a personal computer, etc., or may be another sound collection device procured by the user 1 himself. Further, the sound collection unit 2 may have a recording function, and may record and save the voice of the user 1 in advance and then send the audio data to the duration extraction unit 3, or the voice collected by the sound collection unit 2 may be The configuration may be such that the data is sent to the duration extraction section 3 as is.

The duration extraction unit 3 analyzes the voice of the user 1 collected by the sound collection unit 2, divides the voice into each phoneme included therein, and extracts the duration of each phoneme. Speech segmentation techniques such as DP matching and HMM (Hidden Markov Model) are used to analyze and divide the duration of each phoneme.

The duration holding unit 4 records the duration of each phoneme of a voice with the same content as the task voice of a model speaker who can utter a voice that is easy to hear even for elderly and middle-aged people. . Note that the duration holding unit 4 does not record the duration of each phoneme in advance, but records the voice data of the model speaker, and the duration extraction unit 3 analyzes it each time. The configuration may also be such that the results are recorded in the duration holding section 4.

The duration comparison unit 6 compares the duration of each phoneme included in the voice uttered by the user 1 extracted by the duration extraction unit 3 and the model speaker recorded in the duration storage unit 4. Compare the duration of each phoneme in the speech of each phoneme. As a comparison method, a threshold value is set in advance for the difference between both durations, and information about phonemes that are longer or shorter than that threshold is sent to the comparison result presentation unit 7, or the duration of both A possible method is to calculate the ratio of , and set a threshold value for that ratio.

The comparison result presentation unit 7 presents the comparison result of the duration of each phoneme output from the duration comparison unit 6 to the user 1. The presentation method may be a method of displaying the phonemes included in the task speech content in text and clearly indicating the length of the duration of each phoneme compared to the voice of a model speaker. A method of illustrating the waveform, sound spectrogram, etc. of a model speaker's voice, and then illustrating the section to which each phoneme corresponds in the waveform, etc., to visually clarify the merits and demerits of each phoneme to the user 1. You may also use

On the other hand, the simulated hearing loss conversion unit 5 performs a simulated hearing loss conversion on the voice uttered by the user 1 that has been collected by the sound collection unit 2, and transforms the voice into a voice that would be heard by elderly and middle-aged people. Convert. Examples of simulated hearing loss conversion methods include a method of simulating the spread of the auditory filter of elderly and middle-aged people using signal processing in the frequency domain, and synthesizing it using FFT and overlap add processing, and a method described in Patent Document 2. etc. are possible.

The user 1 can listen to the simulated hearing loss converted audio by pressing a play button on the user screen or the like. Furthermore, if a playback button is provided to listen to the original voice that has not been converted into a simulated hearing loss, the original voice of the model speaker's voice, and the simulated hearing loss converted voice, user 1 will be able to listen to a variety of voices, and Be able to know the weak points of the student's speech and carry out specific training to improve them.

3, 4, and 5 show examples of screens presented on a smartphone screen when the speech training system of the present invention is implemented as a smartphone application.

User 1 presses the recording button according to the screen shown in FIG. 3, and then speaks into the smartphone the training task voice content (in this example, "Thank you, always") displayed at the top of the screen. A microphone built into the smartphone functions as a sound collection unit 2 and collects the content of the user's 1 utterance.

The duration extraction unit 3 of the smartphone application performs analysis processing such as DP matching on the collected voice of the user 1, and extracts the phonemes “i”, “ts”, “u” that are “Thank you always”. ”, ”m”, “o”, “ ”, ”a”, “r”, “i”, ”g”, “a”, “t”, ”o”, “ ” and divide each phoneme into Extract the duration (intervals without phoneme notation are between words).

The duration holding unit 4 records the duration of each phoneme of the voice of "Always, thank you" by a model speaker who can utter a voice that is easy to hear even for elderly and middle-aged people. A duration comparison section 6 compares the duration of each phoneme in both cases.

FIG. 4 is an example of the comparison result presentation section 7. Here, we present only the results for the consonant "ts" in "tsu" in "always" and "a" in "arigatou", for which the difference in duration was particularly large, and the results for improving speech. Displaying advice. In this example, as the output of the duration comparison unit 6, in particular, "ts" uttered by user 1 has a shorter duration than that of the model speaker, and "a" has a too long duration, so The children are encouraged to speak each phoneme carefully and with their mouths wide open.

FIG. 5 is a screen for the user 1 to listen to the simulated hearing loss converted sound and experience how the spoken sound is heard by elderly people and middle-aged people. It is possible to listen to the voice uttered by the user 1 and the simulated hearing loss converted voice and original voice of the model speaker, respectively. User 1 can listen to how elderly and middle-aged people actually hear the phonemes pointed out in Figure 4. It is possible to carry out speech improvement training with specificity.

FIG. 2 is a block diagram of a system according to a second embodiment of the present invention, including a sound collection unit 2 that collects the voice uttered by a user, and the duration of each phoneme of the collected user's voice. a duration extraction unit 3 that extracts the duration of each phoneme of a pre-recorded model voice, a duration holding unit 4 that extracts and/or holds the duration of each phoneme of a model voice recorded in advance, and converts the collected user's voice into a simulated hearing loss. A simulated hearing loss conversion unit 5, a duration comparison unit 6 that compares the duration of each phoneme of the user's voice with a duration of each phoneme of the model voice, and a comparison result of the duration comparison unit is presented to the user. It is comprised of a comparison result presentation section 7 and a comparison result reproduction section 8 that reproduces only specific phonemes presented by the comparison result presentation section 7.

The comparison result reproduction unit 8 has a function of reproducing only the phonemes displayed by the comparison result presentation unit 7 that have a particularly large difference in duration.

FIG. 8 shows an example of a screen displayed on a smartphone screen when the comparison result playback section 8 according to the second embodiment of the present invention is implemented as a smartphone application.

Here, as an example, a speech waveform is illustrated, and phoneme sections with particularly large differences in duration are clearly indicated by hatching, and when the shaded portion is tapped, that phoneme section is played back. When actually playing back, so that user 1 can easily hear the phoneme section, playback is started several milliseconds to several hundred milliseconds before the phoneme section, and several milliseconds before the phoneme section. It would be necessary to end the playback several hundred milliseconds after the start, or to include some of the preceding and succeeding phonemes.

Furthermore, in this embodiment, the phoneme section is clearly indicated by the waveform and shading, but this is not limited to the method of illustrating by the waveform, and it is also possible to use a speech power fluctuation diagram or sound spectrogram instead of the waveform. It is also possible to display the phoneme in text and play only that section.

By the way, in this embodiment, the length of the duration of each phoneme in the voices of the user 1 and the model speaker outputted by the duration comparison unit 6 is displayed as is in the comparison result presentation unit 7. When humans listen to speech and try to recognize its content, they do not concentrate equally on all phonemes.

In terms of psychoacoustics, it has been reported that in order for humans to accurately recognize the content of the speech they hear, it is most important to accurately hear the consonant at the beginning of a word. Therefore, each phoneme of the task speech content is weighted in advance, and the comparison result presentation unit 7 clearly indicates the initial consonant of each clause even if there is a slight difference in length, while encouraging the user 1 to train. Regarding vowels at the end of words, even if the difference in length is large, the comparison result presentation section 7 may not display the vowels.

For example, if the task audio content presented on the user screen etc. is “Hello “k” “o” “n” “n” “i” “ch” “i” “w” “a”, then the most important Regarding "k", which is the initial consonant of a word, the difference in duration between the utterances of the user 1 and the model speaker calculated by the duration comparison unit 6 is doubled and compared with a threshold value to find the slight difference. It is also possible to prompt user 1 to train even if the difference is 1/2, and compare it with the threshold, so that training is not prompted even if there is a slight difference. It is.

Note that the voice of a speaker trained by the speech training system of the present invention becomes easier to hear not only for middle-aged and elderly people but also for hearing-impaired people in general (including young hearing-impaired people). It can also be used as a system for training people with hearing loss in general to make their speech easier to understand.

1... User, 2... Sound collection section, 3... Duration extraction section, 4... Duration holding section, 5... Simulated hearing loss conversion section, 6... Duration comparison section, 7... Comparison result presentation section, 8...
Comparison result playback section.

Claims (2)

This is a speech training system for uttering a voice that is easy to hear for elderly and middle-aged people , which includes a sound collection unit for collecting the user's voice, and a voice training system for each phoneme of the collected user's voice. a duration extracting unit that extracts the duration; a duration holding unit that extracts and/or holds the duration of each phoneme of a model voice recorded in advance; A simulated hearing loss conversion unit that simulates how hearing loss sounds to people of the older generation , a play button for listening to the audio converted by the simulated hearing loss conversion unit, and a continuation of each phoneme of the user's voice. A speech training system comprising a duration comparison section that compares time and duration of each phoneme of the model voice, and a comparison result presentation section that presents advice for conversation improvement to the user based on the comparison result of the duration comparison section. . 2. The speech training system according to claim 1, further comprising a comparison result reproduction section that reproduces only specific phonemes presented by the comparison result presentation section.

Images