JP5605731B2 - Voice feature amount calculation device - Google Patents

Description

  The present invention relates to a technique for comparing and evaluating a model inflection and a learner's pronunciation inflection.

In language learning, when practicing pronunciation, a learning method is widely used in which an exemplary voice recorded on a recording medium such as a CD (CompactDisk) is reproduced and pronounced by imitating the exemplary voice. The purpose of this is to acquire correct pronunciation by imitating model voices. In this kind of learning, the learner understands his / her pronunciation, recognizes the difference from the model voice and improves the pronunciation, but whether or not the same pronunciation as the model voice can be made. It is difficult for learners to listen to their own pronunciation and grasp it objectively.
Thus, for example, as disclosed in Patent Document 1, a technique has been devised that enables a learner to objectively grasp his / her voice. The language learning device disclosed in Patent Literature 1 extracts speech information such as intonation from a speech that serves as an example of pronunciation and a learner's speech. Then, the similarity between the model voice and the learner's voice is obtained, the learner's pronunciation is scored based on the similarity, and the score such as inflection is displayed on the display. According to the technique disclosed in Patent Document 1, the learner's pronunciation is objectively evaluated and the evaluation result is displayed, so that the learner knows whether or not his / her confidence pronunciation is close to a model. Can do.

JP 2000-347560 A

  Patent Document 1 discloses a method for judging the similarity of intonation by superimposing a curve showing inflection of a model voice and a curve showing inflection of a learner's voice. It is possible to know whether the pronunciation is close to a model. However, it is only possible to know whether or not they are similar only by the degree of similarity, and it is not possible to know how to approximate the pronunciation of the model. For this reason, until a pronunciation that matches the pronunciation of the model can be achieved, it is necessary to carry out a persistent learning that repeats improvement and evaluation of pronunciation by trial and error.

  The present invention has been made under the above-described background, and an object of the present invention is to provide a technique for allowing the learner's voice inflection to approach that of the model voice in language learning.

The present invention generates voice input means for inputting a voice, and a pitch curve indicating a temporal change in pitch of the voice inputted to the voice input means, and the generated pitch curve path has a magnitude of inflection. There is provided an audio feature amount calculation device having a calculation unit that calculates a parameter that represents a path, and an output unit that outputs a route calculated by the calculation unit.
In this aspect, the calculation means generates a pitch curve by interpolating the pitch of the unvoiced section according to the pitch of the voice before and after the unvoiced section for the unvoiced section of the voice input to the voice input means. It may be.

  According to the present invention, in language learning, a learner can approximate the inflection of speech to the inflection of a model speech.

It is the figure which showed the hardware constitutions of the language learning apparatus which concerns on embodiment of this invention. It is the figure which illustrated the format of example sentence table TB1. It is the figure which illustrated the structure of the functional block implement | achieved when CPU102 concerning 1st Embodiment runs a program. It is the flowchart which showed the flow of the process which CPU102 concerning 1st Embodiment performs. It is a figure for demonstrating the process of step SA6. It is the figure which illustrated the structure of the functional block implement | achieved when CPU102 concerning 2nd Embodiment runs a program. It is the flowchart which showed the flow of the process which CPU102 concerning 2nd Embodiment performs. It is the figure which illustrated the pitch curve of a learner's voice, and the pitch curve of a model voice.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
[Configuration of the embodiment]
FIG. 1 is a block diagram illustrating a hardware configuration of a language learning device 1 according to an embodiment of the invention. As shown in FIG. 1, each unit of the language learning device 1 is connected to a bus 101, and signals and data are exchanged between the units via the bus 101.

  The microphone 109 is connected to the sound processing unit 108, converts input sound into an analog electrical signal (hereinafter referred to as a sound signal), and outputs the analog signal to the sound processing unit 108. The speaker 110 is connected to the sound processing unit 108 and outputs a sound corresponding to the signal output from the sound processing unit 108. The audio processing unit 108 converts the audio signal input from the microphone 109 into digital data (hereinafter referred to as “learner data”) and outputs it, or converts the digital data representing the audio into an analog audio signal, and the speaker. The function to output to 110 is provided.

  The display unit 106 includes, for example, a display device such as a liquid crystal display, and displays a character string, various messages, a menu screen for operating the language learning device 1, and the like under the control of the CPU 102. The input unit 107 includes an input device such as a keyboard and a mouse (both not shown), and outputs a signal corresponding to the operation content to the CPU 102 in response to a key press or a mouse operation.

  The storage unit 105 includes an HDD (Hard Disk Drive) device that permanently stores data, and stores various data. Specifically, the storage unit 105 stores learner data output from the voice processing unit 108. In addition, the storage unit 105 includes example sentence text data representing example sentences used for language learning, and digital data (hereinafter referred to as model voice data) representing voices (hereinafter referred to as model voices) when the native speaker reads the example sentences. ) Is remembered. The storage unit 105 stores the example sentence table TB1 in the format illustrated in FIG. 2, and the table corresponds to the example sentence text data, the file name of the model voice data, and the identifier for uniquely identifying each example sentence text data. It is attached and stored.

  A CPU (Central Processing Unit) 102 executes a program stored in a ROM (Read Only Memory) 103 using a RAM (Random Access Memory) 104 as a work area. When the CPU 102 executes the program, each part is controlled by the CPU 102, the model voice is compared with the input learner's voice (hereinafter referred to as the learner voice), and the evaluation result of the inflection of the learner voice is output. Function is realized.

  FIG. 3 is a functional block diagram showing a configuration of functions realized by executing a program. The time axis correction unit 10 corrects the voice represented by the learner data so that the pronunciation time of the voice represented by the model voice data is the same as the pronunciation time of the voice represented by the learner data stored in the storage unit 105. To do. The pitch extraction unit 20 divides the sound indicated by the input data into a plurality of frames at predetermined time intervals on the reproduction time axis, and extracts the pitch of the sound of each frame for each divided frame. The pitch interpolating unit 30 performs linear interpolation or cubic processing between frames adjacent to these frames, such as unvoiced sections and unvoiced consonant pronunciation sections, for which the pitch extraction unit 20 could not extract the pitch. Interpolation such as spline interpolation is performed to determine the pitch of these frames. The pitch curve generation unit 40 generates a pitch curve by connecting the pitch obtained for each frame by the pitch extraction unit 20 and the pitch interpolated by the pitch interpolation unit 30. The journey calculation unit 50 calculates the journey of the pitch curve generated by the pitch curve generation unit 40.

[Operation of the embodiment]
Next, the operation of this embodiment will be described. First, when the learner performs an operation to instruct display of a list of example sentences, the CPU 102 reads out example sentence text data stored in the example sentence table TB1 (FIG. 4: step SA1), and displays a list of example sentences represented by the read data. The information is displayed on the display unit 106 (step SA2). Thereafter, when the learner operates the input unit 107 and performs an operation of selecting one of the displayed example sentences (step SA3; YES), the CPU 102 displays the screen displayed on the display unit 106, the input unit, and the like. Based on the signal sent from 107, the selected example sentence is specified (step SA4). When CPU 102 identifies the selected example sentence, CPU 102 reads out the file name of the model voice data stored in association with the selected example sentence in example sentence table TB1 (step SA5). For example, in the table shown in FIG. 2, when an example sentence with the identifier “001” is selected, the file name “a001” is read.

  Next, the CPU 102 reads out the model voice data specified by the read file name from the storage unit 105 and extracts the pitch of the voice indicated by the read data. Specifically, first, the CPU 102 divides the voice indicated by the model voice data at a predetermined time interval (for example, 5 msec) on the playback time axis as shown in FIG. 5 (step SA6) (hereinafter, referred to as “step SA6”). Each divided section is called a frame). Next, the CPU 102 extracts the audio pitch of each frame for each divided frame (step SA7). Note that the time interval of one frame may be other time intervals such as 10 msec instead of 5 msec. When the CPU 102 extracts the pitch for each divided frame, the CPU 102 generates a pitch curve connecting the pitches determined for each frame (hereinafter, this pitch curve is referred to as a first pitch curve), and the generated first pitch curve. Is stored in the storage unit 105 (step SA8). Note that, in a frame where the pitch cannot be extracted, such as an unvoiced section or an unvoiced consonant pronunciation section, interpolation such as linear interpolation or cubic spline interpolation is performed to generate a first pitch curve.

  When the generation of the first pitch curve is completed, the CPU 102 reads out the model voice data specified by the read file name from the storage unit 105, and outputs the read out model voice data to the voice processing unit 108 (step SA9). When the model voice data is input to the voice processing unit 108, the model voice data which is digital data is converted into an analog signal and output to the speaker 110, and the model voice is reproduced from the speaker 110.

  When the reproduction of the model voice is finished, the CPU 102 controls the display unit 106 to display, for example, a message for prompting pronunciation of the example sentence such as “Press the key to pronounce and then press the key again when the pronunciation is finished” ( Step SA10). The learner listens to the model voice output from the speaker 110, and then operates the input unit 107 according to the message to read the example sentence by imitating the model voice. When the learner pronounces, the learner's voice is converted into a voice signal by the microphone 109, and the converted signal is output to the voice processing unit 108. When the audio signal output from the microphone 109 is input, the audio processing unit 108 converts the audio signal into learner data that is digital data. This learner data is output from the voice processing unit 108 and stored in the storage unit 105.

  Next, the CPU 102 monitors a signal sent from the input unit 107 and determines whether or not the learner has finished pronunciation. When the learner finishes the pronunciation and operates the input unit 107 (step SA11; YES), the CPU 102 generates the pronunciation time of the voice represented by the model voice data and the pronunciation of the voice represented by the learner data stored in the storage unit 105. The voice represented by the learner data is corrected so that the time is the same (step SA12).

  Next, the CPU 102 divides the voice indicated by the learner data into a plurality of frames on the reproduction time axis in the same manner as the process of step SA6 (step SA13), and the voice of each frame is divided for each divided frame. The pitch is extracted (step SA14). When CPU 102 extracts the pitch for each frame, CPU 102 generates a pitch curve (hereinafter referred to as a second pitch curve) connecting the pitches determined for each frame in the same manner as in step SA8, and generates the generated second pitch curve. Is stored in the storage unit 105 (step SA15). In this case as well, in an unvoiced section, an unvoiced consonant pronunciation section, etc., interpolation such as linear interpolation or cubic spline interpolation is performed to generate a second pitch curve.

  When the generation of the second pitch curve is completed, the CPU 102 calculates the path of the first pitch curve, and then calculates the path of the second pitch curve (step SA16). The distance of the pitch curve can be obtained by calculating the integral of the absolute value of the first derivative of the pitch curve f (t) as shown in FIG. As shown in FIG. 8, the voice with a large inflection change has a large pitch change, so the pitch curve has a long path, while the voice with a small inflection has a small pitch change, so the pitch curve The journey is shortened. That is, it can be said that the distance of the pitch curve represents the magnitude of the inflection.

  Next, the CPU 102 compares the path of the first pitch curve with the path of the second pitch curve (step SA17). Since the pitch curve path represents the magnitude of the inflection change, by comparing the pitch curve paths, it is known which of the inflection changes is greater between the model voice inflection and the learner voice inflection. be able to. As shown in FIG. 8, when the path of the first pitch curve is longer than the path of the second pitch curve, the change amount of the inflection of the learner speech is smaller than the change amount of the inflection of the model speech. The message “It is a voice with little change” is displayed on the display unit 106, and information useful for improving the pronunciation is output to the learner (step SA18). Also, if the path of the second pitch curve is longer than the path of the first pitch curve, the amount of change in the inflection of the learner's voice is greater than the amount of change in the inflection of the model voice. Is displayed on the display unit 106, and information useful for improving pronunciation is output to the learner (step SA18). In addition, when the journey of the first pitch curve and the journey of the second pitch curve are the same, the change amount of the inflection of the learner's voice is the same as the change amount of the inflection of the model voice. Is displayed on the display unit 106 (step SA18).

  When the CPU 102 finishes outputting the message to the learner, the CPU 102 controls the display unit 106 to display a menu screen for confirming whether to practice pronunciation of the selected example sentence again or practice pronunciation of another example sentence ( Step SA19). When the learner operates the input unit 107 and performs an operation for practicing pronunciation of another example sentence (step SA20; YES), the CPU 102 returns the process flow to step SA1, and performs the processes after step SA1. Run again. When the learner operates the input unit 107 to perform the pronunciation practice of the selected example sentence again (step SA21; YES), the CPU 102 returns the process flow to step SA6, and after step SA6. The above process is executed again.

  As described above, according to the present embodiment, how to improve the pronunciation is output to the learner, so that the pronunciation is not repeated by repeating trial and error to improve and evaluate the pronunciation. Can be brought closer to the model voice.

[Second Embodiment]
[Configuration of the embodiment]
Next, a second embodiment of the present invention will be described. The hardware configuration of the language learning device 1A according to the present embodiment is the same as that of the language learning device 1 according to the first embodiment. For this reason, the description of the hardware configuration is omitted. In the present embodiment, the function realized by the CPU 102 executing the program is different from the first embodiment.

  FIG. 6 is a functional block diagram showing a configuration of functions realized by the CPU 102 executing the program. In FIG. 6, the same functional blocks as those in the first embodiment are denoted by the same reference numerals as those in FIG. 3, and the description thereof is omitted. The Hz → Cent conversion unit 60 converts the pitch expressed in Hz into Cent. The filter unit 70 functions as a low-pass filter and removes a minute change in pitch.

[Operation of the embodiment]
Next, the operation of this embodiment will be described. FIG. 7 is a flowchart illustrating the processing flow of the CPU 102 in the present embodiment. In FIG. 7, the same processes as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment.

  When the learner selects an example sentence, the CPU 102 reads out model voice data corresponding to the selected example sentence from the storage unit 105 (steps SA1 to SA5). Then, the voice indicated by the model voice data is divided at predetermined time intervals on the reproduction time axis (step SA6). Next, the CPU 102 extracts the audio pitch of each frame for each divided frame (step SA7). When the CPU 102 extracts the pitch for each divided frame, it converts the unit of the extracted pitch from Hz to Cent (step SB1). When the conversion from Hz to Cent is completed, the CPU 102 removes minute changes in pitch such as fluctuations in pronunciation for each frame (step SB2). Then, the CPU 102 generates a pitch curve (first pitch curve) connecting the pitches determined for each frame, and stores the curve data indicating the generated first pitch curve in the storage unit 105 (step SA8). Note that, in a frame where the pitch cannot be extracted, such as an unvoiced section or an unvoiced consonant pronunciation section, interpolation such as linear interpolation or cubic spline interpolation is performed to generate a first pitch curve.

  Thereafter, the CPU 102 displays a message for prompting pronunciation of the example sentence (step SA10). When the learner reads the example sentence by imitating the model voice, the learner voice is converted into learner data. The CPU 102 monitors the signal sent from the input unit 107, and when the learner finishes pronunciation and operates the input unit 107 (step SA <b> 11; YES), the sound generation time of the voice represented by the model voice data and the storage unit 105. The voice represented by the learner data is corrected so that the pronunciation time of the voice represented by the stored learner data is the same (step SA12).

Next, the CPU 102 divides the voice indicated by the learner data into a plurality of frames on the reproduction time axis in the same manner as the process of step SA6 (step SA13). Then, the CPU 102 extracts the audio pitch of each frame for each divided frame (step SA14). When the CPU 102 extracts a pitch for each divided frame, the CPU 102 converts the extracted pitch unit from Hz to Cent (step SB3). When the conversion from Hz to Cent is completed, the CPU 102 removes minute changes in pitch such as fluctuations in pronunciation for each frame (step SB4). Then, the CPU 102 generates a pitch curve (second pitch curve) connecting the pitches obtained for each frame, and stores the curve data indicating the generated second pitch curve in the storage unit 105 (step SA15). Note that, in a frame where the pitch cannot be extracted, such as an unvoiced section or an unvoiced consonant pronunciation section, interpolation such as linear interpolation or cubic spline interpolation is performed to generate a second pitch curve.
Since the flow of processing after step SA15 is the same as that of the first embodiment, description thereof is omitted.

  As described above, this embodiment also outputs to the learner how to improve the pronunciation specifically, so that the pronunciation can be reproduced without repeating the improvement and evaluation of the pronunciation through trial and error. It is possible to approach the model voice. In addition, since the pitch is compared in units of Cent, an evaluation closer to human hearing is possible.

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. For example, the present invention may be implemented by modifying the above-described embodiment as follows.

In the above-described embodiment, the process of step SA12, that is, the process of matching the pronunciation time of the model voice and the pronunciation time of the learner voice may not be performed. In the process of step SA12, the pronunciation time of each word in the model voice may be the same as the pronunciation time of each word in the learner voice. In the process of step SA12, the pronunciation time of each phoneme in the model voice may be the same as the pronunciation time of each phoneme in the learner voice.
Further, when comparing the distances of the first pitch curve and the second pitch curve, the time interval to be compared may be a whole example sentence section, a part of example sentence specified in advance, or a part of section specified by the user. It may be.

  In the embodiment described above, the path of the first pitch curve is compared with the path of the second pitch curve, and according to the comparison result, for example, “Let's pronounce with a little inflection” May be displayed. In the above-described embodiment, for example, a word in speech may be recognized, and an inflection evaluation result may be displayed for each word.

  In the above-described embodiment, a pitch curve may be generated only for the learner's voice to determine the pitch curve path, and the calculated path may be output.

DESCRIPTION OF SYMBOLS 1 ... Language learning apparatus, 101 ... Bus, 102 ... CPU, 103 ... ROM, 104 ... RAM, 105 ... Memory | storage part, 106 ... Display part, 107 ... Input unit 108... Audio processing unit 109. Microphone 110 110 Speaker

Claims (2)

Voice input means for inputting voice;
A calculation means for generating a pitch curve indicating a temporal change in the pitch of the voice input to the voice input means, and calculating a path of the generated pitch curve as a parameter representing the magnitude of the inflection ;
An audio feature quantity calculation apparatus comprising: output means for outputting a journey calculated by the calculation means. The calculation means generates a pitch curve by interpolating the pitch of the unvoiced section according to the pitch of the voice before and after the unvoiced section for the unvoiced section of the voice input to the voice input means. Item 2. The audio feature amount calculation apparatus according to Item 1.

Images