KR101840363B1 - Voice recognition apparatus and terminal device for detecting misprononced phoneme, and method for training acoustic model - Google Patents

Abstract

The present invention relates to a speech recognition system for improving the ability to distinguish error phonology generated by native language interference from standard phonemes of native speakers and thereby improve pronunciation evaluation ability in foreign language learning, The present invention relates to a method and apparatus for performing speech recognition on speech data of a user input using a previously stored acoustic model, and in the case where the speech recognition result is an error, whether the error is an error due to native language interference, , Discrimination learning of the acoustic model is performed in such a direction as to increase the difference between the erroneous pronunciation generated by the mother language interference and the recognition result for the word string corresponding to the user's voice data.

Description

TECHNICAL FIELD [0001] The present invention relates to a terminal and a speech recognition device for detecting an error sound, and a learning method of the acoustic model,

The present invention relates to speech recognition, and more particularly, to a speech recognition system for improving the ability to discriminate between an erroneous pronunciation caused by mother language interference and a standard pronunciation of a native speaker, Device, and its acoustic model learning method.

Speech recognition technology extracts phonological and verbal information from acoustical information included in speech and allows the machine to recognize it. It is a technology that can recognize only a specific speaker according to the type of person who can recognize it, And a speaker-independent technology targeting an unspecified majority. There is a key word detection technology for detecting and recognizing only a specific vocabulary, which is divided into isolated words, connected words, continuous sentences, and consecutive sentence recognition techniques according to pronunciation forms . In addition, depending on the number of vocabulary, it can be categorized as a small scale which handles several hundred words or less, a medium scale of several thousand words, and a large capacity recognition technology capable of recognizing tens of thousands of words.

The speech recognition process will be schematically described. First, a feature vector necessary for recognition is extracted from the voice uttered by the user through sound processing. In this case, LPC (Linear Predictive Coding) and MFCC (Mel Frequency Cepstral Coefficients) are mainly used as feature vectors. The recognition result is then obtained by comparing the extracted feature vector with the trained reference pattern. Here, the most common method of speech recognition is pattern recognition, Dynamic Time Warping (DTW) using a template-based pattern matching method, and HMM (Hidden Markov Model) using statistical pattern recognition. In addition, a method using a neural network is also used.

Here, the HMM constructs a model representing these from the training data using the temporal statistical characteristics of the speech signal, and then adopts a probability model having a high degree of similarity to the actual speech signal as the recognition result. It is easy to implement up to speech recognition and has good recognition performance and is widely used in various application fields.

In order to apply the HMM to speech recognition, firstly, when an order sequence of an arbitrary observation symbol is obtained, such an observation The problem of calculating the probability of occurrence of heat, the problem of finding the optimal state column, and the problem of estimating the model parameter that can best express the observation column.

Here, the first problem can be solved by using a forward-backward algorithm, the second problem can be solved by applying a viterbi algorithm, and the third can be solved by Baum-Welch method and segmental K-means method. .

On the other hand, continuous speech speech recognition is a method of finding sentences or consecutive word strings corresponding to speech signals, unlike the case of isolated word recognition, in which an acoustic model, which is an observation probability of an acoustic signal for a word string, and a language model, to be.

Especially, in the acoustic model, the phoneme is often used as the minimum unit, and it is known that the triphone model considering the before and after phonemes is the best in terms of performance.

In addition, ML (Maximum Likelihood) method is widely used as a training method of acoustic models. Because it considers only the optimization of the model allocated to itself, it does not consider how different from other models, Provide a cause of degradation.

In addition, in the foreign language learning, which is a typical application field in which the speech recognition technology is used, the data used in the foreign language learning includes a lot of error data. In this case, Incorrect errors caused by incorrect pronunciation of phonemes due to native language interference are also included.

Therefore, it can be expected that the performance in foreign language learning will be improved if it is possible to distinguish phonemes erroneously spoken by native language interference and phonemes spoken by native speakers.

The present invention relates to a terminal and a voice recognition device for improving the ability to distinguish an error phonology generated by mother language interference from a standard phoneme generated by native language interference and thereby to improve a pronunciation evaluation ability in foreign language learning, And to provide an acoustic model learning method.

According to an aspect of the present invention, there is provided a voice recognition apparatus for performing voice recognition on voice data of a user input using a pre-stored acoustic model in response to a voice recognition request from a terminal, part; When the speech recognition result is an error, it is confirmed whether the error is an error due to native language interference, and when the error is due to native language interference, the difference between the speech recognition result and the error pronunciation generated by the mother- An acoustic model learning unit for performing discrimination learning of the acoustic model; And a storage unit for storing an acoustic model.

In the speech recognition apparatus according to the present invention, the storage unit further stores a phonological error dictionary that defines an error phoneme that can be generated by the mother language interference, and the acoustic model learning unit compares the phonetic data dictionary with the speech data, If the phoneme corresponding to the error phoneme is included in the voice data, the error can be judged as an error due to the native language interference.

In the speech recognition apparatus according to the present invention, when the error of the recognition result is not an error caused by the native language interference, the acoustic model learning unit determines that the speech recognition itself is an error, It is possible to perform discrimination learning in the direction of increasing the size of the image.

In the speech recognition apparatus according to the present invention, the speech recognition unit may further detect an error pronunciation of the user's voice data by using the acoustic model, and may provide the terminal with the result of the error pronunciation detection.

According to another aspect of the present invention, there is provided an information processing apparatus comprising: an input unit for receiving a request from a user; An audio processor for converting a user's voice signal into voice data and outputting the voice data; When a user's request through the input unit or a voice recognition request based on a request of an application program occurs, voice recognition is performed on the voice data of the user outputted from the audio processing unit using the pre-stored acoustic model, A recognition module for recognizing whether or not the error is an error due to native language interference when the result of speech recognition is an error; if the error is due to native language interference, And an acoustic model learning module for performing discrimination learning in a direction of increasing a difference between the speech recognition result and the speech recognition result. An output unit for outputting speech recognition results to a user; And a storage unit for storing the acoustic model.

In the terminal according to the present invention, the storage unit further stores a phonological error dictionary defining an error phoneme that can be generated by the first language interference, and the acoustic model learning module compares the phonetic data dictionary with the speech data, When the phoneme corresponding to the error phoneme is included in the voice data, the error can be judged as an error due to the native language interference.

The terminal according to the present invention may further include a communication unit for transmitting and receiving data through a network and the control unit may receive a plurality of acoustic model and phoneme error dictionary generated by the speech recognition apparatus through the communication unit and store the received acoustic model and phonetic error dictionary in the storage unit.

In the terminal according to the present invention, the control unit may further include a preprocessing module for extracting a feature vector from the user's speech data output from the audio processing unit, wherein the speech recognition module applies the feature vector to the acoustic model, Word strings similar to voice data can be extracted.

In the terminal according to the present invention, the voice recognition module can further detect and provide erroneous pronunciation included in the recognized voice data.

According to the present invention, in the speech recognition system, the learning of the acoustic model can be performed so as to clearly distinguish the difference between the error pronunciation of the user and the standard pronunciation of the actual native speaker by the mother language interference. As a result, It is possible to improve the recognition performance of the discrimination ability between the correct pronunciation and the correct pronunciation and thereby improve the accuracy and reliability of the pronunciation evaluation of the user in language learning such as a foreign language.

1 is a block diagram schematically illustrating a speech recognition system according to the present invention.
2 is a block diagram illustrating a configuration of a speech recognition apparatus according to an embodiment of the present invention.
3 is a block diagram illustrating a configuration of a UE according to another embodiment of the present invention.
4 is a flowchart illustrating a speech recognition process according to the present invention.
5 is a flowchart illustrating a learning process for an acoustic model in speech recognition according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The speech recognition according to the present invention can be applied to various applications. For example, when applied to a language learning field such as a foreign language, the performance of the language evaluation may be improved, but the present invention is not limited to the application to the language learning field. In addition, the learning of the speech recognition and the acoustic model according to the present invention may be realized as a server-client type or a stand-alone type of a terminal. In the following embodiments, both of these types will be described.

1 is a block diagram illustrating a speech recognition system according to an embodiment of the present invention. Here, speech recognition is implemented through a server-client system.

Referring to FIG. 1, the speech recognition system of the present invention may be implemented through a speech recognition apparatus 100 and a terminal 200 that are linked through a network 10.

Here, the network 10 is not limited to any type of network capable of data communication. For example, the network 10 provides a large-capacity data transmission / reception service through an internet protocol (IP) (IP) network, which is an i-bell-net structure in which different networks are integrated. The network 10 also includes a 3G mobile network including a wired network, a Wibro (Wireless Broadband) network, a WCDMA, a 3.5G mobile network including a High Speed Downlink Packet Access (HSDPA) network and an LTE network, and LTE advanced , A fourth generation mobile network, a satellite network, and a wireless LAN including a Wi-Fi network.

The speech recognition apparatus 100 is a server apparatus for performing a speech recognition service. The speech recognition apparatus 100 detects speech recognition and error pronunciation of a user speech inputted based on a stored acoustic model in response to a request from the terminal 200, And provides the result to the terminal 200.

The acoustic model is learned through discrimination learning in accordance with the present invention. Specifically, error data having an error in the recognition result out of the speech recognition results is collected, and it is determined whether the collected error data is an error caused by native language interference, Extracting error pronunciations caused by mother language interference of words corresponding to the error data in the case of an error due to mother language interference, and increasing the difference between the error pronunciation and the recognition result And discrimination learning is performed in a direction of increasing the difference from the original correct answer in the case of an error of speech recognition itself. That is, in the case of universal discrimination learning, re-learning is performed in a direction of increasing the difference between the incorrect answer and the correct answer, thereby reducing the overlap area between the incorrect answer and the correct answer, If the result is discriminated as an error, discrimination learning is performed only in the direction of increasing the difference between the correct answer and the incorrect answer, and pronunciation errors can not be detected by the mother language interference. Therefore, in the present invention, when the speech recognition result is determined as an error due to the pronunciation error due to the mother language interference of the user, re-learning is performed in a direction of increasing the difference between the error pronunciation generated by the mother language interference Thereby improving the detection performance of pronunciation errors due to native language interference.

The speech recognition apparatus 100 performs speech recognition using the learned acoustic model as described above, thereby improving the detection performance of pronunciation errors due to native language interference. Thus, the reliability of the evaluation results at the time of foreign language learning Can be improved.

The speech recognition apparatus 100 may operate in conjunction with a language learning system such as a foreign language, and may be formed integrally with a language learning system.

In response to a user request, the terminal 200 generates voice data by sensing a user voice, and transmits the voice data to the voice recognition apparatus 100 to request voice recognition. At this time, the terminal 200 may perform a part of speech recognition processing, for example, extraction of a feature vector for user's speech data, and transmit the extracted feature vector to the speech recognition apparatus 100 to request speech recognition . The terminal 200 can receive the speech recognition result and the error detection result transmitted from the speech recognition apparatus 100 and output the result to the user. In addition, the terminal 200 can further provide language learning and evaluation based on the received speech recognition result and the error detection result.

The terminal 200 may be various types of information processing apparatuses used by a user and may be a personal computer (PC), a notebook computer, a mobile phone, a tablet PC, a navigation terminal, a smart phone, a PDA (Personal Digital Assistants), a Smart TV, a Portable Multimedia Player (PMP), and a digital broadcast receiver. Of course, this is merely an example, and it should be construed as a concept including a device that is currently developed, commercialized, or capable of all communication to be developed in the future, in addition to the above-described examples.

2 is a block diagram showing a detailed configuration of a speech recognition apparatus in a speech recognition system according to an embodiment of the present invention. For reference, FIG. 2 shows the configuration of the speech recognition apparatus 100 as functional units, and each of the components may be dispersed in a plurality of devices, or may be implemented as a single device.

2, the speech recognition apparatus 100 according to an embodiment of the present invention may include a speech recognition unit 110, an acoustic model learning unit 120, and a storage unit 130.

The speech recognition unit 110 receives a speech recognition request from the terminal 200 and uses the acoustic model stored in the storage unit 130 to calculate a similarity degree between the user's speech data transmitted from the corresponding terminal 200 and the reference patterns And outputs the recognition result of the user's voice data based on the calculated result. And the recognition result may be transmitted to the corresponding terminal 200. In addition, the voice recognition unit 110 detects the user's pronunciation of the erroneous pronunciation of the user's voice data based on the standard pronunciation of the native speaker, and provides the detection result to the terminal 200. [ At this time, the acoustic model used for speech recognition and error sound detection is learned through the acoustic model learning unit 120.

The acoustic model learning unit 120 collects error data for speech recognition from the speech recognition unit 110, and trains an acoustic model to be used for the speech recognition based on the collected error data. More specifically, the acoustic model learning unit 120 extracts, from among the speech recognition results output from the speech recognition unit 110, speech data of a user whose speech recognition result is determined as an error, Is compared with a phonological error dictionary defining a phonological error dictionary, and if there is a phoneme matched with the phonological error dictionary, the error data is classified as an error due to native language interference. Then, information on error pronunciation generated by the mother language interference with respect to the word sequence is extracted from the error data classified by the error due to the native language interference, and the result of the speech recognition, The discrimination learning for the acoustic model is performed in a direction increasing the difference from the error pronunciation.

On the other hand, when the error data is not determined to be an error due to native language interference, the acoustic model learning unit 120 determines that the speech recognition itself is an error, and recognizes the recognition result, And discrimination learning is performed on the acoustic model in a direction increasing the difference from the result (correct answer).

The acoustic model re-learned by the acoustic model learning unit 120 is used for voice recognition of the next voice data in the voice recognition unit 110, and as a result, the accuracy of the voice recognition result can be improved Let's do it.

The storage unit 130 stores data necessary for speech recognition of the speech recognition unit 110 and learning of the acoustic model learning unit 120. More specifically, the storage unit 130 includes a plurality of acoustic models, Stores phonological error dictionary for error pronunciation.

In the above description, the server-client based embodiment has been described. However, in another embodiment of the present invention, the speech recognition processing may be performed by the stand press operation of the terminal 200. [

3 is a block diagram illustrating a configuration of a terminal 200 that performs a voice recognition function according to another embodiment of the present invention.

3, the terminal 200 includes an input unit 210, a communication unit 220, an audio processing unit 230, an output unit 240, a storage unit 250, and a control unit 260 .

The input unit 210 is a means for generating a user input signal for controlling the terminal 200 or requesting a specific function in response to a user's operation. For example, the input unit 210 may include at least one of a key input unit, a touch input unit, a gesture input unit, and a voice input unit. The key input means generates a signal corresponding to the key according to the key operation, and corresponds to a keypad and a keyboard. The touch input means is a touch pad, a touch screen, and a touch sensor, which recognize an input operation by sensing a user's operation of touching a specific portion. The gesture input means recognizes a specific operation, such as a shaking or moving operation of the terminal device, an approach to the terminal device, a blinking operation, etc., as a specific input signal, such as a geomagnetic sensor, A sensor, a camera, an altimeter, a gyro sensor, and a proximity sensor.

The communication unit 220 is a means for transmitting and receiving data through the network 10. The communication unit 220 communicates with the speech recognition apparatus 100 as necessary in the speech recognition process to generate necessary acoustic models, And receive related programs.

The audio processing unit 230 performs audio processing such as generating audio data by sensing audio output from the terminal 200 and a user's voice. The audio processing unit 230 includes audio sensing means (for example, a microphone (MIC) (For example, a speaker SPK). Particularly, in the speech recognition processing according to the present invention, the audio processing unit 230 receives a signal of a voice of a user, which is a target of speech recognition, through a microphone (MIC) and converts the signal into voice data, which is digital data. At this time, preprocessing such as amplification and filtering for noise can be further performed.

The output unit 240 is a means for interfacing the terminal 200 with a user and outputs a GUI (Graphic User Interface) screen for confirming the execution result of the terminal 200 and for performing a user operation, for example. In particular, in the present invention, the output unit 240 may output a guidance message for inputting a user voice for voice recognition, output of a voice recognition result, and detection information on a pronunciation error of the user.

The output unit 240 may be implemented by various display means such as an LCD (Liquid Crystal Display), a TFT-LCD (Thin Film Transistor-Liquid Crystal Display), an LED (Light Emitting Diodes) Organic Light Emitting Diodes (AMOLED), Active Matrix Organic Light Emitting Diodes (AMOLED), flexible displays, and three-dimensional displays.

The storage unit 250 is a means for storing data or programs necessary for the operation of the terminal 200 and can basically store an operating program (OS) of the terminal 200 and one or more application programs. The operating program and the application program stored in the storage unit 250 are executed by the control unit 260 and execute functions implemented in the terminal 200. [ In particular, in the present invention, the storage unit 140 stores data and programs for learning speech recognition and acoustic models. Specifically, the storage unit 140 stores an acoustic model for speech recognition and a phonological error dictionary for detecting pronunciation errors due to native language interference. The storage unit 250 may be any type of storage such as a RAM (Read Only Memory), a ROM (Read Only Memory), a hard disk (HDD), a flash memory, a CD- Media.

The control unit 260 controls the overall operation of the terminal 200 and basically operates based on the operating program stored in the storage unit 250 to establish a basic execution environment of the terminal 200 and selects Therefore, the application program is executed to provide a random function. Particularly, the control unit 150 receives the voice recognition request from the application program (for example, a learning program) requested by the user through the input unit 210 or through the audio processing unit 230 The speech recognition and error sound detection are performed using the acoustic model stored in the storage unit 250 for the user's voice data, and the sound recognition result is used to minimize the error in the speech recognition result Perform re-learning on the model. The result of speech recognition and error pronunciation detection by the control unit 260 is output to the user through the output unit 240.

For this, the controller 260 may include a preprocessing module 261, a speech recognition module 262, and an acoustic model learning module 263.

The preprocessing module 261 is a module for performing preprocessing for voice recognition of user voice data input from the audio processor 230, and more specifically extracts a feature vector of the voice data.

The speech recognition module 262 substitutes the feature vectors of the user's voice data transmitted from the preprocessing module 261 into a plurality of training models stored in the storage unit 250 to calculate the similarity degree, And performs speech recognition on the voice data of the user. The voice recognition module 262 detects an error sound included in the voice data of the recognized user using the plurality of acoustic models and outputs information on the detected error sound through the output unit 240 do.

The acoustic model learning module 263 confirms whether or not an error has occurred on the basis of the speech recognition result of the speech recognition module 262 and outputs the speech recognition result to the storage unit 250 using the speech recognition result, And executes re-learning on the stored acoustic model. Specifically, the acoustic model learning module 263 compares, among the speech recognition results, the speech data of the user whose speech recognition result is determined as an error, with the phonemic error dictionary in which the error pronunciation that can be generated by the previously stored mother language interference is defined, The method comprising the steps of: classifying the error as an error due to native language interference when an error occurs in a pattern matching the phonological error dictionary; And performs discrimination learning on the acoustic model in a direction to increase the difference between the speech recognition result determined as the error and the error pronunciation. On the other hand, if the error is not an error due to native language interference, it is determined that the speech recognition itself is an error, and the difference between the recognition result determined as the error and the actual recognition result (correct answer) And performs discrimination learning for the acoustic model.

Next, a speech recognition method and a learning method of an acoustic model according to the present invention based on the above-described configuration will be described.

The speech recognition method and the learning method of the acoustic model according to the present invention are performed through the interlocking of the speech recognition apparatus 100 and the terminal 200 according to an embodiment of the present invention. In another embodiment of the present invention, As shown in FIG.

FIG. 4 is a flowchart illustrating a speech recognition process according to the present invention. FIG. 5 is a flowchart illustrating a learning process for an acoustic model in speech recognition according to the present invention.

First, referring to FIG. 4, an acoustic model for speech recognition is generated through clustering of a plurality of speech data samples and stored (S110). The step S110 may be performed through the voice recognition apparatus 100. In the case of the terminal 200, the voice recognition apparatus 100 may receive and store a plurality of acoustic models.

Then, voice data of a user to be a speech recognition target is input (S120). At this time, the terminal 200 directly detects the user's voice through the audio processing unit 230 and converts it into voice data, and the voice recognition device 100 receives voice data of the user through the terminal 200.

When the user's voice data is input, voice recognition is performed on the voice data of the user using the pre-stored acoustic model (S130). More specifically, a feature vector for the voice data of the user is extracted, and the feature vector is substituted for each acoustic model to extract the word sequence having the highest degree of similarity to the user voice data. Here, the feature vector is an abbreviated representation of only the feature signal of the waveform, excluding unnecessary information, in the voice data represented by the waveform that changes with time. For such speech recognition, a Viterbi algorithm or the like can be used. The step S130 may be performed by interlocking the voice recognition apparatus 100 or the voice recognition apparatus 100 with the terminal 200 or the terminal 200 alone. When the terminal 200 interacts with the speech recognition apparatus 100 and the terminal 200, for example, when the terminal 200 extracts a feature vector from the user's speech data and transmits the extracted feature vector to the speech recognition apparatus 100, 100) to the acoustic models, and extracts word strings having a high degree of similarity.

The speech recognition result is output to the user through the terminal 200 or provided to another application program (e.g., a language learning program) of the terminal 200 (S140).

Meanwhile, in the present invention, the speech recognition apparatus 100 or the terminal 200 checks whether the speech recognition result is erroneous (S150). For this purpose, the voice recognition apparatus 100 or the terminal 200 may inquire whether the voice recognition result is correct or not, and receive the feedback from the user.

The acoustic model learning module 120 of the speech recognition apparatus 100 or the acoustic model learning module 263 of the terminal 200 may be configured to perform the speech recognition on the basis of the speech recognition result, (S160), the error data is collected from the speech recognition result confirmed by the error, and the acoustic model is re-learned so that the error is minimized. The step S160 may be performed as shown in FIG.

5, the acoustic model learning unit 120 of the speech recognition apparatus 100 or the acoustic model learning module 263 of the terminal 200 collects error data from the speech recognition result confirmed as the error (S210).

The acoustic model learning unit 120 of the speech recognition apparatus 100 or the acoustic model learning module 263 of the terminal 200 corrects an error of the speech recognition result by an error due to mother language interference, (S220). To this end, if the speech data of the user whose speech recognition result is judged as an error is compared with a phonological error dictionary defining an error pronunciation that can be generated by the previously stored native language interference, if there is a pattern matching the phonological error dictionary, An error due to native language interference, and if there is no pattern matched with the phonological error dictionary, it is determined to be an error of speech recognition itself.

Then, it is determined whether the error is an error of the speech recognition itself or an error due to native language interference (S230). If the error is the speech recognition itself, the difference between the correct answer of the feedback speech recognition result and the incorrect recognition result The discrimination learning for the acoustic model is performed (S240).

Conversely, if the error is due to a first language interference in step S230, the acoustic model learning unit 120 of the speech recognition apparatus 100 or the acoustic model learning module 263 of the terminal 200 first obtains the speech of the user Information corresponding to the data and error pronunciation generated by the mother language interference is extracted (S250). That is, an erroneous pronunciation that can be commonly generated by the mother language interference of the user with respect to the word string corresponding to the user's voice data is extracted.

Then, discrimination learning for the acoustic model is performed in a direction of increasing the difference between the speech recognition result determined as the error and the error pronunciation generated by the mother language interference (S260).

According to the above description, in the speech recognition system according to the present invention, the learning of the acoustic model can be performed so that the difference between the error pronunciation of the user and the standard pronunciation of the actual native speaker can be clearly distinguished by the mother language interference.

The acoustic model learning method according to the present invention can be implemented in software form readable by various computer means and recorded in a computer-readable recording medium. Here, the recording medium may include program commands, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a recording medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. For example, the recording medium may be an optical recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a compact disk read only memory (CD-ROM), a digital video disk (DVD) Includes a hardware device that is specially configured to store and execute program instructions such as a magneto-optical medium such as a floppy disk and a ROM, a random access memory (RAM), a flash memory, do. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be apparent to those skilled in the art. Furthermore, although specific terms are used in this specification and the drawings, they are used in a generic sense only to facilitate the description of the invention and to facilitate understanding of the invention, and are not intended to limit the scope of the invention.

According to the present invention, in the speech recognition system, the learning of the acoustic model can be performed so as to clearly distinguish the difference between the error pronunciation of the user and the standard pronunciation of the actual native speaker by the mother language interference. As a result, It is possible to improve the recognition performance of the discrimination ability between the correct pronunciation and the correct pronunciation and thereby improve the accuracy and reliability of the pronunciation evaluation of the user in language learning such as a foreign language.

10: Network 100: Speech recognition device 110:
120: acoustic model learning unit 130: storage unit 200: terminal
210: input unit 220: communication unit 230: audio processing unit
240: output unit 250: storage unit 260: control unit

Claims (10)

A storage unit for storing a phonemic error dictionary defining an acoustic model and an error phoneme that can be generated by native language interference;
A voice recognition unit for performing voice recognition of voice data of a user input using the stored acoustic model in response to a voice recognition request from the terminal and outputting a voice recognition result to the terminal;
When the speech recognition result is an error, comparing the speech data with the phonemic error dictionary and, if the speech data includes a phoneme matched with an error phoneme included in the phonological error dictionary, An acoustic model learning unit that performs discrimination learning of the acoustic model in a direction that increases the difference between the recognition result and the error phonology generated by the mother language interference when the error is due to native language interference; And
The speech recognition apparatus comprising: delete 2. The apparatus of claim 1, wherein the acoustic model learning unit
And discriminating learning is performed in a direction of determining that the speech recognition itself is an error and increasing the difference between the recognition result determined as the error and the original correct answer if the error of the recognition result is not an error due to native language interference A voice recognition device for detecting an erroneous pronunciation. The speech recognition apparatus according to claim 1,
Further detecting an error pronunciation of the user's voice data using the acoustic model and providing an error pronunciation detection result to the terminal. An acoustic model and a storage unit for storing a phonological error dictionary defining an error phoneme that can be generated by the mother language interference
An input unit for receiving a user request;
An audio processor for converting a user's voice signal into voice data and outputting the voice data;
When a user's request through the input unit or a request for a speech recognition based on a request of an application program is generated, speech recognition is performed on the user's speech data output from the audio processing unit using a previously stored acoustic model to output a speech recognition result When the speech recognition result is in error, comparing the speech data with the phonemic error dictionary, and if the speech data includes a phoneme that matches the error phoneme included in the phonological error dictionary, And an acoustic model learning module for performing discrimination learning in a direction for increasing the difference between the recognition result and the error phonology generated by the mother language interference in the case of an error caused by the mother language interference A control unit; And
And an output unit for outputting the speech recognition result to a user. delete 6. The method of claim 5,
Further comprising a communication unit for transmitting and receiving data through a network,
Wherein the control unit receives the acoustic model for speech recognition and the phonological error dictionary through the communication unit and stores the received acoustic model and the phonological error dictionary in the storage unit. 6. The apparatus of claim 5, wherein the control unit
Further comprising a preprocessing module for extracting a feature vector from voice data of the user output from the audio processor,
Wherein the speech recognition module applies the feature vector to the acoustic model and extracts word strings similar to the speech data through similarity measurement. 6. The method of claim 5, wherein the speech recognition module
And further detects an erroneous pronunciation included in the recognized voice data. Storing a phonological error dictionary defining an error phoneme that can be generated by a native language interference;
A speech recognition method using an acoustic model, comprising the steps of: collecting speech data in which a speech recognition result is determined to be an error;
Comparing the speech data with the phonemic error dictionary to determine an error related to the speech data as an error due to native language interference when the speech data includes a phoneme that matches the error phoneme included in the phonological error dictionary; ;
Extracting an error phoneme generated by mother language interference with a word sequence corresponding to the voice data if it is determined to be an error due to native language interference; And
And performing discriminative learning on the acoustic model in a direction that increases the difference between the speech recognition result and the error phonology caused by the mother language interference.

Images