JP3046029B2 - Apparatus and method for selectively adding noise to a template used in a speech recognition system - Google Patents

Description

Description: BACKGROUND OF THE INVENTION The present invention relates to a general speech recognition system, and more particularly to speech recognition using templates, each of which is generated by the selective addition of noise to increase the probability of speech recognition. About the system.

(Prior art)

General speech recognition methods have been very developed in recent years and are used in many forms. The idea of speech recognition is that the information obtained by the utterance is used to directly drive a computer or other means. Basically, the key element in the recognition of information in vocal sounds in the prior art is the distribution of energy with respect to frequency. The formant frequency is a frequency at which the energy peak is particularly important. Formant frequency is the acoustic resonance of the oral cavity and is controlled by the tongue, jaw and lips. For the listener, the determination of the first two or three formant frequencies is usually sufficient to identify a vowel. Thus, prior art machine recognition involves several means for determining the amplitude or power spectrum of the incoming speech signal. The first step in speech recognition is pre-processing, which converts the speech signal into recognizable properties or parameters and reduces the data flow to a manageable rate. One means for performing this process is to measure the zero-crossing rate of the signal in several broad frequency bands to provide an estimate of the formant frequency in this band.

Another means is to represent the speech signal by the parameters of a filter whose spectrum best matches the spectrum of the input speech signal. This method is known as linear predictive coding (LPC). Linear predictive coding, or LPC, is characterized by its efficiency, accuracy, and simplicity. Recognition characteristics extracted from speech are usually
Averaged over 10 to 40 milliseconds and sampled at 50-100 times / second.

The parameters used to represent and recognize speech are directly or indirectly related to the amplitude or power spectrum. Formant frequencies and linear prediction filter coefficients are examples of parameters that are indirectly related to the speech spectrum. In another example, there is a Cepstral parameter and a log area ratio parameter.

[Problems to be solved by the invention]

These and many other speech parameters used for recognition can be derived from spectral parameters. The present invention relates to selectively adding noise to spectral parameters that generate speech recognition parameters. The invention applies to all forms of speech recognition using speech parameters obtained or obtainable from spectral parameters.

In any case, many common methods of speech recognition in the past use templates to perform matching.
In this method, words are usually represented in the form of parameter sequences. Recognition is performed by comparing the unknown template token with the stored template using a similar predefined method. In many cases, a time alignment algorithm is used to account for the variability in word generation rate. Thus, the template matching system can perform well with a small set of phonetic discrete words. Some researchers have questioned the ability of such systems to ultimately perform precise speech classification of a wide range of speakers. Jay Parker
S.Perkel and DHKlat
t) Paper "Achieving Precise Speech Classification: Template vs. Properties"("Variability and Invariance in Speech Processing"
Hilsdill, New Jersey, published by Lawrence Elbaum Associates, 1985, publishers R. A. Cole, Arm M. Stern, and M. J.
(Lassley).

Therefore, many have proposed alternative methods based on features for speech recognition, such as first identifying a set of speech characteristics that capture speech-related information in the speech signal. Algorithms can be configured to extract features from speech signals based on this knowledge. Classification is then performed to combine the features and arrive at a recognition decision. It has been argued that feature-based systems perform better and better at discriminating speech than template matching techniques. In any case, template matching is a commonly used method for pattern recognition, whereby unknowns are compared to prototypes to determine which is closest.

With this decision, feature-based speech recognition using a multiple change Gaussian model for classification also performs template matching. In this case, only the statistical classifier uses the feature vector as a pattern. Similarly, the spectral amplitude and
Looking at LPC coefficients as features, spectrum-based techniques are also feature-based methods.

In use, template matching and feature-based systems actually represent different points along the continuum.
One of the most important problems with the template matching method is the difficulty in defining a distance measure that is sensitive enough to fine speech discrimination but insensitive to unrelated spectral changes.

One manifestation of this problem is due to excessive weight given to insignificant inter-frame changes in the spectrum of long invariant vowels. Accordingly, prior arts aware of such problems have proposed a number of distance metrics that are sensitive to audio distance and insensitive to irrelevant audio differences. For example, ICASSP-82
(IEEE Catalog No. CH1746-7, pp. 1278-1281, 1982), a paper entitled "Estimation of Acceptable Speech Distance from Critical Band Spectrum" (by D.H.Clatt). Please refer to.

Either way, proceeding Zoo of IEEE (November 1985, Vo7
3, No. 11, pages 1537 to 1696). This article from the IEEE offers a variety of papers on man / machine speech communication systems that will broaden the scope of certain related issues. As can be appreciated, a key point of any speech recognition system is the ability of the system to perform its distribution tasks, i.e., recognize speech for all environment types.

As described above, many speech recognition systems use templates. Basically, in such a system, the utterance is converted into a parameter sequence and stored in a computer. The speech waves are passed from the speaker's mouth through a microphone to an analog-to-digital converter where they are digitized through a filter, for example, with any background noise that may be there. Next, the digitized signal is further converted to recognition parameters through a filter,
In this manner, the stored speech template is compared to make a selection of the most likely spoken words. Yet another example of such a method is IE
There is an EE spectrum (published in April 1977, Vo124, No. 4). See the paper entitled "Performing Speech Recognition" by T. Walch (pages 55-57).

As you can see from this paper, the applications of speech recognition systems are steadily expanding, and many models can already be used in various applications as pointed out in the paper. The formation of templates is also well known in the prior art. Such templates are used in many different types of speech recognition systems. One example of a system is the "Keyword Recognition System" by JSBridle, an article "An Efficient Elastic Template Method for Determining Given Words in Ongoing Speech"
(April 1973, "The Spring Conference of the British Phonetic Society", No. 1
Page to page 4). In this paper the author discusses obtaining an elastic template from the parametric representation of the utterance examples of the keywords to be detected. Similar parametric representations of the incoming speech are continually compared to these templates to determine the similarity between the speech and the keyword from which the template was derived.

If the incoming speech segment is sufficiently close to the corresponding template, the word is determined to have been spoken by the recognizer. Word templates are called "elastic" because they can be expanded and compressed in time due to changes in speaking speed and changes in pronunciation speed of words.

Keyword recognition is similar to conventional speech recognition.
The former is one in which the template is only remembered for arbitrary words, ie "key" words that should be recognized within the context of the sound, while the latter for all of the speech that is expected to be spoken. The template is stored. All such systems, whether keyword recognition systems or conventional speech recognition systems using templates, have the same problem, i.e. words that have been uttered by different individuals or uttered by the same individual in different conditions. The problem is that the system does not have the ability to recognize

Accordingly, it is an object of the present invention to provide an improved apparatus and method for an automatic speech recognition system.

It is a further object of the present invention to provide a speech recognition system that automatically adapts to noisy environments.

[Means for solving the problem]

As can be seen from this specification, many speech recognition systems have reduced performance in noisy conditions. This is especially the case when templates are derived from speech with little or no noise or noise of different nature at the time the recognition is performed. Conventional methods that reduce this difficulty require generating a new template with new noise. This generation requires the collection of new speech and noise. The system of the present invention analytically adds noise to the template, thereby improving the probability of recognition and substantially improving the performance of the system, without the need to collect new speech in generating the template.

The system of the present invention comprises a storage device for storing an initial template indicating a spectrum value of a recognizable speech in a noise-free state, and a spectrum analysis for outputting a spectrum value of an utterance of an input signal indicating a speech in a state of noise A device for generating a motion template;
A recognition module that compares the operating template with the output spectral values from the spectrum analyzer and outputs when a good comparison result is obtained indicating the presence of the recognized speech during the utterance. In the system, an apparatus for generating an operation template is coupled to a spectrum analyzer for outputting an estimated noise signal indicative of noise in an input signal; and an estimated noise signal coupled to the first means. Generating a modified motion template from the initial template based on the estimated noise signal, wherein the first means detects the speech signal in the presence of noise during speech. An audio tracking unit that outputs a first signal indicating a scalar audio level value related to the audio signal and indicating an average power of the audio signal in the presence of noise; Noise and noise level tracking, comprising: a noise tracking unit that detects noise in the signal and outputs a second signal that is a vector of spectrum values indicating an estimated value of the noise and that indicates an average power of the noise for a predetermined time period of the utterance. Means for adjusting the audio level of the initial template based on the first signal; adding spectral values of the noise estimate to the initial template based on the second signal; Generates an action template,
The motion template includes an estimated noise of the utterance of the input signal to obtain an improved speech recognition performance of the recognition module;
It has the same signal-to-noise ratio as the utterance of the input signal.

Also, the method of the present invention forms a motion template for use in a speech recognition system for recognizing speech in the presence of noise in the utterance of the input signal based on a comparison between the spectral value of the input signal and the motion template. The method comprises detecting an audio signal in the presence of noise in the input signal and generating a first signal indicative of a mean power of the audio signal in the presence of noise, which is a scalar audio level value associated with the audio signal. And detecting a noise in the utterance and outputting a second signal indicating a mean power of the noise for a predetermined time period of the utterance, which is a vector of spectral values indicating an estimated value of the noise. A method for outputting a noise signal and forming a motion template having the estimated noise of the utterance of the input signal and the same signal-to-noise ratio as the utterance of the input signal, the improved speech recognition Adjusting the audio level of the initial template based on the first signal and adding spectral values of the noise estimate to the initial template based on the second signal to obtain the function Modifying an initial template indicating recognizable speech in a non-existent state.

〔Example〕

As will be appreciated, the invention applies to all recognition systems that use parameters whose properties are spectral or derived from those whose properties are spectral. The latter requires storing the template in two forms: a spectral template for the analytical addition of noise and an operational template.

Referring to FIG. 1A, there is shown a block diagram of a speech recognition system using recognition parameters derived from a spectrum in accordance with the present invention.

A microphone 10 is shown, which is used by a speaker using the system to input speech. Microphone
10 converts audio waves into electrical signals, which are then amplified
Amplified by 11. The output of amplifier 11 is coupled to spectrum analyzer 12. The spectrum analyzer 12 is a broadband or narrowband spectrum analyzer having a short-term analysis capability. The function and configuration of a spectrum analyzer is basically well known and can be configured in a number of ways.

The spectrum analyzer 12 splits the speech into short frames and outputs at its output a parametric representation of each frame. The particular type of acoustic analysis performed by the spectrum analyzer 12 is not critical to the invention, and many known acoustic or spectral analyzers can be used. Such examples are described in U.S. Patent Application Nos. 439018 (filed November 3, 1982, G. Bensco et al.) And 473422 (March 9, 1983).
Japanese Patent Application, G. Bensco et al.).
Both applications are the IT assignees of the present invention.
Assigned to Corporation and incorporated herein by reference.

U.S. Patent Application No. 655958 (filed September 28, 1984, inventor Ahl Higgins et al., Entitled "Keyword Recognition System and Method Using Template-Concatenation Model") is also a reference.

The spectrum analyzer 12 is provided with a 14-channel bandpass filter array, and the frame size used is 20 milliseconds or more. These spectral parameters are processed as shown in FIG. 1A. As shown, the output of the spectrum analyzer 12 is coupled to a switch 13, which can operate in a recognition mode, a formation template mode, or a modified template mode.

When switch 13 is placed in the forming template mode,
The output of the spectrum analyzer 12 is coupled to a module 14 shown as a template in spectral form. The purpose of module 14 is to help form a template from the output of spectrum analyzer 12. These templates are formed in module 14 and are templates in spectral form, and many methods of forming such templates are well known. Basically, in the form template mode, the output of the spectrum analyzer 12 is processed by a module 14, which outputs a template relating to the utterance made by the speaker through the microphone 10. The speaker is prompted to speak the word to be recognized, and a template is generated that basically indicates the spoken word. These templates are used by module 15 to derive the recognition parameters obtained from the spectral morphology templates to generate the final template as shown by module 16 for low or no noise conditions. You. The noise-free templates are then stored, as shown by module 16,
For example, a specific utterance such as a word or a phrase uttered by a specific speaker is shown.

The stored template is coupled through switch 100 to a processor 160, which executes a recognition algorithm. Accordingly, the processor 160 operates in the recognition mode to compare the unknown speech with a template generated in a noise-free manner and stored in the module 16. Therefore, as shown in FIG. 1A, in the formation template mode, a template in the form of a spectrum is output in order to obtain template parameters. This template parameter is then used to form a template for noiseless or low noise conditions. As will be described, the processor 160 can operate with templates stored in the module 16 for low or no noise conditions. The function of the processor 160 is also well-known and basically operates to output a match based on various distance measurement algorithms and other algorithms. When such a match is made, an indication is made that this is a correct word and that this word or sound is the output of the system.

When the switch 13 is placed in the recognition mode, the switch 13 outputs the output of the spectrum analyzer 12 to the derived parameter module.
Coupled to 161, this module 161 basically derives parameters from the spectrum analyzer, which parameters are compared with the storage templates stored in the module 16, for example, as described above.

As shown in FIG. 1A, switch 13 can also be set to the center position. The center position is the modified template mode position, in this case the spectrum analyzer
The twelve outputs enter the estimated noise statistics module 162. The function of module 162 is basically to perform noise analysis or to process the noise and make noise statistics estimates.
This is a key feature of the present invention, whereby the present invention selectively adds noise to form a template, performs speech recognition, and improves such recognition in the presence of such added noise. Perform

Accordingly, the function of the estimated noise statistics module 162 will be described further below, but in conjunction with module 14
Is to modify the spectral template formed in module 164, which receives information from. Module 16
At 5, a recognition parameter is derived from the output of module 164, which parameter is used to form a template that is used in noisy or low noise levels as indicated by module 166.
To that end, the system shown in FIG. 1A can recognize by switching the switch 100 either a noisy template or a very low level noise or noiseless template.

As briefly indicated above, in the recognition mode, the spectral parameter output of the spectrum analyzer 12 is provided to the input of the processor 160 through the derived parameter module 161. Processor 160 typically executes the algorithm, but again is not important to the invention. Processor 16
0 determines the sequence of the stored templates and provides the best match for incoming speech to be recognized. Thus, the output of the processor is essentially a series of template labels, each label representing one template in the best matching template sequence.

For example, each template is assigned one number and one label. The template may be a multi-bit display of that number. This output is provided to a template search system provided in the processor 160, which may be a comparator with storage for template labels, for example, if there is a multi-bit display. Accordingly, processor 160 operates to compare each of the incoming template labels with the stored template. The subsystem, processor 160, then provides an indication of which word or phrase was spoken as well as which word or phrase was spoken.

In either the form template mode or the modified template mode, the user speaks various words and recognition parameters are derived from the spectral output of the spectrum analyzer 12. In the modified template mode, the system generates various templates to be used in cooperation with the system in the recognition mode, which are modified by the selective addition of noise by the estimated noise statistics module 162 as described above. This module 162
, Resulting in more reliable system operation, as described further below.

Referring to FIG. 1B, a recognition system that uses recognition parameters whose properties are spectra is shown. 1B
In the figure, members having the same functions are indicated by the same reference numerals as those in FIG. 1A. As can be seen, microphone 10 is coupled to the input of amplifier 11 and the output of amplifier 11 is coupled to the input of spectrum analyzer 12. Spectrum analyzer
The output of 12 is again coupled to a switch 13, which can be operated in a forming template mode, a modifying template mode, or a recognition mode.

As can be seen from FIG. 1B, in the forming template mode, the module 170 forms a template for a low noise or noiseless condition. This module 170 forms a template that directly gives recognition parameters whose properties are spectra. This formed template is then stored and module 170 is coupled to module 171. Module 171 is based on the effect of estimated noise statistics generator 172, which basically functions as noise module 162,
For example, the spectrum template obtained from the module 170 is modified. The output of the modified spectrum template module 171 is coupled to module 173 and
3 stores a template for use in a noise state.
In this figure, the processor 177 is also shown.
Can operate on either the template stored in module 170 or the template stored in module 173.

In each case, before further processing, it is known how to generate the template according to the prior art. There are several ways to generate a template. The method of performing the task of template generation is automatic,
Usually, a multi-stage or two-stage process is used. In one such method, speech data from a training utterance (template mode) is divided into segments. These segments are then provided as inputs for statistical cluster analysis, which selects a subset of segments that maximizes a mathematical function based on a measure of the distance between the segments. Segments belonging to the selected subset are used as templates.

Such a technique is described in the above-mentioned US Patent Application No. 655958. In any case, various methods for measuring distance are well known as described in several references cited in the Background of the Invention.
One of the widely used methods of measuring distance is Mahalanobis distance calculation.

An example of this method is described in US Patent Application No. 003971, entitled "Multi-Parameter Speaker Recognition System and Method",
(Transferred to a wrench on January 16, 1987). This document provides various other examples of techniques used in speaker recognition systems, and details some of the algorithms used with such systems. In any event, referring to FIG. 1, a key feature of the present invention is related to the speech recognition system shown in FIG. 1, which uses a template for comparison with incoming speech. And thereby determine which word was spoken. The method may be a keyword recognition system, a speech recognition system, a speaker recognition system, a speaker recognition system, a language recognition system, or any other system that makes decisions regarding spoken sound using a template or a combination of various templates. Such a system can also be used.

Prior to the description of the configuration and method of the present invention, the underlying concept and concept of the invention will be described.

The inventor has recognized that when the S / N ratio of the template is the same as the unknown or uttered speech, the recognition performance is better than using a template that is noisier or less noisy. Therefore, if the signal-to-noise ratio of the audio signal is considered predictable, the template is generated "as if" from speech with the same signal-to-noise ratio as the incoming unknown speech, before the template is used. The recognition performance can be optimized by correcting.

Therefore, in order to put the present invention into practical use, the following considerations must be taken. The first is to anticipate the S / N ratio of the incoming speech, and the second is to modify the template to meet the "as if" requirement. Conjectures are based on both theory and experience. Often, it is expected that a speaker will speak at a relatively constant level, or at a relatively constant level greater than this noise, or absolutely in the case of low level noise or constant noise. it can.
Then, the S / N ratio of the unknown speech can be predicted using the speech and the noise level. As explained below, this is done by using a speech and noise level tracking module. In one example, it is assumed that both the talk level and the noise level in each filter channel change slowly enough that the current value is a valid estimate of the near future value.

Making the template "as if" made from more noisy speech by modifying the noisy or relatively noisy template is based on both experience and theoretical considerations.

Research has determined that this is a very good approximation, assuming that noise and speech power are added in each of the individual filter bank channels. A more accurate approximation is where the combination of speech and noise has a non-central x ² distribution with many degrees of freedom with respect to the filter bank channel bandwidth. From the above additional considerations, a more accurate estimate of the expected value of the combination of noise of known statistical characteristics and known speech power can be made. The increased accuracy resulting from the "addition of noise" obtained in this way increases the accuracy of the generated template, but significantly more than the improvement obtained using the "add power" rule. There is no. Thus, although the process can be made more theoretically more accurate by substituting another method for estimating the expected value of the combination of speech and noise power, the power addition rules are described below. This substitution does not alter the intent or substance of the present invention.

Furthermore, it has been observed that both internal electronic noise and quantum noise combine with acoustic noise and signals with respect to the "power addition" rule. These noises are smaller than the target acoustic noise, but can be applied. Thus, since the results of "power addition" can be used in constructing various models, the application of research work becomes apparent through continued efforts to use many derived from effective models. This is described below.

It has been shown that templates resulting from noise powers equal to that average perform very well with respect to producing reliable recognition output. Therefore, it is not necessary to anticipate the variability of noise power between frames, and it is sufficient to use an average value. The required template parameters are those generated from the same average speech power and the same speech power that is effectively combined in the base form template.

The channel noise power value from the system is an estimate of the noise power and can be taken to be related to the average noise power, which can be determined mathematically. Therefore, in order to fully understand the process and the validity, the description will be made below.

First, it is pointed out that the probability distribution of the output of a single discrete Fourier transform (DFT) of a speech signal corrupted by additive zero-mean Gaussian noise can be easily calculated. The next important factor to extend the model of how to combine speech and noise to make it applicable to each channel of the bandpass filter bank is that the channel bandwidth must be less than that of a single DFT channel. Is also quite large or can be made large. Thus, the noise power parameter and the number of contributing channels can be specified by observing the output of the bandpass filter in the absence of speech and noise.

The next step is to improve the speech recognition template formed in the noisy condition to be used in the noisy condition by modifying it to be equal to the expected value in the noisy condition. . Therefore, the method used is to replace, for each speech sample and each bandpass filter channel represented in the noise-free template, the expected value of the noise-free template as modified by the presence of the current noise. It is.

Therefore, by measuring the average and fluctuation in the output of the bandpass filter channel, it is possible to estimate the characteristics of the channel by how Gaussian noise passes. Basically, as can be seen from the above (and much of the above has been proved mathematically), the practice of the present invention is based on both theory and experience. Basically, thus, a feature of the present invention is the analytical addition of noise to form a template that functions to increase the reliability of a speech recognition system.

There are two ways to add noise to template data collected in a noisy environment, thereby forming a new template for use in a noisy environment. The exact method adds noise to each template token and then averages the results. The approximate method is to average the noise-free tokens to form the basic morphological data, and then add the appropriate noise to the current state using "power addition" or other convenient or more accurate rules. To correct. The exact method requires maintaining all templates and surrounding tokens, and requires excessive storage. Approximate methods provide essentially the same template and recognition results. There are absolute assumptions during execution. That is, there is no noise in the template data as compared with the environment in which the template data is used.

Referring to FIG. 2, there is shown a detailed block diagram of the template formation used by adding noise to the basic form template. The base form template is itself an average formed for a set of words "tokens". Each token consists of a pattern taken from one utterance of a given word. One or more tokens are arranged to form a basic form template. The basic form template is formed in a quiet state.
It is stored in module 16 shown in FIG. 1A or module 170 shown in FIG. 1B. FIG. 3 is a table defining each value shown in FIG. FIG. 2 again shows the microphone 10 at which the speaker speaks. The output of the microphone is coupled to the input of an amplifier 11, the output of which is a BPF, a spectrum analyzer shown as a bandpass filter bank 12.
Combined with 12. Switch 13 is in the correction template position. The output from the spectrum analyzer 12 is a vector of bandpass filter spectral magnitude values which is provided to a module 20, which averages the frame pairs.

Frame pair averaging is a well-known technique and is basically performed by many known circuits. module
The output of 20 is the result of averaging a continuous pair of inputs from spectrum analyzer 12, and module 20 halves the effective frame rate. The output of module 20 is provided to scale bit module 21 and squared component module 22.
The squared component module 22 provides a vector output, which is essentially equal to the squared magnitude of the average frame versus the power value of the output of the module 20.

The output of the scale bit module 21 serves essentially to double the average of successive pairs performed by a series of shifts, making it possible to adapt the vector maximum to a 7-bit scale. To this end, module 21 is a shift register, which basically performs a number of right shifts and performs the operations described above. The output from the scale bit module 21 is directed to a logarithmic converter 23, which generates a scale logarithmic spectral parameter vector at its output. This parameter vector is then averaged by the module 24 over a predetermined set of template tokens, and a scale logarithmic spectral parameter is generated at the output which basically gives one parameter of the basic form template. The output from the squared component module 22 is directed to the input of module 25, a relativized energy module, and to the load of module 26, a speech and noise level tracker.

The output of the relative energy module 25 is a parameter that indicates the relative energy determined, for example, by averaging the energy from the output of the square component module 22. This is averaged over the template tokens by module 36 to produce an average that indicates the output vector, which is the relative energy parameter needed to provide another elementary morphology data value. Speech and noise level tracker
The output from 26 indicates an energy level, which will be averaged again by module 27, as will be described, to produce a further basic form characteristic energy level at the output of module 27. The speech and noise level tracker 26 provides two additional outputs, one of which is a logarithmic representation of the utterance level averaged over word time and channel, which is associated with the word as described further below. Is a scaler. Others are vectors of the noise level in each channel, averaged over time, and not with respect to the channel. This is also a vector associated with the word recognition unit. Thus, the output from module 27 is provided to a first adder module 30, which receives additional output from speech and noise level tracker 26. The output of adder 30 is provided to one of the inputs of adder 31, which receives the output obtained from scale bit module 21 at the other input.
The output of scale bit module 21 is multiplied by a factor K by module 32, where K is equal to 18.172 and a third
It is shown in the figure. This value is then averaged by module 33 to produce at its output a logarithmic base form value which is provided to the other input of adder 31. Adder 31
Is supplied to the adder 32. Adder 32 receives as another input the output from speech and noise level tracker 26, which is again a vector of noise levels in each channel. This output is provided to one input of a functional module 40, which receives the output from module 24 at the other input. The output from the function module 40 is a scale logarithmic spectral parameter vector for the noise-added template. This is a functional module
At its output, a recognition parameter vector, which is a mel-cosine transform matrix for a particular utterance, is generated at the output. Therefore, the operation template data is generated using the output from the module 41 and the output from the module 36.

All outputs associated with the block diagram of FIG. 2 as described above are shown in FIG. As can be seen from FIG. 3, the effective spectrum magnitude for the basic form template obtained from FIG. 2 is basically given by the following equation.

^B = 2 ^SB expb ( ^B ) The effective power is given by:

^B = ^2B = 2 ^2SB exp _b (2 ^B ) See FIG. 3 for definition.

2 so that the average utterance level of the template shown at the output of module 27 in FIG. 2 is the same as the current utterance level indicated by the output of the speech and noise level tracker 26 provided at the input of adder 30. Before adding, the power of each frame is modified. Module 2 because its value is in the recognition unit (0.331 dB)
The effective power in the basic form shown by the output of 6 is changed. In contrast, since the current noise level is added, the effective power level of the noise-added template is obtained, and the effective size of the noise-added template is shown as an output of the module 41.

Thus, all motion recognition patterns are mel-cosine transforms of log spectral parameters, and are a measure of relative energy. FIG. 2 together with the definition of FIG. 3 makes the above clear and mathematically obvious to a person skilled in the art.

Thus, using the same exact technique, the template can be formed by adding noise to the template token and then averaging. Basically, the process of doing this is the same as that shown in FIG. 2, whereby the same exact output as shown in FIG. 2 is obtained except that averaging is performed after functional unit 40. Given.

FIG. 4 shows a detailed block diagram of a typical system using the template formation technique as described above. In FIG. 4, the same reference numbers are used to indicate parts having the same function. As can be seen in FIG. 4, with the output of adder 46 coupled to a coder / decoder (CODEC) module and linear circuit 47, an AGC coupled to one input of adder 46, ie, an automatic gain control module. Four
5 are located. The coder / decoder module is basically an analog / digital converter, followed by a digital / analog converter. The output of the CODEC module is provided to a synthesizer, or bandpass filter bank, or spectrum analyzer 12.

The output from the spectrum analyzer 12 is sent to an average frame pair module 20, which is again associated with a scale module 21 and a speech and noise tracker module 26 described below. The various output template data values are provided from the output lines shown on the right side of FIG. 4 and are used to form a noisy template.

The main functional module is the speech noise tracker 26, which is described further below. Also in FIG. 4, the inputs to the microphone 10 are labeled Nc and Sc, which are important sources of signal and noise. The subscript "c" indicates that these expressions represent the average spectral magnitude for each passband of the filter bank channels forming the spectrum analyzer 12. This subscript "c" has 14 values, one for each filter in the filter bank.
One. Thus, Sc is the magnitude of the spectrum of the acoustic speech signal in channel C. Nc is the magnitude of the root mean square spectrum of the acoustic noise for this channel.
The output from summers 50 and 46 is the magnitude of the spectrum of the electronic noise, which is injected before or after AGC gain control module 45. The output from CODEC 47 contains the magnitude of the spectrum of the quantization noise introduced by the CODEC. In any case, the output of spectrum analyzer 12 is a vector of spectral magnitude values of the bandpass filter, and the output of average frame pair module 20 is the result of averaging a continuous pair of spectral magnitude values. .

The effective output signal of the spectrum analyzer 12 is an estimate of the spectral magnitude of the signal at the filterbank input relative to the filterbank passband, and is shown for each channel in the filterbank. Successive pairs of these values are averaged to produce output from module 20 at a rate of 50 / sec.

Basically, all values of a set for all 14 channels are shifted right by the same number S in module 21, so that the largest occupies 7 bits or less, and the resulting value is It is converted to a number proportional to the logarithm by the index table. The table returns 127 for 127 inputs, so the result is the natural log of the input, 2
It can be considered as 6.2 times, that is, the logarithm for the base b (b is 1.03888). The 20 ms frame value is also used by tracker 26 to generate a measure of peak speech energy and an estimate of the average noise energy for each channel. The utterance level is obtained by adding an arbitrary constant to the logarithmic estimate of the total speech energy at the microphone 10 with respect to the base b.

The effect of AGC gain is not a spectral value because it is basically removed. For example, the effect of this gain is related to the total energy in the passband of the entire filter bank. The utterance level estimates are also word or phrase related. The time constant is like a measure of the level at which short utterances are made.
Thus, there is only one level value associated with the unknown segment of each template or template period.
A temporal constraint on the noise estimate from tracker 26 is that only one noise level estimate must be assigned to each channel for the length of time that it is speaking. Thus, the output value from the speech and noise tracker 26 coupled to the logarithmic circuit 54 of FIG. 4 is an average energy estimate for the output of the filter bank. Thus, these values are affected by the AGC gain and are directly proportional to the average spectral energy without any log transformation.

It is assumed that the signal source and the various noise sources are statistically independent and their energies are added to the average. This has proven to be not only convenient for determining internal noise sources, but also a chosen approximation for both acoustic noise sources and signal sources. Assume further that there is a noise value that can be referred to as the equivalent noise power in the microphone. These values include the acoustic noise power and other system noise power, some of which are reduced by the gain of the AGC 45.

Accordingly, the scale factors derived from FIG. 4 and shown in FIGS. 2 and 3 are provided to generate a noise-related template. Therefore, by using the template averaging process, the same utterance level and S / N
An average template that is the same or equivalent to that obtained by averaging the log spectral parameters of all tokens in the ratio can be generated. Therefore, to simplify the overall problem, assume that the S / N ratios in all templates as well as all template tokens are the same. Since this can be performed by adjusting the utterance level in all tokens to be equal, the result of the same S / N ratio is an equal noise value in all tokens. Under this assumption, all forms of averaging noise equivalents can be made.

Studies have shown that when the S / N ratio of the template is the same as the unknown speech as described above, the recognition performance is better than that of the template where the noise is larger or smaller. Thus, based on the above technique, the S / N of the audio signal can be predicted, thereby generating a template from speech with the same S / N ratio as the incoming unknown speech before using the template. It was shown that recognition performance could be optimized by modifying the template "as if".

Therefore, two steps are used. One is to anticipate the S / N ratio of the incoming speech and modify the template to meet this requirement. Therefore, as described below, the speech and noise tracker 26 does not form a speech power estimate because the speech power estimate in each channel varies between words with each speech content. Since it is not possible to what word is expected either spoken therefore, not expected force the data. It is important to note that each channel is
That is, it has no estimate of the S / N ratio. Therefore, as described above, the template modification procedure avoids using a specific S / N ratio for each channel. So the template obtained from noise power equal to its average value is
Works very well in recognition systems.

That is, since it is sufficient to use the average value, there is no need to consider the variability of noise power between frames. The template parameters are then generated from the same speech power that is effectively present in the "basic form" template combined with the current average noise power. Basically, as described above, the speech and noise tracker 26 is a digital signal processing (DSP) circuit. The DSP circuit operates to execute an algorithm that provides a measure of the power level of the speech signal in the presence of additional acoustic noise and a measure of the average noise power in any form of bandpass filter bank channel. The measure of utterance level found indicates the appropriate speaker conversation level to adjust the S / N ratio for speech recognition. Other measures of utterance level change rapidly and / or change in the relative frequency of voiced and unvoiced utterances in spoken speech. The measures found by speech and noise trackers avoid these problems by detecting slightly smooth peak power in vowel nuclei.

More specifically, it tracks the slightly smoother peak power in the more energetic vowel nuclei. By ignoring power peaks between unstressed syllable nuclei and non-vowel nucleus speech intervals,
The measure is a continuous representation of a common speech level. The tracker has additive noise that is uncorrelated with the speech present when the total noise power changes usually slowly compared to the vowel nucleation rate in the speech (typically 5-15 / sec). It is intended to be used in the state. The tracker also operates to recover from faster changes in the noise level. Speech and noise tracker 26 employs a logarithmic or compression technique, which provides a measure of the total speech power for the frequency domain of interest. This measure is first processed by a slow rise, fast fall filter, where the rise and fall time limits are large positive instantaneous signal powers and filter values during the first few mils of a vowel kernel. The difference between
The choice is made such that no large negative difference occurs.

Unintensified vowel nuclei are usually skipped, and the instantaneous signal power and the fast signal velocities are increased so that the resulting value rises above the appropriate threshold only during normal vowel nuclei during speech intervals or only between stressed vowel nuclei The non-linear function of the difference between the falling and slowly rising time filtered values is directed to the moving boxcar integration process for the appropriate period. The intersection with this threshold is used to identify high signal power intervals due to speech kernels. Only the intervals identified in this way are used for tracking the utterance level. The value from the boxcar integration process that is greater than a second threshold less than the speech kernel threshold is then used to identify intervals that include noise power as well as speech power. The boxcar integral is less than a second (lower) threshold and the instantaneous power is greater than a third threshold greater than a value filtered by a filter having its fast rise and slow rise times. Only the missing interval is used as input to the noise power tracking function.

The noise power tracking module comprises a digital signal processor which is basically constituted by an integrated circuit chip. Many of these chips are available, are essentially programmable, and perform various types of algorithms. The algorithms associated with the noise and signal tracking functions operate to determine both the signal energy content and the noise energy content and operate in the following manner.

First, a mathematical value indicating the channel energy is obtained. This is done on every frame. Next, the total energy is calculated. This allows the system to make adjustments for automatic gain control changes. Once the energy has been calculated, the result is then smoothed for a predetermined period.
After the smoothed energy values are obtained, the log value of the total energy is calculated. After calculating the logarithm of the total energy, a boxcar integration or averaging is performed on the speech level estimates at the input to the pan-pass filter array. The next step uses an asymmetric filter,
Filter the logarithmic energy for speech detection by monitoring the rise time of the speech signal. The speech signal is commonly referred to and the incoming signal may be noise or an artifact signal,
Artifact signals are not noise or speech signals, but are due to intense exhalation and some other other characteristic of the speaker's voice which is essentially neither information nor noise.
In any case, this is also a true speech signal.

Therefore, to determine this, the instantaneous value of the logarithmic energy relative to the smoothed energy is monitored. The algorithm operates to divide the time interval associated with the rise and fall times of the signal into predetermined intervals. When the fall is positive relative to negative, certain decisions are made regarding the characteristics of the incoming signal to be recognized. As indicated above, these decisions determine whether speech, artifacts, or pure noise. For example, during a period when the rising is negative, if the rising is continuously negative, it is unconditionally assumed to be a noise signal. When a noise signal is received, the system smoothes the noise value,
The signal is tracked continuously by using these blood to contribute to the average noise energy and applying this value to the noise estimate using the calculated value. Next, a template is formed using this. Attention about the positive transition is even more difficult.

Positive transitions indicate noise, artifacts, or speech. The integration of the non-linear function is performed for this determination. Thus, based on comparing the integral to a certain threshold, it can be determined whether the positive rise represents speech, noise, or an artifact. The values resulting from the speech and noise tracker module in this way represent true speech values. 5A-5C show the speech and noise tracker program, where the complete program is shown.

FIG. 6 shows the definition of the engineering parameters required to understand the programming format shown in FIGS. 5A to 5C. More specifically, this procedure is performed every single frame, and operates as follows. No.
In the first step of the procedure, as shown in FIG. 5A, the total energy is obtained along with the energy in each channel.
This is shown in steps 1 and 2. And step
Energy is filtered on each channel, taking into account automatic gain control scale changes, as shown in 3,4. In the next step, the energy value is smoothed and the AGC
To obtain a smooth logarithmic value of the energy corrected for This is shown in steps 5,6,7. In the next step 8, a boxcar average of the speech level estimate is taken. Then, as shown in steps 9 and 10, the asymmetric filter value of the energy and the rise of the current energy for the filtered value are obtained.

Then the program moves to FIG. 5B. Steps in Figure 5A
The variable r, shown at 10, is such that the current log energy exceeds its asymmetric smoothness. During vowel nuclei, r is positive and remains positive for a significant period of time.

This is shown to have a special meaning for the positive and negative periods, so special handling is required the first time it goes positive or negative. This is shown in detail in FIG. 5B. When r first becomes positive, it records the frame number as being the beginning of a fixed speech nucleus. It then operates to reset the value P used to determine if it is speech and to interrupt noise tracking. In any case, while r remains positive, the value P is accumulated, and if P exceeds a certain threshold, the artifact and speech flags are set. These are shown on the left side of FIG. 5B. When r becomes positive for the first time, the noise tracker is reset to the last known noise value, and while confirming that the assumed speech level is sufficiently high from the noise level, no speech or artifact is detected. If so, the noise tracking is resumed after a predetermined delay. If speech is detected during this rise, the frame number is recorded as the end of the known speech interval.

Continue to track noise after a predetermined delay while r stays negative. This is all shown in the flow chart, which clearly describes the various operations provided.

FIG. 5C basically shows the generation of output variables used to provide the operation templates shown in FIGS. 2 and 4, for example. Thus, as can be seen from the above, the main idea of the system of the present invention is to generate a template and thereby add noise in the correct expected way to form a template with an associated expected S / N ratio. The noise level associated with the template indicates an estimate of the noise present in the incoming signal. In this way, the recognition probability of the speech recognition system is substantially increased.

Generating such a template by adding noise as described above is an option to determine whether the signal is actually speech, artifact, or noise, compared to the incoming signal using the template. Can be used for the voice recognition system. Thus, the system provides an improved speech recognition template that is first formed in a noisy condition and modified for use in noisy conditions by modifying it to be equal to these expected values of the noisy condition. To work.

[Brief description of the drawings]

FIG. 1A is a block diagram illustrating a speech recognition system using recognition parameters obtained from a spectrum using the present invention. FIG. 1B is a block diagram illustrating another speech recognition system using recognition parameters whose properties are spectra according to the present invention. FIG. 2 is a detailed block diagram showing the technique according to the present invention for forming operation template data. FIG. 3 is a diagram showing a table of definitions of various outputs shown in FIG. FIG. 4 is a detailed block diagram of another embodiment of the present invention. 5A to 5C are detailed flowcharts illustrating the operation of the speech and noise tracker according to the present invention. FIG. 6 is a diagram showing a table of definitions of engineering parameters according to FIGS. 5A to 5C. 10… Microphone, 11… Amplifier, 12… Spectrum analyzer, 13,100… Switch, 14,15,16,20,21,25,27,4
0,162,166 ... module, 26 ... tracker, 160 ... processor, 31, 32 ... adder, 54 ... logarithmic circuit.

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-59-137999 (JP, A) JP-A-62-65088 (JP, A) JP-A-62-42198 (JP, A) 67197 (JP, B2) JP-B 61-2960 (JP, B2) JP-B 5-56519 (JP, B2) U.S. Pat. Proceedings of the 51st Spring Meeting of Spring Meeting, 4-2-10, "Effects of External Noise on Speech Recognition System", pp. 527-528 (issued on May 25, 1976) (58) .Cl. ⁷ , DB name) G10L 15/06 G10L 15/20 G10L 21/02 JICST file (JOIS) WPI (DIALOG)

Claims (15)

(57) [Claims] 1. A storage device for storing an initial template indicating a spectrum value of a recognizable speech in a noise-free state, and a spectrum analyzer for outputting a spectrum value of an utterance of an input signal indicating a speech in a noise-existing state. When,
An apparatus for generating a motion template, and comparing the motion template with an output spectrum value from the spectrum analyzer to output a good comparison result indicating that there is a recognized speech in the utterance. A speech recognition system of the type comprising: a first module coupled to the spectrum analyzer for outputting an estimated noise signal indicative of noise in the input signal. Means coupled to the first means and responsive to the estimated noise signal for generating the modified motion template from the initial template based on the estimated noise signal; The first means detects an audio signal in a state where noise is present during the utterance,
A voice tracking unit that outputs a first signal that is a scalar voice level value related to the voice signal and that indicates an average power of the voice signal in the presence of noise; and detects noise in the voice and estimates noise. A noise tracking unit that outputs a second signal indicating a mean power of noise for a predetermined time period of the utterance, which is a vector of spectral values indicating values, and a voice and noise level tracking unit, Adjusting the audio level of the initial template based on the first signal, and adding a spectral value of an estimated value of noise to the initial template based on the second signal. Generating the motion template, the estimated noise of the utterance of the input signal and the input signal in order to obtain the improved speech recognition performance of the recognition module. A speech recognition system having the same signal-to-noise ratio as the utterance. 2. The spectrum analyzer according to claim 1, further comprising a plurality of bandpass filters arranged in a filter bank array, each filter configured to pass a predetermined spectral component according to a band of the filter. 2. The speech recognition system according to 1. 3. The speech recognition system according to claim 2, wherein said first means comprises means for measuring an average and a fluctuation of said bandpass filter and outputting an estimated value of a noise pass characteristic of each filter. 4. The speech recognition system according to claim 3, wherein said noise estimate is estimated based on said filter response to Gaussian noise. 5. The template generated in the absence of noise is formed from a token without noise, means for forming an average value in response to the token and outputting basic form data, and a current expected noise. 2. The apparatus according to claim 1, further comprising: means for modifying the basic form data based on the signal.
A speech recognition system as described. 6. A processing means coupled to said spectrum analyzer for generating said operational template for storage by modifying said initial template based on an estimated noise signal indicative of the presence of noise. 2. The speech recognition system according to 1. 7. The speech recognition system according to claim 6, wherein the expected value calculated by said processing means indicates the presence of Gaussian noise. 8. A means for averaging a noise-free template to obtain a basic form data output, and modifying the basic form data output by adding calculated noise data to the basic form data. The speech recognition system according to claim 6, comprising: 9. An averaging means for outputting an average value of a continuous pair of the magnitude values of the spectrum output by the analyzer, and a processing means coupled to an output of the averaging means, 7. The voice according to claim 6, further comprising scaling means for outputting a field signal having a length, and means for converting the field signal having the predetermined length into a logarithmic signal and outputting one of the basic form data outputs. Recognition system. 10. A squaring means coupled to said averaging means for outputting a vector signal indicating a squared magnitude of an average value of said continuous pair, and coupled to an output of said squaring means,
The speech recognition system according to claim 9, further comprising: a unit for outputting another output of the basic form data output. 11. A means coupled to an output of said squaring means, comprising: a relative energy forming means for outputting a basic morphological energy parameter in response to said vector signal; and a basic energy indicating means for indicating both speech and noise power levels. 11. The speech recognition system according to claim 10, further comprising voice and noise level tracking means for outputting a morphological parameter. 12. A method for forming a motion template for use in a speech recognition system for recognizing speech in the presence of noise in the utterance of an input signal based on a comparison between a spectral value of the input signal and the motion template. Detects audio signals in the presence of noise in the input signal,
Outputting a first signal, which is a scalar audio level value related to the audio signal, indicating an average power of the audio signal in the presence of noise, detecting noise during the utterance, and calculating an estimated value of the noise. Outputting a second signal, which is a vector of spectral values indicating the average power of the noise for a predetermined period of time of the utterance, thereby outputting an estimated noise signal; Adjusting the speech level of the initial template based on the first signal to form the motion template having the same signal-to-noise ratio as the utterance of the first template and obtain an improved speech recognition function; Adding the spectral value of the noise estimate to the initial template based on the second signal, thereby providing an initial indication of recognizable speech in the absence of noise. A method comprising modifying a template. 13. The method of claim 12, wherein outputting the estimated noise signal comprises measuring a response of a predetermined speech processing channel with respect to noise and estimating the signal to be output based on the measurement. Method. 14. The method of claim 12, wherein said modifying step comprises first forming a relatively noise-free basic form template and modifying the basic form template based on the signal indicating the expected noise level. . 15. The modifying step includes forming a relatively noise-free basic form template, adding noise to each template, and averaging the added noise template data to form a new template based on the analysis data. 13. The method of claim 12, comprising steps.