JP3088357B2 - Unspecified speaker acoustic model generation device and speech recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an unspecified speaker by performing speaker normalization on an initial speaker model using characteristic parameters of speaker-dependent speech data and then performing unspecified speaker conversion. An unspecified speaker acoustic model generation apparatus that generates a hidden Markov model (hereinafter, referred to as HMM) that is a speakerized acoustic model, and a speech recognition apparatus that performs speech recognition using the generated unspecified speaker HMM About.

[0002]

2. Description of the Related Art When considering a speech recognition application, there is a strong demand for an unspecified speaker speech recognition system that can be used without prior speaker registration. However, the current unspecified speaker speech recognition performance is lower than that of the specific speaker speech recognition, and the difference is about two to three times the error rate. In order to improve the performance of the speaker-independent speaker recognition, a speaker adaptation process (for example, using a small amount of adaptation data uttered by a particular speaker) to bring the acoustic model of the speaker-independent speaker recognition closer to the particular speaker.
Prior art document 1 “CLLeggetter et al.,“ MaximumLike
lihood Linear Regression for Speaker Adaptation of
Continuous Density Hidden Markov Models ", Computer
Speech and Language, Vol. 9, pp. 171-185, 1995. " However, a large amount of training adaptation data is required until the performance is equivalent to that of the specific speaker speech recognition.

[0003] In general, an unspecified speaker HM independent of the speaker
Learning of M (hereinafter referred to as SI-HMM) is performed using voice data of a plurality of speakers. In spite of the fact that the learning data contains not only differences depending on speakers, but also differences such as situations (contexts) where units to be learned are placed, an acoustic model for specific speaker speech recognition (HMM depending on speakers) (Less than,
It is called SD-HMM. ). The processing is performed in the same manner as in the learning of ()). As a result, in the SI-HMM, both the variation due to the difference between speakers and the variation in the phonemic context are mixed, resulting in a model having a large spread. This is considered to be one of the factors of the degradation of the identification performance. In the case of a speech recognition system based on a continuous mixture distribution type HMM, the variance of the Gaussian distribution is large, and there is a problem in that recognition units are overlapped with each other, making identification difficult.

[0004] In particular, when speaker adaptation is performed using the conventional multiple regression mapping model disclosed in the prior art document 1, when the adaptation data for learning is small, estimation of adaptation parameters is performed. There is a problem that the accuracy is relatively poor and the speech recognition rate is relatively low.

[0005] In order to solve the above problems, the present applicant has disclosed a speaker normalizing apparatus and a speaker adapting apparatus in Japanese Patent Application No. 09-054596. The speaker normalization apparatus uses a maximum likelihood linear regression method to generate a multiple regression mapping model for an initial model of a predetermined hidden Markov model based on a feature vector of speech data that depends on each of a plurality of speakers. First calculating means for calculating, for each speaker, a first conversion coefficient including a conversion matrix and a constant term vector for converting an average vector based on the plurality of speakers, A second computing means for subtracting a constant term vector computed by the first computing means for each speaker from the vector to compute a feature vector of voice data normalized, and a second computing means By learning the initial model of the hidden Markov model using a predetermined learning algorithm based on the feature vector of the normalized speech data calculated by the means, Further comprising a third calculating means for calculating a model parameter of the normalized HMM "the is characterized. Further, the speaker adaptation apparatus may be configured such that “based on the feature vector of the voice data of the speaker to be speaker-adapted, the hidden Markov model calculated by the third calculation unit of the speaker normalization apparatus is used. A second matrix including a conversion matrix and a constant term vector for converting an average vector based on a multiple regression mapping model by a maximum likelihood linear regression method.
Based on a fourth calculating means for calculating a conversion coefficient of the following, and a second conversion coefficient including a conversion matrix and a constant term vector calculated by the fourth calculating means, by a maximum posterior probability estimating method, Calculating means for calculating a third conversion coefficient including a conversion matrix and a constant term vector for converting an average vector based on the generalized multiple regression mapping model, and a conversion calculated by the fifth calculating means And a sixth operation means for executing an average linear vector of the hidden Markov model after speaker adaptation by executing a predetermined linear conversion process on the third conversion coefficient including the matrix and the constant term vector. That is the feature.

[0006]

However, the above speaker adaptation apparatus has a problem that it is necessary to perform speaker adaptation processing at the time of speech recognition. A system without speaker adaptation processing is desired as an unspecified speaker model. An object of the present invention is to solve the above problems and provide an unspecified speaker acoustic model generation device and a speech recognition device capable of improving the speech recognition rate in the unspecified speaker speech recognition as compared with the related art. Is to do.

[0007]

According to a first aspect of the present invention, there is provided an apparatus for generating an unspecified speaker acoustic model based on a feature vector of speech data which depends on a plurality of speakers.
For the initial model of the predetermined hidden Markov model, the first conversion coefficient including a conversion matrix and a constant term vector for conversion of an average vector based on the multiple regression mapping model is determined by the maximum likelihood linear regression method. The first calculation is to obtain a hidden Markov model adapted for each speaker by calculating
Based on the hidden Markov model adapted for each speaker obtained by the first calculating means, from the voice data and the text data of the uttered content, using a Viterbi algorithm, A second operation of calculating an optimal state sequence and calculating a mixture distribution sequence in which the feature vector of the audio data indicates the maximum output probability for each optimal state at each time;
And speaker normalization of the feature vector of the voice data using the average vector before and after the speaker adaptation of the mixture distribution in each state of the optimal state sequence calculated by the second arithmetic unit. Accordingly, the third calculating means for calculating the feature vector of the voice data normalized by the speaker, and the hidden Markov based on the feature vector of the voice data normalized by the third calculating means. A fourth calculating means for calculating the model parameters of the speaker-normalized hidden Markov model by learning an initial model of the model using a predetermined learning algorithm; and a fourth calculating means for calculating the model parameters. For the speaker-normalized hidden Markov model, maximum likelihood linear regression
A hidden Markov model adapted for each speaker by calculating a second conversion coefficient including a conversion matrix and a constant term vector for converting an average vector based on the multiple regression mapping model for each speaker And an adapted hidden Markov model average vector obtained by the fifth operation means, and speaker-normalized hidden average calculated by the fourth operation means. Based on the model parameters of the Markov model, the average vector and the covariance matrix of the hidden Markov model converted to the unspecified speaker are calculated by making the speaker unspecified, and the hidden Markov converted to the unspecified speaker is calculated. And a sixth calculating means for obtaining a model.

According to a second aspect of the present invention, there is provided a speech recognition apparatus, comprising the steps of inputting a hidden Markov model calculated by a sixth calculating means of the unspecified speaker acoustic model generating apparatus. Voice recognition means for performing voice recognition on the basis of the voice signal of the uttered voice sentence and outputting a voice recognition result.

[0009]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of a speech recognition apparatus according to an embodiment of the present invention. This embodiment is characterized in that a speaker normalization control unit 20 and an unspecified speaker conversion control unit 21 are provided.

In this case, the speaker normalization control section 20 has the following steps:
Speech data 32-1 depending on each of a plurality of M speakers
To 32-M feature vectors O ^m (m = 1, 2,...,
_{^{M) = [o m 1,}} o m 2, ..., o m Tm] based on a predetermined HMM initial model (hereinafter, referred to as initial HMM.) 31
By using the maximum likelihood linear regression method, the first transformation coefficients A ^m and b ^m including the transformation matrix for transforming the average vector based on the multiple regression mapping model and the constant term vector are expressed by the following equation (1).
Each speaker m (m = 1,2, ..., M) by using by calculating for each, after obtaining the HMMramudah ^m which is adapted at the each speaker, each story obtained (b) above H adapted for each person m
Based on MMramudah ^m, the from the voice data and text data of the speech content (. Are stored together with audio data 32-1 to 32-M in memory), using a Viterbi algorithm, the best state sequence p ^m = [ p ^m ₁ , p ^m ₂ , ...,
p ^m _Tm ], and a mixed distribution sequence q ^m = [q ^m ₁ , q ^m ₂ ,..., q ^m _Tm ] in which the feature vector O ^{m of the} audio data indicates the maximum output probability for each optimal state at each time. and it was calculated using the two numbers to be described later, by using the average vector of the front and rear speaker adaptation of mixture distribution q ^m _t in each state p ^m _t of (c) above calculated optimum state sequence, the voice by speaker normalization feature vector data, the feature vector Ob = voice data speaker normalization ^{[Ob 1, O b 2,} ..., Ob M] and calculates using a number 3 to be described later (D) The initial HMM is calculated using the Baum-Welch learning algorithm using Equations 4 to 8 described below, based on the calculated normalized feature vector Ob of the audio data. Speaker-normalized HM
The model parameters of Mλb are calculated. Here, the model parameters include HMM model parameters such as mean vector, variance of Gaussian distribution, and state transition probability.

In addition, the unspecified speaker control unit 21 includes (e)
For the calculated speaker-normalized HMMλb,
Using a maximum likelihood linear regression method, a second transformation coefficient including a transformation matrix for transforming an average vector based on a multiple regression mapping model and a constant term vector is calculated for each speaker m using Expression 9. Thus, the HM adapted for each speaker m
After obtaining the average vector of M (where the parameter to be adaptively learned is the average value of the Gaussian distribution), (f) the average vector of the obtained adapted HMM and the speaker normalization control unit Speaker-normalized HM calculated by 20
Based on the covariance matrix, which is a model parameter of M,
The average vector and the covariance matrix of the HMM λa which is made into an unspecified speaker is calculated by making the speaker unspecified by using Expressions 10 and 11 described later, and the HM which has been made unspecified speaker is calculated.
Obtain Mλa. Here, the speaker-normalized parameters of the HMMλb are used for the transition probability of the HMMλa and the mixing weight coefficient.

Further, the speech recognition apparatus of FIG. 1 performs speech recognition based on the speech signal of the input uttered speech sentence by using the HMM 11 converted to the unspecified speaker, and outputs a speech recognition result.

First, the creation of an unspecified speaker model using speaker normalization in this embodiment will be described. When batch processing is performed using naturally uttered speech data such as conversation, an utterance style is greatly different for each learning speaker, so that an acoustic model having a large spread is generated. Also, the utterance contents are different for each speaker, and the number of learning speakers is biased for each recognition unit,
Fluctuations due to different speakers cannot be correctly expressed. Therefore, the HMM has a large spread as shown in FIG.
It is considered that the output probability distribution is biased and the performance of speaker-independent speech recognition is reduced. Therefore, an unspecified speaker model using naturally uttered speech is created by the following two-stage processing. (A) Speaker normalization is performed, and only the phonemic fluctuation within the speaker is performed.
Learning is performed to obtain the speaker normalization model (SN model: λb) of (b). (B) The variation due to the difference in the speakers is estimated based on the speaker normalization model λb, and the speaker-normalized unspecified speaker model (SN-SI model: λa) shown in FIG. 6C is obtained.

First, speaker normalization of learning data will be described. The speaker normalization is a set O = [O ¹ , O ² …, O of an observation sequence of feature parameters from M learning speaker voice data.
^M ] (The observed sequence of the feature parameter of speaker m is O ^m =
_{^{[O m 1, o m 2}} , ..., o m Tm] (o is an n-dimensional vector, subscripts the time (specifically, the frame number)
It is. ), A set of speaker-normalized observation sequences Ob = [Ob
^1, Ob ^2, ..., carried out by obtaining the Ob ^M]. In the present embodiment, a speaker normalization method using a speaker adaptation method and assuming that the relative position between the speaker adaptation model and the observation vector is a speaker normalized observation vector will be described. The adaptive model λh ^{m of the} speaker ^m (in this specification, the model is an HMM) is obtained by adaptive learning using the initial model as λ and the observation sequence O ^m of the feature vector as learning data. Here, the maximum likelihood linear regression method (Maximum Li
kelihood Linear Regression; hereinafter, referred to as the MLLR method. See, for example, Prior Art Document 1. ), The average vector μ (j, k) of the Gaussian distribution (mixture distribution k in the state j) is mapped to the adaptive average vector μh (j, k) by the following equation.

Equation 1 μh ^m (j, k) = A ^m μ (j, k) + b ^m

Here, A ^m and b ^m are an n × n matrix and an n-dimensional vector, respectively, which are estimated for each Gaussian distribution sharing class. N is the number of dimensions of the feature vector. FIG. 4 shows a conceptual diagram of the processing by the MLLR method.

Next, using the adaptive model λhm of the speaker m, the optimal state sequence p ^m = V ^m by the Viterbi algorithm from the observed sequence O ^m of the feature vector of the speaker ^m and the text data of the utterance content. _{^{[p m 1, p m 2}} , ...,
p ^m _Tm] look, for each optimal state at each time, mixing distribution sequence observation sequence O ^m of feature vectors indicating the maximum output probability q ^m =
[Q ^m ₁ , q ^m ₂ ,..., Q ^m _Tm ] are extracted by the following equation.

[Number 2] _{^{q m t = argmax [c (}} p m t, q) · N (o m t, μ
_{^{h m (p m t, q}} ), U (p m t, q))] q∈Q m t

[0018] Here, Q ^m _t is the set of mixture distribution in the optimal state at time t, c is mixed weighting coefficient, U is the covariance matrix. The function argmax (·), upon changing the variable q by q∈Q ^m _t The condition is a function representing the value of the variable q when the function value is maximized. Further, the function N
(·) Is the output probability when the set characteristic parameter o ^m _t is a variable, the mean vector _{^{μh m (p m t, q}} ) and covariance matrix U (p ^m _t, q) a. Then, speaker normalization observation sequence _{^{Ob m = [ob m 1,}} ob m 2, ..., ob m Tm] is obtained in the above, the mixture distribution q ^m _t in state p ^m _t speaker adaptation before and after The average vector is obtained according to the following equation.

[Number 3] ^{_{ob m t = o m t -μh}} m (p m t, q m t) + μ (p m
_t , q ^m _t )

[0019] That is, the observation sequence o ^m _t of feature parameters of the speech data, the mean vector .mu.H ^m after speaker adaptation
_{^{(P m t, q m t}} ) as well as subtracting the average vector _{^{μ (p m t, q m}} t) before the speaker adaptation by adding, together origin before the speaker adaptation, the the speaker normalizing the observation series o ^m _t. FIG. 5 shows a conceptual diagram of the speaker normalization processing. The above processing is performed for all the learning speakers, that is, for each speaker, and a set of speaker normalized observation sequences Ob = [Ob ¹ , Ob ² ,..., Ob
^m ].

Next, learning of the speaker normalization model will be described. First, re-learning of the initial model λ is performed using the speaker normalized observation sequence Ob. The mean value of the Gaussian distribution and the covariance matrices μb (j, k) and Ub (j, k) are updated by the following equations.

(Equation 4)

(Equation 5)

Here,

(Equation 6)

(Equation 7)

(Equation 8)

Here, γ ^m _t (j, k) is an expected value of the feature parameter ob ^m _t observed in the mixture k of the state j. ｛·｝ ′ Represents a transposed matrix. Other, HMM
Are updated in the same manner. The updated acoustic model is replaced with the above-described initial model λ, the normalization process is repeated a fixed number of times, and the finally obtained model is set as an SN model λb.

Next, creation of a speaker-normalized unspecified speaker model will be described. For the purpose of speaker-independent speech recognition,
A method of creating a speaker normalization model (SN-SI model) in which a variation due to a difference between speakers is expressed will be described. The variation due to the difference between speakers is expressed by creating a speaker adaptation model for each learning speaker using the speaker normalization model as an initial model, and synthesizing a Gaussian distribution.

(A) Using the SN model λb as an initial model,
Similar to the MLLR process of the speaker normalization control unit 20, the MLL
According to the R method, an adaptive model for each learning speaker is created by the following equation. The adapted parameter is the mean value of the Gaussian distribution.

Μhb ^m (k, j) = Ab ^m μb (j, k) + bb ^m (b) Average vector μh ^m (j, k) of adaptive model and S
From the covariance matrix Ub (j, k) of the N model, an average vector μa (j, k) and a covariance matrix Ua (j, k) are obtained by Expressions 10 and 11, and an SN-SI model λa is obtained. Here, the values of the SN model are used for the transition probability and the mixing weight coefficient.

(Equation 10)

[Equation 11]

In FIG. 1, a speaker normalization control unit 20, an unspecified speaker conversion control unit 21, a feature extraction unit 2, a phoneme verification unit 4,
The LR parser 5 is composed of, for example, an arithmetic and control unit such as a digital computer. The buffer memory 3 is, for example, a hard disk memory, and has an initial HMM 31 and speakers 1 to M.
, The speaker-normalized HMM 33, the speaker-independent HMM 11, LR
The table 12 and the context-free grammar 13 are stored in, for example, a hard disk memory. In addition, the voice data 3 of each speaker
Reference numerals 2-1 to 32-M denote feature parameter vectors extracted from the speech waveform signal of each speaker, that is, text data of the feature vector and its utterance content. In this specification, audio data refers to a feature vector.

FIG. 2 is a flowchart showing a speaker normalization process executed by the speaker normalization control unit 20 of FIG. First, in step S1 of FIG. 2, the speech data 32-1 to 32-M of each speaker m is read, and an initial HMM (an initial model of the HMM) 3 which is an unspecified speaker HMM is read.
1 is read out and set as the HMM to be processed. Next, in step S2, as shown in FIG. 4, the feature parameter O ^m of the speech data 32-1 to 32-M for each speaker m is adaptively learned as learning data for the HMM to be processed. Then, the average vector μ (j, k) of the Gaussian distribution is changed to the adaptive average vector μ for each speaker by using Equation 1 by the MLLR method.
By mapping to h ^m (j, k), an adaptive model λhm of speaker ^m is obtained. Then, in step S3, by using an adaptive model .lambda.h ^m for each speaker with observation sequence O ^m by the Viterbi algorithm from the text data of the utterance contents, calculates the optimal state sequence p ^m, for each optimal condition at each time Then, a mixed distribution sequence q ^{m in} which the observation sequence O ^m shows the maximum output probability is calculated using Equation 2. Furthermore, the calculation according to the equation 3 using the average vector of speaker adaptation before and after the mixture distribution q ^m _t speaker in normalized observation sequence Ob ^m states p ^m _t at step S4. Further, in step S5, the initial HMM is re-learned using the speaker normalized observation sequence Ob ^m using the Balm-Welch learning algorithm. Then, it is determined in step S6 whether or not the predetermined number of repetitions has been reached. If not, the HMM after the re-learning is set as the processing target HMM in step S7, and the process returns to step S2 again to repeat the above processing. Execute. On the other hand, in step S6, a predetermined number of repetitions (5 in the preferred embodiment).
When, the HMM after the re-learning is stored in the memory as the speaker-normalized HMM 33 in step S8. Then, the speaker normalization processing ends.

FIG. 3 is a flowchart showing an unspecified speaker conversion process executed by the unspecified speaker conversion control unit of FIG. In step S11 of FIG. 3, the speaker-normalized HM
M33 and voice data 32-1 to 32-M of each speaker are read. Next, in step S12, an adaptive model for each speaker is calculated by using the MLLR method using Expression 9. Further, in step S13, a model λb converted to an unspecified speaker is calculated using equations (10) and (11). Finally, the HMM 11 that has been turned into an unspecified speaker is stored in the memory. Then, the unspecified speaker conversion process ends.

The unspecific speaker HMM 11 is connected to the phoneme matching unit 4 and can be represented as a network in a plurality of states as an HM network. Each state in the HMM 11 can be regarded as one stochastic stationary signal source in the sound space, and each has the following information. (A) state number, (b) acceptable context class, (c) list of preceding and following states,
(D) parameters of a probability distribution assigned to the feature space of speech, (e) self-transition probability and transition probability to a subsequent state. In the unspecific speakerized HMM 11, when input data and its context information are given, a model for the input data is connected by linking the states that can receive the context within the constraints of the preceding and succeeding state lists. Can be uniquely determined. Here, the output probability density function is a mixed Gaussian distribution having a 34-dimensional diagonal covariance matrix (referred to as a Gaussian distribution in this specification), and each Gaussian distribution is controlled by speaker normalization control using an initial HMM 31. Using the HMM 33 that has been speaker-normalized by the unit 20 and speaker-normalized, the unspecific speaker conversion unit 21 converts the speaker into an unspecified speaker.

In general, when speaker adaptation is performed on a model based on a continuous distribution type HMM with a small amount of adaptation data, adaptation of the average value of the Gaussian distribution is known to be more effective than adaptation of other parameters. (For example, Prior Art Document 2, "Kumi Okura et al.," Moving Vector Field Smoothing Speaker Adaptation Method Using Mixed Continuous Distribution HMM ", Proceedings of the Acoustical Society of Japan, 2-Q-17, pp. 191-192, 1992
March ". ). In the present embodiment, only the average value of each Gaussian distribution is applied, and the variance, the state transition probability, and the weight coefficient of the mixed Gaussian distribution are not applied.

Next, the SSS-LR (left-t) using the above-described speaker normalization method and speaker adaptation method according to the present embodiment.
An o-right rightmost type) speaker-independent continuous speech recognition device will be described. This device is a phoneme environment-dependent efficient HM stored in the memory of the HM network including the HMM 11.
M expression format is used. In the SSS, the likelihood of a stochastic model expressing a temporal transition of a speech parameter by a stochastic transition between stochastic stationary signal sources (states) assigned to a feature space of a phoneme is calculated. The refinement of the model is performed sequentially by repeating the operation of dividing each state in the context direction or the time direction based on the criterion of maximization.

In FIG. 1, a uttered voice of a speaker is input to a microphone 1 and converted into a voice signal, and then input to a feature extracting unit 2. After performing A / D conversion on the input audio signal, the feature extraction unit 2 performs, for example, LPC analysis, and performs 34-dimensional feature parameters including logarithmic power, 16th-order cepstrum coefficient, Δlogarithmic power, and 16th-order Δcepstrum coefficient. Is extracted. The time series of the extracted feature parameters is input to the phoneme matching unit 4 via the buffer memory 3.

The phoneme collating unit 4 executes a phoneme collating process in response to a phoneme collation request from the phoneme context-dependent LR parser 5. Then, the likelihood for the data in the phoneme matching section is calculated using the speaker model of the phoneme HMM stored in the memory of the speaker-adapted HMM 11, and the value of the likelihood is used as the phoneme matching score in the LR parser 5. Is returned to
At this time, a forward path algorithm is used.

On the other hand, a predetermined context-free grammar (CFG) in the context-free grammar database 13 is automatically converted as is well known, and an LR table 12 is created and stored in its memory. The LR parser 5 refers to the LR table 12 and processes the input phoneme prediction data from left to right without regression. If there is syntactic ambiguity, the stack is split and the analysis of all candidates is processed in parallel. The LR parser 5 uses the LR table 1
It predicts the next phoneme from 2 and outputs phoneme prediction data to the phoneme matching unit 4. In response, the phoneme matching unit 4
The matching is performed with reference to the information in the HMM 11 corresponding to the phoneme, the likelihood is returned to the LR parser 5 as a voice recognition score, and the continuous voice recognition is performed by sequentially connecting the phonemes. If multiple phonemes are predicted in the above continuous speech recognition, check for the presence of all of them,
By the beam search method, high-speed processing is realized by performing pruning to leave a partial tree having a high likelihood of partial speech recognition.

[0034]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present inventor conducted an evaluation experiment on a speech recognition apparatus having the above-described configuration as follows. As an evaluation experiment, a continuous word recognition experiment was performed using Table 1 as experimental conditions.

[0035]

[Table 1] Experimental conditions 音響 Acoustic analysis ───────サ ン プ リ ン グ Sampling frequency: 12 KHz Frame shift: 10 ms Frame length: 20 ms (Hamming window) Feature vector: 16th order cepstrum Coefficient, 16th order cepstrum coefficient, logarithmic power, Δpower 音 声 audio data ─────────────────────────────────── Travel conversation task learning: 99 men (13K words, 1237 utterances) 131 women (20K words, 1725 utterances) Evaluation: 16 men (2102 words, 196 utterances) 19 women (2844 words, 2 4 utterances) ─────────────────────────────────── HMM ─────────── Ｈ Silence model of HMNet (5 mixture / state) + 1 state (10 mixture) created by ML-SSS ─────────────────────────────── Language model ───────────────── ────────────────── Variable length N-gram learning: 414,326 words (6,396 different words), perplexity: 19.34３４ ─────────────────────────────

As voice data, travel conversation voice data recorded by the present applicant is used, and a male speaker model, a female speaker model, and a gender-unspecified speaker model are created and evaluated from data of 99 men and 131 women. did. The structure of the acoustic model also uses the above-mentioned voice data, and the sound model is a well-known ML-SSS.
(Maximum likelihood successive state splitting)
Was used, and the silent model was 1 state (10 mixtures). SN-S
In the MLLR method used for creating the I model, the sharing class is 1
6 (determined by clustering the Gaussian distribution of the unspecified speaker model based on the known Kullback divergence method), and estimated the diagonal components of the regression matrix A and the constant term b. The number of repetitions of the speaker normalization process is set to 5 times, and a Baum-Wee
lch). The language model is a known variable-length N-gram (for example,
Prior art document 3 “H. Masataki et al.,“ Variable-Order
N-Gram Generation by Word-Class Splitting and Con
secutive Word Grouping ", Proceedings of ICASSP'96, p
188-191, 1996. " ), And the recognition result was evaluated as the first-place candidate of the beam search using the word graph.

Next, the experimental results will be described. SN-
Using the SI model, a continuous speech recognition experiment was performed on 16 males and 19 females (acoustic model open, language model closed). 401, 60 HMM states
Table 2 shows the results of the word accuracy evaluation when the words are 1,801 and 1001. For comparison, the recognition results of the SN model and the conventional speaker-independent (SI) model are also shown.

[0038]

[Table 2] Continuous word recognition result-Word error rate (%) ─────────────────────────────────── Number of states of HMM (number of distributions) ４０１ 401 601 801 1001 (2010) (3010) (4010) (5010) ─────────────────────────────────── Male speaker 44.3 42.6 43.9 40.4 Model 48.1 49.6 50.8 44.9 45.4 46.8 45.7 41.2 ────────── Female speaker 29.9 28.7 30.3 28.6 Model 32.9 32.1 35.0 35.0 31.6 30.3 32.9 31.5 ─────────不 Unspecified gender 36.0 31.9 33.8 33.5 Speaker model 38.9 35.5 37.3 38.5 37.7 33.8 34.7 35.4 (Note) Upper: Speaker-normalized unspecified speaker (SN-SI) model Middle: Speaker-normalized (SN) model Lower: Unspecified speaker (SI) model

As is clear from Table 2, the SN-SI model obtained recognition results that exceeded the conventional SI model in all acoustic model types. It can be seen that it is effective to estimate the fluctuation due to the speaker difference after learning the phoneme fluctuation in the speaker by performing the speaker normalization. The SN model has a lower recognition rate than the conventional SI model. Since this does not include fluctuation due to the difference between speakers, it can be understood that the performance is unreliable in unspecified speaker speech recognition.

As described above, according to this embodiment, a speaker normalization process is performed for an unspecified speaker model using naturally uttered speech having different utterance contents and utterance styles for each speaker. After learning the phonemic fluctuations within the speaker, the speaker-independent processing was performed by re-learning the fluctuations due to the difference between the speakers. That is, after learning using the feature parameters of the speaker-normalized speech data and generating a speaker-normalized model,
Since the variation due to the difference in speaker characteristics is learned, the adverse effect due to the bias of the training speech data is reduced, and the speech recognition is performed by using the obtained unspecified speaker model, thereby improving the speech recognition rate compared to the conventional technology. It can be greatly improved.

[0041]

As described in detail above, according to the apparatus for generating an unspecified speaker acoustic model according to the present invention, a predetermined hidden Markov model is generated based on a feature vector of speech data which depends on a plurality of speakers. By using a maximum likelihood linear regression method with respect to the initial model, a first conversion coefficient including a conversion matrix and a constant term vector for conversion of an average vector based on the multiple regression mapping model is calculated for each speaker. First computing means for obtaining a hidden Markov model adapted for each speaker, and the speech based on the hidden Markov model adapted for each speaker obtained by the first computing means. From the data and the text data of the utterance contents, the optimal state sequence is calculated using the Viterbi algorithm,
A second calculating means for calculating a mixed distribution series in which the feature vector of the audio data shows the maximum output probability for each optimum state at each time; and a state in each state of the optimum state series calculated by the second calculating means. A third calculating means for calculating the speaker-normalized feature data of the speech data by subjecting the feature vector of the speech data to speaker normalization using the average vector before and after the speaker adaptation of the mixture distribution; The speaker normalization is performed by learning the initial model of the hidden Markov model using a predetermined learning algorithm based on the feature vector of the normalized speech data calculated by the third calculation means. A fourth calculating means for calculating model parameters of the hidden Markov model;
The speaker-normalized hidden Markov model calculated by the calculation means is subjected to a maximum likelihood linear regression method to include a conversion matrix and a constant term vector for converting an average vector based on a multiple regression mapping model. Calculating the transformation coefficient of each of the speakers to obtain an average vector of the Hidden Markov Model adapted for each of the speakers, and an adaptive coefficient obtained by the fifth arithmetic means. An unspecified speaker is obtained by converting the hidden Markov model into an unspecified speaker based on the average vector of the hidden Markov model and the model parameter of the speaker-normalized hidden Markov model calculated by the fourth calculating means. Sixth arithmetic means for calculating an average vector and a covariance matrix of the generalized Hidden Markov Model to obtain an unspecific speakerized Hidden Markov Model. Therefore, for an unspecified speaker model using naturally uttered speech having different utterance contents and utterance styles for each speaker, the speaker normalization processing is performed to learn the phonemic fluctuation within the speaker, and then the speaker The speaker-independent processing was performed by re-learning the fluctuations caused by the differences in. That is, learning is performed using the feature parameters of the speaker-normalized voice data, and after generating a speaker-normalized model, fluctuations due to differences in speaker characteristics are learned. By performing speech recognition using the obtained speaker-independent model, the speech recognition rate can be significantly improved as compared with the related art.

According to the speech recognition apparatus of the second aspect of the present invention, the input is performed by using the hidden Markov model calculated by the sixth calculation means of the unspecified speaker acoustic model generation apparatus. Voice recognition means for performing voice recognition based on the voice signal of the uttered voice sentence and outputting a voice recognition result. Therefore, since learning is performed using the feature parameters of the speaker-normalized speech data and a speaker-normalized model is generated, the variation due to the difference in speaker characteristics is learned. By performing speech recognition using the obtained speaker-independent model, the speech recognition rate can be significantly improved as compared with the related art.

[Brief description of the drawings]

FIG. 1 is a block diagram of a voice recognition device according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a speaker normalization process performed by a speaker normalization control unit in FIG. 1;

FIG. 3 is a flowchart showing an unspecified speaker conversion process executed by the unspecified speaker conversion control unit in FIG. 1;

FIG. 4 is a diagram illustrating an MLLR process executed by a speaker normalization control unit in FIG. 1;

FIG. 5 is a diagram illustrating a speaker normalization process performed by a speaker normalization control unit in FIG. 1;

6A and 6B are diagrams illustrating output probability distributions of respective models prepared or generated by the apparatus of FIG. 1, wherein FIG. 6A is a diagram illustrating an output probability distribution of an unspecified speaker model, and FIG.
Is a diagram showing the output probability distribution of the speaker normalization model,
(A) is a figure which shows the output probability distribution of the speaker-independent unspecified speaker model.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Microphone, 2 ... Feature extraction part, 3 ... Buffer memory, 4 ... Phoneme collation part, 5 ... LR parser, 11 ... HMM converted into unspecified speaker, 12 ... LR table, 13 ... Context-free grammar (CFG) Database 20: speaker normalization control unit 21: unspecified speaker conversion control unit 31: initial HMM, 32-1 to 32-M: voice data of speakers 1 to M, 33: speaker normalized HMM.

Continuation of front page (56) References Patent 3035239 (JP, B2) IEICE Technical Report [Voice], Vol. 96, No. 448. SP96-91, Jun Ishii, "Speaker normalization and speaker adaptation using multiple regression mapping model", pp. 29-35 (published January 16, 1997). Conference Papers I, 2-6-16, Jun Ishii, "Speaker normalization method for speaker adaptation using multiple regression mapping model," p. 75-76 (Published March 17, 1997) Proceedings of the Acoustical Society of Japan 1997 Fall Meeting, I, 1-1-9, Jun Ishii, "Unspecified talk using speaker normalization" Learning of Acoustic-Sound Models ”, p. 17-18 (Published September 17, 1997) Proceedings of 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. 1, J. Ishii et al, "Speaker Independence Acoustic Modeling Using Speaker Normalization" p. 97-100, May 12-15, 1998, Seattle, Washington, USA Proceedings of 1997 IEEE International Conferencing on Acoustics, Speech and Signal Processing, Vol. 2, J .; Ishii et al, "Speaker Normalization and Adaptation Based on Linear Transformation", p. 1055-1058 (58) Fields studied (Int. Cl. ⁷ , DB name) G10L 15/14 G10L 15/06 JICST file (JOIS) IEEE / IEEE Electronic Library Online

Claims (2)

(57) [Claims] 1. An average vector based on a multiple regression mapping model by a maximum likelihood linear regression method with respect to an initial model of a predetermined hidden Markov model based on a feature vector of voice data which depends on a plurality of speakers, respectively. First arithmetic means for obtaining a hidden Markov model adapted for each speaker by calculating a first conversion coefficient including a conversion matrix for conversion and a constant term vector for each speaker; Based on the hidden Markov model adapted for each speaker obtained by the first calculating means, an optimum state sequence is calculated from the voice data and the text data of the utterance content using a Viterbi algorithm. A second calculating means for calculating a mixed distribution series in which the feature vector of the audio data shows the maximum output probability for each optimal state at each time; Using the average vector before and after the speaker adaptation of the mixture distribution in each state of the calculated optimal state sequence, the feature vector of the audio data is speaker-normalized. A third calculating means for calculating a feature vector; and an initial model of the hidden Markov model, based on a feature vector of the normalized speech data calculated by the third calculating means, using a predetermined learning algorithm. Computing means for computing model parameters of the speaker-normalized hidden Markov model by learning the speaker-normalized hidden Markov model computed by the fourth computing means. And a second conversion coefficient including a conversion matrix and a constant term vector for converting an average vector based on the multiple regression mapping model by the maximum likelihood linear regression method. 5 to obtain the average vector of the Hidden Markov Model adapted for each speaker.
Calculation means, the average vector of the adapted hidden Markov model obtained by the fifth calculation means, and the model parameter of the speaker-normalized hidden Markov model calculated by the fourth calculation means. A sixth operation for calculating the average vector and the covariance matrix of the hidden Markov model converted to the unspecified speaker based on the specified speaker, thereby obtaining the hidden Markov model converted to the unspecified speaker Means for generating an unspecified speaker acoustic model. 2. A speech recognition based on a speech signal of an input uttered speech sentence, using a hidden Markov model computed by a sixth computing means of the speaker-independent acoustic model generation apparatus according to claim 1. And a speech recognition means for outputting a speech recognition result.