EP1846918B1 - Method of estimating a voice conversion function - Google Patents

Abstract

The invention relates to a method of estimating a voice conversion function between (i) the voice of a speaker, defined from a voice message recorded by said speaker, and (ii) the voice of a reference speaker, defined by a speech synthesis database. According to the invention, the method comprises the following steps consisting in: generating a synthetic recording of the voice message recorded by the speaker from the speech synthesis database, and estimating the voice conversion function using a training operation which is performed on the recorded voice message and the synthetic recording.

Description

The present invention relates to a method for estimating a voice conversion function between, on the one hand, the voice of a speaker defined from a voice message recorded by said speaker, and, on the other hand, the voice of a reference speaker defined by a speech synthesis database.

It also relates to a method for estimating a voice conversion function between, on the one hand, the voice of a source speaker defined from a first voice message recorded by said source speaker, and, on the other hand, on the other hand, the voice of a target speaker defined from a second voice message recorded by said target speaker.

The invention finds an advantageous application whenever it is desired to have a speaker say a voice message recorded by another speaker. It is thus possible, for example, to diversify the voices used in speech synthesis systems, or, conversely, to anonymously render messages recorded by different speakers. It is also conceivable to implement the method according to the invention for making dubbing films.

In general, voice conversion consists of estimating a transformation function, or conversion function, which, applied to a first speaker whose voice is defined from a recorded voice message, makes it possible to reproduce as faithfully as possible the voice of a second speaker. In the context of the invention, said second speaker may be a reference speaker whose voice is defined by a voice synthesis database or a so-called "target" speaker whose voice is also defined from a voice message registered, with the first speaker being called "source".

The vocal identity of a speaker depends on many characteristics, whether they are segmental (timbre, pitch of voice, vocal quality), or suprasegmental (speech style). Of these, timbre remains the most important piece of information, which is why most work in the voice conversion field deals mainly with the modification of the timbre. Nevertheless, during the conversion, a modification of the fundamental frequency, also called "pitch", can also be performed in order to respect overall the voice height of the second speaker.

In essence, the principle of voice conversion consists, in known manner, in a learning operation which aims to estimate a function connecting the tone of the voice of the first speaker to that of the voice of the second speaker. For this, two parallel recordings of the two speakers, that is to say containing the same voice message, are necessary. An analysis is conducted on each of the recordings in order to extract representative parameters of the tone of the voice. Then, after aligning the two recordings, we begin by performing a classification, ie a partition of the acoustic spaces of the two speakers. This classification is then used to estimate the conversion function. Many transformation methods based on this principle have been proposed, for example, conversion by vector quantization ( M. Abe, S. Nakamura, K. Shikano and H. Kuwabara, "Voice conversion through vector quantization", Proceedings of ICASSP, pp 655-658, 1988 ), by multiple linear regression ( H. Valbret, "Voice Conversion System for Speech Synthesis", PhD Thesis ENST Paris, 1992 ), by dynamic frequency alignment ( H. Valbret, E. Moulines, JP Tubach, "Voice transformation using PSOLA technique", Speech Communication, vol 11, pp. 175-187, 1995 ), by neural network ( M. Narendranath, HA Murthy, S. Rajendran and B. Yegnanarayana, "Transformation of Formants for Voice Conversion Using Artificial Neural Networks," Speech Communication, Vol 16, pp. 207-216, 1995 ), or by Gaussian Mixture Model (GMM) proposed in " Y. Stylianou, O. Cappe, C. Moulines, Continuous probabilistic transform for voice conversion, IEEE Transactions on Speech and Audio Processing, Vol. 6 (2), pp. 131-142, March 1998 And improved by Kain ( A. Kain and M. Macon, "Text-to-speech voice adaptation of sparse training data," Proceedings of ICSLP, 1998 ).

The methods for estimating voice conversion functions that have just been presented use recordings, or corpus, of parallel messages of the two speakers. However, it is not always possible to obtain such records. This is why, in parallel with the development of conversion methods based on the use of parallel corpora, other work has been done to make conversion possible in cases where the source and target corpora are not parallel. This work is very largely inspired by the speaker adaptation techniques conventionally used in speech recognition by Hidden Markov Model (HMM). An interesting application has been proposed ( J. Yamagishi, M. Tamura, T. Masuko, K. Tokuda and T. Kobayashi, "A context clustering technique for average voice models", IEICE Trans. Inf & Syst, vol. E86-D (3), pp. 534-542, March 2003 ), where the speaker adaptation module allows you to customize an HMM synthesis system. As a first step, a classification of the HMM models in context by decision tree is carried out to build a model of "average" voice. Then, the parameters of these HMM models are adapted according to the target speaker. Both objective and subjective tests have shown the usefulness of the method in the context of HMM synthesis. But the quality of the converted speech accessible by synthesis systems by HMM is nevertheless very poor.

A technique of adaptation to the speaker is also proposed ( A. Mouchtaris, J. van der Spiegel and P. Mueller, "Non-paralel training for voice conversion by maximum likelihood constrained adaptation," In Proceeding ICASSP, 2004, vol 1, pp 1-4. ) to obtain a voice conversion based on nonparallel corpora. In this application, hypothesis is made that two parallel corpora A and B are available. To carry out the conversion between the non-parallel source C and target D corpora, it is furthermore supposed that the corpora C and D are parallel respectively to part of the corpora A and B. In this case, the conversion function between the speakers C and D is expressed as the compound of three conversion functions, respectively speakers C to A, A to B and B to D. The framework of application of this process seems rather restrictive, because it nevertheless requires parallel recording portions. Moreover, no mechanism to control the parallelism of the corpora used is proposed. Finally, the composition of the three conversion functions may lead to significant transformation errors. In the end, the quality of the converted speech obtained by this method is considered less good than that obtained from parallel corpora.

Also, a technical problem to be solved by the object of the present invention is to propose a method of estimating a voice conversion function between, on the one hand, the voice of a speaker defined from a voice message recorded by said speaker, and, on the other hand, the voice of a reference speaker defined by a voice synthesis database, which would provide a converted speech of better quality than that provided by the methods to non-parallel corpora known.

The document US2002 / 0173962 discloses a method for synthesizing a personalized voice from text where the learning operation is performed between a synthethic voice message obtained from the text and a corresponding voice message spoken by the target speaker.

• générer, à partir dudit message vocal enregistré par le locuteur et de ladite base de données de synthèse vocale, un enregistrement synthétique dudit message vocal,
• estimer ladite fonction de conversion de voix par une opération d'apprentissage effectuée sur ledit message vocal enregistré et ledit enregistrement synthétique.

The solution to the technical problem posed consists, according to the present invention, in that said method comprises the steps of:

• generating, from said voice message recorded by the speaker and said voice synthesis database, a synthetic record of said voice message,
• estimating said voice conversion function by a learning operation performed on said recorded voice message and said synthetic record.

Thus, it will be understood that the method according to the invention makes it possible to obtain two parallel recordings of the same voice message, one being recorded directly by the speaker, which constitutes in a way the basic message, and the other being a Synthetic reproduction of this basic message. The estimation of the conversion function sought is then performed by a conventional learning operation performed on two parallel recordings. The different stages of this treatment will be described in detail below.

Two applications of the method according to the invention can be envisaged, namely, on the one hand, an application to the conversion of voice messages recorded by a source speaker into corresponding messages reproduced by said reference speaker, and, on the other hand , an application to the conversion of synthetic messages recorded by a speaker of reference in corresponding messages reproduced by a target speaker. The first application leads to make anonymous, because reproduced by the same reference speaker, voice messages recorded by different speakers. The second application aims, on the contrary, to diversify the voices used in speech synthesis.

• générer, à partir dudit premier message vocal enregistré par le locuteur source et d'une base de données de synthèse vocale, un enregistrement synthétique dudit premier message vocal,
• estimer une première fonction de conversion de voix entre la voix du locuteur source et la voix d'un locuteur de référence définie par ladite base de données de synthèse vocale, par une opération d'apprentissage effectuée sur ledit premier message vocal enregistré par le locuteur source et ledit enregistrement synthétique du premier message vocal,
• générer, à partir dudit deuxième message vocal enregistré par le locuteur cible et de ladite base de données de synthèse vocale, un enregistrement synthétique dudit deuxième message vocal,
• estimer une deuxième fonction de conversion de voix entre la voix dudit locuteur de référence et la voix du locuteur cible, par une opération d'apprentissage effectuée sur ledit enregistrement synthétique du deuxième message vocal et ledit deuxième message vocal enregistré par le locuteur cible,
• estimer ladite fonction de conversion de voix par composition de ladite première et de ladite deuxième fonctions de conversion de voix.

The same principle of parallelization of messages via a reference speaker can be applied to the conversion of voice between two speakers according to a method of estimating a voice conversion function between, on the one hand, the voice of a source speaker defined from a first voice message recorded by said source speaker, and, on the other hand, the voice of a target speaker defined from a second voice message recorded by said target speaker, which according to the invention is remarkable in that said method comprises the steps of:

• generating, from said first voice message recorded by the source speaker and a voice synthesis database, a synthetic record of said first voice message,
• estimating a first voice conversion function between the voice of the source speaker and the voice of a reference speaker defined by said voice synthesis database, by a learning operation performed on said first voice message recorded by the source speaker and said synthetic record of the first voice message,
• generating, from said second voice message recorded by the target speaker and said voice synthesis database, a synthetic record of said second voice message,
• estimating a second voice conversion function between the voice of said reference speaker and the voice of the target speaker, by a learning operation performed on said synthetic record of the second voice message and said second voice message recorded by the target speaker,
• estimating said voice conversion function by composing said first and said second voice conversion functions.

According to a first embodiment of the invention, said voice synthesis database is a database of a concatenated speech synthesis system.

According to a second embodiment of the invention, said voice synthesis database is a database of a speech synthesis system by corpus.

Recall that concatenation synthesis systems can use mono-represented diphon bases. The choice of the diphone, and not the phone (acoustic realization of a phoneme), results from the importance of the transient zone, thus conserved, between two phones for the intelligibility of the speech signal. Diphone synthesis generally leads to a synthetic signal whose intelligibility is quite good. On the other hand, modifications made by the TD-PSOLA algorithm ( F. Charpentier and E. Moulines, "Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones", Proceedings of Eurospeech, 1989 ), in order to satisfy the prosodic instructions, introduce distortions of the synthesis signal and thus significantly degrade the quality of the rendered synthetic speech.

The recent availability of important computer resources has allowed the emergence of new solutions grouped under the name of synthesis by corpus. In this approach, the acoustic database is not restricted to a dictionary of mono-represented diphones, but contains these same elements recorded in different contexts (grammatical, syntactic, phonemic, phonological or prosodic). Each element thus manipulated, also called "unit", is thus a segment of speech characterized by a set of symbolic descriptors relative to the context in which it was recorded. In this corpus approach, the problematic of the synthesis changes radically: it is no longer a matter of distorting the speech signal with the aim of degrading the quality of the timbre as little as possible, but rather of having a sufficiently rich database. and a fine algorithm allowing the selection of the units best adapted to the context and minimizing the artifacts at the concatenation instants. The selection of units can therefore be likened to a problem of minimizing a cost function composed of two types of metrics: a "target cost" which measures the adequacy of the units with the symbolic parameters resulting from the language processing modules of the system and a "concatenation cost" which accounts for the acoustic compatibility of two consecutive units.

For reasons of algorithmic complexity, enumerating and treating from the outset all the combinations of units corresponding to the phonetization of a given text is difficult to envisage. It is therefore necessary to filter the data before deciding on the choice of the optimal sequence. For this reason, the unit selection module generally operates in two steps: firstly a "pre-selection" which consists in selecting sets of candidate units for each target sequence, then a "final selection" which aims to determine the sequence optimal according to a certain predetermined cost function. The pre-selection methods are mostly variants of the method called Context Oriented Clustering introduced by Nakajima ( S. Nakajima and H. Hiroshi, "Automatic Generation of Synthesis Units Based on Context Oriented Clustering", Proceedings of ICASSP, pp. 659-662, New York, USA, April 1988 ). As an example, we can mention the work of Black and Taylor ( AW Black and P. Taylor, "Automatically clustering similar units for speech synthesis", Proceedings of Eurospeech, Rhodes, Greece, September 1997 ) and Donovan ( RE Donovan, "Trainable Speech Synthesis", PhD Thesis, University of Cambridge, United Kingdom, 1996 ) on this topic. The final selection is done by minimizing a cost function, usually by a Viterbi type algorithm. Many cost functions have been proposed, which differ essentially in the nature of the different costs used and in the way these costs are combined. It should be noted, however, that the determination of such heterogeneous cost functions in an automatic way remains delicate, despite the numerous studies in this area ( H. Peng, Y. Zhong, and M. Chu, "Perpetually Optimizing the Cost Function of Unit Selection in a MOS Evaluation", Proceedings ICSLP, pp. 2613-2616, 2002 ), ( SS Park, Kim CK and Kim NS, "Discriminative weight training for unit-selection based speech synthesis ", Proceedings of Eurospeech, pp. 281-284, 2003 ), ( T. Toda, H. Kawai and M. Tsuzaki, "Optimizing sub-cost functions for segment selection based on perceptual evaluations in concatenative speech synthesis", Proceedings of ICASSP, pp. 657-660, Montreal, Canada, 2004 ).

The following description with reference to the accompanying drawings, given as non-limiting examples, will make it clear what the invention consists of and how it can be achieved.

The figure 1 is a block diagram showing the steps of a voice conversion method between a speaker and a reference speaker.

The figure 2 is a block diagram representing the steps of a voice conversion process between a source speaker and a target speaker.

The figure 3 is a diagram of a voice conversion system implementing the estimation method according to the invention.

On the figure 1 is illustrated a voice conversion estimation method between a speaker and a reference speaker. The voice of said speaker is defined from a recorded voice message while the voice of said reference speaker is defined from an acoustic data base of a concatenated speech synthesis system, preferably by corpus. although a mono-represented diphon synthesis system can also be used.

In a first step, a synthetic record parallel to the voice message recorded by the speaker is generated from said voice synthesis data base.

For this purpose, a first block required for generation, called analysis and annotation block 20, is intended to extract from the record of the speaker concerned information of a symbolic type relating to the message contained in said record.

A first type of treatment envisaged consists in extracting the message delivered in text form from the voice recording. This can be obtained automatically by a voice recognition system, or manually by listening and retranscribing voice messages. In this case, the text thus recognized directly feeds the system 30 of speech synthesis, thereby generating the desired reference synthetic record.

However, it may be advantageous to determine the phonetic string actually performed by the speaker. For this, standard acousto-phonetic decoding procedures, for example based on HMM models, can be used. By this variant, it is possible to constrain the speech synthesizer to reproduce exactly the phonetization thus determined.

More generally, it is desirable to introduce an annotation mechanism of the record in order to extract the maximum of information that can be taken into account by the concatenation synthesis system. Among these, the intonation information seems particularly relevant, because it allows to better control the speech mode of the speaker. Thus, a prosodic annotation algorithm can be integrated in the method or a manual annotation phase of the corpus can be considered to take into account melodic markers deemed relevant.

It is then possible to estimate the conversion function sought by applying to the two available parallel records, namely the recorded voice message and the synthetic reference record, a learning operation which will now be described in detail.

• l'analyse acoustique 40,
• l'alignement 50 des corpus,
• la classification acoustique 60,
• l'estimation 70 de la fonction de conversion.

As can be seen on the figure 1 , the processing applied to the two recordings shows different operations necessary to obtain the desired conversion function. These operations are, in order:

• acoustic analysis 40,
• the alignment 50 of the corpora,
• the acoustic classification 60,
• the estimate 70 of the conversion function.

avec $$h\left(n\right)={\displaystyle \sum _{l=1}^L}{A}_l\left(n\right)\cos \left({\phi }_l\left(n\right)\right)$$

The acoustic analysis is carried out for example by means of the HNM model ("Harmonic plus Noise Model") which supposes that a segment (also called frame) voiced of the speech signal s (n) can be decomposed into a harmonic part h ( n) representing the quasi-periodic component of the signal consisting of a sum of L harmonic sinusoids of amplitudes A _l and of phases φ _l , and a noisy part b (n) representing the friction noise and the variation of the glottal excitation from one period to another, modeled by a noise-excited LPC (Linear Prediction Coefficients) filter. white Gaussian ( Y. Stylianou, "Harmonic plus Noise Model for speech, combined with statistical methods, for speech and modification", PhD thesis, National School of Telecommunications, France, 1996 ). $$s\left(\mathrm{not}\right)=h\left(\mathrm{not}\right)+b\left(\mathrm{not}\right)$$

with $$h\left(\mathrm{not}\right)={\displaystyle \underset{l=1}{\overset{\mathrm{The}}{\Sigma }}}{\mathrm{AT}}_l\left(\mathrm{not}\right)\cos \left({\phi }_l\left(\mathrm{not}\right)\right)$$

For an unvoiced frame, the harmonic part is absent and the signal is simply modeled by a white noise shaped by auto-regressive filtering (AR).

The first step of the HNM analysis is to make a decision as to whether the analyzed frame is voiced. This processing is performed in asynchronous mode using an analysis step set at 10 ms.

où s(n) est le signal original, h(n) est la partie harmonique définie par la relation (5) écrite plus loin, w(n) est la fenêtre d'analyse, et T⁰ _i est la période fondamentale de la trame courante. Il convient de noter que la trame d'analyse a une durée égale à deux fois la période fondamentale (voir l'article de Y. Stylianou cité plus haut). Cette analyse harmonique est importante dans la mesure où elle apporte une information fiable sur la valeur du spectre aux fréquences harmoniques. Une telle information est nécessaire pour avoir une estimation robuste de l'enveloppe spectrale.For a voiced frame, the fundamental frequency F ₀ and the maximum frequency of voicing, that is to say the frequency beyond which the signal is considered to consist solely of noise, are first determined. Then, a synchronized analysis on F ₀ makes it possible to estimate the parameters of the harmonic part (the amplitudes and the phases) as well as the parameters of the noise. The harmonic parameters are calculated by minimizing a weighted least squares criterion (see the article by Y. Stylianou cited above): $$E={\displaystyle \underset{\mathrm{not}=-T_0^l}{\overset{T_0^l}{\Sigma }}}{w}^2\left(\mathrm{not}\right)(s\left(\mathrm{not}\right)-h\left(\mathrm{not}\right){)}^2$$

where s (n) is the original signal, h (n) is the harmonic part defined by relation (5) written later, w (n) is the analysis window, and T ⁰ _i is the fundamental period of the current frame. It should be noted that the analysis frame has a duration equal to twice the fundamental period (see the article by Y. Stylianou cited above). This harmonic analysis is important in the extent to which it provides reliable information on the value of the spectrum at harmonic frequencies. Such information is necessary to have a robust estimate of the spectral envelope.

The parts of the spectrum corresponding to noise (whether the noise component of a voiced frame or an unvoiced frame) are modeled using a simple linear prediction. The frequency response of the AR model thus estimated is then sampled at constant pitch, which provides an estimate of the spectral envelope on the noisy areas.

In the proposed embodiment, given this sampling of the spectral envelope, the parameters modeling this spectral envelope are deduced using the discrete regularized cepstrum method ( O. Cappe, E. Mills, Regularization techniques for discrete cepstrum estimation, IEEE Signal Processing Letters, Vol. 3 (4), pp. 100-102, April 1996 ). The order of cepstral modeling was set at 20. Moreover, to reproduce as closely as possible the properties of the human ear, a Bark scale transformation is performed. These coefficients are thus to be compared with MFCCs (Mel Frequency Cepstral Coefficients) conventionally encountered in speech recognition. Thus, for each speech frame, an acoustic vector consisting of cepstral parameters is calculated.

It should also be noted that other types of parameters modeling the spectral envelope can be used: for example Line Spectral Frequency (LSF) or Log Area Ratio (LAR).

After acoustic analysis, it is necessary to match the different acoustic vectors of the two recordings. For this, a conventional algorithm called Dynamic Alignment (DTW for Dynamic Time Warping) is used.

Advantageously, if an annotation and a segmentation of the two recordings are available (for example a phoneme division) and if this information is concordant between the two recordings, then the alignment path can be constrained so as to respect the segmentation marks.

où N(z;µ;Σ) est la densité de probabilité de la loi normale de moyenne µ et de matrice de covariance Σ, et où les α_i sont les coefficients du mélange (α _i est la probabilité a priori que z soit généré par la i^ième gaussienne).In the proposed embodiment, a joint classification of the acoustic vectors of the two aligned recordings is performed. Let x _{1: N} = [x ₁ , x ₂ , ···, x _N ] and y _{1: N} = [y ₁ , y _{2 ···} , y _N ] be the sequences of aligned acoustic vectors. Let x and y be the random variables relating to the acoustic vectors of each of the records and z = (x, y) the associated pair. In the acoustic classification described here, the random variable z is modeled by a mixture of Gaussian laws (in English GMM for "Gaussian Mixture Model") of order Q. Its probability density is then written in the following form: $$p\left(z\right)={\displaystyle \underset{i=1}{\overset{Q}{\Sigma }}}{\alpha }_i\mathrm{NOT}\left(z;{\mu }_i;{\mathrm{\Sigma }}_i\right),{\displaystyle \underset{i=1}{\overset{Q}{\Sigma }}}{\alpha }_i=1,{\alpha }_i\ge 0$$

where N (z; μ; Σ) is the probability density of the normal law of mean μ and of covariance matrix Σ, and where α _i are the coefficients of the mixture (α _i is the probability a priori that z is generated by the ^ith Gaussian).

The estimation of the parameters of the model is carried out by applying a classical iterative procedure, namely the EM (Expectation-Maximization) algorithm ( AP Dempster, NM Laird, DR Rubin, EM algorithm, Journal of the Royal Statistical Society B, vol. 39, pp. 1-38, 1977 ). The determination of the initial parameters of the GMM model is obtained using a standard vector quantization technique.

où $$h_i\left(x\right)=\frac{{\alpha }_{i}N\left(x,,{\mu }_i^x,,{\mathrm{\Sigma }}_i^{\mathit{xx}}\right)}{{\displaystyle \sum _{i=1}^Q}{\alpha }_{i}N\left(x;{\mu }_i^x;{\mathrm{\Sigma }}_i^{\mathit{xx}}\right)}$$

est la probabilité a posteriori que x soit généré par la gaussienne d'indice i, avec $${\mathrm{\Sigma }}_i=\left[\begin{array}{cc}{\mathrm{\Sigma }}_i^{\mathit{xx}}& {\mathrm{\Sigma }}_i^{\mathit{xy}}\\ {\mathrm{\Sigma }}_i^{\mathit{yx}}& {\mathrm{\Sigma }}_i^{\mathit{yy}}\end{array}\right]{\mathrm{et\; \mu }}_i=\left[\begin{array}{c}{\mu }_i^x\\ {\mu }_i^y\end{array}\right]\mathrm{.}$$

Once the GMM model has been learned, it can be used to regression determine a conversion function between the speaker and the reference speaker. In the case of a conversion of a speaker x to a speaker y, it is written in the form: $$\widehat{\mathrm{there}}=F\left(x\right)=E\left[\mathrm{there}/x\right]={\displaystyle \underset{i=1}{\overset{Q}{\Sigma }}}{h}_i\left(x\right)\left[{\mu }_i^{\mathrm{there}}+{\mathrm{\Sigma }}_i^{\mathit{yx}}({\mathrm{\Sigma }}_i^{\mathit{xx}}{)}^{-1}\left(x-{\mu }_i^x\right)\right],$$

or $$h_i\left(x\right)=\frac{{\alpha }_i\mathrm{NOT}\left(x,,{\mu }_i^x,,{\mathrm{\Sigma }}_i^{\mathit{xx}}\right)}{{\displaystyle \underset{i=1}{\overset{Q}{\Sigma }}}{\alpha }_i\mathrm{NOT}\left(x;{\mu }_i^x;{\mathrm{\Sigma }}_i^{\mathit{xx}}\right)}$$

is the posterior probability that x is generated by the Gaussian index i , with $${\mathrm{\Sigma }}_i=\left[\begin{array}{cc}{\mathrm{\Sigma }}_i^{\mathit{xx}}& {\mathrm{\Sigma }}_i^{\mathit{xy}}\\ {\mathrm{\Sigma }}_i^{\mathit{yx}}& {\mathrm{\Sigma }}_i^{\mathit{yy}}\end{array}\right]{\mathrm{and\; \mu }}_i=\left[\begin{array}{c}{\mu }_i^x\\ {\mu }_i^{\mathrm{there}}\end{array}\right]\mathrm{.}$$

The figure 2 illustrates a method for estimating a voice conversion function between a source speaker and a target speaker whose voices are respectively defined from voice messages recorded by each of the speakers, these recordings being non-parallel.

In a first step, synthetic reference records are generated from said voice messages recorded according to a procedure similar to that just described with regard to the figure 1 .

Two conversion steps are then necessary to convert the voice of the source speaker to that of the target speaker. As a first step, you need to convert the source speaker parameters into those of the reference speaker, then transform them to reproduce the desired target speaker. Thus, a function allowing the desired source-target conversion can be estimated by composing two transformation functions given by (4): $$F_{\mathit{source}\mathit{\to }\mathit{target}}\left(x\right)=F_{\mathit{reference}\mathit{\to }\mathit{target}}\circ {\mathit{F}}_{\mathit{source}\mathit{\to }\mathit{reference}}\left(x\right)\mathrm{.}$$

A voice conversion system incorporating the described estimation method is represented on the figure 3 . In the proposed embodiment, the analysis step still relies on HNM modeling, but this time is conducted in a pitch-synchronous manner, as this allows for better pitch and spectral envelope changes (see FIG. article by Y. Stylianou cited above). The extracted spectral parameters are then transformed using a conversion module 80 performing the conversion determined by the relation (6).

These modified parameters as well as the residual information necessary for the sound generation (fundamental frequency, phase of the harmonics, gain of the noisy part, maximum frequency of voicing) are transmitted to a synthesis module by HNM. The harmonic component of the signal defined by equation (2) and present for the voiced signal frames is generated by summation of sinusoids previously tabulated whose amplitudes are calculated from the converted spectral parameters. The stochastic portion is determined by inverse Fourier Transform (IFFT) on the spectrum calculated from the spectral parameters.

Alternatively, the HNM model can be replaced by other models known to those skilled in the art, such as linear prediction coding (LPC) models, sinusoidal or MBE ("Multi-Band Excited") models. "). The GMM conversion method can be replaced by conventional vector quantization (VQ) or fuzzy vector quantization (Fuzzy VQ) techniques.

The description that has just been given of the estimation method according to the invention only referred to the only transformation of parameters relating to the stamp. But it is understood that the same method can also be applied to the transformation of other types of parameters such as the pitch frequency or parameters related to voice quality.

According to a preferred implementation of the invention, the steps of the method are determined by the instructions of a program for estimating a voice conversion function incorporated in a server, and the method according to the invention is implemented when this program is loaded into a computer whose operation is then controlled by the execution of the program.

Accordingly, the invention also applies to a computer program, including a computer program on or in an information carrier, adapted to implement the invention. This program can use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code such as in a partially compiled form, or in any other form desirable to implement the method according to the invention.

The information carrier may be any entity or device capable of storing the program. For example, the medium may comprise storage means, such as a ROM memory, for example a CD ROM or a microelectronic circuit ROM memory, or a means magnetic recording, for example a floppy disk or a hard disk.

On the other hand, the information medium may be a transmissible medium such as an electrical or optical signal, which may be conveyed via an electrical or optical cable, by radio or by other means. The program according to the invention can be downloaded in particular on an Internet type network.

Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the method in question.

Claims (8)

1. Method of estimating a voice conversion function for converting between, on the one hand, the voice of a speaker defined on the basis of a voice message recorded by said speaker, and, on the other hand, the voice of a reference speaker defined by a voice synthesis database, characterized in that said method comprises the steps consisting in:

   - generating, on the basis of said voice message recorded by the speaker and of said voice synthesis database, a synthetic recording of said voice message,

   - estimating said voice conversion function by a training operation performed on said recorded voice message and said synthetic recording.
2. Method of estimating a voice conversion function for converting between, on the one hand, the voice of a source speaker defined on the basis of a first voice message recorded by said source speaker, and, on the other hand, the voice of a target speaker defined on the basis of a second voice message recorded by said target speaker, characterized in that said method comprises the steps consisting in:

   - generating, on the basis of said first voice message recorded by the source speaker and of a voice synthesis database, a synthetic recording of said first voice message,

   - estimating a first voice conversion function for converting between the voice of the source speaker and the voice of a reference speaker defined by said voice synthesis database, by a training operation performed on said first voice message recorded by the source speaker and said synthetic recording of the first voice message,

   - generating, on the basis of said second voice message recorded by the target speaker and of said voice synthesis database, a synthetic recording of said second voice message,

   - estimating a second voice conversion function for converting between the voice of said reference speaker and the voice of the target speaker, by a training operation performed on said synthetic recording of the second voice message and said second voice message recorded by the target speaker,

   - estimating said voice conversion function by composition of said first and said second voice conversion functions.
3. Method according to one of Claims 1 or 2,
   characterized in that said voice synthesis database is a database of a concatenation-based speech synthesis system.
4. Method according to one of Claims 1 or 2,
   characterized in that said voice synthesis database is a database of a corpus-based speech synthesis system.
5. Application of the method according to Claim 1 to the conversion of voice messages recorded by a source speaker into corresponding messages reproduced by said reference speaker.
6. Application of the method according to Claim 1 to the conversion of synthetic messages recorded by a reference speaker into corresponding messages reproduced by a target speaker.
7. Voice conversion system, characterized in that it comprises a voice conversion module comprising means for implementing the method according to any one of Claims 1 to 4.
8. Computer program on an information medium, said program comprising program instructions suitable for implementing a method according to any one of Claims 1 to 4, when said program is loaded and executed in a computer system.

Images