JP6201205B2 - Speech synthesis apparatus, speech synthesis method, and speech synthesis program - Google Patents

Description

  The present invention relates to a speech synthesizer, a speech synthesis method, and a speech synthesis program that synthesize a speech waveform based on input time-series sound source control information and spectrum characteristic information.

  Speech synthesis technology is a general term for a series of technologies for synthesizing speech waveforms from text. First, we explain the process of synthesizing speech waveforms from the spectrum information and sound source information of the speech that you want to synthesize. To do. In this process, the spectrum information and sound source information of the voice to be synthesized are obtained in advance from the corresponding natural voice or the like.

  A typical speech synthesis waveform synthesis method is a speech synthesis method based on a source filter model. This method is a method of first synthesizing a speech waveform having a desired characteristic by generating an appropriate sound source (source) waveform and passing it through a filter having an appropriate characteristic. For example, if it is considered that the sound source corresponds to the glottal volume flow accompanying vocal cord vibration and the filter corresponds to the vocal tract transfer characteristics, it can be said that the model corresponds to the human voice generation process.

  However, what can be observed from the speech waveform is the physical quantity for the final speech waveform, such as the spectral characteristics of the speech waveform and the fundamental frequency observed in the periodic speech waveform. Matching is difficult. Therefore, in practice, a speech waveform is often synthesized by directly giving a spectral characteristic of speech as a synthesis target by a filter to a spectrally white sound source waveform such as an impulse train or white noise.

  When the speech waveform has periodicity, the spectrum information to be observed includes a fundamental frequency component and its harmonic component derived from the periodicity. Normally, this periodicity is expressed on the sound source side by an impulse train or the like.

  Hereinafter, the spectrum information refers to smoothed spectrum information excluding the influence of the fundamental frequency and its harmonic components. The smoothing method includes a method of connecting only the peak points of the harmonic components on the frequency axis. In addition, although the sound waveform can be regarded as almost steady in a short time, it is time-varying over a long time. Therefore, in general, a characteristic at every certain interval (for example, about 1 to 20 milliseconds) is considered. The stationarity is assumed at each time. Here, the spectrum information of each sample is expressed by, for example, a multi-order mel cepstrum coefficient or a linear prediction coefficient.

  In general, voice with vocal cord vibration is called voiced sound, and voice without voice is called unvoiced sound. In voiced sound, waveform periodicity is usually observed. In speech waveform synthesis based on a source filter, a method is often used in which only an impulse train is used as a voiced sound source and only white noise is used as an unvoiced sound source. Although this method often has no problem in terms of linguistic intelligibility of synthesized speech, the actual voiced sound also includes a noisy component, resulting in a problem that its naturalness is lowered.

  Therefore, a method has been developed that improves the naturalness of synthesized speech by simultaneously generating an impulse train and white noise and using the combined waveform as a sound source waveform. However, usually, the optimum impulse to noise power ratio is not constant in each frequency band, and it differs depending on the type of speech to be synthesized. Therefore, it is necessary to change the amplitude characteristics of the impulse and white noise for each frequency band (subband) using a filter bank or the like.

  At this time, considering the correspondence with the conventional source / filter model, a method of controlling so that the result of adding the sound sources to white is often used. Hereinafter, such a sound source is referred to as a multiband mixed excitation source. Although it is considered that a certain degree of naturalness can be obtained without changing the mixing ratio for each subband, it is possible to synthesize speech with higher naturalness by changing the time as in the spectral information.

  Therefore, for speech synthesis, speech spectral information, voiced / unvoiced information, fundamental frequency information about voiced, and multiband mixed excitation sources are used for each interval on the time axis, and the characteristics are dynamically changed. Information on the mixing ratio of each subband in the case of changing is required. In the form of speech synthesis described below, for convenience of explanation, it is assumed that the power of the sound source is always constant and the power of the synthesized speech is controlled by being included in the spectrum characteristics.

Sei Imai, Kazuo Sumita, Chieko Furuichi, "Mel Log Spectrum Approximation (MLSA) Filter for Speech Synthesis", IEICE Transactions (A), J66-A, 2, Feb.1983, pp.122-129

  In the prior art as described above, a filter having a relatively large calculation amount such as an MLSA (Mel logarithmic spectrum approximation) filter is used as a filter of the source filter model (see Non-Patent Document 1). The MLSA filter employs a technique of constructing a circuit that approximately realizes a target characteristic by directly approximating an exponential function in a z-transform region by a rational approximation on the z-transform region by Padé approximation. Although there is an advantage that the mel cepstrum coefficient can be used as it is as a filter coefficient, the number of product-sum operations per sample of the waveform is approximately the product of the order of the filter and the order of the Padé approximation, and the amount of calculation is relatively large.

  For example, in terms of synthesized speech quality, it is necessary to use a 30-40th order mel cepstrum at the time of 16 kHz sampling. In that case, in order to approximate the exponential function with the required accuracy, a fourth order or fifth order Padé approximation is required. That is, a product-sum operation is required about 150 to 200 times per sample.

  Further, when performing multi-band mixing excitation, it is necessary to filter the impulse train and the white noise so that the specified mixing ratio is obtained, so that the amount of calculation further increases for each filtering process. For this reason, it is difficult to perform speech synthesis processing using a higher-order filter or mixed excitation in an environment where the calculation processing performance of a mobile terminal or the like is limited.

  In order to solve this problem, a method of performing subband encoding including sample rate reduction on a sound source waveform of an impulse train or a white noise train based on a quasi-orthogonal mirror image filter bank or the like can be considered. In this method, after adjusting the amplitude of each band element in the subband coding region, decoding processing is performed to synthesize a speech waveform.

  In the above method, the processing amount per sample can be made logarithmic order with respect to the number of subbands by using the filter bang processing using high-speed cosine transform or the like. In the conventional filter-based method, the processing amount per sample is in a linear order with respect to the filter order, so that the processing amount can be reduced as compared with the conventional method depending on setting conditions.

  Further, when all signal processing is linear processing, a method of combining white noise or impulse train pre-encoded in the subband code region is conceivable. When this method is used, the processing amount can be further reduced because the subband encoding processing at the time of speech synthesis becomes unnecessary.

  On the other hand, it is conceivable to perform subband encoding including sample rate reduction on a white sound source waveform such as an impulse train or a white noise train based on a pseudo orthogonal mirror image filter bank or the like. In that case, the speech waveform can be synthesized by performing the decoding process after adjusting the amplitude of each band element in the subband coding region.

  In this method, the processing amount per sample can be made logarithmic order with respect to the number of subbands by using a filter bang process using a high-speed cosine transform or the like. Since the conventional filter-based method has a linear order with respect to the filter order, the processing amount can be reduced as compared with the conventional method depending on setting conditions. Further, there is shown a method of combining white noise or impulse train pre-encoded in the subband code region when all signal processing is linear processing. When this method is used, the processing amount can be further reduced because the subband encoding processing at the time of speech synthesis becomes unnecessary.

  However, in this method, the resolution in the frequency axis direction is determined by the number of subbands in subband coding in generating the speech spectrum feature. In order to synthesize speech in which an error from a desired spectral feature is suppressed, the number of subbands must be set large, but the processing amount increases as the number of subbands increases. Although this increase in the amount of processing can be suppressed by increasing the frame period, on the other hand, if the frame period is increased, the resolution in the time axis direction of the spectrum feature change is insufficient and the quality is impaired.

  Further, in the quasi-orthogonal mirror image filter bank, it is possible to cancel the alias associated with downsampling with the alias due to upsampling, and such a configuration is usually used in subband coding. However, if the amplitude coefficient of each subband is changed independently for the synthesis of the spectral characteristics, the process of canceling out aliases is lost, and noise due to the aliases occurs.

  In order to prevent such an alias from occurring, a method of limiting downsampling to half of normal subband coding can be considered. However, as a result, in addition to doubling the amount of processing, it is necessary to set the amplitude adjustment coefficient of two adjacent subbands equal to each other across the frequency that causes aliasing. It will be about half of the number.

  The present invention has been made in view of such circumstances, and provides a speech synthesis apparatus, a speech synthesis method, and a speech synthesis program capable of reducing the processing amount while increasing the frequency resolution and time resolution in speech synthesis. The purpose is to do.

  (1) In order to achieve the above object, a speech synthesizer according to the present invention is a speech synthesizer that synthesizes speech waveforms based on input time-series sound source control information and spectrum characteristic information. A component decomposition unit that decomposes the component into a plurality of frequency band components, a subband encoding unit that performs subband encoding on each of the decomposed components, and subband encoding that includes the subband encoded components. A weighted sum calculation unit that calculates a weighted sum for the decomposition component with respect to the waveform vector, and a subband encoded waveform vector for which the weighted sum has been calculated is decoded and combined into a single speech waveform. And a band decoding unit.

  As a result, even in a configuration in which the amplitude coefficient of each subband is changed independently for the synthesis of spectral characteristics, setting and processing for preventing noise caused by aliases are not required, and the processing amount can be reduced. In addition, the processing amount increases by the weighted sum calculation processing of the subband encoded waveform vector accompanying the component decomposition, but the subband encoded waveform vector includes many elements that become 0 when subband encoding is performed. By disassembling the sound source waveform and combining them on the subband code to compose the speech waveform, it is possible to omit much of the actual product-sum operation process in the weighted sum calculation process on the subband code. it can.

  Further, since the method for decomposing the sound source waveform can be arbitrarily determined separately from the subband coding, the frequency resolution of the synthesized speech waveform can be increased by further decomposing the sound source waveform. As a result, it is possible to avoid problems such as an increase in processing amount and a decrease in time resolution when the number of subbands in subband encoding is increased. In this manner, sufficient speech synthesis processing can be performed even in an environment where the calculation processing performance of a portable terminal or the like is limited.

  (2) Further, in the speech synthesizer according to the present invention, the component decomposition unit is configured by a set of bandpass filters, and the sound source waveform can be approximately reproduced by the sum of the decomposed components. Is characterized by component decomposition. In this way, by performing subband coding for each decomposed component and calculating a weighted sum of subband encoded waveform vectors, it is possible to reproduce speech corresponding to the spectrum information.

  (3) In the speech synthesizer of the present invention, the component decomposition unit uses two types of sound source waveforms of an impulse train and white noise as the sound source waveform, and the weighted sum calculation unit receives the input spectrum. It is characterized in that a weighted sum of subband encoded waveform vectors based on the two types of sound source waveforms is obtained by a mixing ratio between an impulse train based on characteristic information and white noise. Thereby, the naturalness of the synthesized speech can be improved by using the impulse train and white noise as the sound source waveform and synthesizing the speech with the optimum impulse to noise power ratio that differs for each type of speech.

  (4) Further, the speech synthesizer of the present invention is characterized in that the component decomposition unit decomposes the sound source waveform into components having the same width as a frequency band to be subjected to the subband encoding and decoding. Thereby, compared with the method without prior component decomposition, the same frequency and time resolution can be obtained with a processing amount of about 3/4.

  (5) The speech synthesis method of the present invention is a speech synthesis method for synthesizing speech waveforms based on input time-series sound source control information and spectrum characteristic information. A step of subdividing into components, a step of subband encoding each of the decomposed components, and a weighting of the decomposed components with respect to a subband encoded waveform vector composed of the subband encoded components A step of calculating a sum; and a step of decoding the subband encoded waveform vector for which the weighted sum has been calculated and synthesizing it into a single speech waveform. Thereby, the processing amount can be reduced while increasing the frequency resolution and time resolution in speech synthesis.

  (6) The speech synthesis program of the present invention is a speech synthesis program for synthesizing speech waveforms based on input time-series sound source control information and spectrum characteristic information, and the sound source waveforms are divided into a plurality of frequency bands. A process of decomposing into components, a process of subband encoding each of the decomposed components, and a weighting of the decomposed components with respect to a subband encoded waveform vector composed of the subband encoded components It is characterized by causing a computer to execute a process of calculating a sum and a process of decoding a subband encoded waveform vector for which the weighted sum has been calculated and synthesizing it into a single speech waveform. Thereby, the processing amount can be reduced while increasing the frequency resolution and time resolution in speech synthesis.

  According to the present invention, it is possible to reduce the processing amount while increasing the frequency resolution and time resolution in speech synthesis. As a result, it is possible to perform sufficient speech synthesis processing even in an environment where the calculation processing performance of a portable terminal or the like is limited.

It is a block diagram which shows the basic composition of the speech synthesizer which concerns on embodiment which becomes foundations. It is a block diagram which shows the specific structure of the speech synthesizer which concerns on embodiment which becomes fundamental. It is a block diagram which shows the actual circuit structure of a subband encoding part. It is a block diagram which shows the theoretical structure of a subband encoding part. It is a block diagram which shows the actual circuit structure of a subband decoding part. It is a block diagram which shows the theoretical structure of a subband decoding part. It is a graph which shows the amplitude characteristic with respect to a frequency about a band division | segmentation filter bank. It is a block diagram which shows the basic composition of the speech synthesizer which concerns on 1st Embodiment. It is a block diagram which shows the specific structure of the speech synthesizer which concerns on 1st Embodiment. It is a flowchart which shows an example of operation | movement of the speech synthesizer which concerns on 1st Embodiment. It is a flowchart which shows an example of operation | movement of the speech synthesizer which concerns on 1st Embodiment.

  Next, embodiments of the present invention will be described with reference to the drawings. In order to facilitate understanding of the description, the same reference numerals are given to the same components in the respective drawings, and duplicate descriptions are omitted.

[Basic embodiment]
(Configuration of speech synthesizer)
FIG. 1 is a block diagram showing a basic configuration of the speech synthesizer 100, and FIG. 2 is a block diagram showing a specific configuration of the speech synthesizer 100. The speech synthesizer 100 sub-band encodes and accumulates the sound source waveform by the sub-band encoding unit 110 and adjusts the amplitude for each sub-band according to the input information. Then, the subband decoding unit 140 synthesizes using the subband encoded waveform vector whose amplitude is adjusted, and synthesizes a speech waveform that approximately has a target spectral characteristic.

  The speech synthesizer 100 synthesizes a speech waveform based on the input time-series sound source control information and spectrum characteristic information. In the present embodiment, the sound source control information is a fundamental frequency. As shown in FIG. 1, speech synthesis apparatus 100 includes subband encoding section 110, subband encoded excitation generating section 120, subband power adjustment section 130, and subband decoding section 140.

  The subband encoding unit 110 divides the excitation waveform into a plurality of frequency bands, and generates a vector sequence by dividing the excitation waveform. The subband encoding unit 110 preferably generates a subband encoded waveform vector for thinning out a vector from a vector sequence within an equal time interval and storing it as a sound source.

Note that the subband encoding unit 110 includes, for example, analysis filter banks E ₀ (z) to E _M-1 (z) and a downsampler D ↓. The analysis filter banks E ₀ (z) to E _M-1 (z) are configured by filter banks that are equally divided into M frequency bands. The downsampler D ↓ subtracts the vector of (D-1) from the vector sequence of D samples (provided that D ≦ M) at equal time intervals with respect to the M-dimensional vector sequence after subband coding. The process of leaving only the vector is performed. Such thinning-out processing can reduce the pre-accumulation size and the processing amount of the synthesis filter bank.

  The subband encoded excitation generation unit 120 divides the subband encoded waveform vector corresponding to the input excitation control information based on the subband encoded waveform vector accumulated by dividing the excitation waveform into a plurality of frequency bands. Generate. At that time, a plurality of subband encoded waveform vectors among the accumulated subband encoded waveform vectors are combined to generate a subband encoded waveform vector corresponding to the input excitation control information.

  The subband coded excitation generation unit 120 further includes an accumulation unit 121 and a selection unit 122. The accumulating unit 121 stores a vector generated as a result of sub-band coding of a relatively short time excitation waveform (sound source waveform segment) generated in advance. This vector is a vector having the same number of dimensions as the number of subband divisions by subband encoding, and this is called a subband encoded waveform vector.

  The selection unit 122 selects a pre-stored subband encoded waveform vector based on the input fundamental frequency information. In this way, the subband encoded excitation generator 120 uses the selected subband encoded waveform vector or configures a plurality of types of subband encoded waveform vectors as subband encoded waveform vectors. Then, a subband-encoded excitation waveform vector is output. Note that the above accumulation is performed in advance as pre-processing, and the subsequent processing is performed when there is input information.

The subband power adjustment unit 130 performs amplitude adjustment for each subband according to the input spectral characteristic information with respect to the generated subband encoded waveform vector. The subband power adjustment unit 130 is provided with a multiplier circuit for controlling the power of each subband. The subband power adjustment unit 130 adjusts the coefficients A _{0 to} A _M−1 for each subband based on the input spectral feature information. As a result, the spectral characteristics of the target speech are reproduced. The spectrum information to be input may be configured directly with the power information of each subband. For example, the mel cepstrum coefficient is input, the power information of each subband is calculated internally, and the result is calculated. It may be used.

The subband decoding unit 140 synthesizes the subband encoded waveform vector whose amplitude has been adjusted into a single speech waveform. That is, the sub-band encoded waveform is synthesized and a final synthesized speech waveform is generated. The subband decoding unit 140 includes, for example, an upsampler D ↑ and synthesis filter banks R ₀ (z) to R _M-1 (z). Up-sampler D ↑ inserts a zero value sample between the band-divided signals with respect to the subband encoded waveform vector whose amplitude has been adjusted, and performs D-times up-sampling. The synthesis filter banks R ₀ (z) to R _M-1 (z) synthesize the subband encoded waveform vector divided into M frequency bands into a single speech waveform.

(Configuration of filter bank)
Multiplying the coefficients of a filter that constitutes a filter bank by the discrete Fourier transform (DFT), discrete cosine transform (DCT), or inverse transform coefficient series, shifts the characteristics of the underlying filter on the frequency axis. A filter characteristic of the shape is obtained. By configuring a filter bank with such a filter, a high-speed technique such as FFT (Fast Fourier Transform) can be used for calculations necessary for the filter bank processing. Thereby, it is possible to speed up the processing of subband encoding / subband synthesis.

  3A and 3B are block diagrams showing an actual circuit configuration and a theoretical configuration of subband encoding section 110, respectively. 4A and 4B are block diagrams showing an actual circuit configuration and a theoretical configuration of subband decoding section 140, respectively. Both examples show configuration examples using discrete cosine transform.

For both the subband encoding unit 110 and the subband decoding unit 140, the actual circuit configuration includes a delay element z ⁻¹ and a discrete cosine transform element DCT or an inverse discrete cosine transform element IDCT. . On the other hand, in a configuration that is theoretically equivalent to the subband encoding unit 110 and the subband decoding unit 140, a form that does not include the above elements is equivalent. In the configuration that is theoretically equivalent to the subband encoding unit 110, since the downsampling is performed after the filter processing, the processing sample rate is large and the processing amount is large. However, in the actual configuration, the downsampling is performed first. Therefore, the processing amount is reduced. The same applies to the subband decoding unit 140.

  FIG. 5 is a graph showing amplitude characteristics with respect to frequency for the equal band division filter bank. When only DFT or DCT is used, a bandpass filter having an impulse response of a rectangular window function can be generally regarded as a filter bank composed of bandpass filters that are shifted on the frequency axis. Hereinafter, the base filter before the shift is referred to as a basic filter. Note that the basic filter is a band-pass filter having a frequency characteristic with a larger attenuation in the cut-off region, which is generally considered to be more preferable (if the frequency 0 is the center, it becomes a low-pass filter). It is also possible. However, it is necessary to design a filter so that the original speech waveform can be restored when subband synthesis is performed on the subband encoding processing result. That condition is called the perfect reconstruction condition. Further, depending on the filter configuration, exact restoration may not be possible. In such a case, the filter is designed so as to be restored approximately. When a DFT of length M is used, a filter bank is configured by M filters obtained by shifting the base filter by 2πk / M (k ≦ 0 <M) by a normalized angular frequency, and the DCT is When used, depending on the definition, in the case of the DCT transformation defined in the following example, a characteristic shifted by π (k + 1/2) / M at the normalized angular frequency, and π (−k + 1/2) A filter bank is composed of M filters having the frequency characteristics of the sum of the features shifted by / M.

In the following example, a pair of DCT transform and inverse DCT transform is used. The input / output of the DFT is defined as a complex number, whereas the input / output of the DCT is a real number, and the processing can be performed more easily. For example, the analysis filter bank can also be configured by using the expression (2) (0 ≦ k <M) as a filter using the M-th order DCT coefficient of the expression (1) as a coefficient.

  Due to the characteristics of the DCT coefficient, this is an M-divided equal-band division filter bank. Further, since this filter bank can be configured to satisfy the complete reconstruction condition, the input waveform can be restored from the band-divided waveform.

  In the above configuration, the number of subbands is a number that can simulate the spectrum described by the spectrum feature information with a predetermined accuracy. For example, when the spectral information of one sample is a k-th order mel cepstrum (the number of parameters including the zeroth order coefficient is k + 1), and k here is the number of dimensions necessary to express the spectral feature, In order to generally simulate such a spectrum, at least (k + 1) subbands are required.

  In addition, the subband power adjustment unit 130 operates to adjust the gain of each subband with respect to a white sound source and generate an audio waveform corresponding to the input spectral feature information. When performing multiband mixed excitation, normalization is performed so that the impulse sound source and the white noise source have equal power in advance, and control is performed so that the sum of the power weights of each subband is 1. A sound source can be obtained.

  As described above, the power value of each subband may be obtained by converting from the mel cepstrum coefficient or the like instead of directly inputting the power value of each subband as spectrum information. By controlling the spectral intensity at the center of the subband as a power value of the subband, the target spectral feature can be obtained approximately. The center of the subband is 0, 2π / M, 4π / M,... On the normalized angular frequency axis when configuring a filter bank based on DFT.

  On the other hand, when a filter bank based on the above-described DCT is configured, ± π / 2M, ± 3π / 2M,... However, in the case of using a basic filter whose input is a real number sequence and whose impulse response is symmetric, the frequency characteristics are all symmetric with respect to the frequency 0. Therefore, for example, only the range from 0 to π in the normalized angular frequency may be considered. . Since the spectral characteristic for each subband can be obtained from the filter bank coefficient, an error from the target spectral feature may be evaluated at a finer interval than the number of subbands on the frequency axis. For example, more accurate control can be realized by obtaining a subband power coefficient set that minimizes the mean square error by iterative approximate estimation or the like. The above example is merely an example, and the pair of DCT transform / inverse DCT transform can be replaced with another reversible transform pair.

(Sound source control method)
Next, a sound source control method will be described. First, it is assumed that the linearity of processing is guaranteed before and after subband coding and subband synthesis. The filter bank based on the DFT or DCT described above satisfies this condition because its processing is configured only by a combination of linear operations.

  At this time, for the impulse train, for example, when 32 bands are divided from the past 32 samples and the analysis filter / synthesis filter of each band can be expressed by the FIR filter, the result of subband coding is as follows: Can be obtained. That is, when an impulse sound source waveform in which an impulse is raised in the first and 20th samples of the input frame is divided into bands, the sound source waveform in which an impulse is raised only in the first sample is subband encoded. It can be obtained by adding each element of the result and the result of subband coding of the sound source waveform in which the impulse stands only in the 20th sample.

  That is, in the case of M band division, it is only necessary to store in advance M types of sound source waveform changes for impulse sound sources. Actually, since the fundamental frequency used in speech synthesis is relatively low, there may be few cases where two or more impulses are included in the M samples of the sound source waveform. In this case, the processing amount of the addition process can be almost ignored.

  Note that the amount of processing for generating waveforms to be created and stored in advance is not a problem because it is not processing at the time of speech synthesis. Therefore, for example, it is also easy to control the position of the impulse with a time accuracy that is virtually equal to or higher than the sampling period, such as an impulse standing at the 1.5th sample. For such a sound source waveform, for example, a corresponding waveform using twice the sampling frequency is first created, and an antialiasing filter that is a high-frequency cutoff filter is applied to remove components higher than the Nyquist frequency at the original sampling frequency. And then thinning out the sample by 2: 1 downsampling.

  Such a method is particularly effective in the case where the sampling frequency is low and the error of the fundamental frequency of the synthesized speech becomes large if the impulse position is rounded to the sampling point. Conversely, when the sampling rate is high, a method of reducing the position accuracy of the impulses and reducing the number of subband encoded waveforms to be stored is conceivable.

  On the other hand, for white noise sources, white noise may be synthesized by adding impulses, but an appropriate number and frame length of white noise sequences are pre-divided into bands and stored randomly for each frame. By selecting, it may be configured approximately. In this case, although it is necessary to accumulate the converted waveform, it is not necessary to calculate the weighted sum, and the amount of processing can be reduced. In order to generate white noise from only a relatively small number of accumulations, a method of constructing a band-divided sound source waveform by adding a plurality of accumulated band-divided sound source waveforms is also conceivable.

(Configuration using non-maximum decimation filter bank)
The decimation rate M in the filter bank is a value from 1 (not decimation at all) to M, and at least when power adjustment is not performed in each subband before recombination, the subband synthesis result is before subband coding. It is theoretically known that it is possible to construct a filter bank that matches the input signal. For example, when a filter bank is constituted only by DFT or DCT and thinning-out rate L is thinned out, it is clear that if the calculation error is ignored, the input waveform can be completely restored by their inverse transformation.

  However, especially in the case of D = M (the thinning rate is the maximum and is called the maximum thinning), using DCT, in the normalized angular frequency (however, only the range from 0 to π is considered here because of its symmetry) ), 0 to π / M, π / M to 2π / M,..., (M−2) π / M to (M−1) π / M are components of a pass band and a cutoff band. Regardless of whether or not, all the subbands are folded and stored. At the time of synthesis, the aliasing noise components of the subbands cancel each other, thereby restoring the input waveform.

  When the filter of each subband is viewed as a band pass, the width of the pass band is also π / M, but in reality, an ideal filter that always has a gain of 1 in the pass band and always 0 in the cut-off band is It cannot be theoretically realized with a finite filter. Actually, there is a certain amount of passage even in the cut-off area, and in the case of maximum thinning, a large aliasing noise is included in each subband. For this reason, if the power of each subband is changed independently for each subband, the structure of aliasing noise canceling out between the subbands is destroyed, and the aliasing noise becomes a problem.

  On the other hand, if a value smaller than M is set in D, the folding width of the sample by thinning out becomes wider than the band width of the bandpass filter in the filter bank, so that the folding noise of each subband is reduced. Even when the power is adjusted independently every time, the influence of aliasing noise can be reduced. Such a setting is called non-maximum decimation. In general, the smaller the thinning rate D, the smaller the influence of aliasing noise, but the amount of information becomes redundant and the amount of data to be stored and processed increases. For this reason, it is preferable to set D as large as possible within a range necessary for suppressing the influence of aliasing noise.

  The aforementioned non-maximum decimation is equivalent to performing an overlap analysis of the frame shift D when viewed from the waveform series before band division and after band synthesis. In addition, for each processing of D samples in the time domain, processing of one sample in the subband division region by subband coding is performed. Here, for simplicity, it is assumed that D is a divisor of M. It is assumed that a filter bank that satisfies the complete reconstruction condition is used.

  First, the impulse sound source is controlled by the same method as the sound source control method described above, even if it is non-maximum decimation. However, for example, in a frame of length M, if the Nth sample (where M> N ≧ D) stands from the beginning, the NDth sample from the beginning in the next frame due to the frame shift of D samples. Impulse stands on the sample. At this time, the impulse sound source outputs a corresponding pre-accumulated subband encoded waveform vector at each timing.

  On the other hand, white noise can be generated by repeating the same waveform at an M × N sample period, for example, as the simplest method. In that case, a method of pre-accumulating M × N × (M / D) length M waveforms corresponding to the frame shift and sequentially outputting them in accordance with the frame shift is conceivable. Here, N may be any value as long as the noise period does not cause any problem in hearing. For example, the noise period M × N may be longer than the period corresponding to the lower limit of the audible frequency (for example, 20 Hz).

  Alternatively, there is a method in which several white noise waveform segments having a length M are prepared in advance and are connected at random. Here, for one white noise waveform segment of length M, all sample values are treated as 0 outside the segment range on the time axis. The appearance pattern in one frame in the time domain of the white noise waveform segment alone is determined by the difference in the waveform start point in the frame, and −M + D, −M + 2D,..., −D, 0, D There are a total of (2 (M / D) -1) patterns of M-D.

  The noise waveform for one frame can be expressed by one type (when the starting point is 0) or a combination of two types of noise waveforms of length M. Therefore, when the number of white noise waveform segments of the sample of length M to be created in advance is N, one or two subband encoded waveforms from a total of N (2 (M / D) -1) prestores. Can be obtained, and a white noise source can be realized by adding the subband division regions in the subband coding.

  In addition, by reducing the length of the white noise waveform segment to M / 2, M / 4,..., The number of appearance patterns in one frame is reduced, and the addition processing necessary for the sound source is reduced. It will increase. Conversely, the length of the white noise waveform segment can be increased. In this case, since the number of appearance patterns increases, the number of necessary accumulations increases, but the number of cases where a summing process is required with a sound source is reduced.

[First Embodiment]
(Configuration of equipment for component decomposition)
In the above embodiment, amplitude adjustment is performed for each band that is frequency-divided by subband encoding. However, in the embodiment of the present invention, component decomposition is performed before subband encoding, and separately from encoding. The amplitude is adjusted for each decomposed component.

  FIG. 6 is a block diagram showing a basic configuration of a speech synthesizer 200 that performs component decomposition and amplitude adjustment separately from subband coding, and FIG. 7 is a block diagram showing a specific configuration of speech synthesizer 200. . The basic configuration of the speech synthesizer 200 is the same as that of the speech synthesizer 100. The sound source waveform is subband encoded by the subband encoder 210 and stored, and after adjusting the amplitude according to the input information, the subband code is converted. Decrypt. As shown in FIG. 6, the speech synthesizer 200 includes a component decomposition unit 205, a subband encoding unit 210, a weighted sum calculation unit 220, and a subband decoding unit 140.

  The component decomposition unit 205 includes an impulse side decomposition unit 206a and a white noise side decomposition unit 206b, and uses two types of sound source waveforms of an impulse train and white noise as a sound source waveform, and decomposes the sound source waveform into components of a plurality of frequency bands. To do. The component decomposing unit 205 is composed of a set of band-pass filters, and decomposes the sound source waveform into components so that the sound source waveform can be approximately reproduced by the sum of the decomposed components. In this way, the speech corresponding to the spectrum information can be reproduced by performing subband coding for each decomposed component and performing a weighted sum.

  The component decomposing unit 205 includes M and N filters Fp and Fa, respectively, and divides the impulse train and white noise source waveform into M and N source element waveforms, respectively. As for the division, an appropriate sound source waveform may be divided so that the sum of the divided waveforms reproduces the original sound source waveform or approximate features are reproduced. As a filter that realizes such division, an N-th band filter, which is an expansion of a 1/2 half-band filter to a 1 / N band limit, is used as a basic filter, and its coefficient is cosine-modulated on the frequency axis. It is possible to use a filter bank constructed by shifting with. Note that the band dividing method in the component decomposing unit 205 can be determined independently of the band dividing process in the subsequent subband encoding process.

  Further, it is preferable that the component decomposing unit 205 takes a large cutoff amount in the cutoff region of the band limiting filter so that the amplitude intensity in the cutoff region is virtually regarded as zero. Thereby, in the vector of the subband encoding result, the value of the subband element in the frequency band that completely overlaps the cutoff band of the band limiting filter can be set to zero. Then, a subband encoded waveform vector including many 0 elements can be used for speech synthesis processing. As a result, in the weighted sum calculation processing in the weighted sum calculation unit 220, the actual amount of processing can be reduced by omitting the zero-element product-sum operation processing.

  The subband encoding unit 210 performs subband encoding on each of the decomposed components by D division. That is, the divided sound source element waveform is converted into a D-dimensional vector. The subband encoding unit 210 includes an impulse side encoding unit 211a and a white noise side encoding unit 211b. The impulse side encoding unit 211a performs subband encoding on the impulse sound source, and the white noise side encoding unit 211b performs subband encoding on the white noise source.

  The weighted sum calculation unit 220 multiplies the decomposed component with respect to the subband encoded waveform vector composed of the subband encoded components, and obtains a weighted sum. Then, a weighted sum of the subband encoded waveform vectors based on the two types of sound source waveforms is obtained based on the mixing ratio of the impulse train and the white noise based on the input spectral characteristic information. Thereby, the naturalness of the synthesized speech can be improved by using the impulse train and white noise as the sound source waveform and synthesizing the speech with the optimum impulse to noise power ratio that differs for each type of speech.

  The weighted sum calculation unit 220 includes an impulse side accumulation unit 221a, an impulse side selection unit 222a, an impulse side weighting multiplication unit 223a, a white noise side accumulation unit 221b, a white noise side selection unit 222b, a white noise side weighting multiplication unit 223b, and an addition. Part 224 is provided.

  Each of the impulse side accumulation unit 221a and the white noise side accumulation unit 221b accumulates the sound source element waveform converted into a D-dimensional vector. The impulse side accumulation unit 221a accumulates subband encoded waveform vectors based on the impulse sound source. The white noise side accumulation unit 221b accumulates the subband encoded waveform vector based on the white noise source. The operations up to accumulation are performed before speech synthesis.

  The impulse side selection unit 222a and the white noise side selection unit 222b select a subband encoded waveform vector necessary for constructing a synthesized speech waveform based on sound source information and spectrum feature information during speech synthesis. The impulse side selection unit 222a selects a subband encoded waveform vector based on the impulse sound source accumulated in advance based on the inputted information on the fundamental frequency. The white noise side selection unit 222b selects a subband encoded waveform vector based on a prestored white noise source based on, for example, the “sound source control method” described above.

The impulse side weighting multiplication unit 223a and the white noise side weighting multiplication unit 223b calculate a weighted sum of the selected vectors to generate a subband encoded waveform vector constituting the synthesized speech waveform. The impulse-side weighting multiplication unit 223a multiplies each element of the selected subband encoded waveform vector by a weighting coefficient _{Ap0 to} Ap _(M-1) . The white noise side weighting multiplication unit 223b multiplies each element of the selected subband encoded waveform vector by a weighting coefficient A _{a0 to} A _{a (N−1)} . Each coefficient is determined so that A _px + A _ax = 1.

  The weighted sum calculation unit 220 can use a method of combining an element obtained by band-dividing an impulse train and an element obtained by band-dividing white noise in sound source waveform element selection. As the weighting factor, first, the spectral feature of only the impulse train component and the spectral feature of only the noise component are obtained from the mixing ratio of the impulse train and noise included as the sound source information. Then, it is only necessary to obtain the weighting coefficient so that the target spectrum feature and the amplitude intensity of the spectrum feature of the sound source element waveform match at each center frequency of the band-divided element.

  The impulse side accumulation unit 221a, the impulse side selection unit 222a, and the impulse side weighting multiplication unit 223a constitute an impulse side weighted sum calculation unit 220a. The white noise side accumulation unit 221b, the white noise side selection unit 222b, and the white noise side weighting multiplication unit 223b constitute a white noise side weighted sum calculation unit 220b.

  Adder 224 adds the subband encoded waveform vectors that have been weighted and multiplied on the impulse side and white noise side, respectively. Thus, a plurality of types of subband encoded waveform vectors are generated as one subband encoded waveform vector based on the excitation information. When the mixed excitation source is used as a sound source, the mixing ratio adjustment between the impulse train and the noise source is simultaneously performed based on the sound source information. As described above, the speech synthesizer 200 calculates the result of subband coding according to the type of the sound source waveform at the time of speech synthesis, and combines the pre-stored band subband encoded waveform vector with the sub-band that becomes the sound source. A band encoded waveform vector is generated.

  The subband decoding unit 140 decodes the generated subband encoded waveform vector with a configuration that cancels aliases associated with subband encoding, and synthesizes it into a single speech waveform. Then, the generated subband encoded waveform vector is converted into a final synthesized speech waveform.

  As a result, even in a configuration in which the amplitude coefficient of each subband is changed independently for the synthesis of spectral characteristics, setting and processing for preventing noise caused by aliases are not required, and the processing amount can be reduced. Also, the amount of processing increases by the amount of processing associated with component decomposition. However, when subband coding is performed, a sound source waveform that contains many elements that become 0 in the subband encoded waveform vector is decomposed, and these are subbanded. By configuring the speech waveform by combining on the code, much of the actual product-sum calculation process can be omitted in the calculation process of the weighted sum on the subband code.

  Further, since the method for decomposing the sound source waveform can be arbitrarily determined separately from the subband coding, the frequency resolution of the synthesized speech waveform can be increased by further decomposing the sound source waveform. As a result, it is possible to avoid problems such as an increase in processing amount and a decrease in time resolution when the number of subbands in subband encoding is increased. In this manner, sufficient speech synthesis processing can be performed even in an environment where the calculation processing performance of a portable terminal or the like is limited.

(Operation example of speech synthesizer)
Next, an operation example of the speech synthesizer 200 will be described. For the noise element, it is necessary to consider the influence of the number of taps of the band limiting filter on the sound source waveform in the component decomposition unit 205 and the number of taps of the filter in the subband coding unit 210. Although it is possible to use a method of forming noise by adding randomly selected pieces, the number of additions increases and the processing becomes complicated.

  Therefore, as a simpler method, it is easy to configure a noise source by a method of repeatedly outputting a finite-length sound source waveform. The sound source waveform of a finite length is obtained by cutting out the result of passing white noise through a band limiting filter by the amount of a sample of the period. If the period is sufficiently long (for example, several thousand samples or more), it can be regarded as substantially random.

  In this case, for example, when the number of subbands is D and the period is L, the noise element ID can be expressed by (n, i). n is a divided band number and corresponds to the subscript of the band limiting filter Fa. i is a noise element ID. A process of changing i from 0 to (L / D) -1 in order and outputting it, and returning to 0 is performed.

  FIG. 8 and FIG. 9 are flowcharts showing an example of the operation of the speech synthesizer 200. In the figure, A and B indicate points connecting the flows of FIGS. 8 and 9.

  First, the divided band number n is set to 0 (step T1). Next, the presence / absence of input data is determined (step T2). If there is no input data, the process ends. If there is input data, the fundamental frequency, mixing weight, and spectrum feature information are acquired as input data (step T3).

  The position of the impulse is determined from the input fundamental frequency (step T4). A subband encoded waveform vector corresponding to each impulse is acquired from the accumulated subband encoded waveform vector (step T5). The number of acquisitions is the same as the number of impulses. Then, the sub-band encoded waveform vector acquired on the impulse side is multiplied by the weight reflecting the mixing weight and the spectral feature information (step T6). Then, the sum is calculated on the impulse side (step T7).

  On the other hand, it acquires from the subband encoded waveform vector accumulated based on the information of n which is the subband encoded waveform vector of white noise (step T8). Then, the obtained sub-band encoded waveform vector is multiplied by a weight reflecting (1-mixing weight coefficient) and spectrum feature information (step T9). Then, the sum is calculated on the white noise side (step T10).

  Next, the divided band number n is set to n + 1 (step T11). It is determined whether or not the divided band number n is smaller than L / N (step T12). If smaller, the process proceeds to step T14. If the divided band number n is greater than or equal to L / D, the divided band number n is set to 0 (step T13).

  Next, the sum of the weighted subband encoded waveform vectors on the impulse side and the white miscellaneous side is calculated as the subband encoded waveform vector of the mixed excitation source (step T14). Then, subband synthesis processing is performed (step T15), D samples are output (step T16), and the process returns to step T2. Such processing reduces the amount of processing and enables sufficient speech synthesis processing and mixed excitation.

(Reduction of processing amount)
The reduction of the processing amount of the present invention will be described with reference to an example in which a sound source waveform is decomposed into components with the same width as the frequency band to be subjected to subband encoding and decoding. In this example, compared with a method in which component decomposition is not performed in advance, the same frequency and time resolution can be obtained with a processing amount of about 3/4.

  In order to prevent aliasing noise, a method of limiting downsampling to half of normal subband coding can be employed. According to this method, the thinning-out rate is suppressed to half of the maximum thinning-out, and an equal amplitude adjustment coefficient is set between adjacent subbands across the frequency at which aliasing noise occurs. In this way, speech synthesis that eliminates the influence of aliasing noise can be realized. On the other hand, in the present invention, such consideration is unnecessary.

  For example, when the final output sampling rate is 16 kHz and the number of subbands in subband coding is 32, the subband code sampling rate under the maximum thinning-out condition is 500 Hz. This sample rate is usually sufficient as a time resolution when representing audio features. Therefore, there is no problem with the maximum thinning from the viewpoint of time resolution, and the frequency resolution does not deteriorate. As a result, the number of multiplications per output sample in the subband code decoding unit can be reduced to about 1/4 or less by simple calculation as compared with the case where the method for preventing the aliasing noise is employed.

  On the other hand, the amplitude multiplier determined from the mixture weight and the spectral feature information needs to be multiplied by all the multipliers of the subband encoded waveform vector. For example, when the number N of excitation waveform divisions is equal to the number D of subbands in subband encoding, the number of multiplications in this process is N (= D) times that of the same process in the above method.

  However, when the amplitude characteristic in the cutoff band of the band limiting filter E (z) at the time of subband coding can be regarded as 0, it actually corresponds to the pass band of the band limiting filter among the subband encoded waveform vectors. There are three subbands, and all values other than the elements of these three subbands are zero. Therefore, the number of times of multiplication processing can be suppressed to 3 times instead of N times. In this way, the processing amount can be reduced even when compared with a method that does not decompose components in advance.

  The operation of the apparatus as described above is performed by executing a program by a computer in the apparatus. In the above embodiment, the amplitude-adjusted sine wave is synthesized so as to simulate the spectral characteristics of the output target speech, and the sub-band divided sound source waveform vector and sine wave that are amplitude-adjusted in a specific subband are synthesized. One subband division waveform vector may be generated by combining the combined sine wave components. In this case, the generated subband division waveform vector and the subband division excitation waveform vector whose amplitude has been adjusted can be combined into a single speech waveform.

DESCRIPTION OF SYMBOLS 100 Speech synthesizer 110 Subband encoding part 120 Subband encoding excitation generation part 121 Storage part 122 Selection part 130 Subband power adjustment part 140 Subband decoding part 200 Speech synthesizer 205 Component decomposition part 206a Impulse side decomposition part 206b White Noise side decomposition unit 210 Subband encoding unit 211a Impulse side encoding unit 211b White noise side encoding unit 220 Weighted sum calculation unit 220a Impulse side weighted sum calculation unit 220b White noise side weighted sum calculation unit 221a Impulse side accumulation Unit 221b white noise side accumulation unit 222a impulse side selection unit 222b white noise side selection unit 223a impulse side weighting multiplication unit 223b white noise side weighting multiplication unit 224 addition unit

Claims (6)

A speech synthesizer that synthesizes speech waveforms based on input time-series sound source control information and spectrum characteristic information,
A component decomposition unit that decomposes the sound source waveform into components of a plurality of frequency bands by a band limiting filter ;
A subband encoding unit that converts each of the decomposed components into subband encoded waveform vectors that have a predetermined dimension and are configured by subband encoded components;
A weighted sum calculation unit for calculating a weighted sum for the decomposition component with respect to the subband encoded waveform vector;
A speech synthesizer comprising: a subband decoding unit that decodes the subband-encoded waveform vector for which the weighted sum is calculated and synthesizes it into a single speech waveform.   The component decomposition unit is configured by a set of band-pass filters, and decomposes the sound source waveform into components so that the sound source waveform can be approximately reproduced by a sum of the decomposed components. Voice synthesizer. The component decomposition unit uses two types of sound source waveforms of an impulse train and white noise as the sound source waveform,
The weighted sum calculation unit obtains a weighted sum of subband encoded waveform vectors based on the two types of excitation waveforms based on a mixing ratio between an impulse train and white noise based on input spectrum characteristic information. The speech synthesizer according to claim 1 or 2.   The speech according to any one of claims 1 to 3, wherein the component decomposition unit performs component decomposition on the sound source waveform with the same width as a frequency band to be subjected to the subband encoding and decoding. Synthesizer. A speech synthesis method for synthesizing a speech waveform based on input time-series sound source control information and spectrum characteristic information,
Decomposing the sound source waveform into components of a plurality of frequency bands by a band limiting filter ;
Converting each of the decomposed components into subband encoded waveform vectors each having a predetermined dimension and composed of subband encoded components ;
Calculating a weighted sum for the decomposition component for the subband encoded waveform vector;
Decoding the subband-encoded waveform vector for which the weighted sum has been calculated, and synthesizing it into a single speech waveform. A speech synthesis program that synthesizes a speech waveform based on input time-series sound source control information and spectrum characteristic information,
Processing to decompose a sound source waveform into components of a plurality of frequency bands by a band limiting filter ;
A process of converting each of the decomposed components into a subband encoded waveform vector having a predetermined dimension composed of the subband encoded components by performing subband encoding ;
A process of calculating a weighted sum for the decomposition component with respect to the subband encoded waveform vector;
A speech synthesis program for causing a computer to execute a process of decoding the subband encoded waveform vector for which the weighted sum has been calculated and synthesizing it into a single speech waveform.

Images