JP6369189B2 - Echo suppression device, echo suppression program, echo suppression method, and communication terminal - Google Patents

Description

  The present invention relates to an echo suppression device, an echo suppression program, an echo suppression method, and a communication terminal, and is applied to, for example, an echo suppression device, an echo suppression program, an echo suppression method, and a communication terminal used in a video conference system, a telephone conference system, and the like. To get.

  For example, in a loudspeaker system such as a video conference system or a telephone conference system, sound emitted from a speaker (where “sound” includes sound, voice, etc.) wraps around a microphone and is transmitted to the transmitting side. A returning acoustic echo signal is generated. Since the acoustic echo signal significantly hinders a call, many researches and developments on acoustic echo suppression methods have been conducted so far.

  One technique for suppressing acoustic echo signals is to use an echo suppressor (echo suppression device). The echo suppressor obtains echo path characteristics, estimated echo signal and echo suppress gain from the far end signal and the near end input signal, and suppresses the acoustic echo signal by multiplying the echo suppress gain and the near end input signal. It is. By using this echo suppressor, the acoustic echo signal is suppressed. A method for suppressing an acoustic echo signal using this echo suppressor is proposed in Non-Patent Document 1.

  A conventional echo suppression device described in Non-Patent Document 1 will be described with reference to FIG. The echo suppressor 200 disclosed in Non-Patent Document 1 obtains echo path characteristics based on the far-end signal and the near-end input signal of the past frame. Then, the estimated echo signal calculation unit 209 estimates a signal obtained by multiplying the obtained echo path characteristic and the far-end signal as an estimated echo signal. The echo suppression gain calculation unit 210 obtains an echo suppression gain based on the near-end input signal and the estimated echo signal, and the integration unit 211 multiplies the echo suppression gain by the near-end input signal to suppress the acoustic echo signal.

  In the conventional echo suppression device, when the echo path characteristic is calculated during double talk, a value completely different from the actual echo path characteristic is calculated and erroneously learned. For this reason, the conventional echo suppressor stops updating the echo path characteristics during double talk.

  Here, the single talk is a state in which the acoustic echo signal has circulated to the near-end microphone and the near-end speaker is not speaking. Double talk is a state in which an acoustic echo signal goes around the microphone on the near end side, and at the same time, a speaker on the near end side speaks on the near end side.

C. Faller and C.M. Turnery, “ESTIMATION THE DELAY AND COLORATION EFFECT OF THE ACOUSTIC ECHO PATH FOR LOW COMPLEITY ECHO SUPPRESSION”, Proc. Intl. Works, On Acoustic. Echo and Noise Control (IWAENC) 2005, pp. 53-56, Oct. 2005

  However, if the doubleke state continues, the update of the echo path characteristic stops, and the state where the echo path characteristic cannot be updated continues. If the actual echo path characteristic is changed while the doubleke state continues, there may be a problem that the state in which the acoustic echo signal cannot be suppressed continues.

  Therefore, an echo suppressor, an echo suppression program, an echo suppression method, and a communication terminal that can maintain echo suppression performance by calculating echo path characteristics and updating the echo path characteristics even during double talk are desired. Yes.

  The present invention has been made in view of the above problems, and determines whether the near-end input signal is a frequency component of an acoustic echo signal or a frequency component of a near-end sound signal for each frequency bin, The frequency bin determined to be a frequency component calculates and updates the echo path characteristic of the frequency bin, while the frequency bin determined to be the frequency component of the near-end sound signal calculates the echo path characteristic of the frequency bin. By not updating, it is possible to calculate and update the echo path characteristics by updating only the frequency bin of the frequency component of the acoustic echo signal of the near-end input signal even during double talk. An echo suppression device, an echo suppression program, an echo suppression method, and a communication terminal are provided.

  An echo suppression apparatus according to a first aspect of the present invention is an echo suppression apparatus that suppresses an acoustic echo signal based on a near-end input signal and a far-end signal, and (1) converts an input far-end signal into a frequency domain signal. A far-end signal amplitude spectrum calculating means for obtaining an amplitude spectrum of the far-end signal; and (2) a near-end input for converting the input near-end input signal into a frequency domain signal to obtain an amplitude spectrum of the near-end input signal. A signal amplitude spectrum calculating means; (3) an estimated echo signal calculating means for obtaining an amplitude spectrum of the estimated echo signal using the amplitude spectrum of the far-end signal and the held echo path characteristic; and (4) an estimated echo signal Using the amplitude spectrum and the amplitude spectrum of the near-end input signal, the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal or the frequency of the sound signal on the near-end side. Near-end input signal frequency component determination means for determining whether each component is a frequency component, and (5) the frequency bin determined as the frequency component of the acoustic echo signal by the determination result of the near-end input signal frequency component determination means, Using the amplitude spectrum of the far-end signal and the amplitude spectrum of the near-end input signal, the echo path characteristics of the current frame of the frequency bin are calculated, and the echo path characteristics for the frequency bin determined to be the frequency component of the near-end sound signal A frequency bin echo path characteristic calculating means that does not calculate the frequency bin, and (6) a frequency bin determined as a frequency component of the acoustic echo signal using the held echo path characteristic and the echo path characteristic of the current frame calculated by the frequency bin echo path characteristic calculating means. Echo path characteristic updating means for updating the above echo path characteristic, and (7) an estimated echo signal and a near-end input Using the amplitude spectrum of the signal, the echo suppression gain is calculated, characterized in that it comprises a echo suppression means for suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal.

  An echo suppression program according to a second aspect of the present invention is an echo suppression program for suppressing an acoustic echo signal based on a near-end input signal and a far-end signal, and (1) converting an input far-end signal into a signal in the frequency domain. A far-end signal amplitude spectrum calculating means for obtaining an amplitude spectrum of the far-end signal by conversion, (2) a near-end signal for obtaining an amplitude spectrum of the near-end input signal by converting the input near-end input signal into a frequency domain signal. End input signal amplitude spectrum calculating means, (3) estimated echo signal calculating means for obtaining the amplitude spectrum of the estimated echo signal using the amplitude spectrum of the far end signal and the retained echo path characteristics, and (4) the estimated echo signal Whether the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal using the amplitude spectrum and the amplitude spectrum of the near-end input signal Is a near-end input signal frequency component determining means for determining for each frequency bin whether the frequency component of the sound signal on the near-end side, (5) The frequency component of the acoustic echo signal is determined based on the determination result of the near-end input signal frequency component determining means. Using the amplitude spectrum of the far-end signal and the amplitude spectrum of the near-end input signal for the determined frequency bin, the echo path characteristics of the current frame of the frequency bin are calculated and determined as the frequency component of the near-end sound signal. Frequency bin echo path characteristic calculation means that does not calculate the echo path characteristics for the frequency bins, and (6) the echo path characteristics that are held and the echo path characteristics of the current frame calculated by the frequency bin echo path characteristic calculation means, Echo path characteristic updating means for updating the echo path characteristic of the frequency bin determined as (7) estimated echo Using the amplitude spectrum of the No. and the near-end input signal, the echo suppression gain is calculated, characterized in that to function as an echo suppression means for suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal.

  According to a third aspect of the present invention, there is provided an echo suppression method for suppressing an acoustic echo signal based on a near-end input signal and a far-end signal, wherein: (1) the far-end signal amplitude spectrum calculating means is configured to input the far-end signal; Is converted to a frequency domain signal to obtain the amplitude spectrum of the far end signal. (2) The near end input signal amplitude spectrum calculating means converts the input near end input signal to a frequency domain signal, The amplitude spectrum of the end input signal is obtained, and (3) the estimated echo signal calculation means obtains the amplitude spectrum of the estimated echo signal using the amplitude spectrum of the far end signal and the held echo path characteristic, and (4) near The end input signal frequency component determination means uses the amplitude spectrum of the estimated echo signal and the amplitude spectrum of the near end input signal, and the frequency component of the amplitude spectrum of the near end input signal is the acoustic echo signal. It is determined for each frequency bin whether it is a wave number component or a frequency component of the near-end sound signal, and (5) the frequency bin echo path characteristic calculation means determines the frequency of the acoustic echo signal based on the determination result of the near-end input signal frequency component determination means Using the amplitude spectrum of the far-end signal and the amplitude spectrum of the near-end input signal for the frequency bin determined to be a component, the echo path characteristics of the current frame of the frequency bin are calculated, and the frequency component of the sound signal on the near-end side is calculated. The echo path characteristic is not calculated for the determined frequency bin, and (6) the echo path characteristic update means uses the held echo path characteristic and the echo path characteristic of the current frame calculated by the frequency bin echo path characteristic calculation means. The echo path characteristic of the frequency bin determined to be the frequency component of the signal is updated, and (7) the echo suppression means Using the amplitude spectrum of the signal and the near-end input signal, the echo suppression gain is calculated, characterized by suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal.

  A communication terminal according to a fourth aspect of the present invention includes the echo suppression device according to the first aspect of the present invention.

  According to the present invention, the echo path characteristic is calculated and updated only by the frequency bin of the frequency component of the acoustic echo signal with the frequency component of the amplitude spectrum of the near-end input signal. Calculate, update, and maintain echo suppression performance.

It is a block diagram which shows the structure of the echo suppression apparatus of embodiment of this invention. It is a block diagram for demonstrating the conventional echo suppression apparatus.

(A) Main Embodiments Hereinafter, main embodiments of an echo suppression device, an echo suppression program, an echo suppression method, and a communication terminal according to the present invention will be described in detail with reference to the drawings.

  This embodiment exemplifies a case where the present invention is applied to, for example, an echo suppression device, an echo suppression program, an echo suppression method, and a communication terminal of an audio transmission / reception device of a loudspeaking call system such as a video conference system or a telephone conference system.

(A-1) Configuration of Embodiment FIG. 1 is a block diagram showing a configuration of an echo suppression apparatus 100 according to an embodiment of the present invention.

  In FIG. 1, an echo suppression apparatus 100 according to the embodiment includes a far-end signal input terminal 101, a DA converter 102, a speaker 103, a microphone 104, an AD converter 105, a far-end signal frequency domain converter 106, and a far-end signal amplitude. Spectrum calculation unit 107, echo path characteristic holding unit 108, estimated echo signal calculation unit 109, near-end input signal frequency domain conversion unit 110, near-end input signal amplitude spectrum calculation unit 111, near-end input signal frequency component determination unit 112, frequency bin echo path A characteristic calculation unit 113, a frequency bin echo path characteristic update unit 114, an echo suppression gain calculation unit 115, an echo suppression unit 116, a near-end output signal time domain conversion unit 117, and a near-end signal output terminal 118 are included.

  The echo suppressor 100 according to this embodiment is an implementation of the echo suppressor according to the present invention. The echo suppression device 100 performs echo suppression processing (echo suppression processing) on the near-end input signal even when the far-end signal is input as an acoustic echo signal to the near-end microphone, and the acoustic echo signal Is appropriately suppressed.

  The echo suppression device 100 may be constructed as a dedicated board, for example. The echo suppression apparatus 100 may be realized by writing an echo suppression program to a DSP (digital signal processor), for example, and is realized by a CPU and software (echo suppression program) executed by the CPU. It may be. Even in that case, the function of the echo suppressor 100 can be represented by FIG.

  Furthermore, the echo suppression apparatus 100 can be mounted on a communication terminal. Communication terminals equipped with the echo suppression device 100 can be widely applied to devices having a communication function, and include, for example, personal computers, tablet computers, mobile phones, smartphones, game terminals, wearable terminals, and the like.

  The far-end signal input terminal 101 is connected to a radio wave of a network such as an Internet protocol (IP) network or a wireless network such as a mobile phone, and the far-end signal is input via a connected line. The

  The far-end signal input to the far-end signal input terminal 101 is output to the DA converter 102, and the DA converter 102 converts the digital sound signal into an analog sound signal. The analog sound signal is output to the near end side through the speaker 103.

  On the other hand, sound signals such as voices emitted by the near-end speaker, environmental sounds, and acoustic echo signals (for example, analog sound signals output from the speaker 103 wrap around the near-end space) The analog sound signal on which the signal is superimposed is received by the microphone 104 and output to the AD converter 105. The analog sound signal is converted into a digital sound signal by the AD converter 105 and input to the echo suppression device 100 as a near-end input signal.

  The far end signal frequency domain transforming unit 106 transforms the far end signal into a frequency domain signal by, for example, fast Fourier transform (FFT) or the like. The far-end signal frequency domain conversion unit 106 outputs the frequency spectrum of the far-end signal to the far-end signal amplitude spectrum calculation unit 107.

  The far-end signal amplitude spectrum calculation unit 107 calculates the amplitude spectrum of the far-end signal based on the frequency spectrum of the far-end signal output from the far-end signal frequency domain conversion unit 106. The far-end signal amplitude spectrum calculation unit 107 outputs the calculated amplitude spectrum of the far-end signal to the estimated echo signal calculation unit 109 and the frequency bin echo path characteristic calculation unit 113.

  The echo path characteristic holding unit 108 holds the echo path characteristic. The echo path characteristic holding unit 108 outputs the held echo path characteristic to the estimated echo signal calculation unit 109 and the frequency bin echo path characteristic update unit 114.

  The estimated echo signal calculation unit 109 calculates the amplitude spectrum of the estimated echo signal by multiplying the amplitude spectrum of the far-end signal by the echo path characteristic held in the echo path characteristic holding unit 108. The estimated echo signal calculation unit 109 outputs the calculated amplitude spectrum of the estimated echo signal to the near-end input signal frequency component determination unit 112 and the echo suppression gain calculation unit 115.

  The near-end input signal frequency domain conversion unit 110 converts the near-end input signal into a frequency domain signal by, for example, fast Fourier transform (FFT). The near-end input signal frequency domain conversion unit 110 outputs the frequency spectrum of the near-end input signal obtained by the conversion to the echo suppression unit 116 and the near-end input signal amplitude spectrum calculation unit 111.

  The near-end input signal amplitude spectrum calculation unit 111 calculates the amplitude spectrum of the near-end input signal based on the frequency spectrum of the near-end input signal from the near-end input signal frequency domain conversion unit 110. The near-end input signal amplitude spectrum calculation unit 111 outputs the calculated amplitude spectrum of the near-end input signal to the near-end input signal frequency component determination unit 112, the frequency bin echo path characteristic calculation unit 113, and the echo suppression gain calculation unit 115.

  The near-end input signal frequency component determination unit 112 is based on the amplitude spectrum of the estimated echo signal estimated by the estimated echo signal calculation unit 109 and the amplitude spectrum of the near-end input signal from the near-end input signal amplitude spectrum calculation unit 111. Thus, it is determined for each frequency bin whether the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal or the frequency component of the sound signal on the near-end side. The near-end input signal frequency component determination unit 112 outputs the determination result determined for each frequency bin to the frequency bin echo path characteristic calculation unit 113.

  The frequency bin echo path characteristic calculation unit 113 includes an amplitude spectrum of the far end signal from the far end signal amplitude spectrum calculation unit 107, an amplitude spectrum of the near end input signal from the near end input signal amplitude spectrum calculation unit 11, and a near end input signal. Based on the determination result from the frequency component determination unit 112, the echo path characteristic is calculated for the frequency bin determined as the frequency component of the acoustic echo signal. The frequency bin echo path characteristic calculation unit 113 outputs the calculated echo path characteristic of the frequency bin to the frequency bin echo path characteristic update unit 114.

  The frequency bin echo path characteristic updating unit 114 is configured for each frequency bin based on the echo path characteristic for each frequency bin from the frequency bin echo path characteristic calculation unit 113 and the echo path characteristic of the corresponding frequency bin held in the estimated echo signal calculation unit 109. The echo path characteristic is updated. The frequency bin echo path characteristic updating unit 114 outputs and holds the updated echo path characteristic for each frequency bin to the echo path characteristic holding unit 108.

  The echo suppression gain calculator 115 uses the amplitude spectrum of the estimated echo signal from the estimated echo path calculator 109 and the amplitude spectrum of the near-end input signal from the near-end input signal frequency domain converter 110 to use the near-end input signal. The echo suppression gain for suppressing the acoustic echo signal superimposed on is calculated. The echo suppression gain calculation unit 115 outputs the calculated echo suppression gain to the echo suppression unit 116.

  The echo suppression unit 116 multiplies the echo suppression gain from the echo suppression gain calculation unit 115 by the frequency spectrum of the near-end input signal, thereby obtaining a frequency spectrum that suppresses the acoustic echo signal superimposed on the near-end input signal. It is what you want. The echo suppressor 116 outputs the frequency spectrum in which the acoustic echo signal is suppressed to the near-end output signal time domain conversion unit 117 as the frequency spectrum of the near-end output signal.

  The near-end output signal time domain conversion unit 117 converts the frequency spectrum of the near-end output signal from the echo suppression unit 116 into a time-domain digital sound signal by, for example, inverse fast Fourier transform (Inverse FFT), and outputs the near-end output signal. The signal is output to the near end signal output terminal 118.

  The near end signal output terminal 118 is connected to, for example, a radio wave of a network such as the Internet Protocol (IP) network or a wireless network such as a mobile phone, and the near end signal output terminal 118 receives the near end output signal via the connected line. Outputs to the far end that is the communication partner.

(A-2) Operation of Embodiment Next, the operation of echo suppression processing in the echo suppression apparatus 100 according to this embodiment will be described in detail.

  First, after the operation of the echo suppression apparatus 100 is started, for example, by a radio wave of a network such as an Internet protocol (IP) network or a wireless network such as a mobile phone. The far end signal is input to the far end signal input terminal 101.

  The far-end signal input to the far-end signal input terminal 101 is output to the DA converter 102, and the far-end signal is converted from a digital sound signal to an analog sound signal by the DA converter 102. Then, the far-end signal that is an analog sound signal is output to the near-end side through the speaker 103.

  On the other hand, a sound signal such as a voice uttered by a near-end speaker, an analog sound signal superimposed with an environmental sound, an acoustic echo signal, or the like is received by the microphone 104 and output to the AD converter 105. The analog sound signal is converted into a digital sound signal by the AD converter 105, and the analog sound signal is input to the echo suppressor 100 as a near-end input signal.

  The far-end signal frequency domain transforming unit 106 transforms the far-end signal into a frequency-domain signal by, for example, fast Fourier transform (FFT), and the far-end signal frequency spectrum ROUT (i, ω) is The signal is output to the end signal amplitude spectrum calculation unit 107.

  The far-end signal amplitude spectrum calculation unit 107 obtains the far-end signal amplitude spectrum | ROUT (i, ω) | according to the equation (1) using the far-end signal frequency spectrum ROUT (i, ω).

ここで、ｉはフレーム、ωは周波数ビン、ＲＯＵＴ＿ｒｅａｌ（ｉ，ω）とＲＯＵＴ＿ｉｍａｇｅ（ｉ，ω）は、フレームｉにおける周波数ビンωの遠端信号の周波数スペクトルの実数部と虚数部を示しており、遠端信号の周波数スペクトルＲＯＵＴ（ｉ，ω）は、（２）式で表すことができる。

Here, i is a frame, ω is a frequency bin, and ROUT_real (i, ω) and ROUT_image (i, ω) indicate the real part and imaginary part of the frequency spectrum of the far-end signal of the frequency bin ω in frame i. The frequency spectrum ROUT (i, ω) of the far-end signal can be expressed by equation (2).

（２）式のｊは虚数を表している。そして、遠端信号振幅スペクトル計算部１０７により求められた遠端信号の周波数スペクトル｜ＲＯＵＴ（ｉ，ω）｜は、周波数ビンエコーパス特性計算部１１３及び推定エコー信号計算部１０９に出力する。

(2) j represents an imaginary number. Then, the frequency spectrum | ROUT (i, ω) | of the far-end signal obtained by the far-end signal amplitude spectrum calculating unit 107 is output to the frequency bin echo path characteristic calculating unit 113 and the estimated echo signal calculating unit 109.

  The estimated echo signal calculation unit 109 uses the echo path characteristic | H (i−1, ω) | held by the echo path characteristic holding unit 108 and the amplitude spectrum | ROUT (i, ω) | of the far-end signal. , (3), the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal is obtained.

（３）式は、遠端信号の振幅スペクトル｜ＲＯＵＴ（ｉ，ω）｜に、エコーパス保持部１０８に保持しているエコーパス特性｜Ｈ（ｉ−１，ω）｜の対応する周波数ビンを乗じて、当該周波数ビンの推定エコー信号の振幅スペクトル｜ＥＣＨＯ（ｉ，ω）｜を求める。そして、推定エコー信号計算部１０９により求められた推定エコー信号の振幅スペクトル｜ＥＣＨＯ（ｉ，ω）｜は、近端入力信号周波数成分判定部１１２及びエコーサプレ部１１５に出力される。

Equation (3) multiplies the amplitude spectrum | ROUT (i, ω) | of the far-end signal by the corresponding frequency bin of the echo path characteristic | H (i−1, ω) | held in the echo path holding unit 108. Thus, the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal of the frequency bin is obtained. The amplitude spectrum | ECHO (i, ω) | of the estimated echo signal obtained by the estimated echo signal calculation unit 109 is output to the near-end input signal frequency component determination unit 112 and the echo suppression unit 115.

  On the other hand, the near-end input signal frequency domain transforming unit 110 transforms the near-end input signal into a frequency-domain signal by, for example, fast Fourier transform, and the frequency spectrum SIN (i, ω of the transformed near-end input signal. ) Is output to the near-end input signal amplitude spectrum calculation unit 111 and the echo suppression unit 116.

  The near-end input signal amplitude spectrum calculation unit 111 uses the frequency spectrum SIN (i, ω) of the near-end input signal and the amplitude spectrum | SIN (i, ω) | Desired.

ここで、ＳＩＮ＿ｒｅａｌ（ｉ，ω）とＳＩＮ＿ｉｍａｇｅ（ｉ，ω）とは、フレームｉにおける周波数ビンωの近端入力信号の周波数スペクトルの実数部と虚数部を示しており、近端入力信号の周波数スペクトルＳＩＮ（ｉ，ω）は（５）式で表すことができる。

Here, SIN_real (i, ω) and SIN_image (i, ω) indicate the real part and the imaginary part of the frequency spectrum of the near-end input signal of the frequency bin ω in frame i, and the frequency of the near-end input signal. The spectrum SIN (i, ω) can be expressed by equation (5).

（５）式のｊは虚数を表している。近端入力信号振幅スペクトル計算部１１１により求められた近端入力信号の振幅スペクトル｜ＳＩＮ（ｉ，ω）｜は、近端入力信号周波数成分判定部１１２、周波数ビンエコーパス特性計算部１１３及びエコーサプレスゲイン計算部１１５に出力される。

In the formula (5), j represents an imaginary number. The amplitude spectrum | SIN (i, ω) | of the near-end input signal obtained by the near-end input signal amplitude spectrum calculation unit 111 is a near-end input signal frequency component determination unit 112, a frequency bin echo path characteristic calculation unit 113, and an echo suppression gain. It is output to the calculation unit 115.

  The near-end input signal frequency component determination unit 112 uses the near-end input using the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal and the amplitude spectrum | SIN (i, ω) | of the near-end input signal. Whether the frequency component of the amplitude spectrum of the signal is the frequency component of the acoustic echo signal or the frequency component of the near-end sound signal is determined for each frequency bin.

  Here, a method for determining whether the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal or the frequency component of the near-end sound signal will be described.

  For example, the near-end input signal frequency component determination unit 112 compares the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal with the amplitude spectrum | SIN (i, ω) | of the near-end input signal for each frequency bin. If the condition of equation (6) is satisfied, the frequency component of the frequency bin is determined as the frequency component of the acoustic echo signal. If the condition of equation (6) is not satisfied, the frequency component of the frequency bin is The frequency component of the sound signal is determined.

（６）式のＴＨは閾値である。（６）式は、近端入力信号の振幅スペクトル｜ＳＩＮ（ｉ，ω）｜が、推定エコー信号の振幅スペクトル｜ＥＣＨＯ（ｉ，ω）｜に閾値ＴＨを減算した値から、推定エコー信号の振幅スペクトル｜ＥＣＨＯ（ｉ，ω）｜に閾値ＴＨを加算した値までの範囲にあれば、その周波数ビンの周波数成分は音響エコー信号であることを示している。ここで、閾値ＴＨは、小さい値（例えばＴＨ＝１０程度の値）とすることができ、固定値としても良いし又は変動値としても良い。

In the equation (6), TH is a threshold value. (6) is obtained by subtracting the threshold TH from the amplitude spectrum | SIN (i, ω) | of the near-end input signal and the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal. If the amplitude spectrum | ECHO (i, ω) | is within the range up to the value obtained by adding the threshold value TH, it indicates that the frequency component of the frequency bin is an acoustic echo signal. Here, the threshold value TH can be a small value (for example, a value of about TH = 10), and may be a fixed value or a variable value.

  That is, the near-end input signal frequency component determination unit 112 determines whether the near-end input signal amplitude spectrum | SIN (i, ω) | and the estimated echo signal amplitude spectrum | ECHO (i, ω) | If each frequency bin is determined and approximated, it can be determined that the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal.

  Further, as a modification of the equation (6), the amplitude spectrum | SIN (i, ω) | of the near-end input signal is divided by the amplitude spectrum | ECHO (i, ω) | However, it may be determined whether it is within a predetermined range.

  Note that the method for determining for each frequency bin whether the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal or the sound signal of the near-end side is limited to the above method. Instead, various methods can be widely applied.

  For example, the near-end input signal frequency component determination unit 112 may perform determination using the sparsity of speech. Here, the sparsity is a property that the energy of the audio signal exists only sparsely in the time frequency domain. Specifically, the near-end input signal frequency component determination unit 112 determines, for each frequency bin, whether or not there is a frequency component using the amplitude spectrum of the estimated echo signal. A method of determining that the corresponding frequency bin of the frequency component of the end input signal is the frequency bin of the frequency component of the acoustic echo signal can be applied.

  Further, for example, the near-end input signal frequency component determination unit 112 may perform determination using an error for each frequency bin between the amplitude spectrum of the near-end input signal and the amplitude spectrum of the estimated echo signal. That is, for each frequency bin, it may be determined whether the difference value between the amplitude spectrum of the near-end input signal and the amplitude spectrum of the estimated echo signal is small. Specifically, the near-end input signal frequency component determination unit 112 calculates a difference for each frequency bin between the amplitude spectrum of the near-end input signal and the amplitude spectrum of the estimated echo signal, and the difference value for each frequency bin is predetermined. A method for determining whether or not the frequency bin is within a range and determining that the corresponding frequency bin of the frequency component of the near-end input signal is a frequency bin of the frequency component of the acoustic echo signal when the difference value is within the predetermined range Can be applied.

  Further, for example, the near-end input signal frequency component determination unit 112 performs determination using a correlation function for each frequency bin between the amplitude spectrum of the near-end input signal and the amplitude spectrum of the estimated echo signal, and the amplitude spectrum of the near-end input signal Can determine that the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal.

  Specifically, the near-end input signal frequency component determination unit 112 obtains a correlation coefficient for each frequency bin from the amplitude spectrum of the near-end input signal and the amplitude spectrum of the estimated echo signal, and generates a correlation coefficient for each frequency bin. When the correlation coefficient is within the predetermined range, the corresponding frequency bin of the frequency component of the near-end input signal is the frequency bin of the frequency component of the acoustic echo signal. A determination method can be applied.

  Then, the near-end input signal frequency component determination unit 112 gives the determination result to the frequency bin echo path characteristic calculation unit 113.

Based on the determination result from the near-end input signal frequency component determination unit 112, the frequency bin echo path characteristic calculation unit 113 applies the frequency bin in which the frequency component of the amplitude spectrum of the near-end input signal is determined as the frequency component of the acoustic echo signal. , (7), the echo path characteristic | H ₁ (i, ω) | of the current frame of the frequency bin is obtained.

Here, the frequency bin echo path characteristic calculation unit 113 includes the far-end signal amplitude spectrum | ROUT (i, ω) | from the far-end signal amplitude spectrum calculation unit 107 and the near-end output signal amplitude spectrum calculation unit 109. Using the amplitude spectrum | SIN (i, ω) | of the output signal, the echo path characteristic | H ₁ (i, ω) | of the current frame of the frequency bin is obtained according to the equation (7).

現フレームのエコーパス特性｜Ｈ₁（ｉ，ω）｜が求まれば、周波数ビンエコーパス特定計算部１１３は、周波数ビンエコーパス特性更新部１１４に現フレームの周波数ビン毎のエコーパス特性を出力する。

When the echo path characteristic | H ₁ (i, ω) | of the current frame is obtained, the frequency bin echo path specifying calculation unit 113 outputs the echo path characteristic for each frequency bin of the current frame to the frequency bin echo path characteristic update unit 114.

  Based on the determination result from the near-end input signal frequency component determination unit 112, for frequency bins in which the frequency component of the amplitude spectrum of the near-end input signal is determined to be the frequency component of the sound signal on the near-end side. The frequency bin echo path characteristic calculation unit 113 does not calculate the echo path characteristic of the current frame of the frequency bin.

When the frequency bin echo path characteristic update unit 114 outputs the echo path characteristic | H ₁ (i, ω) | of the current frame of the frequency bin determined as the acoustic echo component from the frequency bin echo path characteristic calculation unit 113, the frequency bin echo path characteristic update unit 114 outputs the echo path of the current frame. Using the characteristic | H ₁ (i, ω) | and the echo path characteristic | H (i−1, ω) | of the corresponding frequency bin held in the estimated echo signal calculation unit 109, the echo path of the frequency bin Update characteristics for each frequency bin.

  At this time, the frequency bin echo path characteristic update unit 114 updates the echo path characteristic only for the frequency bin calculated by the frequency bin echo path characteristic update unit 114. However, the frequency bin echo path characteristic updating unit 114 does not update the echo path characteristic for the frequency bin determined to be the near-end sound signal. Specifically, the frequency bin echo path characteristic updating unit 114 updates the echo path characteristic | H (i, ω) | according to the equation (8).

（８）式のａは時定数フィルタの係数であり、ａは０より大きく１以下の値である。エコーパス特性の更新を遅くしたい場合は、ａは１に近い値が望ましく（例えばａ＝０．９９等の値）、エコーパス特性の更新を早くしたい場合は、ａは０に近い値が望ましい（例えばａ＝０．０１等の値）。

In equation (8), a is a coefficient of the time constant filter, and a is a value greater than 0 and less than or equal to 1. When it is desired to delay the update of the echo path characteristics, a is desirably a value close to 1 (for example, a value such as a = 0.99), and when it is desired to update the echo path characteristics earlier, a is desirably a value close to 0 (for example, a = value such as 0.01).

For the frequency bin whose frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal, the frequency bin echo path characteristic updating unit 114 holds the frequency bin held in the echo path characteristic holding unit 108. And the echo path characteristic | H ₁ (i, ω) | of the current frame obtained by the echo path characteristic calculation unit 113 is updated.

  On the other hand, for the frequency bin in which the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the near-end sound signal, the frequency bin echo path characteristic updating unit 114 does not update the estimated echo path. That is, for the frequency bin whose frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the sound signal on the near-end side, the echo path characteristic | H (i− of the frequency bin held in the echo path characteristic holding unit 108 1, ω) |.

  The echo suppression gain calculation unit 115 uses the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal and the amplitude spectrum | SIN (i, ω) | of the near-end input signal according to the equation (9): An echo suppression gain G (i, ω) is obtained.

（９）式は、周波数ビン毎に、近端入力信号の振幅スペクトル｜ＳＩＮ（ｉ，ω）｜から推定エコー信号の振幅スペクトル｜ＥＣＨＯ（ｉ，ω）｜を差し引いた振幅スペクトルを、近端入力信号の振幅スペクトル｜ＳＩＮ（ｉ，ω）｜で除することで、エコーサプレスゲインＧ（ｉ，ω）を求める。

Equation (9) is obtained by subtracting the amplitude spectrum obtained by subtracting the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal from the amplitude spectrum | SIN (i, ω) | of the near-end input signal for each frequency bin. The echo suppression gain G (i, ω) is obtained by dividing by the amplitude spectrum | SIN (i, ω) | of the input signal.

  The echo suppression gain G (i, ω) obtained by the echo suppression gain calculation unit 115 is output to the echo suppression unit 116.

The echo suppressor 116 uses the near-end input signal spectrum SIN (i, ω) and the echo-suppress gain G (i, ω), according to the equations (10) and (11), and the near-end input signal spectrum. The acoustic echo signal superimposed on SIN (i, ω) is suppressed.

  Here, SOUT_real (i, ω) and SOUT_image (i, ω) indicate the real part and the imaginary part of the frequency spectrum of the near-end output signal of the frequency bin ω in the frame i, and the frequency spectrum of the near-end output signal. SOUT (i, ω) can be expressed by equation (12).

（１２）式のｊは虚数を表している。（１０）式と（１１）式は、周波数スペクトルの実数部、虚数部にエコーサプレスゲインＧ（ｉ，ω）を周波数ビン毎に乗じて、音響エコー信号を抑圧した近端出力信号の周波数スペクトルを求めるものである。

In the expression (12), j represents an imaginary number. Equations (10) and (11) are obtained by multiplying the real part and imaginary part of the frequency spectrum by the echo suppression gain G (i, ω) for each frequency bin, and the frequency spectrum of the near-end output signal in which the acoustic echo signal is suppressed. Is what you want.

  Then, the echo suppress unit 116 outputs the frequency spectrum SOUT (i, ω) of the near-end output signal in which the acoustic echo signal is suppressed to the near-end output signal time domain conversion unit 117.

  In the near-end output signal time domain conversion unit 117, the spectrum SOUT (i, ω) of the near-end output signal is converted into a time-domain digital sound signal by, for example, inverse fast Fourier transform (Inverse FFT), and the near-end output signal is output. The signal is output to the near end signal output terminal 118.

  The near-end signal output terminal 118 is connected to, for example, a radio network such as an Internet protocol (IP) network or a wireless network such as a mobile phone, and is connected via a line to which the near-end output signal is connected. Outputs to the far end that is the other party.

(A-3) Effect of Embodiment As described above, according to this embodiment, even in the case of double talk, only the frequency bin of the frequency component of the acoustic echo signal is used in the echo path characteristic using the amplitude spectrum of the near-end input signal. By calculating and updating, the convergence of the echo path characteristics is accelerated and the echo suppression performance is maintained.

(B) Other Embodiments Although various modified embodiments have been described in the above-described embodiments, the present invention can also be applied to the following modified embodiments.

  The echo suppression device described in each of the above-described embodiments may be mounted on a device including a voice communication device used in a video conference system, a telephone conference system, or the like. Further, the echo suppression device of the present invention may be mounted on a mobile terminal such as a mobile phone or a smartphone.

  DESCRIPTION OF SYMBOLS 100 ... Echo suppression apparatus, 101 ... Far end signal input terminal, 102 ... DA converter, 103 ... Speaker, 104 ... Microphone, 105 ... AD converter, 106 ... Far end signal frequency domain conversion calculation part, 107 ... Far end signal Amplitude spectrum calculation unit, 108 ... Echo path characteristic holding unit, 109 ... Estimated echo signal calculation unit, 110 ... Near end input signal frequency domain conversion unit, 111 ... Near end input signal amplitude spectrum calculation unit, 112 ... Near end input signal frequency component Determination unit 113... Frequency bin echo path characteristic calculation unit 114. Frequency bin echo path characteristic update unit 115 115 Echo suppression gain calculation unit 116. Echo suppression unit 117 117 Near end output signal time domain conversion unit 118 118 Near end signal output Terminal.

Claims (4)

In an echo suppression device that suppresses an acoustic echo signal based on a near-end input signal and a far-end signal,
A far-end signal amplitude spectrum calculating means for converting an input far-end signal into a frequency-domain signal and obtaining an amplitude spectrum of the far-end signal;
A near-end input signal amplitude spectrum calculating means for converting the input near-end input signal into a frequency domain signal and obtaining an amplitude spectrum of the near-end input signal;
Estimated echo signal calculation means for obtaining the amplitude spectrum of the estimated echo signal using the amplitude spectrum of the far-end signal and the retained echo path characteristic;
Using the amplitude spectrum of the estimated echo signal and the amplitude spectrum of the near-end input signal, the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal or the frequency component of the sound signal on the near-end side. Near-end input signal frequency component determining means for determining whether for each frequency bin;
Based on the determination result of the near-end input signal frequency component determination means, for the frequency bin determined as the frequency component of the acoustic echo signal, using the amplitude spectrum of the far-end signal and the amplitude spectrum of the near-end input signal, A frequency bin echo path characteristic calculating means for calculating the echo path characteristic of the current frame of the frequency bin and not calculating the echo path characteristic for the frequency bin determined as the frequency component of the near-end sound signal;
The echo path characteristic update for updating the echo path characteristic of the frequency bin determined to be the frequency component of the acoustic echo signal, using the held echo path characteristic and the echo path characteristic of the current frame calculated by the frequency bin echo path characteristic calculating means. Means,
Echo suppression means for calculating an echo suppression gain using the estimated echo signal and the amplitude spectrum of the near-end input signal, and for suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal. Echo suppression device. In an echo suppression program that suppresses acoustic echo signals based on near-end input signals and far-end signals,
Computer
A far-end signal amplitude spectrum calculating means for converting the input far-end signal into a frequency-domain signal and obtaining an amplitude spectrum of the far-end signal;
Near-end input signal amplitude spectrum calculating means for converting the input near-end input signal into a frequency domain signal to obtain the amplitude spectrum of the near-end input signal,
Estimated echo signal calculating means for obtaining an amplitude spectrum of the estimated echo signal using the amplitude spectrum of the far-end signal and the retained echo path characteristic;
Using the amplitude spectrum of the estimated echo signal and the amplitude spectrum of the near-end input signal, the frequency component of the amplitude spectrum of the near-end input signal is the frequency component of the acoustic echo signal or the frequency component of the sound signal on the near-end side. Near-end input signal frequency component determining means for determining whether for each frequency bin,
Based on the determination result of the near-end input signal frequency component determination means, for the frequency bin determined as the frequency component of the acoustic echo signal, using the amplitude spectrum of the far-end signal and the amplitude spectrum of the near-end input signal, A frequency bin echo path characteristic calculating means for calculating the echo path characteristic of the current frame of the frequency bin and not calculating the echo path characteristic for the frequency bin determined as the frequency component of the near-end sound signal;
The echo path characteristic update for updating the echo path characteristic of the frequency bin determined to be the frequency component of the acoustic echo signal, using the held echo path characteristic and the echo path characteristic of the current frame calculated by the frequency bin echo path characteristic calculating means. means,
An echo suppression gain is calculated using the estimated echo signal and the amplitude spectrum of the near-end input signal, and functions as an echo suppression unit that suppresses the estimated echo signal from the amplitude spectrum of the near-end input signal. Echo suppression program. In an echo suppression method for suppressing an acoustic echo signal based on a near-end input signal and a far-end signal,
The far-end signal amplitude spectrum calculating means converts the input far-end signal into a frequency-domain signal to obtain the amplitude spectrum of the far-end signal,
The near-end input signal amplitude spectrum calculating means converts the input near-end input signal into a frequency domain signal to obtain the near-end input signal amplitude spectrum,
The estimated echo signal calculation means obtains the amplitude spectrum of the estimated echo signal using the amplitude spectrum of the far-end signal and the retained echo path characteristic,
The near-end input signal frequency component determining means uses the amplitude spectrum of the estimated echo signal and the amplitude spectrum of the near-end input signal to determine whether the frequency component of the amplitude spectrum of the near-end input signal is a frequency component of the acoustic echo signal. Or determine whether the frequency component of the near-end sound signal for each frequency bin,
The frequency bin echo path characteristic calculation means, for the frequency bin determined as the frequency component of the acoustic echo signal based on the determination result of the near-end input signal frequency component determination means, the amplitude spectrum of the far-end signal and the near-end input signal Using the amplitude spectrum, calculate the echo path characteristic of the current frame of the frequency bin, do not calculate the echo path characteristic for the frequency bin determined to be the frequency component of the near-end sound signal,
The echo path characteristic update means uses the held echo path characteristic and the echo path characteristic of the current frame calculated by the frequency bin echo path characteristic calculation means, and uses the echo path characteristic of the frequency bin determined as the frequency component of the acoustic echo signal. Update
Echo suppression means calculates an echo suppression gain using the estimated echo signal and the amplitude spectrum of the near-end input signal, and suppresses the estimated echo signal from the amplitude spectrum of the near-end input signal. Echo suppression method.   A communication terminal comprising the echo path suppression device according to claim 1.

Images