JP2006067127A - Method and apparatus of reducing reverberation - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and an apparatus of reducing a reverberation which can remove a reverberation sound even if an auditory range sound is not sounded. <P>SOLUTION: An inverted filter processing unit 10 obtains the transfer function H(k) of a reverberation space using the impulse response of a feedback path H<SB>AC</SB>in which a first echo canceler 30A identifies in an accommodative manner by an adaptive filter 31A, namely, a filter coefficient h^<SB>i</SB>(j). Further, a reverberation voice signal Z'(k) is divided by the amplitude ¾H(k)¾ of the transfer function H(k), and the original voice signal (sound source signal) Z"(k) is restored, and a reverberation component is removed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a reverberation removal method and apparatus used for clearly collecting sound in a reverberant place such as a bathroom.

  In recent years, loudspeaker devices that perform loudspeaker calls using microphones and speakers have become widespread in intercom systems, etc., but when this kind of loudspeaker device is installed in a place with reverberation (for example, in a bathroom, etc.) Since the reverberation of the person's voice is collected by the microphone, the voice may become unclear. On the other hand, various methods (reverberation removal methods) and reverberation removal devices for removing reverberation from sound collected by a microphone have been proposed.

For example, Non-Patent Document 1 proposes a method of removing only the minimum phase component of room transfer characteristics from a signal collected by a single microphone and recovering it. However, this method is effective only when the room sound field has a minimum phase characteristic. Further, in Non-Patent Document 2, when one or more microphones are arranged with respect to the number of sound sources and the zeros of the transfer characteristics between the sound sources and the microphones do not overlap, the system does not have the minimum phase characteristics. The sound field inverse filter theory that can accurately restore the sound source waveform itself has been proposed. In the method of realizing the inverse characteristics of these transfer characteristics with the inverse filter means, the inverse filter parameters (reverberation impulse response) must be measured in advance to determine the inverse filter. However, since the indoor transmission system fluctuates with time due to various fluctuations in the indoor environment, the transmission system must be measured and adaptively processed each time in order to maintain high recovery accuracy. Further, Non-Patent Document 3 proposes a method that does not require measurement of indoor transfer characteristics, focusing on the fact that reverberation characteristics affect not only spectral distortion but also the envelope of the signal waveform. This is realized as a power envelope inverse filter process for the purpose of recovering the power envelope, not the signal waveform itself, by modeling the sound source signal and the transmission system based on a modulation transfer function (MTF).
JP-A-9-321860 Stephen T. Neely and Jont B. Allen "Invertibility of a room impulse response", J.Acoust.Soc.Am.Vol.66, No.1, July 1979 Miyoshi, M. and Kaneda, Y., `` Inverse filtering of room acoustics, '' IEEE Trans.ASSP, Vol.36, No.2, pp.145-152, Feb. 1988 Shigeki Hirobayashi, Hiroaki Nomura, Mikio Higashiyama "Recovery of reverberant speech by inverse filtering of power envelope transfer function" IEICE Transactions A, Vol. J81-A, No. 10, pp. 1323-1330, 2000 Masakazu Furukawa, Yuji Kashiwagi, Masato Akagi, "Recovery method of reverberant voice power envelope based on MTF" IEICE Technical Report, EA2002-15, SP2002-15 (2002-04)

  However, in the power envelope recovery method disclosed in Non-Patent Document 3, the method for determining the parameters (amplitude and reverberation time) of the modeled room transfer characteristic is unclear, and the TSP method is a general parameter determination method. In an impulse response measurement method such as the M-sequence method, an audible range sound must be output from a speaker at the time of measurement, which limits the development of a practical application problem.

  On the other hand, in Non-Patent Document 4, based on the power envelope inverse filter processing of Non-Patent Document 3, (1) a method for extracting power envelopes, and (2) a method for determining parameters of an indoor impulse response (amplitude term and reverberation time) It proposes the improvement of the problems in principle. However, in the method disclosed in Non-Patent Document 4, the carrier signal of the audio signal is approximated by white noise, and the influence of the reverberant sound field that the carrier signal receives is not taken into consideration. There is a problem that it is not restored. In particular, when such a method is applied to an intercom telephone terminal installed in a bathroom, the visitor knows that the speaker is talking in the bathroom, which violates the privacy of the resident. was there.

  On the other hand, Patent Document 1 discloses a dereverberation apparatus and method applicable even when an audible area sound cannot be measured in advance or when the transfer function changes from moment to moment. This dereverberation apparatus has at least two microphones, and an inverse filter unit and a transfer function simulation filter unit corresponding to these two microphones, respectively. However, this method and apparatus will be applied to a loudspeaker device such as an interphone. Then, since it is necessary to add a microphone and a memory area for calculation and use a high-performance signal processing arithmetic unit, the price to provide to the user becomes high, which becomes a barrier to the spread to general households. There was a problem of being.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a reverberation removing method and apparatus capable of removing reverberant sound without sounding an audible area sound.

  In order to achieve the above object, the invention of claim 1 is a dereverberation method for removing a reverberation component from a reverberant speech signal collected by a microphone in a reverberation space and restoring an original sound source signal. A first step of adaptively identifying an impulse response of a feedback path formed by acoustic coupling between an existing speaker and a microphone by an adaptive filter and estimating an echo component of the feedback path from a reverberant speech signal collected by the microphone; The second step of subtracting the echo component estimated by the adaptive filter in the first step from the output signal of the feedback path, and the estimation error of the echo component estimation value included in the subtraction result in the second step is minimized. A third step of updating the filter coefficients of the adaptive filter so that the estimation error of the echo component estimation value in the third step The fourth step of substituting the filter coefficient when the signal becomes the minimum into the impulse response of the reverberant space to obtain the transfer function of the reverberant space from the filter coefficient, the transfer function of the reverberant space obtained in the fourth step and the microphone And a fifth step of obtaining an original audio signal from a calculation with the collected reverberant audio signal.

  In the invention of claim 2, in the invention of claim 1, in the fourth step, a transfer function in the frequency domain is obtained by Fourier transforming the filter coefficient, and in the fifth step, the reverberant speech signal is Fourier transformed. The inverse Fourier transform is performed after dividing by the magnitude of the transfer function in the frequency domain obtained in the fourth step.

  The invention of claim 3 is characterized in that the adaptive filter in the first step is an FIR type filter.

  The invention of claim 4 is characterized in that, in the invention of claim 3, in the third step, the filter coefficient of the adaptive filter is updated by a least mean square algorithm.

  According to a fifth aspect of the present invention, in the third aspect of the invention, in the third step, it is determined whether or not the reverberant speech signal includes speech, and the filter coefficient of the adaptive filter is only included when speech is included. It is characterized by updating.

  The invention of claim 6 is the adaptive filter according to the invention of claim 4, wherein in the third step, the instantaneous power ratio of the reverberant voice signal to the instantaneous power of the voice signal output from the speaker is larger than a predetermined threshold value. The step gain is set to a relatively small value.

  According to a seventh aspect of the present invention, in the third aspect of the present invention, in the third step, it is determined whether or not sound is included in both the signal collected by the microphone and the signal output from the speaker. Is characterized by not updating the filter coefficient of the adaptive filter.

  The invention of claim 8 is characterized in that, in the invention of claim 4, in the third step, the filter coefficient is initialized when the filter coefficient diverges.

  The invention of claim 9 is the feedback path in the invention of claim 7, even if the third step includes sound in both the signal collected by the microphone and the signal output from the speaker. When the value fluctuates, the filter coefficient is continuously updated.

  According to a tenth aspect of the present invention, in the first or second aspect of the present invention, in the fifth step, it is determined whether or not sound is included in both the signal collected by the microphone and the signal output from the speaker. A reverberant audio signal when no audio is included in at least one of the signal collected by the microphone and the signal output from the speaker, and the estimation error of the echo component estimation value is smaller than a predetermined threshold value Is set to zero.

  In order to achieve the above object, an eleventh aspect of the invention is a dereverberation apparatus that removes a reverberation component from a reverberant speech signal collected by a microphone in a reverberation space and restores the original sound source signal. An adaptive filter that adaptively identifies the impulse response of the feedback path formed by the acoustic coupling between the existing speaker and microphone and estimates the echo component of the feedback path from the reverberant speech signal collected by the microphone, and an estimation by the adaptive filter Filter coefficient update that subtracts the echo component from the output signal of the feedback path and updates the filter coefficient of the adaptive filter so that the estimation error of the echo component estimation value contained in the subtraction result by the subtraction means is minimized And the filter coefficient when the estimation error of the echo component estimated value is minimized in the filter coefficient updating means. Transfer function calculation means for obtaining the transfer function of the reverberation space from the filter coefficient in place of the impulse response of the reverberation space, and from the calculation of the reverberation space transfer function obtained by the transfer function calculation means and the reverberant speech signal collected by the microphone. Reverberation calculation means for obtaining an audio signal is provided.

  According to a twelfth aspect of the invention, in the invention of the eleventh aspect, the transfer function calculating means obtains a transfer function in the frequency domain by subjecting the filter coefficient to Fourier transform, and the reverberation calculating means performs Fourier transform on the reverberant speech signal and The inverse Fourier transform is performed after the reverberation signal is divided by the magnitude of the transfer function in the frequency domain.

  The invention of claim 13 is characterized in that, in the invention of claim 11 or 12, the adaptive filter is an FIR type filter.

  According to a fourteenth aspect of the present invention, in the thirteenth aspect, the filter coefficient updating means updates the filter coefficient of the adaptive filter by a least mean square algorithm.

  According to a fifteenth aspect of the present invention, in the fourth aspect of the present invention, the filter coefficient updating means determines whether or not the reverberant sound signal includes sound, and only when the sound is included, the filter coefficient of the adaptive filter. A voice / silence determination unit for updating

  According to a sixteenth aspect of the present invention, in the fourteenth aspect of the present invention, the filter coefficient updating means is an adaptive filter when the instantaneous power ratio of the reverberant voice signal to the instantaneous power of the voice signal output from the speaker is greater than a predetermined threshold value. A step gain switching unit for setting the step gain to a relatively small value.

  According to a seventeenth aspect of the present invention, in the fourteenth aspect, the filter coefficient updating means determines whether or not sound is included in both the signal collected by the microphone and the signal output from the speaker. And the filter coefficient of the adaptive filter is not updated when both are included in the determination unit.

  The invention of claim 18 is characterized in that, in the invention of claim 14, the filter coefficient updating means comprises a divergence detection unit for detecting the divergence of the filter coefficient and initializing the filter coefficient when the divergence is detected.

  According to a nineteenth aspect of the present invention, in the seventeenth aspect, the filter coefficient updating means includes a feedback path fluctuation detecting unit that detects a fluctuation of the feedback path, and it is determined by the determining unit that the sound is included in both. Even in this case, when the feedback path fluctuation detecting means detects the fluctuation of the feedback path, the filter coefficient is continuously updated.

  According to a twentieth aspect of the invention, in the eleventh or twelfth aspect of the invention, it is detected whether the output signal of the subtracting means and the signal output from the speaker include sound, and the estimation error of the echo component estimation value is predetermined. If at least one of the signals does not contain speech and the estimation error is smaller than the threshold, it is judged that the reverberant speech signal contains a nonlinear echo component. And non-linear echo suppression means for setting the reverberant speech signal to zero.

  According to the present invention, the impulse response of the feedback path formed by the acoustic coupling between the speaker and the microphone existing in the reverberation space is substituted with the filter coefficient of the adaptive filter, and the reverberation component is removed using the filter coefficient. Using only a single microphone, there is an effect that reverberant sound can be removed without sounding an audible area sound as in the conventional example.

  Hereinafter, an embodiment of a dereverberation apparatus that realizes the dereverberation method of the present invention will be described. In this embodiment, the case where the dereverberation device is installed in the loudspeaker device that is configured in the interphone system and installed in the bathroom is illustrated. However, the present invention is not limited to this, and the sound is heard using a microphone and a speaker. The dereverberation method and the dereverberation apparatus of the present invention can be applied to all loudspeakers that amplify sound.

  FIG. 2 shows a block diagram of an interphone master unit (hereinafter abbreviated as “master unit”) M as a loudspeaker device, and a door phone slave unit S as a counterpart call terminal. The base unit M includes a microphone 1, a speaker 2, a 2-wire to 4-wire conversion circuit 3, a microphone amplifier G1, a line output amplifier G2 for amplifying a transmission signal to a line (two-wire transmission line), and a reception signal from the line. Is composed of a line input amplifier G3, a speaker amplifier G4, a transmission volume adjustment amplifier G5, a reception volume adjustment amplifier G6, and first and second echo cancellers 30A and 30B. The doorphone slave unit S includes a microphone 1 ', a speaker 2', a two-wire / four-wire conversion circuit 3 ', a microphone amplifier G1', and a speaker amplifier G4 '.

The first echo canceller 30A is made adaptive filter 31A and a subtractor 32A, adaptively the impulse response of the feedback path (acoustic echo path) H _AC formed by the acoustic coupling between the speaker 2 microphone 1 by the adaptive filter 31A The subtractor 32A subtracts the echo component (acoustic echo) G ^ (j) identified and estimated from the reference signal (input signal to the speaker amplifier G4) X (j) from the output signal Y (j) of the microphone amplifier G1. Thus, the echo component is canceled and erased. The second echo canceller 30B also includes an adaptive filter 31B and a subtractor 32B. Reflection due to impedance mismatch between the two-wire / four-wire conversion circuit 3 and the transmission line, and the speaker 2′− in the doorphone slave unit S. The impulse response of the feedback path (line echo path) H _LIN formed by the acoustic coupling between the microphones 1 'is adaptively identified by the adaptive filter 31B, and the reference signal (input signal to the line output amplifier G2, that is, transmission) The echo component (line echo) estimated from the signal) is subtracted from the received signal by the subtractor 32B to cancel and cancel the echo component.

Thus, since the first and second echo cancelers 30A and 30B cancel the closed loop by canceling the echo components of the feedback paths H _AC and H _LIN , unpleasant echoes and howling can be suppressed. Further, a component other than an echo included in the output signal of the microphone amplifier G1, that is, a voice signal uttered by the caller to the base unit M and a noise around the base unit M do not give any loss, and the door phone slave unit is not lost. Similarly, components other than the echo included in the received signal, that is, the voice signal uttered by the caller to the doorphone slave unit S and the noise around the doorphone slave unit S are completely lost. Can be transmitted to the base unit M side without giving. Therefore, two-way simultaneous calls can be realized.

  Next, the dereverberation apparatus A which is the gist of the present invention will be described. As shown in FIG. 1, the dereverberation apparatus A according to the present embodiment transmits the microphone 1, the speaker 2, the first echo canceller 30A, and the first echo canceller 30A in the signal path on the transmission side of the base unit M and the speech. The inverse filter processing unit 10 is provided between the volume adjusting amplifier G5. However, the first echo canceler 30A and the inverse filter processing unit 10 are realized by controlling DSP (Digital Signal Processor) hardware with dedicated software, and convert an analog audio signal into a digital signal. An A / D converter 37 and a D / A converter 38 for converting a digital audio signal into an analog signal are provided.

  The adaptive filter 31A is an FIR type that is the most stable among various structures such as FIR type, IIR type, and lattice type, and that is resistant to changes in the characteristics of the input signal, and adaptively updates variable filter coefficients. As an algorithm for estimating the echo component of the feedback path (the wraparound component of the received signal via the feedback path), the least mean square (LMS) algorithm that minimizes the mean square value of the output signal of the subtractor 32A Is used.

The operation of the adaptive filter 31A will be described in more detail. In the LMS algorithm, the filter coefficient (also referred to as “tap weight”) h ^ _i (j) is recursively updated by the following equation.

h ^ _i (j + 1) = h ^ _i (j) + μE (j) · X (j−i)
However, i is a tap number and j is a sample time.

Here, E (j) is in a so-called single talk state in which speech is made only from the far end side (doorphone slave unit S) and no speech is made on the near end side (base unit M). If the echo component is G (j), E (j) = G (j) −G ^ (j), which represents the estimation error (instantaneous error) of the echo component G (j) at the sample time j, and μ is It represents a step gain (also referred to as a “step size parameter”) that is a constant for controlling the magnitude of the correction amount in repetition (that is, the speed of convergence). The estimated value G ^ (j) of the echo component G (j) is obtained by the following equation (1) from the filter coefficient h ^ _i (j) and the received signal X (j).

Here, I is the number of filter taps, and i is the tap number.

When the optimal solution that minimizes the mean square error of the estimation error E (j) is reached (converged) by recursively updating the filter coefficient h ^ _i (j), the filter coefficient of the optimal solution By subtracting the estimated value G ^ (j) of the echo component obtained from h ^ _i (j) from the transmission signal Y (j), an output signal E (j) in which the echo component is canceled is obtained.

Therefore, to break the closed loop by the first echo canceller 30A cancels the echo component of the acoustic side feedback path H _AC, it is possible to suppress an unpleasant echo. Further, a component other than an echo included in the output signal of the microphone amplifier G1, that is, a voice signal uttered by the caller to the base unit M and a noise around the base unit M do not give any loss, and the door phone slave unit is not lost. Similarly, components other than the echo included in the received signal, that is, the voice signal uttered by the caller to the doorphone slave unit S and the noise around the doorphone slave unit S are completely lost. Can be transmitted to the base unit M side without giving.

By the way, when the adaptive filters 31A and 31B of the echo cancellers 30A and 30B continue to update the filter coefficient h ^ _i (j) in a so-called double talk state in which the base unit M and the intercom unit S are simultaneously uttered, the filter The coefficient h ^ _i (j) may diverge without converging. For example, in the first echo canceller 30A, when the double talk component N (j) input from the microphone 1 exists, the transmission signal Y (j) is Y (j) = N (j) + G (j), and is estimated. The error E (j) is expressed as E (j) = N (j) + (G (j) −G ^ (j)). At this time, if the filter coefficient h ^ _i (j) is recursively updated to obtain an optimal solution that minimizes the mean square error of the estimation error E (j), the reference signal (received signal X (j) ) Includes a term of the double talk component N (j) that is not correlated with the estimation error E (j), the filter coefficient h ^ _i (j) may not converge and may diverge. That is, the double talk component N (j) becomes a disturbance component in the operation of the adaptive filter 31A.

Therefore, in the echo canceller 30A of the present embodiment, the adaptive filter 31A includes the voice / silence determination unit 13 that determines whether or not the speech signal is included in the input signal X (j) from the far end side, and double talk is performed. A double-talk detection unit 14 is provided for detection, and as will be described later, the sound coefficient / no-sound determination unit 13 determines that the voice is included, and the filter coefficient h ^ only when double-talk is not detected by the double-talk detection unit 14. _In addition to updating _i (j), the filter coefficient h ^ _i (j) is not updated but fixed to the previous value in other states to prevent divergence of the filter coefficient h ^ _i (j). ing.

Furthermore, in the echo canceller 30A of the present embodiment, when the instantaneous power ratio of the near-end side signal to the instantaneous power of the far-end side signal is larger than a predetermined threshold, the step gain μ in the adaptive filter 31A is relatively set. A step gain switching unit 15 for setting to a small value is provided, whereby it is determined whether or not the ratio is greater than a predetermined threshold, and if it is determined that the ratio is greater than the threshold, the adaptive filter 31A In order to set the step gain μ at a relatively small value, regardless of whether or not double talk, the speed of convergence of the filter coefficient h ^ _i (j) in the adaptive filter 31A if the ratio is larger than the threshold value. By relatively slowing down, it is possible to prevent and suppress divergence.

By the way, in the system in which the gain of the echo path frequently fluctuates by holding the hand in front of the microphone 1 or the speaker 2 or bringing the face close to the microphone 1 or the speaker 2 as in the base unit M of the interphone system, the double talk state and the echo path There is a possibility that the state in which the gain has fluctuated cannot be discriminated, and the filter coefficient h ^ _i (j) that should be updated may not be updated in accordance with the fluctuation in the gain of the echo path. Therefore, in the present embodiment, the acoustic echo path change detector 16 is provided to the echo canceller 30A, when detecting a variation in the acoustic echo path H _AC, so as to continue updating the filter coefficients h ^ _i (j) ing. For this reason, it is possible to quickly converge the filter coefficient h ^ _i (j), and it is possible to suppress only the echo component early and accurately.

In the present embodiment, the echo canceller 30A includes the acoustic echo divergence detection unit 17 that detects that the filter coefficient h ^ _i (j) diverges in the adaptive filter 31A. If divergence of h ^ _i (j) is detected, since it is difficult to converge as it again be continued updating of the filter coefficients h ^ _i (j), the initial filter coefficients h ^ _i (j) It has come to become. For this reason, the filter coefficient h ^ _i (j) can be quickly converged, and only the echo component can be suppressed quickly and accurately.

  By the way, the echo canceller 30A of the present embodiment includes a nonlinear echo suppression unit 18 in order to remove residual echo components that cannot be suppressed even by the various methods described above. As will be described later, the nonlinear echo suppressor 18 suppresses the residual echo only when the input signal (transmission signal) Y (j) from the near-end side does not include a voice signal to be transmitted. Thus, the stability of the call can be improved.

  Next, the dereverberation apparatus that is the gist of the present invention will be described. In general, when a sound source signal is x (t) (t is an index representing time) and an indoor impulse response is h (t), a reverberation signal (observation signal) is expressed by the following equation (2) (Non-patent Document 3). reference).

y (t) = x (t) * h (t) (2)
Note that * is an operator indicating a convolution operation.
Expression (2) is expressed by the following expression (3) in the frequency domain (see Non-Patent Document 3).

Y (k) = X (k) H (k) (3)
Here, k is an index representing the frequency domain.
Accordingly, the sound source signal X (k) is obtained from the following equation (4).

X (k) = Y (k) H ⁻¹ (k) (4)
Here, in order to estimate the sound source signal from the observation signal in the linear system, it is necessary to estimate the transmission system. However, since the transfer function H (k) is generally a time-varying system, adaptive estimation is required.

On the other hand, when the filter coefficient h ^ _i (j) is sufficiently converged in the echo canceller 30A, that is, when the optimum solution that minimizes the mean square error of the estimation error E (j) has been reached, the filter coefficients h ^ _i (j) is a good approximation of the impulse response of the acoustic side feedback path H _AC. If the room impulse response h (t) in the above equation (2) is substituted with the impulse response of the acoustic side feedback path H _AC , the transfer function H (k) is calculated using the filter coefficient h ^ _i (j). The sound source signal can be restored from the reverberation signal (reference signal), and the inverse filter processing unit 10 executes such calculation processing. That is, in this embodiment, the inverse filter processing unit 10 serves as a transfer function calculation unit and a reverberation calculation unit.

When the main unit M is installed in a reverberant place such as a bathroom, the sound signal Z (j) collected by the microphone 1 includes not only the sound source signal (the sound signal emitted by the caller) but also the reverberation signal. The reverberation component remains as it is in the audio signal Z ′ (j) that is included and the echo component only is suppressed by the first echo canceller 30A. Therefore, the reverberation component is removed by the inverse filter processing unit 10. Thus, an audio signal (sound source signal) Z ″ (j) that does not include a reverberation component is restored. Specifically, the inverse filter processing unit 10 performs the following five steps 1 to 5 arithmetic processing.
<Step 1: Step of fast Fourier transform of room impulse response h (t)>
A filter coefficient h ^ _i (j) substituting the indoor impulse response h (t) is acquired from the first echo canceller 30A, and this filter coefficient h ^ _i (j) is subjected to fast Fourier transform to transfer function H (k). Ask for. This transfer function H (k) is expressed in a complex form as shown in the following equation. However, A is a parameter for adjusting the amplitude, and i is an imaginary unit.

H (k) = A {h_real (k) + i · h_img (k)}
<Step 2: Step of fast Fourier transform calculation of reverberant speech signal Z ′ (j)>
A speech signal (reverberation speech signal) Z ′ (j) including a reverberation component output from the first echo canceller 30A is subjected to fast Fourier transform to obtain a reverberation speech signal Z ′ (k) in the frequency domain. This reverberant speech signal Z ′ (k) is also expressed in a complex form as shown in the following equation.

Z ′ (k) = z′_real (k) + i · z′_img (k)
<Step 3: Step of calculating magnitude | H (k) | of transfer function H (k)>
The magnitude | H (k) | of the transfer function H (k) is obtained by the following equation.

｜ H (k) | = {h_real ² (k) + h_img ² (k)} ^1/2
<Step 4: Step of performing calculation to recover sound source signal Z ″ (k)>
Using H (k), | H (k) |, Z ′ (k) obtained in Steps 1 to 3 respectively, a sound source signal, that is, a speech signal Z ″ (k) from which a reverberation component is removed is obtained. From (4)
Z ″ (k) = Z ′ (k) / H (k) = z ″ _real (k) + i · z ″ _img (k)
However,
z ″ _real (k) = {z′_real (k) · h_real (k) + z′_img (k) · h_img (k)} / | H (k) | ²
z ”_img (k) = {z′_img (k) · h_real (k) −z′_real (k) · h_img (k)} / | H (k) | ²
<Step 5: Inverse Fast Fourier Transform Calculation of Sound Source Signal Z ″ (k)>
The frequency domain sound source signal Z ″ (k) is subjected to inverse fast Fourier transform to obtain the time domain sound source signal Z ″ (j).

  Through the arithmetic processing in steps 1 to 5, a speech signal (sound source signal) Z ″ (j) obtained by removing the reverberation component from the reverberant speech signal Z ′ (j) is output from the inverse filter processing unit 10 to the far end side.

  Here, regarding the processing performed by the first echo canceller 30A and the inverse filter processing unit 10 when the base unit M makes a call with the other party's telephone device (such as the door phone slave unit S), refer to the flowcharts of FIG. 3 and FIG. Will be explained.

  For example, when the response button of the master unit M is operated in response to a call from the door phone slave unit S, a communication path is established between the master unit M and the door phone slave unit S, and the master unit M shifts to a call state. At the same time, the DSP executes a program (software) for realizing the echo cancellers 30A, 30B and the inverse filter processing unit 10.

As shown in FIG. 3, first, a variable initialization process (filter coefficient ＾ _i (0) = 0, step gain μ = μ _MAX , estimation error E (0) = 1) 39 is performed, and then Acquisition processing 40 and 41 of the input signal X (j + 1) on the far end side (door phone slave unit S side) and the input signal Y (j + 1) on the near end side (microphone 1 side) is performed, and the acquired input signal X (j + 1) ), Y (j + 1) are stored as the latest data in a FIFO type memory (not shown).

Next, a determination process (coefficient update determination process 42) of whether to update the filter coefficient ＾ _i (j), stop the update (do not update), or restart the process from the variable initialization process 39 is performed. In the filter coefficient update determination process 42, a divergence determination process, a sound / silence determination process, and a double talk determination process are performed as shown in the flowchart of FIG. In the divergence determination process by the acoustic echo divergence detection unit 17, first, the code of both is determined based on the product of the near-end side input signal Y (j) and the echo component estimated value G ^ (j). After the processing 51 for incrementing the count value divcount is performed only when the time is divergence, it is determined in the divergence determination time non-elapsed determination processing 52 whether or not the time for performing divergence detection determination (for example, 200 milliseconds) has elapsed. If it has not elapsed, the voice / silence determination processing is executed. If it has elapsed, the time count is initialized to 0, and the count value divcount, that is, the ratio of the different sign exceeds the divergence determination threshold div _slice. It is determined whether or not. Then, if the count value divcount exceeds the divergence determination threshold div _slice , it is determined as a divergence state, and after a process 54 for initializing the count value divcount to 0 is performed, a variable initialization process 39 is performed. On the other hand, if the count value divcount does not exceed the divergence determination threshold div _slice , it is determined as a non-divergence state, and after the process 56 for initializing the count value divcount to 0 is performed, the sound / silence determination unit 13 performs sound. / Silence determination processing is executed.

In the sound / silence determination processing, it is determined whether or not the absolute value average LX (j + 1) of the accumulated input signal X (j + 1) exceeds the sound / silence determination threshold LX _SLICE , and the absolute value average LX (j + 1) ) _Does not exceed the sound / silence determination threshold LX _SLICE , it is determined that there is a silence, and the update of the filter coefficient h ^ _i (j) is stopped. On the other hand, if the absolute value average LX (j + 1) exceeds the sound / silence determination threshold LX _SLICE , it is determined that the sound is present, and the double talk determination process is executed. Further, in the double talk determination process, it is determined whether or not the absolute value average LY (j + 1) of the accumulated input signal Y (j + 1) exceeds the double talk determination threshold LY _SLICE , and the absolute value average LY (j + 1) is doubled. If the talk determination threshold value LY _SLICE is _exceeded , it is determined that the state is a double talk state, and the update of the filter coefficient h ^ _i (j) is stopped. On the other hand, if the absolute value average LY (j + 1) does not exceed the double talk determination threshold LY _SLICE , it is determined that the double talk state is not established, and the filter coefficient h ^ _i (j) is updated in the adaptive filter 31A.

As shown in FIG. 3, when the filter coefficient h ^ _i (j) is updated, the step gain switching unit 43 executes the step gain switching process 43 to stop the update of the filter coefficient h ^ _i (j). In this case, the processing 44 ′ for substituting the previous filter coefficient h ^ _i (j) for the current filter coefficient h ^ _i (j) is performed, and then the calculation process 45 of the echo component estimated value G ^ (j + 1) is performed. .

In the step gain switching process 43, instantaneous values (X instantaneous value, Y instantaneous value) obtained by averaging the input signals X (j) and Y (j) from the latest to a predetermined time (for example, 2 milliseconds). (= X instantaneous value / Y instantaneous value) is compared with a predetermined threshold value α _slice, and if the ratio does not exceed the threshold value α _slice , the step gain μ used in the filter coefficient update processing 44 is set to the minimum value μ _MIN . If the instantaneous value ratio is equal to or greater than the threshold α _slice , the step gain μ is set to the maximum value μ _MAX to prevent the filter coefficient h ^ _i (j) from diverging.

In the filter coefficient updating process 44, the accumulated echo component estimation error E (j) and the input signal X (j) are acquired, and the filter coefficient ＾ _i (j) is updated for each tap number. Subsequently, the processing 45 for calculating the echo component estimated value G ^ (j + 1) is performed according to the equation (1), and then the echo component estimated value G ^ (j + 1) is subtracted from the input signal Y (j + 1). A process 46 for calculating the estimation error E (j + 1) is performed, and a nonlinear echo removal process 47 is further performed. In this nonlinear echo cancellation processing 47, the absolute value average LY (j + 1) of the input signal Y (j + 1) stored in the memory is smaller than the threshold LY _slice for determining single talk and double talk, that is, in a single talk state. If the estimated error E (j + 1) of the echo component is smaller than the clipping threshold Eclip, it is determined as a non-linear echo component, and the output signal (reverberant speech signal) Z ′ (j + 1) is set to 0 and removed. In other cases, the echo component estimation error E (j + 1) is directly used as the output signal Z ′ (j + 1).

In the inverse filter processing 48 by the inverse filter processing unit 10, the sound signal (sound source signal) Z ″ (j) obtained by removing the reverberation component from the reverberation sound signal Z ′ (j) in the five steps 1 to 5 as described above. After the processing 49 and 50 for outputting the restored voice signal Z ″ (j) to the signal path on the transmission side is performed, the input signals X (j + 1) and Y (j + 1) are acquired again. It returns to the process 40 and the above-mentioned process is repeated. Note that the filter coefficient 空間_i (j) used in the inverse filter process 48 is an impulse in a space (for example, a bathroom) that is a time-varying system by the coefficient update determination process 42, the step gain switching process 43, and the nonlinear echo removal process 47. Since the response can be approximated with high accuracy, the reverberation component can be removed with high accuracy.

Further, in this embodiment, it is determined whether or not the echo suppression amount in the first echo canceller 30A exceeds a predetermined reference value. If the echo suppression amount exceeds the reference value, the filter coefficient h _i (j) is Assuming that the impulse response h (j) is a good approximation, the filter coefficient h ^ _i (j) is output to the inverse filter processing unit 10, and the inverse filter processing unit 10 executes the above-described inverse filter processing 48 to generate reverberation. When the audio signal Z ″ (j) is obtained by removing the reverberation component from the audio signal Z ′ (j), and the reference value is not exceeded, the filter coefficient h ^ _i (j) is the room impulse response h (j). Is not output to the inverse filter processing unit 10 and the reverberant speech signal Z ′ (j) passes through the inverse filter processing unit 10 as it is and does not output the filter coefficient h ^ _i (j). This is output as (j).

For example, when the sampling frequency of the A / D converter 37 is 8 kHz and the impulse response length is 256 milliseconds, the number of filter taps I in the equation (1) is 2048, and the inverse filter processing unit 10 performs the inverse filter processing 48. The determination as to whether or not to perform is 20 log when the echo suppression amount is defined as the ratio E (j) / G ^ (j) between the echo component estimation error E (j) and the echo component estimated value G ^ (j). This is performed depending on whether the value of {E (j) / G ^ (j)} (echo suppression amount) exceeds the reference value −8 dB. That is, the first echo canceller 30A when the echo suppression amount is less than -8dB are sufficiently suppress acoustic echoes goes around the acoustic side feedback path H _AC, i.e. the filter coefficients h ^ _i (j ) Is sufficiently approximated to the indoor impulse response h (j) of the feedback path H _AC , and the filter coefficient h _i (j) is passed from the first echo canceller 30A to the inverse filter processing unit 10. When the inverse filter processing unit 10 performs the inverse filter processing 48 and the echo suppression amount is −8 dB or more, the acoustic echo is not sufficiently suppressed, that is, the filter coefficient h ^ _i (j) is the room impulse of the feedback path H _AC . it is determined that no similar response h a (j), the inverse filtering unit 10 to the inverse filtering unit 10 from the first echo canceller 30A to not passed the filter coefficients h ^ _i (j) is the inverse filtering 8 without reverberation sound signal Z '(j) is output as it is. Here, the waveform of the reverberant audio signal when a man utters the phrase “energy saving is up to you” in the bathroom (room dimensions: 2.0 m × 1.7 m × 2.2 m) The waveform of the audio signal after the reverberation component is removed by the filter processing unit 10 performing the inverse filter processing 48 is shown in FIG. 6, and 2048 filter coefficients ｈ _i (j when the first echo canceller 30A converges. ) Are shown in FIG. As is clear from comparison between FIG. 5 and FIG. 6, it can be seen that the reverberation component included in the audio signal is removed by the dereverberation apparatus A of the present embodiment, and the audio signal is easy to hear.

  Thus, when making a loudspeaking call between the main unit M installed in a bathroom with reverberation and the doorphone slave unit S installed at the entrance, the audio signal with the reverberation component in the bathroom is the parent signal. Since the sound is collected by the microphone 1 of the machine M, the reverberation component masks the sound source signal (caller's voice signal), making it difficult to hear the sound output from the speaker of the doorphone slave unit S. Since the dereverberation apparatus A according to the present invention is installed in the master unit M as described above, an audio signal from which the reverberation component is removed is transmitted from the master unit M to the door phone slave unit S. The voice output from the speaker is easy to hear and a comfortable telephone call environment can be realized. In addition, if the sound heard from the speaker of the doorphone slave unit S contains a reverberation component, it may be known that the other party is in the bathroom, and privacy may be infringed. If there is a problem of crime prevention, the other party's caller may know that it may be invaded into the house or may be damaged by theft, but the sound heard from the speaker of the doorphone slave unit S has a reverberation component. Otherwise, the other party's caller will not know that he is in the bathroom, so the privacy of the resident can be protected and crime prevention can be improved.

  Even in a loudspeaker system installed in a concert hall or auditorium, the speaker's voice may be masked by reverberation components reflected from the space, making it difficult for the listener to hear the content. If the dereverberation method and device are applied, the same effect as when the main unit M of the intercom system is installed in the bathroom is produced, and the reference sound based on the TSP method or M-sequence method is output in the concert hall or auditorium. It is possible to remove the reverberation component at least by inverse filtering.

It is a block diagram which shows embodiment of this invention. It is a block diagram of the door phone cordless handset which comprises a main phone carrying the same as above and an intercom system with the main phone. It is a flowchart for operation | movement description same as the above. It is a flowchart for operation | movement description same as the above. It is a wave form diagram of a reverberant voice signal. It is a wave form diagram of an audio signal from which a reverberation component was removed using the same as above. It is a figure which shows the filter coefficient in the same as the above.

Explanation of symbols

A dereverberation device 1 microphone 2 speaker 10 inverse filter processing unit 30A first echo canceller 31A adaptive filter 32A subtractor

Claims (20)

  A reverberation removal method for restoring an original sound source signal by removing a reverberation component from a reverberant speech signal collected by a microphone in a reverberation space, and a feedback path formed by acoustic coupling between a speaker and a microphone existing in the reverberation space The first step of estimating the echo component of the feedback path from the reverberant speech signal collected by the microphone by adaptively identifying the impulse response of the signal from the microphone, and the echo component estimated by the adaptive filter in the first step A second step of subtracting from the output signal of the feedback path, and a third step of updating the filter coefficient of the adaptive filter so that the estimation error of the echo component estimation value included in the subtraction result in the second step is minimized. And the filter coefficient when the estimation error of the echo component estimation value is minimized in the third step A fourth step for obtaining a transfer function of the reverberation space from the filter coefficient instead of the impulse response, and an original sound from the calculation of the transfer function of the reverberation space obtained in the fourth step and the reverberant sound signal collected by the microphone. And a fifth step of obtaining a signal.   In the fourth step, a transfer function in the frequency domain is obtained by Fourier transforming the filter coefficients, and in the fifth step, the reverberant speech signal is Fourier transformed and the magnitude of the transfer function in the frequency domain obtained in the fourth step is obtained. The dereverberation method according to claim 1, wherein an inverse Fourier transform is performed after dividing by the above.   3. The dereverberation method according to claim 1, wherein the adaptive filter in the first step is an FIR type filter.   4. The dereverberation method according to claim 3, wherein in the third step, the filter coefficient of the adaptive filter is updated by a least mean square algorithm.   5. The reverberation according to claim 4, wherein in the third step, it is determined whether or not speech is included in the reverberant speech signal, and the filter coefficient of the adaptive filter is updated only when speech is included. Removal method.   In the third step, when the instantaneous power ratio of the reverberant audio signal to the instantaneous power of the audio signal output from the speaker is larger than a predetermined threshold, the step gain in the adaptive filter is set to a relatively small value. 5. The dereverberation method according to claim 4, wherein   In the third step, it is determined whether or not both the signal collected by the microphone and the signal output from the speaker include sound, and if both include sound, the filter of the adaptive filter 5. The dereverberation method according to claim 4, wherein the coefficient is not updated.   5. The dereverberation method according to claim 4, wherein in the third step, the filter coefficient is initialized when the filter coefficient diverges.   In the third step, the filter coefficient is continuously updated when the feedback path fluctuates even when both the signal collected by the microphone and the signal output from the speaker include sound. The dereverberation method according to claim 7.   In the fifth step, it is determined whether or not both the signal collected by the microphone and the signal output from the speaker include sound, and the signal collected by the microphone and the signal output from the speaker are 3. The reverberant speech signal is set to zero when at least one of them does not include speech and the estimation error of the echo component estimated value is smaller than a predetermined threshold value. Reverberation removal method.   A reverberation removing device that removes a reverberation component from a reverberant speech signal collected by a microphone in a reverberation space and restores the original sound source signal, and is a feedback path formed by acoustic coupling between a speaker and a microphone that exist in the reverberation space An adaptive filter that adaptively identifies the impulse response of the signal and estimates the echo component of the feedback path from the reverberant speech signal collected by the microphone, and a subtractor that subtracts the echo component estimated by the adaptive filter from the output signal of the feedback path Filter coefficient updating means for updating the filter coefficient of the adaptive filter so that the estimation error of the echo component estimated value included in the subtraction result by the subtracting means is minimized, and the estimation error of the echo component estimated value in the filter coefficient updating means The filter coefficient at the time of the minimum is substituted for the impulse response in the reverberation space, and the residual is obtained from the filter coefficient. Transfer function calculating means for obtaining a transfer function in space, and reverberation calculating means for obtaining an original sound signal from a calculation of a reverberation space transfer function obtained by the transfer function calculating means and a reverberant sound signal collected by a microphone. A dereverberation device characterized by that.   The transfer function calculation means obtains a transfer function in the frequency domain by performing Fourier transform on the filter coefficient, and the reverberation calculation means performs Fourier transform on the reverberant speech signal and divides the reverberation signal by the size of the transfer function in the frequency domain. 12. The dereverberation apparatus according to claim 11, wherein an inverse Fourier transform is performed later.   13. The dereverberation apparatus according to claim 11 or 12, wherein the adaptive filter is an FIR type filter.   The dereverberation apparatus according to claim 13, wherein the filter coefficient updating means updates the filter coefficient of the adaptive filter by a least mean square algorithm.   The filter coefficient update means includes a sound / silence determination unit that determines whether or not sound is included in the reverberant sound signal and updates the filter coefficient of the adaptive filter only when the sound is included. The dereverberation apparatus according to claim 14, characterized in that:   The filter coefficient updating means sets the step gain in the adaptive filter to a relatively small value when the instantaneous power ratio of the reverberant audio signal to the instantaneous power of the audio signal output from the speaker is larger than a predetermined threshold value. The dereverberation apparatus according to claim 14, further comprising a switching unit.   The filter coefficient updating unit includes a determination unit that determines whether or not both the signal collected by the microphone and the signal output from the speaker include sound, and the determination unit includes sound in both. 15. The dereverberation apparatus according to claim 14, wherein the filter coefficient of the adaptive filter is not updated when it is present.   15. The dereverberation apparatus according to claim 14, wherein the filter coefficient updating means includes a divergence detection unit that detects divergence of the filter coefficient and initializes the filter coefficient when the divergence is detected.   The filter coefficient updating means includes a feedback path fluctuation detecting unit that detects fluctuations in the feedback path, and the feedback path fluctuation detecting means is used even if it is determined by the determining unit that the voice is included in both. 18. The dereverberation apparatus according to claim 17, wherein the update of the filter coefficient is continued when a change in the frequency is detected.   It detects whether or not the output signal of the subtracting means and the signal output from the speaker contain sound, compares the estimated error of the echo component estimated value with a predetermined threshold value, and outputs sound to at least one of the signals. Is included, and when the estimation error is smaller than the threshold, it is determined that the reverberant speech signal includes a non-linear echo component, and non-linear echo suppression means for setting the reverberant speech signal to zero is provided. The dereverberation apparatus according to claim 11 or 12, wherein the dereverberation apparatus is provided.

Images