JPH11168791A - Device and method for detecting sound source - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device for detecting a sound source composed of a microphone for receiving sound signals and a detection means for detecting sound from the received sound signals. SOLUTION: This device is provided with means 15 and 17 for discriminating the incoming direction of received signals, the means 17 for storing the estimated incoming direction of the sound of a certain specified source and a means 18 for comparing the incoming direction of the received signals with the estimated incoming direction. The device is further provided with the means 18 for displaying that the sound source is a certain specified source in the case of discriminating by comparison that the incoming direction of the received signals matched with the estimated incoming direction with certain allowable tolerance.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound source detecting method and a detecting device comprising a microphone means for receiving a sound signal and a detecting means for detecting sound in the received sound signal.

[0002]

BACKGROUND OF THE INVENTION Telephone conversations are often disturbed by echoes. This is especially true for four different types: idle, near-end, far-end, and double-talk (do).
This is a case of a full-duplex telephone having a call state of (uble-talk). Echo typically occurs when a call enters at the far end and the received far end signal is played back on the speaker and returned to the far end via a microphone. The echo problem is especially true when the speakers play loud sound to the surroundings,
This occurs in a hands-free calling method in which the sound from the speaker is easily returned to the microphone.

[0003] Adapted signal processing is employed to eliminate echo. In hands-free mobile phone applications, known echo cancellers and echo suppressors can be used to effectively remove the jammed acoustic feedback from the speaker to the microphone, ie, the acoustic echo. The echo canceller can be implemented using an adaptive digital filter that suppresses the echo signal from the output signal, that is, the signal that normally comes from the far end when the far end signal exists on the receiving side. In this way, efforts are made to prevent far end signals from returning to the far end. The parameters of the adaptive filter are usually updated whenever a far-end call occurs, in order to take into account the conditions of any situation as accurately as possible. For echo suppressors, it is used to attenuate the transmitted near-end signal.

A situation in which near-end and far-end calls occur simultaneously is called a double-talk situation. During double talk, the echo canceller cannot effectively remove the echo signal. This is because the echo signals are summed in the transmitted near-end signal, in which case the echo canceller cannot form an accurate model of the echo signal to be removed. In such a case, the adaptive filter of the echo canceller cannot properly adapt to the acoustic response of the space between the loudspeaker and the microphone, and therefore, if near-end speech signals are present, the acoustic filter Echo cannot be removed. For this reason, a double-talk detector is often used to remove the double-talk interference effect on the echo canceller. Typically, a double talk situation is detected by detecting whether a near end call is present at the same time as a far end call.
During double talk, the parameters of the adaptive filter of the echo canceller are not updated, but the update of the adaptive filter must be interrupted while the near end is talking.
The echo suppressor also needs information about the call activity of the near-end talker so that the signal transmitted while the near-end talks is not inappropriately (excessively) attenuated.

[0005] Interruptible transmissions used in GMS mobile phones, in addition to echo canceling and suppression,
Information on near-end call activity is needed. The concept of interruptible transmission is to send a call signal only during call activity, that is, while the near-end talker is dormant,
That is, no near-end signal is transmitted to save power. To avoid excessive fluctuations in the background noise level due to interruptible transmissions, it is possible to transmit some comfortable noise while idle and save the bits required during transmission. . Therefore, in order not to degrade the sound quality of the call sent by GSM interruptible transmission,
Near-end call activity must be accurately, quickly and reliably detected.

FIG. 1 shows a conventionally known configuration 1 for echo canceling and double talk detection. The near-end signal 3 arrives from the microphone 2 and is detected using the near-end speech activity detector 4 and VAD (voice activity detector). The far-end signal 5 arrives at the input connection I (which may be the input connector for hands-free devices, the wire connector for stationary phones, and the path from the antenna to the receiving branch of the phone for mobile phones), It is detected in the far-end speech activity detector 6 and VAD, and is finally reproduced by the speaker 7. The near-end signal 3 and the far-end signal 5 are both a double talk detector 8 for detecting double talk,
And an adaptive filter 9 for adapting to the acoustic response of the echo path 13. The adaptive filter 9 also receives the output of the double talk detector 8 as input, since it does not adapt to the filter during double talk (because the parameters are not updated). To perform echo cancellation, the model 10 formed by the adaptive filter is added to an adder / subtracter 11
Is subtracted from the near-end signal 3. (The output connector O may be an output connector for a hands-free device, a wire connector for a stationary telephone, and a path from a transmission branch to an antenna for a mobile telephone).
From which the echo (part of it) has already been cancelled. The echo canceller shown in FIG. 1 may be integrated into a telephone (e.g., comprising a speaker and a microphone for hands-free speaker calling) or may be implemented in a separate hands-free device.

Several methods have been proposed for detecting double talk. However, many of them are quite simple and some are unreliable. Most double talk detectors are based on the power ratio between the loudspeaker signal and / or the microphone signal and / or the signal after the echo canceller. The advantage of these detectors is simplicity and speed, the disadvantage of which is that they are not reliable.

[0008] Detectors based on correlating loudspeaker signals and / or microphone signals and / or signals after an echo canceller are also known. These detectors are based on the concept that the mere echo signal (the signal after passing through the echo canceller) in the speaker and the microphone is strongly correlated, but when the near-end signal is added to the microphone signal, the correlation is increased. Is reduced. The disadvantages of these detectors are that the detection speed is slow, that the near-end signal and the far-end signal are uncorrelated (partially incorrect), and that the loudspeaker signal due to the echo path is affected by changes. That is, the correlation decreases even when there is no near-end signal.

[0009] Also known are double talk detectors based on a comparison of the autocorrelation of the same signal, in which the detector can recognize speech in the near-end signal and thus detect the presence of the near-end signal. . Although the power required to calculate such a detector is small, the same problems occur as described above because the detector is based on correlation.

[0010] Kuo S. M., Pan Z. Author's book, "Microphone system for canceling acoustic echo for large-scale video conferencing,"ICSPAT'94 Bulletin, 1994.
Yearly, pp. 7-12, uses two microphones oriented in opposite directions to remove noise and acoustic echo and to recognize the different call situations mentioned at the outset. However, the above method does not provide any particular improvement in the recognition of double talk performed only by the output power of the echo canceller.

Affes S., Grenier Y.
Authored book "Microphone sound source subspace tracking array for double talk situations", ICSPA
T'96 Bulletin, Volume 2, 1996, 909-912
On the page, an echo and background noise canceller with a microphone vector structure is proposed. The proposed echo canceller filters signals arriving from spatially selected directions while retaining signals arriving from a desired direction. The above echo canceller can operate during a double talk situation. However, the above references do not present near-end speech activity detection using a multi-microphone solution (also referred to as microphone vector) nor do double-talk detection.

[0012]

SUMMARY OF THE INVENTION A method and apparatus for detecting near-end speech activity and recognizing a double talk situation has been invented.

[0013]

The present invention is based on the concept of detecting a near-end speech signal based on the direction of arrival of the signal. In hands-free applications where the speaker signal arrives from a direction that is distinctly different from the direction of the near-end talker's call signal, the near-end talk signal can be distinguished from the speaker signal based on its arrival angle. In the present invention, detection is performed using several microphones (microphone vectors) that pick up sound from different directions and / or different points.

The output of the microphone vector is first bandpass filtered into a narrowband signal, and the angle of arrival is estimated on a signal matrix formed by the filtered signals. The spatial spectrum is restored by this estimation, and tracking in the direction of arrival is performed based on the peaks generated in the spectrum from the spatial spectrum. The arrival direction of the near-end speech signal and the arrival direction of the speaker signal are updated based on the obtained (determined) arrival direction. These estimates of the direction of arrival make the final VAD determination easier. If the direction-of-arrival estimator detects a sufficiently strong spectral peak in the direction of arrival sufficiently close to the estimated direction of arrival of the near-end talk signal, the near-end talker is considered to be talking. That is, near-end call activity can be detected.

For the determination of double talk, information on the far-end speech activity is required in addition to the near-end speech activity. This information can be detected using a known voice activity detector, such as a power level based voice activity detector.

The apparatus according to the present invention comprises: means for determining the direction of arrival of a received signal; means for storing the estimated direction of arrival of the sound of a particular source; and the direction of arrival of the received signal and the estimated direction of arrival. Means for comparing the arrival direction of the received signal with the estimated arrival direction within the range of a certain tolerance by the comparison. Means for indicating that the event has occurred.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, the present invention will be described in detail with reference to the drawings.

FIG. 2 shows a block diagram of a detector according to the present invention for detecting near-end voice activity and recognizing double talk. In the present invention, several microphones 2a, 2b,. . . , 2M are used, and their microphones are preferably connected as a so-called microphone vector 2. The vector has at least two microphones, but preferably three or four or more. Each microphone has a single signal 3a, 3
b,. . . , 3M, and M microphones (M
Is an integer), M is variable in the time domain
Signals are obtained, which are variable in the time domain.
To form one signal vector of the elements.

The outputs 3a of the microphone vector 2
3b,. . . , 3M are first bandpass filtered in bandpass filter 14 to produce narrowband signals 19a, 19b,. . . , 19M. Since the accurate method of estimating the super-resolution spectrum only works on narrowband signals, bandpass filtering is performed to estimate the directional angle. Bandpass filtering can be achieved using, for example, fast Fourier transform (FFT), windowing and interleaving. The frequency range of the bandpass filter is determined based on the distance between each microphone in the microphone vector. Since, according to Nyquist sampling theory, the spatial sampling frequency must be at least twice the spatial frequency of the signal, the following is obtained as the band frequency (point frequency) of the bandpass filter 14: f = C / 2d, where C is The speed of sound in the air (343 m / s at 20 ° C.), and d is the distance between each microphone.

The filtered signals 19a, 19b,. . . ,
The estimation of the directional angle (i.e. the direction of arrival) on the signal matrix formed by the 19M is performed by the estimator 15 using some known estimating method such as, for example, MUSIC (Multiple Signal Classification). Done.

This estimation method restores the spatial spectrum,
The direction of arrival of the signal is determined based on the peaks generated in the spectrum therefrom. FIG. 3 shows an example of the spatial spectrum of such a microphone vector signal.
The arrival direction can be determined from the spectrum diagram shown in FIG. 3 by examining the derivative of the spectrum curve, for example. Such a zero of the derivative is restored as the direction of arrival. At zero, the derivative changes from positive to negative, which, as is known, indicates the position of each peak in the curve. Thus, in FIG. 3, two signals arrive at the microphone vector. That is, one from a direction of 10 ° or another from a direction of 40 °.
Further, it may be required that the spectral peaks considered to be in the direction of arrival have a certain minimum amplitude (eg, 5 dB). In the drawing, the effective range (coverage) of the spectrum is shown as 90 °. In practice, ± 90
° can be detected. Calculating the derivative,
Checking whether the minimum condition of the amplitude is satisfied should preferably be performed (by programming) using a digital signal processor. The estimation device 15 gives (estimates) the arrival direction 16 of the signal as its output.

The estimated directions of arrival of the near-end speech signal 3 and the loudspeaker signal 5 are updated in block 17 based on the obtained (determined) directions of arrival. The possible directions of arrival are evaluated by averaging the directions of arrival obtained from the spectral peaks. If it is almost known from which direction the signal has arrived, the effect of error peaks occurring in the spatial spectrum at that point can be minimized.
Unless an error peak occurs in the estimated direction of arrival, it is not noted. FIG. 4 shows the arrangement of the microphone 2 and the loudspeaker 7 of a conventional hands-free device in a car, the loudspeaker usually being in the direction of 0 ° ± 40 °, but rather just before the microphone vector 2. The position of the loudspeakers may vary significantly with respect to the microphone vector. The microphones 2a, 2b,. . . , 2M are arranged at a certain distance from each other in a certain direction. The distance and direction must be determined by the direction-of-arrival estimation algorithm described below. Hereinafter, the averaging of the arrival directions at both the far end and the near end performed in the signal source position / direction determination block 17 will be described in more detail.

The estimation of the far end arrival direction is performed based on the averaging of the arrival angle 16 obtained from the spectrum estimating device 15. Averaging occurs only when there is a call at the far end,
This is the far end VA whose output is sent to decision block 17.
The determination is made using the output of D6. The averaging is preferably performed in the time domain, for example using IIR-filtering. Two signal sources arriving from different directions, namely the near-end signal 3
It is basically assumed that the far-end signal 5 exists. It is further assumed that the direction of arrival of the signal changes relatively slowly compared to the frequency of observations performed. When the spectrum estimator 15 emits the arrival direction vector doa (in degrees) as its output, the estimated value fdoa (in degrees) of the far-end arrival direction vector is averaged so that each new direction estimate is the closest Updated to affect components of fdoa. The updating is weighted such that the closer the detected component is to the related component, the more the detected direction updates the fdoa component. The direction of the loudspeaker signal and thereby the reverberation (reve) induced in the spectrum
The direction of the reverberation signal changes very little, in which case the weighting reduces the effects of accidental, error peaks in the spectrum. At the same time, the probability of occurrence of the problem fdoa component, pdoa, is updated as the new value is closer to the direction estimation value. In addition, the strength of the relevant fdoa component, powder
a is updated based on the power of the corresponding spectral peak. In this case, the far-end arrival direction estimation vector fdoa has directions of arrival angles of (M-1) signals. The component pdoa is the probability of the corresponding arrival direction in the range [0, 1] and the range [0, 1] corresponding to the powder.
1].

Here, the arrival direction of the far-end signal 5 has the highest probability and strength corresponding thereto, while the component of the far-end arrival direction vector estimated value fdoa closest to the finally determined far-end signal arrival direction. Can be assumed. Since the estimate is updated only when the call is at the far end, it can be assumed that the near end signal 3 (in this case, double talk) occurs less than 50% of the time. Thus, it is basically assumed that double talk will occur in less than half of the far-end active time. The far-end signal arrival direction (the direction of the speaker) can be separated from the arrival direction of the reverberated speaker signal based on the power of the spectral peak corresponding to the arrival direction. Signals that arrive directly at the microphone produce stronger peaks in the spatial spectrum than signals that are normally attenuated in the reverberation path.

The following is an explanation of the algorithm for estimating the arrival direction with reference to FIG. In step 100, an initialization is performed which comprises determining: fdoa, pd
oa and powder contain (M-1) components. doa includes L components (1 ≦ L ≦ M−1). The fdoa component is initialized with different values. fdoa (n) = − 90 + n ^* 180 / M; (1 ≦ n ≦
M-1)

Step 101: Detected direction of arrival (do
The estimated value (each component of fdoa) is tracked as follows corresponding to a). Calculate the distance in each direction of arrival from each estimate. The shortest estimated value doa (i) and its corresponding closest estimated value fdoa (n) are selected.

Step 102: Update the estimated value fdoa (n) according to how close it is to the estimated value of the arrival direction doa (i). The closer, the more the detected direction changes the estimate. That is, fdoa (n) = α ₀ ^* fdoa
(N) + (1−α ₀ ) ^* doa (i), where α ₀ is, for example, a linear function or exponential function of the distance (see FIG. 5 for linear (linear) dependence). Upper limit and lower limit of update coefficient α ₀ and distance d, α ₀ _max, α ₀ min and d max, d Adjusting min can affect not only the speed of the update, but also at which distance the peak located affects the estimate. For example, the maximum value of the distance is kept at 40 ° (d mi
n = 0 °, d max = 40 °), when the maximum value of the update coefficient is kept at 1 (α ₀ min = 0.99, α ₀
max = 1.0), peak values of spectral errors more than 40 ° do not update the estimate and thus do not induce any errors. In this way, it is possible to remove the influence of the above-mentioned spurious signal on the estimated value.

Step 103: Increase the probability of occurrence of the estimated value, pdoa, again according to how close the arrival direction is to the estimated value. In the following, the function of distance is assumed to be a linear function. Other functions, such as an exponential function, are possible. pdoa (n) = α ₁ ^* pdoa (n) + (1−α ₁ )
(1-dist / 180) where α ₁ is, for example, 0.9, and dist is [0, 1
80] is the distance between the observed and estimated values.

Step 104: Estimated power pow using the power of the detected spectral peak as follows:
doa is also updated. powder (n) = α ₃ ^* powder (n) + (1
−α ₃ ) ^* Pow / Powmax where α ₃ is, for example, 0.9, Pow is the power of the spectrum peak, and Powmax is the maximum power.

In step 105, it is determined whether or not other arrival directions and estimated values can be found. If yes, steps 101-1 are performed on the remaining pairs of arrival directions and estimated values.
Repeat 04. Step 106: Reduce the frequency and power of estimation values for which no direction of arrival has been detected, for example by setting dist = 180 and Pow = 0.

Thereafter, at step 107, for example, by maximizing the following equation, the direction of the loudspeaker has the highest probability of occurrence and power, and is closest to the latest rating of the direction of the loudspeaker. Select the direction of the estimate. a ^* pdoa (k) + b ^* powdoa (k) + c ^* distance (k); K = 1. . . M-1, (1) where a, b and c are weighting factors, for example, 1/3, and distance (k) is an estimated value fdoa (k)
Is the distance in degrees between the direction of the loudspeaker evaluated so far (previously).

The estimation of the arrival direction of the far-end speech signal has been described above. Hereinafter, the estimation of the arrival direction of the near-end speech signal will be described. The estimation of the direction of arrival of the near-end signal is performed according to the procedure and algorithm described above, and thus the estimated value ndoa of the direction of arrival of the near end is obtained by replacing fdoa with ndoa in the above algorithm. The estimation is performed when the far-end speech activity detector 6 indicates that there is no call arriving from the far end. Estimation device 1
When detecting this spectrum within 5, there is no expected peak (direction of angle of arrival) or there are 1 to (M-1) peaks according to the direction of the near-end signal and / or spurious signal and reverberation. . As the direction of the near end talker, the direction indicated by the most frequently repeated and strongest spatial spectrum is selected as described above. In addition, the near-end talker is approximately 0 ° ± 3 ° with respect to the microphone vector.
It is assumed to be sitting in the 0 ° direction, in which case the initial value of the near-end talker's direction estimate can be set to 0 °, and the choice of direction has been previously (previously) rated. Directions can be strongly weighted.

These assumed arrival direction values fdoa,
The ndoa is incorporated in a detection block 18 that performs a final detection. When the arrival direction estimating device 15 detects a sufficiently strong spectrum peak in the arrival direction whose peak is sufficiently close to the assumed arrival direction of the near-end speech signal, it is determined that the near-end talker is talking. That is, near-end call activity is detected. This comparison is made at detector 18 based on the signals arriving from blocks 15 and 17. The final determination of near-end speech activity is made using estimation (averaging) of the spectral peak and direction of arrival. If any of the spectral peaks are closer to the near-end arrival direction estimate (or its reverberation estimate) than the far-end estimate, and closer to the near-end estimate than a predetermined error tolerance: The presence of a call at the near end is detected. The value of the tolerance is, for example, 10 degrees.

For the determination of double talk, information on the far-end speech activity is required in addition to the near-end speech activity. This information is transmitted from the far-end speech activity detector 6 to the double talk detector 1
8 and thus this detector 18 will detect a double talk situation when the near-end speech activity detector (described above) detects speech and the far-end speech activity detector 6 simultaneously detects speech. Is detected. As far as the far-end signal is concerned, any VAD algorithm may be used to detect call activity. The result of the double talk is a simple AND operation on the values of the near-end and far-end speech activity, namely 1 (during a call) and 0
(During a non-call).

The function of the transient state (transition) detector TD will be described below with reference to FIG. This detector is optional in the call activity detector / double talk detector according to the invention and is therefore shown in the drawing with a dotted line.
Since the estimation of the arrival direction is performed using a narrowband signal, it is difficult to detect a sudden change (transient state) of the near-end signal. Therefore, a parallel detector TD optimized for transient detection
It is possible to use After each transient position is detected, a direction-of-arrival detector is used to check the accuracy of the determination. The detector according to the invention is for example 20 m
If a sufficiently rapid signal change of less than s is detected, it is not necessary to use the transient detector TD.

In principle, a normal V
AD can be used. However, since a specific arrival direction angle can be attenuated by the multiple microphone structure, the transient state detector TD can be realized in such a manner that the estimated direction of the speaker signal is attenuated. In this case, the probability that the detected transient state is linked to the near-end signal increases. Attenuation in the speaker direction can be achieved in many different ways. The easiest way is to use two adaptive microphone structures. In principle, these two microphones are the microphones 2 of the microphone vector 2, for example the microphones furthest from each other.
a, 2b,. . . , 2M can be used. Two microphone signals are sufficient to achieve attenuation. If the adaptation is controlled using the determination of the direction-of-arrival estimation device (that is, the adaptation is performed only when there is only the far-end signal), attenuation in the desired direction can be obtained. (For example, 1KHz-2KHz
If detection is performed in a certain frequency range (such as
Adaptation becomes easier. In the transient detector, for example, F
An FT or bandpass filter can be used to perform frequency division directly on the signal obtained from the microphone.

The actual transient detector TD compares the instantaneous power P (n) of the signal at instant n with the noise estimate N (n). Here, P (n) is the power of the microphone signal (or the power of the microphone signal in which the direction of the loudspeaker signal is attenuated), the noise estimation value N (n) is averaged using the previous value, and the call is terminated. In the absence of any, it is the corresponding power controlled by the overall system decision. Information about moments when there is no call is block 18
To the transient state detector TD (dotted arrow). The associated values P (n) and N (n) can be calculated using a transient detector based on the signal arriving from the microphone. (Methods for calculating the signal output values P (n) and N (n) are known. For example, a transient state detector TD using an IIR (infinite impulse response) filter is used.
Can be run within If the difference is sufficiently large, it is determined that a transient state has been detected. Noise estimation value N
Iterative averaging to update (n), N (n + 1) = α
N (n) + (1-α) P (n) is used. Where α is a time constant for controlling averaging (generally about 0.9).

The transient detector complements the function of the spatial detector according to the invention. Although it is possible to detect the near-end call itself with the transient state detector TD, reliable detection can be obtained by the direction determination by the arrival direction estimating device 15. The detection of incorrect transients at the mere echo source (not the near-end signal) can be corrected by the direction at the direction-of-arrival estimator 15. If the directional attenuation works well enough, it is not necessary to be aware of transients induced by echoes during near-end calls. Near-end calls initiated during the echo can again be detected as distinct transients, and the results can be checked using a direction-of-arrival detector. The output of the transient detector TD is taken to block 18 (dotted line).

Near-end speech activity and double talk can also be determined by a statistical pattern recognition approach based on the output of direction of arrival estimator 15. According to this approach, speech activity detection based on DOA angle estimation could be improved using statistical information. Pattern recognition techniques such as neural networks and the Hidden Markov Model (HMM) have been successfully applied to many similar tasks. The strength of the pattern recognition method is that training is possible. Given a sufficient amount of training data, a model can be estimated for each state of the system (near-end talk, far-end talk, double talk, silence). These models can then be used to optimally detect the state of the system. It goes without saying that the detection process is optimal only if the modeling assumptions are correct.

Hereinafter, an embodiment in which the HMM is used to detect the activity of a multi-microphone call will be schematically described. Since the input to the system is still derived from the spatial spectrum, the DOA angle of the signal (s) is still a decisive factor according to the invention. In addition, the transient detection component (reference TD) described above can be used as before.

The first step in pattern recognition using HMMs is to define a model network.
As mentioned above, there are four states (models) in a full-duplex telephone system: near-end talk, far-end talk, double talk, and silence. Each model can be modeled with a multi-state HMM, but the starting point is a single-state HM
M can be used. Other potential improvements include using a minimum forced period on each state to prevent oscillations between the states.

Although in theory a transient can occur between any two models, in practice there are direct transients between silence and the double talk model, and between the near and far end models. Since the state can be neglected, the actual transient state is as shown in FIG.

Once the model structure has been defined, it must be decided what kind of probability distribution to use. The standard approach to speech recognition is the Gaussian probability density function (p
df) to model each state, which is also a preferred starting point in this embodiment. Instead, any pdf could be used. Model p
Training df is ideally done by estimating the parameters that are most likely to be trained from the labeled training data as shown in FIG. 9 (which allows us to know what state the system is in at a given moment). Done.
An alternative is to start with a certain basic model,
Unsupervised training (unsupervised tr)
There is a way to adapt the system online, called aing. Referring again to speech recognition, there are several online talker adaptation techniques that can be applied to this. In summary, using the current data, the condition that produces the highest probability is adapted with the highest weight. The more adaptive data, the more weight is added in the update.
The obvious problem with unsupervised training is that there is a risk of adapting the incorrect model in case of misclassification.
If the initial parameters can be estimated with a small number of monitoring training samples, the adaptation is likely to be better. Further, the far-end channel (speaker) is separated from the remaining channels, and this information is available. If there is far-end activity, only the far-end and double-talk models can be applied, and so on.

The actual detection (recognition) is very simple.
You only need to select the model with the highest probability at any time. Of course, additional information such as far-end speech activity can be used to further enhance detection performance. A logical refinement of this alternative approach is to use HMMs in some situations. For example, the HMM representing each system state is composed of three states. That is, a transient state to the model, a state representing a stationary part of the model, and a transient state deviating from the model. Gaussian pdfs could also be used together to increase the accuracy of the pdf modeling.

When using the detector according to the invention for in-car hands-free applications, the transient detector can be modified in such a way as to take into account the direction of the final reverberation of the signal. In such a case, the detection of transients can be improved by attenuating some estimated directions of arrival of the loudspeakers, rather than one of the directions of arrival of the loudspeakers.

An advantage of the spatial call activity detector according to the invention compared to the conventional method is that it can recognize both double talk situations and near-end call activity, its speed and reliability. The detector according to the invention based on the direction of arrival of the speech signal is very reliable due to its main features. The difference between the power levels of each speech signal does not significantly affect the result, but the detector also recognizes near-end speech signals that have significantly lower power than the speaker signal. In addition, the result of the decision is not affected by the operation of a separate device, such as the operation of an adaptive echo canceller. In the double talk detector, there are often threshold levels according to the speech signal and the surrounding noise level, and it is determined whether there is a double talk situation based on the threshold level.
The parameters of this detector are largely constant and therefore do not have the problems described above. The speed of recognition can be increased by using an optional transient detector.

In any case, in this hands-free facility,
A number of operations, such as detection of far-end speech activity and estimation of ambient noise, required for the spatial detector according to the invention are performed,
Thus, already performed calculation operations can be used in the detector according to the invention.

The detector according to the invention can be used in hands-free installations, for example in on-board kits for mobile phones or in hands-free installations for car phones (for example as part of an echo canceller and transmission logic). The invention is also suitable for use in so-called hands-free telephone applications where the hands-free equipment is included in the telephone.

FIG. 6 shows by way of example a mobile station according to the invention in which a spatial near-end speech / double-talk detector 80 according to the invention is used. The speech signal to be transmitted out of the microphone vector 2 is sampled in the A / D converter 20 and then processed in a base frequency signal (such as speech coding, channel coding, interleaving), Mixing and modulation to radio frequency and transmission to block TX are performed. Block T
From X, the signal is transmitted to the aerial path via the duplex filter DPLX and the antenna ANT. For example, to control the echo canceller or to transmit TX with intermittent transmission
The detector 80 can be utilized to control
On the receiving side, normal operations of the receiver branch RX such as demodulation, cancellation of interleaving, channel decoding, and voice decoding are performed, and thereafter, the far-end speech activity is detected by the detector 6, and the signal is D / A converted. The data is converted into an analog form by the device 23 and reproduced by the speaker 7. Blocks 2, 7, 20, 23 and 80 according to FIG. 6 are installed in a separate hands-free device with connections for mobile stations for input, output and control signals (30, 50, near end VAD, DT). By doing so, it is possible to carry out the present invention with a separate hands-free device. The present invention is further directed to a conference calling device with one or more microphones and speakers on a desk for conference calling, or for example,
The microphone and the speaker can be used by connecting to a computer for making a call via an Internet network, which can be built in a video display device, for example. Thus, the present invention is suitable for all kinds of hands-free type devices.

The implementation method and the embodiment of the present invention have been described using the embodiments. It will be obvious to those skilled in the art that the present invention is not limited to the details of the embodiments presented above, and that the present invention may be implemented in other embodiments without departing from its features. The disclosed embodiments are to be considered illustrative and not restrictive. Therefore, the possibilities of implementing and using the invention are limited only by the following claims. Thus, different embodiments of the invention, as defined by the claims, and equivalents, are within the scope of the invention.

[Brief description of the drawings]

FIG. 1 is a block diagram of a conventionally known echo canceller.

FIG. 2 is a block diagram of a detector according to the present invention.

FIG. 3 is a spatial spectrum diagram of a microphone vector signal.

FIG. 4 is an installation diagram of a microphone and a speaker in an automobile.

FIG. 5 shows update coefficients used to estimate (in degrees) the direction of arrival as a function of distance.

FIG. 6 shows a mobile station according to the invention.

FIG. 7 shows the estimation of the direction of arrival in the form of a flowchart.

FIG. 8 is a diagram illustrating transition states between different states in an alternative embodiment.

FIG. 9 shows labeled training data.

[Explanation of symbols]

 2 microphone vector 3 near-end signal 5 far-end signal 6 far-end speech activity detector 7 speaker 14 band-pass filter 15 arrival direction estimating device 17 arrival direction determination block 18 double-talk detector TD Transient state detector

Claims (10)

[Claims] An apparatus for detecting an audio source, comprising: microphone means (2; 2a, 2b, 2M) for receiving an audio signal; and means for detecting audio from the received audio signal. Means for determining the direction of arrival (15, 17); means for storing the estimated direction of arrival of a particular sound source; and means for determining the direction of arrival of the received signal and the estimated direction of arrival. Means (18) for comparing, if the comparison shows that the direction of arrival of the received signal coincides with the estimated direction of arrival within a certain tolerance, the source of the audio is present A device for detecting a voice source, comprising: means (18) for indicating that the source is a specific source. 2. The microphone means (2; 2a, 2)
b, 2M) is M microphones (2a, 2b)
Wherein M is an integer, wherein the microphone is arranged to generate M microphone signals as output, the device forms a spatial spectrum based on the microphone signals, and Apparatus according to claim 1, comprising means (15) for determining the direction of arrival in the spectrum based on peaks occurring in the spectrum. 3. A means (15) for calculating the derivative of the spectral curve and determining the arrival direction by restoring the zero point of the derivative in which the derivative changes from positive to negative. 3. The device according to claim 2, wherein: 4. The microphone means (2; 2a, 2)
b, 2M), comprising means for reproducing sound in a certain direction (7), said means (17) for storing an estimated direction of arrival of a source, wherein said first source is a device. Is configured to store the estimated direction of arrival of the sound from at least two different sources when the second source is the sound reproducing means (7), in which case the first Is the estimated direction of arrival of the sound from the first source, the second estimated direction of arrival is the estimated direction of arrival of the sound from the second source, and The means (18) for detecting a sound source may be configured to determine that a direction of arrival of the received signal is closer to the first estimated direction of arrival than to the second estimated direction of arrival as a result of the comparison. The user of the device is the audio source Apparatus according to claim 2, characterized in that it is configured to display. 5. The apparatus of claim 4, wherein the means for detecting an audio source is configured to indicate a situation in which audio is simultaneously received from the first source and the second source.
An apparatus according to claim 1. 6. A means for two-way voice transmission, wherein voice from said first source is a near-end call to be transmitted and voice from said second source is voice. Apparatus according to claim 4 or 5, characterized in that the received far-end call is arranged to be reproduced using a reproduction means (7). 7. A method for receiving an audio signal and detecting an audio source wherein the audio is detected from the received audio signal, wherein a direction of arrival of the received signal is determined, and audio from a particular source is determined. The estimated direction of arrival of
The arrival direction of the received signal is compared with the estimated arrival direction, and as a result of the comparison, the arrival direction of the received signal matches the estimated arrival direction within a certain tolerance. A method of detecting an audio source, wherein if found, the audio source is displayed as being the particular source. 8. An audio signal comprising M microphones (2
a, 2b), where M is an integer, where M microphone signals are provided as outputs of each microphone, a spatial spectrum in the direction of arrival is generated based on the microphone signals, and within the spectrum The method of claim 7, wherein the direction of arrival is determined from the spectrum based on a resulting peak. 9. The arrival direction is determined by calculating a derivative of the spectral curve and restoring a zero point of the derivative where the derivative changes from positive to negative. 7. The method according to 7. 10. A parameter (fdoa) that describes the direction of arrival of each of the peaks in the spatial spectrum of the source and a parameter (pdo) that describes the probability that the direction of arrival will appear.
a) and a parameter (powdoa), wherein the strength of the sound from the source and the direction of arrival of the source are determined by averaging each parameter (fdoa, pdoa, powdoa) individually. 9. The method of claim 8, wherein the method comprises: