CN112904279A - Sound source positioning method based on convolutional neural network and sub-band SRP-PHAT space spectrum - Google Patents
Sound source positioning method based on convolutional neural network and sub-band SRP-PHAT space spectrum Download PDFInfo
- Publication number
- CN112904279A CN112904279A CN202110059164.1A CN202110059164A CN112904279A CN 112904279 A CN112904279 A CN 112904279A CN 202110059164 A CN202110059164 A CN 202110059164A CN 112904279 A CN112904279 A CN 112904279A
- Authority
- CN
- China
- Prior art keywords
- srp
- phat
- sub
- band
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000001228 spectrum Methods 0.000 title claims abstract description 37
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 59
- 239000011159 matrix material Substances 0.000 claims abstract description 20
- 230000006870 function Effects 0.000 claims description 21
- 238000011176 pooling Methods 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 8
- 230000004807 localization Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000005316 response function Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000009432 framing Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 230000001902 propagating effect Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 abstract description 36
- 230000008569 process Effects 0.000 abstract description 6
- 230000000694 effects Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000037433 frameshift Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 241001333991 Rugopharynx tau Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000021615 conjugation Effects 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008909 emotion recognition Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/18—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
- G01S5/22—Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
- G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/45—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of analysis window
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L2021/02082—Noise filtering the noise being echo, reverberation of the speech
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
- G10L2021/02166—Microphone arrays; Beamforming
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Theoretical Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
The invention discloses a sound source positioning method based on a convolutional neural network and a sub-band SRP-PHAT space spectrum, which comprises the following steps: the microphone array collects voice signals, and carries out frame division and windowing pretreatment on the collected voice signals to obtain single-frame signals; calculating a sub-band SRP-PHAT spatial spectrum matrix of each frame signal; and inputting the sub-band SRP-PHAT space spectrum matrixes of all the frame signals into the trained convolutional neural network, outputting the probability that the voice signal belongs to each azimuth angle, and taking the azimuth angle with the highest probability as the sound source azimuth angle estimation value of the voice signal. The invention can improve the sound source positioning performance of the microphone array in a complex acoustic environment and improve the generalization ability to the sound source space structure, reverberation and noise; the training process of the convolutional neural network can be completed off line, the trained convolutional neural network is stored in the memory, and real-time sound source positioning can be realized only by one frame of signal during testing.
Description
Technical Field
The invention belongs to the field of sound source positioning, and particularly relates to a sound source positioning method based on a convolutional neural network and a sub-band SRP-PHAT spatial spectrum.
Background
The sound source positioning technology based on the microphone array has wide application prospect and potential economic value in the front-end processing of voice recognition, speaker recognition and emotion recognition systems, video conferences, intelligent robots, intelligent homes, intelligent vehicle-mounted equipment, hearing aids and the like. The SRP-PHAT (stepped Response Power-Phase Transform) method, which is the most popular and commonly used method among conventional sound source localization methods, achieves sound source localization by detecting the peak of the spatial spectrum, but noise and reverberation often cause the spatial spectrum to exhibit a multi-peak characteristic, and especially in a strong reverberation environment, the spatial spectrum peak generated by reflected sound may be larger than the peak of direct sound, resulting in an error in sound source position detection. In recent years, model-based sound source positioning methods are used for positioning in complex acoustic environments, and such methods implement sound source positioning by modeling spatial characteristic parameters and constructing a mapping relationship between a sound source position and the spatial characteristic parameters, but at present, the generalization capability of such algorithms to unknown environments (noise and reverberation) is low, and the performance needs to be further improved. The spatial characteristic parameters and the modeling method are main factors influencing the performance of the model-based sound source positioning method.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the problems in the prior art, the invention discloses a sound source positioning method based on a Convolutional Neural Network and a sub-band SRP-PHAT space spectrum.
The technical scheme is as follows: in order to achieve the purpose, the invention adopts the following technical scheme: a sound source positioning method based on a convolutional neural network and a sub-band SRP-PHAT space spectrum is characterized by comprising the following steps:
s1, the microphone array collects voice signals, and the collected voice signals are subjected to frame division and windowing preprocessing to obtain single-frame signals;
s2, calculating a sub-band SRP-PHAT spatial spectrum matrix of each frame signal;
s3, inputting the sub-band SRP-PHAT space spectrum matrixes of all frame signals into the trained convolutional neural network, outputting the probability that the voice signal belongs to each azimuth angle, and taking the azimuth angle with the maximum probability as the sound source azimuth angle estimation value of the voice signal.
Preferably, in step S2, the calculating the sub-band SRP-PHAT spatial spectrum matrix of each frame signal includes the following steps:
s21, performing discrete Fourier transform on each frame of signal:
wherein x ism(i, n) is the ith frame signal of the mth microphone in the microphone array, M is 1,2, …, M is the number of microphones, Xm(i, k) is xm(i, N) indicating a frequency domain signal of an ith frame of the mth microphone, K being a frequency point, K being a length of discrete fourier transform, N being a frame length, K being 2N, and DFT (·) indicating discrete fourier transform;
s22, designing an impulse response function of the Gamma filter bank:
wherein j represents the serial number of the Gamma atom filter; c is the gain of the Gamma filter; t is tRepresenting a continuous time; a is the order of the Gamma-tone filter;
represents the phase; f. ofjRepresents the center frequency of the j Gamma filter; bjRepresents the attenuation factor of the j Gamma filter, bjThe calculation formula is as follows:
bj＝1.109ERB(fj)
ERB(fj)＝24.7(4.37fj/1000+1)
discrete fourier transform of the impulse response function for each gamma filter:
wherein G isj(k) Is the frequency domain expression of the j Gamma filter, K is the frequency point, K is the length of discrete Fourier transform, N is the frame length, K is 2N, fsRepresents the signal sampling rate, DFT (-) represents the discrete fourier transform;
s23, calculating a sub-band SRP-PHAT function of each frame signal:
wherein, P (i, j, r) represents the j sub-band SRP-PHAT function of the i frame signal when the beam direction is r; m is the number of microphones in the microphone array; tau ismn(r) represents a time difference of the sound wave propagating from the beam direction r to the m-th microphone and the n-th microphone, and is calculated by the formula:
where r denotes the coordinates of the beam direction, rmDenotes the position coordinate of the m-th microphone, rnPosition coordinates of the nth microphone, and c is the speed of sound in air；
S24, carrying out normalization processing on the sub-band SRP-PHAT function of each frame signal:
s25, combining all sub-band SRP-PHAT functions of the same frame signal into a matrix form to obtain a sub-band SRP-PHAT spatial spectrum matrix:
wherein, y (i) represents the sub-band SRP-PHAT spatial spectrum matrix of the signal of the ith frame, J is the number of the sub-bands, namely the number of Gamma filters, and L is the number of beam directions.
Preferably, in step S23, when the sound source is set to be at the same level as the microphone array and the sound source is located in the far field of the microphone array, τ ismnThe equivalent calculation formula of (r) is:
where xi is [ cos θ, sin θ ═]TAnd θ is the azimuth of the beam direction r.
Preferably, the convolutional neural network comprises an input layer, three convolutional-pooling layers, a full-link layer and an output layer which are connected in sequence;
in the convolution-pooling layer, each convolution layer adopts convolution kernels with the size of 3 multiplied by 3, the step length is 1, the number of the convolution kernels of the three convolution layers is 24, 48 and 96 in sequence, after each convolution layer is subjected to convolution operation, batch normalization is firstly carried out, then ReLU function activation is used, and the characteristic dimensions before and after convolution are kept unchanged by adopting a zero filling mode in the convolution operation; the pooling layer adopts a maximum pooling mode, the size of the pooling is 2 multiplied by 2, and the step length is 2;
after convolution-pooling, straightening and deforming the characteristic data into one-dimensional vector characteristic data;
adding a Dropout connection mode into the connection of the full connection layer and the one-dimensional vector characteristic data;
the output layer uses a Softmax classifier.
Preferably, the training step of the convolutional neural network is as follows:
s1, convolving the clean voice signal with the room impulse response of different azimuth angles, and adding different degrees of noise and reverberation to generate a plurality of directional voice signals of different specified azimuth angles:
xm(t)＝hm(t)*s(t)+vm(t),m＝1,2,...,M
wherein x ism(t) a directional speech signal representing a specified azimuth received by an mth microphone of the array of microphones; m is the serial number of the microphone, M is 1,2, …, and M is the number of the microphones; s (t) is a clean speech signal; h ism(t) represents the room impulse response from the specified azimuth to the mth microphone; v. ofm(t) represents noise;
s2, preprocessing all directional voice signals by framing and windowing to obtain single-frame signals, and calculating a sub-band SRP-PHAT spatial spectrum matrix of each frame of signals;
s3, taking the sub-band SRP-PHAT space spectrum matrixes of all the directional voice signals as training samples, taking the designated azimuth angles of all the directional voice signals as class labels of the corresponding training samples, taking the training samples and the class labels as training data sets, and training the convolutional neural network by adopting a random gradient descent algorithm with momentum to minimize a loss function.
Has the advantages that: the invention has the following remarkable beneficial effects:
1. the invention can improve the sound source positioning performance of the microphone array in a complex acoustic environment and improve the generalization ability to the sound source space structure, reverberation and noise;
2. the invention adopts the sub-band SRP-PHAT space spectrum as the space characteristic parameter, and the parameter not only can represent the whole acoustic environment information, but also has the advantage of strong robustness; modeling spatial characteristic parameters of directional voice data under various reverberation and noise environments by adopting a convolutional neural network, establishing a mapping relation between an azimuth and the spatial characteristic parameters, and converting a sound source positioning problem into a multi-classification problem;
3. the invention can complete the training process of the convolutional neural network off line, and the trained convolutional neural network is stored in the memory, and the real-time sound source positioning can be realized only by one frame of signal during testing.
Drawings
FIG. 1 is a flow chart of the algorithm of the present invention;
FIG. 2 is a diagram of a model architecture of a convolutional neural network of the present invention;
FIG. 3 is a graph comparing the success rate of positioning between the method of the present invention and the conventional SRP-PHAT algorithm when the testing environment and the training environment are consistent and the reverberation time is 0.5 s;
FIG. 4 is a graph comparing the success rate of positioning between the method of the present invention and the conventional SRP-PHAT algorithm when the testing environment and the training environment are consistent and the reverberation time is 0.8 s;
FIG. 5 is a comparison graph of the positioning success rate of the method of the present invention and the conventional SRP-PHAT algorithm when the noise environments of the test environment and the training environment are not consistent and the reverberation time is 0.5 s;
FIG. 6 is a comparison graph of the positioning success rate of the method of the present invention and the conventional SRP-PHAT algorithm when the noise environments of the test environment and the training environment are not consistent and the reverberation time is 0.8 s;
FIG. 7 is a comparison graph of the positioning success rate of the method of the present invention and the conventional SRP-PHAT algorithm when the reverberation environments of the test environment and the training environment are not consistent and the reverberation time of the test environment is 0.6 s;
fig. 8 is a comparison graph of the positioning success rate of the method of the present invention and the conventional SRP-PHAT algorithm when the reverberation environments of the test environment and the training environment are not consistent and the reverberation time of the test environment is 0.9 s.
Detailed Description
The present invention will be further described with reference to the accompanying drawings.
The sub-band SRP-PHAT space spectrum represents the whole acoustic environment space information, including sound source direction, room size, room reflection characteristics and the like, has strong robustness and can be used as space characteristic parameters in a positioning system. The deep neural network can simulate the information processing mode of a nervous system, can describe the fusion relation and structural information among the spatial characteristic parameters, has strong expression and modeling capabilities, and meanwhile, does not need to assume the distribution of data during modeling. The convolutional neural network is a neural network specially used for processing data with a similar network structure, and is applied to image or time series data. The speech signal collected by the microphone array is just a kind of time series data.
Therefore, the present invention provides a sound source localization method based on a convolutional neural network and a subband SRP-phot spatial spectrum, as shown in fig. 1, comprising the following steps:
the method comprises the following steps: convolving the clean speech signal with the room impulse response at different azimuth angles, and adding different degrees of noise and reverberation to generate a plurality of directional speech signals at different specified azimuth angles, namely microphone array signals:
xm(t)＝hm(t)*s(t)+vm(t),m＝1,2,...,M
wherein x ism(t) a directional speech signal representing a specified azimuth received by an mth microphone of the array of microphones; m is the serial number of the microphone, M is 1,2, …, and M is the number of the microphones; s (t) is a clean speech signal; h ism(t) represents the room impulse response from the specified azimuth to the mth microphone, hm(t) is related to sound source orientation, room reverberation; v. ofm(t) represents noise.
In this embodiment, the microphone array is a uniform circular array composed of 6 omnidirectional microphones, and the array radius is 0.1 m. And setting the sound source and the microphone array to be in the same horizontal plane, wherein the sound source is positioned in the far field of the microphone array. It is defined that the right front of the horizontal plane is 90 °, the azimuth angle of the sound source is in the range of [0 °,360 °, the interval is 10 °, the number of training azimuths is marked as F, and then F equals 36. The reverberation time of the training data comprises 0.5s and 0.8s, and the Image algorithm generates room impulse responses h of different azimuth angles under different reverberation timesm(t)。vm(t) is white Gaussian noise, and the signal-to-noise ratios of the training data include 0dB, 5dB, 10dB, 15dB, and 20 dB.
And step two, preprocessing the microphone array signal obtained in the step one to obtain a single-frame signal.
The pre-processing includes framing and windowing, wherein:
the framing method comprises the following steps: using the preset frame length and frame shift to convert the directional voice signal x of the m-th microphone with the appointed azimuth anglem(t) division into a plurality of single-frame signals xm(iN + N), wherein i is a frame number, N represents a sampling number iN one frame, N is more than or equal to 0 and less than N, and N is a framing length. Signal sampling rate f in this embodimentsAt 16kHz, the length of the frame taken, N, is 512 (i.e., 32ms) and the frame shift is 0.
The windowing method comprises the following steps: x is the number ofm(i,n)＝wH(n)xm(iN + n) where xm(i, n) is the ith frame signal of the mth microphone after the windowing processing,
is a hamming window.
And step three, extracting spatial characteristic parameters of microphone array signals, namely a sub-band SRP-PHAT spatial spectrum matrix. The method specifically comprises the following steps:
and (3-1) carrying out discrete Fourier transform on each frame of signal obtained in the step two, and converting the time domain signal into a frequency domain signal.
The discrete fourier transform calculation formula is:
wherein, Xm(i, k) is xmThe discrete fourier transform of (i, N) represents the frequency domain signal of the ith frame of the mth microphone, K is a frequency point, K is the length of the discrete fourier transform, K is 2N, and DFT (·) represents the discrete fourier transform. In this embodiment, the length of the discrete fourier transform is set to 1024.
And (3-2) designing a Gamma atom filter bank.
gj(t) is the impulse response function of the j-th Gamma filter, and the expression is:
wherein j represents the serial number of the Gamma atom filter; c is the gain of the Gamma filter; t represents a continuous time; a is the order of the Gamma-tone filter;
represents the phase; f. ofjRepresents the center frequency of the j Gamma filter; bjRepresents the attenuation factor of the j Gamma filter, bjThe calculation formula is as follows:
bj＝1.109ERB(fj)
ERB(fj)＝24.7(4.37fj/1000+1)
in this embodiment, the order a is 4, and the phase
Set to 0, the number of gamma filters is 36, i.e. j is 1,2, …,36, the central frequency f of the gamma filtersjIn the range of [200Hz, 8000Hz]。
Performing discrete Fourier transform on the impulse response function of each Gamma filter to obtain a frequency domain expression:
wherein G isj(k) Is gj(n/fs) Represents the frequency domain expression of the j-th Gamma transform filter, K is the frequency point, K is the length of the discrete Fourier transform, K is 2N, DFT ((-) represents the discrete Fourier transform, fsRepresenting the sampling rate. In this embodiment, the length of the discrete fourier transform is set to 1024.
(3-3) calculating a sub-band SRP-PHAT function of each frame signal, wherein the calculation formula is as follows:
wherein, P (i, j, r) represents the j sub-band SRP-PHAT function of the i frame signal when the beam direction of the array is r; (.)*Represents a conjugation; tau ismn(r) represents a time difference of the sound wave propagating from the beam direction r to the m-th microphone and the n-th microphone, and is calculated by the formula:
where r denotes the coordinates of the beam direction, rmDenotes the position coordinate of the m-th microphone, rnC is the sound velocity in the air, and is about 342m/s at normal temperature, fsIs the signal sampling rate; i | · | | represents a 2 norm.
In this embodiment, if the sound source and the microphone array are set to be in the same horizontal plane, and the sound source is located in the far field of the microphone array, then τ ismnThe equivalent calculation formula of (r) is:
where xi is [ cos θ, sin θ ═]TAnd θ is the azimuth of the beam direction r. Tau ismn(r) is independent of the received signal and can therefore be calculated off-line and stored in memory.
The normalization processing is carried out on the sub-band SRP-PHAT function P (i, j, r), and the calculation formula is as follows:
(3-4) combining all sub-bands SRP-PHAT functions of the same frame signal
Combining the two into a matrix form to obtain a sub-band SRP-PHAT spatial spectrum matrix:
where y (i) represents the spatial characteristic parameter of the ith frame signal, i.e., the subband SRP-PHAT spatial spectrum matrix, J is the number of subbands, i.e., the number of gamma filters, and J is 36 in this embodiment. The number of beam directions L is 72 because the azimuth range of the beam directions of the array is [0 ° and 360 °) defined as 90 ° directly in front of the horizontal plane and the interval is 5 ° in the present embodiment. Generally, the number L of beam directions is greater than the number F of training azimuths, so that the accuracy of the spatial characteristic parameters of the signals can be improved, and the training accuracy of the CNN model can be improved.
Step four, preparing a training set: according to the first to third steps, extracting spatial characteristic parameters of the directional voice signals under all training environments (the implementation setting of the training environments is detailed in the first step), using the spatial characteristic parameters as CNN training samples, marking the corresponding specified azimuth angle of each training sample, and using the corresponding specified azimuth angle as a class label of the training sample.
And fifthly, constructing a CNN model, and training by taking the training samples and the class labels obtained in the fourth step as a training data set of the CNN so as to obtain the CNN model. The method specifically comprises the following steps:
and (5-1) setting a CNN model structure.
The CNN architecture employed in the present invention is shown in fig. 2 and includes an input layer followed by three convolutional-pooling layers, then a fully-connected layer, and finally an output layer.
The input signal of the input layer is a J × L two-dimensional subband SRP-PHAT spatial spectrum matrix, i.e., training samples, in this embodiment, J is 36, and L is 72.
The input layer is followed by three convolution-pooling layers, each convolution layer adopts a convolution kernel with the size of 3 multiplied by 3, the step length is 1, and the characteristic dimensionality before and after convolution is kept unchanged by adopting a zero filling mode in the convolution operation. The numbers of convolution kernels of 1 st, 2 nd and 3 rd convolution layers are respectively 24, 48 and 96. After each convolution layer is subjected to convolution operation, batch normalization is carried out firstly, and then ReLU function activation is used. The pooling layer adopts a maximum pooling mode, the pooling size is 2 multiplied by 2, and the step length is 2.
After three convolution-pooling operations, the 36 × 72 two-dimensional sub-band SRP-PHAT spatial spectrum matrix becomes 5 × 9 × 96 feature data, and the feature data is straightened and deformed into 4320 × 1 one-dimensional vector feature data. The neurons in the fully connected layer are connected to all the feature data in the previous layer and the connected mode of Dropout is added to prevent overfitting, with the Dropout rate set to 0.5.
And the output layer adopts a Softmax classifier, a Softmax function converts the characteristic data of the full connection layer into the probability of the voice signal relative to each azimuth angle, and the azimuth angle with the maximum probability is taken as the predicted sound source direction.
And (5-2) training network parameters of the CNN model.
The training process of CNN includes two parts, forward propagation and backward propagation.
The forward propagation is to calculate the output of input data under the current network parameters, and is the layer-by-layer transfer process of the characteristics, and the forward propagation expression at the position (u, v) in the d-th layer is as follows:
Sd(u,v)＝ReLU((Sd-1*wd)(u,v)+βd(u,v))
wherein d represents a layer identifier and the d-th layer is a convolutional layer, SdRepresents the output of the d-th layer, Sd-1Denotes the output of layer d-1, the symbol denotes the convolution operation, wdRepresents the convolution kernel weight, β, of the d-th layerdIndicating the bias of layer d, ReLU is the activation function. The layers in the CNN structure employed by the present invention include the input layer, the convolutional and pooling layers in the convolutional-pooling layer, the fully-connected layer, and the output layer.
D represents an output layer, and the expression of the output layer is as follows:
SD＝Softmax((wD)TSD-1+βD)
wherein S isDRepresenting the output of the output layer, SD-1Represents the output of the fully connected layer, wDTo representConvolution kernel weight, beta, of the output layerDIndicating the bias of the output layer.
The goal of the back propagation phase is to minimize the cross-entropy loss function E (w, β):
wherein the subscript f denotes the f-th azimuth angle,
representing the desired output of the output layer at the f-th azimuth,
representing the actual output of the output layer at the f-th azimuth. F denotes the number of training positions, and in this embodiment, F is 36. The invention adopts a random Gradient Descent (SGDM) algorithm with Momentum to minimize a loss function, and relevant parameters of the SGDM are as follows: the Momentum parameter Momentum is set to 0.9, the L2 regularization coefficient is 0.0001, the initial learning rate is set to 0.01, the learning rate is reduced by 0.2 times every 6 rounds, and the mini-batch is set to 200.
The invention adopts a 7: 3 cross validation mode in the training process. And (5) carrying out iterative training for multiple times until convergence. So far, the CNN model training is completed.
And step six, processing the test signal according to the step two and the step three to obtain the spatial characteristic parameter of the single-frame test signal, namely the sub-band SRP-PHAT spatial spectrum matrix, and taking the spatial characteristic parameter as a test sample.
And step seven, taking the test sample as the input characteristic of the CNN model trained in the step four, outputting the probability that the test signal belongs to each azimuth angle by the CNN, and taking the azimuth with the highest probability as the sound source azimuth angle estimation value of the test sample.
Compared with the prior art, the method comprises two stages of training and testing. In the training stage, spatial characteristic parameters are extracted from directional voice signals in various reverberation and noise environments, and the directional voice signals are input into a CNN (computer-aided network) for training to obtain a CNN model. And in the testing stage, the spatial characteristic parameters of the test signals are extracted, the trained CNN model is input, and the azimuth with the highest probability is taken as the azimuth estimation value of the target sound source. The invention can complete the CNN training process off line, and stores the trained CNN model in the memory, and can realize real-time sound source positioning only by one frame of signal during testing. Compared with the traditional SRP-PHAT algorithm, the algorithm provided by the invention has the advantages that the positioning performance under a complex acoustic environment is obviously improved, and the generalization capability on the sound source space structure, reverberation and noise is better.
Fig. 3 and 4 show the positioning effect of the method of the present invention and the conventional SRP-phot algorithm when the testing environment and the training environment are consistent: in fig. 3, the reverberation time of the test environment and the training environment is 0.5s, and in fig. 4, the reverberation time of the test environment and the training environment is 0.8s, and the positioning effects of the test environment and the training environment under the conditions that the signal-to-noise ratios are 0dB, 5dB, 10dB, 15dB and 20dB are respectively researched, so that the positioning success rate of the method disclosed by the invention is far higher than that of the traditional SRP-PHAT algorithm.
Fig. 5 and fig. 6 show the positioning effect of the method of the present invention and the conventional SRP-phot algorithm when the signal-to-noise ratios of the test environment and the training environment are not consistent: in fig. 5, the reverberation time of the test environment and the training environment is 0.5s, the signal-to-noise ratio of the test environment is different from that of the training environment, in fig. 6, the reverberation time of the test environment and the training environment is 0.8s, and the signal-to-noise ratio of the test environment is different from that of the training environment, and the positioning effects of the test environment under the signal-to-noise ratios of-2 dB, 3dB, 8dB, 13dB and 18dB are respectively researched, so that the positioning success rate of the method disclosed by the invention is far higher than that of the traditional SRP-PHAT algorithm.
Fig. 7 and 8 show the positioning effect of the method of the present invention and the conventional SRP-PHAT algorithm when the reverberation times of the test environment and the training environment are inconsistent: in fig. 7, the reverberation time of the test environment is different from that of the training environment, the reverberation time of the test environment is 0.6s, in fig. 8, the reverberation time of the test environment is different from that of the training environment, and the reverberation time of the test environment is 0.9s, and the positioning effects of the test environment and the training environment under the signal-to-noise ratios of 0dB, 5dB, 10dB, 15dB and 20dB are respectively researched, so that the positioning success rate of the method disclosed by the invention is far higher than that of the traditional SRP-PHAT algorithm.
As can be seen from fig. 5 to fig. 8, even in the non-training environment, the success rate of the method of the present invention is still much higher than that of the conventional SRP-PHAT algorithm, which illustrates that the method of the present invention has better robustness and generalization capability to the unknown environment.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.
Claims (5)
1. A sound source positioning method based on a convolutional neural network and a sub-band SRP-PHAT space spectrum is characterized by comprising the following steps:
s1, the microphone array collects voice signals, and the collected voice signals are subjected to frame division and windowing preprocessing to obtain single-frame signals;
s2, calculating a sub-band SRP-PHAT spatial spectrum matrix of each frame signal;
s3, inputting the sub-band SRP-PHAT space spectrum matrixes of all frame signals into the trained convolutional neural network, outputting the probability that the voice signal belongs to each azimuth angle, and taking the azimuth angle with the maximum probability as the sound source azimuth angle estimation value of the voice signal.
2. The sound source localization method according to claim 1, wherein the step S2 of calculating the sub-band SRP-PHAT spatial spectrum matrix of each frame signal comprises the following steps:
s21, performing discrete Fourier transform on each frame of signal:
wherein x ism(i, n) is the ith frame signal of the mth microphone in the microphone array, M is 1,2, …, M is the number of microphones, Xm(i, k) is xm(i, N) indicating a frequency domain signal of an ith frame of the mth microphone, K being a frequency point, K being a length of discrete fourier transform, N being a frame length, K being 2N, and DFT (·) indicating discrete fourier transform;
s22, designing an impulse response function of the Gamma filter bank:
wherein j represents the serial number of the Gamma atom filter; c is the gain of the Gamma filter; t represents a continuous time; a is the order of the Gamma-tone filter;
represents the phase; f. ofjRepresents the center frequency of the j Gamma filter; bjRepresents the attenuation factor of the j Gamma filter, bjThe calculation formula is as follows:
bj＝1.109ERB(fj)
ERB(fj)＝24.7(4.37fj/1000+1)
discrete fourier transform of the impulse response function for each gamma filter:
wherein G isj(k) Is the frequency domain expression of the j Gamma filter, K is the frequency point, K is the length of discrete Fourier transform, N is the frame length, K is 2N, fsRepresents the signal sampling rate, DFT (-) represents the discrete fourier transform;
s23, calculating a sub-band SRP-PHAT function of each frame signal:
wherein P (i, j)R) represents the jth sub-band SRP-PHAT function of the ith frame signal when the beam direction is r; m is the number of microphones in the microphone array; tau ismn(r) represents a time difference of the sound wave propagating from the beam direction r to the m-th microphone and the n-th microphone, and is calculated by the formula:
where r denotes the coordinates of the beam direction, rmDenotes the position coordinate of the m-th microphone, rnRepresenting the position coordinate of the nth microphone, and c is the sound velocity in the air;
s24, carrying out normalization processing on the sub-band SRP-PHAT function of each frame signal:
s25, combining all sub-band SRP-PHAT functions of the same frame signal into a matrix form to obtain a sub-band SRP-PHAT spatial spectrum matrix:
wherein, y (i) represents the sub-band SRP-PHAT spatial spectrum matrix of the signal of the ith frame, J is the number of the sub-bands, namely the number of Gamma filters, and L is the number of beam directions.
3. The sound source localization method according to claim 2, wherein in step S23, when the sound source and the microphone array are set to be at the same horizontal plane and the sound source is located in the far field of the microphone array, τ ismnThe equivalent calculation formula of (r) is:
where xi is [ cos θ, sin θ ═]TAnd θ is the azimuth of the beam direction r.
4. The sound source localization method based on the convolutional neural network and the subband SRP-PHAT spatial spectrum as claimed in claim 1, wherein the convolutional neural network comprises an input layer, three convolutional-pooling layers, a full-link layer and an output layer which are connected in sequence;
in the convolution-pooling layer, each convolution layer adopts convolution kernels with the size of 3 multiplied by 3, the step length is 1, the number of the convolution kernels of the three convolution layers is 24, 48 and 96 in sequence, after each convolution layer is subjected to convolution operation, batch normalization is firstly carried out, then ReLU function activation is used, and the characteristic dimensions before and after convolution are kept unchanged by adopting a zero filling mode in the convolution operation; the pooling layer adopts a maximum pooling mode, the size of the pooling is 2 multiplied by 2, and the step length is 2;
after convolution-pooling, straightening and deforming the characteristic data into one-dimensional vector characteristic data;
adding a Dropout connection mode into the connection of the full connection layer and the one-dimensional vector characteristic data;
the output layer uses a Softmax classifier.
5. The sound source localization method based on the convolutional neural network and the subband SRP-PHAT spatial spectrum as claimed in claim 1, wherein the training step of the convolutional neural network is as follows:
s1, convolving the clean voice signal with the room impulse response of different azimuth angles, and adding different degrees of noise and reverberation to generate a plurality of directional voice signals of different specified azimuth angles:
xm(t)＝hm(t)*s(t)+vm(t),m＝1,2,...,M
wherein x ism(t) a directional speech signal representing a specified azimuth received by an mth microphone of the array of microphones; m is the serial number of the microphone, M is 1,2, …, and M is the number of the microphones; s (t) is a clean speech signal; h ism(t) denotes a designationRoom impulse response of azimuth to mth microphone; v. ofm(t) represents noise;
s2, preprocessing all directional voice signals by framing and windowing to obtain single-frame signals, and calculating a sub-band SRP-PHAT spatial spectrum matrix of each frame of signals;
s3, taking the sub-band SRP-PHAT space spectrum matrixes of all the directional voice signals as training samples, taking the designated azimuth angles of all the directional voice signals as class labels of the corresponding training samples, taking the training samples and the class labels as training data sets, and training the convolutional neural network by adopting a random gradient descent algorithm with momentum to minimize a loss function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110059164.1A CN112904279B (en) | 2021-01-18 | 2021-01-18 | Sound source positioning method based on convolutional neural network and subband SRP-PHAT spatial spectrum |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110059164.1A CN112904279B (en) | 2021-01-18 | 2021-01-18 | Sound source positioning method based on convolutional neural network and subband SRP-PHAT spatial spectrum |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112904279A true CN112904279A (en) | 2021-06-04 |
| CN112904279B CN112904279B (en) | 2024-01-26 |
Family
ID=76114123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110059164.1A Active CN112904279B (en) | 2021-01-18 | 2021-01-18 | Sound source positioning method based on convolutional neural network and subband SRP-PHAT spatial spectrum |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112904279B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113589230A (en) * | 2021-09-29 | 2021-11-02 | 广东省科学院智能制造研究所 | Target sound source positioning method and system based on joint optimization network |
| CN113655440A (en) * | 2021-08-09 | 2021-11-16 | 西南科技大学 | Self-adaptive compromising pre-whitening sound source positioning method |
| CN114897033A (en) * | 2022-07-13 | 2022-08-12 | 中国人民解放军海军工程大学 | Three-dimensional convolution kernel group calculation method for multi-beam narrow-band process data |
| CN114994608A (en) * | 2022-04-21 | 2022-09-02 | 西北工业大学深圳研究院 | Multi-device self-organizing microphone array sound source positioning method based on deep learning |
| CN115201753A (en) * | 2022-09-19 | 2022-10-18 | 泉州市音符算子科技有限公司 | Low-power-consumption multi-spectral-resolution voice positioning method |
| CN115331691A (en) * | 2022-10-13 | 2022-11-11 | 广州成至智能机器科技有限公司 | Pickup method and device for unmanned aerial vehicle, unmanned aerial vehicle and computer readable storage medium |
| CN116859336A (en) * | 2023-07-14 | 2023-10-10 | 苏州大学 | High-precision implementation method for sound source localization |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110222707A1 (en) * | 2010-03-15 | 2011-09-15 | Do Hyung Hwang | Sound source localization system and method |
| CN109164415A (en) * | 2018-09-07 | 2019-01-08 | 东南大学 | A kind of binaural sound sources localization method based on convolutional neural networks |
| CN109490822A (en) * | 2018-10-16 | 2019-03-19 | 南京信息工程大学 | Voice DOA estimation method based on ResNet |
| CN110133572A (en) * | 2019-05-21 | 2019-08-16 | 南京林业大学 | A kind of more sound localization methods based on Gammatone filter and histogram |
| CN110133596A (en) * | 2019-05-13 | 2019-08-16 | 南京林业大学 | A kind of array sound source localization method based on frequency point signal-to-noise ratio and biasing soft-decision |
| CN110517705A (en) * | 2019-08-29 | 2019-11-29 | 北京大学深圳研究生院 | A kind of binaural sound sources localization method and system based on deep neural network and convolutional neural networks |
| CN110544490A (en) * | 2019-07-30 | 2019-12-06 | 南京林业大学 | sound source positioning method based on Gaussian mixture model and spatial power spectrum characteristics |
| WO2020042708A1 (en) * | 2018-08-31 | 2020-03-05 | 大象声科（深圳）科技有限公司 | Time-frequency masking and deep neural network-based sound source direction estimation method |
| CN111123202A (en) * | 2020-01-06 | 2020-05-08 | 北京大学 | Indoor early reflected sound positioning method and system |
| CN111583948A (en) * | 2020-05-09 | 2020-08-25 | 南京工程学院 | Improved multi-channel speech enhancement system and method |
| CN111707990A (en) * | 2020-08-19 | 2020-09-25 | 东南大学 | Binaural sound source positioning method based on dense convolutional network |
| CN111968677A (en) * | 2020-08-21 | 2020-11-20 | 南京工程学院 | Voice quality self-evaluation method for fitting-free hearing aid |
-
2021
- 2021-01-18 CN CN202110059164.1A patent/CN112904279B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110222707A1 (en) * | 2010-03-15 | 2011-09-15 | Do Hyung Hwang | Sound source localization system and method |
| WO2020042708A1 (en) * | 2018-08-31 | 2020-03-05 | 大象声科（深圳）科技有限公司 | Time-frequency masking and deep neural network-based sound source direction estimation method |
| CN109164415A (en) * | 2018-09-07 | 2019-01-08 | 东南大学 | A kind of binaural sound sources localization method based on convolutional neural networks |
| CN109490822A (en) * | 2018-10-16 | 2019-03-19 | 南京信息工程大学 | Voice DOA estimation method based on ResNet |
| CN110133596A (en) * | 2019-05-13 | 2019-08-16 | 南京林业大学 | A kind of array sound source localization method based on frequency point signal-to-noise ratio and biasing soft-decision |
| CN110133572A (en) * | 2019-05-21 | 2019-08-16 | 南京林业大学 | A kind of more sound localization methods based on Gammatone filter and histogram |
| CN110544490A (en) * | 2019-07-30 | 2019-12-06 | 南京林业大学 | sound source positioning method based on Gaussian mixture model and spatial power spectrum characteristics |
| CN110517705A (en) * | 2019-08-29 | 2019-11-29 | 北京大学深圳研究生院 | A kind of binaural sound sources localization method and system based on deep neural network and convolutional neural networks |
| CN111123202A (en) * | 2020-01-06 | 2020-05-08 | 北京大学 | Indoor early reflected sound positioning method and system |
| CN111583948A (en) * | 2020-05-09 | 2020-08-25 | 南京工程学院 | Improved multi-channel speech enhancement system and method |
| CN111707990A (en) * | 2020-08-19 | 2020-09-25 | 东南大学 | Binaural sound source positioning method based on dense convolutional network |
| CN111968677A (en) * | 2020-08-21 | 2020-11-20 | 南京工程学院 | Voice quality self-evaluation method for fitting-free hearing aid |
Non-Patent Citations (5)
| Title |
|---|
| S. JIANG, W. L., P. YUAN, Y. SUN AND H. LIU: "Deep and CNN fusion method for binaural sound source localization", 《THE JOURNAL OF ENGINEERING》, pages 511 * |
| VECCHIOTTI等: "End-to-end Binaural Sound Localisation from the Raw Waveform", 《IEEE》, pages 451 - 455 * |
| XIAOYAN ZHAO 等: "Sound Source Localization Based on SRP-PHAT Spatial Spectrum and Deep Neural Network", 《COMPUTERS, MATERIALS & CONTINUA 》, pages 253 - 271 * |
| 张文涛;韩莹莹;黎恒;: "基于卷积神经网络的交通声音事件识别方法", 现代电子技术, no. 14 * |
| 王茜茜: "基于神经网络的鲁棒双耳声源定位研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 136 - 129 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113655440A (en) * | 2021-08-09 | 2021-11-16 | 西南科技大学 | Self-adaptive compromising pre-whitening sound source positioning method |
| CN113589230A (en) * | 2021-09-29 | 2021-11-02 | 广东省科学院智能制造研究所 | Target sound source positioning method and system based on joint optimization network |
| CN114994608A (en) * | 2022-04-21 | 2022-09-02 | 西北工业大学深圳研究院 | Multi-device self-organizing microphone array sound source positioning method based on deep learning |
| CN114994608B (en) * | 2022-04-21 | 2024-05-14 | 西北工业大学深圳研究院 | Multi-device self-organizing microphone array sound source positioning method based on deep learning |
| CN114897033A (en) * | 2022-07-13 | 2022-08-12 | 中国人民解放军海军工程大学 | Three-dimensional convolution kernel group calculation method for multi-beam narrow-band process data |
| CN114897033B (en) * | 2022-07-13 | 2022-09-27 | 中国人民解放军海军工程大学 | Three-dimensional convolution kernel group calculation method for multi-beam narrow-band process data |
| CN115201753A (en) * | 2022-09-19 | 2022-10-18 | 泉州市音符算子科技有限公司 | Low-power-consumption multi-spectral-resolution voice positioning method |
| CN115331691A (en) * | 2022-10-13 | 2022-11-11 | 广州成至智能机器科技有限公司 | Pickup method and device for unmanned aerial vehicle, unmanned aerial vehicle and computer readable storage medium |
| CN116859336A (en) * | 2023-07-14 | 2023-10-10 | 苏州大学 | High-precision implementation method for sound source localization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112904279B (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112904279B (en) | Sound source positioning method based on convolutional neural network and subband SRP-PHAT spatial spectrum | |
| CN107703486B (en) | Sound source positioning method based on convolutional neural network CNN | |
| CN109490822B (en) | Voice DOA estimation method based on ResNet | |
| CN110068795A (en) | A kind of indoor microphone array sound localization method based on convolutional neural networks | |
| CN112151059A (en) | Microphone array-oriented channel attention weighted speech enhancement method | |
| US20040175006A1 (en) | Microphone array, method and apparatus for forming constant directivity beams using the same, and method and apparatus for estimating acoustic source direction using the same | |
| CN107452389A (en) | A kind of general monophonic real-time noise-reducing method | |
| CN109782231B (en) | End-to-end sound source positioning method and system based on multi-task learning | |
| Morito et al. | Partially Shared Deep Neural Network in sound source separation and identification using a UAV-embedded microphone array | |
| CN113111765B (en) | Multi-voice source counting and positioning method based on deep learning | |
| Vesperini et al. | Localizing speakers in multiple rooms by using deep neural networks | |
| Aroudi et al. | Dbnet: Doa-driven beamforming network for end-to-end reverberant sound source separation | |
| CN110888105A (en) | DOA estimation method based on convolutional neural network and received signal strength | |
| CN112180318B (en) | Sound source direction of arrival estimation model training and sound source direction of arrival estimation method | |
| CN111123202B (en) | Indoor early reflected sound positioning method and system | |
| Salvati et al. | Time Delay Estimation for Speaker Localization Using CNN-Based Parametrized GCC-PHAT Features. | |
| CN111443328A (en) | Sound event detection and positioning method based on deep learning | |
| CN113593596B (en) | Robust self-adaptive beam forming directional pickup method based on subarray division | |
| CN112201276B (en) | TC-ResNet network-based microphone array voice separation method | |
| CN110838303B (en) | Voice sound source positioning method using microphone array | |
| Wang et al. | U-net based direct-path dominance test for robust direction-of-arrival estimation | |
| CN116559778A (en) | Vehicle whistle positioning method and system based on deep learning | |
| CN116859336A (en) | High-precision implementation method for sound source localization | |
| Firoozabadi et al. | Combination of nested microphone array and subband processing for multiple simultaneous speaker localization | |
| CN114758670A (en) | Beam forming method, beam forming device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |