JP2019128402A - Signal processor, sound emphasis device, signal processing method, and program - Google Patents

Abstract

To calculate information to be used for signal processing with high precision.SOLUTION: A signal processor comprises a storage part, a similarity calculation part, a weight calculation part, an update part, and a signal processing part. The storage part stores a first feature quantity representing features of a first input signal. The similarity calculation part calculates similarity between the first feature quantity and a second feature quantity representing features of a second input signal. The weight calculation part calculates a first weight for the first feature quantity upon the basis of the similarity and second feature quantity. The update part calculates a third feature quantity upon the basis of the first quantity having been multiplied by the first weight and the second feature quantity, and updates the first feature quantity stored in the storage part with the third feature quantity. The signal processing part executes signal processing using the first feature quantity having been updated.SELECTED DRAWING: Figure 2

Description

  Embodiments described herein relate generally to a signal processing device, a speech enhancement device, a signal processing method, and a program.

  In order to improve the recognition rate of a speech recognition system, a technique for executing signal processing for enhancing speech has been proposed. As a technique used in a speech enhancement apparatus, beam forming that enhances speech in a specific direction using spatial information of a signal is known. In order to execute signal processing with higher accuracy, it is desirable to calculate information (features and the like) used for signal processing with higher accuracy.

Patent No. 5044581 gazette

Heymann et al., “NEURAL NETWORK BASED SPECTRAL MASK ESTIMATION FOR ACOUSTIC BEAMFORMING”, ICASSP 2016

  However, in the prior art, information used for signal processing may not be calculated with high accuracy. For example, in beam forming, there is a case where the current sound source position is preferentially emphasized by providing a forgetting function. However, the forgetting function works even when the sound source does not move, and the enhancement effect may be reduced.

  The signal processing device according to the embodiment includes a storage unit, a similarity calculation unit, a weight calculation unit, an update unit, and a signal processing unit. The storage unit stores a first feature amount representing a feature of the first input signal. The similarity calculation unit calculates the similarity between the first feature value and the second feature value representing the feature of the second input signal. The weight calculation unit calculates a first weight for the first feature amount based on the similarity and the second feature amount. The update unit calculates a third feature amount based on the first feature amount multiplied by the first weight and the second feature amount, and updates the first feature amount stored in the storage unit with the third feature amount. Do. The signal processing unit executes signal processing using the updated first feature amount.

FIG. 1 is a hardware diagram of a signal processing device according to a first embodiment. FIG. 1 is a block diagram of a signal processing device according to a first embodiment. 6 is a flowchart of signal processing in the first embodiment. The figure for demonstrating the flow of the process which calculates and updates the feature-value. The hardware block diagram of the signal processing apparatus concerning 2nd Embodiment. FIG. 7 is a block diagram of a signal processing device according to a second embodiment. The flowchart of the signal processing in 2nd Embodiment.

  Hereinafter, preferred embodiments of a signal processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings. In the following description, an apparatus that mainly performs signal processing for enhancing speech will be described as an example, but applicable signal processing is not limited to speech enhancement processing. It can be applied to the processing of any signal other than voice. Further, signal processing other than enhancing the signal may be applied.

  In beam forming, it is usually assumed that the direction of arrival of a sound source is constant. For this reason, when the speaker is switched, and when the speaker moves relative to a voice input device (such as a microphone) for inputting voice, the emphasis effect is exhibited more than when the sound source is fixed. It is difficult. Therefore, a technique has been proposed in which the above-described forgetting function is provided and the current sound source position is prioritized and emphasized over the past sound source positions. However, even when the speaker does not move relatively, the effect of emphasis may not be obtained compared to the case where the forgetting function is not set because the forgetting function works.

  On the other hand, a technique for dealing with speaker switching by using clustering has been proposed. However, such a method is a rule-based method and includes non-differentiable components. For this reason, it is difficult to adjust a parameter for improving the accuracy of clustering by using an output criterion, for example, a criterion (maximum SNR criterion) representing maximization of a signal-to-noise ratio (SN ratio). there were.

First Embodiment
The signal processing apparatus according to the first embodiment stores feature quantities representing spatial information of a speaker in each of a plurality of storage areas. The signal processing apparatus inputs, to the neural network, the degree of similarity between the feature quantity stored in the storage unit and the input feature quantity and the input feature quantity each time the feature quantity for the audio signal is input. The neural network outputs a weight having a number of dimensions equal to the number of storage areas. The weight to be output includes, for example, a weight for the stored feature amount (erasing weight), a weight for the input feature amount (write weight), and a weight for the feature amount read from the storage area (read weight). The feature amount read from the storage area is used for signal processing such as beam forming.

  In the present embodiment, it is possible to make the neural network learn an appropriate feature amount rewriting and reading method using learning data. For this reason, it becomes possible to learn that no forgetting is performed when it is convenient for emphasizing to hold the feature amount without forgetting.

  Further, in the present embodiment, the similarity between the stored feature amount and the current feature amount, which is information highly relevant to necessity of oblivion, is included in the input of the neural network. Thereby, data required for learning can be reduced as compared with the case where the similarity is not input. Even if the similarity is not input, it can be learned to change the output depending on whether the stored feature quantity is similar to the current feature quantity, but more data is required for this purpose. It is. Although there is a possibility that the data for learning increases, it may be configured not to include the similarity in the input of the neural network.

  Thus, according to the present embodiment, information used for signal processing can be calculated with higher accuracy while introducing the forgetting function. For example, even when the speaker does not move relatively, the emphasis effect can be maintained. In addition, as described below, in this embodiment, a model that does not include a non-differentiable component is used. Therefore, the parameters that define each function including the forgetting function maximize the evaluation criteria (such as SN ratio) defined by the output. Can be adjusted to

  Next, the hardware configuration of the signal processing apparatus according to the first embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram illustrating a hardware configuration example of the signal processing apparatus 100 according to the first embodiment.

  The signal processing device 100 includes a central processing unit (CPU) 51, a read only memory (ROM) 52, a random access memory (RAM) 53, a storage device 54, and an operation device 55, which are connected via a bus. ing.

  The CPU 51 uses the RAM 53 as a work area, executes various processes in cooperation with the program recorded in the RAM 53, and centrally controls the operation of the signal processing apparatus 100.

  The ROM 52 stores a program relating to the operation of the signal processing apparatus 100, media data necessary for learning, and the like in a non-rewritable form.

  The RAM 53 is a storage medium such as SDRAM (Synchronous Dynamic Random Access Memory). The RAM 53 functions as a work area of the CPU 51 and plays a role of holding intermediate data.

  The storage device 54 is a medium capable of storing information magnetically or optically, and stores various setting information and learning results.

  The operating device 55 is, for example, a keyboard and a mouse, and outputs a user's input to the CPU 51.

  FIG. 2 is a block diagram illustrating an example of the configuration of the signal processing apparatus 100. As shown in FIG. 2, the signal processing apparatus 100 includes a generation unit 101, an analysis unit 111, a feature quantity calculation unit 112, a similarity calculation unit 113, a weight calculation unit 114, an update unit 115, and signal processing. Unit 121, learning unit 122, and storage unit 141.

  The storage unit 141 stores a feature amount (first feature amount) calculated for a speech signal (first input signal) input in the past. The storage unit 141 can be configured by, for example, the RAM 53 of FIG. The storage unit 141 includes a plurality of storage areas, and stores the feature amount in each of the plurality of storage areas.

  The generation unit 101 generates learning data used for learning. For example, the generation unit 101 generates learning data including an audio signal (third input signal) and reference data. The reference data is data representing a processing result of signal processing on the audio signal. The reference data is referred to at the time of learning by the learning unit 122.

  The generation unit 101 generates learning data that increases diversity and improves robustness after learning, for example, by processing learning data prepared in advance, and outputs the generated learning data to the analysis unit 111. As described above, the learning data to be generated can include reference data for use in the learning unit 122. In that case, the reference data need not be input to other than the learning unit 122.

  When data corresponding to learning data generated by the generation unit 101 is prepared, if the data is configured to be used, the generation unit 101 need not be provided.

  The audio signal is a signal recorded by an audio input device such as a microphone array. The microphone array includes a plurality of microphones provided at different positions in the space, and acquires audio signals of a plurality of channels corresponding to the plurality of microphones. In the following, the case of using audio signals of a plurality of channels will be described as an example, but the same method can be applied to an audio signal of one channel.

Although any method may be used to generate learning data, for example, the following method can be used.
Generate an impulse response in a region (such as a room) in which the sound source is present and fold it into the original signal.
Add noise.
・ Drop samples randomly.
Add random delays between channels.
・ Change duration and pitch with phase vocoder.

  Further, the generation unit 101 may generate learning data for reproducing a situation where a speaker changes. For example, to reproduce the situation when the speaker changes from A to B to A, the generation unit 101 generates a signal having an interchannel correlation corresponding to A and a signal having an interchannel correlation corresponding to B. It is also possible to generate learning data in which noises are further connected in a staggered manner. Thereby, it can be expected that the follow-up speed of the speech enhancement when a speaker who has spoken in the past speaks again is improved.

  The analysis unit 111 analyzes the input learning data, and outputs information used in the subsequent processing as an analysis result. For example, the analysis unit 111 performs a short-time Fourier transform with a window function on the input audio signal, and outputs a spectrogram. Similarly to Non-Patent Document 1, it may be configured to cause the neural network to execute signal / noise determination in each time frequency bin of the spectrogram and to add the determination result to the output in order to calculate the subsequent feature amount.

  The feature amount calculation unit 112 calculates a feature amount based on the information output from the analysis unit 111. For example, the feature amount calculation unit 112 calculates a spatial correlation between signals of a plurality of channels included in the input signal as a feature amount. Examples of spatial correlation include the spatial correlation of the entire input, the noise spatial correlation calculated only from the spectrogram region estimated to contain a lot of noise, and the spectrogram region estimated to contain a lot of signal. Signal space correlation.

The similarity calculation unit 113 calculates the similarity between the feature amount stored in each storage area of the storage unit 141 and the feature amount (second feature amount) calculated by the feature amount calculation unit 112. The degree of similarity uses, for example, a complex correlation coefficient Real (v ^H r _i ) / (| v | r _i |) between a vector v obtained by vectorizing spatial correlation and the content r _i of the i-th storage area . The symbol H represents Hermitian transpose.

  The spatial correlation is calculated, for example, for each frequency. The vector v may be obtained by concatenating and vectorizing all the feature quantities calculated for each frequency. The vector v may individually vectorize the spatial correlation calculated for each frequency. In the latter case, securing of the storage area of the storage unit 141 and processing of the latter stage such as calculation of the degree of similarity are also executed independently for each spatial correlation.

  The weight calculation unit 114 calculates the erase weight, the write weight, and the read weight described above. The erasure weight is a weight (first weight) for the feature amount stored in the storage unit 141. The erasure weight corresponds to, for example, a forgetting factor used in the forgetting function described above. The writing weight is a weight (second weight) for the feature amount calculated by the feature amount calculating unit 112. The read weight is a weight (third weight) for the feature value read from the storage area in order to calculate the feature value used for signal processing.

  The weight calculation unit 114 calculates a weight based on, for example, the similarity calculated by the similarity calculation unit 113 and the feature amount calculated by the feature amount calculation unit 112. For the calculation of the weight, a neural network that inputs the similarity and the feature amount and outputs each weight can be used. The model for calculating the weight is not limited to the neural network. For example, another model that performs regression analysis, such as a Gaussian process, may be applied.

  The weight calculation unit 114 uses, for example, a neural network that receives the similarity and the spatial correlation (vectorized vector v) and outputs three weight vectors representing the erasure weight, the write weight, and the read weight. Each weight vector is a vector having the same number of dimensions as the number of storage areas for storing feature amounts. Each weight vector element takes a real value ranging from 0 to 1.

  In the present embodiment, the weights can be calculated so as to have different values according to the degree of similarity. For example, when the stored feature value is similar to the feature value for the input audio signal, in other words, when the sound source does not move, the effect of the forgetting function is suppressed by increasing the erasure weight. Is possible. In order to suppress the effect of the forgetting function, at least the erasure weight may be calculated according to the degree of similarity, and other weights (write weight, read weight) may be determined by another method. For example, a method of setting other weights to a fixed value, a method of calculating other weights according to the value of the erasure weight, and the like may be applied.

  The updating unit 115 updates the feature amount stored in each storage area of the storage unit 141 using the calculated weight vector and the feature amount calculated by the feature amount calculating unit 112. For example, the update unit 115 multiplies the stored feature value by the erasure weight, multiplies the feature value calculated by the feature value calculation unit 112 by the write weight, and adds each multiplication result to obtain the feature value (third Calculate the feature amount). The feature quantities calculated in this manner are vectors having the same dimensions as the stored feature quantities, and the number is the same as the number of stored feature quantities (the number of storage areas). The update unit 115 updates the feature amount stored in the storage unit 141 with the calculated feature amount.

  Since the spatial correlation has the property of being Hermitian symmetric, the feature quantity calculated by the updating unit 115 needs to satisfy the property of being Hermitian symmetric when interpreted as a matrix. Since feature quantities are calculated by performing operations (multiplication, addition, etc.) that maintain Hermitian symmetry using feature quantities (spatial correlation) that are Hermitian symmetric, the feature quantities calculated by the updating unit 115 are also Hermitian symmetric. Meet the property of being

  The signal processing unit 121 performs signal processing using the updated feature amount. The signal processing is, for example, a voice enhancement process that emphasizes some of the voice signals of a plurality of channels. For example, the signal processing unit 121 generates a filter that emphasizes a signal based on the feature amount (spatial correlation) read from the storage unit 141, and causes the generated filter to act on the input to obtain an output. As a filter calculation method, for example, a method based on the maximum SNR standard as described in Non-Patent Document 1 can be used. A post filter may be further applied to the output signal. For example, as described in Non-Patent Document 1, BAN (Blind Analytical Normalization) can be used.

  The learning unit 122 learns the parameters of the neural network used when calculating the weight. For example, the learning unit 122 executes processing up to signal processing by the signal processing unit 121 using learning data, evaluates the processing result of the signal processing, and updates parameters of the neural network according to the evaluation result. The learning unit 122 performs a learning process using the learning data generated by the generation unit 101, for example. When the analysis unit 111 uses a neural network, the learning unit 122 may also learn parameters of this neural network.

  For example, the learning unit 122 calculates an evaluation value from reference data and a processing result of the signal processing unit 121, and updates parameters of the neural network by error back propagation. When the reference data is a signal on which no noise is superimposed, a square error from the output can be used as the evaluation value. When the reference data is a signal and noise, an SN ratio that can be calculated from the applied filter can be used as the evaluation value.

  The learning unit 122 determines whether to end learning from the transition of the evaluation value. As a criterion (end criterion) for determining the end, for example, a criterion such as no improvement in the transition of the evaluation value calculated from the past 10000 inputs can be considered. If the end criterion is not satisfied, the learning unit 122 outputs a command to the generation unit 101 to newly generate learning data, for example. When the end criterion is satisfied, the learning unit 122 stores the learned parameter in the storage unit 141 or the like, and ends the learning process.

  Each of the units (generation unit 101, analysis unit 111, feature quantity calculation unit 112, similarity calculation unit 113, weight calculation unit 114, update unit 115, signal processing unit 121, and learning unit 122) is, for example, one or more Realized by the processor of For example, each unit may be realized by causing a processor such as the CPU 51 to execute a program, that is, software. The respective units may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, hardware. Each of the above units may be realized by using software and hardware together. When using a plurality of processors, each processor may realize one of the respective units, or may realize two or more of the respective units.

  The storage unit 141 can be configured by any generally used storage medium such as a hard disk drive (HDD), an optical disk, a memory card, and a RAM. The storage areas of the storage unit 141 may be physically different storage media, or may be realized as different storage areas of the physically same storage medium. Furthermore, each of the storage areas of the storage unit 141 may be realized by a plurality of physically different storage media.

  Next, signal processing by the signal processing apparatus 100 configured as described above will be described with reference to FIG. FIG. 3 is a flowchart illustrating an example of signal processing in the first embodiment.

  First, when the start of signal processing is instructed through the operation device 55 or the like, the generation unit 101 executes initialization processing (step S101). For example, the generation unit 101 reserves a storage area for various settings of the learning process and a storage area for storing feature amounts in the storage unit 141.

  Further, the generation unit 101 reads out learning data stored in advance in the storage unit 141 and stores the learning data in the RAM 53. The learning data may be read out and stored all at once or may be read out and stored sequentially. The generation unit 101 may discard the read data.

  The learning data is divided into two types of signals, for example, a signal to be emphasized and a signal to be suppressed. The signal to be emphasized is typically voice (voice signal). It is assumed that a signal that is not an object of enhancement is sufficiently small even if it exists. For example, learning data whose SN ratio is equal to or higher than a predetermined threshold value (for example, 40 dB (decibel)) is used. The following description will be made on the assumption that the object to be emphasized is speech, but it is to be noted that the following procedure can be applied even if the object to be emphasized is not speech. For example, it can be applied to an arbitrary signal having a characteristic pattern in the time-frequency domain, such as a sound of a musical instrument. Moreover, not only a sound wave but the electromagnetic wave containing the reflected laser beam etc. can also be made into object. Hereinafter, the signal to be suppressed is referred to as noise (noise signal).

  As for voice and noise, signals that can be regarded as the same are observed over a plurality of channels, and signals of at least one channel are different from signals of other channels. Such a signal is obtained, for example, by recording using a microphone array. Such a signal may be generated by simulating multi-channel recording by convoluting an impulse response of a region (such as a room) where a sound source is present with a signal of one channel. Also note that the number of voice and noise channels is equal.

  Next, the production | generation part 101 produces | generates the learning data used by the learning process by the learning part 122 from the learning data prepared in advance (step S102). For example, the generation unit 101 randomly selects voice and noise, adjusts the amplitude with a random signal-to-noise ratio, and superimposes them on all channels. For example, the generation unit 101 determines the SN ratio by sampling from a uniform distribution in a predetermined range (for example, a range of −5 dB to 10 dB). At this time, voice start times of all channels may be commonly delayed by random time. For example, when the noise is sufficiently longer than the speech, the generation unit 101 determines the delay time by sampling from a uniform distribution in which the range of the time delay is set such that the speech is included in the noise range.

  There may be a plurality of voices to be superimposed on the noise. In that case, the generation unit 101 uses a plurality of sounds that do not overlap each other. The generation unit 101 may convolve a common impulse response with a plurality of sounds. As a result, it is possible to simulate a situation where the user speaks from the same position. The generation unit 101 may be configured to convolute an impulse response at a slightly different position, for example, an impulse response from a position moved about 20 cm to 50 cm. This makes it possible to simulate a situation where the sound source has moved slightly.

  From the data obtained as described above, the generation unit 101 may cut a signal in a range not including speech and use it as learning data.

  Next, the analysis unit 111 inputs the generated learning data (input signal), and executes signal analysis processing on the input signal (step S103). For example, the analysis unit 111 performs time-frequency analysis on the input signal for each channel, outputs an analysis result represented by the time frequency, and stores the analysis result in, for example, the RAM 53. As a method of time frequency analysis, for example, short time Fourier transform and filter bank analysis such as wavelet transform can be used.

Then analyzer 111 receives the analysis result to the neural network _{N 1,} stores the intermediate output and the final output of the neural network _{N 1,} for example, in RAM 53. As a method of providing input, a plurality of channels may be input collectively, or may be processed independently for each channel. When processing each channel independently, post-processing is added to obtain the final output. For example, a method of obtaining the median value of the obtained output of each channel for each time frequency coordinate is conceivable.

Here, the number of dimensions of the final output of the neural network N ₁ is twice the characteristic quantity number for each frame of the analysis results. The constituent elements of the neural network N _1, feedforward connection, convolution connection, and, like structures using LSTM (Long short-term memory) , may employ any structure. When using a type of structure that uses information of the entire sequence such as Bidirectional LSTM, note that online processing can not be performed at the time of execution after learning.

Only the absolute value discards the phase information of the analysis result, may enter additional values it took the natural logarithm of the absolute value to the neural network N _1. With this configuration, the dynamic range of the input is narrowed, and the stability at the time of parameter updating in the subsequent stage can be improved.

Analyzer 111 applies a sigmoidal function to the final output of the neural network N _1. The sigmoid function is used, for example, to make the output range from 0 to 1. A function other than the sigmoid function having the same function may be used. The analysis unit 111 separates the output of the sigmoid function into two, one as a speech mask and the other as a noise mask.

Next, the feature amount calculation unit 112 calculates a feature amount for each of speech and noise (step S104). For example, the feature amount calculation unit 112 obtains an estimated value of spatial correlation of speech and an estimated value of spatial correlation of noise using each mask for the analysis result. More specifically, the feature quantity calculation unit 112 calculates the speech mask m _S (t, ω) and the noise mask m _N (t, ω) with respect to the input vector x (t, ω) at time t and frequency ω. And the feature quantity (spatial correlation) ξ _X is calculated by the following equation (1).
ξ _X (t, ω) = m _X (t, ω) x (t, ω) x ^H (t, ω) (1)

“X” of ξ _X and m _X indicates that either “S” indicating speech or “N” indicating noise is set. The following processing is executed independently for voice and noise. For convenience of explanation, variable names with “X” are used when it is not necessary to distinguish them. Each element of the input vector x (t, ω) corresponds to each channel.

Next, the similarity calculation unit 113 calculates the similarity between each feature amount stored in the storage unit 141 and the feature amount calculated in step S104 (step S105). Let L be the number of storage areas for storing feature quantities. The L vectors indicating the feature quantities stored in the L storage areas are denoted as r ₁ , r ₂ ,..., R _L. Also, in the following, a matrix in which L vectors are arranged is represented as R = {r ₁ , r ₂ ,..., R _L }.

For example, the similarity calculation unit 113 calculates, as the similarity, the correlation coefficient between each of the _L vectors r ₁ , r ₂ , ..., r _L and v _X obtained by vectorizing the feature amount ξ _X Do. As the correlation coefficient, the above-mentioned complex correlation coefficient Real (v ^H _x r _i ) / (| v _x || r _i |) (1 ≦ i ≦ L) or the like can be used. Further, v _X may be generated by connecting and vectorizing all feature quantities (ξ _S or ξ _N ) calculated for each frequency, or may be divided appropriately and managed. For example, it may generate a v _X and vectorized for each frequency. Each of L vectors r ₁ , r ₂ ,..., R _L stored in the storage unit 141 is a vector equal to the dimensionality of v _X.

Next, the weight calculation unit 114 calculates a weight using the calculated similarity and the feature amount (step S106). For example the weight calculation unit 114 inputs the L number of similarities, and v _X that vector the feature amount, the neural network N _2. Three weight vector _W D neural network _{N 2} is the number of dimensions _L, W W, and outputs the _{W R.} Each element of each weight vector is a real number greater than or equal to 0, and the sum of each element is 1. The weight vectors W _D , W _W , and W _R correspond to the erase weight, the write weight, and the read weight, respectively.

v The _X number of dimensions is fixed, it is or is optional depending on the configuration of the neural network N _2. For example, when the dimensionality of the input and output is fixed, as in a feedforward type structure of full coupling, the dimensionality of v _X uses a common fixed value during learning and speech enhancement. On the other hand, such as convolutional network, v case of adopting a computable structure without depending on the number of dimensions of _X, v the number of dimensions of _X is arbitrary. Even in any case, as long as each feature amount stored in the storage area is not newly initialized, the number of dimensions of the subsequently input v _X is equal to that previously input.

Next, the updating unit 115 updates the feature amount stored in the storage unit 141 using the calculated weight (step S107). For example, the updating unit 115 updates the matrix R including the stored L vectors according to the following equation (2). Diag (.) Represents a diagonal matrix having vectors as diagonal elements.
^{_{R ← RDiag (W D) +}} v X W H W ··· (2)

Updating unit 115 uses the updated R follows the output phi _X (3) is calculated by equation.
φ _X = W ^H _R R (3)

In the above procedure, assuming that the number of feature quantities stored in the storage unit 141 is 1 (L = 1) and a fixed value independent of the input is used as a weight (forgetting factor), the following equation (4) and a constant Matches except double. Equation (4) represents an estimation method with a forgetting coefficient for online estimation of spatial correlation. α represents a fixed forgetting factor. Thus, it can be seen that the above procedure includes in special cases the existing method with a fixed forgetting factor.
R ← αR + v _X (4)

  In this embodiment, by calculating the weight according to the above-described procedure, the weight can be flexibly controlled by adapting to the input, as compared with the existing case.

The output φ _X can be regarded as an estimated value of spatial correlation when each feature quantity stored in the storage unit 141 can be regarded as spatial correlation. For this purpose, the information stored in the storage unit 141 needs to be initialized so that it can be regarded as an estimated value of the spatial correlation. For example, an initial value initialized by a method such as adding cc ^H to each storage area a sufficient number of times using a random complex vector c satisfies this condition. The number of dimensions of c is equal to the number of input channels. The sufficient number of times is, for example, about twice the number of dimensions of c. As a sampling method of the complex vector, for example, a method of sampling the real part and the imaginary part from uniform distribution in the range of -1 to 1 can be used.

In this way, outputs φ _S and φ _N having the same number of dimensions corresponding to speech and noise are obtained.

The signal processing unit 121 executes signal processing using these outputs (step S108). For example, the signal processing unit 121 designs the filter f on the basis of the maximum SNR for the outputs φ _S and φ _N. This can be solved by a generalized eigenvalue problem. For example, the signal processing unit 121 can generate a filter by the method described in Non-Patent Document 1. The signal processing unit 121 applies the generated filter to the time-frequency representation of the mixed speech, and further applies BAN, if necessary, to output a time-frequency representation of noise-suppressed speech.

Next, the learning unit 122 updates the parameters of the neural network using the processing result of the signal processing unit 121 (step S109). For example, the learning unit 122 calculates an SN ratio for the time-frequency expression of the noise-suppressed speech calculated by the signal processing unit 121. Assuming that a signal containing only speech is s (t, ω) and a signal containing only noise is n (t, ω), the SN ratio E _CN is obtained by the following equation (5).
E _CN = | (f ^H s) / (f ^H n) | (5)

Learning unit 122 obtains a differential of the calculated SN ratio, for example, to update the parameters of the neural network N ₁ and N ₂ by the error backpropagation. When updating, instead of using the derivative value as it is, a modified value may be used by applying Adam or the like.

In order to stabilize the parameter update of the neural network N ₁ , the evaluation value is the cross-entropy error of the correct mask reflecting the S / N ratio and the calculated speech mask m _S (t, ω) or noise mask m _N (t, ω). And the parameters may be updated.

  For example, when the SN ratio is equal to or higher than the upper limit value (for example, 10 dB), the correct mask is set to 1. When the SN ratio is equal to or lower than the lower limit value (for example, −10 dB), the noise mask is set to 1. Created on the basis of

The learning unit 122 repeats the above processing until learning converges. For example, the learning unit 122 determines whether or not an end condition is satisfied (step S110). The termination condition may be any condition. For example, the following condition can be applied.
-It is considered that it has converged when the number of updates reaches a certain value (for example, 1 million times).
-Assess that the SN ratio has improved with respect to the average SN ratio of the evaluation data each time the number of updates reaches a fixed value (for example, 1 million times). When no improvement is observed for a predetermined number of times (for example, 5 times), it is considered that the image has converged. For example, the learning unit 122 separates part of the learning data without using it for learning, and uses it as evaluation data.

  When the end condition is not satisfied (step S110: No), the process returns to step S103 and the process is repeated. When the end condition is satisfied (step S110: Yes), the learning unit 122 stores the updated parameter in the storage unit 141, for example.

  Next, the process of calculating and updating the feature amount will be further described. FIG. 4 is a diagram for explaining the flow of processing for calculating and updating feature amounts.

The analysis unit 111 and the feature amount calculation unit 112 calculate a feature amount from the input signal. The feature amount is represented by, for example, a spatial correlation matrix that represents a spatial correlation between signals of a plurality of channels. Feature amount is vectorized into v _X.

On the other hand, the storage unit 141 stores L vectors r ₁ , r ₂ ,..., R _{L of the} same dimensional number as v _X. The entire storage unit 141 stores a matrix R = {r ₁ , r ₂ ,..., R _L } in which _L vectors are arranged.

Similarity calculating unit 113 calculates the similarity between the vector v _X, respectively the L vectors. The calculated similarity is input to the neural network, and the neural network outputs a weight. The dimension number of the weight is L corresponding to L vectors. The output weight includes at least a weight (erase weight) for the stored feature amount.

The updating unit 115 updates the feature amount stored in the storage unit 141 using the output weight, the calculated feature amount, and the feature amount stored in the storage unit 141, and the updated feature amount Is used to calculate a feature value φ _X (φ _S and φ _N ) for signal processing.

  As described above, in the signal processing apparatus according to the first embodiment, since the weight (erasing weight) for the stored feature amount is used, the same function as the conventional forgetting function can be realized. Furthermore, since the weight is calculated according to the similarity between the calculated feature amount and the stored feature amount, it is possible to calculate information (feature amount) used for signal processing with higher accuracy.

Second Embodiment
The signal processing apparatus according to the second embodiment is an apparatus that performs signal processing (for example, speech enhancement processing) using a model whose parameters have been learned by the signal processing apparatus or the like according to the first embodiment. It may be configured to have both the function of the signal processing device (the device that executes the learning process) of the first embodiment and the function of the signal processing device of the present embodiment.

  FIG. 5 is an explanatory diagram illustrating a hardware configuration example of the signal processing device 100-2 according to the second embodiment.

  The signal processing device 100-2 includes a CPU 61, a ROM 62, a RAM 63, a storage device 64, an operating device 65, an input device 66, and an output device 67, which are connected via a bus.

  The functions of the CPU 61, the ROM 62, the RAM 63, the storage device 64, and the operation device 65 are the same as those of the signal processing device 100, and thus the description thereof is omitted.

  The input device 66 is, for example, a microphone array that inputs sound. The input device 66 acquires a plurality of independent signals from a plurality of microphones constituting the microphone array.

  The output device 67 is a device for outputting various information. For example, the output device 67 is one or a plurality of audio output devices such as a speaker, an earphone, and a headphone. The audio output device converts an electrical signal into air vibration and outputs the air vibration. The output device 67 may be a display. The display displays, for example, a speech recognition result.

  FIG. 6 is a block diagram illustrating an example of a configuration of a signal processing device 100-2 according to the second embodiment. As illustrated in FIG. 6, the signal processing apparatus 100-2 includes a reception unit 131-2, an analysis unit 111, a feature quantity calculation unit 112, a similarity calculation unit 113, a weight calculation unit 114, and an update unit 115. A signal processing unit 121 and a storage unit 141.

  The second embodiment is different from the first embodiment in that the generation unit 101 and the learning unit 122 are deleted and the reception unit 131-2 is added. The other configurations and functions are the same as those in FIG. 2 which is a block diagram of the signal processing apparatus 100 according to the first embodiment, and thus the same reference numerals are given and the description thereof is omitted here.

  The accepting unit 131-2 accepts input of information to be subjected to signal processing and outputs it to the analyzing unit 111. For example, the reception unit 131-2 receives an input signal that is multi-channel waveform data acquired by the microphone array. The reception unit 131-2 digitizes the input signal by AD (analog-digital) conversion, and stores the digitized signal in, for example, a work area in the storage unit 141. The reception unit 131-2 outputs the digitized signal to the analysis unit 111.

  The processing after the analysis unit 111 is the same as in the first embodiment. The signal processing unit 121 outputs a processing result for the received waveform data. For example, the signal processing unit 121 outputs a time frequency expression (spectrum) of the noise-suppressed speech. The signal processing unit 121 may output the processing result converted into the format used in the processing of the latter stage. For example, the signal processing unit 121 may convert the spectrum after enhancement processing into an output waveform using an overlap add to which a synthesis window is applied, and output the output waveform. When a speech recognition system is connected to the latter stage, the spectrum may be output directly without converting it into a waveform.

  Next, signal processing by the signal processing apparatus 100-2 according to the second embodiment configured as described above will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of signal processing according to the second embodiment.

  First, when the start of signal processing is instructed via the operation device 65 or the like, the reception unit 131-2 executes an initialization process (step S201). For example, the receiving unit 131-2 secures in the storage unit 141 a storage area for the learned parameter and a storage area for storing the feature amount.

  The accepting unit 131-2 accepts input of signals of a plurality of channels acquired by, for example, a microphone array (Step S202). The reception unit 131-2 digitizes the signal by AD conversion, and stores the digitized waveform in the storage unit 141.

  The processes from step S203 to step S208 are the same as the processes from step S103 to step S108 in the signal processing apparatus 100 according to the first embodiment, and thus the description thereof will be omitted.

  The signal processing result (for example, the spectrum of the emphasized speech) is obtained by the signal processing in step S208. The above procedure is repeated until the end of the operation is instructed. For example, the reception unit 131-2 determines whether or not the operation has been instructed via the operation device 65 or the like (step S209). When the end of the operation is not instructed (Step S209: No), the process is repeated from Step S202 for the next input signal. When the end of the operation is instructed (step S209: Yes), the signal processing ends.

  At the end of the process, the feature quantities stored in the respective storage areas of the storage unit 141 may be stored in another non-volatile storage medium (for example, the storage device 64). Then, the feature amount stored in the storage medium may be read as an initial setting value at the next activation and set in the storage unit 141. Thereby, the initialization process of the storage area of the storage unit 141 can be omitted.

  As described above, the signal processing apparatus according to the second embodiment can apply the same method as that of the first embodiment at the time of signal processing such as speech enhancement processing.

  As described above, according to the first and second embodiments, information (feature amount) used for signal processing can be calculated with higher accuracy.

  The program executed by the signal processing device (the signal processing device 100, the signal processing device 100-2) of the above embodiment is provided by being incorporated in advance in the ROM 52 or the like.

  A program executed by the signal processing apparatus is an installable or executable file, which is a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), a DVD ( The program may be configured to be recorded as a computer program product by being recorded on a computer readable recording medium such as Digital Versatile Disk).

  Furthermore, the program executed by the signal processing apparatus may be stored on a computer connected to a network such as the Internet, and may be provided by being downloaded via the network. The program executed by the signal processing apparatus may be provided or distributed via a network such as the Internet.

  The program executed by the signal processing device can cause the computer to function as each unit of the signal processing device described above. This computer can read out a program from the computer readable storage medium onto the main storage device and execute it.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

100, 100-2 Signal processing apparatus 101 Generation unit 111 Analysis unit 112 Feature quantity calculation unit 113 Similarity calculation unit 114 Weight calculation unit 115 Update unit 121 Signal processing unit 122 Learning unit 131-2 Reception unit 141 Storage unit

Claims (11)

A storage unit for storing a first feature amount representing a feature of the first input signal;
A similarity calculation unit that calculates a similarity between the first feature amount and a second feature amount representing a feature of the second input signal;
A weight calculator configured to calculate a first weight for the first feature amount based on the similarity and the second feature amount;
A third feature amount is calculated based on the first feature amount multiplied by the first weight and the second feature amount, and the first feature amount stored in the storage unit according to the third feature amount. Updating section, and
A signal processing unit that performs signal processing using the updated first feature value;
A signal processing apparatus comprising: The weight calculation unit calculates the first weight using a model that receives the similarity and the second feature and outputs the first weight.
The signal processing apparatus according to claim 1. The model is a neural network,
The signal processing device according to claim 2. The system further includes a learning unit that evaluates the processing result of the signal processing using learning data and updates parameters of the model.
The signal processing apparatus according to claim 2. It further comprises a generation unit that generates learning data including a third input signal and reference data representing the processing result of the signal processing,
The learning unit executes a learning process using the generated learning data.
The signal processing device according to claim 4. The first feature quantity, the second feature quantity, and the third feature quantity are spatial correlations based on a plurality of input signals input from different positions in space.
The signal processing apparatus according to claim 1. The weight calculation unit further calculates a second weight for the second feature amount based on the similarity and the second feature amount.
The update unit calculates the third feature amount based on the first feature amount multiplied by the first weight and the second feature amount multiplied by the second weight.
The signal processing apparatus according to claim 1. The weight calculation unit further calculates a third weight for the first feature value read from the storage unit based on the similarity and the second feature value.
The signal processing unit executes signal processing using the first feature amount multiplied by the third weight.
The signal processing apparatus according to claim 1. A storage unit for storing a first feature amount representing a feature of a first input signal including a plurality of channels of audio signals;
A similarity calculation unit that calculates a similarity between the first feature amount and a second feature amount representing a feature of a second input signal including audio signals of a plurality of channels;
A weight calculator configured to calculate a first weight for the first feature amount based on the similarity and the second feature amount;
A third feature amount is calculated based on the first feature amount multiplied by the first weight and the second feature amount, and the first feature amount stored in the storage unit according to the third feature amount. Updating section, and
A signal processing unit that executes signal processing for enhancing a part of audio signals of a plurality of channels using the updated first feature amount;
A speech enhancement device comprising A storage step of storing a first feature amount representing a feature of the first input signal in a storage unit;
A similarity calculation step of calculating a similarity between the first feature and a second feature representing a feature of the second input signal;
A weight calculation step of calculating a first weight for the first feature amount based on the similarity and the second feature amount;
A third feature amount is calculated based on the first feature amount multiplied by the first weight and the second feature amount, and the first feature amount stored in the storage unit according to the third feature amount. Update step to update
A signal processing step of performing signal processing using the updated first feature value;
Signal processing method including: On the computer,
A storage step of storing a first feature amount representing a feature of the first input signal in a storage unit;
A similarity calculation step of calculating a similarity between the first feature and a second feature representing a feature of the second input signal;
A weight calculation step of calculating a first weight for the first feature amount based on the similarity and the second feature amount;
A third feature amount is calculated based on the first feature amount multiplied by the first weight and the second feature amount, and the first feature amount stored in the storage unit according to the third feature amount. Update step to update
A signal processing step of performing signal processing using the updated first feature value;
A program to run a program.

Images