KR20190085062A - Apparatus and method for decomposing an audio signal using a ratio as separation characteristic - Google Patents

Abstract

An apparatus for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150) comprises: a block generator (110) for generating a time sequence of blocks of audio signal values; An audio signal analyzer (120) for determining a block characteristic of a current block of an audio signal and determining an average characteristic for a group of blocks, the group of blocks including at least two blocks; And a separator 130 for separating the current block into a background portion and a foreground portion in response to a ratio of a block characteristic of a current block to an average characteristic of a group of blocks, Background component, and the foreground component signal 150 includes the foreground portion of the current block.

Description

Apparatus and method for decomposing an audio signal using a ratio as separation characteristic

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to audio processing, and more particularly, to the decomposition of audio signals into background component signals and foreground component signals.

There are a considerable amount of references relating to audio signal processing, some of which are related to audio signal decomposition. Exemplary references include:

[1] S. Disch and A. Kuntz, A Dedicated Decorrelator for Parametric Spatial Coding of Applied-Like Audio Signals. Springer-Verlag, January 2012, pp. 355-363.

ackstr

m, and J. Robilliard, "The Transient Steering Decorrelator Tool in the Upcoming MPEG Unified Speech and Audio Coding Standard," in 131st Convention of the AES, New York, USA, 2011.[2] A. Kuntz, S. Disch, T. B

ackstr

m, and J. Robilliard, "The Transient Steering Decorrelator Tool in the Upcoming MPEG < / RTI > Speech and Audio Coding Standard," 131st Convention of the AES, New York, USA, 2011.

[3] A. Walther, C. Uhle, and S. Disch, "Transient Suppression in Blind Multi-channel Upmix Algorithms," Proceedings, 122nd AES Pro Audio Expo and Convention, May 2007.

[4] G. Hotho, S. van de Par, and J. Breebaart, "Multichannel coding of applause signals", EURASIP J. Adv. Signal Process, vol. 2008, Jan. 2008. [Online]. Available: http://dx.doi.org/10.1155/2008/531693

[5] D. FitzGerald, "Harmonic / Percussive Separation Using Median Filtering," Proceedings of the 13th International Conference on Digital Audio Effects (DAFx-10), Graz, Austria, 2010.

[6] J. P. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M. B. Sandler, "A Tutorial on Onset Detection in Music Signals," IEEE Transactions on Speech and Audio Processing, vol. 13, no. 5, pp. 1035-1047, 2005.

[7] M. Goto and Y. Muraoka, "Beat tracking based on multiple-agent architecture, a real-time beat tracking system for audio signals," Proceedings of the 2nd International Conference on Multiagent Systems, 1996, pp. 103-110.

[8] A. Klapuri, "Sound onset detection by applying psychoacoustic knowledge," in Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 6, 1999, pp. 3089-3092 vol.6.

Furthermore, WO 2010017967 discloses an apparatus for determining a spatial output multi-channel audio signal based on an input audio signal, the apparatus comprising a first decomposed signal that is a foreground signal portion and a second decomposed signal that is a background signal portion And a semantic decomposer for decomposing the input audio signal. Furthermore, the renderer is configured to render the foreground signal portion using amplitude panning and render the background signal portion by decorrelation. Finally, the first rendered signal and the second rendered signal are processed to obtain a spatial output multi-channel audio signal.

Furthermore, references [1] and [2] disclose a transient steering decorrelator.

European Application No. 16156200.4, which has not yet been published, initiates high resolution envelope processing. The high-resolution envelope processing is a tool for improved coding of signals consisting mainly of many dense transient events such as applause, raindrop sounds, and the like. On the encoder side, the tool analyzes the input signal, attenuates the high frequency portion of the transient events and thereby temporally flattens them, producing a small amount of side information, such as 1 to 4 kbps for the stereo signals It acts as a preprocessor with a high temporal resolution prior to the actual perceptual audio codec. On the decoder side, the tool acts as a post-processor after the audio codec by using the side information that was generated during encoding, by boosting the high frequency portion of the transient events and temporally shaping accordingly.

Upmix generally involves signal decomposition into direct and peripheral signal portions where the direct signal is panned between the loudspeakers and the peripheral portion is subjected to a decorrelation over a given number of channels And distributed. The remaining direct components, such as transients in the surrounding signals, result in a perceived impairment of the ambience in the upmixed sound scene. In [3], transient detection and processing are proposed that reduce the transients detected in the surrounding signal. One proposed method for transient detection involves a comparison between the frequency weighted sum of the bins in one time block and the weighted long time drive means for determining whether a particular block should be suppressed.

In [4], efficient spatial audio coding of applause signals is discussed. Both of the proposed downmix and upmix methods operate on the entire applause signal.

Furthermore, reference [5] discloses a harmonic / percussive separation, where the signals are applied to harmonic and percussive signal components by applying median filters to the spectrogram in the horizontal and vertical directions Separated.

Reference [6] expresses a tutorial that includes time domain approaches such as frequency domain approaches, envelope followers or energy followers in the context of onset detection. Reference [7] initiates power tracking, such as a sharp increase in power in the frequency domain, and reference [8] initiates novelty measurements for purposes of inception detection.

The separation of the signal into foreground and background signal portions as described in the prior art references is disadvantageous due to the fact that such known procedures can result in reduced audio quality of the resulting signal or the decomposed signals.

It is an object of the present invention to provide an improved concept for the purpose of decomposing an audio signal into a background component signal and a foreground component signal.

This object is achieved by an apparatus for decomposing an audio signal into a background component signal and a foreground component signal according to claim 1, a method for decomposing an audio signal into a background component signal and a foreground component signal according to claim 22, Or by a computer program according to claim 23.

In an aspect, an apparatus for decomposing an audio signal into a background component signal and a foreground component signal includes a block generator for generating a time sequence of blocks of audio signal values, an audio signal analyzer coupled to the block generator, And a separator connected to the analyzer. According to a first aspect, an audio signal analyzer is configured to determine a block characteristic of a current block of an audio signal and an average characteristic for a group of blocks, the group of blocks comprising at least two blocks, such as a preceding block, Subsequent blocks or even further preceding blocks or further subsequent blocks.

The separator is configured to separate the current block into a background portion and a foreground portion in response to a ratio of the block characteristic and the average characteristic of the current block. Thus, the background component signal includes the background portion of the current block, and the foreground component signal includes the foreground portion of the current block. Therefore, the current block is not simply determined to be either the background or the foreground. Instead, the current block is actually divided into a non-zero background portion and a non-zero foreground portion. This procedure typically reflects the situation where the foreground signal is always present in the signal alone, but is always coupled to the background signal component. Therefore, according to this first aspect of the present invention, when no threshold is reached or a specific threshold is reached by a ratio, regardless of whether or not a specific thresholding is performed, actual separation is performed, and in addition to the foreground part, Reflects the situation that is always maintained.

Furthermore, the separation is done by a very specific separation means, i.e. the ratio of the average characteristic derived from the group of blocks derived from the at least two blocks and the block characteristic of the current block. Thus, depending on the size of the group of blocks, a very slowly varying moving average or a very rapidly varying moving average can be set. For a large number of blocks in a group of blocks, the moving average changes relatively slowly, whereas for a small number of blocks in a group of blocks, the moving average changes very rapidly. Moreover, the use of the relationship between the characteristic from the current block and the average characteristic across the group of blocks means that if the ratio between the characteristics of such a block with respect to the average is at a particular value, The perceptual situation is recognized. However, according to this aspect, this particular value does not necessarily have to be a threshold. Instead, the ratio itself may already be used to perform quantitative separation of the current block into the background portion and the foreground portion. The high ratio causes the high portion of the current block to be the foreground portion, while the low percentage means that most or all of the current block remains in the background portion, and the current block has only a small foreground portion, or no foreground portion at all Which would result in a situation that does not exist.

Preferably, the amplitude-related characteristic is determined, and such amplitude-related characteristic, such as the energy of the current block, is compared to the average energy of the group of blocks to obtain a ratio, based on what separation is performed. In order to confirm that a background signal remains in response to the separation, a gain factor is determined, and then this gain factor is used to determine how much the average energy of a particular block remains in the background or noise type signal and, And controls which part goes into the foreground signal portion, which can be a transient signal, such as a clap signal or a raindrop signal.

In a further, second aspect of the invention, which may be used in addition to or in addition to the first aspect, the apparatus for decomposing an audio signal comprises a block generator, an audio signal analyzer and a separator. The audio signal analyzer is configured to analyze characteristics of a current block of the audio signal. The characteristic of the current block of the audio signal may be a ratio as discussed with respect to the first aspect, but alternatively it may also be a block characteristic derived only from the current block without any averaging. Moreover, the audio signal analyzer is configured to determine the variability of the characteristics within a group of blocks, wherein the group of blocks comprises at least two blocks, and preferably at least two preceding blocks with or without a current block, At least two subsequent blocks with or without a block, or at least two preceding blocks with or without a current block, and at least two subsequent blocks. In preferred embodiments, the number of blocks is greater than 30 or even 40.

Further, the separator is configured to separate the current block into a background portion and a foreground portion, wherein the separator determines a separation threshold based on the variability determined by the signal separator, and wherein the characteristic of the current block is a predetermined relationship , Such as to be greater than or equal to the isolation threshold, to separate the current block. Of course, if the threshold is defined to be some sort of inverse value, the predetermined relationship may be smaller than the relationship, or less than or equal to the relationship. Thus, if the property is within a predetermined relationship with the separation threshold, separation into the background portion and the foreground portion is performed, whereas if the property is not within a predetermined relationship with the separation threshold, Anger can always be done.

According to a second aspect, using a variable threshold that depends on the variability of the characteristics in the group of blocks, the separation may be a complete separation, i. E., When an entire block of audio signal values is introduced into the foreground component, If the predetermined relationship regarding the variable separation threshold is not satisfied, the entire block of audio signal values is similar to the background signal portion. In a preferred embodiment, this aspect is such that as soon as the variable threshold is found to be in predetermined relationship to the characteristic, non-binary separation is performed, i.e. only a part of the audio signal values enter the foreground signal portion, It is combined with the first aspect in that it is left behind.

Preferably, the separation of the portion for the foreground signal portion and for the background signal portion is determined based on the gain factor, i.e. the same signal values are eventually in the foreground signal portion and the background signal portion, but the energy of the signal values in the different portions Are determined by a separation gain that eventually depends on the characteristics such as the block characteristic of the current block itself or the ratio of the average characteristic for the group of blocks associated with the current block to the current block between the block characteristic for the current block do.

The use of variable thresholds reflects situations in which individuals perceive foreground signal portions, even when there is a small deviation from a very static signal, i. E. When a particular signal is considered to be very static, i. do. Subsequently, even a small variation is already recognized as being the foreground signal portion. However, in the presence of a highly fluctuating signal, the highly fluctuating signal itself is perceived to be a background signal component, and it appears that a small deviation from variations in this pattern is not perceived as being a foreground signal portion. Only stronger deviations from the mean or expected value are perceived as being the foreground signal portion. Thus, it is desirable to use very small separation thresholds for signals with small dispersion and to use higher separation thresholds for signals with high dispersion. However, when inverse values are taken into account, the situation is opposite to the above.

Both aspects, i.e., the foreground signal portion based on the ratio between the block characteristic and the average characteristic, and the first aspect having the non-binary separation into the background signal portion and the variable threshold dependent on the variability of the characteristic in the group of blocks The second aspect may be used separately from each other, or even together, i. E. In combination with each other. The latter alternative constitutes a preferred embodiment as described below.

Embodiments of the present invention relate to a system in which an input signal is decomposed into two signal components to which individual processing may be applied and the processed signals are resynthesized to form an output signal. The applause and also other transient signals can be seen as a superposition of distinctly perceptible transient applause events and a more noise background signal. It is advantageous to be able to apply individual processing to each signal portion in order to modify properties such as the ratio of the foreground and background signal density of such signals. Additionally, signal separation motivated by human perception is obtained. Moreover, the concept can also be used as a measurement device for measuring signal characteristics on the transmitter site, for example, and for restoring these characteristics on the receiver site.

Exemplary embodiments of the present invention do not aim to exclusively generate multi-channel spatial output signals. The mono input signal is decomposed and the individual signal portions are processed and reconstructed into a mono output signal. In some embodiments, the concept as defined in the first or second aspect outputs measurements or side information instead of an audible signal.

Additionally, the separation is based on a perceptual aspect and preferably a quantitative characteristic or value rather than a semantic aspect.

According to embodiments, the separation is based on a deviation of the instantaneous energy with respect to the average energy in the short time frame considered. Events with high energy deviations can be distinguished from background signals, although transient events having energy levels close to or below the average energy in such a time frame are not perceived as being substantially different from the background. This kind of signal separation employs the principle and allows processing closer to the perception of a person of transient events and closer to the perception of foreground events than background events.

Subsequently, preferred embodiments of the present invention are discussed with reference to the accompanying drawings.

1A is a block diagram of an apparatus for decomposing an audio signal in dependence on a ratio, according to a first aspect.
1B is a block diagram of one embodiment of a concept for decomposing an audio signal in dependence on a variable separation threshold, according to a second aspect.
1C illustrates a block diagram of an apparatus for decomposing an audio signal according to either the first, second, or aspects.
Figure 1D illustrates a preferred example of an audio signal analyzer and separator according to the first, second or aspects of the invention.
1e illustrates an embodiment of a signal separator according to the second aspect.
1F illustrates a conceptual description for decomposing an audio signal according to the first and second aspects and by reference to different thresholds.
Figure 2 illustrates two different schemes for separating the audio signal values of the current block into foreground and background components, according to either the first, second, or aspects.
Figure 3 illustrates a schematic representation of the generation of overlapping blocks generated by a block generator, and temporal domain foreground component signals and background component signals subsequent to the separation.
Figure 4A illustrates a first alternative for determining a variable threshold based on smoothing of raw variabilities.
Figure 4B illustrates the determination of a variable threshold based on smoothing of primitive thresholds.
Figure 4c illustrates different functions for mapping (smoothed) variabilities to thresholds.
Figure 5 illustrates a preferred implementation for determining variability as required in the second aspect.
Figure 6 illustrates a general overview of isolation, foreground processing and background processing, and subsequent signal re-synthesis.
Figure 7 illustrates the measurement and reconstruction of signal characteristics with or without metadata.
Figure 8 illustrates a block diagram for an encoder-decoder use case.

Figure 1A illustrates an apparatus for decomposing an audio signal into a background component signal and a foreground component signal. The audio signal is input to the audio signal input 100. The audio signal input is coupled to the block generator 110 to generate a time sequence of blocks of audio signal values output on line 112. Furthermore, the apparatus comprises an audio signal analyzer 120 for determining a block characteristic of a current block of an audio signal and additionally determining an average characteristic for a group of blocks, wherein the group of blocks comprises at least two Blocks. Preferably, the group of blocks includes at least one preceding block or at least one subsequent block, and additionally a current block.

Furthermore, the apparatus includes a separator 130 for separating the current block into a background portion and a foreground portion in response to a ratio of the block characteristic and the average characteristic of the current block. Therefore, the ratio of the block characteristic and the average characteristic of the current block is used as the characteristic, and on this basis, the separation of the current block of audio signal values is performed. In particular, the background component signal at the signal output 140 includes the background portion of the current block, and the foreground component signal output at the foreground component signal output 150 includes the foreground portion of the current block. The procedure illustrated in FIG. 1A is performed on a block-by-block basis, that is, one block of a time sequence of blocks is processed after another block, eventually a sequence of blocks of audio signal values input at the input 100 is processed , The corresponding sequence of blocks of the background component signal and the same sequence of blocks of the foreground component signal are present in lines 140 and 150 as discussed below with respect to FIG.

Preferably, the audio signal analyzer is configured to analyze the amplitude-related measurement as a block characteristic of the current block, and additionally, the audio signal analyzer 120 may also be configured to further analyze the amplitude- .

Preferably, the power measurement or energy measurement for the current block and the average power measurement or average energy measurement for the group of blocks are determined by the audio signal analyzer, and the ratio between these two values for the current block Is used by the separator 130.

Figure 2 illustrates the procedure performed by the separator 130 of Figure Ia in accordance with the first aspect. Step 200 expresses the determination of a characteristic according to a second aspect, which may for example be a ratio according to the first aspect or not necessarily but also a block characteristic.

In step 202, the separation gain is calculated from the ratio or characteristic. Then, a threshold comparison at step 204 may optionally be performed. If a threshold comparison is made at step 204, the result may be that the characteristic is in a predetermined relationship with the threshold. In this case, control proceeds to step 206. However, if it is determined in step 204 that the characteristic does not relate to a predetermined threshold, no separation is performed and control proceeds to the next block in the sequence of blocks.

의 함수인 분리 이득은 직접적으로 사용되는 것이 아니라 차이의 형태로 사용되며, 즉 함수는 1로부터 감산된다. 대안적으로, 배경 성분(N)은,

의 함수에 의해 오디오 신호 A(k,n)를 실제로 가중시킴으로써 직접 계산될 수 있다.According to the first aspect, a threshold comparison at step 204 may be performed or alternatively may not be performed as illustrated by the dashed line 208. In a case where the characteristic is determined in block 204 that the characteristic is in a predetermined relationship with the separation threshold, or alternatively in line 208, in any case, step 206 is performed wherein the audio signals are separated using a separation gain Weighted. To this end, step 206 receives audio signal values of the input audio signal in a temporal or, preferably, spectral representation, as illustrated by line 210. Then, depending on the application of the separation gain, the foreground component C is calculated as illustrated by the equation immediately below in Fig. Specifically, g _N and the ratio

Is not used directly but is used in the form of a difference, that is, the function is subtracted from 1. Alternatively, the background component (N)

Can be directly calculated by actually weighting the audio signal A (k, n)

FIG. 2 illustrates several possibilities for calculating foreground and background components that may all be performed by separator 130. FIG. One possibility is that both components are calculated using the separation gain. An alternative is that only the foreground component is computed using the separation gain and the background component N is computed by subtracting the foreground component from the audio signal values, as illustrated at 210. [ Another alternative, however, is that the background component N is directly calculated using the separation gain by the block 206, and then the background component N is subtracted from the audio signal A to produce the foreground component C finally . Thus, FIG. 2 illustrates three different embodiments for calculating the background component and the foreground component, each of these at least including the weighting of the audio signal values using the separation gain.

Subsequently, Figure IB is illustrated to illustrate the second aspect of the present invention which relies on a variable separation threshold.

1b representing the second aspect relies on the audio signal 100 input to the block generator 110 and the block generator is coupled to the audio signal analyzer 120 via the connection line 122. [ Moreover, the audio signal may be input directly to the audio signal analyzer via the additional connection line 111. The audio signal analyzer 120 is configured to, on the one hand, determine the characteristics of the current block of the audio signal and additionally determine the variability of the characteristics within the group of blocks, wherein the group of blocks includes at least two blocks , Preferably at least two preceding blocks or two subsequent blocks or at least two preceding blocks, at least two subsequent blocks and a current block.

Both the characteristics of the current block and the variability of the characteristics are forwarded to the separator 130 via the connection line 129. The separator is then configured to separate the current block into a background portion and a foreground portion to produce a background component signal 140 and a foreground component signal 150. [ In particular, the separator determines a separation threshold based on the variability determined by the audio signal analyzer according to the second aspect, and if the characteristic of the current block is in a predetermined relationship with the separation threshold, Component signal portion. However, if the characteristics of the current block are not in a predetermined relationship with the (variable) separation threshold, no separation of the current block is performed and the entire current block is forwarded or used or assigned as the background component signal 140.

Specifically, the separator 130 is configured to determine a first separation threshold for the first variability and a second separation threshold for the second variability, wherein the first separation threshold is lower than the second separation threshold, The variability is lower than the second variability, and the predetermined relationship is "greater ".

An example is illustrated in the left portion of FIG. 4c, where a first separation threshold is indicated at 401, a second separation threshold is indicated at 402, a first variability is indicated at 501, and a second variability is indicated at 502 . In particular, a reference is made to the piecewise linear function 410 representing the separation threshold, while the linear function 412 of the subdivision in FIG. 4C illustrates a release threshold to be described later. FIG. 4C illustrates a situation where, for increasing variability, there are thresholds that allow increasing thresholds to be determined. However, if the situation is implemented in such a way that, for example, the inverse threshold values for the threshold values of Figure 4c are taken, the separator may have a first separation threshold for the first variability and a second separation threshold for the second variability Wherein the first separation threshold is greater than the second separation threshold and the first variability is greater than the second variability, and in such a situation, the predetermined relationship is determined by the first Lower than "higher" as in the alternative.

Depending on the particular implementations, the separator 130 may use the table access in which the functions illustrated in the left or right portion of Figure 4C are stored, or using the first and second separation thresholds 401 and 402, (Variable) separation threshold is determined according to a monotone interpolation function that interpolates between the third variability 504 and the fourth variance 504 to obtain a third separation threshold 403 for the third variability 503 and a fourth threshold for the fourth variability 504 Wherein the first separation threshold 401 is associated with a first variability 501 and the second separation threshold 402 is associated with a second variability 502 and the third and fourth variabilities 503 , 504 are located between the first and second variabilities with respect to their values and the third and fourth separation thresholds 403,404 are associated with the first and second separation thresholds 401 , 402).

As illustrated in the left part of Figure 4c, the forged interpolation is a linear function, or as illustrated in the right portion of Figure 4c, the forged interpolation function may be a cube function with a degree greater than one, It is a power function.

Figure 6 shows a top-level block diagram of applause signal separation, processing, and synthesis of processed signals.

6 separates the input audio signal a (t) into a background signal n (t) and a foreground signal c (t), and the background signal separates the background audio signal a (T) and c '(t) are both input to the stage 602 and the foreground signal is input to the foreground processing stage 604, and following processing, both the signals n' (t)) < / RTI >

Preferably, based on the signal separation / decomposition of the input signal a (t) into the distinctively perceivable clap sounds c (t) and the more noise background signals n (t) Respectively. After processing, the modified foreground and background signals c '(t) and n' (t) are re-synthesized to generate an output signal a '(t).

Figure 1C illustrates a top-level diagram of the preferred clap separation stage. The clapping model is given in Equation 1 and illustrated in Figure 1f, where the clapping signal A (k, n) includes distinct and perceptible foreground clapping sounds C (k, n) and a more noise- Signal (N (k, n)). The signals are considered in the frequency domain with high temporal resolution, but k and n represent distinct frequency k and time n indexes of the short-time frequency transform, respectively.

In particular, the system of FIG. 1C includes additional signal separator stages such as the DFT processor 110 as a block generator, the functions of the audio signal analyzer 120 and separator 130 of FIG. 1A or 1B and the weighting device 152 A foreground detector that performs the functions discussed with respect to step 206 of FIG. 2, and a subtractor 154 that implements the functions illustrated in step 210 of FIG. 2. Further, from the corresponding frequency domain representation, there is provided a signal synthesizer for synthesizing the time domain foreground signal c (t) and the background signal n (t), wherein the signal synthesizer is configured to perform a DFT Blocks 160a and 160b.

The input signal including the applause input signal a (t), i.e., background components and clap components, is supplied to the foreground detector 150 as well as to a signal switch (not shown in FIG. 1C) , Frames corresponding to foreground applause sounds are identified. Detector stage 150 is separated to be supplied to the signal switch, the gain (g _{s (n))} and outputs the distinct and individually perceptible applause signal (C (k, n)) and the more noise type signal (N (k , n)). The signal switch is illustrated in block 170 to illustrate that only a binary switch, a particular frame or time / frequency tile, i.e. a particular frequency bin of a particular frame, is routed to either C or N, in accordance with the second aspect. According to a first aspect, the gain is used to separate each frame or several frequency bins of the spectral representation (A (k, n)) into foreground and background components, so that the block characteristics and average characteristics The entire frame or at least one or more time / frequency tiles or frequency bins are separated such that the corresponding beans in each of the signals C and N have the same value (g _{s (n)} ), , But with different amplitudes, where the relationship of the amplitudes depends on g _{s (n)} .

) 및 평균 에너지(

)를 초래한다.1D illustrates a more detailed embodiment of foreground detector 150 that specifically illustrates the functions of the audio signal analyzer. In one embodiment, the audio signal analyzer receives the spectral representation generated by the block generator with the DFT (Discrete Fourier Transform) block 110 of FIG. 1C. Moreover, the audio signal analyzer is configured to perform high-pass filtering using a predetermined, predetermined cross-over frequency, The audio signal analyzer 120 of FIG. 1A or 1B then performs an energy extraction procedure at block 172. The energy extraction procedure determines the instantaneous or current energy of the current block (

) And average energy (

).

The signal separator 130 of FIG. 1A or 1B then determines the ratio, as illustrated at 180, and additionally determines an adaptive or non-adaptive threshold, and the corresponding thresholding operation 182, .

Furthermore, when the adaptive thresholding operation according to the second aspect is performed, the audio signal analyzer additionally performs an envelope variability estimation as illustrated in block 174, and, as will be described below, (v (n)) is forwarded to the separator, and particularly to the adaptive thresholding processing block 182, to finally obtain the gain g _s (n).

) 및 순시(

) 에너지가 추정된다. 신호(X(k, n))의 순시 에너지들은

에 의해 주어지며, 여기서,

는 벡터 놈(vector norm)을 나타내고, 평균 에너지는,A flow diagram of the interior of the foreground signal detector is shown in FIG. If only the upper path is considered, this corresponds to the case without adaptive thresholding, but if the lower path is also considered, adaptive thresholding is possible. The signal fed to the foreground signal detector is high pass filtered and its average (

) And instantaneous (

) Energy is estimated. The instantaneous energies of the signal X (k, n)

, Where < RTI ID = 0.0 >

Represents the vector norm, and the average energy is the vector norm,

)은 다음에 따라 사용된다., Where w (n) is a weighted window with window length L _w = 2M + 1 applied to instantaneous energy estimates. As an indication of whether the distinctive applause is active in the input signal, the ratio of the instantaneous and average energy energy

) Is used as follows.

)를 초과하는 시간 인스턴스의 경우, 입력 신호로부터 뚜렷한 박수소리 부분을 추출하는 분리 이득이 1로 세팅되며; 결과적으로, 잡음형 신호가 이들 시간 인스턴스들에서 제로이다. 하드(hard) 신호 스위칭을 갖는 시스템의 블록 다이어그램이 도 1e에 도시된다. 잡음형 신호에서 신호 드롭 아웃(drop out)들을 피할 필요가 있다면, 보정 항(correction term)이 이득으로부터 감산될 수 있다. 양호한 시작점은 입력 신호의 평균 에너지가 잡음형 신호 내에 남아있게 하는 것이다. 이것은 이득으로부터

또는

를 감산함으로써 행해진다. 평균 에너지의 양은 또한, 얼마나 많은 평균 에너지가 잡음형 신호 내에 남아있는지를 제어하는 이득 g_N≥0을 도입함으로써 제어될 수 있다. 이것은 다음과 같은 분리 이득의 일반적인 형태를 유도한다:In a simpler case without adaptive thresholding, if the energy ratio exceeds the attack threshold (

), The separation gain for extracting the distinctive clapping portion from the input signal is set to one; As a result, the noise-like signal is zero at these time instances. A block diagram of a system with hard signal switching is shown in Figure IE. If it is necessary to avoid signal dropouts in a noise-like signal, the correction term can be subtracted from the gain. A good starting point is that the average energy of the input signal remains within the noise-like signal. From this gain

or

. The amount of average energy can also be controlled by introducing a gain g _N > 0 that controls how much of the average energy remains in the noise-like signal. This leads to the general form of the separation gain as follows:

In a further embodiment, the above equation is replaced by the following equation: < RTI ID = 0.0 >

이면, 뚜렷한 박수소리로 라우팅되는 신호의 양은 에너지 비율(

) 및 고정된 이득(g_N)에만 의존하여, 신호 의존적 연판정(soft decision)을 산출한다. 매우 양호하게 튜닝된 시스템에서, 에너지 비율이 공격 임계치들을 초과하는 시간 기간은 실제 트랜션트 이벤트만을 포착한다. 일부 경우들에서, 공격이 발생된 이후 더 긴 시간 기간의 프레임들을 추출하는 것이 바람직할 수 있다. 이것은, 예를 들어, 분리 이득이 다시 제로로 세팅되기 전에 공격 이후 에너지 비율 (

)이 감소해야 하는 레벨을 표시하는 릴리즈 임계치(

)를 도입함으로써 다음과 같이 행해질 수 있다.Note:

, The amount of signal routed to the distinctive applause is the energy ratio (

) And the fixed gain g _N , to produce a signal-dependent soft decision. In a very well tuned system, the time period over which the energy ratio exceeds the attack thresholds captures only the actual transient events. In some cases, it may be desirable to extract frames of a longer time period after an attack has occurred. This means, for example, that the energy rate after the attack (before the separation gain is again set to zero

≪ / RTI > indicating the level at which the < RTI ID = 0.0 &

) Can be introduced as follows.

In a further embodiment, the immediately preceding equation is replaced by the following equation: < RTI ID = 0.0 >

An alternative but more static method is to simply route a certain number of frames to a distinctive applause signal after the detected attack.

및

을 각각 초래할 수 있다. 임계치들은, 박수 입력 신호의 엔벨로프의 가변성의 추정에 의해 제어되며, 여기서, 높은 가변성은 뚜렷하고 개별적으로 인지가능한 박수소리들의 존재를 표시하고, 다소 낮은 가변성은 더 잡음형이고 정적인 신호를 표시한다. 가변성 추정은 시간 도메인 뿐만 아니라 주파수 도메인에서 행해질 수 있다. 이러한 경우, 바람직한 방법은 주파수 도메인에서 추정을 행하는 것이며:To increase the flexibility of thresholding, the thresholds are selected in a signal-adaptive manner,

And

Respectively. The thresholds are controlled by estimating the variability of the envelope input signal's envelope where the high variability indicates the presence of distinct, individually recognizable clapping sounds and the somewhat lower variability indicates a more noise-like, static signal. The variability estimation can be done in the frequency domain as well as in the time domain. In such a case, a preferred method is to make an estimate in the frequency domain:

는 분산 연산을 나타낸다. 더 안정된 신호를 산출하기 위해, 추정된 가변성은 저역 통과 필터링에 의해 평활화되어, 다음과 같은 최종 엔벨로프 가변성 추정을 산출하며,here,

Represents a distributed operation. In order to produce a more stable signal, the estimated variability is smoothed by low-pass filtering to yield a final envelope variability estimate,

Here, * denotes a convolution. The mapping of the envelope variability to the corresponding threshold values can be done by the mapping functions f _attack (x) and f _release (x) as follows:

In one embodiment, the mapping function may be realized as clipped linear functions corresponding to linear interpolation of the thresholds. The configuration for this scenario is shown in Figure 4c. Moreover, cubic mapping functions or functions, which generally have a higher order, can also be used. In particular, saddle points can be used to define extra threshold levels for variability values between values defined for thin clapping and dense clapping. This is illustrated by way of example on the right side of Figure 4c.

The separated signals are obtained by:

Figure 1F illustrates the mathematical formulas discussed above with respect to and in an overview of the functional blocks of Figures 1A and 1B.

Further, FIG. 1F illustrates a situation where a single threshold or a double threshold is applied, or no threshold is applied, depending on the particular embodiment.

Furthermore, adaptive thresholds may be used, as illustrated with respect to equations (7) through (9) in FIG. 1F. Naturally, any single single threshold is used as a single adaptive threshold. Then, only equation (8) will be active, and equation (9) will not be active. However, in certain preferred embodiments it is desirable to perform dual adaptive thresholding to implement the features of the first and second aspects together.

7 and 8 illustrate additional implementations as to how the invention may be implemented for a particular application of the present invention.

Particularly, the left part of FIG. 7 illustrates a signal characteristic measuring device 700 for measuring the signal characteristics of the background component signal or the foreground component signal. In particular, the signal characteristic measurement 700 may be configured to determine the foreground density in block 702, which illustrates a foreground density calculator using the foreground component signal, or, alternatively or additionally, is used to perform foreground contour calculations using a foreground prominence calculator 704 that calculates a fraction of the foreground associated with the original input signal a (t).

)에 의존한다.Alternatively, there may be a foreground processor 604 and a background processor 602, as illustrated in the right-hand portion of Figure 7, where these processors, contrary to Figure 6, Metadata, or any other metadata that may be useful for performing foreground processing and background processing

).

The separated applause signal portions can be supplied to the measurement stages where specific (perceptually synchronized) characteristics of the transient signals can be measured. An exemplary configuration for such use cases is shown in Fig. 7A, where the energy ratio of the foreground applause sounds on the total signal energy as well as the density of the distinctively perceptible foreground applause sounds is estimated.

)를 추정하는 것은 초당 이벤트 레이트, 즉 초당 검출된 박수소리들의 수를 카운팅함으로써 행해질 수 있다. 전경 현저성(

)은 다음과 같이, 추정된 전경 박수소리 신호(C(n) 및 A(n))의 에너지 비율에 의해 주어진다:Foreground density (

) Can be done by counting the event rate per second, i.e. the number of detected applause sounds per second. Foreground

Is given by the energy ratio of the estimated foreground applause sound signals C (n) and A (n) as follows:

및 파선들은 사이드 정보를 나타낸다.A block diagram of the reconstruction of the measured signal characteristics is shown in Figure 7B,

And dashed lines indicate side information.

In the previous embodiment, only the signal characteristics were measured, but the system is used to modify the signal characteristics. In one embodiment, foreground processing may output a reduced number of detected foreground clapping sounds, resulting in a density modification towards the resulting output signal of lower density. In another embodiment, the foreground processing may output an increased number of foreground applause sounds, for example, by adding a delayed version of the foreground applause sound signal to itself, resulting in a density modification towards the increased density. Moreover, by applying weights in the individual processing stages, the balance of the foreground applause sounds and background in the form of noise can be modified. Additionally, any processing, such as filtering, reverberation, delay, etc., in both paths can be used to modify the characteristics of the applause signal.

Furthermore, Figure 8 relates to an encoder stage for encoding an encoded representation of a foreground component signal for transmission or storage and a foreground component signal and a background component signal to obtain a separate encoded representation of the background component signal. In particular, a foreground encoder is illustrated at 801 and a background encoder is illustrated at 802. The separately encoded representations 804 and 806 are forwarded to the decoder-side device 808 consisting of a foreground decoder 810 and a background decoder 812 that eventually decode distinct representations, Are combined by the combiner 606 to finally output the decoded signal a '(t).

Subsequently, further preferred embodiments are discussed with respect to Fig. In particular, FIG. 3 illustrates a schematic representation of an input audio signal given on a time line 300, wherein the schematic representation illustrates the situation of temporally overlapping blocks. Illustrated in FIG. 3 is a situation where there is an overlap range 302 of 50%. Other overlap ranges are also available, such as multi-overlap ranges with overlap ranges of more than 50% or less than 50% (where less than 50% overlap).

In the embodiment of FIG. 3, the block typically has fewer than 600 sampling values and preferably only 256 or only 128 sampling values to obtain a high temporal resolution.

The overlapping blocks exemplarily illustrated are, for example, made up of a preceding block 303 or a current block 304 that overlaps with the succeeding block 305 and within the overlapping range. Thus, if the group of blocks includes at least two preceding blocks, the group of such blocks consists of a preceding block 303 for the current block 304, and an additional preceding block denoted by degree 3 in FIG. Furthermore, and similarly, if the group of blocks comprises at least two subsequent blocks (in time), these two following blocks include a subsequent block 305 denoted by degree 6 and an additional block 7 exemplified by degree 7 will be.

These blocks are formed by a block generator 110 which also preferably performs a time-spectral transform, such as the DFT or FFT (Fast Fourier Transform) previously described, for example.

The result of the time-spectrum transform is a sequence of spectral blocks I through VIII, wherein each spectral block illustrated in FIG. 3 below block 110 corresponds to one of the eight blocks of time line 300 .

Preferably, the separation is then performed in the frequency domain, i.e. using a spectral representation, wherein the audio signal values are spectral values. Following separation, a foreground spectral representation consisting of blocks I through VIII and a background representation consisting of I through VIII are obtained again. Of course, and depending on the thresholding operation, each block of the foreground representation following separation 130 need not necessarily have values that are different from zero. Preferably, however, it is at least confirmed by the first aspect of the invention that each block of the spectral representation of the background component has values different from zero to avoid dropout of energy in the background signal component.

For each component, foreground component and background component, a spectral-temporal transformation is performed as discussed in the context of FIG. 1C, and a subsequent fade-out / fade-in the fade-in is performed for both the components as illustrated in block 161a and block 161b, respectively, for foreground and background components. Thus, eventually both the foreground signal and the background signal have the same length L as the original audio signal prior to separation.

Preferably, as illustrated in FIG. 4B, the separator 130, which computes variabilities or thresholds, is smoothed.

In particular, step 400 illustrates the determination of the ratio or general characteristic between the block characteristic and the average characteristic for the current block, as illustrated in block 400.

At block 402, the raw variability is calculated for the current block. At block 404, the raw variabilities for the preceding or subsequent blocks are computed to obtain a sequence of raw variabilities by the output of blocks 402 and 404. At block 406, the sequence is smoothed. Thus, at the output of block 406, there is a smoothed sequence of variabilities. The variabilities of the smoothed sequence are mapped to corresponding adaptive thresholds as illustrated in block 408 to obtain a variable threshold for the current block.

An alternative embodiment is illustrated in FIG. 4B, where the thresholds are smoothed, as opposed to smoothing the variabilities. To do this, once again, the property / ratio for the current block is determined as illustrated in block 400.

At block 403, the sequence of variabilities is computed, for example, using Equation 6 in Figure IF for each current block denoted by the integer m.

At block 405, the sequence of variabilities is mapped to the sequence of primitive thresholds according to (8) and (9), but to the non-smoothed variants as opposed to (7) in FIG.

At block 407, the sequence of primitive thresholds is smoothed to ultimately obtain a (smoothed) threshold for the current block.

Subsequently, FIG. 5 is discussed in more detail to illustrate the different ways to calculate the variability of the characteristics within a group of blocks.

Once again, in step 500, a characteristic or ratio between the current block characteristic and the average block characteristic is calculated.

In step 502, an average or generally expected value over the properties / ratios for the group of blocks is calculated.

At block 504, the differences between the properties / ratios and the average / expected value are calculated and the addition of the specific values derived from the differences or differences, as illustrated in block 506, is preferably normalized . If squared differences are added, the sequence of steps 502, 504, 506 reflects the computation of variance as described with respect to equation (6). However, for example, if different powers of the differences or different powers of 2 are added together, different statistical values derived from the differences between the properties and the average / expected value are used as variability.

Alternatively, however, as illustrated in step 508, differences between time-following characteristics / ratios for adjacent blocks are also calculated and used as a variability measure. Thus, block 508 determines the variability that is dependent on the change from block to block, not from the average value, where the differences between the properties for adjacent blocks, as illustrated in Figure 6, , Their magnitudes, or their powers, to finally obtain another value from the variability that is different from the variance. It will be apparent to those skilled in the art that other variability measures that are different from those discussed with respect to FIG. 5 may also be used.

Subsequently, examples of embodiments that may be used separately from the following examples or in combination with any of the following examples are defined:

1. An apparatus for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150)

A block generator (110) for generating a time sequence of blocks of audio signal values;

An audio signal analyzer (120) for determining a block characteristic of a current block of an audio signal and determining an average characteristic for a group of blocks, the group of blocks including at least two blocks; And

And a separator (130) for separating the current block into a background portion and a foreground portion in response to a ratio of a block characteristic of a current block and an average characteristic of a group of blocks,

The background component signal 140 includes the background portion of the current block and the foreground component signal 150 includes the foreground portion of the current block as the background component signal 140 and the foreground component signal 150 ). ≪ / RTI >

Example 2 [0050] In Example 1,

The audio signal analyzer includes an audio signal (100) configured to analyze an amplitude-related measurement as a characteristic of a current block and to analyze an amplitude-related characteristic as an average characteristic for a group of blocks, Into component signal (150).

Example 3 In Example 1 or Example 2,

The audio signal analyzer 120 is adapted to analyze the power measurement or energy measurement for the current block and to analyze the average power measurement or the average energy measurement for a group of blocks by comparing the audio signal 100 with the background component signal 140 And a foreground component signal (150).

Embodiment 4 In any one of Embodiments 1 to 3,

The separator 130 calculates the separation gain from the ratio, weighs the audio signal values of the current block using the separation gain to obtain the foreground portion of the current frame, and determines the background component, Or

The separator is configured to calculate the separation gain from the ratio, weight the audio signal values of the current block using the separation gain to obtain the background portion of the current frame, and determine the foreground component so that the foreground component signal constitutes the remaining signal To the background component signal (140) and the foreground component signal (150).

Embodiment 5 In any one of Embodiments 1 to 4,

The separator 130 is configured to calculate the separation gain using the weighting factor using a predetermined weighting factor different from zero by multiplying the audio signal 100 by the background component signal 140 and the foreground component signal 150 ). ≪ / RTI >

Example 6 In Example 5,

을 사용하여 분리 이득을 계산하도록 구성되며,The separator 130,

Is used to calculate the separation gain,

는 비율이고, p는 제로보다 크고 정수 또는 비-정수인 거듭제곱이고, n은 블록 인덱스이고, max는 최대 함수인, 오디오 신호(100)를 배경 성분 신호(140) 및 전경 성분 신호(150)로 분해하기 위한 장치.g _N is a predetermined factor,

Where n is the block index and max is the maximum function of the audio signal 100 as the background component signal 140 and the foreground component signal 150 Apparatus for disassembly.

Embodiment 7 In any one of Embodiments 1 to 6,

The separator 130 is configured to compare the ratio of the current block to the threshold and to separate the current block if the ratio of the current block is in a predetermined relationship with the threshold,

The separator 130 is configured not to separate additional blocks,

The additional block has a ratio that does not have a predetermined relationship with the threshold and the additional block decomposes the audio signal 100 into the background component signal 140 and the foreground component signal 150, / RTI >

Example 8 [0141] In the same manner as in Example 7,

The separator 130 is configured to separate subsequent blocks following the current block in time using a ratio of subsequent blocks to an additional release threshold,

The additional release threshold is determined by decomposing the audio signal 100 into a background component signal 140 and a foreground component signal 150, wherein the audio signal 100 is set such that a block ratio that is not in a predetermined relationship to the threshold is in a predetermined relationship with an additional release threshold .

[Example 9]

The predetermined relationship is "larger ", the release threshold is lower than the separation threshold, or

The apparatus for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150), wherein the predetermined relationship is "lower" and the release threshold is greater than a separation threshold.

Embodiment 10. In any one of the embodiments 1 to 9,

Block generator 110 may be configured to determine temporally overlapping blocks of audio signal values, or

Wherein the temporally overlapping blocks have a plurality of sampled values less than or equal to 600, into the background component signal (140) and the foreground component signal (150).

Embodiment 11. In any one of embodiments 1 to 10,

Block generator is configured to perform a block-wise transform of the time domain audio signal into the frequency domain to obtain a spectral representation for each block,

The audio signal analyzer is configured to calculate a property using a spectral representation of the current block,

The separator 130 divides the spectrum representation into a background portion and a foreground portion so that the background portion and the foreground portion of the spectrum bins corresponding to the same frequency have a spectral value different from zero And,

The apparatus for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150), wherein the relationship between the spectral value of the foreground part and the spectral value of the background part in the same frequency bin depends on the ratio.

12. Embodiment 12 In any one of embodiments 1 to 11,

Block generator 110 is configured to perform block-to-unit conversions to the frequency domain of the time domain to obtain a spectral representation for each block,

Blocks adjacent in time overlap in the overlap range 302,

The apparatus further includes signal synthesizers (160a, 161a, 160b, 161b) for synthesizing background component signals and synthesizing foreground component signals,

The signal synthesizer performs frequency-to-time transforms (161a, 160a, 160b) on the background component signal and the foreground component signal and performs cross-fade temporal representations of temporally- (150) for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150), wherein the audio component (161a, 161b) is configured to obtain a time domain foreground component signal and a separate time domain background component signal .

Embodiment 13. In any one of embodiments 1 to 12,

The audio signal analyzer 120 is configured to determine an average characteristic for a group of blocks using a weighted addition of individual characteristics of blocks within a group of blocks, Into component signal (150).

Embodiment 14. In any one of embodiments 1 to 13,

The audio signal analyzer 120 is configured to perform a weighted addition of individual characteristics of the blocks within the group of blocks,

The weighting value for the property of the block nearest in time to the current block is greater than the weighting value for the property of the additional block that is closest in time to the current block, and the audio signal 100 is divided into the background component signal 140 and the foreground component signal 150 ). ≪ / RTI >

Example 15. In Example 13 or Example 14,

The audio signal analyzer 120 is configured to determine a group of blocks such that the group of blocks comprises at least 20 blocks or at least 20 blocks following the current block before the corresponding block, Into a background component signal (140) and a foreground component signal (150).

Embodiment 16. In any one of the embodiments 1 to 15,

The audio signal analyzer is configured to use the normalized value in dependence on the number of blocks in the group of blocks or in dependence on the weighted values for the blocks in the group of blocks, Into a foreground component signal (150).

Embodiment 17. In any one of the embodiments 1 to 16,

Further comprising a signal characteristic meter (702, 704) for measuring signal characteristics of at least one of the background component signals or the foreground component signals, wherein the background component signal (140) and the foreground component signal (150).

Example 18 [0141] In the same manner as in Example 17,

The signal characteristic meter is configured to determine the foreground density 702 using the foreground component signal or to determine the foreground prominence 704 using the foreground component signal and the audio input signal. ) Into a background component signal (140) and a foreground component signal (150).

Embodiment 19. In any one of embodiments 1 to 18,

The foreground component signal includes clap signals,

The apparatus corrects the energy relationship between the background component signal, which is a noise type signal, and the foreground clapping sound signal, by increasing the number of applause sounds or by reducing the number of applause sounds or by applying a weight to the foreground component signal or background component signal To a background component signal (140) and a foreground component signal (150), the audio signal (100) further comprising a signal characteristic modifier for modifying the foreground component signal.

Embodiment 20. In any of the embodiments 1 to 19,

Further comprising a blind upmixer for upmixing the audio signal with a representation having a number of output channels that is greater than the number of channels of the audio signal,

The upmixer is configured to spatially distribute the foreground component signal to the output channels and to correlate the foreground component signals within the plurality of output channels to spectrally distribute the background component signal to the output channels,

Wherein the background component signals in the output channels are less correlated or correlated than the foreground component signals into the background component signal (140) and the foreground component signal (150).

Embodiment 21. In any one of embodiments 1 to 20,

An encoder stage 804 for separately encoding the foreground component signal and the background component signal to obtain an encoded representation of the foreground component signal 804 and a separate encoded representation of the background component signal 806 for transmission or storage or decoding 801, 802) for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150).

22. A method of decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150)

Generating (110) a time sequence of blocks of audio signal values;

Determining a block characteristic of a current block of an audio signal and determining an average characteristic for the group of blocks (120), the group of blocks including at least two blocks; And

(130) separating the current block into a background portion and a foreground portion in response to a ratio of a block characteristic of a current block to an average characteristic of a group of blocks,

The background component signal 140 includes the background portion of the current block and the foreground component signal 150 includes the foreground portion of the current block as the background component signal 140 and the foreground component signal 150 ). ≪ / RTI >

Subsequently, further examples are defined which may be used separately from the above examples or in combination with any of the above examples:

1. An apparatus for decomposing an audio signal into a background component signal and a foreground component signal,

A block generator (110) for generating a time sequence of blocks of audio signal values;

An audio signal analyzer (120) for determining a characteristic of a current block of an audio signal and for determining a variability of the characteristic within a group of blocks comprising at least two of the sequences of blocks; And

And a separator 130 for separating the current block into a background portion 140 and a foreground portion 150,

The separator 130 determines 182 the separation threshold based on the variability and determines the current block as the background component signal 140 and the foreground component signal 150 if the characteristic of the current block is in a predetermined relationship with the separation threshold. Or when the characteristic of the current block is in a predetermined relationship with the separation threshold, the entire current block is determined as the foreground component signal, or if the characteristic of the current block is not in a predetermined relationship with the separation threshold, And to determine the current block as a background component signal.

Example 2 [0050] In Example 1,

The separator 130 is configured to determine a first separation threshold 401 for the first variability 501 and a second separation threshold 402 for the second variability 502,

The first separation threshold 401 is lower than the second separation threshold 402 and the first variability 501 is lower than the second variability 502 and the predetermined relationship is greater or the first separation threshold is lower than the second variance 502, Wherein the first variability is lower than the second variability (502), and the predetermined relationship is lower, wherein the first variability is lower than the separation threshold, and the first variability is lower than the second variability (502).

Example 3 In Example 1 or Example 2,

The separator 130 determines the separation threshold using a monotone interpolation function that interpolates between the first separation threshold 401 and the second separation threshold 402 or using a table access to determine the third variability 503 A third separation threshold 403 is obtained for the fourth variability 504 and a fourth separation threshold 404 is obtained for the fourth variance 504,

The first separation threshold 401 is associated with a first variability 501 and the second separation threshold 402 is associated with a second variability 502,

The third variability 503 and the fourth variability are located between the first variability 501 and the second variability 502 with respect to their values and the third and fourth separation thresholds 403 and 404 are located between Wherein the first separation threshold value (401) and the second separation threshold value (402) are in relation to their values.

Example 4 [0060] In Example 3,

Wherein the monotone interpolation function is a linear function or a quadratic function or a power function having a cubic function or a degree greater than three, into an background component signal and a foreground component signal.

Embodiment 5 In any one of Embodiments 1 to 4,

The separator 130 determines (405) a raw separation threshold based on the variability of the characteristics of the current block and determines at least one additional raw separation threshold based on the variability of the at least one preceding or succeeding block (405), and determining (407) a separation threshold for the current block by smoothing the sequence of raw separation thresholds, the sequence comprising a raw separation threshold and at least one additional raw separation threshold, or

The separator 130 is configured to determine the raw variability 402 of the characteristic for the current block and additionally to compute 404 the raw variability for the preceding or subsequent block, To obtain a smoothed sequence of variabilities, and to determine isolation thresholds based on the smoothed variability of the current block, and to determine the isolation thresholds based on the smoothed variability of the current block. ≪ RTI ID = 0.0 & And to decompose the audio signal into a background component signal and a foreground component signal.

Embodiment 6 In any one of Embodiments 1 to 5,

The audio signal analyzer 120 is configured to determine the variability by calculating a characteristic of each block within a group of blocks to obtain a group of characteristics and by calculating a variance of the group of characteristics,

Wherein the variability corresponds to a variance or depends on a variance of a group of characteristics, for decomposing an audio signal into a background component signal and a foreground component signal.

Embodiment 7 In any one of Embodiments 1 to 6,

The audio signal analyzer (120)

The average or expected characteristics 502 and the variability 504 using the differences between the characteristics within the group of characteristics and the average or expected characteristics,

To calculate the variability using differences 508 between the characteristics of the group of characteristics that follow in time

Wherein the audio signal is composed of a background component signal and a foreground component signal.

Embodiment 8. In any one of the embodiments 1 to 7,

The audio signal analyzer 120 is configured to calculate the variability of the property within a group of properties including at least two blocks preceding the current block or at least two blocks following the current block, Signal and foreground component signal.

Embodiment 9 In any one of the embodiments 1 to 8,

The audio signal analyzer (120) is configured to calculate a variability of a characteristic within a group of blocks of at least 30 blocks, for decomposing an audio signal into a background component signal and a foreground component signal.

Embodiment 10. In any one of the embodiments 1 to 9,

The audio signal analyzer 120 is configured to calculate a characteristic as a ratio of the block characteristic of the current block and the average characteristic to a group of blocks comprising at least two blocks,

The separator (130) is configured to compare the ratio to a separation threshold determined based on a variability of the ratio associated with a current block in the group of blocks, for decomposing the audio signal into a background component signal and a foreground component signal.

Example 11 [0141] In Example 10,

The audio signal analyzer (120) is configured to use the same group of blocks for calculation of the average characteristic and for calculation of variability, for decomposing the audio signal into a background component signal and a foreground component signal.

12. Embodiment 12 In any one of embodiments 1 to 11,

The audio signal analyzer is configured to analyze an amplitude-related measurement as a characteristic of a current block and to analyze the amplitude-related characteristic as an average characteristic for a group of blocks, for decomposing the audio signal into a background component signal and a foreground component signal Device.

Embodiment 13. In any one of embodiments 1 to 12,

The separator 130 calculates the separation gain from the characteristic, weighs the audio signal values of the current block using the separation gain to obtain the foreground portion of the current frame, and determines the background component, Or

The separator 130 calculates the separation gain from the characteristic, weights the audio signal values of the current block using the separation gain to obtain the background portion of the current frame, determines the foreground component, Configured to compose an audio signal into a background component signal and a foreground component signal.

Embodiment 14. In any one of embodiments 1 to 13,

The separator 130 is configured to separate subsequent blocks subsequent to the current block in time using a comparison of the characteristics of the following blocks with an additional release threshold,

Wherein the additional release threshold is set such that a property that is not in a predetermined relationship to the threshold is set to a predetermined relationship with an additional release threshold.

Example 15. In Example 14,

The separator 130 is configured to determine the release threshold based on the variability and to separate the subsequent blocks if the characteristics of the current block are in an additional predetermined relationship to the release threshold. .

Example 16 In Example 14 or Example 15,

The predetermined relationship is "larger ", the release threshold is lower than the separation threshold, or

The apparatus for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150), wherein the predetermined relationship is "lower" and the release threshold is greater than a separation threshold.

Embodiment 17. In any one of the embodiments 1 to 16,

Block generator 110 may be configured to determine temporally overlapping blocks of audio signal values, or

Wherein the temporally overlapping blocks have a plurality of sampled values less than or equal to 600, and for decomposing the audio signal into a background component signal and a foreground component signal.

Embodiment 18. In any one of embodiments 1 to 17,

Block generator is configured to perform a block-wise transform of the time domain audio signal into the frequency domain to obtain a spectral representation for each block,

The audio signal analyzer is configured to calculate a property using a spectral representation of the current block,

The separator 130 divides the spectrum representation into a background portion and a foreground portion so that the background portion and the foreground portion of the spectrum bins corresponding to the same frequency have a spectral value different from zero And,

A device for decomposing an audio signal into a background component signal and a foreground component signal, wherein the relationship between the spectral value of the foreground part and the spectral value of the background part in the same frequency bin is characteristic.

Embodiment 19. In any one of embodiments 1 to 18,

The audio signal analyzer 120 is configured to calculate a characteristic using a spectral representation of a current block to compute a variance for a current block using a spectral representation of a group of blocks, A device for decomposition into signals.

20. A method for decomposing an audio signal into a background component signal and a foreground component signal,

Generating (110) a time sequence of blocks of audio signal values;

Determining a characteristic of a current block of the audio signal and determining a variability of the characteristic within the group of blocks comprising at least two of the sequences of blocks (120); And

(130) the current block into a background portion (140) and a foreground portion (150)

The separation threshold is determined based on the variability, and if the current block has a predetermined relationship with the separation threshold, the current block is separated into the background component signal 140 and the foreground component signal 150, The entire current block is determined as the foreground component signal or the current current block is determined as the background component signal if the characteristic of the current block is not in a predetermined relationship with the separation threshold, A method for decomposing an audio signal into a background component signal and a foreground component signal.

The encoded audio signal of the present invention may be stored on a digital or non-temporal storage medium, or may be transmitted on a transmission medium such as a wireless transmission medium or on a wired transmission medium such as the Internet.

Although some aspects have been described in the context of a device, it is clear that these aspects also represent a description of the corresponding method, where the block or device corresponds to a feature of the method step or method step. Similarly, the aspects described in the context of a method step also represent a corresponding block or item or description of the feature in the corresponding device.

Depending on the specific implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be implemented in a digital storage medium, such as a floppy disk, a DVD, a CD, a ROM, a PROM, or the like, in which electronically readable control signals cooperate (or may cooperate) , EPROM, EEPROM or FLASH memory.

Some embodiments in accordance with the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system to perform one of the methods described herein.

In general, embodiments of the present invention may be implemented as a computer program product having program code, wherein the program code is operated to perform one of the methods when the computer program product is run on a computer. The program code may be stored on, for example, a machine readable carrier.

Other embodiments include a computer program for performing one of the methods described herein, stored on a machine readable carrier or non-temporary storage medium.

That is, therefore, an embodiment of the method of the present invention is a computer program having program code for performing one of the methods described herein when the computer program is run on a computer.

Thus, a further embodiment of the methods of the present invention is a data carrier (or digital storage medium, or computer-readable medium) comprising a computer program (recorded on top) for performing one of the methods described herein, to be.

Thus, a further embodiment of the method of the present invention is a sequence of data streams or signals representing a computer program for performing one of the methods described herein. The sequence of data streams or signals may be configured to be transmitted, for example, over the Internet, e.g., via a data communication connection.

Additional embodiments include a processing means, e.g., a computer, or a programmable logic device, configured or adapted to perform one of the methods described herein.

Additional embodiments include a computer on which a computer program for performing one of the methods described herein is installed.

In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the field programmable gate array may cooperate with the microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device.

The embodiments described above are merely illustrative of the principles of the present invention. It will be appreciated that variations and modifications of the arrangements and details described herein will be apparent to those skilled in the art. Accordingly, it is intended that the specific details presented by the description and the description of the embodiments herein be limited only by the scope of the imminent patent claims.

Claims (23)

An apparatus for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150)
A block generator (110) for generating a time sequence of blocks of audio signal values;
An audio signal analyzer (120) for determining a block characteristic of a current block of the audio signal and determining an average characteristic for a group of blocks, the group of blocks including at least two blocks; And
And a separator (130) for separating the current block into a background portion and a foreground portion in response to a ratio of a block characteristic of the current block and an average characteristic of the group of blocks,
The background component signal 140 includes a background portion of the current block and the foreground component signal 150 comprises a foreground portion of the current block, Into component signal (150). The method according to claim 1,
The audio signal analyzer is configured to analyze an amplitude-related measurement as a characteristic of the current block and to analyze the amplitude-related characteristic as an average characteristic for a group of the blocks. 140) and a foreground component signal (150). 3. The method according to claim 1 or 2,
The audio signal analyzer (120) is configured to analyze a power measurement or an energy measurement for the current block, and to analyze an average power measurement or an average energy measurement for the group of blocks. Into a signal (140) and a foreground component signal (150). 4. The method according to any one of claims 1 to 3,
The separator 130 calculates a separation gain from the ratio and obtains the foreground portion of the current block by weighting the audio signal values of the current block using the separation gain, Signal is configured to configure the remaining signal, or
Wherein the separator calculates the separation gain from the ratio and obtains the background portion of the current block by weighting the audio signal values of the current block using the separation gain and determines a foreground component, To the background component signal (140) and the foreground component signal (150), the audio signal (100) being configured to compose the remaining signal. 5. The method according to any one of claims 1 to 4,
The separator (130) is configured to calculate the separation gain using weighting the ratio using a predetermined weighting factor different from zero, wherein the separator (130) is configured to calculate the separation gain using the background component signal (140) and the foreground component signal (150). 6. The method of claim 5,
The separator (130)

To calculate the separation gain,
Wherein g _N is the predetermined factor,

Where n is the block index and max is the maximum function, the audio signal 100 is divided into a background component signal 140 and a foreground component signal < RTI ID = 0.0 > 150). 7. The method according to any one of claims 1 to 6,
The separator 130 is configured to compare the ratio of the current block with the threshold and separate the current block if the ratio of the current block is in a predetermined relationship with the threshold,
The separator 130 is configured not to separate additional blocks,
Wherein the additional block has a ratio that does not have a predetermined relationship with the threshold such that the additional block includes an audio signal 100 that completely belongs to the background component signal 140 as the background component signal 140 and the foreground component signal 140 150). 8. The method of claim 7,
The separator 130 is configured to separate subsequent blocks subsequent to the current block in time using a ratio of the following blocks to an additional release threshold,
Wherein the additional release threshold is set such that an audio signal (100) is divided into a background component signal (140) and a foreground component signal (150), wherein the background component signal (140) and the foreground component signal (150) are set such that a block ratio, which is not in a predetermined relationship with the threshold, . 9. The method of claim 8,
The predetermined relationship is "greater ", the release threshold is lower than the separation threshold, or
Wherein the predetermined relationship is a " lower "and the release threshold is greater than the separation threshold, into an background component signal (140) and a foreground component signal (150). 10. The method according to any one of claims 1 to 9,
The block generator 110 may be configured to determine temporally overlapping blocks of audio signal values,
Wherein the temporally overlapping blocks have a plurality of sampled values less than or equal to 600. The apparatus of claim 1, 11. The method according to any one of claims 1 to 10,
Wherein the block generator is configured to perform block-wise transforms to a frequency domain of a time domain audio signal to obtain a spectral representation for each block,
Wherein the audio signal analyzer is configured to calculate a property using a spectral representation of the current block,
The separator (130) separates the spectral representation into the background portion and the foreground portion, and for spectral bins of the background portion and the foreground portion corresponding to the same frequency, each of the background portion and the foreground portion is zero Lt; RTI ID = 0.0 > a < / RTI >
(150) for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150), wherein the relationship between the spectral value of the foreground part and the spectrum value of the background part in the same frequency bin depends on the ratio. . 12. The method according to any one of claims 1 to 11,
The block generator (110) is configured to perform block-to-frequency transforms into a frequency domain of a time domain to obtain a spectral representation for each block,
Blocks adjacent in time overlap in the overlap range 302,
The apparatus further comprises signal synthesizers (160a, 161a, 160b, 161b) for synthesizing the background component signals and synthesizing the foreground component signals,
The signal synthesizer performs frequency-to-time transforms (161a, 160a, 160b) on the background component signal and the foreground component signal and performs cross-fading on temporal representations of temporally- frequency component signal and a separate time-domain background component signal to produce a background component signal 140 and a foreground component signal 150, which are configured to cross-fade (161a, 161b) Apparatus for disassembly. 13. The method according to any one of claims 1 to 12,
The audio signal analyzer (120) is configured to determine an average characteristic for a group of blocks using a weighted addition of individual characteristics of the blocks within the group of blocks. An audio signal analyzer (120) And a foreground component signal (150). 14. The method according to any one of claims 1 to 13,
The audio signal analyzer 120 is configured to perform a weighted addition of individual characteristics of the blocks within the group of blocks,
Wherein the weighting value for a characteristic of a block closest in time to the current block is greater than a weighting value for a characteristic of an additional block that is closest in time to the current block, (150). The method according to claim 13 or 14,
The audio signal analyzer (120) is configured to determine a group of blocks such that the group of blocks includes at least 20 blocks before the corresponding block or at least 20 blocks following the current block. (100) into a background component signal (140) and a foreground component signal (150). 16. The method according to any one of claims 1 to 15,
Wherein the audio signal analyzer is configured to use the normalization value depending on the number of blocks in the group of blocks or depending on the weighting values for the blocks in the group of blocks. 140) and a foreground component signal (150). 17. The method according to any one of claims 1 to 16,
Further comprising a signal characteristic meter (702, 704) for measuring signal characteristics of at least one of the background component signals or the foreground component signals, wherein the background component signal (140) and the foreground component signal (150). 18. The method of claim 17,
The signal characteristic meter may be configured to determine a foreground density 702 using the foreground component signal or to determine a foreground component 704 using the foreground component signal and the audio input signal. An apparatus for decomposing a signal (100) into a background component signal (140) and a foreground component signal (150). 19. The method according to any one of claims 1 to 18,
Wherein the foreground component signal comprises clap signals,
The apparatus comprises means for increasing the number of clapping sounds or reducing the number of clapping sounds or applying a weight to the foreground component signal or the background component signal, Further comprising a signal characteristic modifier for modifying the foreground component signal by modifying the energy relationship to divide the audio signal into a background component signal (140) and a foreground component signal (150). 20. The method according to any one of claims 1 to 19,
Further comprising a blind upmixer for upmixing the audio signal with a representation having a number of output channels greater than the number of channels of the audio signal,
Wherein the upmixer is configured to spatially distribute the foreground component signal to the output channels and to correlate the foreground component signals within the plurality of output channels to spatially distribute the background component signal to the output channels And,
Wherein the background component signals in the output channels are less correlated or uncorrelated than the foreground component signals into a background component signal (140) and a foreground component signal (150). 21. The method according to any one of claims 1 to 20,
Separately encoding the foreground component signal and the background component signal to obtain an encoded representation (804) of the foreground component signal and a separate encoded representation of the background component signal (806) for transmission or storage or decoding Further comprising an encoder stage (801, 802) for decoding the audio signal (100) into a background component signal (140) and a foreground component signal (150). A method for decomposing an audio signal (100) into a background component signal (140) and a foreground component signal (150)
Generating (110) a time sequence of blocks of audio signal values;
Determining (120) a block characteristic of a current block of the audio signal and determining an average characteristic for the group of blocks, the group of blocks including at least two blocks; And
(130) separating the current block into a background portion and a foreground portion in response to a ratio of a block characteristic of the current block to an average characteristic of the group of blocks,
The background component signal 140 includes a background portion of the current block and the foreground component signal 150 comprises a foreground portion of the current block, Component signal (150). 22. A computer program for performing the method of claim 22 when running on a computer or processor.

Images