EP2158588B1 - Spectral smoothing method for noisy signals - Google Patents
Spectral smoothing method for noisy signals Download PDFInfo
- Publication number
- EP2158588B1 EP2158588B1 EP08784249A EP08784249A EP2158588B1 EP 2158588 B1 EP2158588 B1 EP 2158588B1 EP 08784249 A EP08784249 A EP 08784249A EP 08784249 A EP08784249 A EP 08784249A EP 2158588 B1 EP2158588 B1 EP 2158588B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- short
- smoothing method
- transformation
- smoothing
- term
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Not-in-force
Links
- 238000000034 method Methods 0.000 title claims abstract description 84
- 238000009499 grossing Methods 0.000 title claims abstract description 78
- 230000003595 spectral effect Effects 0.000 title claims description 59
- 238000001228 spectrum Methods 0.000 claims abstract description 99
- 230000009466 transformation Effects 0.000 claims abstract description 99
- 230000009467 reduction Effects 0.000 claims abstract description 15
- 238000000844 transformation Methods 0.000 claims description 18
- 238000013507 mapping Methods 0.000 claims description 9
- 238000000926 separation method Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000000873 masking effect Effects 0.000 claims description 2
- 230000001131 transforming effect Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 47
- 238000012545 processing Methods 0.000 description 15
- 230000002123 temporal effect Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 3
- 230000002596 correlated effect Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000005236 sound signal Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000003094 perturbing effect Effects 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/24—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being the cepstrum
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
Definitions
- the invention relates to a smoothing method for suppressing fluctuating artifacts in noise reduction.
- noise suppression is an important aspect.
- the audio signals recorded with a microphone and subsequently digitized contain, in addition to the useful signal ( FIG. 1 ) still ambient noise, which are superimposed on the useful signal ( FIG. 2 ).
- FIG. 1 the useful signal
- FIG. 2 the useful signal
- hearing aids are constantly changing ambient noise such as traffic noise or people talking in the background, such as in a restaurant.
- the noise reduction aims accordingly to a relief of Speech understanding. Therefore, reducing the noise should not audibly distort the speech signal.
- the spectral representation is a favorable representation of the signal.
- the signal is displayed in frequency broken down.
- a practical realization of the spectral representation are short-term spectra which result from a division of the signal into short frames ( FIG. 3 ), which are subjected to a spectral transformation separately from each other ( FIG. 4 ).
- a transformed frame then consists of M so-called frequency bins.
- the squared amplitude value of a frequency bin corresponds to the energy that contains the signal in the narrow frequency slice of about 31 Hz bandwidth represented by the respective frequency bin.
- 129 bins Due to the symmetry properties of the spectral transformation, only M / 2 + 1 of the M frequency bins, ie 129 bins in the previous example, are relevant for the signal representation. With 129 relevant bins and 31 Hz bandwidth per bin, a total of spectral band from 0 Hz to about 4000 Hz is covered. This is sufficient to describe many speech sounds with sufficient spectral resolution. Another common bandwidth is 8000 Hz, which can be achieved by a higher sampling rate and thus more frequency bins with the same frame duration.
- the frequency bins are indexed with ⁇ . The index for frames is ⁇ .
- the amplitudes of the short-term spectrum of a frame ⁇ are generally noted here as a spectral quantity G ⁇ ( ⁇ ).
- a common form of presentation of short-term spectra are so-called spectrograms, which are formed by juxtaposing temporally successive short-term spectra (cf., for example FIGS. 6 to 9 ).
- the advantage of the spectral representation is that the essential speech energy is concentrated in a relatively small number of frequency bins ( FIGS. 4 and 6 ), while in the time signal all digital samples are equally relevant ( FIG. 3 ).
- the signal energy of the disturbance is in most cases distributed over a larger number of frequency bins. Since the frequency bins contain different amounts of speech energy, it is possible to suppress the noise in those bins that contain little speech energy. The narrowband the frequency bins are, the better this separation succeeds.
- a spectral weighting function is estimated, which can be calculated according to different optimization criteria. It results in low values or zero in frequency bins that are predominantly perturbing, and values close to or equal to one for bins in which voice energy dominates ( FIG. 5 ).
- the weighting function is generally re-estimated for each signal frame in each frequency bin.
- the totality of the weighting values of all the frequency bins of a frame is also referred to herein as the "short-term spectrum of the weighting function" or simply as the "weighting function".
- Multiplication of the weighting function with the short-term spectrum of the noisy signal yields the filtered spectrum in which the amplitudes of the frequency bins in which interference dominates are greatly reduced, while speech components remain almost unaffected ( FIGS. 8 and 9 ).
- tonal artifacts musical noise
- FIGS. 10 and 11 A single tonal artifact has the duration of a signal frame and its frequency is determined by the frequency bin in which the outlier occurred.
- these spectral magnitudes can be smoothed by an averaging method and thus freed from excessive values.
- Spectral magnitudes of several spectrally adjacent or temporally successive frequency bins are calculated to an average, so that the amplitude of individual outliers is relativized. Smoothing is above the frequency [1: Tim Fingscheidt, Christophe Beaugeant and Suhadi Suhadi. Overcoming the statistical independence assumption wrt frequency in speech enhancement. Proceedings, IEEE Int. Conf.
- the cepstrum principally consists of a non-linear mapping, namely the logarithmization, a spectral value of magnitude and a subsequent transformation of this logarithmic magnitude spectrum with a transformation.
- the advantage of a cepstral representation of the amplitudes is that speech is no longer comb-like across the frequency ( FIGS. 4 and 6 ), but the essential information about the speech signal is represented in the cepstral small index bins. In addition, substantial speech information is still represented in the relatively easily detectable cepstral bin of higher index, which represents the so-called pitch frequency of the speaker.
- a smoothed short-term spectrum can be calculated by setting cepstral bins to zero with relatively small amounts, and then re-transforming the altered cepstrum back to a short-term spectrum.
- cepstral bins to zero with relatively small amounts
- re-transforming the altered cepstrum back to a short-term spectrum.
- strong fluctuations or outliers to correspondingly high Amplitudes in the cepstrum these artifacts can not be detected by these methods and suppressed.
- the object of the invention is to provide a smoothing method for suppressing fluctuations in the weighting function or in spectral intermediate variables or outliers in filtered short-term spectra, which neither reduces the frequency resolution of the short-term spectra nor impairs the temporal dynamics of the speech signal.
- the smoothing method according to the invention makes use of a transformation such as cepstrum in order to describe a broadband speech signal with as few transformation coefficients as possible in its essential structure.
- the transformation coefficients are not set to zero independently of each other if they fall below a threshold value. Instead, the values of transformation coefficients from at least two consecutive frames are offset by smoothing over time.
- the degree of smoothing is made dependent on the extent to which the spectral structure represented by the coefficient is decisive for the description of the useful signal.
- the degree of temporal smoothing of a coefficient therefore depends, for example, on whether a transformation coefficient contains a lot of speech energy or little. This is easier to determine in cepstrum or similar transformations than in the short-term spectrum.
- Coefficients with a large amount of speech information are only smoothed to the extent that their temporal dynamics do not become lower than with an unbounded speech signal. If necessary, these coefficients are not even smoothed. Speech distortions are thus prevented. Since spectral fluctuations and outliers represent a short-term change in the fine structure of a short-term spectrum, they form in the transformed short-term spectrum as a short-term change of those transformation coefficients that represent the fine structure of the short-term spectrum.
- G ⁇ ' smooth cepst ⁇ .
- G ⁇ ' smooth cepst ⁇ .
- Transformations differ in their basic functions used.
- the process of transformation means that the signal is correlated with the various basis functions.
- the resulting degree of correlation between the signal and a basis function is then the associated transformation coefficient.
- Orthogonal transformation bases contain only basis functions that are uncorrelated. In the case that the signal is identical to one of the basic functions, transform coefficients with the value zero are produced in the case of orthogonal transformations, with the exception of the one coefficient which is identical to the signal. The selectivity of an orthogonal transformation is therefore high.
- Non-orthogonal transforms use function bases that are correlated with each other.
- Another feature is that the basic functions for the considered application are discrete and finite, since the processed signal frames are discrete signals of the length of one frame.
- DFT Discrete Fourier Transform
- FFT Fast Fourier Transform
- DCT discrete cosine transform
- DST discrete sine transform
- the invertibility of the transforms also makes it possible to interchange the transform and its inverse in the back and forth transformations.
- the use of the DFT from (2) is thus also possible, for example, if the IDFT from (1) is used in (2).
- the spectral coefficients of the short-term spectra are mapped before non-linear transformation.
- a principal feature of non-linear imaging which is advantageous for the invention is dynamic compression of relatively large amplitudes and dynamic expansion of relatively small amplitudes.
- the spectral coefficients of the smoothed short-term spectra after the inverse transformation can be mapped non-linearly, the non-linear mapping after the inverse transformation being the inverse of the non-linear mapping before the forward transformation.
- the spectral coefficients are imaged non-linearly by logarithmization before the forward transformation.
- the smoothing method is applied to the magnitude or power of the magnitude of the short-term spectra.
- time constants can be chosen in such a way that the transformation coefficients, which primarily represent language, are less well smoothed.
- the transformation coefficients, the mainly fluctuating background noise and Artifacts of noise reduction algorithms describe being heavily smoothed.
- the spectral weighting function of a noise reduction algorithm can be provided.
- the spectral weighting function of a postfilter for multi-channel noise reduction methods can also be used as the short-term spectrum.
- the spectral weighting function results here from the minimization of an error criterion.
- a filtered short-term spectrum can also be provided.
- a spectral weighting function of a multi-channel method for noise reduction is provided as the short-term spectrum.
- an estimated coherence or an estimated magnitude squared coherence can also be provided between at least two microphone channels.
- a spectral weighting function of a multi-channel method for speaker or source separation is provided as the short-term spectrum.
- a spectral weighting function of a multi-channel method for speaker separation on the basis of phase differences of signals in the different channels is provided as a short-term spectrum.
- GCC generalized cross-correlation
- spectral magnitudes containing both speech and noise components can also be provided.
- an estimate of the signal-to-noise ratio in the individual frequency bins can also be provided as the short-term spectrum. Further, as the short-term spectrum, an estimate of the noise power can be used.
- the line of an image is interpreted as a signal frame that can be transformed into the spectral range.
- the resulting frequency bins are called spatial frequency bins here.
- algorithms equivalent to those used in audio signal processing are used. Possible fluctuations that generate these algorithms in the spatial frequency range result in the processed image in optical artifacts. These are equivalent to tonal noise in audio processing.
- signals are derived from the human body that can be noisy, such as acoustic signals.
- the noisy signal can be transformed into the spectral range frame by frame.
- the resulting spectrograms can be processed like audio spectra.
- the smoothing method can be used in a telecommunications network and / or in broadcasting to improve speech and / or picture quality and to suppress artifacts.
- the speech coding redundancy-reducing speech compression
- the associated quantization noise and on the other by the interference caused by the transmission channel.
- the latter fluctuate greatly in terms of time and spectral and lead to a noticeable deterioration of the voice quality.
- the receiver side or network used Signal processing ensure that the quasi-random artifacts are reduced.
- so-called post-filters and error concealment methods have hitherto been used.
- the smoothing method can thus be used as a postfilter, in a postfilter, in combination with a postfilter, as part of an error concealment method or in connection with a method for speech and / or picture coding (decompression method or decoding method), in particular on the receiver side.
- a postfilter it is meant that the method is used for post-filtering, that is to say that the data produced in the applications are processed with an algorithm implementing the method.
- FIG. 1 is an unencumbered signal in the form of amplitude over time.
- the duration of the signal is 4 seconds, the amplitudes range from about -0.18 to about 0.18.
- FIG. 2 the signal is shown in noisy form. One recognizes a random background noise over the entire time course.
- FIG. 3 the signal of a single signal frame ⁇ is shown.
- the signal frame has a segment duration of 32 milliseconds.
- the amplitude of both graphs ranges between -0.1 and 0.1.
- the individual samples of the digital signals are connected to graphs.
- the noisy graph represents the input signal containing the noisy signal. A separation of signal and noise in the noisy signal is hardly possible in this representation of the signal.
- FIG. 4 is a representation of the same signal frame after the transformation into the frequency domain.
- the individual frequency bins ⁇ are connected to graphs.
- the frequency bins are noisy and noisy, again with the noisy signal in the noisy signal is included speech signal.
- the abscissa shows the frequency bins ⁇ from 0 to 128. They have amplitudes of about -40 decibels (dB) to about 10 dB. From the comparison of the graphs, it can be seen that the energy of the speech signal in some frequency bins is concentrated in a comb-like structure, while the noise is also present in the intervening bins.
- FIG. 5 is a weighting function for the noisy frame FIG. 4 shown.
- a factor between 0 and 1 results depending on the ratio of speech and noise energy.
- the individual weighting factors are connected to a graph. One recognizes the comb-like structure of the speech spectrum again.
- FIGS. 6 and 7 are spectrograms from a series of noisy or noisy short-term spectra ( FIG. 4 ).
- the frame index ⁇ is plotted, on the ordinate of the frequency bin index ⁇ .
- the amplitudes of the individual frequency bins are shown as gray values.
- FIGS. 6 and 7 It becomes clear how language is concentrated in a few frequency bins. It also trains regular structures. The noise, however, is distributed over all frequency bins.
- FIG. 8 the spectrogram of a filtered signal is shown.
- the axes correspond to those from the FIGS. 6 and 7 . From a comparison with FIG. 6 It can be seen that estimation errors in the weighting function leave high amplitudes in frequency bins which contain no speech. To suppress these outliers is the aim of the method according to the invention.
- FIG. 9 is the spectrogram of a signal is shown, which has been filtered according to a preferred embodiment of the method according to the invention with a smoothed weighting function.
- the axes correspond to those of the previous spectrograms.
- the outliers are greatly reduced.
- the speech components in the spectrogram are preserved in their essential form.
- FIGS. 10 and 11 the time signals are shown, each resulting from the filtered spectra of the FIGS. 8 and 9 result. Plotted is the amplitude over time. The signals are 4 seconds long and have amplitudes between about -0.18 and 0.18.
- the outliers in the spectrogram out FIG. 8 result in the associated time signal in FIG. 10 clearly visible tonal artifacts, which in the noisy signal FIG. 1 are not available.
- the time signal in FIG. 11 has a much quieter course of residual noise. This time signal results from the spectrogram of FIG. 9 that was generated by filtering with the smoothed weighting function.
- the unsmoothed weighting function is shown for all frames. Frequency bins ⁇ are plotted along the ordinate for each frame ⁇ . The values of the weighting function are shown as gray tones. The fluctuations resulting from estimation errors are recognizable as irregular patches.
- FIG. 13 the smoothed weighting function is shown for all frames.
- the axes correspond to those from FIG. 12 . Due to the smoothing, the fluctuations are smeared and greatly reduced in value. The structure of the voice frequency bins, however, remains clearly recognizable.
- FIG. 14 is the amount of the cepstrum of an unbroken signal across all frames.
- the cepstral bins ⁇ ' are plotted along the ordinate.
- the values of the amounts of cepstral coefficients G ⁇ ' cepst ⁇ are shown as shades of gray.
- a comparison with FIG. 6 shows that language in cepstrum is concentrated on an even smaller number of coefficients. In addition, these coefficients are less variable in their position.
- Clearly recognizable is the course of the cepstral coefficient, which represents the pitch frequency.
- FIG. 15 a signal flow graph according to a preferred embodiment of the invention is shown.
- a noisy input signal is transformed into a sequence of short-term spectra, and then a weighting function for filtering is then estimated via spectral intermediate quantities. It will be respectively a frame currently being edited.
- the short-term spectra of the weighting function are subjected to a non-linear, logarithmic mapping. This is followed by a transformation into the cepstral area.
- the thus transformed short-term spectra are thus represented by transformation coefficients of the basis functions.
- the transformation coefficients calculated in this way are smoothed separately using different time constants.
- the recursive nature of the smoothing is indicated by the return of the output of the smoothing to its input.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
- Color Television Image Signal Generators (AREA)
- Holo Graphy (AREA)
- Photoreceptors In Electrophotography (AREA)
- Networks Using Active Elements (AREA)
- Spectrometry And Color Measurement (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Optical Communication System (AREA)
- Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)
Abstract
Description
Die Erfindung betrifft ein Glättungsverfahren zur Unterdrückung von fluktuierenden Artefakten bei der Störgeräuschreduktion.The invention relates to a smoothing method for suppressing fluctuating artifacts in noise reduction.
In der digitalen Sprachsignalübertragung ist die Störgeräuschunterdrückung ein wichtiger Aspekt. Die mit einem Mikrofon erfassten und anschließend digitalisierten Audiosignale enthalten neben dem Nutzsignal (
Für die Störgeräuschreduktion ist die Spektraldarstellung eine günstige Repräsentation des Signals. Hierbei wird das Signal in Frequenzen aufgeschlüsselt dargestellt. Eine praktische Realisierung der Spektraldarstellung sind Kurzzeitspektren, die durch eine Zerteilung des Signals in kurze Rahmen entstehen (
Vorteil der Spektraldarstellung ist, dass die wesentliche Sprachenergie in einer relativ geringen Anzahl von Frequenzbins konzentriert vorliegt (
Für die Störgeräuschreduktion wird eine spektrale Gewichtungsfunktion geschätzt, die nach unterschiedlichen Optimierungskriterien berechnet werden kann. Sie ergibt niedrige Werte oder Null in Frequenzbins, in denen hauptsächlich Störung vorliegt, und Werte nahe oder gleich Eins für Bins, in denen Sprachenergie dominiert (
Eine Multiplikation der Gewichtungsfunktion mit dem Kurzzeitspektrum des verrauschten Signals ergibt das gefilterte Spektrum, in dem die Amplituden der Frequenzbins, in denen Störung dominiert, stark verringert sind, während Sprachkomponenten nahezu unbeeinflusst bleiben (
Schätzfehler bei der Berechnung der spektralen Gewichtungsfunktion, sogenannte Fluktuationen, ergeben gelegentlich zu hohe Gewichtungswerte für Frequenzbins, die hauptsächlich Störung enthalten (
Zur Unterdrückung von Fluktuationen in der Gewichtungsfunktion oder in spektralen Zwischengrößen bzw. zur Unterdrückung von Ausreißern im gefilterten Spektrum können diese spektralen Größen durch ein Mittelungsverfahren geglättet und somit von überhöhten Werten befreit werden. Spektrale Größen mehrerer spektral benachbarter oder zeitlich aufeinanderfolgender Frequenzbins werden dabei zu einem Mittelwert verrechnet, so dass die Amplitude einzelner Ausreißer relativiert wird. Eine Glättung ist über der Frequenz [1:
Eine weitere bekannte Form der Glättung einzelner Kurzzeitspektren über der Frequenz ist ein als "Liftering" bekanntes Verfahren [4:
mit IDFT{·} der inversen diskreten Fourier-Transformation (DFT) einer Folge von Werten der Länge M. Diese Transformation resultiert in M Transformationskoeffizienten
with IDFT {·} of the inverse discrete Fourier transformation (DFT) of a sequence of values of length M. This transformation results in M transformation coefficients
Ein geglättetes Kurzzeitspektrum kann berechnet werden, indem cepstrale Bins mit relativ kleinen Beträgen zu Null gesetzt werden und anschließend das veränderte Cepstrum wieder in ein Kurzzeitspektrum rücktransformiert wird. Da allerdings starke Fluktuationen bzw. Ausreißer zu entsprechend hohen Amplituden im Cepstrum führen, können diese Artefakte durch diese Verfahren nicht detektiert und unterdrückt werden.A smoothed short-term spectrum can be calculated by setting cepstral bins to zero with relatively small amounts, and then re-transforming the altered cepstrum back to a short-term spectrum. However, since strong fluctuations or outliers to correspondingly high Amplitudes in the cepstrum, these artifacts can not be detected by these methods and suppressed.
Alternativ zum Liftering gibt es noch das Verfahen nach [6:
Das Dokument
Hiervon ausgehend liegt der Erfindung die Aufgabe zugrunde, für die Rauschreduktion ein Glättungsverfahren zur Unterdrückung von Fluktuationen in der Gewichtungsfunktion oder in spektralen Zwischengrößen bzw. von Ausreißern in gefilterten Kurzzeitspektren aufzuzeigen, das weder die Frequenzauflösung der Kurzzeitspektren verringert noch die zeitliche Dynamik des Sprachsignals beeinträchtigt.Proceeding from this, the object of the invention is to provide a smoothing method for suppressing fluctuations in the weighting function or in spectral intermediate variables or outliers in filtered short-term spectra, which neither reduces the frequency resolution of the short-term spectra nor impairs the temporal dynamics of the speech signal.
Die Lösung dieser Aufgabe besteht in einem Glättungsverfahren mit den Maßnahmen von Patentanspruch 1. Vorteilhafte Weiterbildungen sind Gegenstand der Unteransprüche.The solution to this problem consists in a smoothing process with the measures of
Das erfindungsgemäße Glättungsverfahren umfasst folgende Schritte:
- Bereitstellen von Kurzzeitspektren einer Folge von digitalen Abtastwerten umfassenden Signalrahmen,
- Transformieren jedes Kurzzeitspektrums durch eine Hintransformation, die das Kurzzeitspektrum durch Transformationskoeffizienten beschreibt, welche das Kurzzeitspektrum in seine groben und seine feinen Strukturen unterteilt repräsentieren,
- Glätten der Transformationskoeffizienten jeweils gleicher Koeffizientenindizes durch Kombination von wenigstens zwei aufeinanderfolgenden transformierten Kurzzeitspektren und
- Transformieren der geglätteten Transformationskoeffizienten in geglättete Kurzzeitspektren durch eine Rücktransformation.
- Providing short-term spectra of a sequence of digital samples comprising signal frames,
- Transforming each short-term spectrum by a forward transformation that describes the short-term spectrum by transform coefficients that represent the short-term spectrum divided into its coarse and fine structures,
- Smoothing the transformation coefficients of each same coefficient indices by combining at least two consecutive transformed short-term spectra and
- Transform the smoothed transform coefficients into smoothed short-term spectra by inverse transform.
Das erfindungsgemäße Glättungsverfahren bedient sich einer Transformation wie des Cepstrums, um ein breitbandiges Sprachsignal mit möglichst wenig Transformationskoeffizienten in seiner wesentlichen Struktur zu beschreiben. Anders als in bekannten Verfahren werden die Transformationskoeffizienten aber nicht unabhängig voneinander zu Null gesetzt, wenn sie einen Schwellwert unterschreiten. Es werden stattdessen die Werte von Transformationskoeffizienten aus mindestens zwei aufeinanderfolgenden Rahmen durch eine Glättung über die Zeit miteinander verrechnet. Hierbei wird der Grad der Glättung davon abhängig gemacht, inwieweit die durch den Koeffizienten repräsentierte spektrale Struktur entscheidend für die Beschreibung des Nutzsignals ist. Der Grad der zeitlichen Glättung eines Koeffizienten hängt daher beispielsweise davon ab, ob ein Transformationskoeffizient viel Sprachenergie enthält oder wenig. Dies ist im Cepstrum oder ähnlichen Transformationen leichter zu bestimmen als im Kurzzeitspektrum. So kann beispielsweise angenommen werden, dass die ersten vier cepstralen Koeffizienten mit Indizes µ' = 0 ... 3 und zusätzlich der Koeffizient mit maximalem Betrag und Index µ' größer 16 und kleiner 160 bei fs = 8000 Hz (Pitch) Sprache repräsentieren. Koeffizienten mit viel Sprachinformation werden nur soweit geglättet, dass ihre zeitliche Dynamik nicht geringer wird als bei einem unverrauschten Sprachsignal. Gegebenenfalls werden diese Koeffizienten gar nicht geglättet. Sprachverzerrungen werden so verhindert. Da spektrale Fluktuationen und Ausreißer eine kurzzeitige Änderung in der Feinstruktur eines Kurzzeitspektrums darstellen, bilden sie sich im transformierten Kurzzeitspektrum als kurzzeitige Änderung derjenigen Transformationskoeffizienten ab, die die Feinstruktur des Kurzzeitspektrums repräsentieren. Da diese Transformationskoeffizienten bei unverrauschter Sprache eine relativ geringe zeitliche Änderungsrate haben, können eben diese Koeffizienten stärker geglättet werden. Eine verstärkte zeitliche Glättung wirkt somit der Ausbildung von Ausreißern entgegen, ohne die Struktur der Sprache zu beeinflussen. Das Glättungsverfahren resultiert somit nicht in einer verringerten spektralen Auflösung für Sprachlaute. Die Änderung der Feinstruktur des Kurzzeitspektrums bei aufeinanderfolgenden Rahmen ist derart verzögert, dass nur schmalbandige spektrale Änderungen mit Zeitkonstanten kleiner als derjenigen von unverrauschter Sprache unterbunden werden.The smoothing method according to the invention makes use of a transformation such as cepstrum in order to describe a broadband speech signal with as few transformation coefficients as possible in its essential structure. Unlike in known methods, however, the transformation coefficients are not set to zero independently of each other if they fall below a threshold value. Instead, the values of transformation coefficients from at least two consecutive frames are offset by smoothing over time. In this case, the degree of smoothing is made dependent on the extent to which the spectral structure represented by the coefficient is decisive for the description of the useful signal. The degree of temporal smoothing of a coefficient therefore depends, for example, on whether a transformation coefficient contains a lot of speech energy or little. This is easier to determine in cepstrum or similar transformations than in the short-term spectrum. Thus, for example, it can be assumed that the first four cepstral coefficients with indices μ '= 0... 3 and additionally the coefficient with maximum magnitude and index μ' represent greater than 16 and less than 160 at f s = 8000 Hz (pitch) speech. Coefficients with a large amount of speech information are only smoothed to the extent that their temporal dynamics do not become lower than with an unbounded speech signal. If necessary, these coefficients are not even smoothed. Speech distortions are thus prevented. Since spectral fluctuations and outliers represent a short-term change in the fine structure of a short-term spectrum, they form in the transformed short-term spectrum as a short-term change of those transformation coefficients that represent the fine structure of the short-term spectrum. Since these transformation coefficients have a relatively low rate of change of time in the case of unobtrusive speech, it is precisely these coefficients that can be smoothed out more effectively. Increased temporal smoothing thus counteracts the formation of outliers without affecting the structure of the language. The smoothing method thus does not result in a reduced spectral resolution for speech sounds. The change in the fine structure of the short-term spectrum in successive frames is delayed so that only narrow-band spectral changes with time constants smaller than that of noisy speech are prevented.
Aus der geglätteten Größe, notiert als
mit DFT{} der diskreten Fourier-Transformation und exp() der Exponentialfunktion, die in (2) elementweise angewendet wird.From the smoothed size, noted as
with DFT {} of the discrete Fourier transform and exp () of the exponential function, which is applied element by element in (2).
Die Vorteile, die sich aus der erfindungsgemäßen Glättung von kurzzeitspektren ergeben, sind:
- eine effektive Unterdrückung von Fluktuationen bzw. Ausreißern,
- Beibehaltung der spektralen Auflösung für Sprachsignale und
- keine hörbare Beeinflussung von Sprache.
- an effective suppression of fluctuations or outliers,
- Maintaining the spectral resolution for speech signals and
- no audible influence on speech.
Es ist wichtig anzumerken, dass die für das Cepstrum verwendete inverse DFT in (1) und die DFT für die Rücktransformation in (2) durch andere Transformationen ersetzt werden können, ohne dass dabei die prinzipiellen Eigenschaften der Transformationskoeffizienten bzgl. der kompakten Repräsentation von Sprache verloren gehen. Genauso verhält es sich mit der Logarithmierung in (1) und der entsprechenden Umkehrfunktion in (2), der Exponentialfunktion. Auch hier sind andere nicht-lineare Abbildungen und auch lineare Abbildungen denkbar.It is important to note that the inverse DFT in (1) used for the cepstrum and the DFT for inverse transformation in (2) can be replaced by other transforms without losing the principal properties of the transform coefficients with respect to the compact representation of speech walk. The same is true of the logarithmization in (1) and the corresponding inverse function in (2), the exponential function. Again, other non-linear mappings and even linear mappings are conceivable.
Transformationen unterscheiden sich in ihren verwendeten Basisfunktionen. Der Vorgang der Transformation bedeutet, dass das Signal mit den verschiedenen Basisfunktionen korreliert wird. Der resultierende Grad der Korrelation zwischen dem Signal und einer Basisfunktion ist dann der zugehörige Transformationskoeffizient. Bei einer Transformation entstehen so viele Transformationskoeffizienten wie es Basisfunktionen gibt. Ihre Anzahl ist hier mit M bezeichnet. Für die Erfindung wichtige Transformationen sind solche, durch deren Basisfunktionen das zu transformierende Kurzzeitspektrum in seiner Grobstruktur und seiner Feinstruktur aufgeschlüsselt wird.Transformations differ in their basic functions used. The process of transformation means that the signal is correlated with the various basis functions. The resulting degree of correlation between the signal and a basis function is then the associated transformation coefficient. In a transformation arise like this many transformation coefficients as there are basis functions. Their number is denoted by M here. Transformations that are important for the invention are those by means of whose basic functions the short-term spectrum to be transformed is broken down into its coarse structure and its fine structure.
Ein Unterscheidungsmerkmal von Transformationen ist die Orthogonalität. Orthogonale Transformationsbasen enthalten nur Basisfunktionen, die unkorreliert sind. Für den Fall, dass das Signal mit einer der Basisfunktionen identisch ist, entstehen bei orthogonalen Transformationen Transformationskoeffizienten mit dem Wert Null, bis auf den einen Koeffizienten, der identisch zum Signal ist. Die Trennschärfe einer orthogonalen Transformation ist demnach hoch. Nicht-orthogonale Transformationen verwenden Funktionsbasen, die miteinander korreliert sind.A distinguishing feature of transformations is orthogonality. Orthogonal transformation bases contain only basis functions that are uncorrelated. In the case that the signal is identical to one of the basic functions, transform coefficients with the value zero are produced in the case of orthogonal transformations, with the exception of the one coefficient which is identical to the signal. The selectivity of an orthogonal transformation is therefore high. Non-orthogonal transforms use function bases that are correlated with each other.
Ein weiteres Merkmal ist, dass die Basisfunktionen für den betrachteten Anwendungsfall diskret und endlich sind, da es sich bei den verarbeiteten Signalrahmen um diskrete Signale von der Länge eines Rahmens handelt.Another feature is that the basic functions for the considered application are discrete and finite, since the processed signal frames are discrete signals of the length of one frame.
Wichtiges Merkmal einer Transformation ist die Invertierbarkeit. Existiert zu einer Transformation (Hintransformation) eine inverse Transformation, so entsteht durch Transformation eines Signals in Transformationskoeffizienten und anschließender inverser Transformation (Rücktransformation) dieser Koeffizienten wieder das Ausgangssignal, falls die Transformationkoeffizienten nicht verändert wurden.An important feature of a transformation is the invertibility. If there is an inverse transformation to a transformation (forward transformation), the transformation of a signal into transformation coefficients and subsequent inverse transformation (inverse transformation) of these coefficients again produces the output signal if the transformation coefficients were not changed.
In der Signalverarbeitung, wie sie hier beschrieben wird, ist die diskrete Fourier-Transformation (DFT) eine bevorzugte Transformation. Ein dazugehöriger wichtiger Algorithmus in der diskreten Signalverarbeitung ist die "Fast-Fourier-Transformation" (FFT). Außerdem sind die diskrete Cosinus-Transformation (DCT) und die diskrete Sinus-Transformation (DST) häufig verwendete Transformationen. Diese Transformationen werden hier unter dem Begriff "Standardtransformationen" zusammengefasst. Eine für die Erfindung entscheidende bereits erwähnte Eigenschaft der Standardtransformationen ist, dass die Amplituden der verschiedenen Transformationskoeffizienten unterschiedliche Grade an Feinstruktur des transformierten Signals repräsentieren. So beschreiben Koeffizienten mit kleinen Indizes die groben Strukturen des transformierten Signals, weil die zugehörigen Basisfunktionen niederfrequente harmonische Funktionen sind. Je höher der Index eines Transformationskoeffizienten bis hin zu µ' = M/2, desto feiner sind die Strukturen des transformierten Signals, die durch diesen Koeffizienten beschrieben werden. Für darüber hinausgehende Koeffizienten dreht sich diese Eigenschaft wegen der Symmetrie der Koeffizienten um. In der Regel werden bei der Signalverarbeitung nur die Koeffizienten mit Indizes µ' = 0 bis µ' = M/2 verarbeitet und die restlichen Werte durch spiegeln der Resultate ermittelt.In signal processing as described herein, Discrete Fourier Transform (DFT) is a preferred transformation. An associated important algorithm in discrete signal processing is the "Fast Fourier Transform" (FFT). In addition, discrete cosine transform (DCT) and discrete sine transform (DST) are commonly used transformations. These transformations are summarized here under the term "standard transformations". An already mentioned characteristic of the standard transformations, which is decisive for the invention, is that the amplitudes of the different transformation coefficients are different Represent levels of fine structure of the transformed signal. Thus, coefficients with small indices describe the coarse structures of the transformed signal because the associated basis functions are low-frequency harmonic functions. The higher the index of a transformation coefficient up to μ '= M / 2, the finer are the structures of the transformed signal which are described by this coefficient. For further coefficients, this property revolves because of the symmetry of the coefficients. As a rule, only the coefficients with indices μ '= 0 to μ' = M / 2 are processed during signal processing and the remaining values are determined by mirroring the results.
Die Invertierbarkeit der Transformationen macht es außerdem möglich, die Transformation und ihre Inverse bei der Hin- und Rücktransformation zu vertauschen. In (1) ist also auch beispielsweise die Verwendung der DFT aus (2) möglich, wenn in (2) die IDFT aus (1) verwendet wird.The invertibility of the transforms also makes it possible to interchange the transform and its inverse in the back and forth transformations. In (1), the use of the DFT from (2) is thus also possible, for example, if the IDFT from (1) is used in (2).
Vorteilhaft werden die spektralen Koeffizienten der Kurzzeitspektren vor der Hintransformation nicht-linear abgebildet. Prinzipielle, für die Erfindung vorteilhafte Eigenschaft der nicht-linearen Abbildung ist eine Dynamik-Kompression relativ großer Amplituden und eine Dynamik-Expansion relativ kleiner Amplituden.Advantageously, the spectral coefficients of the short-term spectra are mapped before non-linear transformation. A principal feature of non-linear imaging which is advantageous for the invention is dynamic compression of relatively large amplitudes and dynamic expansion of relatively small amplitudes.
Entsprechend können die spektralen Koeffizienten der geglätteten Kurzzeitspektren nach der Rücktransformation nicht-linear abgebildet werden, wobei die nicht-lineare Abbildung nach der Rücktransformation die Umkehrung der nicht-linearen Abbildung vor der Hintransformation ist.Accordingly, the spectral coefficients of the smoothed short-term spectra after the inverse transformation can be mapped non-linearly, the non-linear mapping after the inverse transformation being the inverse of the non-linear mapping before the forward transformation.
Zweckmäßigerweise werden die spektralen Koeffizienten vor der Hintransformation durch Logarithmierung nicht-linear abgebildet.Expediently, the spectral coefficients are imaged non-linearly by logarithmization before the forward transformation.
Eine Form der zeitlichen Glättung kann durch ein rekursives System vorzugsweise erster Ordnung erreicht werden:
Mögliche Werte für die Glättungskonstanten für Koeffizienten der Standardtransformationen im Falle von Sprachsignalen sind βµ' = 0 für µ' = 0 ... 3, βµ' = 0,8 für µ' = 4 ... M/2 mit Ausnahme der Transformationskoeffizienten durch die die Pitch-Frequenz eines Sprechers repräsentiert wird, und βµ' = 0,4 für Transformationskoeffizienten, die die Pitch-Frequenz repräsentieren. Verfahren zur Bestimmung des Pitch-Koeffizienten sind zahlreich in der Literatur verfügbar. Beispielsweise kann zur Bestimmung des Koeffizienten der Pitch derjenige Koeffizient gewählt werden, dessen Index zwischen µ' = 16 und µ' = 160 liegt und der die maximale Amplitude aller Koeffizienten in diesem Indexbereich aufweist. Für die verbleibenden Transformationskoeffizienten mit Indizes µ'= M/2 + 1 ... M - 1 gilt die Symmetriebedingung βM-µ' = βµ'. Die Werte sind für die Standardtransformationen sowie Kurzzeitspektren, die aus Signalen mit fs = 8000 Hz entstanden sind, geeignet. Sie sind durch verhältnismäßige Umrechnung an andere Systeme anpassbar. Die Wahl βµ' = 0 bedeutet, dass die betreffenden Koeffizienten nicht geglättet werden. Es ist eine entscheidende Eigenschaft der Erfindung, dass Koeffizienten, die den groben Verlauf des Kurzzeitspektrums beschreiben, möglichst wenig geglättet werden, wenn es sich um die Entrauschung von Sprachsignalen handelt. So werden die groben Strukturen des breitbandigen Sprachspektrums vor Glättungseffekten geschützt. Die feinen Strukturen von Fluktuationen bzw. spektralen Ausreißern bilden sich bei den Standardtransformationen in den Transformationskoeffizienten zwischen µ' = 4 und µ' = M/2 ab, weshalb diese bis auf den Pitch der Sprache stark geglättet werden.Possible values for the smoothing constants for coefficients of the standard transformations in the case of speech signals are β μ ' = 0 for μ' = 0 ... 3, β μ '= 0.8 for μ' = 4 ... M / 2 with the exception of Transform coefficients by which the pitch frequency of a speaker is represented, and β μ ' = 0.4 for transform coefficients representing the pitch frequency. Methods for determining the pitch coefficient are numerous available in the literature. For example, to determine the coefficient of the pitch, that coefficient can be selected whose index lies between μ '= 16 and μ' = 160 and which has the maximum amplitude of all the coefficients in this index range. For the remaining transformation coefficients with indices μ '= M / 2 + 1... M-1, the symmetry condition β M-μ' = β μ ' applies. The values are suitable for the standard transformations as well as short-time spectra, which have been generated from signals with f s = 8000 Hz. They are adaptable by relative conversion to other systems. The choice β μ ' = 0 means that the respective coefficients are not smoothed. It is a crucial feature of the invention that coefficients that describe the coarse course of the short-term spectrum are smoothed as little as possible when it comes to the denoising of speech signals. Thus, the coarse structures of the broadband speech spectrum are protected from smoothing effects. The fine structures of fluctuations or spectral outliers are reflected in the standard transformations in the transformation coefficients between μ '= 4 and μ' = M / 2, which is why they are greatly smoothed to the pitch of the language.
Vorteilhafterweise wird das Glättungsverfahren auf den Betrag oder eine Potenz des Betrags der Kurzzeitspektren angewendet.Advantageously, the smoothing method is applied to the magnitude or power of the magnitude of the short-term spectra.
Besonders vorteilhaft ist es, wenn zum Glätten der jeweiligen Transformationskoeffizienten unterschiedliche Zeitkonstanten verwendet werden. Die Zeitkonstanten können so gewählt werden, dass die Transformationskoeffizenten, die vornehmlich Sprache repräsentieren, wenig geglättet werden. Zweckmäßigerweise können die Transformationskoeffizenten, die hauptsächlich fluktuierende Hintergrundgeräusche und Artefakte der Geräuschreduktionsalgorithmen beschreiben, stark geglättet werden.It is particularly advantageous if different time constants are used to smooth out the respective transformation coefficients. The time constants can be chosen in such a way that the transformation coefficients, which primarily represent language, are less well smoothed. Conveniently, the transformation coefficients, the mainly fluctuating background noise and Artifacts of noise reduction algorithms describe being heavily smoothed.
Als Kurzzeitspektrum kann die spektrale Gewichtungsfunktion eines Geräuschreduktionsalgorithmus bereitgestellt werden. Vorteilhaft kann als Kurzzeitspektrum auch die spektrale Gewichtungsfunktion eines Postfilters für mehrkanalige Verfahren zur Geräuschreduktion verwendet werden. Zweckmäßigerweise ergibt sich die spektrale Gewichtungsfunktion hierbei aus der Minimierung eines Fehlerkriteriums.As a short-term spectrum, the spectral weighting function of a noise reduction algorithm can be provided. Advantageously, the spectral weighting function of a postfilter for multi-channel noise reduction methods can also be used as the short-term spectrum. Expediently, the spectral weighting function results here from the minimization of an error criterion.
Als Kurzzeitspektrum kann auch ein gefiltertes Kurzzeitspektrum bereitgestellt werden.As a short-term spectrum, a filtered short-term spectrum can also be provided.
Nach einer anderen Weiterbildung des Verfahrens, wird als Kurzzeitspektrum eine spektrale Gewichtungsfunktion eines mehrkanaligen Verfahrens zur Geräuschreduktion bereitgestellt.According to another development of the method, a spectral weighting function of a multi-channel method for noise reduction is provided as the short-term spectrum.
Als Kurzzeitspektrum kann auch eine geschätzte Kohärenz oder eine geschätzte "Magnitude Squared Coherence" zwischen wenigstens zwei Mikrofonkanälen bereitgestellt werden.As a short-term spectrum, an estimated coherence or an estimated magnitude squared coherence can also be provided between at least two microphone channels.
Vorteilhaft wird als Kurzzeitspektrum eine spektrale Gewichtungsfunktion eines mehrkanaligen Verfahrens zur Sprecher- oder Quellentrennung bereitgestellt.Advantageously, a spectral weighting function of a multi-channel method for speaker or source separation is provided as the short-term spectrum.
Weiterhin ist vorgesehen, dass als Kurzzeitspektrum eine spektrale Gewichtungsfunktion eines mehrkanaligen Verfahrens zur Sprechertrennung auf Basis von Phasenunterschieden von Signalen in den verschiedenen Kanälen (Phase Transform - PHAT) bereitgestellt wird.It is further provided that a spectral weighting function of a multi-channel method for speaker separation on the basis of phase differences of signals in the different channels (phase transform - PHAT) is provided as a short-term spectrum.
Ferner ist es möglich, als Kurzzeitspektrum eine spektrale Gewichtungsfunktion eines mehrkanaligen Verfahrens auf Basis einer "Generalized Cross-Correlation" (GCC) zu verwenden.Furthermore, it is possible to use a spectral weighting function of a multi-channel method based on a generalized cross-correlation (GCC) as the short-term spectrum.
Als Kurzzeitspektrum können auch spektrale Größen, die sowohl Sprach- als auch Störanteile enthalten, bereitgestellt werden.As a short-term spectrum, spectral magnitudes containing both speech and noise components can also be provided.
So kann als Kurzzeitspektrum auch eine Schätzung des Signal-zu-Rausch-Verhältnisses in den einzelnen Frequenzbins bereitgestellt werden. Ferner kann als Kurzzeitspektrum eine Schätzung der Rauschleistung verwendet werden.Thus, an estimate of the signal-to-noise ratio in the individual frequency bins can also be provided as the short-term spectrum. Further, as the short-term spectrum, an estimate of the noise power can be used.
Das Problem von Fluktuationen in Kurzzeitspektren ist nicht nur in der Audiosignalverarbeitung bekannt. Weitere vorteilhafte Anwendungsgebiete sind die Bild- und die medizinische Signalverarbeitung.The problem of fluctuations in short-term spectra is known not only in audio signal processing. Further advantageous fields of application are image and medical signal processing.
In der Bildverarbeitung kann z.B. die Zeile eines Bildes als Signalrahmen interpretiert werden, der in den Spektralbereich transformiert werden kann. Die entstehenden Frequenzbins werden hier Ortsfrequenzbins genannt. Bei der Verarbeitung von Bildern im Ortsfrequenzbereich werden Algorithmen verwendet, die denen in der Audiosignalverarbeitung äquivalent sind. Mögliche Fluktuationen, die diese Algorithmen im Ortsfrequenzbereich erzeugen, resultieren im verarbeiteten Bild in optischen Artefakten. Diese sind äquivalent zum tonalen Rauschen in der Audioverarbeitung.In image processing, e.g. the line of an image is interpreted as a signal frame that can be transformed into the spectral range. The resulting frequency bins are called spatial frequency bins here. When processing images in the spatial frequency domain, algorithms equivalent to those used in audio signal processing are used. Possible fluctuations that generate these algorithms in the spatial frequency range result in the processed image in optical artifacts. These are equivalent to tonal noise in audio processing.
In der medizinischen Signalverarbeitung werden vom menschlichen Körper Signale abgeleitet, die wie akustische Signale verrauscht sein können. Das verrauschte Signal kann entsprechend rahmenweise in den Spektralbereich transformiert werden. Die entstehenden Spektrogramme lassen sich wie Audiospektren verarbeiten.In medical signal processing, signals are derived from the human body that can be noisy, such as acoustic signals. The noisy signal can be transformed into the spectral range frame by frame. The resulting spectrograms can be processed like audio spectra.
Das Glättungsverfahren kann in einem Telekommunikationsnetzwerk und/oder bei einer Rundfunkübertragung zur Verbesserung der Sprach- und/oder Bildqualität sowie zur Unterdrückung von Artefakten eingesetzt werden. In der mobilen Sprachkommunikation treten Verzerrungen des Sprachsignals auf, die zum einen durch die eingesetzten Sprachcodierverfahren (redundanzvermindernde Sprachkompression) und das damit verbundene Quantisierungsrauschen und zum anderen durch die vom Übertragungskanal hervorgerufenen Störungen bedingt sind. Letztere sind wiederum stark zeitlich und spektral fluktuierend und führen zu einer deutlich wahrnehmbaren Verschlechterung der Sprachqualität. Auch hier muss die empfängerseitig oder im Netzwerk eingesetzte Signalverarbeitung sicherstellen, dass die quasi zufälligen Artefakte reduziert werden. Zur Qualitätsverbesserung werden bisher sogenannte Postfilter und Fehlerverdeckungsverfahren eingesetzt. Während das Postfilter überwiegend die Reduktion von Quantisierungsrauschen zur Aufgabe hat, werden Fehlerverdeckungsverfahren zur Unterdrückung von übertragungsbedingten Kanalstörungen eingesetzt. In beiden Anwendungen können Verbesserungen erzielt werden, wenn in das Postfilter oder das Verdeckungsverfahren das erfindungsgemäße Glättungsverfahren integriert wird. Das Glättungsverfahren kann somit als Postfilter, in einem Postfilter, in Kombination mit einem Postfilter, im Rahmen eines Fehlerverdeckungsverfahrens oder in Zusammenhang mit einem Verfahren zur Sprach- und/oder Bildcodierung (Dekompressionsverfahren bzw. Dekodierungsverfahren) insbesondere empfängerseitig eingesetzt werden. Mit der Verwendung des Verfahrens als Postfilter ist dabei gemeint, dass das Verfahren zum Postfiltern eingesetzt wird, dass also mit einem das Verfahren umsetzenden Algorithmus die in den Anwendungen entstehenden Daten prozessiert werden. Weiterhin ist es möglich, die Qualität des Sprachsignals im Telekommunikationsnetzwerk zu verbessern, indem das Sprachsignalspektrum oder eine davon abgeleitete Größe mit dem erfindungsgemäßen Glättungsverfahren geglättet wird.The smoothing method can be used in a telecommunications network and / or in broadcasting to improve speech and / or picture quality and to suppress artifacts. In the mobile voice communication distortions of the speech signal occur, which are caused on the one hand by the speech coding (redundancy-reducing speech compression) and the associated quantization noise and on the other by the interference caused by the transmission channel. The latter, in turn, fluctuate greatly in terms of time and spectral and lead to a noticeable deterioration of the voice quality. Again, the receiver side or network used Signal processing ensure that the quasi-random artifacts are reduced. To improve quality, so-called post-filters and error concealment methods have hitherto been used. While the postfilter is primarily intended to reduce quantization noise, error concealment techniques are used to suppress transmission-related channel interference. Improvements can be achieved in both applications if the smoothing process according to the invention is integrated into the postfilter or the masking process. The smoothing method can thus be used as a postfilter, in a postfilter, in combination with a postfilter, as part of an error concealment method or in connection with a method for speech and / or picture coding (decompression method or decoding method), in particular on the receiver side. By using the method as a postfilter, it is meant that the method is used for post-filtering, that is to say that the data produced in the applications are processed with an algorithm implementing the method. Furthermore, it is possible to improve the quality of the speech signal in the telecommunications network by smoothing the speech signal spectrum or a variable derived therefrom with the smoothing method according to the invention.
Die Erfindung wird nachfolgend anhand von in den Figuren dargestellten Abbildungen näher erläutert. Es zeigen:
Figur 1- ein unverrauschtes Zeitsignal;
Figur 2- ein verrauschtes Zeitsignal;
Figur 3- einen einzelnen Signalrahmen im Zeitbereich;
Figur 4- einen einzelnen Signalrahmen im Spektralbereich;
- Figur 5
- eine Gewichtungsfunktion für einen einzelnen Rahmen;
- Figur 6
- das Spektrogramm eines unverrauschten Signals;
- Figur 7
- das Spektrogramm eines verrauschten Signals;
- Figur 8
- das Spektrogramm eines mit der ungeglätteten Gewichtungsfunktion gefilterten Signals;
- Figur 9
- das Spektrogramm eines mit einer erfindungsgemäß geglätteten Gewichtungsfunktion gefilterten Signals;
Figur 10- ein gefiltertes Zeitsignal mit tonalen Artefakten;
- Figur 11
- ein gemäß der Erfindung gefiltertes Zeitsignal;
- Figur 12
- das Spektrogramm einer ungeglätteten Gewichtungsfunktion;
- Figur 13
- das Spektrogramm einer erfindungsgemäß geglätteten Gewichtungsfunktion;
- Figur 14
- den Betrag des Cepstrums eines unverrauschten Sprachsignals und
- Figur 15
- den Signalflussgraphen gemäß einer bevorzugten Ausführungsform der Erfindung.
- FIG. 1
- a noisy time signal;
- FIG. 2
- a noisy time signal;
- FIG. 3
- a single signal frame in the time domain;
- FIG. 4
- a single signal frame in the spectral range;
- FIG. 5
- a weighting function for a single frame;
- FIG. 6
- the spectrogram of an unrestrained signal;
- FIG. 7
- the spectrogram of a noisy signal;
- FIG. 8
- the spectrogram of a signal filtered with the unsmoothed weighting function;
- FIG. 9
- the spectrogram of a signal filtered with a smoothed weighting function according to the invention;
- FIG. 10
- a filtered time signal with tonal artifacts;
- FIG. 11
- a time signal filtered according to the invention;
- FIG. 12
- the spectrogram of an unsmoothed weighting function;
- FIG. 13
- the spectrogram of a weighting function smoothed according to the invention;
- FIG. 14
- the amount of the cepstrum of a noisy speech signal and
- FIG. 15
- the signal flow graph according to a preferred embodiment of the invention.
In
In
In
In den
In
In
In den
In
In
In
In
Claims (34)
- Smoothing method for suppressing fluctuating artifacts during noise reduction, having the following steps:• short-term spectra for a series of signal frames comprising digital samples are provided,• each short-term spectrum is transformed by forward transformation, which describes the short-term spectrum using transformation coefficients which represent the short-term spectrum divided into its coarse and its fine structures,• the transformation coefficients with the same coefficient indices in each case are smoothed by combining at least two successive transformed short-term spectra, and• the smoothed transformation coefficients are transformed into smoothed short-term spectra by backward transformation.
- Smoothing method according to the preceding claim, characterized in that the inverse of the forward transformation is used for the backward transformation.
- Smoothing method according to Claim 1 or 2, characterized in that transformation with an orthogonal base is used.
- Smoothing method according to Claim 1 or 2, characterized in that transformation with a nonorthogonal base is used.
- Smoothing method according to Claim 1 or 2, characterized in that the discrete Fourier transformation and the inverse thereof are used for the transformations.
- Smoothing method according to Claim 1 or 2, characterized in that the fast Fourier transformation and the inverse thereof are used for the transformations.
- Smoothing method according to Claim 1 or 2, characterized in that the discrete cosine transformation and the inverse thereof are used for the transformations.
- Smoothing method according to Claim 1 or 2, characterized in that the discrete sine transformation and the inverse thereof are used for the transformations.
- Smoothing method according to one of the preceding claims, characterized in that the short-term spectra are mapped nonlinearly before the forward transformation.
- Smoothing method according to the preceding claim, characterized in that the smoothed short-term spectra are mapped nonlinearly after the backward transformation, wherein the nonlinear mapping of the backward transformation is the reversal of the nonlinear mapping of the forward transformation.
- Smoothing method according to one of the two preceding claims, characterized in that the short-term spectra are mapped nonlinearly before the forward transformation by logarithmization.
- Smoothing method according to one of Claims 1 to 11, characterized in that recursive smoothing is used for smoothing the transformation coefficients.
- Smoothing method according to one of Claims 1 to 11, characterized in that nonrecursive smoothing is used for smoothing the transformation coefficients.
- Smoothing method according to one of the preceding claims, characterized in that the smoothing is applied to the absolute value or to a power of the absolute value of the short-term spectra.
- Smoothing method according to one of the preceding claims, characterized in that different time constants are used for smoothing the respective transformation coefficients.
- Smoothing method according to the preceding claim, characterized in that time constants are chosen such that the transformation coefficients which typically describe spectral structures of voice are smoothed little.
- Smoothing method according to one of the two preceding claims, characterized in that time constants are chosen such that the transformation coefficients which describe spectral structures of fluctuating spectral magnitudes and of artifacts of noise reduction algorithms are smoothed much.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is a spectral weighting function of a noise reduction algorithm.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum used is a spectral weighting function of a post filter for multichannel methods for noise reduction.
- Smoothing method according to one of the two preceding claims, characterized in that the spectral weighting function results from the minimization of an error criterion.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is a filtered short-term spectrum.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is a spectral weighting function of a multichannel method for noise reduction.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is an estimated coherence or an estimated "Magnitude Squared Coherence" between at least two microphone channels.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is a spectral weighting function of a multichannel method for speaker or source separation.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is a spectral weighting function of a multichannel method for speaker separation on the basis of phase differences for signals in the different channels (Phase Transform - PHAT).
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is a spectral weighting function of a multichannel method for noise reduction on the basis of a "Generalized Cross-Correlation" (GCC).
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is spectral magnitudes which contain both voice and noise components.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is an estimate of the signal-to-noise ratio.
- Smoothing method according to one of Claims 1 to 17, characterized in that the short-term spectrum provided is an estimate of the noise power.
- Smoothing method according to one of Claims 1 to 15, characterized in that the short-term spectrum provided is transformed signal frames of an image signal, and the coefficients of the transformed image signal which are calculated row by row or column by column or two-dimensionally are subjected to spatial smoothing with different smoothing parameters.
- Smoothing method according to the preceding claim, characterized in that the image signal is a video signal.
- Smoothing method according to one of Claims 1 to 15, characterized in that the short-term spectrum used is a transformed medical signal derived from the human body.
- Smoothing method according to one of Claims 1 to 32, characterized in that the smoothing method is used in a post filter, in combination with a post filter, as part of an error masking method or in connection with a method for voice and/or image coding, particularly at the receiver end.
- Smoothing method according to one of Claims 1 to 33, characterized in that the smoothing method is used in a telecommunication network and/or during a broadcast transmission to improve the voice and/or image quality and to suppress artifacts.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102007030209A DE102007030209A1 (en) | 2007-06-27 | 2007-06-27 | smoothing process |
PCT/DE2008/001047 WO2009000255A1 (en) | 2007-06-27 | 2008-06-25 | Spectral smoothing method for noisy signals |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2158588A1 EP2158588A1 (en) | 2010-03-03 |
EP2158588B1 true EP2158588B1 (en) | 2010-10-13 |
Family
ID=39767094
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP08784249A Not-in-force EP2158588B1 (en) | 2007-06-27 | 2008-06-25 | Spectral smoothing method for noisy signals |
Country Status (6)
Country | Link |
---|---|
US (1) | US8892431B2 (en) |
EP (1) | EP2158588B1 (en) |
AT (1) | ATE484822T1 (en) |
DE (2) | DE102007030209A1 (en) |
DK (1) | DK2158588T3 (en) |
WO (1) | WO2009000255A1 (en) |
Families Citing this family (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ATE454696T1 (en) * | 2007-08-31 | 2010-01-15 | Harman Becker Automotive Sys | RAPID ESTIMATION OF NOISE POWER SPECTRAL DENSITY FOR SPEECH SIGNAL IMPROVEMENT |
US8588138B2 (en) * | 2009-07-23 | 2013-11-19 | Qualcomm Incorporated | Header compression for relay nodes |
US8577186B1 (en) * | 2011-02-14 | 2013-11-05 | DigitalOptics Corporation Europe Limited | Forward interpolation approach using forward and backward mapping |
US8675115B1 (en) | 2011-02-14 | 2014-03-18 | DigitalOptics Corporation Europe Limited | Forward interpolation approach for constructing a second version of an image from a first version of the image |
EP2689419B1 (en) | 2011-03-21 | 2015-03-04 | Telefonaktiebolaget L M Ericsson (PUBL) | Method and arrangement for damping dominant frequencies in an audio signal |
WO2012128678A1 (en) | 2011-03-21 | 2012-09-27 | Telefonaktiebolaget L M Ericsson (Publ) | Method and arrangement for damping of dominant frequencies in an audio signal |
GB201114737D0 (en) * | 2011-08-26 | 2011-10-12 | Univ Belfast | Method and apparatus for acoustic source separation |
US9026451B1 (en) * | 2012-05-09 | 2015-05-05 | Google Inc. | Pitch post-filter |
JP5772723B2 (en) * | 2012-05-31 | 2015-09-02 | ヤマハ株式会社 | Acoustic processing apparatus and separation mask generating apparatus |
CN105144290B (en) * | 2013-04-11 | 2021-06-15 | 日本电气株式会社 | Signal processing device, signal processing method, and signal processing program |
US20150179181A1 (en) * | 2013-12-20 | 2015-06-25 | Microsoft Corporation | Adapting audio based upon detected environmental accoustics |
DE102014210760B4 (en) * | 2014-06-05 | 2023-03-09 | Bayerische Motoren Werke Aktiengesellschaft | operation of a communication system |
JP6521056B2 (en) * | 2015-03-31 | 2019-05-29 | 日本電気株式会社 | Spectroscopic analyzer, spectral analysis method, and program |
US9721581B2 (en) * | 2015-08-25 | 2017-08-01 | Blackberry Limited | Method and device for mitigating wind noise in a speech signal generated at a microphone of the device |
US9972134B2 (en) | 2016-06-30 | 2018-05-15 | Microsoft Technology Licensing, Llc | Adaptive smoothing based on user focus on a target object |
EP4224833A3 (en) | 2018-05-09 | 2023-10-11 | Nureva Inc. | Method and apparatus utilizing residual echo estimate information to derive secondary echo reduction parameters |
EP3573058B1 (en) * | 2018-05-23 | 2021-02-24 | Harman Becker Automotive Systems GmbH | Dry sound and ambient sound separation |
JP7278092B2 (en) * | 2019-02-15 | 2023-05-19 | キヤノン株式会社 | Image processing device, imaging device, image processing method, imaging device control method, and program |
CN113726348B (en) * | 2021-07-21 | 2022-06-21 | 湖南艾科诺维科技有限公司 | Smoothing filtering method and system for radio signal frequency spectrum |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH02195400A (en) * | 1989-01-24 | 1990-08-01 | Canon Inc | Speech recognition device |
US5365592A (en) * | 1990-07-19 | 1994-11-15 | Hughes Aircraft Company | Digital voice detection apparatus and method using transform domain processing |
US5737485A (en) * | 1995-03-07 | 1998-04-07 | Rutgers The State University Of New Jersey | Method and apparatus including microphone arrays and neural networks for speech/speaker recognition systems |
US6070140A (en) * | 1995-06-05 | 2000-05-30 | Tran; Bao Q. | Speech recognizer |
DE19629132A1 (en) * | 1996-07-19 | 1998-01-22 | Daimler Benz Ag | Method of reducing speech signal interference |
US7272556B1 (en) * | 1998-09-23 | 2007-09-18 | Lucent Technologies Inc. | Scalable and embedded codec for speech and audio signals |
US6226606B1 (en) * | 1998-11-24 | 2001-05-01 | Microsoft Corporation | Method and apparatus for pitch tracking |
US6766292B1 (en) * | 2000-03-28 | 2004-07-20 | Tellabs Operations, Inc. | Relative noise ratio weighting techniques for adaptive noise cancellation |
JP3566197B2 (en) * | 2000-08-31 | 2004-09-15 | 松下電器産業株式会社 | Noise suppression device and noise suppression method |
US7054810B2 (en) * | 2000-10-06 | 2006-05-30 | International Business Machines Corporation | Feature vector-based apparatus and method for robust pattern recognition |
US6964023B2 (en) * | 2001-02-05 | 2005-11-08 | International Business Machines Corporation | System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input |
US7124075B2 (en) * | 2001-10-26 | 2006-10-17 | Dmitry Edward Terez | Methods and apparatus for pitch determination |
US20040002856A1 (en) * | 2002-03-08 | 2004-01-01 | Udaya Bhaskar | Multi-rate frequency domain interpolative speech CODEC system |
US7725314B2 (en) * | 2004-02-16 | 2010-05-25 | Microsoft Corporation | Method and apparatus for constructing a speech filter using estimates of clean speech and noise |
US7689419B2 (en) * | 2005-09-22 | 2010-03-30 | Microsoft Corporation | Updating hidden conditional random field model parameters after processing individual training samples |
US7680663B2 (en) * | 2006-08-21 | 2010-03-16 | Micrsoft Corporation | Using a discretized, higher order representation of hidden dynamic variables for speech recognition |
US8145488B2 (en) * | 2008-09-16 | 2012-03-27 | Microsoft Corporation | Parameter clustering and sharing for variable-parameter hidden markov models |
-
2007
- 2007-06-27 DE DE102007030209A patent/DE102007030209A1/en not_active Ceased
-
2008
- 2008-06-25 DK DK08784249.8T patent/DK2158588T3/en active
- 2008-06-25 US US12/665,526 patent/US8892431B2/en not_active Expired - Fee Related
- 2008-06-25 EP EP08784249A patent/EP2158588B1/en not_active Not-in-force
- 2008-06-25 DE DE502008001543T patent/DE502008001543D1/en active Active
- 2008-06-25 WO PCT/DE2008/001047 patent/WO2009000255A1/en active Application Filing
- 2008-06-25 AT AT08784249T patent/ATE484822T1/en active
Also Published As
Publication number | Publication date |
---|---|
WO2009000255A1 (en) | 2008-12-31 |
EP2158588A1 (en) | 2010-03-03 |
DE502008001543D1 (en) | 2010-11-25 |
US8892431B2 (en) | 2014-11-18 |
ATE484822T1 (en) | 2010-10-15 |
US20100182510A1 (en) | 2010-07-22 |
DE102007030209A1 (en) | 2009-01-08 |
DK2158588T3 (en) | 2011-02-07 |
WO2009000255A9 (en) | 2010-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2158588B1 (en) | Spectral smoothing method for noisy signals | |
DE60024501T2 (en) | Improvement of Perceptual Quality of SBR (Spectral Band Replication) AND HFR (Radio Frequency Reconstruction) Coding method by adaptively adding noise floor and limiting the noise substitution | |
DE60131639T2 (en) | Apparatus and methods for determining noise cancellation performance values for a voice communication system | |
DE112009000805B4 (en) | noise reduction | |
DE69428119T2 (en) | REDUCING BACKGROUND NOISE FOR LANGUAGE ENHANCEMENT | |
DE19747885B4 (en) | Method for reducing interference of acoustic signals by means of the adaptive filter method of spectral subtraction | |
DE60225130T2 (en) | IMPROVED TRANSIENT PERFORMANCE FOR LOW-BITRATE CODERS THROUGH SUPPRESSION OF THE PREVIOUS NOISE | |
EP1143416B1 (en) | Time domain noise reduction | |
DE60116255T2 (en) | NOISE REDUCTION DEVICE AND METHOD | |
DE60027438T2 (en) | IMPROVING A HARMFUL AUDIBLE SIGNAL | |
DE69630580T2 (en) | Noise canceller and method for suppressing background noise in a noisy speech signal and a mobile station | |
DE3689035T2 (en) | NOISE REDUCTION SYSTEM. | |
DE60104091T2 (en) | Method and device for improving speech in a noisy environment | |
DE60031354T2 (en) | Noise cancellation before voice coding | |
DE112012006876T5 (en) | Formant-dependent speech signal enhancement | |
EP3197181B1 (en) | Method for reducing latency of a filter bank for filtering an audio signal and method for low latency operation of a hearing system | |
AT509570B1 (en) | METHOD AND APPARATUS FOR ONE-CHANNEL LANGUAGE IMPROVEMENT BASED ON A LATEN-TERM REDUCED HEARING MODEL | |
EP3065417B1 (en) | Method for suppressing interference noise in an acoustic system | |
EP1239455A2 (en) | Method and system for implementing a Fourier transformation which is adapted to the transfer function of human sensory organs, and systems for noise reduction and speech recognition based thereon | |
EP2080197B1 (en) | Apparatus for noise suppression in an audio signal | |
DE602004006912T2 (en) | A method for processing an acoustic signal and a hearing aid | |
EP1453355B1 (en) | Signal processing in a hearing aid | |
DE69802431T2 (en) | METHOD AND DEVICE FOR IMPROVING A DIGITAL VOICE SIGNAL | |
AT408286B (en) | METHOD FOR SUPPRESSING NOISE IN A SIGNAL FIELD | |
DE102018131687B4 (en) | METHODS AND DEVICES FOR REDUCING CLOPPING NOISE |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20091210 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA MK RS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
DAX | Request for extension of the european patent (deleted) | ||
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REF | Corresponds to: |
Ref document number: 502008001543 Country of ref document: DE Date of ref document: 20101125 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: NV Representative=s name: SIEMENS SCHWEIZ AG |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20101013 |
|
LTIE | Lt: invalidation of european patent or patent extension |
Effective date: 20101013 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110113 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FD4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110113 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110213 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110214 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110114 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110124 Ref country code: IE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 |
|
26N | No opposition filed |
Effective date: 20110714 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502008001543 Country of ref document: DE Effective date: 20110714 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 |
|
BERE | Be: lapsed |
Owner name: RUHR-UNIVERSITAT BOCHUM Effective date: 20110630 Owner name: SIEMENS AUDIOLOGISCHE TECHNIK G.M.B.H. Effective date: 20110630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110630 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20120626 Year of fee payment: 5 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110625 Ref country code: CY Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION Effective date: 20101013 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20101013 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130625 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 484822 Country of ref document: AT Kind code of ref document: T Effective date: 20130625 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130625 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 502008001543 Country of ref document: DE Representative=s name: FDST PATENTANWAELTE FREIER DOERR STAMMLER TSCH, DE |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 502008001543 Country of ref document: DE Representative=s name: FDST PATENTANWAELTE FREIER DOERR STAMMLER TSCH, DE Ref country code: DE Ref legal event code: R081 Ref document number: 502008001543 Country of ref document: DE Owner name: RUHR-UNIVERSITAET BOCHUM, DE Free format text: FORMER OWNERS: RUHR-UNIVERSITAET BOCHUM, 44801 BOCHUM, DE; SIEMENS AUDIOLOGISCHE TECHNIK GMBH, 91058 ERLANGEN, DE Ref country code: DE Ref legal event code: R081 Ref document number: 502008001543 Country of ref document: DE Owner name: SIVANTOS GMBH, DE Free format text: FORMER OWNERS: RUHR-UNIVERSITAET BOCHUM, 44801 BOCHUM, DE; SIEMENS AUDIOLOGISCHE TECHNIK GMBH, 91058 ERLANGEN, DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DK Payment date: 20190624 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20190626 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20190624 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20190624 Year of fee payment: 12 Ref country code: GB Payment date: 20190624 Year of fee payment: 12 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 502008001543 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: EBP Effective date: 20200630 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20200625 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200630 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200625 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200630 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200630 |