JP4597919B2 - Acoustic signal feature extraction method, extraction device, extraction program, recording medium recording the program, acoustic signal search method, search device, search program using the features, and recording medium recording the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound signal retrieval device capable of accurately retrieving a sound signal at high speed in a flexible way, even when frequency of each sound signal is fluctuating. <P>SOLUTION: In the sound signal retrieval device 1, a base frequency detection section 11 extracts a predetermined base feature part for each time interval, from each time frequency spectrogram of a reference signal and an accumulation signal, and frequency of an extracted predetermined base feature part is made into base frequency of each of the reference signal and the accumulation signal. A feature extraction section 12 extracts a reference feature amount in a whole signal of the reference signal based on a power spectrum value of the frequency which is uniquely defined with the base frequency as a base. A similarity calculation section 13 extracts a feature amount of the accumulation signal for each time interval, based on the extracted reference feature amount, and calculates an interval feature amount of the extracted accumulation signal for each interval, and a similarity degree is calculated based on the calculated interval feature amount and the reference feature amount. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to an acoustic signal search in which a reference signal that is an acoustic signal is used as a search key, and a position of a signal similar to the reference signal is searched from an accumulated signal that is an acoustic signal longer than the reference signal. Acoustic signal feature extraction method, extraction device, extraction program, recording medium on which the program is recorded, and acoustic signal search method and search device using the feature, which can cope with frequency fluctuation (transposition or pitch shift) of each acoustic signal The present invention relates to a search program and a recording medium on which the program is recorded.

  In recent years, acoustic signal search technology using a feature extraction method and a feature matching method as elements has been put to practical use in broadcast monitoring, playlist distribution, music recognition services, and the like. In this method, when music (or a part thereof) is input, the feature is automatically extracted, and desired information can be detected instantaneously by collating with the feature of the music stored in the database.

  For example, a “time-series active search (TAS) method” (see Patent Document 1) is known as a fast search method for acoustic signals. This is to generate a histogram as feature data from the time series signal and calculate the similarity from the overlapping rate of the generated histogram. Also, when matching the features, calculate the interval that does not require matching in the time axis direction. Since the search can be skipped, useless collation can be omitted, and the search speed can be greatly improved while maintaining a practically sufficient search accuracy.

  Further, as a technique for searching for background music from an acoustic signal covered with narration, such as a TV drama BGM or a radio DJ program, there is a “division match search (DAL) method” disclosed in Patent Document 2. The principle of the DAL method is to divide a sound signal serving as a search key into small regions in the time and frequency directions, and to search for a small region that matches each small region from the search target signal. Thus, it is possible to find an area where no narration is superimposed and to detect a section in which the music being played is the same as the search key.

  However, with these methods, for example, even if the sound signals were originally the same, it was not possible to detect the similarity for the sound signals reproduced by changing the pitch. There is a humming search method as a method for searching for the same music with different pitches. Ghias et al., In particular, uses a melodic contour to represent a single melodic pitch sequence as a string of relative pitches of “S (same)”, “U (up)”, and “D (down)”. And a search method for similar music that searches for similar single melodies using Hamming as a search key (see Non-Patent Document 1).

The melodic contour is simpler in terms of melodic expression than a musical score, but it is effective for identifying and collating simple melodies. It is also used as a feature in the search for simple melodies in other studies. Many similar music searches so far use a single melody pitch sequence input by humming or the like as a search key, and search for similar ones from a database consisting of single melody pitch sequences. The search was based on collation. Here, in many methods, the pitch sequence of the database is a single melody pitch sequence from which the main melody has already been extracted, which is obtained from the input of the main melody by humming as well as the MIDI, the score information, or the search key. . Therefore, in order to use these humming search methods for searching for an acoustic signal using a normal music CD or a sound being broadcast, which is usually a multiplayer, the main melody extraction from the multitone or the multiplayer and the multiplayer There are many difficult problems such as the need to check the main melody between the two.
Japanese Patent No. 30653314 JP 2004-102023 A A. Ghias, J. Logan, D. Chamberlin and BC Smith “Query by humming: Musical information retrieval in an audio database,” Proc. ACM Multimedia '95, pp.231-236, San Francisco, USA, Nov. 1995.

  As described above, the conventional search for matching acoustic signals has a problem that it is difficult to search for an acoustic signal reproduced with a different pitch even for the same acoustic signal.

  In addition, the method of adding frequency fluctuations to an input acoustic signal has a problem that the amount of information required for collation is greatly increased.

  The present invention has been made to solve the above problem, and when a reference signal is input as a search key, a section in an accumulated signal including a matching signal can be flexibly dealt with a frequency variation of each acoustic signal. Acoustic signal feature extraction method, extraction device, extraction program, recording medium storing the program, and acoustic signal search method, search device, search program using the feature Another object is to provide a recording medium on which the program is recorded.

In order to solve the above problem, the present invention provides an acoustic signal feature extraction method in an acoustic signal feature extraction device that extracts a feature amount of an acoustic signal based on a time-frequency spectrogram of the input acoustic signal, A predetermined reference feature portion of the time frequency spectrogram is extracted for each time interval of the time frequency spectrogram, and the extracted frequency of the predetermined reference feature portion is set as a reference frequency for each time interval, and is predetermined on a logarithmic axis. And extracting a plurality of frequencies at intervals with reference to the reference frequency, and extracting a feature amount of the acoustic signal at each time interval based on the extracted power spectrum value of each frequency. Is the method.

  The present invention is characterized in that, in the invention described above, the predetermined reference feature portion is a portion where a power spectrum value becomes a maximum value in each time interval.

  The present invention is characterized in that, in the invention described above, the predetermined reference feature portion is a portion in which a power spectrum value is a maximum value in several time frequency spectrograms continuous on a time axis. To do.

  The present invention is characterized in that, in the above-described invention, the predetermined reference feature portion is a portion serving as a center of gravity of power spectrum values of several time-frequency spectrograms continuous on a time axis.

The present invention is an acoustic signal search method in an acoustic signal search device for searching a section including a sound similar to the reference sound signal from an input search target sound signal based on a reference sound signal, the reference sound A predetermined reference feature for each time interval of the time frequency spectrogram is extracted from a time frequency spectrogram of each of the signal and the search target acoustic signal, and the predetermined reference feature portion extracted from the time frequency spectrogram of the reference acoustic signal the frequency with the reference frequency of the reference acoustic signal for each of said time intervals, the search target acoustic signal of the time the search target acoustic signal of the reference of each said time interval the frequency of the predetermined reference feature extracted from the frequency spectrogram and frequency, a plurality of frequencies of predetermined intervals on a logarithmic axis, of the reference acoustic signal Extracting, based on the quasi-frequency, based on the extracted power spectrum value of each frequency extracts a feature quantity of the audio signal for each of the time intervals of the reference acoustic signal, the reference acoustic signal on the basis of the extracted feature quantity Calculate a reference feature value with the time length of the entire signal as one section , extract a plurality of frequencies at predetermined intervals on the logarithmic axis with reference to the reference frequency of the search target acoustic signal , and extract each frequency The feature quantity for each time interval of the search target acoustic signal is extracted based on the power spectrum value of the search target section, and the section feature quantity for each section is calculated based on the extracted feature quantity of the search target acoustic signal. The similarity is calculated based on the section feature quantity and the reference feature quantity, and the section of the search target acoustic signal similar to the sound of the reference acoustic signal is searched based on the calculated similarity degree. Is an acoustic signal search method characterized by outputting a section of the search target acoustic signal similar to the sound of the reference acoustic signal detected by the search.

The present invention is an acoustic signal feature extraction device that extracts a feature amount of an acoustic signal based on a temporal frequency spectrogram of an input acoustic signal, and for each time interval of the temporal frequency spectrogram, the temporal frequency spectrogram includes: A predetermined reference feature portion is extracted, a reference frequency detecting means that uses the extracted frequency of the predetermined reference feature portion as a reference frequency for each time interval, and a plurality of frequencies at predetermined intervals on the logarithmic axis, An acoustic signal feature comprising: feature amount extraction means for extracting the feature amount of the acoustic signal for each time interval based on the extracted power spectrum value of each frequency based on the reference frequency It is an extraction device.

The present invention is an acoustic signal search device for searching an input search target acoustic signal for a section including a sound similar to the reference acoustic signal based on the reference acoustic signal, the reference acoustic signal and the search target Extracting a predetermined reference feature for each time interval of the time frequency spectrogram from the time frequency spectrogram of each acoustic signal, and extracting the frequency of the predetermined reference feature extracted from the time frequency spectrogram of the reference acoustic signal as the time interval a reference frequency of the reference acoustic signal of each, the reference frequency as a reference frequency of the search target acoustic signal of each of the search target said time interval the frequency of the predetermined reference feature extracted from time-frequency spectrogram of the acoustic signals detection means, a plurality of frequencies of predetermined intervals on a logarithmic axis, based on the reference acoustic signal The frequency is extracted as a reference, based on the extracted power spectrum value of each frequency and extracts a feature of each said time interval of said reference sound signal, based on the extracted feature quantity of the whole signal of the reference acoustic signal time A reference feature value having a length as one section is calculated, a plurality of frequencies at predetermined intervals on the logarithmic axis are extracted with reference to a reference frequency of the search target acoustic signal, and a power spectrum value of each extracted frequency Based on the feature extraction means for extracting the feature quantity for each time interval of the search target acoustic signal, and the section feature quantity for each section based on the feature quantity of the search target acoustic signal extracted by the feature extraction means A similarity is calculated based on the calculated section feature and the reference feature, and the search pair similar to the sound of the reference acoustic signal is calculated based on the calculated similarity. An acoustic signal retrieval device comprising: a similarity calculation unit that retrieves an acoustic signal section and outputs the retrieval target acoustic signal section similar to the sound of the reference acoustic signal detected by the search. is there.

The present invention provides a computer of an acoustic signal feature extraction apparatus that extracts a feature quantity of an acoustic signal based on the temporal frequency spectrogram of the input acoustic signal, and transmits the time frequency spectrogram of the time frequency spectrogram for each time interval of the time frequency spectrogram. A step of extracting a predetermined reference feature, a step of setting the extracted frequency of the predetermined reference feature as a reference frequency for each time interval, and a plurality of frequencies at predetermined intervals on a logarithmic axis, An acoustic signal feature extraction program for executing a step of extracting a feature amount of the acoustic signal for each time interval based on a power spectrum value of each extracted frequency.

The present invention relates to a computer of an acoustic signal search device that searches an input search target acoustic signal for a section including a sound similar to the reference acoustic signal based on the reference acoustic signal. Inputting a time frequency spectrogram of the reference frequency, extracting a predetermined reference feature for each time interval of the time frequency spectrogram from the time frequency spectrogram of each of the reference sound signal and the search target sound signal, and the reference sound The frequency of the predetermined reference feature extracted from the time frequency spectrogram of the signal is set as the reference frequency of the reference acoustic signal for each time interval, and the frequency of the predetermined reference feature extracted from the time frequency spectrogram of the search target acoustic signal the search target acoustic signal of the group of each said time interval a frequency A step of frequency, multiple frequencies of predetermined intervals on a logarithmic axis, the reference frequency of the reference acoustic signal extracted as a reference, extracted of the reference acoustic signal on the basis of the power spectrum value of each frequency Extracting a feature value for each time interval, calculating a reference feature value having one time as a whole time length of the signal of the reference acoustic signal based on the extracted feature value, and predetermined on a logarithmic axis Extracting a plurality of frequencies in the interval with reference to a reference frequency of the search target acoustic signal, and extracting a feature amount for each time interval of the search target acoustic signal based on the extracted power spectrum value of each frequency; Calculating a section feature amount for each section based on the extracted feature amount of the search target acoustic signal, and the similarity calculation unit calculates the section feature A step of calculating a similarity based on the amount and the reference feature, and searching for a section of the search target acoustic signal similar to the sound of the reference acoustic signal based on the calculated similarity and detecting by searching And a step of outputting a section of the search target acoustic signal similar to the sound of the reference acoustic signal.

  The present invention is a computer-readable recording medium on which the acoustic signal feature extraction program described above is recorded.

  The present invention is a computer-readable recording medium in which the above-described acoustic signal search program is recorded.

  According to the present invention, for each time interval of the time frequency spectrogram, a predetermined reference feature portion of the time frequency spectrogram is extracted, and the extracted frequency of the predetermined reference feature portion is set as a reference frequency for each time interval, and the reference frequency Based on the power spectrum value of the frequency uniquely determined with reference to the above, the feature amount of the acoustic signal for each time interval is extracted. Thereby, from each time frequency spectrogram of the acoustic signal, a predetermined reference feature portion of the power spectrum is detected and set as a reference frequency in each time interval, and by extracting the feature of the power spectrum based on the reference frequency, Even if fluctuation due to frequency occurs, feature extraction that is not affected by the fluctuation can be performed.

  Further, according to the present invention, the predetermined reference feature portion is a portion where the power spectrum value becomes a maximum value at each time interval. This makes it possible to perform feature extraction using the frequency value at which the power spectrum value is maximum as the reference frequency.

  In addition, according to the present invention, the predetermined reference feature portion is a portion in which the power spectrum value becomes the maximum value among several time-frequency spectrograms continuous on the time axis. This makes it possible to set the most appropriate reference frequency for feature extraction within a continuous time.

  Further, according to the present invention, the predetermined reference feature portion is a portion that becomes the center of gravity of the power spectrum values of several time-frequency spectrograms that are continuous on the time axis. This makes it possible to set the most appropriate reference frequency for feature extraction even if the acoustic signal contains noise or the like.

  Further, according to the present invention, a predetermined reference feature for each time interval of the time frequency spectrogram is extracted from the time frequency spectrogram of each of the reference acoustic signal and the search target acoustic signal, and the frequency of the extracted predetermined reference feature is extracted. Is the reference frequency of each of the reference acoustic signal and the search target acoustic signal for each time interval, and the characteristics of the reference acoustic signal for each time interval based on the power spectrum value of the frequency uniquely determined based on the reference frequency of the reference acoustic signal A reference feature value having a time length of the entire signal of the reference acoustic signal as one section based on the extracted feature value, and a frequency of a frequency uniquely determined based on the reference frequency of the search target acoustic signal. Based on the power spectrum value, the feature quantity for each time interval of the search target acoustic signal is extracted, and the extracted search target A section feature amount for each section is calculated based on the feature amount of the sound signal, a similarity is calculated based on the calculated section feature amount and the reference feature amount, and a reference acoustic signal is calculated based on the calculated similarity degree. The section of the search target acoustic signal similar to the sound of the search target acoustic signal is searched, and the section of the search target acoustic signal similar to the sound of the reference acoustic signal detected by the search is output. As a result, a predetermined reference feature of the power spectrum is detected from each time frequency spectrogram of the reference sound signal and the search target sound signal and set as a reference frequency at each time interval, that is, fluctuation due to the frequency has occurred. However, it is possible to set a reference frequency capable of extracting features that do not depend on the frequency variation. Then, the frequency spectrum feature of each signal is extracted based on the reference frequency, and the acoustic signal can be detected corresponding to the vertical fluctuation of the frequency based on the extracted frequency spectrum feature.

(Principle of the present invention)
First, the principle of the present invention will be described with reference to FIGS. The present invention is characterized in that, at the time of extracting an acoustic feature, the frequency at which the acoustic feature is extracted is shifted in accordance with a change in the frequency of the acoustic signal at each time point when the acoustic feature is extracted. For example, it is assumed that the power spectrogram of an acoustic signal is expressed as shown in FIG. 1 when the frequency axis is expressed logarithmically. At this time, as shown in FIG. 2, the acoustic feature is extracted as the power of each frequency of f1, f2,..., Fc,. At this time, when the frequency of the acoustic signal is reproduced by multiplying by a, the power spectrogram is shifted by a times in the frequency axis direction while maintaining the characteristics of the change of the power spectrogram value as shown in FIG. Become. That is, if this a-times is used as transposition or pitch shift, if the value of a is known, it is possible to absorb the influence of frequency fluctuation due to transposition or pitch shift.

  Therefore, in the present invention, fc is used as a reference frequency for extracting acoustic features, for example, the peak of the spectrum at that time, and f1, f2,..., Fc,. fk is extracted, an average value of the extracted power spectrum values for each frequency is calculated, and the power spectrum value for each frequency is normalized by the calculated average value. Then, as shown in FIG. 4, the afc is obtained as the reference frequency by matching the reference frequency with the peak of the spectrum at that time in the same manner for the acoustic signal subjected to transposition and pitch shift corresponding to a times. , Afc,..., Afk are extracted as frequencies indicating acoustic features with the afc as a reference, and the power spectrum value for each extracted frequency is normalized in the same manner as described above. Although the two acoustic features obtained in this way have a difference in frequency value, when observed with fc and afc as a reference, the normalized power spectrum values at the corresponding frequencies are the same value, and are due to frequency fluctuations. It is possible to extract acoustic features whose influence is absorbed. Hereinafter, embodiments of the present invention that realize the principle will be described with reference to the drawings.

(First embodiment of the present invention)
Next, the acoustic signal search device 1 according to the first embodiment of the present invention will be described. The acoustic signal search device 1 inputs an acoustic signal to be searched, that is, a reference signal, and detects a section including a signal similar to the reference signal from stored signals that are longer acoustic signals. Further, the acoustic signal search device 1 is robust against the fluctuation even if there is a frequency fluctuation of the acoustic signal by applying the above-described principle, that is, the search is made possible by absorbing the fluctuation. In the present specification, detecting a section including a signal similar to the reference signal in the accumulated signal is defined as detecting the beginning time of the section in which this section starts.

  FIG. 5 is a block diagram illustrating a configuration of the acoustic signal search device 1 according to the first embodiment. The acoustic signal search device 1 includes a reference frequency detection unit 11, a feature extraction unit 12, and a similarity calculation unit 13. In the acoustic signal search device 1, when a reference signal and an accumulated signal are input, first, the signal is converted by a conversion unit (not shown), and a time-frequency spectrogram (hereinafter referred to as a spectrogram) of each signal is output.

  The reference frequency detector 11 receives the spectrograms of the reference signal and the accumulated signal from the conversion unit, detects a characteristic part of the power spectrum from the input reference signal and the accumulated signal spectrogram, A reference frequency fc at the time is set. The feature extraction unit 12 calculates the features of the reference signal and the accumulated signal based on the reference frequency fc at each time point detected by the reference frequency detection unit. The similarity calculation unit 13 sets an attention window having the same length as the reference signal as an accumulated signal, calculates the similarity between the characteristics of the reference signal and the characteristics of the accumulated signal within the attention window, Similarity is determined by comparing with a threshold.

  Next, the operation of the above-described acoustic signal search device 1 will be described. FIG. 6 is a flowchart showing an operation example of the acoustic signal search apparatus 1 of FIG. The acoustic signal search apparatus 1 performs input by reading a reference signal that is a search key (step S201), and a spectrogram is output by the conversion unit described above (step S202). Here, the conversion to the spectrogram by the conversion unit is performed, for example, by performing a fast Fourier transform or by using a plurality of band pass filters.

  The spectrogram of the reference signal output by the conversion unit is input to the reference frequency detection unit 11, and the reference frequency detection unit 11 has a power spectrum at each time point of the frequency spectrum constituting the spectrogram as shown in FIG. The highest frequency is detected as a characteristic part, and the detected frequency is output as the reference frequency fc at this time (step S203).

  Note that the reference frequency detection unit 11 does not determine the reference frequencies for all the frequency spectra sampled as shown in FIGS. 7A and 7B, but instead uses time axes as shown in FIG. The frequency having the maximum intensity among a plurality of consecutive frequency spectra may be set as the reference frequency of these frequency spectra. In addition, the center of gravity of a plurality of frequency spectra continuous in the time axis direction may be obtained, and the frequency corresponding thereto may be set as the reference frequency of the frequency spectrum within this time.

  In addition, the reference frequency determination method in the reference frequency detection unit 11 is not limited to the above-described method. For example, a plurality of power spectrum peaks are detected at each sampling time, and the Xth frequency is selected, or the intensity is selected. The reference frequency can be determined by selecting a point shifted by several Hz from the frequency at which Y becomes Y. Further, a point that becomes a trough appearing between peaks of the plurality of detected power spectra may be used as a reference frequency.

  The feature extraction unit 12 selects a plurality of power spectrum values at predetermined intervals on the frequency axis based on the reference frequency fc detected by the reference frequency detection unit 11 for the reference signal. For example, a plurality of power spectrum values may be selected at equal intervals on the logarithmic axis. Then, the feature extraction unit 12 extracts a feature vector having the selected power spectrum value as an element as a frequency spectrum feature (a feature amount of the reference signal) at this time. Next, the feature extraction unit 12 encodes a feature vector that is a frequency spectrum feature at each time point by vector quantization (step S204). Then, the feature extraction unit 12 further generates a histogram (hereinafter, the histogram is referred to as a histogram feature) in which the number of appearances of these codes within a predetermined time is counted for each code, and the generated histogram feature is used for the acoustic signal. It transmits to the similarity calculation part 13 as intrinsic | native information, ie, an acoustic fingerprint (step S205). Note that the predetermined time for generating the histogram is equal to the time length of the reference signal.

  On the other hand, the acoustic signal search device 1 reads and inputs the acoustic waveform signal of the accumulated signal to be searched (step S206), and extracts the time-frequency spectrogram by the same method as the reference signal (step S207). Then, the reference frequency detection unit 11 and the feature extraction unit 12 detect the reference frequency fc similarly to the reference signal (step S208), obtain a feature vector that becomes a frequency spectrum feature at each time point, and encode it by vector quantization. (Step S209). In the case of an accumulated signal, the feature extraction unit 12 does not output a histogram feature but outputs a code sequence obtained by encoding a feature vector to the similarity calculation unit 13.

  In general, the code sequence of the accumulated signal is stored in the accumulated signal database, and when the search process is started, the accumulated signal database is sequentially accessed to read the accumulated signal code sequence. However, it is not limited to this configuration. For example, the reference signal and the accumulated signal may be input at the same time, the histogram characteristics of the reference signal may be stored in the similarity calculation unit 13, and the code sequence of the accumulated signal input in real time may be collated. it can.

  When the histogram feature of the reference signal is input from the feature extraction unit 12, the similarity calculation unit 13 sets an attention window having the same time length as that of the reference signal from the head of the code sequence of the input accumulated signal (step) S210). Then, a histogram is generated by counting the number of appearances of the code in the attention window for each code (step S211), and the similarity between the generated histogram and the histogram feature of the reference signal is calculated (step S212). As a calculation method, for example, a time series active search (TAS) method (Patent Document 1) can be applied.

Here, the TAS method will be described with reference to FIG. TAS method, the similarity S _IR with histograms overlapping ratio between the reference signal histogram feature is intended to explore all the sections on the storage signal having a histogram feature that exceeds the search threshold value theta. Specifically, as described above, the similarity calculation unit 13 first sets an attention window having the same time length as that of the reference signal from the top in the code sequence that is a characteristic of the accumulated signal. Next, the similarity is calculated S _IR with histogram feature H _I in the target window with histogram feature H _R of the reference signal stored signal code sequence. The similarity _SIR can be calculated by the following equation (1).

However, In formula (1), L is the number of bins in the histogram, D is the total frequency of the _{histogram, h} lγ, _h Rγ, respectively _H I, the power contained in the γ-th bin of _{H R} Represents. In the present embodiment, as the spectrogram corresponding to the input in FIG. 8, the frequency spectrum features of the reference signal arranged in time series and the frequency spectrum features of the accumulated signal arranged in time series are applied to each frequency. Vector quantization is performed on the spectral features.

When the calculated similarity value exceeds the threshold θ, the similarity calculation unit 13 determines that the section in the accumulated signal is similar and outputs a detection result. When the similarity calculation for the set attention window is completed, the similarity calculation unit 13 sets a new attention window for the code sequence of the accumulated signal. At this time, based on the already calculated similarity _SIR , it is a value representing where the next window of interest can be set by omitting collation with a section on the stored signal that is guaranteed not to be similar. A skippable width (movable amount) k is calculated (step S213). The skippable width k can be calculated based on the following equation (2).

  That is, the window of interest is set by obtaining the upper limit value of the similarity in the vicinity of the current position of the window of interest from the similarity based on the histogram overlap rate, and determining the position of the next window of interest in the accumulated signal based on the upper limit value. Will do.

  The similarity calculation unit 13 determines whether or not the set position of the target window has reached the end of the stored signal code sequence (step S214), and performs the processing from step S210 to step S213 until the end of the stored signal code sequence is reached. The similarity calculation is performed while appropriately moving the target window on the accumulated signal. When the window of interest reaches the end of the accumulated signal and the collation processing is completed for the accumulated signal, the acoustic signal search device 1 outputs a detection result and ends all the search processing (step S215).

  Note that the order of setting the attention window is not limited to the configuration in which the code sequence that is the feature amount of the accumulated signal shown in step S210 is a target. For example, as shown in the flowchart of FIG. May be performed on the signal waveform (FIG. 9: step S311). In the case of FIG. 9, spectrogram extraction, reference frequency detection, feature amount extraction, and histogram generation are performed on the information in the window of interest (FIG. 9: Step S312 to Step S315), and the similarity calculation with the histogram feature of the reference signal is performed. This is performed (step S212).

  As in the flowchart shown in FIG. 10, first, similarly to the flowchart in FIG. 6, feature extraction is performed from the accumulated signal (steps S206 to S209), and histogram features at each time point are calculated in advance ( FIG. 10: Step S321), the window of interest may be set for the histogram series of accumulated signals (FIG. 10: Step S322).

  Moreover, there is an acoustic signal search device 1a shown in FIG. 11 as another form of the acoustic signal search device 1 in the first embodiment described above. In addition to the configuration of the acoustic signal search device 1 illustrated in FIG. 5, the acoustic signal search device 1 a includes a coding unit 14 that codes the histogram feature extracted by the feature extraction unit 12. In the acoustic signal search device 1a, the histogram feature is encoded by vector quantization or the like, and the acoustic fingerprint of each acoustic signal is expressed as a code sequence, so that the similarity calculation unit 13a obtains a reference signal and an interval in the accumulated signal. The similarity can be calculated from the similarity between the code sequences, which greatly reduces the amount of calculation required for collation.

  Note that, instead of encoding a histogram feature for a code sequence that is an acoustic fingerprint of each acoustic signal output from the encoding unit 14 of the accumulated signal search device 1a, a frequency spectrum feature (or a frequency sequence feature calculated by the feature extraction unit 12) (or A plurality of spectral features that are a series of consecutive feature vectors) may be calculated by direct vector quantization.

  Furthermore, there is an acoustic signal retrieval apparatus 1b shown in FIG. 12 as another form of the acoustic signal retrieval apparatus 1 in the first embodiment described above. The acoustic signal search device 1b is configured to include an index generation unit 15 in addition to the configuration of the acoustic signal search device 1a illustrated in FIG. The index generating unit 15 generates index information indicating the location on the accumulated signal in advance for each code of the accumulated signal output by the encoding unit 14. With this configuration, the position of the code on the accumulated signal that is similar to the code of the reference signal can be easily detected from the index information, whereby the similarity calculation unit 13b can detect the similar code as the accumulated signal. The search process above is omitted, and the similarity in the section on the accumulated signal can be calculated at higher speed. In FIG. 12, the solid line indicates the flow of processing for the spectrogram of the reference signal, and the broken line indicates the flow of processing for the spectrogram of the accumulated signal.

  FIG. 13 is a flowchart illustrating an operation example of the acoustic signal search device 1b illustrated in FIG. In the flowchart, the same operations as those described in FIG. 6 are denoted by the same step numbers, and different operations will be described below.

  First, in the acoustic signal search apparatus 1b, the encoding unit 14 encodes the histogram feature of each acoustic signal calculated by the feature extraction unit 12 (steps S601 and S602). The index generation unit 15 generates index information for each code of the accumulated signal (step S801). The similarity calculation unit 13b detects the code of the accumulated signal similar to the code of the reference signal by index search (step S803: Yes), and collates in the accumulated signal that may be similar from the detected code of the accumulated signal. The section is output (step S804). Next, the similarity calculation unit 13b repeatedly selects a matching section until there is no output matching section, and calculates the similarity with the reference signal for each matching section, that is, the section similarity (steps S805 to S807). Then, the similarity calculation unit 13b compares the section similarity with a search threshold that is a predetermined threshold, determines that the section similarity exceeds the search threshold, is similar, and the section similarity is the search threshold. The detection results are output as those not exceeding similarity if not exceeding (steps S808 and S809). In step S803, when the similarity calculation unit 13b cannot detect the code of the accumulated signal similar to the code of the reference signal by index search (step S803: No), the result is output as a detection result (step S809). .

(Example of implementation of the first embodiment)
Next, an implementation example of the above-described first embodiment will be described.
First, an acoustic signal (sampling frequency 44.1 kHz, 16-bit quantization, stereo) used as a music CD or the like is down-sampled, monauralized, and the number of quantization bits is changed (sampling frequency 11.025 kHz). , 8-bit quantization, monaural). The acoustic signal is first subjected to 1024 FFT (Fast Fourier Transform) every 10 ms, and the power spectrum at that time is extracted every 10 ms. Then, the spectrogram is obtained assuming that the spectra are arranged in the time direction every 10 ms. Moreover, power is extracted at each frequency of ((1 / x ³ ) × fc, (1 / x ² ) × fc, (1 / x) × fc, x × fc, x ² × fc, x ³ × fc). Then, the average value of the power of each extracted frequency is calculated, and the characteristic vector obtained by normalizing the power of each frequency with the calculated average value is used. Here, x is ^71/6 .

And let fc be the frequency where the power is the maximum among the frequencies of 10% above and below from 1300 Hz in the previously extracted spectrum. Based on this fc, at each time point, ((1 / x ³ ) × fc, (1 / x ² ) × fc, (1 / x) × fc, x × fc, x ² × fc, x ³ × fc) Of the frequencies obtained from the FFT result, the power of the frequency closest to these frequencies is ((1 / x ³ ) × fc, (1 / x ² ) × fc, (1 / x) × fc, x × fc, x ² × fc, x ³ × fc), the average value of the power of each frequency is calculated, and the power of each frequency is normalized by the calculated average value Extracted as a feature vector.

  Further, for the frequency corresponding to each component of the feature vector, the power spectrum may be extracted again using a bandpass filter whose center frequency is the frequency.

  In this way, by extracting the feature vector, it is possible to perform a search even with an acoustic signal whose pitch varies up and down by 10%.

The dimension of these feature vectors (7 in the above implementation example), the value of x (7 ^{1/6 in the} above implementation example), and the setting range of fc (10% in the above implementation example) are signals to be searched. It can be changed according to the width of the pitch variation (up and down 10% in the above mounting example), and is not limited to the above specific numerical values.

  Further, in the above implementation example, the frequency sequence corresponding to each component of the feature vector is changed from the first component to the geometric sequence. However, if the ratio between the components is determined, they do not need to be equal. For example, the frequency corresponding to the second component may be 1.2 times the frequency corresponding to the first component, the frequency corresponding to the third component may be twice the frequency corresponding to the first component, and the like.

(Second Embodiment)
Next, the acoustic signal search device 2 according to the second embodiment of the present invention will be described. The acoustic signal search device 2 inputs an acoustic signal to be searched, that is, a reference signal, and detects a section including a signal similar to the reference signal from accumulated signals that are longer than this. In addition, the acoustic signal search device 2 is robust against the fluctuation, that is, absorbs the fluctuation, even when a large disturbance sound (additive noise) is superimposed on the acoustic signal and the frequency fluctuation occurs. Search.

FIG. 14 is a block diagram illustrating a configuration of the acoustic signal search device 2 according to the second embodiment.
The acoustic signal search device 2 in FIG. 9 includes a spectrogram dividing unit 20, a reference frequency detecting unit 21, a feature extracting unit 22, an encoding unit 24, an index generating unit 25, and a similarity calculating unit 23. Yes. In FIG. 14, the solid line shows the flow of processing for the spectrogram of the reference signal, and the broken line shows the flow of processing for the spectrogram of the accumulated signal.

  The spectrogram dividing unit 20 divides the spectrogram of the inputted reference signal and accumulated signal into spectrograms of small regions called small region spectrograms, and outputs them as a small region reference signal spectrogram and a small region accumulated signal spectrogram, respectively. The reference frequency detection unit 21 detects a characteristic portion of the power spectrum from the reference signal and each small region signal spectrogram of the accumulated signal, and sets the reference frequency fc at each time point. The feature extraction unit 22 calculates the small region spectrogram features of the reference signal and the accumulated signal based on the reference frequency fc at each time point detected by the reference frequency detection unit 21.

  The encoding unit 24 encodes the small region spectrogram features of the reference signal and the accumulated signal, and outputs them as a reference signal small region code and an accumulated signal small region code, respectively. The index generation unit generates index information indicating the location in the accumulated signal for the accumulated signal small region code. The similarity calculation unit 23 detects a code similar to the reference signal small region code by referring to a previously generated small region code similarity table, which will be described later, and performs an index search for the detected similar code. Further, when the similarity calculation unit 23 detects that there is a similar code, that is, a small region accumulated signal spectrogram similar to the small region reference signal spectrogram, in the accumulated signal, the accumulated signal including the small region accumulated signal spectrogram. The similarity between the middle section and the reference signal, that is, the section similarity is calculated.

  Next, the operation of the above-described acoustic signal search device 2 will be described. FIG. 15 is a flowchart illustrating an operation example of the acoustic signal search device 2. When the acoustic signal search device 2 inputs the reference signal and the stored acoustic signal by reading them (steps S901 and S911), the waveform signal is converted by the conversion unit in the same manner as the acoustic signal search device 1 according to the first embodiment. A power spectrum is obtained from the time spectrum, and a time-frequency spectrogram is extracted based on the obtained power spectrum (step S911, step S912).

Next, the spectrogram dividing unit 20 converts a spectrogram of a certain size (narrowband / short time), that is, a small-region spectrogram (small-region spectrogram) from the time frequency spectrogram of the reference signal and the accumulated signal in the frequency axis direction and time. Cut out at predetermined intervals in the axial direction, and output as a small region reference signal spectrogram and a small region accumulated signal spectrogram, respectively (steps S903 and S913). The small region spectrograms may be output so as to overlap each other or may not be overlapped. Here, the small region reference signal spectrogram in the reference signal in which the first time point is tn and the frequency band is ωm is represented as F _{tn, ωm} . Similarly, in the stored signal spectrogram, the first time point is t, and the small region stored signal spectrogram having the same size as F _{tn, ωm in} the frequency band _ωm is expressed as G _{t, ωm} .

In the following description, in the spectrogram of the reference signal, a set of all time points tn obtained by cutting out the small region spectrograms F _{tn, ωm} is TR (TR = {t1, t2,...}), And all frequency bands ωm The set is described as W (W = {ω1, ω2,...}).

  The reference frequency detection unit 21 detects the peak of the power spectrum in the frequency band ωm for each small region spectrogram of the reference signal and the accumulated signal output from the spectrogram dividing unit 20 (steps S904 and S914).

Note that the method of setting the reference frequency f _c ^ωm of the small region spectrogram is determined for each of the sampled frequency spectra in the small region spectrogram, similarly to the reference frequency detector 11 in the first embodiment described above. Or a frequency corresponding to the center of the power spectrum value or the frequency at which the power spectrum value becomes maximum in a certain time can be set as the reference frequency of the frequency spectrum at each point in time. Similarly to the first embodiment, a plurality of power spectrum peaks are detected at each sampling time point, and the Xth frequency is selected, or a point shifted by several Hz from the frequency at which the intensity is Yth is selected. To determine the reference frequency, or instead of paying attention to the peak of the power spectrum, a method that uses the point that becomes the valley between the peaks in the detected power spectrum as the reference frequency. You may make it use.

Next, the feature extraction unit 22 uses the frequency spectrum at each point in the small region spectrogram based on the reference frequency f _c ^ωm detected by the reference frequency detection unit 21 for each small region spectrogram of the reference signal and the accumulated signal. Features are calculated (steps S905 and S915). In the second embodiment, in order to cope with fluctuations in sound volume, the frequency spectrum feature is normalized by the average value at each extracted time point. A histogram (histogram feature) in which feature vectors of frequency spectrum features having normalized power spectrum values as elements are encoded by vector quantization, and the number of appearances of these codes in a small region spectrogram is counted for each code. (Step S906, Step S916).

  The encoding unit 24 encodes the histogram feature of each small region spectrogram of the reference signal and the accumulated signal calculated by the feature extraction unit 22 by vector quantization for each frequency band ωm (steps S907 and S917). Specifically, a feature vector whose element is the bin value of the histogram, that is, the number of appearances of each spectrum code in the small region spectrogram, is vector-quantized, and a code is assigned by a vector quantization codebook generated in advance. Use the method. Note that the encoding unit 24 uses the same vector codebook for the small region reference signal spectrogram and the small region accumulated signal spectrogram belonging to the same frequency band ωm when the small region spectrogram is encoded.

The vector quantization by the encoding unit 24 is performed by calculating a distance between the histogram feature and each representative vector of the vector quantization codebook and outputting a code corresponding to the representative feature having the minimum distance value. Hereinafter, the code obtained by encoding the histogram feature is referred to as a reference signal small region code and an accumulated signal small region code, and the reference signal small region code of the small region reference signal spectrogram F _{tn, ωm} is represented by c (F _{tn , Ωm} ), and the accumulated signal small region code of the small region accumulated signal spectrogram G _{t, ωm} is defined as c (G _{t, ωm} ). Each of c (F _{tn, ωm} ) and c (G _{t, ωm} ) in the band ωm is one of codes q (ωm, 1), q (ωm, 2),..., Q (ωm, z). Will be allocated.

  The encoding of these small region spectrograms is performed by frequency spectrum features (or a series of feature vectors in which a plurality of spectral features are consecutively included) in the small region reference signal spectrogram and small region accumulated signal spectrogram extracted by the feature extraction unit 22. Can be directly vector-quantized without generating a histogram to calculate a reference signal small region code and an accumulated signal small region code, respectively.

  The index generation unit 25 generates index information that lists at which point in the accumulated signal the code appears in each code of the small region accumulated signal spectrogram output from the encoding unit 24 (step S920). The index information is configured as shown in FIG. 16, and the list (arrangement at the present time; column in the horizontal direction) indicated by the code q (ωm, j) has a small size classified into the code q (ωm, j). Time points appearing on the accumulation signal of the area accumulation signal spectrogram are stored in time series.

  The similarity calculation unit 23 first detects a small region accumulated signal spectrogram similar to the small region reference signal spectrogram from the index information based on the similarity between the reference signal small region code and the code of the accumulated signal small region code. The similarity between all the small region codes in the band ωm is calculated in advance from the distance between the representative vectors of the codes, and the similarity for each small region code pair as shown in the small region code similarity table shown in FIG. Is defined on the table, that is, stored in an internal storage area. That is, the similarity υ (ωm, i, j) between the small area codes of the reference signal small area code q (ωm, i) and the accumulated signal small area code q (ωm, j) can be listed from this table. is there.

  The small area code similarity υ (ωm, i, j) is calculated based on the distance between the representative vectors corresponding to the small area codes q (ωm, i) and q (ωm, j). 23, when the distance value is small, it is determined that the similarity is high, and when the distance value is large, it is determined that the similarity is low. In this embodiment, υ (ωm, i, j) is defined as a real value from 0 to 1. That is, when the distance value is maximum in the band ωm, υ (ωm, i, j) = 0, and when the distance value is minimum, υ (ωm, i, j) = 1.

The similarity calculation unit 23 sequentially reads and selects reference signal subregion codes from the set of reference signal subregion codes (step S930), refers to the subregion code similarity table shown in FIG. 17 (step S931), and A code similar to the reference signal small region code, that is, a code having a small region code similarity υ (ωm, i, j) exceeding a small region search threshold s ^P th which is a predetermined threshold is detected (step S932). ). Here, for a certain reference signal small area code, at least one or more similar codes including codes having the same code value are detected.

Note that the small region search threshold value s ^P th can be set, for example, by repeating detection processing for a large number of acoustic signals in advance and statistically obtaining a value with no detection omission. Further, although the small region search threshold value s ^P th is a constant value in the entire band, a different value may be set for each band.

Then, the similarity calculation unit 23 generates, for a certain reference signal small region spectrogram, a small region spectrogram having a value of the above-described at least one code similar to the reference signal small region code as the code in the accumulated signal. Whether there exists or not is searched by the index information shown in FIG. 16 (step S933). In the accumulated signal, when the presence of the small region accumulated signal spectrogram G _{t + tn, ωm} similar to the small region reference signal spectrogram F _{tn, ωm} is confirmed _{, the} accumulated signal including the small region accumulated signal spectrogram G _{t + tn, ωm} It is determined that there is similarity to the reference signal in the section, and the similarity S ^P (F _{tn, ωm} , G _{t + tn,} between the appearance position t + tn in the accumulated signal of the similar small region accumulated signal spectrogram and the small region reference signal spectrum _ωm ) is output (step S934). Here, the similarity between the small region spectra, that is, the small region similarity is defined by the following equation (3).

  Subsequently, the similarity calculation unit 23 calculates the section start current time t in the similar accumulated signal from the appearance position t + tn in the accumulated signal of the detected similar accumulated signal small region spectrogram (step S935), The small area similarity between the reference signal spectrogram and the small area spectrogram in the stored signal similar to the reference signal spectrogram is integrated by the voting method at the corresponding section start time t (step S936).

  The similarity calculation unit 23 repeats the processing from step S930 to step S936 (step S937), sequentially performs index detection for all the small region spectrograms of the reference signal, and when similar small region accumulated signal spectrograms are detected. Then, the small area similarity is integrated with respect to the section start current time t in the accumulated signal including the small area accumulated signal spectrogram.

  When all the small region similarity integration processing is completed for the small region accumulated signal spectrogram similar to each small region reference signal spectrogram, the similarity calculating unit 23 uses the following equation (4) to calculate the integrated small region similarity as a reference signal. Is divided by the number of all small region reference signal spectrograms and normalized, and the interval similarity S (t) between the interval in the accumulated signal starting from time t and the reference signal is calculated (step S940).

Here, | TR | represents the number of elements at the time point TR (sampled time-series discrete time points), and | TW | represents the number of elements in the frequency band ωm. Then, the similarity calculation unit 23 uses, as a reference signal, a section that starts at time t on the accumulated signal in which the section similarity S (t) calculated as described above exceeds the search threshold Sth that is a predetermined value. It outputs as a similar section (step S941, step S942). As another form, instead of outputting a plurality of sections exceeding the search threshold, only a section exceeding the search threshold and having the largest section similarity may be output.

  Note that the search threshold Sth may be arbitrarily set experimentally, or separately, a standard deviation is calculated from the statistical distribution of the interval similarity S (t) calculated by the similarity calculation unit 23. It is also possible to set the value of −bσ as the search threshold value Sth for the maximum value of the obtained section similarity S (t). Here, an appropriate value of b is set through experiments.

(Example of implementation of the second embodiment)
Next, an implementation example of the above-described second embodiment will be described.
The acoustic signal is an acoustic signal (sampling frequency of 11.025 kHz, 8 bits) obtained by down-sampling, monauralizing, and changing the number of quantization bits of an acoustic signal (sampling frequency 44.1 kHz, 16-bit quantization, stereo) used in a music CD or the like (Quantization, monaural) is used. The acoustic signal is first subjected to 1024 FFT (Fast Fourier Transform) every 10 ms, and the power spectrum at that time is extracted every 10 ms. Then, the spectrogram is obtained assuming that the spectra are arranged in the time direction every 10 ms. The number of spectrogram divisions in the frequency axis direction is 4, which is used in the apparatus described in this embodiment, and in each band, ((1 / x ³ ) × fc, (1 / x ² ) × fc, ( 1 / x) × fc, x × fc, x 2 × fc, extracts power at each frequency x ³ × fc), extracted to calculate the average value of each frequency of the power that is, each frequency at the calculated average value A feature vector is obtained by normalizing the power of. Here, x is set to ^71/27 .

Then, fc is set to be floating around each band at 620 Hz, 1030 Hz, 1710 Hz, and 2840 Hz (the floating center frequency of each band is described here). Of the frequencies 10% above and below the center frequency, the frequency at which the power is maximum is assumed to be fc. Based on this fc, at each time point, ((1 / x ³ ) × fc, (1 / x ² ) × fc, (1 / x) × fc, x × fc, x ² × fc, x ³ × fc) Of the frequencies obtained from the FFT result, the power of the frequency closest to these frequencies is ((1 / x ³ ) × fc, (1 / x ² ) × fc, (1 / x) × fc, x × fc, x ² × fc, x ³ × fc), the average value of the power of each frequency is calculated, and the power of each frequency is normalized by the calculated average value Extracted as a feature vector.

  Further, for the frequency corresponding to each component of the feature vector, the power spectrum may be extracted again using a bandpass filter having the frequency as the center frequency.

  In this way, by extracting the feature vector, it is possible to perform a search even with an acoustic signal whose pitch varies up and down by 10%.

The spectrogram division number (4 in the above implementation example), the dimension of the feature vector (7 in the above implementation example), the value of x (7 ^{1/27 in the} above implementation example), and the setting range of fc (in the above implementation example) 10%) can be changed according to the signal to be searched and the range of pitch fluctuation (up and down 10% in the above mounting example), and is not limited to a specific numerical value.

  Further, in the above implementation example, the frequency sequence corresponding to each component of the feature vector is changed from the first component to the geometric sequence. However, if the ratio between the components is determined, they do not need to be equal. For example, the frequency corresponding to the second component may be 1.2 times the frequency corresponding to the first component, the frequency corresponding to the third component may be twice the frequency corresponding to the first component, and the like.

  With the configuration of the first and second embodiments described above, a characteristic part of the power spectrum is detected from each time frequency spectrogram of the reference signal and the accumulated signal, and the reference frequency at each time point is set floatingly, that is, the frequency Even if fluctuations due to the above occur, it is possible to set a reference frequency capable of extracting the characteristics of the entire spectrogram. Then, the frequency spectrum feature of each signal is extracted based on the reference frequency, and the acoustic signal can be detected corresponding to the fluctuation in the frequency up and down based on the extracted frequency spectrum feature.

  In addition, with the configuration of the second embodiment, the time-frequency spectrogram of each signal is divided into small regions in the time / frequency direction, the peaks in the small regions are detected, and feature extraction is performed. By configuring so that the similarity to the section is judged from the similarity of each small area, even if a larger disturbance sound is superimposed on the acoustic signal whose frequency has been changed, the signal detection processing is performed accurately and at high speed. It is possible to do.

  In addition, with the configuration of the first and second embodiments, by expressing the characteristics of each signal as a set of codes, it is possible to significantly reduce the amount of information required for matching, and speed up the calculation processing of similarity It becomes possible to do. Furthermore, by preparing the code of the accumulated signal as index information in advance, it becomes possible to easily detect the position of a similar code, and it is possible to detect a section in the accumulated signal containing a signal similar to the reference signal. Furthermore, it can be performed at high speed.

  In addition, according to the above-described embodiment, it is possible to search for an acoustic signal processed according to the atmosphere of a story, such as an inserted song of a drama in TV broadcasting or background music, and the monitoring of multimedia broadcasting can be performed at high speed. It becomes possible to perform with high precision. In addition, it is possible to search music information and CM information of music broadcast on TV and radio in real time on multiple channels and create a playlist. Playlists include, for example, publicity materials for broadcasters to respond to inquiries from viewers, CM confirmation data that allows an advertiser to investigate CM exposure status of competitors, record company marketing data, It can be widely used as basic data for collecting copyright fees.

  In addition, the configuration of the above embodiment can be applied to infringement of copyright infringement, identification of unknown acoustic information, and Internet copyright prevention applications, and is expected to expand into the market such as metadata analysis and distribution services. The

  Further, the acoustic signal feature extraction device described in the present invention is a configuration including the reference frequency detection unit 11 and the feature extraction unit 12 in the acoustic signal search devices 1, 1 a, and 1 b, and in the acoustic signal search device 2. This is an apparatus corresponding to the configuration including the reference frequency detection unit 21 and the feature extraction unit 22. The reference acoustic signal described in the present invention corresponds to the reference signal, and the search target acoustic signal corresponds to the accumulated signal.

  In the above embodiment, detection includes search / recognition, and feature extraction includes analysis of an acoustic signal. Further, “each time point” described in the above embodiment means each time point for each time interval of the time-frequency spectrogram, and indicates a certain time range from which the time-frequency spectrogram is calculated on the time axis. And

  Further, when actually constructing the above-described acoustic signal retrieval devices 1, 1a, 1b, and 2, the program for realizing the functions of the acoustic signal retrieval device is recorded on a computer-readable recording medium, and this recording is performed. The stored signal may be registered in the database and the input signal may be detected by executing a program recorded on the medium by a computer. The “computer system” here includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  In addition, the above-described program may be transmitted from a computer system that stores the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the technical idea of the present invention.

It is the schematic (the 1) of the spectrogram for demonstrating the principle of this invention. It is the schematic (the 2) of the spectrogram for demonstrating the principle of this invention. It is the schematic (the 3) of the spectrogram for demonstrating the principle of this invention. It is the schematic (the 4) of the spectrogram for demonstrating the principle of this invention. 1 is a schematic block diagram of an acoustic signal search device according to a first embodiment of the present invention. It is a flowchart (the 1) which shows operation | movement of the acoustic signal search device which concerns on the same embodiment. It is a conceptual diagram explaining the outline | summary of the reference frequency detection which concerns on the same embodiment. It is a conceptual diagram explaining the TAS method which concerns on the same embodiment. It is a flowchart (the 2) which shows operation | movement of the acoustic signal search device which concerns on the same embodiment. It is a flowchart (the 3) which shows operation | movement of the acoustic signal search device which concerns on the same embodiment. It is other embodiment (the 1) of the acoustic signal search device concerning the embodiment. It is other embodiment (the 2) of the acoustic signal search device concerning the embodiment. It is a flowchart which shows operation | movement of the acoustic signal search apparatus of other embodiment (the 2) of 1st Embodiment. It is a schematic block diagram of the acoustic signal search device concerning a 2nd embodiment of the present invention. It is a flowchart which shows operation | movement of the acoustic signal search apparatus which concerns on the same embodiment. It is a conceptual diagram for demonstrating the structure of the index information which concerns on the same embodiment. It is a conceptual diagram for demonstrating the structure of the similarity table between small area codes which concerns on the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Acoustic signal search device 11 Reference frequency detection part 12 Feature extraction part 13 Similarity calculation part

Claims (11)

An acoustic signal feature extraction method in an acoustic signal feature extraction device that extracts a feature amount of the acoustic signal based on a time-frequency spectrogram of an input acoustic signal,
For each time interval of the time frequency spectrogram, extract a predetermined reference feature of the time frequency spectrogram,
The extracted frequency of the predetermined reference feature is set as a reference frequency for each time interval,
Extracting a plurality of frequencies at predetermined intervals on the logarithmic axis with reference to the reference frequency, and extracting feature values of the acoustic signal for each time interval based on the extracted power spectrum value of each frequency. An acoustic signal feature extraction method.   The acoustic signal feature extraction method according to claim 1, wherein the predetermined reference feature portion is a portion where a power spectrum value becomes a maximum value at each time interval. The acoustic signal feature extraction according to claim 1, wherein the predetermined reference feature portion is a portion in which a power spectrum value becomes a maximum value among several time-frequency spectrograms continuous on a time axis. Method. The acoustic signal feature extraction method according to claim 1, wherein the predetermined reference feature portion is a portion serving as a center of gravity of power spectrum values of several time-frequency spectrograms continuous on a time axis. An acoustic signal search method in an acoustic signal search device that searches an input search target acoustic signal for a section including a sound similar to the reference acoustic signal based on a reference acoustic signal,
From the time frequency spectrogram of each of the reference acoustic signal and the search target acoustic signal, a predetermined reference feature for each time interval of the time frequency spectrogram is extracted,
The frequency of the predetermined reference feature extracted from the time frequency spectrogram of the reference sound signal is set as the reference frequency of the reference sound signal for each time interval ,
The frequency of the predetermined reference feature extracted from time-frequency spectrogram of the search target sound signal and the reference frequency of the search target acoustic signal of each of the time intervals,
A plurality of frequencies at predetermined intervals on the logarithmic axis are extracted based on the reference frequency of the reference acoustic signal, and the sound for each time interval of the reference acoustic signal is extracted based on the extracted power spectrum value of each frequency. Extracting a feature quantity of the signal, calculating a reference feature quantity based on the extracted feature quantity, with a time length of the entire signal of the reference acoustic signal as one section,
A plurality of frequencies at predetermined intervals on the logarithmic axis are extracted with reference to the reference frequency of the search target acoustic signal, and for each time interval of the search target acoustic signal based on the extracted power spectrum value of each frequency Features of the
A section feature amount for each section is calculated based on the extracted feature amount of the search target acoustic signal,
A similarity is calculated based on the calculated section feature and the reference feature,
Based on the calculated similarity, the search target sound signal section similar to the sound of the reference sound signal is searched, and the search target sound signal section similar to the sound of the reference sound signal detected by the search is output. An acoustic signal search method characterized by: An acoustic signal feature extraction device that extracts a feature amount of the acoustic signal based on a time-frequency spectrogram of the input acoustic signal,
For each time interval of the time frequency spectrogram, a predetermined reference feature portion of the time frequency spectrogram is extracted, and a reference frequency detection unit that uses the extracted frequency of the predetermined reference feature portion as a reference frequency for each time interval;
A feature amount that extracts a plurality of frequencies at predetermined intervals on the logarithmic axis with reference to the reference frequency, and extracts a feature amount of the acoustic signal for each time interval based on the extracted power spectrum value of each frequency. Extraction means;
An acoustic signal feature extraction apparatus comprising: Based on a reference acoustic signal, an acoustic signal search device that searches an input search target acoustic signal for a section including a sound similar to the reference acoustic signal,
Extracting a predetermined standard feature for each time interval of the time frequency spectrogram from the time frequency spectrogram of each of the reference acoustic signal and the search target acoustic signal, and extracting the predetermined standard extracted from the time frequency spectrogram of the reference acoustic signal The frequency of the feature portion is set as a reference frequency of the reference acoustic signal for each time interval, and the frequency of the predetermined reference feature portion extracted from the time frequency spectrogram of the search target acoustic signal is set to the search target acoustic signal for each time interval. a reference frequency detecting means for the issue of the reference frequency,
A plurality of frequencies at predetermined intervals on the logarithmic axis are extracted based on the reference frequency of the reference acoustic signal, and the characteristics of the reference acoustic signal for each time interval based on the extracted power spectrum value of each frequency Extracting a quantity, calculating a reference feature quantity having a time length of the entire signal of the reference acoustic signal as one section based on the extracted feature quantity, and calculating a plurality of frequencies at predetermined intervals on a logarithmic axis, A feature extraction unit that extracts a reference frequency of a search target acoustic signal as a reference , and extracts a feature amount for each time interval of the search target acoustic signal based on a power spectrum value of each extracted frequency;
A section feature amount for each section is calculated based on the feature amount of the search target acoustic signal extracted by the feature extraction unit, and a similarity is calculated based on the calculated section feature amount and the reference feature amount. The search target acoustic signal section similar to the sound of the reference acoustic signal is searched based on the calculated similarity, and the search target acoustic signal section similar to the sound of the reference acoustic signal detected by the search is searched. A similarity calculation means to output;
An acoustic signal retrieval device comprising: Based on the time-frequency spectrogram of the input acoustic signal, the computer of the acoustic signal feature extraction device that extracts the feature quantity of the acoustic signal,
Extracting predetermined reference features of the time-frequency spectrogram for each time interval of the time-frequency spectrogram;
Setting the extracted frequency of the predetermined reference feature as a reference frequency for each time interval;
Extracting a plurality of frequencies at predetermined intervals on the logarithmic axis with reference to the reference frequency, and extracting a feature quantity of the acoustic signal for each time interval based on the extracted power spectrum value of each frequency; ,
Acoustic signal feature extraction program for executing Based on the reference acoustic signal, the computer of the acoustic signal retrieval device that retrieves the section including the sound similar to the reference acoustic signal from the input retrieval target acoustic signal,
Inputting a time frequency spectrogram of a reference acoustic signal and a search target acoustic signal;
Extracting a predetermined reference feature for each time interval of the time-frequency spectrogram from the time-frequency spectrogram of each of the reference sound signal and the search target sound signal;
The predetermined standard extracted from the time-frequency spectrogram of the search target acoustic signal, with the frequency of the predetermined reference feature extracted from the time-frequency spectrogram of the reference acoustic signal as the reference frequency of the reference acoustic signal for each time interval the method comprising the frequency of the feature and the reference frequency of the search target acoustic signal of each of the time intervals,
A plurality of frequencies at predetermined intervals on the logarithmic axis are extracted based on the reference frequency of the reference acoustic signal, and the characteristics of the reference acoustic signal for each time interval based on the extracted power spectrum value of each frequency Extracting a quantity, and calculating a reference feature quantity having a time length of the entire signal of the reference acoustic signal as one section based on the extracted feature quantity;
A plurality of frequencies at predetermined intervals on the logarithmic axis are extracted with reference to the reference frequency of the search target acoustic signal, and for each time interval of the search target acoustic signal based on the extracted power spectrum value of each frequency Extracting a feature quantity of
Calculating a section feature value for each section based on the extracted feature value of the extracted search target acoustic signal;
The similarity calculating means calculating a similarity based on the calculated section feature and the reference feature;
Based on the calculated similarity, the search target sound signal section similar to the sound of the reference sound signal is searched, and the search target sound signal section similar to the sound of the reference sound signal detected by the search is output. And steps to
Acoustic signal search program for executing   A computer-readable recording medium on which the acoustic signal feature extraction program according to claim 8 is recorded.   The computer-readable recording medium which recorded the acoustic signal search program of Claim 9.

Images