JP2015114361A - Acoustic signal analysis device and acoustic signal analysis program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an acoustic signal analysis device and an acoustic signal analysis program in which the accuracy of estimating a beat point, code progress, and the position of a measure line.SOLUTION: An acoustic signal of a music is taken in, and a first beat point candidate series OR1 and a second beat point candidate series OR2 out of beat by half with the first beat point candidate series OR1 are detected. Next, a first beat synchronous type code feature amount series CR1 and a second beat synchronous type code feature amount series CR2 representing the feature of code between two adjacent beat point candidates of the first beat point candidate series and second beat point candidate series are calculated. From among probability models representing the code progress of the music, probability models in which the first beat synchronous type code feature amount series CR1 and the second beat synchronous type code feature amount series CR2 satisfy a prescribed standard are selected one for each, and the beat point, code progress, and the position of a measure line of the music are estimated on the basis of one of the two selected probability models that has greater likelihood.

Description

  The present invention relates to an acoustic signal analysis apparatus and an acoustic signal analysis program for analyzing a sound signal representing a music and detecting a beat point (beat timing) in the music and a chord (chord) generated in each section of the music.

  2. Description of the Related Art Conventionally, for example, as shown in Non-Patent Document 1, an acoustic signal analyzer that detects beat points in music and chords (chords) generated in each section of the music is known. In this acoustic signal analyzer, first, an acoustic signal is analyzed to determine a beat point of a music piece. Then, under the assumption that a chord change occurs at the detected beat point and a chord change occurs at the beginning of the bar, the chord and bar line positions of each section of the music are detected.

M.M. Goto et al. “SONGLE: A WEB SERVICE FOR ACTIVE MUSIC LISTENING IMPROVED BY USER CONTRIBUTIONS”, ISMIR, 2011, p. 311-316

  In Non-Patent Document 1, there is a possibility that the back beat is selected as a beat point (front beat) by mistake, or a beat point at which the tempo of the music is a tempo that is double the true tempo is selected. Although it is described that a possible beat point is selected in consideration of the possibility, the selection means is not specifically disclosed. In addition, when the back beat is selected as the beat (front beat), or when the beat is selected so that the tempo of the music is double the true tempo, the chord detection accuracy and bar line position The accuracy of detection decreases.

  The present invention has been made to address the above problems, and an object thereof is to provide an acoustic signal analysis apparatus and an acoustic signal analysis program that improve the estimation accuracy of beat points, chord progressions, and bar line positions. It is in. In addition, in the description of each constituent element of the present invention below, in order to facilitate understanding of the present invention, reference numerals of corresponding portions of the embodiment are described in parentheses, but each constituent element of the present invention is The present invention should not be construed as being limited to the configurations of the corresponding portions indicated by the reference numerals of the embodiments.

  In order to achieve the above object, the present invention is characterized in that an acoustic signal acquisition means (S11) that captures an acoustic signal representing a performance sound of a musical piece to be analyzed, and a plurality of musical pieces based on the captured acoustic signal. A first beat point candidate series (OR1) consisting of a plurality of beat point candidates, and a second beat consisting of a plurality of beat point candidates each shifted by half a beat with respect to the plurality of beat point candidates constituting the first beat point candidate series. A beat point candidate series detecting means (S15) for detecting a beat point candidate series (OR2) and two adjacent beat points among a plurality of beat point candidates constituting the first beat point series on the basis of the acquired acoustic signal. A first beat-synchronized chord feature amount sequence (CR1) composed of beat-synchronized chord feature amounts each representing a chord feature between two beat point candidates is calculated, and a plurality of the second beat point sequences are constructed. Beat point candidates Beat-synchronized chord feature amount sequence for calculating a second beat-synchronized chord feature amount sequence (CR2) composed of beat-synchronized chord feature amounts each representing a chord feature between two adjacent beat point candidates. A calculation means (S16) and a probability model representing chord progression of the music piece, and the chords between beat points according to the combination of the number of beats within one measure, the key of the music piece, and the beat position of the first beat point. Among the probability models in which transition probabilities are set, a probability model in which the first beat-synchronized chord feature amount sequence satisfies a predetermined criterion and a probability model in which the second beat-synchronized chord feature amount sequence satisfies a predetermined criterion, respectively Select one by one and estimate the beat point, chord progression, and bar line position of the song based on the probability model with the highest likelihood of the two selected probability models And estimating means (S17~S21), in that the sound signal analysis device provided with a.

In this case, the beat-synchronized chord feature amount calculating means calculates a chord feature amount (XC) representing a chord feature amount for each of a plurality of sections (t _i ) located between the two adjacent beat point candidates. Code feature amount calculating means for calculating the beat-synchronized chord feature amount by smoothing chord feature amounts of a plurality of sections located between the two adjacent beat point candidates Means.

  In this case, the beat point candidate series detecting means calculates beat / tempo feature quantity calculating means for calculating a first feature quantity representing a feature relating to the presence of a beat in each section of the music and a second feature quantity representing a feature relating to the tempo. And among the plurality of probability models described as a series of states classified by combinations of physical quantities related to the presence of beats and physical quantities related to tempo in each section of the music, the first feature quantity and the second feature quantity are Beat point and tempo estimation means for simultaneously estimating beat points and tempo transitions in the music piece by selecting a probability model in which a series of observation likelihoods representing the probability observed simultaneously in each section of the music piece satisfies a predetermined criterion It is good to provide.

  In general, chord changes are more likely to occur at the beat. For this reason, if the back beat is selected as the front beat by mistake, there is a high possibility that the chord changes between beat points. Therefore, in this case, the beat synchronization type chord feature amount cannot accurately represent the chord feature. In other words, when the back beat is selected as the front beat, the likelihood of the beat-synchronized code feature quantity sequence is lower than when the true beat point (ie, the front beat) is selected. Therefore, the acoustic signal analysis apparatus according to the present invention includes a first beat point candidate series and a second beat point composed of beat point candidates shifted by half a beat with respect to each beat point candidate constituting the first beat point candidate series. Candidate sequences are detected, and the likelihoods of the beat-synchronized code feature amount sequences relating to the first beat point candidate sequence and the second beat point candidate sequence are calculated. Then, both likelihoods are compared, and a beat point candidate series having a high likelihood is selected. Thereby, it can suppress selecting a back beat accidentally as a front beat. Also, the beat position and key combination that maximizes the likelihood of the beat-synchronized chord feature quantity sequence is selected, and the beat point, chord progression, and bar line position in the music are estimated simultaneously (integrally). The Therefore, according to the acoustic signal analysis device of the present invention, it is possible to improve the estimation accuracy of beat points, chord progressions, and bar line positions as compared to the conventional art.

  The present invention can also be implemented as a computer program applied to a computer provided in the acoustic signal analyzer.

It is a block diagram showing the structure of the acoustic signal analyzer which concerns on one Embodiment of this invention. It is a flowchart showing a beat point and chord estimation process. It is a graph showing the waveform of the acoustic signal to be analyzed. It is a conceptual diagram of the probability model for calculating a beat point candidate. It is a block diagram of a comb filter. It is a graph which shows the calculation result of a BPM feature-value. It is a table | surface which shows the structure of a template. It is a conceptual diagram of a code feature amount. It is a conceptual diagram of a beat synchronous chord feature amount.

  An acoustic signal analyzer 10 according to an embodiment of the present invention will be described. As will be described below, the acoustic signal analysis apparatus 10 takes in an acoustic signal representing music and detects transitions in beat points and tempos in the music. As shown in FIG. 1, the acoustic signal analyzer 10 includes an input operator 11, a computer unit 12, a display 13, a storage device 14, an external interface circuit 15, and a sound system 16, which are connected via a bus BS. Connected.

  The input operator 11 includes a switch corresponding to an on / off operation (for example, a numeric keypad for inputting a numerical value), a volume or rotary encoder corresponding to a rotation operation, a volume or linear encoder corresponding to a slide operation, a mouse, a touch panel, etc. Composed. These operators are operated by the performer's hand to select the music to be analyzed, start or stop the analysis of the sound signal, play or stop the music (output or stop from the sound system 16 described later), sound signal It is used to set various parameters related to the analysis. When the input operator 11 is operated, operation information indicating the operation content is supplied to the computer unit 12 described later via the bus BS.

  The computer unit 12 includes a CPU 12a, a ROM 12b, and a RAM 12c connected to the bus BS. The CPU 12a reads an acoustic signal analysis program and its subroutine, which will be described later in detail, from the ROM 12b and executes them. In addition to the acoustic signal analysis program and its subroutine, the ROM 12b stores various data such as initial setting parameters, graphic data for generating display data representing an image displayed on the display 13, and character data. . The RAM 12c temporarily stores data necessary for executing the acoustic signal analysis program.

  The display 13 is configured by a liquid crystal display (LCD). The computer unit 12 generates display data representing contents to be displayed using graphic data, character data, and the like, and supplies the display data to the display unit 13. The display device 13 displays an image based on the display data supplied from the computer unit 12. For example, when selecting a song to be analyzed, a title list of songs is displayed. For example, at the end of the analysis, a graph representing beat points and bar lines and a chord name series representing chord progression are displayed.

  The storage device 14 includes a large-capacity nonvolatile recording medium such as an HDD, FDD, CD-ROM, MO, and DVD, and a drive unit corresponding to each recording medium. The storage device 14 stores a plurality of pieces of music data representing a plurality of pieces of music. The music data is composed of a plurality of sample values obtained by sampling the music at a predetermined sampling period (for example, 1/444100 seconds), and each sample value is sequentially recorded at successive addresses in the storage device 14. Title information representing the title of the song, data size information representing the capacity of the song data, and the like are also included in the song data. The music data may be stored in advance in the storage device 14, or may be taken in from an external device via the external interface circuit 15 described later. The music data stored in the storage device 14 is read by the CPU 12a, and the transition of beat points and tempo in the music is analyzed.

  The external interface circuit 15 includes a connection terminal that enables the acoustic signal analyzer 10 to be connected to an external device such as an electronic music device or a personal computer. The acoustic signal analyzer 10 can be connected to a communication network such as a LAN (Local Area Network) or the Internet via the external interface circuit 15.

  The sound system 16 includes a D / A converter that converts music data into an analog sound signal, an amplifier that amplifies the converted analog sound signal, and a pair of left and right speakers that convert the amplified analog sound signal into an acoustic signal and output it. It has. When the user uses the input operator 11 to instruct the reproduction of the music to be analyzed, the CPU 12a supplies the music data to be analyzed to the sound system 16. Thereby, the user can audition the music to be analyzed.

  Next, the operation of the acoustic signal analyzer 10 will be specifically described. When the user turns on a power switch (not shown) of the acoustic signal analyzer 10, the CPU 12a reads the beat point / code estimation program shown in FIG. 2 from the ROM 12b and executes it. In FIG. 2, the “judgment” step is indicated by a hexagon.

  The CPU 12a starts beat point / code estimation processing at step S10, reads title information included in each of a plurality of music data stored in the storage device 14 at step S11, and lists the titles of the music in a list format. Is displayed on the display 13. The user uses the input operator 11 to select music data to be analyzed from the music displayed on the display 13. In addition, when selecting the music data of analysis object in step S11, you may comprise so that the content of music data can be confirmed by reproducing | regenerating part or all of the music which the music data to select selects.

  Next, CPU12a performs the initial setting for an acoustic signal analysis in step S12. Specifically, a storage area corresponding to the data size information of the selected music data is secured in the RAM 12c, and the selected music data is read into the secured storage area. In addition, an area for temporarily storing later-described onset feature amounts XO, BPM feature amounts XB, and the like is secured in the RAM 12c. In addition, the user inputs the time signature (or the number of beats included in one measure) of the selected music using the input operator 11. That is, in this embodiment, it is assumed that the time signature (or the number of beats included in one measure) of the selected music is known.

In step S13, the CPU 12a divides the selected music piece at predetermined time intervals and divides it into a plurality of frames t _i {i = 0, 1,..., I} as shown in FIG. To do. The length of each frame is common. In order to simplify the explanation, in this embodiment, the length of each frame is set to 125 ms. As described above, since the sampling period of each piece of music is 1/444100 seconds, each frame is composed of about 5000 sample values.

Next, the CPU 12a calculates a beat point candidate series OR1 and a beat point candidate series OR2 composed of a plurality of beat point candidates. Here, the outline of the calculation procedure of the beat point candidate series OR1 and the beat point candidate series OR2 will be described. First, the beat point candidate series OR1 is calculated as follows. An onset feature value XO representing a feature related to the presence of a beat and a BPM (beats per minute) feature value XB representing a feature related to a tempo are calculated for each frame t _i . It is described as a series of states q _{b, n} classified according to the combination of the value of the beat period b in each frame t _i (value proportional to the reciprocal of the tempo) and the value of the number of frames n up to the next beat. Among the obtained probability models (hidden Markov models), a probability model in which the series of observation likelihoods representing the probability that the onset feature quantity XO and the BPM feature quantity XB as observation values are observed simultaneously is most likely (FIG. 4). reference). Thereby, the beat point candidate series OR1 in the music to be analyzed is detected. Then, using the detected beat point candidate series OR1, a beat point candidate series OR2 composed of a plurality of beat point candidates shifted by half a beat from the plurality of beat points constituting the beat point candidate series OR1 is calculated. (See FIG. 9). The beat period b is represented by the number of frames. Therefore, the value of the beat period b is an integer satisfying “1 ≦ b ≦ b _max ”, and in the state where the value of the beat period b is “β”, the value of the number of frames n is “0 ≦ n <β”. It is an integer that satisfies.

  Next, the calculation procedure of the beat point candidate series OR1 and the beat point candidate series OR2 will be specifically described. First, in step S14, the CPU 12a calculates an onset feature value XO and a BPM feature value XB for each frame.

The onset feature value XO (t _i ) of the frame t _i is calculated as follows. First, the CPU 12a performs a short-time Fourier transform for each frame to calculate the signal intensity of each frequency bin. Next, the CPU 12a calculates the signal intensity M (fb _x , t _i ) of each frequency band fb _x (for example, x = 1, 2,..., 20) using the mel filter bank. Next, the CPU 12a calculates an increase amount R (fb _x , t _i ) of the signal strength in each frequency band between frames. As shown in the following equation (1), the sum of the increase amounts of the signal strength of each frequency band between frames is the onset feature amount XO (t _i ).

The BPM feature value XB (t _i ) of the frame t _i is calculated as follows. First, the CPU 12a inputs onset feature values XO (t ₀ ), XO (t ₁ ),... In this order to the filter bank FBB (see FIG. 5). Filter bank FBB is composed of a plurality of comb filters CF _b respectively provided in accordance with the value of the beat period b. The comb filter CF _b outputs one data every time one data is input. The comb filter CF _b has a FIFO (= First In First Out) memory for storing past output data by the number corresponding to the value of the beat period b, and is stored in the storage means with the input data. The oldest data among the existing data is added at a predetermined ratio (for example, 1: 1 (that is, α = 0.5)) and output. The sequence XD _b (t) of the data XD _b obtained by inputting the sequence XO (t) {= XO (t ₀ ), XO (t ₁ )...} Of the onset feature quantity XO to the filter bank FBB. By reversing {= XD _b (t ₀ ), XD _b (t ₁ )... In time series and inputting them again into the filter bank FBB, the BPM feature quantity series XB _b (t ) {= XB _b (t ₀ ), XB _b (t ₁ ). The BPM feature value XB (t _i ) of the frame t _i is represented as a set of BPM feature values XB _{b = 1, 2...} (T _i ) calculated for each beat period b (see FIG. 6).

  Next, in step S15, the CPU 12a estimates the maximum likelihood state sequence using the Viterbi algorithm. Thereby, the beat point candidate series OR1 is estimated. When the beat point candidate series OR1 is estimated, the series of values of the beat period b (that is, tempo transition) is also estimated simultaneously (integrally).

Specifically, CPU 12a first calculates onset feature quantity XO _{(t i)} and BPM feature value XB observation likelihood LO _{(t i)} of _{(t i)} and the observation likelihood LB a _{(t i),} respectively . Here, the onset feature amount XO (t _i ) follows a normal distribution set according to the value of the number of frames n up to the next beat point. In other words, the observation likelihood LO (t _i ) of the onset feature quantity XO substitutes the onset feature quantity XO as a normal distribution random variable set according to the value of the number of frames n up to the next beat point. Is calculated by For example, when the value of the number of frames n is “0”, a normal distribution having an average value of “3” and a variance of “1” is used. Further, when the value of the beat period b is “β” and the value of the number of frames n is “β / 2”, the normal value is “1” and the variance is “1”. A distribution is used. Further, when the value of the number of frames n is different from both “0” and “β / 2”, a normal distribution having an average value of “0” and a variance of “1” is used.

Further, the observation likelihood LB of the BPM feature quantity XB (t _i ) corresponds to the adaptability of the BPM feature quantity XB to the template TMP _b provided for each beat period b. That is, as shown in the following equation (2), the inner product of the template TMP _b and the BPM feature quantity XB (t _i ) is the observation likelihood LB (t _i ). Note that “ν _b ” in this arithmetic expression is a coefficient that determines the weight of the BPM feature quantity XB (t _i ) with respect to the onset feature quantity XO (t _i ). That is, the larger the value of “ν _b ” is, the greater the importance is placed on the BPM feature value XB (t _i ). Further, Z (ν _b ) in this arithmetic expression is a normalization coefficient that depends on “ν _b ”. In other words, the template TMP _b, the coefficient [delta] _b, which are respectively multiplied to the BPM feature value XB _{(t i)} constituting the BPM feature value _XB b _{(t i),} from the series of _{γ {= 1,2 ···}} (See FIG. 7). Coefficient [delta] _b constituting the template TMP _b, of _gamma, as a factor equal to an integer multiple of the index gamma is the beat period b equal to the coefficient and the beat period b is maximum, template TMP _b is set.

Next, the CPU 12a uses a logarithmic observation likelihood LOB (t _i ) (see Equation (3) below) that is a logarithm of the product of the observation likelihood LO (t _i ) and the observation likelihood LB (t _i ). Then, the state series having the maximum likelihood is calculated. In calculating the maximum likelihood state sequence, a Viterbi algorithm is used.

  In the present embodiment (from the state where the value of the beat period b is “βs” and the value of the number of frames n is “ηs”, the value of the beat period b is “βe” and the frame The value of the logarithmic transition probability T to the state where the value of the number n is “ηe” is set as follows (see FIG. 4): “ηe = 0”, “βe = βs”, and “ηe” = Βe−1 ”, the value of the logarithmic transition probability T is“ −0.2. ”When“ ηs = 0 ”,“ βe = βs + 1 ”, and“ ηe = βe−1 ”, The value of the logarithmic transition probability T is “−0.6”, and when “ηs = 0”, “βe = βs−1”, and “ηe = βe−1”, the value of the logarithmic transition probability T. Is “−0.6.” When “ηs> 0”, “βe = βs”, and “ηe = ηs−1”, the logarithmic transition probability T is “0”. Logarithm other than the above The value of the transition probability T is “−∞.” That is, when the transition from the state where the value of the number of frames n is “0” (ηs = 0) to the next state, the value of the beat period b is “1”. In this case, the value of the frame number n is set to a value smaller by “1” than the value of the beat period b after the transition. Also, the value of the frame number n is not “0” ( When transitioning from (ηs ≠ 0) to the next state, the value of the beat period b is not changed, and the value of the number of frames n is decreased by “1”.

Of the states constituting the maximum likelihood state sequence calculated as described above, a sequence in which the number of frames n is “0” is the beat candidate sequence OR1. Of the states constituting the maximum likelihood state sequence, a sequence in which the values of the beat period b and the number of frames n satisfy the following formula (4) is the beat candidate sequence OR2.

  Thereby, from a plurality of beat point candidates shifted from each other by a half beat (for example, the length of an eighth note in the case of a four-beat music) as seen from a plurality of beat point candidates constituting the beat point candidate series OR1. The beat point candidate series OR2 is obtained.

  Next, in step S16, the CPU 12a, in step S16, beat-synchronized chord feature amount sequence CR1 that is a sequence of beat-synchronized chord feature amount XBC1 representing chord features in each beat point candidate constituting beat point candidate sequence OR1; A beat-synchronized chord feature amount series CR2 that is a series of beat-synchronized chord feature amounts XBC2 representing the chord features in each beat point candidate constituting the beat point candidate sequence OR2 is calculated.

The beat-synchronized chord feature amount series CR1 and the beat-synchronized chord feature amount series CR2 are calculated as follows. First, the power of each frequency bin of each frame t _i, is mapped to the nearest pitch frequency to the frequency (e.g., the fundamental frequency of each pitch in equal temperament). Of the power mapped to each pitch as described above, the power belonging to the low frequency range (for example, “B1” or lower) is added for each pitch class (C, C #, D,..., B #) ( (Or accumulating). The 12-dimensional feature amount composed of the power of each pitch class calculated in this way is called a base feature amount HPCP ^(B) (see FIG. 8). Further, among the power mapped to each pitch, power belonging to a high pitch range (for example, “C2” or higher) is added (or integrated) for each pitch class (C, C #, D,..., B #). To do. The 12-dimensional feature quantity composed of the power of each pitch class calculated in this way is called a treble feature quantity HPCP ^(T) .

Further, the L2 norm of the low frequency range power is referred to as base power ρ ^(B), and the L2 norm of the high frequency range power is referred to as treble power ρ ^(T) .

A 26-dimensional feature amount consisting of a base feature amount HPCP ^(B) , a treble feature amount HPCP ^(T) , a base power ρ ^(B), and a treble power ρ ^{(T) for} each frame t _i is converted into a code feature amount XC (t _i ). Call it.

The beat-synchronized code feature value XBC1 (m) smoothes the code feature value XC (t _i ) of the frame between the “m” -th beat point candidate and the “m + 1” -th beat point candidate in the beat point candidate series OR1. This is a 26-dimensional feature amount obtained by converting to The beat-synchronized chord feature amount XBC2 (m) is the chord feature amount XC (t _i ) of the frame between the “m” -th beat point candidate and the “m + 1” -th beat point candidate in the beat point candidate series OR2. Is a 26-dimensional feature amount obtained by smoothing. Note that the smoothing means, for example, calculating the average of the base feature amount HPCP ^(B) of the frames between the beat point candidates, and calculating the treble feature amount HPCP ^(T) of the frame between the beat point candidates. It means calculating the median of power ρ ^(B) and treble power ρ ^(T) , respectively. The beat-synchronized chord feature amount series CR1 is calculated by calculating the beat-synchronized chord feature amount XBC1 for all the beat point candidates constituting the beat point candidate sequence OR1. Further, the beat-synchronized chord feature amount series CR2 is calculated by calculating the beat-synchronized chord feature amount XBC2 for all the beat point candidates constituting the beat point candidate sequence OR2.

  Next, in step S17, the CPU 12a calculates the likelihood LK1 of the maximum likelihood beat-synchronized code feature amount sequence CR1 and the likelihood LK2 of the maximum likelihood beat-synchronized code feature amount sequence CR2. The procedure for calculating the likelihood LK1 of the maximum likelihood beat-synchronized code feature amount sequence CR1 and the likelihood LK2 of the maximum likelihood beat-synchronized code feature amount sequence CR2 is common. Therefore, in the following description, the beat synchronization type code feature value sequence CR1 and the beat synchronization type code feature value sequence CR2 are simply referred to as a beat synchronization type code feature value sequence CR. Further, the beat synchronization type chord feature amount XBC1 and the beat synchronization type chord feature amount XBC2 are simply referred to as a beat synchronization type chord feature amount XBC.

Here, it is assumed that the base feature amount HPCP ^(B) and the treble feature amount HPCP ^(T) follow a vMF (= von Miss Fisher) distribution. Further, it is assumed that the base power ρ ^(B) and the treble power ρ ^(T) follow a gamma distribution. General music has strong power in a section where chords are generated. On the other hand, the power is weak in the section where no chord is generated. Further, the distribution of the power of each pitch class constituting the base feature amount HPCP ^(B) and the treble feature amount HPCP ^(T) in a section where no chord is generated, and each pitch class in the section where a chord is generated. The power distribution is different. Therefore, the base feature amount HPCP ^{(B), the} treble feature amount HPCP ^(T) , the base power ρ ^(B), and the treble power ρ ^(T) are related to the state where the chord is generated and the state where the chord is not generated. Learn at the same time. Also, instead of using a single vMF distribution and gamma distribution for one chord, a plurality of models (mixed models) are set, and the beat-synchronized chord feature amount is calculated as a weighted linear sum. Define the observation likelihood. The observation likelihood of the beat-synchronized code feature value XBC includes the base feature value HPCP ^(B) , the treble feature value HPCP ^(T) , the base power ρ ^(B), and the treble power as components of the beat-synchronized code feature value XBC. Using ρ ^(T) and the mean μ _{k of} the vMF distribution, the variance κ _{k of the} vMF distribution, the scale parameter u of the gamma distribution, the shape parameter v of the gamma distribution, and the weight w _{k of the} linear sum, It is expressed as equation (5). Note that “k” is an index for identifying the distribution constituting the mixed model. In addition, “B” is described in parentheses at the upper right of the variable related to the low frequency range. In addition, “T” is written in parentheses at the upper right of the variable relating to the middle and high pitch range. “Θ _j ” represents a parameter related to the chord j. For example, “Θ _j ” represents the shape (power distribution of each pitch class ⁾ of the base feature amount HPCP ^(B) and the treble feature amount HPCP ^(T) .

In general, the transition probability from chord to chord depends on the key of the music. For example, the transition from the chord “C” to the chord “F” is likely to occur in C major music. The transition probability from chord to chord depends on the beat position s of the beat point (the number of beats counted from the immediately preceding bar line). For example, when the chord of the fourth beat (that is, s = 4) is “G7” in the music of four quarters, there is a high possibility that the first chord of the next measure is “C” ( Dominant motion). Therefore, the acoustic signal analysis apparatus 10 includes a plurality of databases that store chord transition probabilities. Each database corresponds to a time signature. That is, the acoustic signal analysis apparatus 10 analyzes, for example, a database used when analyzing music of 3/4 time, a database used when analyzing music of 4/4 time, and music of 6/8 time. A database to be used when In each database, a transition probability from a chord to a chord is stored in association with the key and the beat position s. The transition probability from chord to chord is determined by learning the transition of chords in various musical pieces. These databases are stored in the ROM 12b. Here, the probability that the chord in the “m−1” th beat point candidate is “j ′” and the chord in the “m” th beat point candidate is “j” is expressed by the following equation (6). It describes as follows. A database corresponding to each beat number in one measure may be provided.

Then, under the condition that the time position s and the key of the first beat are known, the likelihood LK (s, key) of the beat-synchronized code feature amount sequence CR is expressed by the following equation (7). expressed.

Note that “Z _j (m)” in Equation (7) is a binary variable as described below. That is, “Z _j (m)” is “1” when the chord of the “m” -th beat point candidate is “j”, and “0” in other cases. In addition, “Z _{j ′} (m−1) Z _j (m)” indicates that the chord in the “m−1” -th beat point candidate is “j ′” and the “m” -th beat point candidate. It is “1” only when the chord is “j”, and “0” otherwise.

Here, the likelihood LK (s, key) for the beat point candidate series OR1 is expressed as the likelihood LK ⁽¹⁾ (s, key). The likelihood LK1 of the maximum likelihood beat-synchronized code feature quantity sequence CR1 is calculated based on Expression (8).

Further, the likelihood LK (s, key) for the beat point candidate series OR2 is expressed as likelihood LK ⁽²⁾ (s, key). The likelihood LK2 of the maximum likelihood beat-synchronized code feature quantity sequence CR2 is calculated based on Expression (9).

  The CPU 12a calculates the likelihood LK1 and the likelihood LK2 using the Viterbi algorithm. Further, when calculating the likelihood LK1 and the likelihood LK2, the CPU 12a determines a chord transition probability by referring to a database corresponding to the time signature (or the number of beats in one measure) input by the user in step S12. .

  Next, the CPU 12a compares the likelihood LK1 and the likelihood LK2 in step S18. When the likelihood LK1 is larger than the likelihood LK2, the CPU 12a determines “Yes”, and in step S19, the estimated beat point candidate series OR1, the maximum likelihood beat-synchronized code feature quantity series CR1, and The beat position s is output. Specifically, a graph representing beat points and bar lines is displayed on the display 13 based on the beat point candidate series OR1 and the beat position s. Further, the chord progression Z1 (chord name sequence) is calculated based on the maximum likelihood beat-synchronized chord feature amount sequence CR1 and displayed on the display 13. The chord progression Z1 is a chord name sequence corresponding to each beat-synchronized chord feature amount XBC1 (m) constituting the most likely beat-synchronous chord feature amount sequence CR1. And CPU12a complete | finishes a beat point and chord estimation process in step S20.

  On the other hand, when the likelihood LK1 is less than or equal to the likelihood LK2 in step S18, the CPU 12a determines “No”, and in step S21, the estimated beat point candidate series OR2, the maximum likelihood beat synchronization type. The chord feature amount series CR2 and the time signature position s are output. Specifically, a graph representing a beat point and a bar line is displayed on the display 13 based on the beat point candidate series OR2 and the beat position s. Further, the chord progression Z2 (chord name sequence) is calculated based on the maximum likelihood beat-synchronized chord feature amount sequence CR2 and displayed on the display 13. The chord progression Z2 is a chord name sequence corresponding to each beat-synchronized chord feature amount XBC2 (m) constituting the most likely beat-synchronous chord feature amount CR2. And CPU12a complete | finishes a beat point and chord estimation process in step S20.

In general, chord changes are more likely to occur at the beat. Therefore, the chord feature value XC (t _i ) calculated for each frame t _i is highly likely to change greatly in the table beat. Therefore, if the back beat is selected as the front beat by mistake, there is a high possibility that the chord feature amount XC (t _i ) changes greatly between beat points. Therefore, in this case, the beat synchronization type chord feature amount cannot accurately represent the chord feature. In other words, when the back beat is selected as the front beat, the likelihood of the beat-synchronized code feature quantity sequence is lower than when the true beat point (ie, the front beat) is selected. Therefore, the acoustic signal analysis device 10 detects the beat point candidate series OR1 and the beat point candidate series OR2 including the beat point candidates shifted by half a beat with respect to each beat point candidate constituting the beat point candidate series OR1. The likelihood LK1 and the likelihood LK2 of the beat-synchronized code feature amount series relating to the beat point candidate series OR1 and the beat point candidate series OR2 are calculated. Then, the likelihood LK1 and the likelihood LK2 are compared, and a beat point candidate series having a high likelihood is selected. Thereby, it can suppress selecting a back beat as a front beat accidentally. In addition, the combination of the beat position s and the key key that maximizes the likelihood of the beat-synchronized chord feature quantity sequence is selected, and the beat point, chord progression, and bar line position in the music are simultaneously (integrated). Presumed. Therefore, according to the acoustic signal analyzer 10, the estimation accuracy of the beat point, chord progression, and bar line position can be improved as compared with the conventional technique.

  Furthermore, in carrying out the present invention, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the object of the present invention.

  For example, in the above embodiment, the performance sound of the entire music is stored as music data, and the music data is analyzed to estimate the beat point, chord progression, and bar line position. However, instead of this, while capturing the performance sound of the music in real time, the data representing the captured performance sound is analyzed in the same manner as in the above embodiment, and the beat point, chord progression, and bar line position are estimated. Also good.

DESCRIPTION OF SYMBOLS 10 ... Acoustic signal analyzer, CR1 ... Beat synchronous type chord feature amount series, CR2 ... Beat synchronous type chord feature amount sequence, HPCP ^(T) ... Treble feature amount, HPCP ^(B) ... Base feature value, j ... chord, key ... key, OR1 ... beat point candidate series, OR2 ... beat point candidate series, s ... beat position, XB ... BPM feature quantity, XBC1 ... beat-synchronized chord feature, XBC2 ... beat-synchronized chord feature, XC ... chord feature, XO ... onset feature, ρ ^(T) ... treble power, ρ ^(B) Base power

Claims (4)

An acoustic signal acquisition means for capturing an acoustic signal representing a performance sound of a music piece as an analysis target;
Based on the acquired sound signal, the first beat point candidate series composed of a plurality of beat point candidates of the music and the half beat difference with respect to the plurality of beat point candidates constituting the first beat point candidate series, respectively. Beat point candidate series detecting means for detecting a second beat point candidate series comprising a plurality of beat point candidates;
Consists of beat-synchronized chord feature quantities each representing a chord feature between two adjacent beat point candidates out of a plurality of beat point candidates constituting the first beat point series based on the acquired sound signal Beats representing the chord features between two adjacent beat point candidates out of a plurality of beat point candidates constituting the second beat point series Beat-synchronized code feature value sequence calculating means for calculating a second beat-synchronized code feature value sequence composed of synchronized code feature values;
Probability model representing the chord progression of the music piece, the probability that the chord transition probability between beat points is set according to the combination of the number of beats within one measure, the key of the music piece, and the beat position of the first beat point Among the models, a probability model in which the first beat synchronization type chord feature amount sequence satisfies a predetermined criterion and a probability model in which the second beat synchronization type chord feature amount sequence satisfies a predetermined criterion are selected one by one, An acoustic signal analyzer comprising: estimation means for estimating a beat point, chord progression, and bar line position of the music piece based on a probability model having a high likelihood among the two selected probability models. The acoustic signal analyzer according to claim 1,
The beat synchronization type chord feature quantity calculation means
Code feature amount calculating means for calculating individual feature amounts representing the feature amount of the code for each of a plurality of sections located between the two adjacent beat point candidates;
An acoustic signal comprising: a chord feature amount smoothing unit that calculates the beat synchronous chord feature amount by smoothing chord feature amounts of a plurality of sections located between the two adjacent beat point candidates. Analysis equipment. In the acoustic signal analyzer according to claim 1 or 2,
The beat point candidate series detecting means includes
Beat / tempo feature amount calculating means for calculating a first feature amount representing a feature relating to the presence of a beat in each section of the music and a second feature amount representing a feature relating to the tempo;
Among the plurality of probability models described as a series of states classified by combinations of physical quantities related to the presence of beats and physical quantities related to tempo in each section of the music piece, the first feature quantity and the second feature quantity are those of the music piece. Beat point and tempo estimation means for simultaneously estimating beat points and tempo transitions in the music piece by selecting a probability model in which a series of observation likelihoods representing the probability observed simultaneously in each section satisfies a predetermined criterion; An acoustic signal analyzing apparatus. In the computer provided in the acoustic signal analyzer,
An acoustic signal acquisition step for capturing an acoustic signal representing a performance sound of a music piece as an analysis target;
Based on the acquired sound signal, the first beat point candidate series composed of a plurality of beat point candidates of the music and the half beat difference with respect to the plurality of beat point candidates constituting the first beat point candidate series, respectively. A beat point candidate sequence detecting step for detecting a second beat point candidate sequence comprising a plurality of beat point candidates;
Consists of beat-synchronized chord feature quantities each representing a chord feature between two adjacent beat point candidates out of a plurality of beat point candidates constituting the first beat point series based on the acquired sound signal Beats representing the chord features between two adjacent beat point candidates out of a plurality of beat point candidates constituting the second beat point series A beat-synchronized chord feature amount sequence calculating step for calculating a second beat-synchronized chord feature amount sequence composed of the synchronized chord feature amounts;
Probability model representing the chord progression of the music piece, the probability that the chord transition probability between beat points is set according to the combination of the number of beats within one measure, the key of the music piece, and the beat position of the first beat point Among the models, a probability model in which the first beat synchronization type chord feature amount sequence satisfies a predetermined criterion and a probability model in which the second beat synchronization type chord feature amount sequence satisfies a predetermined criterion are selected one by one, A computer program for executing an estimation step of estimating a beat point, chord progression, and bar line position of the music piece based on a probability model having a high likelihood among the two selected probability models.

Images