JP6281211B2 - Acoustic signal alignment apparatus, alignment method, and computer program - Google Patents

Description

  In the present invention, a first performance in which a plurality of performance parts constituting a musical piece are played, and an acoustic signal each representing a second performance in which a part of the plurality of performance parts is played are recorded. The present invention relates to an alignment apparatus that analyzes the first and second acoustic data and associates the tone generation timings of the musical sounds constituting the first and second performances, respectively. In the following description, the correspondence relationship between the sound generation timings of the respective musical sounds constituting the plurality of performances is simply referred to as alignment.

  Conventionally, for example, as shown in Non-patent Documents 1 and 2 below, acoustic signal alignment apparatuses are known. In the alignment apparatus described in Non-Patent Document 1, first, each pronunciation information (for example, note-on data and note-off data) constituting score data (for example, Standard MIDI File) representing the score of the music to be analyzed, and analysis Each musical tone constituting each of the target first performance and second performance is associated with each other. Then, using the correspondence information between the pronunciation information of the score data and each musical tone of the first performance, and the correspondence information between the pronunciation information of the musical score data and each musical tone of the second performance, the first performance and the second performance are used. The performance alignment is calculated.

  In the alignment apparatus described in Non-Patent Document 2, the alignment between the first performance and the second performance is calculated using a dynamic time warping method.

Sebastian Ewert, Mineral Muller, Peter Grosche, “HIGH RESOLUTION AUDIO SYNCLONIZATION USING CHROMA ONSET FEATURES”, Acustics, Speech 1869-p. 1872 Simon Dixon, Gerhard Widner, “MATCH: A MUSIC ALIGNMENT TOOL CHEST”, ISMIR 2005, 6th International Conference on Music Information Retrieval, p. 492-p. 497

  According to the alignment apparatus of Non-Patent Document 1, musical score data representing the music to be analyzed is required. Therefore, it is not possible to calculate the alignment of music that does not have musical score data. Also, as the first stage of analysis, the pronunciation information constituting the musical score data is associated with each musical tone constituting the first performance and the second performance to be analyzed. Then, in the second stage of analysis, the alignment of the first performance and the second performance is calculated using the analysis result of the first stage. For this reason, there is a possibility that the analysis accuracy may be reduced by accumulating errors at each analysis stage.

  In the alignment apparatus of Non-Patent Document 2, a distance measure (for example, Euclidean distance) having symmetry is used when calculating the difference (distance) between the first performance and the second performance. According to this, when the number of performance parts included in the first performance is substantially the same as the number of performance parts included in the second performance, the difference (distance) between the performances when both performances are evaluated as acoustic signals. ) Is small, a good analysis result can be obtained. However, for example, when the alignment of the performance of all the performance parts of a predetermined symphony and the performance of only a part of the performance parts is calculated using the alignment apparatus of Non-Patent Document 2, each performance Since the difference (distance) between them is large when each is evaluated as an acoustic signal, alignment cannot be calculated with high accuracy.

  In addition, when the pitches of the musical sounds of the performance parts common to the first performance and the second performance are shifted, it is determined that the distance between the two is large, and the alignment cannot be calculated with high accuracy.

The present invention has been made to address the above problems, and an object of the present invention is an alignment apparatus , an alignment method, and a computer program capable of calculating the alignment of a plurality of performances without using musical score data. The present invention provides an alignment apparatus , an alignment method, and a computer program capable of calculating alignment with high accuracy even when a difference when the signal is evaluated as an acoustic signal is large. In addition, in the description of each constituent element of the present invention below, in order to facilitate understanding of the present invention, reference numerals of corresponding portions of the embodiment are described in parentheses, but each constituent element of the present invention is The present invention should not be construed as being limited to the configurations of the corresponding portions indicated by the reference numerals of the embodiments.

In order to achieve the above object, the present invention is characterized by a first performance in which a plurality of performance parts constituting a musical piece are played, and a second performance in which a part of the plurality of performance parts is played. An alignment device (1) that analyzes first and second acoustic data (d1, d2) each recording an acoustic signal that represents a performance, and associates the sound generation timing of each musical sound that constitutes the first and second performances ( 10, 20), one spectrum (Xp (tp)) of a plurality of spectra constituting the spectrogram of the second performance and a plurality of spectra constituting the spectrogram of the first performance A frequency component in which the spectrum of the second performance of the set of spectra including one spectrum (Xm (tm)) exceeds the spectrum of the first performance is obtained. A first weight is assigned, and a second performance spectrum of the set of spectra is included in the first performance spectrum when a second weight smaller than the first weight is assigned. Evaluation value calculation means (S12, S22) for calculating evaluation values (D _{tp, tm} , L _{tp, tm} ) related to the similarity of the set of spectra using a scale set to be performed, and the similarity Calculating the evaluation value of the set of spectrum series using the evaluation value for the set of spectrums, and estimating the set of spectrum series satisfying a predetermined criterion by the evaluation value of the set of spectrum series, The alignment apparatus includes alignment calculation means (13, 23) for associating the sound generation timings of the respective musical sounds constituting the first and second performances. Note that “corresponding to the sound generation timing of each musical tone” means that the first and second performances are reproduced when the first and second performances are simultaneously reproduced using the first and second acoustic data. This means that the sound generation timings of the musical sounds are associated with each other so that they can be synchronized.

In this case, the evaluation value related to the similarity of the set of spectra is a distance (D _{tp, tm} ) of the set of spectra, and the scale is a spectrum of a second performance of the set of spectra. Has a frequency component exceeding the spectrum of the first performance, the distance of the spectrum of the second performance viewed from the spectrum of the first performance is the first distance viewed from the spectrum of the second performance. When the spectrum of the performance is larger than the distance of the performance spectrum and the spectrum of the second performance of the set of spectra is included in the spectrum of the first performance, the second spectrum viewed from the spectrum of the first performance Is a non-symmetrical distance measure in which the spectrum distance of the performance is smaller than the spectrum distance of the first performance viewed from the spectrum of the second performance, Evaluation value of the sequence, may is the sum of said distance. Note that when a simple increase function (for example, an exponential function) is applied to the distance and the cumulative value of the distance is used as an evaluation value of the set of spectrum series, the total sum of the distances is substantially calculated. This is the same as the evaluation value of the spectrum series. Therefore, the above case is also included in the present invention.

In this case, the evaluation value related to the similarity of the set of spectra is the observation likelihood (L _{tp, tm} ) of the spectrum of the second performance in the probability distribution corresponding to the scale, and The evaluation value of the spectrum series may be a likelihood model (HMM) likelihood described as a series of states classified by a combination of the spectrum of the first performance and the spectrum of the second performance.

  According to this, when the spectrum of the second performance has a frequency component exceeding the spectrum of the first performance, the evaluation value of similarity is calculated by assigning the first weight. On the other hand, when the spectrum of the second performance is included in the spectrum of the first performance, the evaluation value of similarity is calculated with a second weight smaller than the first weight. Thereby, the inclusion relationship between the first performance and the second performance can be more accurately evaluated. That is, compared with the case where the evaluation value regarding the similarity of the set of spectra is calculated using a strictly symmetric scale (that is, a scale in which the first weight and the second weight are the same), the evaluation value is It can be calculated more accurately. Therefore, even when the second performance is a subset of the first performance and the two performances are largely separated as acoustic signals, the alignment of the first performance and the second performance is more accurate. Can be calculated well.

  Another feature of the present invention is that the evaluation value calculating means converts one frequency component of the first performance spectrum and the second performance spectrum of the set of spectra to the other frequency. Pitch shift means for shifting relative to the component in the frequency axis direction, and addition means for adding an evaluation value corresponding to the shift amount of each frequency component to the evaluation value related to the similarity of the set of spectra. In addition to that. When an exponential function is applied to the evaluation value related to the similarity, the evaluation value corresponding to the shift amount of each frequency component is integrated into the evaluation value related to the similarity of the set of spectra. Also in this case, it is considered that an evaluation value corresponding to the shift amount of each frequency component is substantially added to an evaluation value related to the similarity of the set of spectra.

  According to this, the pitch of the first performance and the pitch of the second performance are relatively shifted, and the distance as the cost corresponding to the shift amount is added as the distance between both spectra. Thereby, even if the pitch of the second performance is slightly deviated from the first performance, the alignment between the first performance and the second performance can be calculated with high accuracy.

It is a conceptual diagram which shows the structure of the acoustic signal (acoustic data) of analysis object. It is a block diagram which shows the structure of the alignment apparatus which concerns on 1st and 2nd embodiment of this invention. It is a flowchart which shows the calculation procedure of the alignment of the alignment apparatus which concerns on 1st Embodiment of this invention. It is a conceptual diagram which shows the series of a grid point. It is a graph which shows the state where the spectrum of the 2nd performance is included in the spectrum of the 1st performance. It is a graph which shows the state which has a frequency component in which the spectrum of a 2nd performance exceeds the spectrum of a 1st performance. It is a flowchart which shows the calculation procedure of the alignment of the alignment apparatus which concerns on 2nd Embodiment of this invention. It is a key map showing the course of a state.

(First embodiment)
An alignment apparatus 10 according to a first embodiment of the present invention will be described. As will be described below, the alignment apparatus 10 configures each performance so that the two acoustic data d1 and acoustic data d2 representing the performance of the music can be synchronized when the performances are reproduced. Associate the playback timing of each musical sound. The acoustic data d1 and d2 are recorded as digital data of acoustic signals respectively representing sounds generated by playing one or more performance parts described in the same score. The acoustic data d1 includes performances of all the performance parts of the music (hereinafter referred to as the first performance), and the acoustic data d2 includes performances of some of the performance parts (hereinafter referred to as the second performance). (Refer to FIG. 1). In the example of FIG. 1, the acoustic data d1 to be analyzed is data in which acoustic signals generated by playing all performance parts of a predetermined music are recorded. On the other hand, the acoustic data d2 is data in which an acoustic signal generated by playing only the violin performance part of the predetermined music is recorded.

  As shown in FIG. 2, the alignment apparatus 10 includes an input operator 11, a computer unit 12, a display 13, a storage device 14, an external interface circuit 15, and a sound system 16, which are connected via a bus BS. Has been.

  The input operator 11 includes a switch corresponding to an on / off operation (for example, a numeric keypad for inputting a numerical value), a volume or rotary encoder corresponding to a rotation operation, a volume or linear encoder corresponding to a slide operation, a mouse, a touch panel, etc. Composed. These operators are operated by the player's hand to select acoustic data to be analyzed, start or stop the analysis of the acoustic data, and play or stop the performance using the selected acoustic data (from the sound system 16 described later). Output or stop), and setting of various parameters relating to the analysis of the acoustic signal. When the input operator 11 is operated, operation information indicating the operation content is supplied to the computer unit 12 described later via the bus BS.

  The computer unit 12 includes a CPU 12a, a ROM 12b, and a RAM 12c connected to the bus BS. The CPU 12a reads a program representing an alignment calculation procedure described later from the ROM 12b and executes it. In addition to the program, the ROM 12b stores various data such as initial setting parameters, graphic data for generating display data representing an image displayed on the display 13, and character data. The RAM 12c temporarily stores data necessary for executing the program.

  The display 13 is configured by a liquid crystal display (LCD). The computer unit 12 generates display data representing contents to be displayed using graphic data, character data, and the like, and supplies the display data to the display unit 13. The display device 13 displays an image based on the display data supplied from the computer unit 12. For example, when selecting acoustic data to be analyzed, a list of selectable acoustic data is displayed on the display unit 13.

  The storage device 14 includes a large-capacity nonvolatile recording medium such as an HDD, FDD, CD, or DVD, and a drive unit corresponding to each recording medium. The storage device 14 stores acoustic data d1 and d2. The acoustic data d1 and d2 are each composed of a plurality of sample values obtained by sampling the performance of the predetermined music at a predetermined sampling period (for example, 1/444100 sec), and each sample value is a continuous address in the storage device 14. Are recorded in order. Each acoustic data d1, d2 includes title information for identifying the data, data size information representing the capacity, and the like. The acoustic data d1 and d2 may be stored in the storage device 14 in advance, or may be acquired from the outside via the external interface circuit 15 described later.

  The external interface circuit 15 includes a connection terminal that enables the alignment device 10 to be connected to an external device such as an electronic music device or a personal computer. The alignment apparatus 10 can be connected to a communication network such as a LAN (Local Area Network) or the Internet via the external interface circuit 15.

  The sound system 16 includes a D / A converter that converts the acoustic data d1 and d2 into an analog sound signal, an amplifier that amplifies the converted analog sound signal, and a left and right that converts the amplified analog sound signal into an acoustic signal and outputs the sound signal. A pair of speakers is provided. When instructed to reproduce the performance using the acoustic data d1 or the acoustic data d2, the CPU 12a supplies the acoustic data d1 or the acoustic data d2 to the sound system 16. Thereby, the user can audition the performance to be analyzed.

Next, the operation (alignment calculation procedure) of the alignment apparatus 10 configured as described above will be described. In the present embodiment, first, as shown in FIG. 3, an alignment calculation process is started in step S10. Next, in step S11, the spectrogram of each acoustic signal is calculated using the acoustic data d1 and the acoustic data d2. In the following description, the time (or frame number) in each acoustic signal is described as time tm (= 1, 2,..., TM) and time tp (= 1, 2,..., TP). To do. Correspondence between the sequence of the spectrum Xm (tm) constituting the spectrogram of the acoustic signal represented by the acoustic data d1 and the sequence of the spectrum Xp (tp) constituting the spectrogram of the acoustic signal represented by the acoustic data d2 Calculated using dynamic time stretching method. As shown in FIG. 4, on the plane with the time axis of each of the first performance and the second performance as coordinate axes, the correspondence between time tp and time tm is the relationship between the lattice points c _{tp and tm} on the plane. Expressed as a series.

Specifically, in step S12, the distance D _{tp, tm} between the spectrum Xp (tp) and the spectrum Xm (tm) is calculated based on the following equation (1).

The distance D _{tp, tm} corresponds to the evaluation value related to the similarity of the present invention. Further, the first term of the expression (1) corresponds to the Itakura Saito distance of the spectrum Xp (tp) viewed from the spectrum Xm (tm). Further, “Xp (f, tp)” in the first term represents the amplitude (power) at the frequency f of the spectrum Xp (tp). “Α” represents a pitch shift amount. That is, “Xm (αf, tm)” represents the amplitude (power) at the frequency αf of the spectrum obtained by multiplying the frequency f of each frequency component constituting the spectrum Xm (tm) by α (that is, pitch shifted). Further, the second term of the formula (1) is a distance corresponding to the cost for the above-described pitch shift. In the present embodiment, the distance corresponding to the cost for the pitch shift is defined as following a lognormal distribution whose average is “1”.

Next, in step S13, a series of lattice points c _{tp, tm} that minimizes the sum of the distances D _{tp, tm} is calculated using a dynamic time expansion / contraction method similar to that of Non-Patent Document 2. The As described above, the alignment of the first performance and the second performance (that is, a series of combinations of time tp and time tm) is calculated, and the alignment calculation process ends in step S14.

As described above, since the first performance is a performance of all parts and the second performance is a performance of some parts, the second performance is a subset of the first performance. Therefore, when calculating the distance between the spectrum Xp (tm) and the spectrum Xp (tp), the weight used when the spectrum Xm (tm) is included in the spectrum Xp (tp) (FIG. 5A) is used as the spectrum Xm (tm). ) May be smaller than the weight used when it has a frequency component exceeding the spectrum Xp (tp) (the hatched portion in FIG. 5B). According to the alignment apparatus 10, the distance D _{tp, tm} is calculated using the Itakura Saito distance as a distance scale. In other words, when calculating the distances D _{tp, tm} , an asymmetric distance scale was used so that the measurement distances differed depending on the inclusion relationship between the first performance spectrum and the second performance spectrum. Specifically, when the spectrum of the second performance has a frequency component exceeding the spectrum of the first performance, the distance of the spectrum of the second performance viewed from the spectrum of the first performance is the second performance spectrum. When a weight that is larger than the spectrum distance of the first performance viewed from the spectrum (the first weight of the present invention) is added, and the spectrum of the second performance is included in the spectrum of the first performance The weight of the spectrum of the second performance viewed from the spectrum of the first performance is smaller than the distance of the spectrum of the first performance viewed from the spectrum of the second performance (the second of the present invention). ) And a distance D _{tp, tm} are calculated. Therefore, the distance of the spectrum of the second performance with respect to the spectrum of the first performance can be expressed more accurately than in the case of using a strictly symmetric distance scale. Therefore, even when the second performance is a subset of the first performance and the two performances are largely separated as acoustic signals, the alignment of the first performance and the second performance is more accurate. Can be calculated well.

  Moreover, in the said 1st Embodiment, the distance as a cost according to the shift amount was added to the distance of both spectra while the 2nd performance was pitch-shifted. Thereby, even if the pitch (tuning) of the second performance is slightly shifted from the pitch of the first performance, the alignment between the first performance and the second performance can be accurately calculated.

In step S13 of the first embodiment _, a series of lattice points c _{tp, tm} that minimizes the sum of the distances D _{tp, tm} is calculated, but the lattice points c _tp satisfy a predetermined reference value. _{, Tm} series may be calculated. For example, a cost for transition of lattice points may be set, and a series of lattice points c _{tp, tm} that minimizes the sum of the sum of costs for the transition of lattice points and the sum of distances D _{tp, tm} may be calculated.

In the first embodiment, the distance D _{tp, tm} is calculated based on the equation (1), but the distance D _{tp, tm} may be calculated based on the following equation (2). Equation (2) represents an expected value of Itakura Saito's distance for “α”.

The distance D _{tp, tm} may be calculated based on an arithmetic expression obtained by applying a monotonically increasing function to Expression (1). For example, it may be calculated based on Expression (3) to which an exponential function is applied.

In this case, in step S13, a series of lattice points c that minimizes the accumulation of the distances D _{tp and tm} is calculated.

In the first embodiment, the Itakura Saito distance is adopted as the distance scale, but the present invention is not limited to this. Bregman divergence generated from a convex function such that the following expression (4) holds for an arbitrary value X and a non-negative value a may be adopted as a distance measure. For example, generalized KL divergence may be employed.

  In the first embodiment, the pitch of the first performance is configured to be shiftable. However, it is only necessary that the pitch of the first performance is relatively shiftable with respect to the pitch of the second performance. That is, instead of or in addition to the pitch of the first performance, the pitch of the second performance may be shiftable.

(Second Embodiment)
Next, an alignment apparatus 20 according to a second embodiment of the present invention will be described. Since the configuration of the alignment apparatus 20 is the same as that of the alignment apparatus 10, the description thereof is omitted. The operation of the alignment apparatus 20 is different from that of the first embodiment. That is, in the second embodiment, the program to be executed is different from that in the first embodiment. In general, the Itakura Saito distance minimization of “Y” viewed from “X” may be equivalent to the maximum likelihood estimation of Y when “Y” is observed in an exponential distribution with an expected value of “X”. Are known. Therefore, in the second embodiment, it is assumed that the spectrum Xp (tp) follows an exponential distribution having the spectrum Xm (tm) as an average. Of the hidden Markov model HMM expressed as a series of states classified by a combination of the spectrum Xp (tp) and the spectrum Xm (tm) (that is, a series of correspondence between the time tp and the time tm), The alignment is calculated by selecting a model whose likelihood for the series of time tm as an observed value satisfies a predetermined criterion.

Specifically, as shown in FIG. 6, alignment calculation processing is started in step S20. Next, in step S21, spectrograms of the respective acoustic signals are calculated using the acoustic data d1 and the acoustic data d2 as in the first embodiment. In step S22, the observation likelihood L _{tp, tm} is calculated based on the following equation (5) using the spectrum Xm (tm) and the spectrum Xp (tp) constituting the calculated spectrogram. . That is, a value calculated by substituting the spectrum Xp (tp) as a random variable of the exponential distribution is set as an observation likelihood L _{tp, tm} .

Note that the observation likelihood L _{tp, tm} corresponds to the evaluation value related to the similarity of the present invention. Further, the transition probabilities between the states in the hidden Markov model HMM of this embodiment are set as follows. That is, the time tp is always set to advance by “1” in the state transition. The probability that the time tm advances by “1” when the time tp advances by “1” is “u” (0 <u <1), and the time tm becomes the same time when the time tp advances by “1”. Let the probability of staying be “1-u”. The probability of other state transitions is “0”. Therefore, the state path in the hidden Markov model HMM is expressed as shown in FIG.

The likelihood of each hidden Markov model HMM is calculated as the cumulative value of the observation likelihood L _{tp, tm} in the state on each path and the transition probability between the states. For example, the likelihood C _R model of the path R shown by the thick solid line in FIG. 7 is calculated by the following equation (6).

Next, in step S23, the maximum likelihood path among the paths (i.e., the model likelihood C _R becomes maximum) is calculated using the Viterbi algorithm. In this case, it is preferable to set “L _1,1 = 1” and “L _{TP, TM} = 1”. As described above, the alignment of the first performance and the second performance (that is, a series of combinations of time tp and time tm) is calculated, and the alignment calculation process ends in step S24.

In the present embodiment, it is assumed that the spectrum Xp (tp) follows an exponential distribution that averages the spectrum Xm (tm) (that is, a distribution corresponding to the Itakura Saito distance), and the observation likelihood L _tp, of the spectrum Xp (tp) _tm was calculated. That is, when the spectrum Xm (tm) has a frequency component exceeding the spectrum Xp (tp), the observation likelihood L _{tp, tm} is calculated with the first weight. On the other hand, when the spectrum Xm (tp) is included in the spectrum Xp (tp), the observation likelihood L _{tp, tm} is calculated with a second weight smaller than the first weight. Thereby, the distance of the 2nd performance with respect to a 1st performance can be expressed more correctly compared with the case where the distribution corresponding to a strictly symmetrical distance scale is used. Therefore, the same effect as that of the first embodiment can be obtained by the alignment apparatus 20 configured as described above. In other words, even when the second performance is a subset of the first performance and the two performances are largely separated as acoustic signals, the alignment between the first performance and the second performance is more accurate. Can be calculated well.

In the second embodiment, unlike the first embodiment, the pitch difference between the first performance and the second performance is not considered. However, also in the second embodiment, the observation likelihood L _{tp, tm} may be calculated as in the following equation (7) in order to take into account the pitch deviation between the first performance and the second performance. .

  In Equation (7), the likelihood as the cost for the pitch shift may be subtracted. In Expression (7), the pitch of the first performance is configured to be shiftable. However, the pitch of the first performance may be relatively shiftable with respect to the pitch of the second performance. . That is, instead of or in addition to the pitch of the first performance, the pitch of the second performance may be shiftable.

Further, in the second embodiment the step S23, the path of maximum likelihood (that is, the likelihood C _R is the path with the maximum), but is calculated, the path that satisfies a predetermined reference value may be calculated . For example, the state having the maximum likelihood at each time may be selected.

  In the second embodiment, an exponential distribution corresponding to the Itakura Saito distance is used. However, the present invention is not limited to this, and any distribution corresponding to Bregman divergence can be used. For example, a Poisson distribution corresponding to generalized KL divergence may be employed.

  In the second embodiment, “u” used to represent the transition probability between states is a constant, but is not limited thereto. For example, a Bernoulli distribution having “u” as a random variable may be set as a prior distribution, and an appropriate “u” may be determined for the observed value when the maximum posterior probability estimation of the state series is executed.

Further, observation likelihood L _tp, taking the logarithm of _tm as well as the log observation likelihood, if the transition probabilities between states and log transition probabilities, likelihood C _R of the path R, each state on the path R Is calculated as the sum of the logarithmic observation likelihood and the log transition probability between each state.

10,20 ... Alignment device 12 ... a computer unit, d1, d2 ... acoustic _{data, L tp, tm} ... observation _{likelihood, C} R ... likelihood, Xp, Xm ... Spectrum, R ... Path, HMM ... Hidden Markov Model

Claims (6)

First and first recordings each recording an acoustic signal representing a first performance in which a plurality of performance parts constituting a musical piece are performed and a second performance in which a part of the plurality of performance parts are performed. An alignment device that analyzes the acoustic data of 2 and associates the sound generation timing of each musical sound that constitutes the first and second performances, respectively,
A first of a set of spectra comprising one spectrum of a plurality of spectra constituting the spectrogram of the second performance and one spectrum of a plurality of spectra constituting the spectrogram of the first performance. When the spectrum of the two performances has a frequency component exceeding the spectrum of the first performance, a first weight is assigned, and the spectrum of the second performance of the set of spectra is included in the spectrum of the first performance. Evaluation value calculation means for calculating an evaluation value related to the similarity of the set of spectra using a scale set so that a second weight smaller than the first weight is applied when
An evaluation value of the set of spectrum series is calculated using the evaluation value related to the similarity, and the set of spectrum series satisfying a predetermined criterion for the evaluation value of the set of spectrum series is estimated. By means of this, alignment calculation means for associating the sound generation timing of each musical sound constituting the first and second performances,
An alignment apparatus comprising: The alignment apparatus according to claim 1,
The evaluation value calculation means includes
Pitch shifting means for shifting one frequency component of the first performance spectrum and the second performance spectrum of the set of spectra relative to the other frequency component in the frequency axis direction. When,
An alignment apparatus, further comprising: addition means for adding an evaluation value corresponding to the shift amount of each frequency component to an evaluation value relating to the similarity of the set of spectra. The alignment apparatus according to claim 1 or 2,
The evaluation value related to the similarity of the set of spectra is a distance between the set of spectra,
The scale is a spectrum of the second performance viewed from the spectrum of the first performance when the spectrum of the second performance of the set of spectra has a frequency component exceeding the spectrum of the first performance. Is larger than the distance of the spectrum of the first performance viewed from the spectrum of the second performance, and the spectrum of the second performance of the set of spectra becomes the spectrum of the first performance. When included, the asymmetry of the second performance spectrum seen from the first performance spectrum is smaller than the first performance spectrum distance seen from the second performance spectrum. Is a distance scale of
The alignment apparatus, wherein the evaluation value of the set of spectrum series is a sum of the distances. The alignment apparatus according to claim 1 or 2,
The evaluation value related to the similarity of the set of spectra is an observation likelihood of the spectrum of the second performance in the probability distribution corresponding to the scale,
The evaluation value of the set of spectrum series is a likelihood of a probability model described as a series of states classified by a combination of the spectrum of the first performance and the spectrum of the second performance. An alignment device. First and first recordings each recording an acoustic signal representing a first performance in which a plurality of performance parts constituting a musical piece are performed and a second performance in which a part of the plurality of performance parts are performed. 2 is an alignment method for analyzing the sound data of 2 and associating the sound generation timing of each musical sound constituting the first and second performances,
A first of a set of spectra comprising one spectrum of a plurality of spectra constituting the spectrogram of the second performance and one spectrum of a plurality of spectra constituting the spectrogram of the first performance. When the spectrum of the two performances has a frequency component exceeding the spectrum of the first performance, a first weight is assigned, and the spectrum of the second performance of the set of spectra is included in the spectrum of the first performance. An evaluation value calculating step of calculating an evaluation value related to similarity of the set of spectra using a scale set so as to be given a second weight smaller than the first weight,
An evaluation value of the set of spectrum series is calculated using the evaluation value related to the similarity, and the set of spectrum series satisfying a predetermined criterion for the evaluation value of the set of spectrum series is estimated. An alignment calculation step for associating the sound generation timings of the musical tones constituting the first and second performances,
An alignment method comprising: First and first recordings each recording an acoustic signal representing a first performance in which a plurality of performance parts constituting a musical piece are performed and a second performance in which a part of the plurality of performance parts are performed. 2 is a computer program that causes the computer to execute an alignment process that analyzes the acoustic data of 2 and associates the sound generation timings of the musical sounds that constitute the first and second performances, respectively.
A first of a set of spectra comprising one spectrum of a plurality of spectra constituting the spectrogram of the second performance and one spectrum of a plurality of spectra constituting the spectrogram of the first performance. When the spectrum of the two performances has a frequency component exceeding the spectrum of the first performance, a first weight is assigned, and the spectrum of the second performance of the set of spectra is included in the spectrum of the first performance. An evaluation value calculating step of calculating an evaluation value related to similarity of the set of spectra using a scale set so as to be given a second weight smaller than the first weight,
An evaluation value of the set of spectrum series is calculated using the evaluation value related to the similarity, and the set of spectrum series satisfying a predetermined criterion for the evaluation value of the set of spectrum series is estimated. An alignment calculation step for associating the sound generation timings of the musical tones constituting the first and second performances,
A computer program for causing a computer to execute a process including:

Images