JP5131220B2 - Singing pitch difference identification device and program - Google Patents

Description

  The present invention relates to a singing pitch difference specifying device and a program for generating information used for implementing functions of a karaoke device such as suggesting a recommended song and executing scoring.

  Conventionally, in a karaoke apparatus in which a user enjoys singing along with accompaniment music, what is proposed as a recommended song that the user has estimated to be able to sing, and what scores the skill of the user's singing are known. Yes.

  In general, in these karaoke devices, when proposing a recommended song, it is suggested that a song that is easy for the user to sing is recommended as a recommended song. It is required to derive the scoring results.

  On the other hand, for example, when proposing a recommended song, it has been proposed to select, as a recommended song, a song whose pitch of component sounds transitions within a range where the user can sing (for example, Patent Document 1). reference). In addition, when performing scoring, in addition to scoring so that the higher the degree of coincidence between the basic frequency of the singing voice and the reference value prepared in advance, the better the result (hereinafter referred to as basic scoring) Thus, it has been proposed to derive a scoring result in which a weight is given to a range where a singer can sing (for example, see Patent Document 2).

  In other words, in the karaoke apparatuses described in Patent Document 1 and Patent Document 2, a weight that is determined by using a sound range that the user can sing (hereinafter referred to as a singing sound range) or added to the result of the basic scoring. Is derived.

JP-A-2005-107313 Japanese Patent No. 3290945

  By the way, the ease of singing and the skill of singing are not determined only by the singing range, but also vary by other various singing abilities.

  However, in the karaoke apparatuses described in Patent Literatures 1 and 2, in order to score (evaluate) the determination of recommended songs and the skill of singing using only the singing sound range among the singing abilities possessed by the user. There is a problem that it is not possible to make a song that is easy to sing for the user as a recommended song or to accurately score the user's singing skill.

  Then, this invention aims at providing the technique which can produce | generate the information used in order to determine the recommended music which is easy to sing for a user, or to accurately mark the skill of a singing.

  The present invention made to achieve the above object is a singing pitch difference identifying device used in a karaoke device that plays music according to music data representing the pitch and value of each of the constituent sounds constituting each music. .

  In the singing pitch difference identifying device of the present invention, the voice signal acquisition means is the voice that is input during the performance of the music (hereinafter referred to as the voice signal) by the karaoke device, and the user who is playing the voice The song data is generated in association with the identification information for identifying the song, and the song data generation means analyzes the frequency of the acquired voice signal to generate song data representing pitch transition.

  Then, the transition section specifying means, in the generated singing data, from the end of the singing section for the transition source sound (hereinafter referred to as section end) to the beginning of the singing section for the arrival sound (hereinafter referred to as section starting end). ) To the singing section (hereinafter referred to as singing transition section). However, the reaching sound referred to here is after the pitch is switched among two consecutive constituent sounds (hereinafter referred to as specific constituent sounds) so that the pitch is switched in the music specified from the identification information. The component sound that arrives, and the transition source sound referred to here is the other of the specific component sounds (the component sound that is the transition source before the pitch is switched).

  Further, for each of the identified singing transition sections, the transition value deriving means derives a pitch transition value that becomes a larger value as the degree of coincidence between the singing transition section and the pitch transition model is higher. However, the pitch transition model referred to here is a transition mode when the utterance pitch is ideally shifted from the pitch corresponding to the transition source sound to the pitch corresponding to the arrival sound.

  The pitch transition value derived in this way is summed up for each pitch difference in each singing transition section by the singing pitch difference specifying means, and the counting result (hereinafter referred to as the total transition value) is defined. A pitch difference that is greater than or equal to the value is specified as a pitch difference that can be sung by the user specified from the identification information.

  According to such a singing pitch difference specifying device of the present invention, it is possible to specify a pitch difference that the user can sing, that is, a pitch that the user can sing.

  In particular, in a state in which the user uttered forcibly, it is difficult to smoothly sing the pitch transition from the pitch corresponding to the transition source sound to the pitch corresponding to the arrival sound, According to the singing pitch difference specifying device of the present invention, it is possible to specify a pitch difference that can be sung without the user's excessive effort.

  And in the karaoke apparatus having the singing pitch difference identifying device of the present invention, the music whose pitch difference between two consecutive constituent sounds is mainly within the range of the singable pitch difference is recommended music. Proposing to the user can make the user more entertaining karaoke. Further, in the karaoke apparatus having the singing pitch difference specifying device of the present invention, if a weight that becomes larger as the specified singing pitch difference is larger is given to the result of the basic scoring, the user's singing skill is improved. Can be scored more accurately.

  By the way, in general, if the pitch difference between the pitch of the transition source sound and the pitch of the arrival sound (hereinafter referred to as the pitch difference between constituent sounds) is large, it reaches from the end of the transition source sound in the music It is difficult to smoothly sing the section between the beginning of the sound (hereinafter referred to as a specific section). However, even if the pitch difference between the constituent sounds is large, it becomes easier to sing the specific section more smoothly as the time length of the specific section is longer.

  That is, even if the pitch difference between the constituent sounds is the same, whether or not the user can sing varies depending on the time length of the specific section.

  For this reason, in the present invention, the singing pitch difference specifying means is configured to classify and sum the pitch transition values for each section length of each singing transition section, as described in claim 2. It is desirable that

  According to the singing pitch difference specifying device configured as described above, the pitch difference that can be sung by the user can be specified for each time length of the singing section. That is, according to the singing pitch difference specifying device configured as described above, the singing ability of the user can be specified in detail.

  In addition, since the distribution of the pitch difference between the constituent sounds in one musical piece is usually greatly different depending on each musical piece, when the user sings only the musical piece that includes a lot of specific constituent pitch differences, The total transition value for the specific inter-sound pitch difference becomes a large value.

  Therefore, in the singing pitch difference specifying device of the present invention, it is desirable to normalize the total transition value.

  In this way, when the total transition value is normalized, the distribution of the normalized total transition value is as shown in FIG. 9, as long as the user is a good singer (ie, has high singing ability). The tabulation results for each pitch difference (ie, normalized tabulated transition values) are not varied, and the tabulation results are close to the maximum value for all pitch differences. On the other hand, if the user is not good at singing (that is, the singing ability is low), the tabulation results for each pitch difference (that is, the normalized total transition value) vary, and the pitch difference that can be sung. The difference between the totaling result in and the totaling result in the pitch difference that cannot be sung becomes large.

  Therefore, if the specified value used in the singing pitch difference specifying means is a ratio specified in advance with respect to the maximum value of the total transition value as described in claim 3, it depends on the level (skill) of the user. Thus, it is possible to appropriately specify the pitch difference that the user can sing.

  By the way, in the present invention, as described in claim 4, the transition section specifying means has a section length that is set in advance so that the variance value deriving means continues to the song data as time progresses. A plurality of time windows are set, and for each of the set time windows, a dispersion value that is a variance of the singing data within the time window is derived, and the start / end identification means are continuous with each other among the derived dispersion values. It is also possible to derive the difference between the variance values for the time window to be determined, and specify the centers of the two time windows where the difference is minimum and maximum as the end of the section and the start of the section.

  In the transition section specifying means configured as described above, the variance value deriving means is, in particular, the switching section specifying means, in the song data, the singing section for the transition original sound, the singing section for the reaching sound, and both singing sections. It may be configured to specify a switching section composed of singing sections sandwiched between and set a plurality of time windows in the specified switching section. And the method of specifying a song section may acquire the music data specified from identification information, and may collate song data and music data.

  The present invention may be a program executed on a computer used in a karaoke apparatus.

  In this case, as described in claim 5, the program of the present invention acquires the audio signal in association with the identification information in the audio signal acquisition procedure, and uses the acquired audio signal as the song data generation procedure. By performing frequency analysis, singing data representing pitch transitions is generated, and in the generated singing data, the singing section from the end of the section to the beginning of the section is set as the singing transition section by the transition section specifying procedure. Identify. At the same time, for each specified singing transition section, the transition value deriving procedure derives a pitch transition value that becomes larger as the degree of coincidence between the singing transition section and the pitch transition model is higher, and is derived. The pitch transition values are counted for each pitch difference in each singing transition section in the singing pitch difference identification procedure, and the pitch difference for which the counting result is a specified value or more is identified from the identification information. It is necessary to be specified as a pitch difference that the user can sing.

  That is, the program described in claim 5 is a program for causing a computer to realize each means constituting the singing pitch difference specifying device described in claim 1. Therefore, by causing the computer to execute each procedure, it is possible to obtain the same effect as the singing pitch difference specifying device described in claim 1.

  If it is such a program, for example, it is recorded on a computer-readable recording medium such as a DVD-ROM, a CD-ROM, a hard disk, etc., and loaded into a computer as necessary, and started. It can be used by being acquired by a computer via a communication line and starting up.

It is a block diagram which shows schematic structure of a karaoke system. It is a flowchart which shows the process sequence of a song pitch difference specific process. It is a flowchart which shows the process sequence of a singing transition area extraction process. It is drawing which showed typically the processing content of song pitch difference specific processing. It is drawing which showed typically the processing content of song pitch difference specific processing. It is drawing which showed typically the processing content of the singing transition area extraction process. It is drawing which showed typically the processing content of the singing transition area extraction process. It is drawing which showed typically the processing content of song pitch difference specific processing. It is explanatory drawing shown about the totalization transition value normalized (outline | summary of the invention which concerns on Claim 3).

  Embodiments of the present invention will be described below with reference to the drawings.

  First, FIG. 1 is a block diagram showing a schematic configuration of a karaoke system for a user to sing according to the performance of a song processed in advance for karaoke (hereinafter referred to as karaoke song).

<Configuration of the entire karaoke system>
As shown in FIG. 1, the karaoke system 1 includes a karaoke device 20 that reproduces karaoke music specified by a user, and a server 30 that distributes music data, which is data necessary for reproducing karaoke music, to the karaoke device 20. The karaoke apparatus 20 and the server 30 are connected via a network (for example, a dedicated line, a WAN, or the like). That is, the karaoke system 1 is configured as a so-called communication karaoke system.

  Among these, the server 30 is a known device having at least a processing program (hereinafter referred to as a karaoke processing program) necessary for playing karaoke music and a storage device (not shown) for storing music data, and a ROM, RAM, and CPU. This is a well-known karaoke service server device composed of an information processing device mainly composed of a microcomputer (not shown).

  That is, the server 30 centrally manages information about the user who uses the karaoke system 1 (more precisely, the karaoke device 20) (hereinafter referred to as user information), the karaoke scoring result transmitted from the karaoke device 20, and the like. In addition, music data and a karaoke processing program are transmitted in response to a request from the karaoke apparatus 20.

  By the way, the music data is prepared in advance for each karaoke music, and is, for example, well-known karaoke playback data described in the MIDI (Musical Instrument Digital Interface) format.

  Therefore, the music data is composed of music information that is data related to karaoke music, a guide melody that is data related to the melody that the user should sing, and lyrics information that is data related to the lyrics of the karaoke music.

  The music information includes music number data for specifying the karaoke music, music name data indicating the music name, and time data indicating the performance time of the karaoke music.

  The guide melody is composed of pitches and tone values of constituent sounds that form the melody of karaoke music. Specifically, in the guide melody of the present embodiment, the tone output start time and tone output end time of each component sound are represented together with the pitch of each component sound. However, the tone output start time referred to here is the time from the start of the performance of the karaoke music until the output of the constituent sound is started, and the tone output end time is the time until the output of the constituent sound is ended. This is the time since the start of karaoke music.

<About karaoke equipment>
Next, the configuration of the karaoke apparatus 20 will be described.

  The karaoke apparatus 20 includes a communication unit 22 for executing data communication with the server 30, a storage unit 21 for storing a karaoke processing program and music data acquired from the server 30 via the communication unit 22, and various types. A display unit 23 for displaying an image and an operation reception unit 24 for receiving an instruction from the user are provided. Further, the karaoke apparatus 20 includes a microphone 26 for inputting sound, a speaker 27 for outputting sound, a sound input / output unit 25 for controlling sound input / output through the microphone 26 and the speaker 27, and karaoke. And a control unit 28 for controlling the units 21, 22, 23, 24, and 25 constituting the apparatus 20.

  Among these, the communication unit 22 is a communication interface for connecting the karaoke apparatus 20 to a network (for example, a dedicated line or WAN) to communicate with the outside, and outputs various data to the server 30 and also the server 30. Acquire various data and processing programs.

  The display unit 23 is a display device configured from, for example, a liquid crystal display, and the operation receiving unit 24 is input via, for example, an input device configured from a plurality of key switches or the like, or a known remote controller. It comprises a receiving device that accepts instructions.

  The voice input / output unit 25 is configured as an AD converter that converts voice (analog signal) input via the microphone 26 into a digital signal and inputs the digital signal to the control unit 28. At the same time, the audio input / output unit 25 is configured to control the output of audio from the speaker 27. In the following, the sound that is input via the microphone 26 and converted into a digital signal is referred to as sound data.

  Furthermore, the storage unit 21 is a storage device (for example, a hard disk drive) configured to retain stored contents even when the power is turned off and to be able to read and write the stored contents. The storage unit 21 includes a program storage area for storing a karaoke processing program, a music data storage area for storing music data, and a specifying data storage area for storing audio data. The data storage area for specification is a voice data storage area for storing voice data for a predetermined number of karaoke songs (for example, for five songs), and a predetermined number of users (for example, for five people). ) Is prepared for the minutes.

  Next, the control unit 28 stores the ROM 28a that stores programs and data that need to retain stored contents even when the power is turned off, the RAM 28b that temporarily stores programs and data, and the ROM 28a and RAM 28b. It is mainly configured by a known microcomputer having at least a CPU 28c that executes control and various operations for the respective units 21, 22, 23, 24, and 25 constituting the karaoke apparatus 20 according to programs and data.

  The karaoke processing program is read from the storage unit 21 into the RAM 28b, and the CPU 28c executes each process according to the karaoke processing program stored in the RAM 28b.

  As the karaoke processing program, there is a karaoke performance processing program for performing (reproducing) a karaoke piece designated via the operation reception unit 24 and executing a well-known karaoke performance processing for displaying lyrics on the display unit 23. . Also, as a karaoke processing program, the pitch and tempo of the singing extracted from the voice input via the microphone 26 during the performance of the karaoke music are collated with a scoring standard (that is, a guide melody), and the degree of adaptation is scored. There is a scoring processing program for executing a well-known scoring process as a scoring result.

  In addition, as the karaoke processing program, there is a user information processing program for executing user information processing for receiving user information (for example, name, sex, identification number (ID), age) via the operation receiving unit 24. Further, as the karaoke processing program, an accumulation process for accumulating data generated during the performance of karaoke music (for example, voice data) or a history when the karaoke apparatus 20 is used in the storage unit 21 or the server 30 is executed. There is a storage processing program.

  Moreover, the singing pitch which specifies the pitch difference (henceforth singing pitch difference) which a user can sing based on the audio | voice data stored in the data storage area for specification of the memory | storage part 21 as a karaoke processing program. There is a pitch difference specifying process program for executing the difference specifying process. That is, when the CPU 28c executes the singing pitch difference specifying process, the karaoke device 20 (more precisely, the control unit 28) functions as the singing pitch difference specifying device of the present invention.

<Operation of Karaoke System 1>
Next, the operation (operation) of the karaoke system 1 will be described.

  When the karaoke system 1 is used, the karaoke device 20 (more precisely, the control unit 28) executes a user information processing program to receive user information for the number of users who use the karaoke device 20, Each received user information is associated with each audio data storage area.

  Furthermore, in the karaoke apparatus 20, by executing a karaoke performance processing program, the karaoke music designated by the user is played and the lyrics of the karaoke music are displayed on the display unit 23. However, the control unit 28 acquires user information via the operation receiving unit 24 before the performance of each karaoke piece is started.

  When the performance of the karaoke music is started, the user sings according to the karaoke music being played at that time (hereinafter referred to as the corresponding karaoke music). Then, when the performance of the corresponding karaoke song is completed, in the control unit 28, the CPU 28c executes the accumulation processing program and associates the voice data generated from the user's singing voice with the song number data of the corresponding karaoke song. (Hereinafter, data in which audio data and song number data are associated with each other is also referred to as correspondence data) and is stored in the audio data storage area. However, the audio data storage area in which the correspondence data is stored is associated with the user information of the user who sang the corresponding karaoke song. Thereby, audio | voice data, music number data, and user information are matched.

  Thereafter, the karaoke apparatus 20 repeats a series of cycles from the execution of the karaoke performance processing program to storing the voice data in the voice data storage area until the user finishes specifying the karaoke music.

<Singing pitch difference identification processing>
Next, the singing pitch difference specifying process executed by the CPU 28c will be described.

  Here, FIG. 2 is a flowchart showing a processing procedure of the singing pitch difference identification processing.

  This singing pitch difference specifying process is started when a prescribed number of voice data is stored in the voice data storage area of the storage unit 21, that is, when one user sings a prescribed number of karaoke songs. It is.

  As shown in FIG. 2, when the singing pitch difference specifying process is started, first, in S110, one piece of voice data is acquired from all the voice data stored in the voice data storage area of the storage unit 21. To do. That is, as shown in FIG. 4A, an audio signal whose signal level changes along with the performance of karaoke music (that is, the progress of performance time) is acquired as audio data.

  Subsequently, in S120, the audio data acquired in S110 is extracted for each reference time window m set so as to continue along with the progress of the performance time, and frequency analysis is performed for each extracted audio data (this embodiment). Then, FFT (Fast Fourier Transform) is performed. As a result of the frequency analysis, the frequency spectrum (that is, the distribution of frequency components) of the audio data within each reference time window m is generated by the number of the set reference time windows m. The reference time window m is a period having a predetermined time length Tw (for example, several tens of ms), and the subscript m is the index number of the reference time window (therefore, m is a natural number of 1 or more). ).

  Furthermore, in S130, based on the result (frequency spectrum) of the frequency analysis in S120, a fundamental frequency transition f0v (m) corresponding to the song data of the present invention is derived.

  Specifically, in S130 in the present embodiment, a harmonic structure model represented by a comb shape is prepared in advance, and the reference time is determined using a known method for collating the harmonic structure model with each frequency spectrum. The fundamental frequency f0 in the window m is detected from the frequency spectrum. The detected fundamental frequency f0 is collected as a fundamental frequency transition f0v (m) in accordance with the time transition of the reference time window m, that is, the progress of the performance time of the corresponding karaoke piece.

  Thereby, the fundamental frequency transition f0v (m) representing the temporal transition of the fundamental frequency f0 when the user sings the corresponding karaoke piece as shown in FIG. 4B is derived.

  In subsequent S140, according to the music number data associated with the audio data acquired in S110, the guide melody of the karaoke music corresponding to the music number data is acquired from the storage unit 21.

  In S150, the basic frequency transition f0v (m) derived in S130 and the guide melody acquired in S140 are collated to derive the time delay amount tl (k).

  However, the time delay amount tl (k) means that after the pitch is switched between two component sounds (hereinafter referred to as specific component sounds) that are continuous so that the pitches are switched and the lyrics are associated with each other. It represents the utterance delay time with respect to the constituent sound that reaches (hereinafter referred to as the arrival sound). Hereinafter, among the specific component sounds, a component sound that is a transition source before the pitch is switched is referred to as a transition source sound.

  Here, a method of calculating the time delay amount tl (k) in the present embodiment will be described in detail. However, in the present embodiment, k represents the order of performance of the corresponding component sound (here, the arrival sound), and represents that the kth performance is performed from the start of the performance of the corresponding karaoke piece. . Therefore, k is a natural number with the total number of constituent sounds as a maximum value.

  First, the time change of the pitches of all the constituent sounds represented by the guide melody (that is, the melody by the guide melody) is set as the guide melody pitch.

  Then, as shown in FIG. 4C, by comparing the guide melody pitch with the fundamental frequency transition f0v (m), it can be considered that the utterance to the constituent sound k is started in the fundamental frequency transition f0v (m). Timing (hereinafter referred to as constituent sound singing start timing) is detected. In addition, as a method for collating the guide melody pitch with the fundamental frequency transition f0v (m), a method described in JP-A-2005-107330 may be used.

  Furthermore, the difference between the detected component sound singing start timing and the musical sound output start time st (k) for the component sound k is calculated as a time delay amount tl (k).

  As described above, a series of flow from the comparison between the guide melody pitch and the fundamental frequency transition f0v (m) to the calculation of the time delay amount tl (k) is performed according to the time progress of the karaoke music (that is, k is 1). Repeat until k is reached, increasing k sequentially. Thereby, all the constituent sound singing start timings are detected, and the time delay amount tl (k) for these all constituent sound singing start timings is obtained.

  In subsequent S160, in the fundamental frequency transition f0v (m), the end of the section for the transition source sound (hereinafter referred to as section end) and the start of the section for the arrival sound (hereinafter referred to as section start) are specified. Then, a singing transition section extraction process is performed to extract a continuous section from the end of the specified section to the start of the section as a singing transition section f0vt (m).

<Singing transition section extraction processing>
Here, the song transition section extraction process will be described in detail.

  FIG. 3 is a flowchart showing a processing procedure of the singing transition section extraction process, and FIGS. 6 and 7 are schematic diagrams showing processing contents of the singing transition section extraction process.

When this singing transition section extraction process is started in S160 of the singing pitch difference specifying process, as shown in FIG. 3, first, in S400, all the constituent sound singing start timings detected earlier are included. A reference time window (hereinafter referred to as a search center time window) m ₀ (k) including one component sound singing start timing is derived according to the following equation (1).

However, in the expression (1), round means rounding to an integer value.

  In S410, one analysis time window #p having a predetermined section length (hereinafter referred to as a window section length) is set for the fundamental frequency transition f0v (m). In the present embodiment, the window section length is 2M + 1, and M is the number of reference time windows m.

Specifically, the analysis time window #p is set so that the center of the window section length (that is, the reference time window m serving as the center) becomes the specified center value. However, the specified center value has an exploration center time window m ₀ (k) as an initial value. That is, when the present singing transition section extraction process is activated and first proceeds to S410, the center of the window section length of the analysis time window #p is the search center time window m ₀ (k).

Here, #p is the index number of the analysis time window, and takes from a negative value to a positive value with the search center time window m ₀ (k) as the origin.

  Further, in S420, a variance value vf0 (#p) for the fundamental frequency transition f0v (m) within the analysis time window #p set in S410 is derived by the following equation (2). However, in the equation (2), mf0 (#p) is an average value of the fundamental frequency transition f0v (m) within the analysis time window #p, and is derived from the following equation (3).

Subsequently, in S430, the difference of the variance value vf0 (#p) with respect to the analysis time windows #p continuous with each other is obtained by the following equation (4). Hereinafter, the value obtained by the equation (4) is referred to as a variance difference dvf0 (#p).

Subsequently, in S440, the variance value difference (hereinafter referred to as the now variance value difference) dvf0 (#p) derived in S430 of the current cycle is stored. Hereinafter, the dispersion value difference dvf0 (#p) derived in S430 of the previous cycle is referred to as a previous dispersion value difference dvf0.

  In S450, it is determined whether or not the absolute value of the current variance value dvf0 (#p) is less than a predetermined allowable value. If the absolute value of the current variance value dvf0 (#p) is less than the allowable value as a result of the determination, that is, if the current variance value difference dvf0 (#p) can be regarded as 0, the process proceeds to S460.

  In S460, it is determined whether or not the absolute value of the previous variance difference dvf0 is greater than or equal to the allowable value. As a result of the determination, if the absolute value of the previous variance value difference dvf0 is greater than or equal to the allowable value, that is, if the previous variance value difference dvf0 cannot be regarded as 0, the process proceeds to S470.

  In S470, the inversion number counter is incremented, and the process proceeds to S480. However, the initial value of the pole change counter is 0.

  In S480, it is determined whether or not the pole change counter is 2 or more. If the pole change counter is less than 2 as a result of the determination, the process proceeds to S490.

  As a result of the determination in S450, when the absolute value of the current variance value difference dvf0 (#p) is greater than or equal to the allowable value, or as a result of the determination in S460, the absolute value of the previous variance value difference dvf0 is less than the allowable value. In some cases, the process proceeds to S490.

  In S490, the specified center value is set by shifting the reference time window m by p (where p ≦ M) from the specified center value set in the current cycle. Thereby, when returning to S410 next time, an analysis time window #p in which the reference time window m is displaced by p pieces is set for the center of the window section length.

  However, in S490 of the present embodiment, when the inversion number counter is 0, the specified center value is displaced along the direction opposite to the time direction in the time progress of the fundamental frequency transition f0v (m). . Further, when the change counter is 1, the specified center value is displaced along the direction of time progression in the time progression of the fundamental frequency transition f0v (m).

  Here, the operation when the processing from S410 to S490 (hereinafter referred to as specific processing) is repeated will be described.

First, as shown in FIG. 6 (A), if the specific process is repeated when the inversion number counter is 0, the time of the fundamental frequency transition f0v (m) is determined with reference to the search center time window m ₀ (k). An analysis time window #p is set along the direction opposite to the progression (to the left in FIG. 6A). At this time, the index number #p of the analysis time window set in S410 is decreased every time the analysis time window #p is set (the value increases in the minus direction).

On the other hand, if the specific process is repeated when the inversion number counter is 1, along the time progress at the fundamental frequency transition f0v (m) with reference to the search center time window m ₀ (k) (FIG. 6A). Analysis right window #p is set. At this time, the index number #p of the analysis time window set in S410 is incremented every time the analysis time window #p is set (the value increases in the plus direction).

  The analysis time window #p set in S410 is set so that at least a plurality of analysis time windows #p adjacent to each other are set by repeating the specific processing because the center of the window section length is displaced by p. Is done.

  Furthermore, when the specific process is repeated, the variance value vf0 (#p) derived in S420 is a section in which the singing pitch transitions from the transition source sound to the arrival sound (hereinafter, referred to as “B”) as shown in FIG. If it is within the transition section), the value itself is large although the change for each analysis time window #p is small. On the other hand, if it is within the section (hereinafter referred to as a steady section) with respect to the singing pitch of the transition source sound and the reaching sound, the value itself is large although the change for each analysis time window #p is small.

  Therefore, when the specific process is repeated, the variance difference dvf0 (#p) derived in S430 is the value if it is within the steady section and the transition section as shown in FIG. 7A. It can be regarded as 0 itself. On the other hand, the value itself cannot be regarded as 0 when switching from the steady section to the transition section or switching from the transition section to the steady section.

  That is, the variance difference dvf0 becomes a maximum value when switching from the steady section to the transition section, and becomes a minimum value when switching from the transition section to the steady section. Therefore, in this embodiment, in S450 and S460, whether or not the absolute value of the current variance value difference dvf0 (#p) is less than the allowable value, and the absolute value of the previous variance value difference dvf0 is greater than or equal to the allowable value. It is determined whether or not there is a variance, and it is determined whether or not the variance difference dvf0 (#p) is an extreme value.

  In other words, when the specific processing is repeated, if the pole number counter becomes 2 or more, two extreme values are detected from the variance difference dvf0 (#p).

Here, returning to FIG. 3, if the result of determination in S480 is that the number of pole change counter is 2 or more, the process proceeds to S500. In S500, a reference time window corresponding to the section end (hereinafter referred to as section end window m _fin (k−1)) and a reference time window corresponding to the section start end (hereinafter referred to as section start end window m _st (k)) ) Is detected.

Specifically, in S500 of the present embodiment, as shown in FIG. 7A, the specified center value corresponding to the analysis time window #p in which the variance difference dvf0 (#p) is a maximum value is set as the section end. The defined center value corresponding to the analysis time window #p in which the variance value difference dvf0 (#p) is the minimum value is set as the section start end. Therefore, the number of reference time windows m between the specified center value at the end of the section and the search center time window m ₀ (k) is pb, and the search center time window m ₀ from the specified center value at the start of the section. Let pf be the number of reference time windows m up to (k). Then, the section end window m _fin (k−1) is represented by m _fin (k−1) = m ₀ (k) −pb, and the section start window mst (k) is m _st (k) = m ₀ (k) + pf.

  As a technique for detecting the maximum value and the minimum value, the maximum or minimum value among all the variance value differences stored in S440 may be set as the maximum value and the minimum value, respectively. .

  Subsequently, in S510, the singing pitch f0 (k-1) for the section with respect to the transition source sound and the singing pitch f0 (k) for the section with respect to the reaching sound are respectively expressed by the following formulas (5) and (6). At the same time, the pitch difference f0d (k) between these singing pitches is derived according to the following equation (7).

That is, the pitch difference f0d (k) is obtained by quantizing the ratio of the singing pitch f0 (k-1) and the singing pitch f0 (k) with a semitone and expressed as a so-called pitch. is there.

In S520, the time from the start of the performance of the corresponding karaoke piece in the section end window m _fin (k-1) and the section start window m _st (k) is derived according to the following equations (8) and (9). .

At the same time, in S520, the period length from the section end window m _fin (k−1) to the section start window m _st (k) is derived as the transition period trt (k) according to the following equation (10).

However, the transition period trt (k) derived by the equation (10) is represented by the number of reference time windows m.

  In S530, the pitch difference f0d (k) derived in S510 and the transition period trt (k) derived in S470 are stored.

  In S540, according to the following equation (11), in the fundamental frequency transition f0v (m), the section corresponding to the transition period trt (k) derived in the previous S520 is extracted as the singing transition section f0vt (m). However, mm in the formula (11) is a reference time window m from 1 to trt (k).

Then, the process returns to S170 of the singing pitch difference identification process.

That is, in the singing transition section extraction process, as shown in FIG. 7 (B), a section ending window is selected from the singing section for the transition source sound, the singing section for the reaching sound, and the singing section sandwiched between these singing sections. _Specify m _fin (k−1) and the section start window m _st (k). Then, the period from the identified section end window m _fin (k−1) to the section start window m _st (k) is derived as a transition period trt (k). At the same time, the section corresponding to the transition period trt (k) is extracted as the singing transition section f0vt (m) in the fundamental frequency transition f0v (m).

  Here, it returns to S170 of a song pitch difference specific process (namely, FIG. 2).

  In S170, the degree of similarity between the singing transition section f0vt (m) extracted by the singing transition section extraction process and the pitch transition model representing the ideal transition mode of the utterance pitch from the transition source sound to the arrival sound is calculated. From the following equations (12) to (15): Hereinafter, the similarity degree derived in S170 is referred to as a pitch transition score tsc (m).

  Here, a specific method for deriving the pitch transition score tsc (m) will be described.

  Note that the pitch transition model in the present embodiment is a time function expressed by the following equation (12) so that the transition of the pitch from the transition source sound to the arrival sound can be heard accurately and smoothly. is there. Hereinafter, the pitch transition model represented by the following equation (12) is referred to as a pitch transition model curve f0model (m).

That is, the pitch transition model curve f0model (m) varies according to the time length of each singing transition section f0vt (m) to be verified. However, M ₀ represents the transition period trt (k).

  Then, as shown in FIG. 5A, the pitch transition score tsc (m) is derived by matching the pitch transition model curve f0model (m) with the singing transition section f0vt (m). The pitch transition model curve f0model (m) and the singing transition section f0vt (m) are collated by the following equation (13).

However, MV _VO in the equation (13) indicates an average value of the singing transition section f0vt (m) and is derived according to the following equation (14). The MV _MO in the equation (13) is a pitch transition model curve f0model. The average value of (m) is shown and derived according to the following equation (15).

That is, the pitch transition score tsc (k) is higher as the degree of similarity between the singing transition section f0vt (m) and the pitch transition model curve f0model (m) is higher (that is, the pitch transition is smoother). ) Large value.

  In subsequent S180, the singing transition section f0vt (m) corresponding to all the time delay amounts tl (k) calculated in S150 is extracted, and collation with the pitch transition model curve f0model (m) is performed. Determine whether. As a result of the determination, if collation has not been performed, the process returns to S160, where the singing transition section f0vt (m) corresponding to all the time delay amounts tl (k) is extracted, and the pitch transition model curve f0model (m S160 and S170 are repeated until collation with () is performed. On the other hand, if it is determined that the determination has been made, the process proceeds to S190.

  In S190, the pitch transition scores tsc (k) derived in S170 are totaled according to the following equation (16). Hereinafter, the result obtained by the aggregation by the equation (16) is defined as an aggregation transition score mtsc (trt, f0d).

  Note that k ′ in the equation (16) is an identification number that represents a constituent sound (hereinafter referred to as the same constituent sound) in which the pitch difference f0d (k) and the transition period trt (k) are included in the same classification category. This is a set (corresponding to the k-th and k + 1-th mentioned above), and K0 is the total number of identical constituent sounds.

In other words, the total transition score mtsc (trt, f0d) is classified into the pitch transition score tsc (m) derived in S170 with the same pitch difference f0d (k) and transition period trt (k). These are tabulated for each section (hereinafter referred to as pitch difference section (trt, f0d)). Therefore, the total transition score distribution obtained by mapping the total transition score mtsc (trt, f0d) is, as shown in FIG. 5B, pitch difference f0d (k), transition period trt (k), total transition score mtsc. This is a three-dimensional matrix with (trt, f0d) as the respective axes. However, in FIG. 5B, the size of the total transition score (trt, f0d) is shown using shades of color.

  In the present embodiment, when the number of times of advance to S180 is the second time or later, mtsc (trt, f0d) derived in that cycle (that is, current S190) is changed to the previous cycle (that is, the previous time). Before S190) is added to the derived mtsc (trt, f0d). In other words, the total transition score mtsc (trt, f0d) derived from each audio data is integrated with the total transition score (trt, f0d) finally generated.

  Further, in S200, it is determined whether or not the processing from S110 to S190 (herein referred to as the regulation processing) has been executed for all the audio data stored in the audio data storage area of the storage unit 21. To do.

  As a result of the determination, if the prescribed process has not been executed for all the audio data, the process returns to S110, and in S110, from among all the audio data stored in the audio data storage area of the storage unit 21 The voice data for which the regulation process has not been executed is acquired, and the process proceeds to S120. On the other hand, if the result of determination in S200 is that definition processing has been executed for all audio data, the process proceeds to S210.

  In S210, whether or not each of the total transition scores mtsc (trt, f0d) in each pitch difference section (trt, f0d) is equal to or greater than a specified value is determined by the equation (17).

That is, if the total transition score mtsc (trt, f0d) is equal to or greater than the specified value, the pitch difference section (trt, f0d) is determined as the singable pitch difference section ITM ₁ (trt, f0d). On the other hand, if the total transition score mtsc (trt, f0d) is less than the specified value, the pitch difference section (trt, f0d) is determined as the unsongable pitch difference section ITM ₀ (trt, f0d).

  Note that the specified value in the present embodiment is a ratio A (for example, half of the maximum value, 1/3, etc.) defined in advance with respect to the maximum value Max_tsc of the total transition score mtsc (trt, f0d) (specified) Value = Max_tsc × A).

Subsequently, in S220, singing possible pitch difference division ITM ₁ (trt, f0d) specified in S210 processing using (hereinafter, a particular pitch difference use processing) executes.

Specifically, in this embodiment, on the basis of singing possible pitch difference was identified in S210 Category ITM ₁ (trt, f0d), and singing impossible pitch difference division ITM ₀ (trt, f0d), the result matrix ITM (trt , F0d), and the generated result matrix ITM (trt, f0d) is displayed on the display unit 23. At the same time, the result matrix ITM (trt, f0d) is stored in the storage unit 21 and the server 30 in association with the corresponding user information.

As a result, as shown in FIG. 8, the matrix ITM (trt, f0d) includes the singable pitch difference section ITM ₁ (trt, f0d) and the unsingable pitch difference section ITM ₀ (trt, f0d). It is arranged along the sequence of the height difference sections (trt, f0d).

Furthermore, in the specific pitch difference use processing of the present embodiment, the music whose pitch difference of the specific component sound is mainly within the range of the singable pitch difference category ITM ₁ (trt, f0d) is recommended to the user. suggest. At the same time, in the specific pitch difference use processing, a weight that becomes larger as the maximum pitch difference f0d of the singable pitch difference section ITM ₁ (trt, f0d) is larger is given to the scoring result by the scoring processing.

  Then, the singing pitch difference identification process is terminated.

That is, in the singing pitch difference specifying process of the present embodiment, each singing transition section f0vt (m) extracted in the singing transition section extracting process is compared with the corresponding pitch transition model curve f0model (m), and A high transition score tsc (k) is derived. Further, it is determined whether or not the total transition score mtsc (trt, f0d) obtained by tabulating the derived pitch transition scores tsc (k) for each pitch difference section (trt, f0d) is equal to or greater than a specified value. Then, according to the determination result, the singable pitch difference section ITM ₁ (trt, f0d), that is, the pitch difference section that the user can sing is specified.
[Effect of the embodiment]
As described above, according to the karaoke apparatus 20 of the present embodiment, it is possible to specify a pitch difference that the user can sing, that is, a pitch that the user can sing.

  In particular, according to the karaoke apparatus 20 of the present embodiment, the total transition score mtsc (trt, f0d) is totaled for each pitch difference f0d (k) and for each transition period trt (k). A possible pitch difference f0d can be specified for each time length of the transition period trt.

  That is, according to the karaoke apparatus 20, a user's singing capability can be specified in detail.

  In addition, in the state which the user uttered and forced, it is difficult to sing smoothly the pitch transition in the transition period trt. However, in such a case, in the karaoke apparatus 20 of this embodiment, the pitch transition score tsc (k) is derived as a small value.

  Therefore, according to the karaoke apparatus 20 of the present embodiment, it is possible to specify the pitch difference f0d that can be sung by the user without overdoing it.

And in the karaoke apparatus 20 of this embodiment, the music whose pitch difference of a specific component sound is mainly in the range of the singable pitch difference division ITM ₁ (trt, f0d) is proposed to the user as a recommended song. In addition, a higher weight is assigned to the scoring result by the scoring process as the maximum pitch difference f0d of the singable pitch difference section ITM ₁ (trt, f0d) is larger.

As a result, according to the karaoke apparatus 20 of the present embodiment, karaoke can be more enjoyed by allowing the user to sing the recommended song, and the user's skill in singing can be more accurately scored. Can do.
[Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  For example, the karaoke system 1 of the above embodiment is one in which one karaoke device 20 and one server 30 are connected, but the karaoke system 1 is not limited to this, and a plurality of karaoke devices 20 and One server 30 may be connected, or a plurality of karaoke apparatuses 20 and a plurality of servers 30 may be connected.

In the singing transition section extraction process of the above embodiment, the reference time window m in which the variance difference dvf0 (#p) with respect to the consecutive analysis time windows is maximized or minimized is defined as the section end m _fin (k−1) and It derived as the section starting m _st (k), but the method for deriving a section termination m _fin (k-1) and the section beginning m _st (k) is not limited thereto. For example, the section which is singing stably about each of the transition source sound and the arrival sound is specified, and the reference time window m corresponding to the end and start of those sections is set to the section end m _fin (k−1) and the section start, respectively. m _st (k) may be used.

In the pitch difference specifying process in the above embodiment, the pitch transition score tsc (k) is tabulated according to the pitch difference f0d (k) and the transition period trt (k), but the pitch transition score tsc (k ) Is not limited to this, and the pitch transition score tsc (k) may be totaled according to only the pitch difference f0d (k).
[Correspondence between Embodiment and Present Invention]
Next, a correspondence relationship between the above embodiment and the claims will be described.

  The function obtained by executing S110 in the pitch difference specifying process of the above embodiment corresponds to the audio signal acquisition means of the present invention, and the function obtained by executing S120 and S130 is the song data generation means. Equivalent to. Furthermore, the function obtained by executing S160 of the pitch difference specifying process corresponds to the transition section specifying means, and the function obtained by executing S170 corresponds to the transition value deriving means. The function obtained by executing S190 and S210 in the pitch difference specifying process corresponds to the pitch difference specifying means.

  Of the functions obtained by executing S160 of the pitch difference specifying process, the functions obtained by executing S410 and S420 in the singing transition section extraction process correspond to the variance value deriving means of the present invention, The function obtained by executing S430, S450 to S480, S500 corresponds to the start / end identification means of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Karaoke system 20 ... Karaoke apparatus 21 ... Memory | storage part 22 ... Communication part 23 ... Display part 24 ... Operation reception part 25 ... Voice input / output part 26 ... Microphone 27 ... Speaker 28 ... Control part 30 ... Server

Claims (5)

A singing pitch difference identifying device used in a karaoke device that plays the music according to music data representing the pitch and value of each of the constituent sounds constituting each music,
An audio signal for acquiring the audio signal in association with identification information for identifying the user who emitted the audio and the music being played, using the audio input during the performance of the music in the karaoke apparatus as an audio signal Acquisition means;
Singing data generating means for generating singing data representing a transition in pitch by analyzing the frequency of the sound signal acquired by the sound signal acquiring means;
Among two constituent sounds that are continuous so that the pitch is switched in the music specified from the identification information, the constituent sound that arrives after the pitch is switched is the arrival sound, and the transition before the pitch is switched In the singing data generated by the singing data generating means, the original component sound is the transition original sound, from the end of the singing section for the transition original sound, at the beginning of the singing section for the reaching sound A transition section specifying means for specifying a singing section up to a certain section start point as a singing transition section;
A transition mode when an utterance pitch is ideally transitioned from a pitch corresponding to the transition source sound to a pitch corresponding to the arrival sound is a pitch transition model, and is specified by the transition section specifying unit For each singing transition section, transition value deriving means for deriving a pitch transition value that becomes a larger value as the degree of coincidence between the singing transition section and the pitch transition model is higher,
The pitch transition values derived by the transition value deriving means are totaled for each pitch difference in each singing transition section, and the total transition value that is the total result is a pitch that is equal to or greater than a predetermined value. A singing pitch difference specifying device for specifying a difference as a pitch difference that can be sung by a user specified by the identification information. The singing pitch difference specifying means is:
2. The singing pitch difference identifying device according to claim 1, wherein the pitch transition values are classified and totaled for each section length of each singing transition section. The singing pitch difference specifying means is:
Normalizing the aggregate transition value,
The singing pitch difference identifying device according to claim 1 or 2, wherein a ratio defined in advance with respect to the normalized maximum value of the totalized transition value is set as the specified value. The transition section specifying means includes
In the singing data, in a switching section consisting of a singing section for the transition source sound, a singing section for the reaching sound, and a singing section sandwiched between the two singing sections, so as to continue with the progress of time A variance value deriving means for deriving a variance value that is a variance of the singing data within the time window for each of the set time windows, and setting a plurality of time windows having a preset section length;
Of the variance values derived by the variance value deriving means, a difference between variance values for successive time windows is derived, and the centers of two time windows where the difference is minimum and maximum are respectively defined as the end of the section and the The singing pitch difference specifying device according to any one of claims 1 to 3, further comprising start / end specifying means for specifying the section start end. A program to be executed by a computer used in a karaoke apparatus that plays the music according to music data representing the pitch and value of each of the constituent sounds that make up each music,
An audio signal for acquiring the audio signal in association with identification information for identifying the user who emitted the audio and the music being played, using the audio input during the performance of the music in the karaoke apparatus as an audio signal Acquisition procedure;
A singing data generation procedure for generating singing data representing a transition in pitch by analyzing the frequency of the audio signal acquired in the audio signal acquisition procedure;
Among two constituent sounds that are continuous so that the pitch is switched in the music specified from the identification information, the constituent sound that arrives after the pitch is switched is the arrival sound, and the transition before the pitch is switched In the singing data generated in the singing data generation procedure, the original component sound is a transition original sound, and from the end of the singing section for the transition original sound, at the beginning of the singing section for the reaching sound A transition section specifying procedure for specifying a singing section up to the beginning of a section as a singing transition section,
A transition mode when an utterance pitch is ideally transitioned from a pitch corresponding to the transition source sound to a pitch corresponding to the arrival sound is defined as a pitch transition model, and specified by the transition section specifying procedure. For each singing transition section, a transition value derivation procedure for deriving a pitch transition value that becomes a larger value as the degree of coincidence between the singing transition section and the pitch transition model is higher,
The pitch transition values derived by the transition value deriving procedure are totaled for each pitch difference in each singing transition section, and the total transition value as a result of the totaling is equal to or higher than a predetermined value. A program that causes the computer to execute a singing pitch difference specifying procedure that specifies a difference as a pitch difference that can be sung by a user specified from the identification information.