JP2021033260A - Training method, speaker identification method, and recording medium - Google Patents

Abstract

To identify a speaker accurately.SOLUTION: Provided is a training method of training a speaker identification model 20 which, when voice data is input, outputs speaker identification information for identifying a speaker of an utterance included in the voice data. The training method includes: performing voice quality conversion of first voice data of a first speaker to generate second voice data of a second speaker; and performing training of the speaker identification model 20 using, as training data, the first voice data and the second voice data.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to techniques for identifying speakers.

Conventionally, a technique for identifying a speaker using a speaker identification model is known (see, for example, Non-Patent Document 1).

David Snyder, Daniel Garcia-Romero, Gregory Sell, Daniel Povey, Sanjeev Khudanpur, “X-VECTORS: ROBUST DNN EMBEDDINGS FOR SPEAKER RECOGNITION” ICASSP 2018: 5329-5333.

I want to identify the speaker accurately.

The learning method according to one aspect of the present disclosure is a learning method of a speaker identification model that outputs speaker identification information for identifying a speaker of a speech included in the voice data when voice data is input. By performing voice quality conversion processing on the first voice data of the speaker, the second voice data of the second speaker is generated, and the first voice data and the second voice data are combined. The speaker identification model is trained as training data.

In the speaker identification method according to one aspect of the present disclosure, voice data is input to the speaker identification model that has been previously trained by the learning method, and the speaker identification model outputs the speaker identification information. ..

The program according to one aspect of the present disclosure executes a process of learning a speaker identification model that outputs speaker identification information for identifying a speaker of a speech included in the voice data when voice data is input to the computer. The first program is to generate the second voice data of the second speaker by performing the voice quality conversion process on the first voice data of the first speaker. And a second step of performing the learning process of the speaker identification model using the first voice data and the second voice data as training data.

It should be noted that these general or specific embodiments may be implemented in a system, method, integrated circuit, computer program or computer-readable recording medium such as a CD-ROM, system, method, integrated circuit, computer. It may be realized by any combination of a program and a recording medium.

According to the learning method and the like according to the present disclosure, the speaker can be identified with high accuracy.

FIG. 1 is a block diagram showing a configuration example of a speaker identification device according to an embodiment. FIG. 2 is a schematic view showing an example of how the voice data holding unit according to the embodiment stores the voice data and the speaker identification information in association with each other. FIG. 3 is a schematic diagram showing how the voice quality conversion unit according to the embodiment converts the voice data of one speaker into the voice data of a plurality of other speakers and outputs the data. FIG. 4 is a block diagram showing a configuration example of the voice quality conversion unit according to the embodiment. FIG. 5 is a flowchart of the speaker identification model learning process according to the embodiment. FIG. 6 is a flowchart of the voice quality conversion model learning process according to the embodiment. FIG. 7 is a flowchart of the speaker identification process according to the embodiment.

(History of obtaining one aspect of the present disclosure)
There is known a speaker identification technique for identifying a speaker by using a speaker identification model in which learning processing is performed in advance using voice data associated with the identification information for identifying the speaker as learning data.

Conventionally, in order to increase the number of learning data (hereinafter, "increasing the number of learning data" is also referred to as "extension of learning data"), noise addition, reverberation, etc. are added to the original learning audio data. Is being done. However, it is not possible to increase the utterance content and language (Japanese, English, etc.) of one speaker by expanding the learning data by adding noise, reverberation, etc. as described above. Therefore, it may not be possible to sufficiently reduce the influence of the utterance content and language in the learning process of the speaker identification model.

Therefore, the inventors have conducted diligent studies and experiments in order to accurately identify the speaker in the speaker identification performed by using the speaker identification model. As a result, the inventors came up with the following learning methods and the like.

The learning method according to one aspect of the present disclosure is a learning method of a speaker identification model that outputs speaker identification information for identifying a speaker of a speech included in the voice data when voice data is input. By performing voice quality conversion processing on the first voice data of the speaker, the second voice data of the second speaker is generated, and the first voice data and the second voice data are combined. The speaker identification model is trained as training data.

According to the above learning method, in the extension of the learning data in the learning process of the speaker identification model, the number of voice data of the second speaker can be increased without being limited by the utterance content and the language. Therefore, the accuracy of speaker identification by the speaker identification model can be improved.

Therefore, according to the above learning method, the speaker can be identified with high accuracy.

Further, the voice quality conversion process may be a process based on the voice data of the first speaker and the voice data of the second speaker.

Further, in the voice quality conversion process, the first speaker is applied to a voice quality conversion model that has been previously trained so as to output the voice data of the second speaker when the voice data of the first speaker is input. By inputting the voice data, the process of outputting the second voice data from the voice quality conversion model may be included.

Further, the voice quality conversion model may include a deep neural network that inputs WAV format audio data and outputs WAV format audio data.

Further, the voice quality conversion process may be a process based on the voice data of the first speaker and the voice data of the third speaker.

Further, the speaker identification model may include a deep neural network that receives an utterance feature amount indicating the utterance feature included in the voice data as an input and outputs a speaker characteristic amount indicating the speaker characteristic.

In the speaker identification method according to one aspect of the present disclosure, voice data is input to the speaker identification model that has been previously trained by the learning method, and the speaker identification model outputs the speaker identification information. ..

According to the speaker identification method, the number of voice data of the second speaker can be increased without being limited by the utterance content and the language in the extension of the learning data in the learning process of the speaker identification model. Therefore, the accuracy of speaker identification by the speaker identification model can be improved.

Therefore, according to the speaker identification method described above, the speaker can be identified with high accuracy.

The program according to one aspect of the present disclosure executes a process of learning a speaker identification model that outputs speaker identification information for identifying a speaker of a speech included in the voice data when voice data is input to the computer. The first program is to generate the second voice data of the second speaker by performing the voice quality conversion process on the first voice data of the first speaker. And a second step of performing the learning process of the speaker identification model using the first voice data and the second voice data as training data.

According to the above program, in the extension of the learning data in the learning process of the speaker identification model, the number of voice data of the second speaker can be increased without being limited by the utterance content and the language. Therefore, the accuracy of speaker identification by the speaker identification model can be improved.

Therefore, according to the above program, the speaker can be identified with high accuracy.

It should be noted that these comprehensive or specific embodiments may be implemented in a system, method, integrated circuit, computer program or computer-readable recording medium such as a CD-ROM, system, method, integrated circuit, computer. It may be realized by any combination of a program and a recording medium.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Each of the embodiments described below is a specific example of the present disclosure. The numerical values, shapes, components, steps, order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present disclosure. Moreover, in all the embodiments, each content can be combined.

(Embodiment)
Hereinafter, the speaker identification device according to the embodiment will be described. This speaker identification device acquires voice data and outputs identification information for identifying the speaker of the utterance included in the voice data.

<Structure>
FIG. 1 is a block diagram showing a configuration example of the speaker identification device 1 according to the embodiment.

As shown in FIG. 1, the speaker identification device 1 includes a voice data expansion unit 10, a speaker identification model 20, a learning unit 30, and an identification target voice data acquisition unit 40.

The voice data expansion unit 10 expands the learning data for performing the learning process of the speaker identification model 20 (that is, increases the number of learning data). The voice data expansion unit 10 may be realized by, for example, a computer including a microprocessor, a memory, a communication interface, and the like. In this case, various functions of the voice data expansion unit 10 are realized by the microprocessor executing a program stored in the memory. Further, the voice data expansion unit 10 may be realized by, for example, distributed computing or cloud computing by a plurality of computers communicating with each other.

As shown in FIG. 1, the voice data expansion unit 10 includes a voice data holding unit 11, a first voice data acquisition unit 12, a voice quality conversion unit 13, a noise reverberation adding unit 14, and a first feature amount calculation unit 15. A comparison unit 16, an audio data storage unit 17, and an extended audio data holding unit 18.

The learning unit 30 performs learning processing of the speaker identification model 20 using the learning data expanded by the voice data expansion unit 10. The learning unit 30 may be realized by, for example, a computer including a microprocessor, a memory, a communication interface, and the like. In this case, various functions of the learning unit 30 are realized by the microprocessor executing a program stored in the memory. Further, the learning unit 30 may be realized by, for example, distributed computing or cloud computing by a plurality of computers communicating with each other.

As shown in FIG. 1, the learning unit 30 includes a second voice data acquisition unit 31, a second feature amount calculation unit 32, and a first learning unit 33.

When the speaker identification model 20 inputs voice data, the speaker identification model 20 outputs speaker identification information for identifying the speaker of the utterance included in the voice data. The speaker identification model 20 may be realized by, for example, a computer including a microprocessor, a memory, a communication interface, and the like. In this case, the various functions of the speaker identification model 20 are realized by the microprocessor executing a program stored in the memory. Further, the speaker identification model 20 may be realized by distributed computing or cloud computing by a plurality of computers communicating with each other, for example.

As shown in FIG. 1, the speaker identification model 20 includes a third feature amount calculation unit 21, a deep neural network (DNN: Deep Neural Network) 22, and a determination unit 23.

The identification target voice data acquisition unit 40 acquires the voice data to be identified in the speaker identification performed by the speaker identification model 20. The identification target voice data acquisition unit 40 may have, for example, a communication interface that communicates with an external device, and may acquire voice data from the external device via the communication interface. Further, the identification target voice data acquisition unit 40 may have, for example, an input / output port (for example, a USB port) and acquire voice data from an external storage device (for example, a USB memory) connected to the input / output port. .. Further, the identification target voice data acquisition unit 40 may, for example, have a microphone and acquire voice data by converting the voice input to the microphone into an electric signal.

Hereinafter, each component constituting the voice data expansion unit 10 will be described.

The voice data holding unit 11 stores the voice data and the speaker identification information associated with the voice data, which identifies the speaker of the utterance included in the voice data, in association with each other.

FIG. 2 is a schematic diagram showing an example of how the voice data holding unit 11 stores the voice data and the speaker identification information in association with each other.

As shown in FIG. 2, the voice data holding unit 11 stores a plurality of voice data associated with a plurality of different speaker identification information. The voice data and the speaker identification information stored in the voice data holding unit 11 are used as learning data for performing the learning process of the speaker identification model 20.

Returning to FIG. 1 again, the description of the speaker identification device 1 will be continued.

Even if the voice data holding unit 11 has a communication interface for communicating with the external device, and stores the voice data acquired from the external device via the communication interface and the speaker identification information associated with the voice data. Good. Further, the voice data holding unit 11 has, for example, an input / output port (for example, a USB port), and is linked to the voice data acquired from an external storage device (for example, a USB memory) connected to the input / output port and the voice data. The attached speaker identification information may be stored.

Here, the audio data will be described as being in WAV format. However, the audio data does not necessarily have to be limited to the WAV format, and may be, for example, an AIFF format, an AAC format, or the like.

The first voice data acquisition unit 12 acquires voice data and speaker identification information associated with the voice data from the voice data holding unit 11.

The voice quality conversion unit 13 uses the voice data acquired by the first voice data acquisition unit 12 as a speaker other than the speaker identified by the speaker identification information associated with the voice data (hereinafter, "another talk"). It is converted into voice data uttered by a person) and output. More specifically, the voice quality conversion unit 13 generates and outputs voice data spoken by another speaker by changing the frequency component of the utterance included in the voice data.

The voice quality conversion unit 13 converts the voice data of one speaker into the voice data of a plurality of other speakers and outputs the voice data so that the speakers are different from each other but the same speech content is obtained. Can be output. Further, when the voice data of one speaker is voice data including utterances in Japanese, the voice quality conversion unit 13 is a voice including utterances in Japanese of another speaker who cannot necessarily speak Japanese. Can be converted to data. That is, the voice quality conversion unit 13 converts the voice data of one speaker into the voice data of a plurality of other speakers and outputs the voice data without being limited by the utterance content and language of the voice data before conversion. Can be done.

FIG. 3 is a schematic diagram showing how the voice quality conversion unit 13 converts the voice data of one speaker into the voice data of a plurality of other speakers and outputs the data.

As shown in FIG. 3, the voice quality conversion unit 13 can increase the number of voice data used as learning data for performing the learning process of the speaker identification model 20 without being limited by the utterance content and the language. it can.

Returning to FIG. 1 again, the description of the speaker identification device 1 will be continued.

The voice quality converter 13 may be realized, for example, by a widely available conventional voice quality converter. Further, the voice quality conversion unit 13 is realized by using, for example, a voice quality conversion model that has been previously trained so as to output the voice data of the second speaker when the voice data of the first speaker is input. May be done. Here, the voice quality conversion unit 13 is realized by using a voice quality conversion model that has been previously trained so as to output the voice data of the second speaker when the voice data of the first speaker is input. It will be explained as.

FIG. 4 is a block diagram showing a configuration example of the voice quality conversion unit 13.

As shown in FIG. 4, the voice quality conversion unit 13 includes a voice quality conversion learning data holding unit 131, a second learning unit 132, and a voice quality conversion model 133.

When the voice data of the first speaker, which is one speaker of the speaker pair, is input for each of the plurality of speaker pairs, the voice quality conversion model 133 is a second speaker, which is the other speaker of the speaker pair. A deep neural network (DNN:) that has been previously trained to output the voice data of the speaker and to output the voice data of the first speaker when the voice data of the second speaker is input. Deep Neural Network). Here, as an example, the voice quality conversion model 133 outputs the WAV format audio data of the second speaker when the WAV format audio data of the first speaker is input for each of the plurality of speaker pairs. As described above, the cycleVAE is preliminarily trained so as to output the WAV format audio data of the first speaker when the WAV format audio data of the second speaker is input. However, the voice quality conversion model 133 outputs the voice data of the second speaker when the voice data of the first speaker is input for each of the plurality of speaker pairs, and the second speaker. If the DNN has been previously trained to output the voice data of the first speaker when the voice data of the above is input, it is not necessarily limited to the above cycleVAE.

The voice quality conversion learning data holding unit 131 stores learning data for performing the learning process of the voice quality conversion model 133. More specifically, the voice quality conversion learning data holding unit 131 stores voice data (here, voice data in WAV format) of each of a plurality of speakers targeted by the voice quality conversion model 133.

The second learning unit 132 uses the learning data stored in the voice quality conversion learning data holding unit 131, and for each of the plurality of speaker pairs, the first speaker who is one speaker of the speaker pair. When the voice data of the second speaker is input, the voice data of the second speaker who is the other speaker of the speaker pair is output, and when the voice data of the second speaker is input, the voice data of the first speaker is output. The learning process of the voice quality conversion model 133 is performed so as to output the voice data of.

Returning to FIG. 1 again, the description of the speaker identification device 1 will be continued.

The noise reverberation unit 14 adds noise (for example, 4 types) and reverberation (for example, 1 type) to each of the voice data output from the voice quality conversion unit 13, and adds noise and the voice data after adding noise. Output the later audio data. As a result, the noise reverberation unit 14 can further increase the number of voice data.

The first feature amount calculation unit 15 is a utterance feature showing the characteristics of the utterance included in the voice data from the voice data output from the voice quality conversion unit 13 and the voice data output from the noise reverberation addition unit 14. Calculate the amount. Here, as an example, the first feature amount calculation unit 15 will be described as calculating MFCC (Mel-Freeency Cepstrum Cofficients) indicating the vocal tract characteristics of the speaker as the utterance feature amount. However, the first feature amount calculation unit 15 is not necessarily limited to the example of calculating the MFCC as long as the utterance feature amount indicating the characteristics of the speaker can be calculated. The first feature amount calculation unit 15 may calculate, for example, the utterance voice signal multiplied by the mel filter bank as the utterance feature amount, or, for example, calculate the spectrogram of the utterance voice signal as the utterance feature amount. May be.

The comparison unit 16 describes the first speaker feature amount and its first speaker feature amount for each of the speaker feature amounts (hereinafter, also referred to as “first speaker feature amount”) output from the first feature amount calculation unit 15. It is compared with the speaker feature amount of the speaker of the utterance (hereinafter, also referred to as "second speaker feature amount") included in the voice data which is the calculation source of the speaker feature amount of 1.

As a result of comparison, the comparison unit 16 determines that (1) the first speaker feature amount and the second speaker feature amount are within a predetermined range when the similarity between the first speaker feature amount and the second speaker feature amount is within a predetermined range. The speaker identification information that identifies the speaker of the utterance included in the voice data is associated with the voice data that is the calculation source. As a result, the comparison unit 16 can increase the number of voice data associated with one speaker identification information. Then, the comparison unit 16 outputs the voice data and the speaker identification information associated with the voice data.

As a result of comparison, the comparison unit 16 calculates (2) the first speaker feature amount when the similarity between the first speaker feature amount and the second speaker feature amount is not within a predetermined range. The original voice data is associated with identification information that identifies a third party that is different from the speaker of the utterance included in the voice data. As a result, the comparison unit 16 can increase the number of speaker identification information associated with the voice data. That is, the comparison unit 16 can increase the number of speakers in the learning data for performing the learning process of the speaker identification model 20. By increasing the number of speakers, it is possible to suppress overfitting in the learning process of the speaker identification model 20, which will be described later. Thereby, the generalization performance of the speaker identification model 20 can be improved. Then, the comparison unit 16 outputs the voice data and the speaker identification information associated with the voice data.

Similar to the voice data holding unit 11, the extended voice data holding unit 18 provides the voice data and the speaker identification information associated with the voice data to identify the speaker of the speech included in the voice data. Store in association with each other.

The voice data storage unit 17 describes the voice data and the speaker identification information associated with the voice data, which are output from the comparison unit 16, with the voice data and the speaker identification information associated with the voice data. Are associated with each other and stored in the extended audio data holding unit 18. Further, the voice data storage unit 17 is associated with the voice data and the voice data for each of the voice data and the speaker identification information associated with the voice data acquired by the first voice data acquisition unit 12. The speaker identification information is associated with each other and stored in the extended voice data holding unit 18. As a result, the extended voice data holding unit 18 stores the voice data output from the comparison unit 16 in addition to the voice data stored as learning data for the voice data holding unit 11 to perform the learning process of the speaker identification model 20. Is also stored as training data for performing the training process of the speaker identification model.

Hereinafter, each component constituting the speaker identification model 20 will be described.

The third feature amount calculation unit 21 calculates the utterance feature amount indicating the utterance feature included in the voice data from the voice data acquired by the identification target voice data acquisition unit 40. Here, as an example, the third feature amount calculation unit 21 will be described as calculating the MFCC indicating the vocal tract characteristics of the speaker as the utterance feature amount. However, the third feature amount calculation unit 21 is not necessarily limited to the example of calculating the MFCC as long as the utterance feature amount indicating the characteristics of the speaker can be calculated. The third feature amount calculation unit 21 may calculate, for example, the utterance voice signal multiplied by the mel filter bank as the utterance feature amount, or, for example, calculate the spectrogram of the utterance voice signal as the utterance feature amount. May be.

When the utterance feature amount calculated by the third feature amount calculation unit 21 is input, the deep neural network 22 shows the speaker characteristic of the utterance included in the voice data from which the utterance feature amount is calculated. It is a deep neural network (DNN) that has been preliminarily trained so as to output an amount. Here, as an example, when the MFCC indicating the vocal tract characteristics of the speaker is input, the deep neural network 22 makes the x-Vector, which is the acoustic feature amount of the utterance that maps the variable-length utterance to the fixed-dimensional embedding, the speaker. It will be described as Kaldi which has been subjected to learning processing in advance so as to be output as a feature amount. However, the deep neural network 22 is a DNN that has been previously trained so as to output a speaker characteristic amount indicating the speaker's characteristics when the speech feature amount calculated by the third feature amount calculation unit 21 is input. If so, it does not necessarily have to be limited to the above Kaldi. Since the details of the calculation method of x-Vector and the like are disclosed in Non-Patent Document 1, the details thereof are omitted here.

The determination unit 23 determines the speaker of the utterance included in the voice data acquired by the identification target voice data acquisition unit 40 based on the speaker characteristic amount output from the deep neural network 22. More specifically, the determination unit 23 stores the x-Vectors of the plurality of speakers, and among the plurality of stored x-Vectors, the x-Vectors most similar to the x-Vectors output from the deep neural network 22. The Vector is specified, and the speaker of the specified x-Vector is determined to be the speaker of the utterance included in the voice data acquired by the identification target voice data acquisition unit 40. Then, the determination unit 23 outputs speaker identification information that identifies the determined speaker.

Hereinafter, each component constituting the learning unit 30 will be described.

The second voice data acquisition unit 31 acquires voice data and speaker identification information associated with the voice data from the extended voice data holding unit 18.

The second feature amount calculation unit 32 calculates the utterance feature amount indicating the utterance feature included in the voice data from the voice data acquired by the second voice data acquisition unit 31. Here, as an example, the second feature amount calculation unit 32 will be described as calculating the MFCC indicating the vocal tract characteristics of the speaker as the utterance feature amount. However, the second feature amount calculation unit 32 is not necessarily limited to the example of calculating the MFCC as long as the utterance feature amount indicating the characteristics of the speaker can be calculated. The second feature amount calculation unit 32 may calculate, for example, the utterance voice signal multiplied by the mel filter bank as the utterance feature amount, or, for example, calculate the spectrogram of the utterance voice signal as the utterance feature amount. May be.

The first learning unit 33 obtains the utterance feature amount calculated by the second feature amount calculation unit 32 and the speaker identification information for identifying the speaker of the utterance included in the voice data from which the utterance feature amount is calculated. When voice data is input as training data, the learning process of the speaker identification model 20 is performed so as to output speaker identification information for identifying the speaker of the utterance included in the voice data.

More specifically, when the first learning unit 33 inputs the MFCC using the MFCC calculated by the second feature amount calculation unit 32 and the speaker identification information corresponding to the MFCC as learning data, the MFCC is calculated. The learning process of the deep neural network 22 is performed so as to output an x-Vector showing the characteristics of the speaker of the speech included in the original voice data.

<Operation>
The speaker identification device 1 having the above configuration performs the speaker identification model learning process, the voice quality conversion model learning process, and the speaker identification process.

Hereinafter, these processes will be described in order with reference to the drawings.

FIG. 5 is a flowchart of the speaker identification model learning process.

The speaker identification model learning process is a process for performing the learning process of the speaker identification model 20.

The speaker identification model learning process is started by, for example, a user using the speaker identification device 1 performing an operation on the speaker identification device 1 to start the speaker identification model learning process.

When the speaker identification model learning process is started, the first voice data acquisition unit 12 receives one voice data and one speaker identification information associated with the one voice data from the voice data holding unit 11. And are acquired (step S100).

When one voice data and one speaker identification information are acquired, the voice data storage unit 17 associates the one voice data with the one speaker identification information with each other and extends the voice data holding unit. It is stored in 18 (step S110).

On the other hand, the voice quality conversion unit 13 selects one speaker from other speakers who are speakers other than the speaker identified by the one speaker identification information (step S120). Then, the voice quality conversion unit 13 converts one voice data into voice data uttered by the one speaker (step S130) and outputs the data.

When the voice data is output from the voice quality conversion unit 13, the noise reverberation unit 14 adds noise and reverberation to the voice data output from the voice quality conversion unit 13 (step S140), and one or more voices. Output data.

When one or more voice data is output from the noise reverberation unit 14, the first feature amount calculation unit 15 has the voice data output from the voice quality conversion unit 13 and one or more voice data output from the noise reverberation unit 14. The utterance feature amount is calculated from each of the voice data (step S150).

When the utterance feature amount is calculated, the comparison unit 16 compares each of the calculated utterance feature amount with the utterance feature amount of the selected one speaker, and compares the calculated utterance feature amount with the one speaker. It is determined whether or not the degree of similarity with the utterance feature amount of is within a predetermined range (step S160).

When a positive determination is made in the process of step S160 (step S160: Yes), the comparison unit 16 identifies one selected speaker in the voice data from which the positively determined utterance feature amount is calculated. The speaker identification information is linked (step S170). Then, the comparison unit 16 outputs the voice data and the speaker identification information associated with the voice data.

When a negative determination is made in the process of step S160 (step S160: No), the comparison unit 16 is different from the selected speaker in the voice data from which the negatively determined utterance feature amount is calculated. The identification information that identifies the third party is associated (step S180). Then, the comparison unit 16 outputs the voice data and the speaker identification information associated with the voice data.

When the processing of step S170 or the processing of step S180 is executed by the comparison unit 16 for all the speech feature quantities to be compared in the processing of step S160, the voice data storage unit 17 outputs from the comparison unit 16. For each of the voice data and the speaker identification information associated with the voice data, the voice data and the speaker identification information are associated with each other and stored in the extended voice data holding unit 18 ( Step S190).

When the process of step S190 is completed, the voice quality conversion unit 13 has one speaker (hereinafter, also referred to as “unselected speaker”) not selected in the process of step S120 among the other speakers. Whether or not it is determined (step S200).

When it is determined in the process of step S200 that there is an unselected speaker (step S200: Yes), the voice quality conversion unit 13 selects one speaker from the unselected speakers (step S210). ), Proceed to the process of step S130.

When it is determined in the process of step S200 that there is no unselected speaker (step S200: No), the first voice data acquisition unit 12 still acquires the voice data stored in the voice data holding unit 11. It is determined whether or not there is unacquired audio data that has not been acquired (step S220).

When it is determined in the process of step S220 that there is unacquired audio data (step S220: Yes), the first audio data acquisition unit 12 acquires one audio data from the unacquired audio data. (Step S230), the process proceeds to step S110.

In the process of step S220, when it is determined that there is no unacquired audio data (step S220: No), the second audio data acquisition unit 31 is changed from the extended audio data holding unit 18 to the extended audio data holding unit 18. For all the voice data to be stored, the voice data and the speaker identification information associated with the voice data are acquired (step S240).

When the voice data and the speaker identification information associated with the voice data are acquired for all the voice data, the second feature amount calculation unit 32 transfers the voice data to the voice data. The utterance feature amount indicating the utterance feature included in the voice data is calculated (step S250).

When the utterance feature amount is calculated for all the utterance feature amounts, the first learning unit 33 includes all the utterance feature amounts in the utterance feature amount and the voice data from which the utterance feature amount is calculated. When voice data is input using the speaker identification information for identifying the speaker of the utterance as learning data, the speaker identification model 20 so as to output the speaker identification information for identifying the speaker of the utterance included in the voice data. (Step S260).

When the process of step S260 is completed, the speaker identification device 1 ends the speaker identification model learning process.

FIG. 6 is a flowchart of the voice quality conversion model learning process.

The voice quality conversion model learning process is a process of performing the learning process of the voice quality conversion model 133.

The voice quality conversion model learning process is started by, for example, a user using the speaker identification device 1 performing an operation on the speaker identification device 1 to start the voice quality conversion model learning process.

When the voice quality conversion model learning process is started, the second learning unit 132 selects one speaker pair from the plurality of speakers targeted by the voice quality conversion model 133 (step S300). Then, the second learning unit 132 uses the learning data for each of the two speakers constituting one of the selected speaker pairs among the learning data held by the voice quality conversion learning data holding unit 131. For one selected speaker pair, when the voice data of the first speaker who is one speaker of the speaker pair is input, the voice data of the second speaker which is the other speaker of the speaker pair is input. And when the voice data of the second speaker is input, the learning process of the voice quality conversion model 133 is performed so as to output the voice data of the first speaker (step S310).

When the second learning unit 132 performs the learning process of the voice quality conversion model 133 for one speaker pair, the unselected speaker pair that has not yet been selected among the plurality of speakers targeted by the voice quality conversion model 133. It is determined whether or not there is (step S320).

When it is determined in the process of step S320 that there is an unacquired speaker pair (step S320: Yes), the second learning unit 132 selects one speaker pair from the unselected speaker pairs. Then (step S330), the process proceeds to step S310.

When it is determined in the process of step S320 that there is no unacquired speaker pair (step S320: No), the speaker identification device 1 ends the voice quality conversion model learning process.

FIG. 7 is a flowchart of the speaker identification process.

The speaker identification process is a process for identifying the speaker of the utterance included in the voice data. More specifically, the speaker identification process is a process in which voice data is input to the speaker identification model 20 that has undergone learning processing in advance, and the speaker identification model 20 outputs speaker identification information.

The speaker identification process is started, for example, when a user who uses the speaker identification device 1 performs an operation on the speaker identification device 1 to start the speaker identification process.

When the speaker identification process is started, the identification target voice data acquisition unit 40 acquires the voice data to be identified (step S400).

When the voice data is acquired, the third feature amount calculation unit 21 calculates the utterance feature amount indicating the utterance feature included in the voice data from the acquired voice data (step S410), and the calculated utterance feature is calculated. The amount is input to the deep neural network 22. Then, the deep neural network 22 outputs a speaker characteristic amount indicating the characteristics of the speaker of the utterance included in the voice data that is the calculation source of the input utterance feature amount (step S420).

When the speaker characteristic amount is output, the determination unit 23 determines the speaker of the utterance included in the voice data acquired by the identification target voice data acquisition unit 40 based on the output speaker characteristic amount. (Step S430). Then, the determination unit 23 outputs the speaker identification information that identifies the determined speaker (step S440).

When the process of step S440 is completed, the speaker identification device 1 ends the speaker identification process.

<Discussion>
As described above, the speaker identification device 1 extends the learning data stored in the voice data holding unit 11 for learning the speaker identification model 20 without being limited by the utterance content and the language. Then, the learning process of the speaker identification model 20 is performed using the expanded learning data. Therefore, according to the speaker identification device 1, it is possible to improve the identification accuracy of the speaker performed by using the speaker identification model 20. Therefore, according to the speaker identification device 1, the speaker can be identified with high accuracy.

(Supplement)
Although the speaker identification device according to the embodiment has been described above, the present disclosure is not limited to this embodiment.

For example, each processing unit included in the speaker identification device according to the above embodiment is typically realized as an LSI which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them.

Further, the integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of circuit cells inside the LSI may be used.

Further, the present disclosure may be realized as a learning method of a speaker identification model executed by the speaker identification device according to the embodiment, or may be realized as a speaker identification method.

Further, in the above-described embodiment, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

Further, the division of the functional block in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be transferred to other functional blocks. You may. Also, the functions of a plurality of functional blocks having similar functions may be processed by a single hardware or software in parallel or in a time division manner.

Further, the order in which each step in the flowchart is executed is for exemplifying the present disclosure in detail, and may be an order other than the above. Further, a part of the above steps may be executed at the same time (parallel) as other steps.

The speaker recognition device according to one or more aspects has been described above based on the embodiment, but the present disclosure is not limited to this embodiment. As long as the gist of the present disclosure is not deviated, various modifications that can be conceived by those skilled in the art are applied to the present embodiment, and forms constructed by combining components in various modifications and the like are also within the scope of one or more embodiments. May be included within.

The present disclosure can be widely used as a device for identifying a speaker and the like.

1 Speaker identification device 10 Voice data expansion unit 11 Voice data retention unit 12 First audio data acquisition unit 13 Voice quality conversion unit 14 Noise reverberation addition unit 15 First feature amount calculation unit 16 Comparison unit 17 Voice data retention unit 18 Extended voice data Holding unit 20 Speaker identification model 21 Third feature amount calculation unit 22 Deep neural network 23 Judgment unit 30 Learning unit 31 Second audio data acquisition unit 32 Second feature amount calculation unit 33 First learning unit 40 Identification target audio data acquisition unit 131 Voice conversion learning data holding unit 132 Second learning unit 133 Voice conversion model

Claims (8)

It is a learning method of a speaker identification model that outputs speaker identification information that identifies a speaker of an utterance included in the voice data when voice data is input.
By performing voice quality conversion processing on the first voice data of the first speaker, the second voice data of the second speaker is generated.
The speaker identification model is trained using the first voice data and the second voice data as learning data.
Learning method. The voice quality conversion process is a process based on the voice data of the first speaker and the voice data of the second speaker.
The learning method according to claim 1. In the voice quality conversion process, the first voice data is applied to a voice quality conversion model that has been previously trained so as to output the voice data of the second speaker when the voice data of the first speaker is input. Including the process of outputting the second voice data from the voice quality conversion model by inputting.
The learning method according to claim 2. The voice quality conversion model includes a deep neural network that inputs WAV format audio data and outputs WAV format audio data.
The learning method according to claim 3. The voice quality conversion process is a process based on the voice data of the first speaker and the voice data of the third speaker.
The learning method according to claim 1. The speaker identification model includes a deep neural network that inputs an utterance feature amount that indicates the utterance feature included in the voice data and outputs a speaker characteristic amount that indicates the speaker characteristic.
The learning method according to claim 1. Voice data is input to the speaker identification model that has been previously trained by the learning method according to claim 1, and the speaker identification model is made to output the speaker identification information.
Speaker identification method. It is a program for executing a process of learning a speaker identification model that outputs speaker identification information for identifying a speaker of an utterance included in the voice data when voice data is input to the computer.
The above processing
The first step of generating the second voice data of the second speaker by performing the voice quality conversion processing on the first voice data of the first speaker, and
A second step of performing a learning process of the speaker identification model using the first voice data and the second voice data as learning data is included.
program.

Images