JP2016017980A - Voice imitation voice evaluation device, voice imitation voice evaluation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To assume whose voice is imitated only by a voice signal to evaluate its similarity.SOLUTION: In a character database storage unit 16, the voice of an object speaker and a related text characterizing the object speaker are stored as a set. A speaker recognition unit 11 calculates a speaker recognition score representing similarity between input voice imitation voice and voice of the object speaker. A voice recognition unit 12 voice-recognizes the voice imitation voice to generate a recognition results text. A text retrieval unit 13 calculates test similarity representing similarity between the recognition results text and a related text. A similarity aggregation unit 14 calculates a voice imitation score aggregating the speaker recognition score and text similarity. An estimation evaluation unit 15 outputs voice imitation evaluation results based on the voice imitation score.SELECTED DRAWING: Figure 1

Description

  In the present invention, when a human is imitating another person's voice ("imitation" of a voice), the person is imitating only from the audio signal, or how similar it is (similarity) The present invention relates to voice imitation voice evaluation technology.

  In Non-Patent Document 1, in order to clarify what acoustic features are controlled in voice mimicry, an example of comparing voice mimicry voices by professional impersonation talents with the voice of a person (a celebrity) Is described.

Tatsuya Kitamura, “Analysis of Imitation Voice by Imitation Talent”, IEICE Technical Report (SP), vol. 107, no. 282, pp. 49-54, 2007

  The first issue is how to evaluate the similarity between voice imitation and the person's voice. In the case of Non-Patent Document 1, an utterance phrase (text) that characterizes the person is premised, and the similarity is evaluated by imitating the imitation talent. That is, it is possible to analyze that the change pattern of the fundamental frequency is similar by uttering the same phrase as the utterance phrase of the person who is imitating the voice. Except for some professional imitation voice imitations, most imitations are not very similar. In some cases it is not similar at all. Still, humans can determine who is imitating by some means. The method is not limited only to the change pattern of the fundamental frequency, and it is estimated that other acoustic feature quantities are also perceived.

  The second problem is how to correctly estimate who is imitating a voice that is not very similar. In the case of Non-Patent Document 1, it is considered that the utterance phrase characterizing the person was in an important position for evaluating the similarity of the change pattern of the fundamental frequency. Based on this idea, if the utterance phrase is specified, it is possible to narrow down who is imitated, and it is considered that it effectively acts on the evaluation of the similarity in terms of the acoustic feature amount. However, even if there is an example in which voice imitation voice is uttered limitedly by presenting the utterance phrase from the system side, the voice utterance voice that is arbitrarily uttered is recognized and the utterance phrase characterizing the person is extracted from it In the past, there was no example used for evaluation information of voice imitation.

  The third problem is high-speed processing. For example, if the target of imitation is a celebrity, the number of celebrities is extremely large, so it must be estimated who is imitating the voice from among many candidates.

  An object of the present invention is to provide a voice imitation voice evaluation technique for estimating who is imitating from only a voice signal and evaluating the similarity.

  In order to solve the above-described problem, a voice imitation voice evaluation device according to a first aspect of the present invention is a person database storage unit that stores a target speaker's voice and related text characterizing the target speaker in pairs. A speaker recognition unit that calculates a speaker recognition score indicating the similarity between the input voice mimic speech and the target speaker speech, and a speech recognition unit that recognizes the voice mimic speech and generates a recognition result text A text search unit that calculates a text similarity indicating the similarity between the recognition result text and the related text, a similarity summation unit that calculates a voice imitation score obtained by adding the speaker recognition score and the text similarity, and a voice imitation And an estimation evaluation unit that outputs a voice imitation evaluation result based on the score.

  A voice imitation voice evaluation device according to a second aspect of the present invention includes a person database storage unit that stores a target speaker's voice and a related text characterizing the target speaker as a set, and an input voice imitation voice. A speech recognition unit that generates speech recognition result text by speech recognition, a text search unit that calculates a text similarity indicating the similarity between the recognition result text and the related text, and a candidate that identifies a target speaker having a high text similarity A high similarity candidate storage unit for storing information, a speaker recognition unit for calculating a speaker recognition score indicating the similarity between the voice imitation voice and the target speaker voice specified by the candidate information, and a speaker recognition score And an estimation evaluation unit that outputs a voice imitation evaluation result based on.

  The voice imitation voice evaluation technology of the present invention is based on the similarity based on the speaker feature and the similarity between the text that recognizes the voice imitation voice and the text that characterizes the person. To evaluate the voice imitation voice. In this way, even when imitating a voice with an arbitrary utterance, the evaluation is based only on the speaker feature based on the composite evaluation based on both the similarity of speech features and the similarity of text. It is possible to identify a person who imitates a voice or to evaluate the similarity of voice imitation more accurately than in the prior art.

FIG. 1 is a diagram illustrating a functional configuration of the voice mimicking voice evaluation apparatus according to the first embodiment. FIG. 2 is a diagram illustrating a processing flow of the voice imitation voice evaluation method of the first embodiment. FIG. 3 is a diagram illustrating a functional configuration of the voice imitation voice evaluation device of the second embodiment. FIG. 4 is a diagram illustrating a processing flow of the voice imitation voice evaluation method of the second embodiment. FIG. 5 is a diagram illustrating a functional configuration of the voice mimic speech evaluation apparatus according to the third embodiment. FIG. 6 is a diagram illustrating a processing flow of the voice imitation voice evaluation method of the third embodiment.

  Prior to the description of the embodiments, the points of the present invention will be described in correspondence with the above-described problems.

  The point of the invention for the first problem is that a speaker recognition technique is used. The similarity with the voice imitation for each person to be searched is calculated by the speaker recognition technique. Specifically, the score output by the speaker recognition technique is used as the similarity evaluation value.

  When humans hear voice imitations to determine whether they are similar or not, they may feel that they are capturing features even if they are clearly different voices. It is not certain whether the speaker recognition score captures this phenomenon, but speaker recognition is a technology that determines whether or not the voice of the same person is the same. The relevance is high among what is required.

  The point of the invention with respect to the second problem is that a speech recognition technique and a text search technique are used. This is based on the fact that attention is paid to the utterance contents when estimating who the person is. For example, in the case of imitation of a celebrity, it is often a voice imitation of the famous person's famous phrase. If it is imitated by an actor, it is often a voice imitating a voice of the role played by the actor. If you are a singer, it is often a voice imitating a famous song.

  Relevant texts of each person are collected in advance, and voice imitation speech is converted into text by speech recognition. Using the speech recognition result text as a query, a person having similar related text is searched by a text search technique. Thereby, even if the imitation is not substantially similar, it is possible to predict who the imitation is.

  The point of the invention for the third problem is that a speaker recognition method based on similarity calculation on a vector space is introduced. However, this is supported by the combined use of text search technology. Specifically, a speaker recognition method based on i-vector and cosine similarity developed in the speaker recognition technology field in recent years is used. Details of i-vectors are described in “H. Aronowitz and O. Barkan,“ Efficient approximated i-vector extraction ”, Proceedings of ICASSP, pp. 4789-4792, 2012. (Reference 1). For details on speaker recognition methods based on cosine similarity, see “N. Dehak, PJ Kenny, R. Dehak, P. Dumouchel, and P. Ouellet,“ Front-end factor analysis for speaker verification ”, IEEE Transactions on Audio. , Speech and Language Processing, vol. 19, no. 4, pp. 788-798, 2011. (Reference 2).

  If this method is generalized, it can be expressed as “speaker recognition method based on similarity calculation in vector space”. This method has a low calculation cost and is extremely fast. It is also relatively high performance. However, there is not necessarily the ability to detect a speaker appropriately even for a large number of people. As mentioned above, impersonation itself may not be similar. However, as mentioned in the point of the invention with respect to the second problem, by focusing on the similarity of text, it is possible to identify who is imitating voice or to greatly limit candidates. From among the limited candidates, if the imitation is similar, it is possible to identify who imitates by a method based on i-vector and cosine similarity. On the other hand, if it can be identified by text, the remainder may be calculated by a method based on i-vector and cosine similarity and used as an imitation evaluation value.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

[First embodiment]
As shown in FIG. 1, the voice imitation voice evaluation device of the first embodiment includes a voice input unit 10, a speaker recognition unit 11, a voice recognition unit 12, a text search unit 13, a similarity summation unit 14, and an estimation evaluation unit 15. The person database storage unit 16 and the similarity storage unit 17 are included, for example.

  The voice imitation voice evaluation device is configured by loading a special program into a known or dedicated computer having a central processing unit (CPU), a main memory (RAM), and the like, for example. It is a special device. For example, the voice imitation voice evaluation device performs each process under the control of the central processing unit. The data input to the voice imitation voice evaluation device and the data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out as necessary for other processing. Used. In addition, at least a part of each processing unit of the voice mimicking voice evaluation apparatus may be configured by hardware such as an integrated circuit.

  Each storage unit included in the voice imitation voice evaluation device includes, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disc, or a flash memory, Alternatively, it can be configured by middleware such as a relational database or key-value store. Each storage unit included in the voice mimicking voice evaluation device only needs to be logically divided, and may be stored in one physical storage device.

  The person database storage unit 16 stores a person database in which a voice uttered by a person to be imitated (hereinafter referred to as a target speaker) and a related text characterizing the target speaker are paired. .

  With reference to FIG. 2, the voice imitation voice evaluation method of the first embodiment will be described.

  In step S <b> 10, a voice mimic voice is input to the voice input unit 10. The voice imitation voice is a voice uttered by a speaker imitating the target speaker. The voice imitation voice may be input by inputting a voice imitation voice that has been recorded in advance, or by inputting a voice imitation voice that is collected in real time by a microphone or the like. The input voice imitation voice is sent to the speaker recognition unit 11 and the voice recognition unit 12.

  In step S <b> 11, the speaker recognition unit 11 calculates a speaker recognition score indicating the similarity between the input voice imitation voice and the voice of each target speaker stored in the person database storage unit 16. The calculated speaker recognition score is sent to the similarity summation unit 14.

  As a method for calculating the speaker recognition score, a known speaker recognition technique can be used. The first form of speaker recognition score is “Shigeo Morishima et al.,“ New Video Technology “Dive Into the Movie” ”, IEICE Journal, Vol. 94, No. 3, pp. 250-268, 2011.3 This is a speaker recognition method based on a Gaussian Mixture Model (GMM) described in “Moon (Reference 3)”. The second form of the speaker recognition score is a speaker recognition technique based on the dynamic time warping (DTW) described in Reference Document 3 above. The third form of speaker recognition score is “Pavel Matejka, Ondrej Glembek, Fabio Castaldo, Md. Jahangir Alam, Oldrich Plchot, Patrick Kenny, Lukas Burget, Jan Cernocky,“ Full-covariance UBM and heavy-tailed PLDA in i This is a speaker recognition method based on i-vector and Probabilistic Linear Discriminant Analysis described in “-vector speaker verification”, ICASSP 2011, pp. 4828-4831 (reference 4). A fourth form of the speaker recognition score is a method based on a combination of the speaker recognition methods described in Reference Document 3.

  In step S12, the voice recognition unit 12 recognizes the input voice imitation voice and generates a recognition result text. A known speech recognition method may be used as the speech recognition method. The generated recognition result text is sent to the text search unit 13.

  In step S <b> 13, the text search unit 13 calculates a text similarity indicating the similarity between the input recognition result text and the related text stored in the person database storage unit 16. A known text search method may be used as the text search method. The calculated text similarity is sent to the similarity summation unit 14.

  In step S14, the similarity summation unit 14 sums the speaker recognition score and the text similarity according to a predetermined method, and calculates a voice mimic score. The calculated voice imitation score is stored in the similarity storage unit 17.

  Methods for adding the speaker recognition score and the text similarity include addition, multiplication, logarithmic addition, and weighted calculation. It is recommended to artificially determine a weight that is considered optimal in terms of search accuracy and the like through preliminary experiments.

  In step S15, the estimation evaluation unit 15 outputs a voice imitation evaluation result based on the voice imitation score. The voice imitation evaluation result is, for example, the names of a predetermined number of target speakers from a higher voice imitation score when estimating who is imitating. Further, when outputting an index as to whether or not the mimicry is similar, for example, a predetermined number of speaker recognition scores from a higher voice imitation score.

[Second Embodiment]
In the first embodiment, the speaker recognition unit needs to calculate a speaker recognition score for all target speakers in the person database. This method is not suitable when the calculation cost is large and the human database is large. In order to avoid this, the text search unit may be limited to only the top candidate target speakers.

  As shown in FIG. 3, the voice imitation voice evaluation device of the second embodiment includes a voice input unit 10, a voice recognition unit 12, a text search unit 13, an estimation evaluation unit 15, a person database storage unit 16, and a similarity storage unit 17. For example, and further includes a high similarity candidate storage unit 20 and a speaker recognition unit 21.

  With reference to FIG. 4, the voice imitation voice evaluation method of 2nd embodiment is demonstrated. Below, it demonstrates centering on difference with the above-mentioned 1st embodiment.

  In step S <b> 13, the text search unit 13 calculates a text similarity indicating the similarity between the input recognition result text and the related text stored in the person database storage unit 16, and selects the related text having a high text similarity. Candidate information for specifying the target speaker is stored in the high similarity candidate storage unit 20.

  In step S <b> 31, the speaker recognizing unit 31 limits the input voice imitation voice to the target speaker specified by the candidate information stored in the high similarity candidate storage unit 20 and the person database storage unit 16. A speaker recognition score is calculated that indicates the degree of similarity with the speech of each stored target speaker. The calculated speaker recognition score is sent to the estimation evaluation unit 15.

  In step S15, the estimation evaluation unit 15 outputs a voice imitation evaluation result based on the speaker recognition score. When estimating who imitates the voice imitation evaluation result, for example, the person names of a predetermined number of target speakers from the higher speaker recognition score are used. Further, when outputting an index as to whether or not the imitation is similar, for example, a predetermined number of speaker recognition scores from the highest one.

  In the second embodiment, since the speaker recognition score is calculated only for the target speaker candidates limited by the text similarity, the voice imitation evaluation result can be obtained quickly. In addition, the number of candidates can be controlled independently for each of the text similarity and the speaker recognition score. However, there is a demerit that when the correct target speaker cannot be listed as a candidate in the text search unit 13, the target speaker cannot be correctly identified no matter how similar the imitation is.

[Third embodiment]
The third embodiment of the present invention is a voice imitation voice evaluation apparatus and method when a person database is large. In the following, the voice imitation voice evaluation device and method of the third embodiment configured based on the first embodiment will be described. However, the voice imitation voice evaluation device and method of the third embodiment are configured based on the second embodiment. It is also possible.

  As shown in FIG. 5, the voice imitation voice evaluation device according to the third embodiment is similar to the first embodiment in the voice input unit 10, the voice recognition unit 12, the text search unit 13, the similarity summation unit 14, and the estimated evaluation. For example, a speaker feature vector space speaker recognition unit 31 and a person database storage unit 36.

  The person database storage unit 36 stores a person database that is a set of related text that characterizes the target speaker to be imitated and the speaker feature vector of the target speaker.

  With reference to FIG. 6, the voice imitation voice evaluation method of 3rd embodiment is demonstrated. Below, it demonstrates centering on difference with the above-mentioned 1st embodiment.

  In step S31, the speaker feature vector space speaker recognition unit 31 extracts the speaker feature vector extracted from the input voice mimicking speech, and the speaker feature vector of each target speaker stored in the person database storage unit 36. Is calculated as a speaker recognition score. The calculated speaker recognition score is sent to the similarity summation unit 14.

  A specific configuration of the speaker feature vector space speaker recognition unit 31 is a method based on the i-vector and the cosine similarity described in the above-mentioned reference 4. The point of this method is that the speech to be recognized (here, voice mimic speech) and the speaker feature vector of each target speaker's speech are expressed as one vector called i-vector. Then, the degree of similarity is calculated by using the similarity measure of two vectors such as cosine similarity that has a low calculation cost. Thereby, high-speed processing can be realized. According to this method, the speaker feature vector space speaker recognizing unit 31 calculates one speaker feature vector from the voice mimic speech, and one for each target speaker registered in the person database in advance. It is comprised by the process which calculates the similarity (here speaker recognition score) between speaker feature vectors with a predetermined similarity scale.

  One form of the speaker feature vector is, for example, a feature value called i-vector. Details of the i-vector are described in Reference Document 1 above. Another form of the speaker feature vector is a vector of speaker-dependent components extracted using Joint Factor Analysis (JFA). Details on the vector obtained by JFA are described in Reference 2 above.

  Both the i-vector and the vector obtained by JFA are connected by connecting the average vector of each Gaussian distribution (GMM: Gaussian Mixture Model) that has been applied to speech data (super vector). ) Is subjected to matrix decomposition by a predetermined method. Considering this, another form of the speaker feature vector is a vector obtained by matrix decomposition so that speaker components can be extracted by a predetermined method from the GMM super vector. Another form of speaker feature vector is the GMM supervector. The GMM supervector is one of the options in that the speaker component remains, but it also contains a large amount of components other than the speaker. In addition, depending on the setting, the dimension becomes extremely higher than other vectors, and there is a concern about the influence on the search speed. In addition, even a vector obtained by a method that does not use the GMM super vector is a category of speaker feature vectors as long as it is one feature vector that is given to speech that exhibits the effectiveness of speaker identification. .

  Any similarity measure can be used for the similarity between the speaker feature vectors. One representative form is cosine similarity. Another representative form is an inner product value.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

DESCRIPTION OF SYMBOLS 10 Speech input part 11, 21 Speaker recognition part 12 Speech recognition part 13 Text search part 14 Similarity summation part 15 Estimate evaluation part 16, 36 Person database storage part 17 Similarity storage part 20 High similarity candidate storage part 31 Speaker Feature vector space speaker recognition unit

Claims (6)

A person database storage unit for storing a voice of the target speaker and a related text characterizing the target speaker in pairs;
A speaker recognition unit that calculates a speaker recognition score indicating the degree of similarity between the input voice mimic speech and the speech of the target speaker;
A voice recognition unit that recognizes the voice imitation voice and generates a recognition result text;
A text search unit for calculating a text similarity indicating a similarity between the recognition result text and the related text;
A similarity summation unit that calculates a voice imitation score that is the sum of the speaker recognition score and the text similarity;
An estimation evaluation unit that outputs a voice imitation evaluation result based on the voice imitation score;
Voice imitation voice evaluation device including. A person database storage unit for storing a voice of the target speaker and a related text characterizing the target speaker in pairs;
A voice recognition unit that recognizes an input voice imitation voice and generates a recognition result text;
A text search unit for calculating a text similarity indicating a similarity between the recognition result text and the related text;
A high similarity candidate storage unit for storing candidate information for specifying the target speaker having a high text similarity;
A speaker recognition unit that calculates a speaker recognition score indicating a similarity between the voice imitation voice and the voice of the target speaker specified by the candidate information;
An estimation evaluation unit that outputs a voice imitation evaluation result based on the speaker recognition score;
Voice imitation voice evaluation device including. The voice imitation voice evaluation device according to claim 1 or 2,
The person database storage unit stores a speaker feature vector extracted from the speech of the target speaker and a related text characterizing the target speaker in pairs.
The voice recognition speech evaluation apparatus, wherein the speaker recognition unit calculates the speaker recognition score from a speaker feature vector extracted from the voice mimic speech and a speaker feature vector of the target speaker. In the person database storage unit, the voice of the target speaker and the related text characterizing the target speaker are stored in pairs,
A speaker recognition step in which a speaker recognition unit calculates a speaker recognition score indicating a degree of similarity between the input voice mimic speech and the speech of the target speaker;
A voice recognition step in which a voice recognition unit recognizes the voice imitation voice to generate a recognition result text;
A text search step in which a text search unit calculates a text similarity indicating a similarity between the recognition result text and the related text;
A similarity summing unit that calculates a voice imitation score obtained by summing the speaker recognition score and the text similarity;
An estimation evaluation step in which an estimation evaluation unit outputs a voice imitation evaluation result based on the voice imitation score;
Voice imitation voice evaluation method including. In the person database storage unit, the voice of the target speaker and the related text characterizing the target speaker are stored in pairs,
A speech recognition step in which a speech recognition unit recognizes the input voice imitation speech and generates a recognition result text;
A text search step in which a text search unit calculates a text similarity indicating a similarity between the recognition result text and the related text;
A high similarity candidate storage step for storing candidate information for specifying the target speaker having a high text similarity in the high similarity candidate storage unit;
A speaker recognition unit for calculating a speaker recognition score indicating a similarity between the voice imitation voice and the voice of the target speaker specified by the candidate information;
An estimation evaluation step in which an estimation evaluation unit outputs a voice imitation evaluation result based on the speaker recognition score;
Voice imitation voice evaluation method including.   The program for functioning a computer as a voice imitation voice evaluation apparatus in any one of Claim 1 to 3.

Images