JP2019109278A - Speech synthesis system, statistic model generation device, speech synthesis device, and speech synthesis method - Google Patents

Abstract

To synthesize a speech with a reading (pronunciation) that a user specified by an ETE (End-To-End) type speech synthesis system.SOLUTION: A speech synthesis system synthesizes a speech by: generating a statistic model to be used for speech synthesis on the basis of learning data in which an utterance text and speech data are made to correspond to each other; generating, as learning data, data in which an utterance text having some words of the utterance text replaced with a pronunciation symbol array and the speech data are made to correspond to each other; storing a user dictionary as data including information in which words and pronunciation symbol arrays for the words are made to correspond to each other; replacing words, included in the user dictionary among words included in an object text as an object of speech synthesis, with the pronunciation symbol arrays made to correspond to words in the user dictionary; and performing speech synthesis processing, based upon the statistic model, on the object text after the replacement.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a speech synthesis system, a statistical model generation device, a speech synthesis device, and a speech synthesis method.

In recent years, speech synthesis technology has been introduced in various fields such as automatic answering of telephones, announcements by public transport organizations and local governments, and reading of information by applications of smartphones and personal computers. Also, in recent years, technologies such as speech recognition, machine translation, dialogue generation, etc. have dramatically improved, and their practical application to speech translation, service robots, etc. are rapidly advanced.

Dioreman, Sander; Zen, Heiga; Simonyan, Karen; Vinyals, Oriol; Graves, Alex; Kalchbrenner, Nal; Senior, Andrew; Kavukcuoglu, Koray (2016-09-12). "WaveNet: A Generative Model for Raw Audio ". Yucoon Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, et al. synthesis model. "Unit selection with Hierarchical Cascaded Long Short Term Memory Bidirectional Recurrent Neural Nets", Vincent Pollet, Enrico Zovato, Sufian Irhimeh, Pier Batzu, Interspeech 2017.

  There are various methods for speech synthesis, and among them, text-to-speech synthesis is attracting attention as one of the important technologies. A typical text-to-speech synthesis method includes front-end processing for converting text into phonetic symbol strings, and back-end processing for generating speech waveforms from intermediate languages. In recent years, those in which a statistical method such as DNN (Deep Neural Network) is applied to each of front-end processing and back-end processing are also put to practical use.

  Recently, a so-called ETE (End-To-End) type speech synthesis system has also appeared, which directly generates speech from input text without passing through an intermediate language. In the ETE speech synthesis method, the relationship between the language feature of the speech text of the speech corpus and the sound feature of the speech waveform is prepared in advance as a statistical model by a statistical method such as DNN, and the speech is synthesized during speech synthesis. A voice is synthesized by generating a sequence having an acoustic feature corresponding to a text to be subjected to speech synthesis based on a model.

  By the way, in the field where speech synthesis technology is used, there are not a few needs to perform speech synthesis by reading (pronunciation) specified by the user for personal names, proper nouns, etc., for example. There is a great need for dictionary functionality. However, the statistical model in the ETE speech synthesis system is configured as a whole large model (such as the DNN model), and a large amount of computers are required for relearning the ETE speech synthesis system using a large amount of data. Resources are required, and it is not always practical to re-learn a statistical model each time you add the contents of a user dictionary to training data or add new words.

  The present invention has been made in view of the above background, and it is an object of the present invention to provide an ETE-type speech synthesis system etc. capable of performing speech synthesis in reading (pronunciation) specified by a user. Do.

  One of the present inventions for solving the above problems is a speech synthesis system configured by using an information processing apparatus, which performs speech synthesis based on learning data in which an utterance text and speech data are associated with each other. A model learning unit that generates a statistical model to be used; a text substitution unit that generates, as the learning data, data in which speech data is associated with speech text in which all or part of words of the speech text are replaced with phonetic symbol strings; A storage unit for storing a user dictionary which is data including information in which a word is associated with a phonetic symbol string for the word, and a target text which is a text to be subjected to speech synthesis in the target text; A user who replaces a word included in the user dictionary among the words with the phonetic symbol string for the word in the user dictionary Comprising a write application unit, and a speech synthesis unit for generating synthetic speech by performing speech synthesis processing based on the statistical model for the target text after the replacement.

  According to the present invention, it is possible to provide an ETE-type speech synthesis system or the like that can perform speech synthesis in reading (pronunciation) specified by a user.

FIG. 1 is a diagram showing a schematic configuration of a speech synthesis system. FIG. 1 is a block diagram of an information processing apparatus shown as an example of hardware used to realize a speech synthesis system. It is a figure explaining the composition of a speech synthesis system. It is a figure explaining the detail of a text substitution part. It is a figure explaining the detail of a substitution word extraction part. It is a figure explaining the detail of an audio | voice feature-value extraction part. It is a figure explaining the detail of a user dictionary application part.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

  FIG. 1 shows a schematic configuration of a system (hereinafter referred to as a speech synthesis system 1) for performing speech synthesis by an ETE (End-To-End) type speech synthesis method described in the present embodiment.

  The speech synthesis system 1 learns the relationship between the language feature of the speech text 51 of the speech corpus 50 and the sound feature of the speech data 52 by learning statistical techniques (such as machine learning) such as DNN (Deep Neural Network). A statistical model 60 is generated in advance, and a series having an acoustic feature corresponding to text (text or phrase) (hereinafter referred to as input text 700 (target text)) to be subjected to speech synthesis is a statistical model 60 To generate speech based on.

  In the present embodiment, DNN will be described as an example of the above-described statistical method, but the statistical method is not necessarily limited. For example, other statistical methods such as Hidden Markov Model (HMM) may be used. You may use. In the following, the case where the language described as text is Japanese will be described as an example, but the language described as text may be another language, and a plurality of languages are mixed in the text. It is also good.

  As shown in the figure, the speech synthesis system 1 generates a speech corpus 50, a statistical model generation unit 100 that generates a statistical model 60 based on the speech corpus 50, and a sequence having acoustic features corresponding to the input text 700. And a speech synthesis unit 200 which synthesizes speech waveforms based on the generated sequence and generates (outputs) a synthesized speech 800.

  The speech corpus 50 includes a speech text 51 and speech data 52 (speech waveform data, coded speech data, etc.) associated with the speech text 51. The speech corpus 50 is used as learning data when the statistical model generation unit 100 generates the statistical model 60.

  The speech synthesis unit 200 uses the statistical model 60 to specify a specified pronunciation or utterance style for the input text 700 (for example, speech characteristics such as speech tone, emotional stagnation, etc., voice intonation, size). The speech is synthesized according to the vocal feature characterized by the elements such as rhythm, speed, and length between).

  FIG. 2 is a block diagram of the information processing apparatus 10 (computer, computer resource) shown as an example of hardware used to realize the speech synthesis system 1. As shown in the figure, the information processing device 10 includes a processor 11, a main storage device 12, an auxiliary storage device 13, an input device 14, an output device 15, and a communication device 16. These are communicably connected to each other via communication means such as a bus (not shown).

  In the information processing apparatus 10, the entire configuration does not necessarily have to be realized by hardware. For example, part or all of the configuration is realized by a virtual resource such as a cloud server of a cloud system. May be

  The processor 11 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), a field programmable gate array (FPGA) or the like. Various functions of the speech synthesis system 1 are realized by the processor 11 reading and executing a program stored in the main storage device 12.

  The main storage device 12 is a device for storing programs and data, and is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile semiconductor memory (NVRAM (Non Volatile RAM)), or the like.

  The auxiliary storage device 13 may be, for example, a hard disk drive, an SSD (Solid State Drive), an optical storage device (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), a storage system, an IC card, an SD memory card, an FD (FD) This is a reading / writing device of a recording medium such as a flexible disk), a storage area of a cloud server, and the like. The programs and data stored in the auxiliary storage device 13 are read into the main storage device 12 as needed. Data managed by the speech synthesis system 1 such as the speech corpus 50 is managed, for example, in a database of a DBMS (DataBase Management System) that uses the auxiliary storage device 13 as a data storage area.

  The input device 14 is an interface (or a user interface) for inputting the voice corpus 50 and the input text 700, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a microphone, an amplifier or the like. The information processing apparatus 10 may be configured to receive input of information with another apparatus via the communication apparatus 16.

  The output device 15 is an interface (or a user interface) for outputting various information, and includes an audio output device (speaker, amplifier, etc.) for outputting synthetic speech. Note that the information processing apparatus 10 is an interface (for example, an audio output device (speaker etc.), a screen display device (liquid crystal monitor, LCD (Liquid Crystal Display), graphic card) that provides the user with various information such as process progress and process result And the like), a printing device and the like) may be further provided as the output device 15. Further, the information processing apparatus 10 may output information with another apparatus via the communication apparatus 16.

  The communication device 16 is a wired or wireless communication interface for realizing communication with another device via a communication means such as a LAN or the Internet, and, for example, a NIC (Network Interface Card), various wireless communication modules , USB (Universal Serial Interface) module, serial communication module, modem or the like.

  The statistical model generation unit 100, the speech synthesis unit 200, the speech corpus 50, and the statistical model 60 do not have to be all realized by common hardware, and a plurality of hardwares connected communicably It may be distributed and arranged. For example, the speech corpus 50 and the statistical model generation unit 100, and the statistical model 60 and the speech synthesis unit 200 may be configured by independent resources. The statistical model 60 and the voice synthesis unit 200 are incorporated into a device such as a car navigation device, a smartphone, a mobile phone, a personal computer, etc., for example.

  The components of the statistical model generation unit 100, which will be described later, may not necessarily be realized by common hardware, and may be distributed and disposed in a plurality of hardware connected communicably. The components of the voice synthesis unit 200 described later do not have to be necessarily realized by common hardware, and may be distributed and arranged in a plurality of hardware connected communicably.

  In addition, the voice corpus 50 and the statistical model 60 are arranged in resources on the communication network such as a cloud server, and the statistical model generation unit 100 and the voice synthesis unit 200 make the voice corpus 50 and the statistical model 60 through the wired or wireless communication network. You may make it access.

  Further, the statistical model generation unit 100 and the speech synthesis unit 200 are arranged in independent hardware, and the statistical model 60 generated by the statistical model generation unit 100 is used as a physical recording medium (optical storage device (CD (Compact Disc (Compact Disc Or the like), a hard disk drive, an SSD, an IC card, an SD memory card, etc.), or may be provided to the voice synthesis unit 200 via a wired or wireless communication network.

  FIG. 3 is a diagram for explaining the configuration of the speech synthesis system 1 shown in FIG. 1 in detail. As shown in the figure, the statistical model generation unit 100 has the functions of a text substitution unit 110 and a model learning unit 120.

  The text replacement unit 110 replaces a part or all of the utterance text 51 of the speech corpus 50 with a phonetic symbol string in text data format. More specifically, the text replacement unit 110 replaces part or all of the utterance text 51 with a phonetic symbol string in text data format.

  FIG. 4 is a diagram for explaining the details of the text replacement unit 110 of FIG. As shown in the figure, the text replacement unit 110 includes a replacement word extraction unit 111, an audio feature extraction unit 112, a phonetic symbol string generation unit 113, and a text replacement processing unit 115.

  The replacement word extraction unit 111 extracts, from the utterance text 51 of the speech corpus 50, a word (for example, proper noun or the like) to be a target of replacement with a phonetic symbol string. In the following description, the word "word" includes all aspects that can be registered in the user dictionary 212 described later, such as one character, one word, multiple word clusters, phrases, clauses, and sentences.

  The reason why the words to be replaced are extracted from the utterance text 51 in this way is that if all the words in all the utterance texts 51 of the speech corpus 50 are replaced with phonetic symbol strings, the speech synthesis unit 200 This is because speech synthesis based on text without phonetic symbol string (hereinafter referred to as normal text) can not be performed during synthesis. Therefore, for example, when the object is to realize the function of the user dictionary, the replacement word extraction unit 111 extracts, for example, a word that may be registered in the user dictionary as a word to be replaced.

  FIG. 5 is a diagram for explaining the details of the replacement word extraction unit 111 shown in FIG. As shown in the figure, the replacement word extraction unit 111 has functions of a morphological analysis unit 1111 and a word extraction unit 1112.

  The replacement word extraction unit 111 also stores word extraction data 106. The word extraction data 106 includes information serving as a reference for determining whether the word extraction unit 1112 extracts a word to be replaced with a phonetic symbol string. The content of the word extraction data 106 is set by the user, for example. The replacement word extraction unit 111 may automatically extract words using, for example, a statistical method (such as machine learning).

  The morphological analysis unit 1111 performs morphological analysis to divide the utterance text 51 into the smallest unit of language (morpheme in the case of Japanese). For example, when the utterance text 51 is "foreign suffrage", the morphological analysis unit 1111 divides this into four words "foreign", "person", "regime", and "right".

  The word extraction unit 1112 extracts, from the words obtained by being divided by the morphological analysis unit 2101, a word to be replaced with the phonetic symbol string. Here, the above extraction method is not necessarily limited, but can be performed using, for example, a rule-based method or a statistic-based method. For example, when constructing the speech synthesis system 1 for railway broadcast, assuming that a station name is registered in a user dictionary 212 described later, the replacement word extraction unit 111 extracts the station name from the utterance text 51. Specifically, for example, the input text 700 is "Shinjuku-san got off the train in Shinjuku.", And the word extraction data 106 is "notation = Shinjuku, Yomi = Shinjuku, attribute = station name". In the case, the word extraction unit 2102 extracts only words in which “Shinjuku” is used as the “station name”. However, when only station names are replaced in this way, bias may occur in the accent type. Therefore, for example, the replacement word extraction unit 111 can balance the phonemes of the words (the phoneme connection of the selected words is biased so that the words that may be registered in the user dictionary can be read out with high quality as a whole. Select words appropriately from the utterance text 51 of the entire speech corpus 50 in consideration of the absence of) and prosody balance (accent type, position in the sentence, etc. not biased) (extracting words other than the station name if necessary) You may make it a target.

  FIG. 6 is a diagram for explaining the details of the audio feature quantity extraction unit 112 shown in FIG. The speech feature quantity extraction unit 112 extracts, for example, pronunciation (phoneme), speech style, prosody etc. as speech feature quantity from the speech data 52 of the speech corpus 50.

  As shown in the figure, the voice feature extraction unit 112 includes a pronunciation (phoneme) extraction unit 1121, a speech style extraction unit 1122, and a prosody feature extraction unit 1123. The pronunciation (phoneme) extraction unit 1121 extracts pronunciation (phoneme) information from the speech data 52 of the speech corpus 50. The speech style extraction unit 1122 extracts speech style information from the speech data 52 of the speech corpus 50. The prosody feature extraction unit 1123 extracts prosody feature information from the voice data 52 of the voice corpus 50. As described above, it is possible to tune the speech style and intonation of synthetic speech by extracting information on not only the pronunciation (phoneme) but also the speech style and prosody into the phonetic symbol string. The voice feature quantity extraction unit 112 may extract further information as the voice feature quantity 104. The processing performed by the voice feature extraction unit 112 can be automatically performed by the information processing apparatus using, for example, a voice recognition technology or a text analysis technology. Note that part or all of the above processing may be performed manually.

  The pronunciation (phoneme) information can be extracted, for example, from the utterance text 51 by language processing technology. For example, whether the word "Tomorrow" is pronounced by "Ashita" or "Asu", etc. In the case of language processing technology, it may not be possible to uniquely determine the pronunciation, but in that case, the correct pronunciation may be extracted based on the voice data 52 using, for example, a speech recognition technology. For example, when the utterance text 51 corresponding to a certain voice data 52 is “Tomorrow is fine.”, The voice feature extraction unit 112 applies a voice recognition technology to the voice data 52 to produce a pronunciation “Ashwa haredes” (phoneme ) Extract information.

  Returning to FIG. 4, the phonetic symbol string generation unit 113 generates the phonetic symbol string 105 based on the voice feature amount 104 extracted by the voice feature amount extraction unit 112 (represents the voice feature amount 104 by a phonetic symbol string). Although the method of describing the phonetic symbol string 105 is not necessarily limited, for example, in the case of Japanese, the phonetic symbol string generation unit 113 uses the JIETA (general corporate entity electronic corporation) based on the speech feature quantity 104 extracted by the speech feature quantity extraction unit 112. Generates Japanese text-to-speech synthesis symbols (JEITA IT-4006) defined by the Information Technology Industries Association.

  The accuracy of reading (pronunciation) designation can be enhanced by using a code (symbol code, special code, etc.) not included in the character code used for general text. For example, the phonetic symbol string generation unit 113 does not include the character code used in the general text “as 'wa Halle' d%.” As the pronunciation “Aswahhaledes” extracted by the speech feature quantity extraction unit 112 Convert to phonetic symbols containing. In addition to the pronunciation (phoneme) information, information such as accent information, prosody boundary information, and pose information may be further added to enhance the quality of the reading designation.

  The text substitution processing unit 115 is a phonetic symbol string generated by the phonetic symbol string generation unit 113 corresponding to the word extracted by the replacement word extraction unit 111 (the extracted word 103 in the figure) for the utterance text 51. A replaced utterance text (hereinafter referred to as utterance text 102 including a phonetic symbol string) is generated.

  Returning to FIG. 3, the text replacement processing unit 115 adds the utterance text 102 generated by the text replacement unit 110 including the phonetic symbol string to the speech corpus 50 as new learning data in combination with the corresponding speech data 52. Do. Thus, learning data is added to the speech corpus 50 each time the text replacement unit 110 generates the utterance text 102 including the phonetic symbol string.

  As a specific example, for example, the utterance text 51 is "the next stop station is Shinjuku", and the replacement word extraction unit 111 extracts the word "Shinjuku" (extracted word 103) from the utterance text 51. In this case, the text substitution processing unit 115 generates the utterance text 102 including the phonetic symbol string that “the next stop station is a syn-ck”. The text substitution processing unit 115 adds the generated utterance text 102 including the phonetic symbol string to the speech corpus 50 in combination with the corresponding speech data 52.

  The text substitution processing unit 115 newly adds the combination of the utterance text 102 including the phonetic symbol string generated by the text substitution unit 110 and the corresponding speech data 52 to the speech corpus 50 as described above. Alternatively, the utterance text 51 in the combination of the utterance text 51 and the speech data 52 in the speech corpus 50 before the substitution to the pronunciation symbol string may be replaced with the utterance text 102 including the pronunciation symbol string ( That is, the existing combination of speech corpus 50 is updated).

  The model learning unit 120 shown in FIG. 3 learns combinations of all speech texts 51 and speech data 52 (including combinations newly added by the text substitution processing unit 115) included in the speech corpus 50. As learning (machine learning etc.) to generate a statistical model 60. Since the learning data includes the utterance text 102 including the phonetic symbol string as described above, using the generated statistical model 60, speech synthesis is performed for not only normal text but also text including the phonetic symbol string. It can be performed.

  Subsequently, the speech synthesis unit 200 shown in FIG. 3 will be described. As shown in the figure, the speech synthesis unit 200 has functions of a user dictionary application unit 210 and a speech synthesis processing unit 220.

  When the user dictionary application unit 210 detects a word registered in the user dictionary in the input text 700 during speech synthesis based on the input text 700, the user pronounces the detected word for the word specified in the user dictionary 212. The text string is substituted to generate text 203 including a phonetic string. Although the function of the user dictionary application unit 210 can be realized by a simple string substitution algorithm, the input text 700 is described in a language such as Japanese in which there is no clear divided character between words. If so, the simple string substitution algorithm may not be able to do the above substitution correctly. For example, when the user dictionary 212 contains data "notation = ginseng, reading (phonetic symbol string) = ninjang", "foreign suffrage" is replaced with "foreign ninjing administration" in the simple string substitution algorithm. There are times when you are Therefore, the user dictionary application unit 210 of this embodiment generates the text 203 including the phonetic symbol string as follows.

  FIG. 7 is a diagram for explaining the details of the user dictionary application unit 210. As shown in FIG. As shown in the figure, the user dictionary application unit 210 includes a morphological analysis unit 2101, a word extraction unit 2102, and a phonetic symbol string replacement unit 2103.

  The user dictionary application unit 210 also stores word extraction data 211 and a user dictionary 212. The word extraction data 211 includes information serving as a reference for determining whether or not the word extraction unit 2102 extracts a word to be replaced. The user dictionary 212 includes information in which words (indications) and readings (phonetic symbol strings) are associated.

  The morphological analysis unit 2101 performs morphological analysis to divide the input text 700 into language minimum units (morphemes in the case of Japanese). For example, when “foreign suffrage” is input as the input text 700, the morphological analysis unit 2101 divides this into four words of “foreign”, “person”, “registrar”, and “right”. Therefore, for example, even when the content such as “notation = ginseng, reading (phonetic symbol string) = ending” is registered in the user dictionary 212, there is no possibility of erroneous replacement.

  The word extraction unit 2102 extracts, from the words divided by the morphological analysis unit 2101, words (for example, proper nouns) to be replaced with the phonetic symbol string. Although the method of this extraction is not necessarily limited, it is performed using, for example, a rule-based method or a statistic-based method. The same as the word extraction unit 1112 described above, but, for example, the input text 700 is "Shinjuku san got off the train in Shinjuku." If the attribute “station name” is registered, the word extraction unit 2102 extracts words in which “Shinjuku” is used as the “station name”.

  The phonetic symbol string replacement unit 2103 replaces the word registered in the user dictionary 212 among the words extracted by the word extraction unit 2102 with the reading (phonetic symbol string) registered in the user dictionary 212 for the word. In the above example, for example, when “indication = Shinjuku, Yomi (phonetic symbol string) = Shinjuku” is registered in the user dictionary 212, the phonetic symbol string replacement unit 2103 is “Shinjuku-san got off the train in Shinjuku. The input text 700 is replaced with the text "Shinjuku-san got off the train at Shinjuku."

  The morphological analysis unit 2101 and the word extraction unit 2102 use the same one (the algorithm is common) with the morphological analysis unit 1111 and the word extraction unit 1112 in the replacement word extraction unit 111 of the statistical model generation unit 100 described above. Is preferred. Further, as the phonetic symbol string replacement unit 2103, it is preferable to use the same one (a common algorithm) as the text replacement processing unit 115 of the text replacement unit 110 described above. As such, by using a common algorithm in the statistical model generation unit 100 and the speech synthesis unit 200, it is possible to improve the quality of synthesized speech.

  Returning to FIG. 3, the speech synthesis processing unit 220 generates synthetic speech 800 from the input text 700 using the statistical model 60. For example, as in Non-Patent Document 1, the voice synthesis processing unit 220 directly generates a voice waveform, as in Non-Patent Document 2, generates voice parameters for each frame and then generates voice, Non-Patent Document 3 The synthesized speech 800 is generated by a method of synthesizing speech by connecting speech segments selected by DNN such as the above.

  Note that, for example, when the speech corpus 50 does not include the utterance text 51 corresponding to the input text 700, the statistical model 60 is generated by analogy with the model learning unit 120 statistical method (such as machine learning). Good. For example, in the case where the speech corpus 50 does not include “singjuk” as the utterance text 102 including the phonetic symbol string, the model learning unit 120 analogizes “singjuk” and the speech data 52 by the statistical method. To generate a statistical model 60 including the correspondence of

  As described above in detail, the speech synthesis system 1 according to the present embodiment includes learning data in which the speech text 51 in which a part of the words of the speech text 51 is replaced with the phonetic symbol string and the speech data 52 are associated. The statistical model 60 is learned in advance by the speech corpus 50, and for the input text 700, a word included in the user dictionary 212 among the words included in the input text 700 is associated with the word in the user dictionary 212. The synthesized speech is generated by performing speech synthesis processing based on the statistical model 60 for the input text 700 after replacement with the phonetic symbol string being replaced. Therefore, it is possible to perform speech synthesis in reading (pronunciation) specified by the user without relearning the statistical model 60. As described above, according to the present embodiment, it is possible to realize a practical speech synthesis system based on an ETE speech synthesis method capable of speech synthesis in reading (pronunciation) specified by a user.

  As mentioned above, although this invention was concretely demonstrated based on embodiment, this invention is not limited to said embodiment, It can change variously in the range which does not deviate from the summary. For example, the above embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations. In addition, with respect to a part of the configuration of the above embodiment, other configurations can be added, deleted, and replaced.

  In addition, each of the configurations, function units, processing units, processing means, etc. described above may be realized by hardware, for example, by designing part or all of them with an integrated circuit. Further, each configuration, function, etc. described above may be realized by software by a processor interpreting and executing a program that realizes each function. Information such as a program, a table, and a file for realizing each function can be placed in a memory, a hard disk, a recording device such as a solid state drive (SSD), or a recording medium such as an IC card, an SD card, or a DVD.

  Further, in each drawing, control lines and information lines indicate what is considered to be necessary for explanation, and not all the control lines and information lines in mounting are necessarily shown. For example, in practice it may be considered that almost all configurations are mutually connected.

  The arrangement of the various functional units, the various processing units, and the various databases of the information processing apparatus described above is merely an example. The arrangement form of the various functional units, the various processing units, and the various databases can be changed to an optimum arrangement form of each information processing apparatus in terms of hardware and software performance, processing efficiency, communication efficiency, and the like.

DESCRIPTION OF SYMBOLS 1 speech synthesis system, 10 information processing apparatus, 50 speech corpus, 51 utterance text, 52 speech data, 60 statistical model, 100 statistical model generation unit, 102 text including a pronunciation symbol string, 103 extracted words, 104 speech feature amount, 105 phonetic symbol string, 106 word extraction data, 110 text substitution unit, 111 substitution word extraction unit, 1111 morpheme analysis unit, 1112 word extraction unit, 112 speech feature amount extraction unit, 1121 pronunciation (phoneme) extraction unit, 1122 speech style Extraction unit, 1123 prosody feature extraction unit, 113 phonetic symbol string generation unit, 115 text substitution processing unit, 120 model learning unit, 200 speech synthesis unit, 203 text including phonetic symbol sequence, 202 text including phonetic symbol sequence, 210 users Dictionary application section, 211 word extraction day , 212 user dictionary, 2101 morphological analysis unit, 2102 word extraction section, 2103 pronunciation symbol string replacing unit, 700 input text 800 synthesized speech

Claims (15)

A model learning unit that generates a statistical model used for speech synthesis based on learning data in which speech text and speech data are associated;
A text substitution unit that generates, as the learning data, data in which speech data is associated with speech text in which a part of words of the speech text is replaced with a phonetic symbol string;
A storage unit storing a user dictionary which is data including information in which a word is associated with a phonetic symbol string for the word;
A statistical model generator having:
Regarding the target text which is a text to be subjected to speech synthesis, the phonetic symbol string in which a word included in the user dictionary among the words included in the target text is associated with the word in the user dictionary User dictionary application part replaced with,
A speech synthesis processing unit that generates synthesized speech by performing speech synthesis processing based on the statistical model on the target text after the replacement;
A speech synthesizer having
Speech synthesis system equipped with The speech synthesis system according to claim 1, wherein
The text replacement unit extracts a speech feature amount from the speech data, generates a phonetic symbol string based on the extracted voice feature amount, and an utterance text in which a part of words of the utterance text is replaced with the phonetic symbol sequence Generating data associated with voice data as the learning data;
Speech synthesis system. The speech synthesis system according to claim 2, wherein
The text substitution unit generates the phonetic symbol string by performing speech recognition on the speech data.
Speech synthesis system. The speech synthesis system according to claim 2, wherein
The speech feature is at least one of pronunciation (phoneme), speech style, and prosody.
Speech synthesis system. The speech synthesis system according to claim 1, wherein
The text substitution unit divides the utterance text into a plurality of words according to a morphological analysis algorithm;
The user dictionary application unit divides the target text into a plurality of words according to an algorithm common to the morphological analysis algorithm.
Speech synthesis system. The speech synthesis system according to claim 1, wherein
The text replacement unit extracts a word to be replaced with a phonetic symbol string from the utterance text by a word extraction algorithm.
The user dictionary application unit extracts a word to be replaced with a phonetic symbol string from the target text by an algorithm common to the word extraction algorithm.
Speech synthesis system. The speech synthesis system according to claim 1, wherein
The training data is a speech corpus
Speech synthesis system. The speech synthesis system according to claim 7, wherein
The text substitution unit extracts a word to be replaced with a phonetic symbol string from the speech text of the speech corpus by a word extraction algorithm taking into consideration the balance of the extracted phoneme or prosody of the extracted word.
Speech synthesis system. The speech synthesis system according to claim 1, wherein
The model learning unit generates the statistical model by DNN (Deep Neural Network).
Speech synthesis system. The speech synthesis system according to claim 1, wherein
ETE (End-To-End) speech synthesis system,
Speech synthesis system. The said statistical model production | generation apparatus in the speech synthesis system of Claim 1, Comprising:
The model learning unit that generates a statistical model used for speech synthesis based on learning data in which speech text and speech data are associated;
The text substitution unit generating, as the learning data, data in which speech data is associated with speech text in which a part of words of the speech text is replaced with a phonetic symbol string;
A statistical model generator comprising: The statistical model generation device according to claim 11, wherein
The training data is a speech corpus
Statistical model generator. The statistical model generation device according to claim 11, wherein
The model learning unit generates the statistical model by DNN (Deep Neural Network).
Statistical model generator. The speech synthesizer according to claim 1, wherein the speech synthesizer comprises:
The storage unit storing a user dictionary which is data including information in which a word is associated with a phonetic symbol string for the word;
Regarding the target text which is a text to be subjected to speech synthesis, the phonetic symbol string in which a word included in the user dictionary among the words included in the target text is associated with the word in the user dictionary The user dictionary application unit replaced by
The speech synthesis processing unit that generates synthesized speech by performing speech synthesis processing based on the statistical model on the target text after the replacement;
, A speech synthesizer. The information processing apparatus
Generating a statistical model to be used for speech synthesis based on learning data in which speech text and speech data are associated;
Generating, as the learning data, data in which an utterance text in which a part of words in the utterance text is replaced with a phonetic symbol string and speech data are associated;
Storing a user dictionary which is data including information in which a word is associated with a phonetic symbol string for the word;
Regarding the target text which is a text to be subjected to speech synthesis, the phonetic symbol string in which a word included in the user dictionary among the words included in the target text is associated with the word in the user dictionary Replacing with, and
Generating synthesized speech by performing speech synthesis processing based on the statistical model on the target text after the replacement;
Perform a speech synthesis method.

Images