JP2014130211A - Speech output device, speech output method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To impart a suitable expression to a synthesized voice of a conversation sentence when outputting the synthesized voice reading the conversation sentence aloud through voice synthesis.SOLUTION: In document read-aloud processing (S115), specified document data is acquired from an information processing server. In document segmentation processing (S125), the specified document data is analyzed to specify sentences included in a document represented by the specified document data respectively, and in sentence structure determination processing (S130), the respective specified sentences are analyzed to specify whether each sentence is a base sentence, a conversation sentence, or a mixed sentence. In expression specification processing (S135), an object explanation sentence is specified according to a sentence structure of each specified sentence and an expression of the specific conversation sentence is specified based upon a result of the specification. In speech synthesis processing (S145), speech synthesis is carried out so as to impart the specified expression to the specific conversation sentence and a synthesized speech is output.

Description

  The present invention relates to an audio output device, an audio output method, and a program that output a synthesized sound based on text data.

2. Description of the Related Art Conventionally, a text reading apparatus that reads out input text data using a known speech synthesis technique is known (see Patent Document 1).
In the text-to-speech reading device (speech output device) described in Patent Document 1, a conversation sentence included in the sentence data is specified according to “quotation marks” included in the input sentence data, and the context of the specified conversation sentence is determined. The attribute of the conversation sentence is specified based on the analyzed result. In the text-to-speech device described in Patent Document 1, speech synthesis is performed so that a synthesized speech having the specified conversational sentence attribute is obtained. Here, the attributes are the gender, age, etc. of the character who issued the conversation.

Japanese Patent Laid-Open No. 05-313685

By the way, in the text-to-speech device, it is required to give a facial expression suitable for the conversation sentence to the synthesized sound obtained by reading the conversation sentence by voice synthesis.
However, in the text-to-speech device described in Patent Literature 1, although the attribute of the conversation sentence is specified, the facial expression in the conversation sentence is not specified. For this reason, in the text-to-speech device described in Patent Document 1, there is a problem that it is not possible to give a facial expression suitable for the conversation sentence to the synthesized sound that is read out by the voice synthesis.

That is, the conventional technique has a problem that it is difficult to give an appropriate expression to the synthesized sound of the conversation sentence when outputting the synthesized sound obtained by reading out the text data by speech synthesis.
In view of the above, an object of the present invention is to provide an appropriate facial expression to a synthesized sound for a conversation sentence when outputting a synthesized sound obtained by reading out the conversation sentence by speech synthesis.

The speech output device of the first invention made to achieve the above object is characterized by comprising a sentence acquisition means, a sentence structure specifying means, a facial expression estimation means, and a speech synthesis means.
The sentence acquisition unit acquires sentence data representing a character string constituting the specified sentence. The sentence structure specifying unit specifies whether at least a part of each sentence included in the sentence represented by the sentence data acquired by the sentence acquiring unit is a conversational sentence or a local sentence. The facial expression estimation means uses one of the conversation sentences specified by the sentence structure specifying means as a specific conversation sentence, and based on the result of analyzing the meaning of the target explanation sentence that is at least one local sentence explaining the specific conversation sentence, The facial expression in the specific conversation sentence is estimated. The voice synthesizing unit outputs a synthesized sound synthesized by voice so that the facial expression estimated by the facial expression estimating unit is reflected in the specific conversation sentence.

  According to such a voice output device, the facial expression of a specific conversation sentence can be estimated based on the target explanatory sentence, and it is suitable for the specific conversation sentence with respect to the synthesized sound that is read out by the voice synthesis. A facial expression can be added.

That is, according to the voice output device of the first invention, when outputting the synthesized sound obtained by reading out the conversation sentence by voice synthesis, an appropriate expression can be given to the synthesized sound for the conversation sentence.
The “expression” in the first invention is a concept including at least emotions, emotions, scenes, and situations in a conversation sentence.

Among sentences, there is also a mixed sentence in which a conversation sentence and a local sentence are included in one sentence. The local sentence included in the mixed sentence usually has a high possibility of explaining the conversation sentence included in the mixed sentence.
Therefore, the facial expression estimation means in the first invention, when the mixed sentence exists in the sentence data as a result of the specification by the sentence structure specifying means, the conversation sentence included in the mixed sentence is a specific conversation sentence, the mixed sentence It is good also considering the text of the ground contained in the object explanatory text.

According to such an audio output device, since the facial expression of the conversation sentence included in the mixed sentence is estimated based on the meaning of the local sentence included in the mixed sentence, the estimation accuracy is high.
That is, according to the voice output device of the first invention, an appropriate facial expression can be given to the conversational sentence included in the mixed sentence.

In addition, a sentence composed of a single sentence in a local area often explains a conversation sentence included in a sentence immediately before or after the local sentence.
For this reason, the facial expression estimation means in the first invention, if there is a conversation sentence in at least one of the previous sentence or the next sentence of the local sentence, A sentence may be used, and a sentence in the area may be used as a target explanatory sentence.

  According to such a voice output device of the first invention, even if it is a single sentence consisting only of a conversational sentence, it is possible to specify an explanation target sentence that describes the conversational sentence, Based on this, the facial expression of the conversation sentence can be estimated.

  Further, the facial expression estimation means in the first invention totals the facial expressions corresponding to the meanings of the words included in the target explanatory sentence, and as a result of the aggregation, the facial expression corresponding to the most frequent value is used as the facial expression of the specific conversation sentence. good.

  According to such a voice output device, since the facial expression having the highest value as a result of aggregation is used as the facial expression in the specific conversation sentence, a more appropriate facial expression can be estimated for each conversation sentence. In particular, according to the voice output device of the present invention, it is possible to estimate an optimal facial expression for a conversational sentence even when there are a plurality of target explanatory sentences for one conversational sentence.

In addition, there is a high possibility that the local sentence explains the conversation sentence included in the same paragraph.
The sentence structure specifying means in the present invention divides a sentence included in sentence data into specific paragraphs that are paragraphs as constituent units, and determines whether at least a part of each sentence included in each specific paragraph is a conversational sentence. You may make it identify whether it is a sentence. In this case, the facial expression estimation means in the present invention may use a local sentence included in the same specific paragraph as the specific conversation sentence as the target explanatory sentence.

According to such an audio output device, since the facial expression in the conversation sentence is estimated based on the local sentence included in the specific paragraph, the estimation accuracy of the facial expression can be further improved.
By the way, the invention concerning this application may be made | formed as an audio | voice output method which outputs the synthetic | combination sound produced | generated by the audio | voice synthesis | combination.

In this case, the audio output method of the present invention includes a sentence acquisition procedure for acquiring sentence data representing a character string constituting a specified sentence, and at least a part of each sentence included in the sentence represented by the acquired sentence data. Is a sentence structure identifying procedure for identifying whether the sentence is a conversation sentence or a local sentence, and one of the identified conversation sentences is a specific conversation sentence, and is at least one local sentence explaining the specific conversation sentence Based on the result of analyzing the meaning of the target explanatory sentence, the facial expression estimation procedure for estimating the facial expression in the specific conversation sentence, and the synthesized speech synthesized so that the estimated facial expression is reflected in the specific conversation sentence is output. And a speech synthesis procedure.

  According to such an audio output method, each procedure can be executed by a plurality of different devices. As a result, according to the sound output method of the present invention, a system including a plurality of devices can be operated as the sound output device according to claim 1.

Furthermore, the invention according to the present application may be made as a program (third invention) to be executed by a computer.
In this case, the program of the present invention includes a sentence acquisition procedure for acquiring sentence data representing a character string constituting a specified sentence, and at least a part of each sentence included in the sentence represented by the acquired sentence data, Sentence structure identification procedure for identifying whether it is a conversational sentence or a local sentence, and one of the identified conversational sentences as a specific conversational sentence, and an object explanation that is at least one local sentence explaining the specific conversational sentence Based on the result of analyzing the meaning of the sentence, the facial expression estimation procedure for estimating the facial expression in the specific conversation sentence, and the voice that outputs the synthesized sound that is synthesized so that the estimated expression is reflected in the specific conversation sentence A synthesis procedure is executed by a computer.

  If the program of the present invention is made in this way, for example, it can be recorded on a computer-readable recording medium such as a DVD-ROM, CD-ROM, hard disk, etc. It can be used by being acquired and activated by a computer via a communication line as necessary. And by making a computer perform each procedure, the computer can be functioned as an audio | voice output apparatus described in Claim 1.

It is a block diagram which shows the whole structure. It is a flowchart which shows the process sequence of the text reading process in 1st embodiment. It is a flowchart which shows the process sequence of a paragraph division process. It is a flowchart which shows the process sequence of the sentence division | segmentation process in 1st embodiment. It is a flowchart which shows the process sequence of a sentence structure determination process. It is a flowchart which shows the process sequence of the facial expression specific process in 1st embodiment. It is a flowchart which shows the process sequence of a sentence type specific process. It is a flowchart which shows the process sequence of a facial expression allocation process. It is a flowchart which shows the process sequence of a facial expression estimation process. It is a flowchart which shows the process sequence of a facial expression candidate derivation process. It is a figure showing the processing result of (A) paragraph division processing, and the processing result of (B) sentence structure judging processing in a first embodiment. (A) The result of a sentence type specific process, (B), (C) It is a figure which shows the process result of the facial expression specific process in 1st embodiment. It is a flowchart which shows the process sequence of a speech synthesis process. It is a flowchart which shows the process sequence of the text reading-out process in 2nd embodiment. It is a flowchart which shows the process sequence of the sentence division | segmentation process in 2nd embodiment. It is a flowchart which shows the process sequence of the facial expression specific process in 2nd embodiment. (A) The process result of the sentence structure determination process in 2nd embodiment, (B), (C) It is a figure which shows the process result of the facial expression specific process in 2nd embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
[First embodiment]
<Speech synthesis system>
A speech synthesis system 1 shown in FIG. 1 is a system that outputs the contents of text data WT designated by a user as synthesized sounds having characteristics designated by the user. The speech synthesis system 1 analyzes the text data WT designated by the user and extracts speech data SD desired by the user of the speech synthesis system 1 from at least a plurality of speech data SD registered in advance. Voice synthesis.

In order to realize this, the speech synthesis system 1 includes at least one information processing server 10 and at least one speech output terminal 60. Note that the speech synthesis system 1 in this embodiment includes a plurality of speech output terminals 60.
<Information processing server>
The information processing server 10 includes a communication unit 12, a control unit 20, and a storage unit 30, and includes at least sentence data WT representing a character string constituting a sentence and at least a voice feature amount of speech input in advance. The server stores voice data SD and facial expression data ET representing the tendency of voice to which facial expressions are added.

In the communication unit 12, the information processing server 10 communicates with the outside through a communication network. The communication network in this embodiment is, for example, a public wireless communication network or a network line.
The control unit 20 includes a ROM 22 that stores processing programs and data that need to retain stored contents even when the power is turned off, a RAM 24 that temporarily stores processing programs and data, and processes stored in the ROM 22 and RAM 24. A known computer having at least a CPU 26 that executes various processes according to a program is mainly configured. The control unit 20 controls the communication unit 12 and the storage unit 30.

The storage unit 30 is a non-volatile storage device configured to be able to read and write stored contents. The storage device is, for example, a hard disk device or a flash memory.
The storage unit 30 stores text data WT, voice data SD, and facial expression data ET.

The text data WT is, for example, data obtained by converting a book into text data, and is prepared in advance for each book. Books here are novels and the like.
The voice data SD is data in which voice parameters PV _i and tag data TG _i are associated with each speaker.

  The voice parameter PV is prepared for each waveform of sound produced by a person, and is at least one feature amount representing the voice waveform i. This feature amount is a feature amount of speech used for so-called formant synthesis, and is prepared for each speaker and for each phoneme. As a feature amount in the speech parameter PV, at least a fundamental frequency F0, a mel frequency cepstrum (MFCC), a phoneme length, a power, and a time difference thereof are provided for each phoneme in the uttered speech.

  The tag data TG is data representing the nature of the sound represented by the speech parameter PV, and includes speaker feature information representing the features of the speaker, and facial expression information representing the expression of the speaker when the speech is spoken. At least. The speaker characteristic information includes, for example, the gender and age of the speaker. The facial expression information may include information representing a scene, emotion, atmosphere, and the like when speaking and information necessary for estimating the facial expression of the speaker in addition to information representing emotion as a facial expression.

  The expression data ET is data representing a tendency of the voice parameter PV expressed for each expression. The facial expression data ET in the present embodiment is generated by clustering the audio data SD, and the representative tag data TD, the representative parameter DPV, and the distributed parameter CPV_V are associated with each corresponding cluster.

The representative tag data TD is tag data TG representing the cluster in the tag data TG group included in each cluster.
The representative parameter DPV is a difference between the average parameter CPV_A and the reference parameter NPV in the cluster. The average parameter CPV_A here is an average value of the voice parameters PV included in each cluster. Further, the reference parameter NPV here is an average value of all voice parameters PV that satisfy the specific condition. However, the specific condition here indicates that the expression in the tag data TG represents a natural body.

The distribution parameter CPV_V is a distribution of the voice parameter PV included in each cluster.
<Audio output terminal>
The audio output terminal 60 includes a communication unit 61, an information receiving unit 62, a display unit 63, a sound input unit 64, a sound output unit 65, a storage unit 66, and a control unit 70. As the audio output terminal 60 in the present embodiment, for example, a known portable terminal may be assumed, or a known information processing apparatus such as a so-called personal computer may be assumed. Note that portable terminals include well-known electronic book terminals, and portable information terminals such as mobile phones and tablet terminals.

  In the communication unit 61, the audio output terminal 60 performs information communication with the outside via a communication network. The information receiving unit 62 receives information input via an input device (not shown). The display unit 63 displays an image based on a signal from the control unit 70.

  The sound input unit 64 is a device that converts sound into an electric signal and inputs the electric signal to the control unit 70, and is, for example, a microphone. The sound output unit 65 is a known device that outputs sound, and includes, for example, a PCM sound source and a speaker. The storage unit 66 is a non-volatile storage device configured to be able to read and write stored contents. The storage unit 66 stores various processing programs and various data.

The control unit 70 is configured around a known computer having at least a ROM 72, a RAM 74, and a CPU 76.
The ROM 72 of the control unit 70 analyzes the text data WT designated by the user and extracts voice data SD desired by the user from a plurality of pre-registered voice data SD based on the analysis result. A processing program for the control unit 70 to execute a text-to-speech process for executing speech synthesis is stored.

When analyzing the text data WT, when the text data WT is read from the storage unit 30, the type of text data of the character data WT is specified from the file extension and the like, and the character symbol ( After identifying the character code), analyze the sentence. For example, character symbols such as line feed (LF) and carriage return (CR) included in text data are distinguished from line feed symbols.
<Text reading process>
The text-to-speech process executed by the control unit 70 of the voice output device is started when a start command is input via the information receiving unit 62 of the voice output terminal 60.

  When the text-to-speech process is started, as shown in FIG. 2, first, information input through the information receiving unit 62 (hereinafter referred to as text designation information) is acquired (S110). The text designation information acquired in S110 is information for designating text data WT that is desired to be read out by speech synthesis.

Subsequently, sentence data WT (hereinafter referred to as “designated sentence data”) corresponding to the sentence designation information obtained in S110 is obtained from the storage unit 30 of the information processing server 10 (S115).
Further, the specified sentence data is analyzed, and a paragraph division process for specifying each paragraph of the sentence represented by the specified sentence data is executed (S120). For each paragraph specified in the paragraph dividing process (S120), a sentence dividing process for specifying each sentence included in each paragraph is executed (S125). Then, each sentence specified in the sentence division process (S125) is analyzed, and whether each sentence is a local sentence or a conversation sentence, or a local sentence and a conversation sentence are mixed in one sentence. A sentence structure determination process for specifying a sentence structure indicating whether it is a mixed sentence is executed (S130).

  Subsequently, according to the sentence structure of each sentence specified in the sentence structure determination process, the target explanatory sentence that is an explanatory sentence for the conversation sentence is specified, and the facial expression in the conversation sentence is specified based on the specified result. A facial expression specifying process is executed (S135).

  Further, the output property information input via the information receiving unit 62 is acquired (S140). This output property information is information corresponding to the speaker characteristic information, and is, for example, the sex and age of the speaker as the property of the sound output as a synthesized sound.

Subsequently, a speech synthesis process for generating and outputting a synthesized speech obtained by reading out the contents of the designated sentence data by speech synthesis is executed (S145).
Thereafter, the text reading process is terminated.
<Paragraph division processing>
When the paragraph division process activated in S120 of the text-to-speech process is activated, as shown in FIG. 3, the first character symbol of the designated sentence data is acquired (S210). The character symbol here is a symbol used in the text, and specifically includes symbols such as angle brackets and exclamation marks in addition to the characters themselves.

  Subsequently, it is determined whether or not the acquired character symbol is “close bracket” (“) (S215). In a general sentence, “” ”is used as the end of the conversation sentence. As a result of the determination in S215, if the acquired character symbol is not“ close bracket ”(S215: NO), the acquired character It is determined whether or not the symbol is “punctuation” (S220) As a result of the determination in S220, if the acquired character symbol is not “punctuation” (S220: NO), the process proceeds to S240 described later in detail. .

  On the other hand, if the acquired character symbol is “punctuation” as a result of the determination in S220 (S220: YES), or if the acquired character symbol is “close bracket” as a result of the determination in S215 (S215: YES). ), The next character symbol along the flow of the sentence in the designated sentence data is acquired (S225).

Then, it is determined whether or not the character symbol acquired in S225 is a “line feed symbol” (S2
30). As a result of this determination, if the character symbol is “line feed symbol” (S230: YES), the acquired character string, which is the character string acquired after the previous execution of S235, is taken as the sentence itself constituting one paragraph, It is stored in association with paragraph identification information for identifying the paragraph (hereinafter referred to as “paragraph ID”) (S235). Thereafter, the process proceeds to S240.

The information stored in S235 that associates the paragraph ID with the sentence itself that forms the paragraph identified by the paragraph ID is referred to as paragraph information.
On the other hand, as a result of the determination in S230, if the acquired character symbol is not the “line feed symbol” (S230: NO), the process proceeds to S240 without executing S235.

  In S240, it is determined whether or not the next character symbol along the flow of the sentence in the designated sentence data is included in the designated sentence data. If the result of determination in S240 is that the next character symbol is included (S240: YES), the next character symbol is acquired (S245), and the process returns to S215.

  On the other hand, if the result of determination in S240 is that the next character symbol is not included (S240: NO), it is assumed that the processing from S215 to S245 has been completed up to the last character symbol of the sentence in the designated sentence data. The division process is terminated, and the process proceeds to S125 of the text reading process.

That is, in the paragraph division processing of this embodiment, the sentence represented by the designated sentence data is analyzed, and each paragraph constituting the sentence is specified. Therefore, as a result of executing the paragraph division processing according to the present embodiment, paragraph information that classifies sentences included in the sentence data shown in FIG. 11A for each paragraph is generated.
<Sentence division processing>
As shown in FIG. 4, the text division process activated in S125 of the text-to-speech process acquires the first character symbol in the first paragraph specified from the paragraph information as shown in FIG. 4 (S310).

  Then, it is determined whether or not the acquired character symbol is “open parenthesis” (“) (S315) Since“ “” is used as the beginning of a conversational sentence in a general sentence, in this S315 As a result of the determination, if the acquired character symbol is not “open bracket” (S315: NO), the process proceeds to S335 described later in detail.

  On the other hand, if the acquired character symbol is “open bracket” as a result of the determination in S315 (S315: YES), the next character symbol along the flow of the sentence in the designated sentence data is acquired (S320). Then, it is determined whether or not the character symbol acquired in S320 is a “close bracket” (S325). As a result of the determination in S325, if the character symbol is “close bracket” (S325: YES), the process returns to S320.

  On the other hand, if the character symbol is not “close bracket” (S325: NO), it is determined whether or not the character symbol acquired in S320 is a line feed symbol (hereinafter referred to as “line feed”) (S330). As a result of the determination in S330, if the character symbol is not “line feed” (S330: NO), the process proceeds to S335.

  In S335, it is determined whether or not the acquired character symbol is a “punctuation point”. As a result of the determination in S335, the acquired character symbol is “punctuation” (S335: YES), or as a result of the determination in S330, the character symbol acquired in S320 is “line feed” (S320). : YES), the process proceeds to S340.

In S340, a string of character symbols acquired after the previous execution of S340 is regarded as one sentence, and is associated with sentence identification information (hereinafter referred to as “sentence ID”) for identifying the sentence and a paragraph ID. Remember. Thereafter, the process proceeds to S345.

The information stored in S340 in which the paragraph ID, the sentence ID, and the string of character symbols constituting the sentence identified by the paragraph ID and the sentence ID are associated is referred to as sentence information.
As a result of the determination in S335, if the acquired character symbol is not “punctuation” (S335: NO), the process proceeds to S345 without executing S340.

  In S345, it is determined whether or not the paragraph in which the steps from S315 to S340 are executed includes the next character / symbol along the flow of the sentence. If the result of determination in S345 is that the next character symbol is included (S345: YES), the next character symbol is acquired (S360), and the process returns to S315.

  On the other hand, if the result of determination in S345 is that the next character symbol is not included (S345: NO), it is determined whether or not the next paragraph specified from the paragraph information exists (S350). As a result of the determination in S350, if the next paragraph exists (S350: YES), the next paragraph is acquired (S355). Furthermore, the first character symbol of the paragraph acquired in S355 is acquired (S360), and the process returns to S315.

  As a result of the determination in S350, if the next paragraph does not exist (S350: NO), it is assumed that the processes from S315 to S340 have been completed up to the last character symbol of the sentence in the designated sentence data. The process ends, and the process proceeds to S130 of the text reading process.

That is, in the sentence division processing of the present embodiment, the sentence represented by the designated sentence data is analyzed, each sentence included in each paragraph is specified, and the paragraph ID, sentence ID, paragraph ID, and sentence ID are used. Sentence information is generated by associating strings of character symbols constituting the sentence to be identified.
<Sentence structure judgment processing>
As shown in FIG. 5, the sentence structure determination process activated in S130 of the sentence reading process acquires the first sentence along the flow of the sentence in the designated sentence data as shown in FIG. 5 (S410).

  Subsequently, it is determined whether or not the first character symbol in the acquired sentence is “open bracket” and the last character symbol is “close bracket” (S415). As a result of the determination, if the first and last character symbols are “parentheses” (S415: YES), “type 1” indicating that the acquired sentence is a conversation sentence is added to the sentence information corresponding to the acquired sentence. "Is added as identification information of the sentence structure type (S420). Thereafter, the process proceeds to S460 described later in detail.

  On the other hand, as a result of the determination in S415, if the first and last character symbols of the acquired sentence are not “parentheses” (S415: NO), it is determined whether or not parentheses are present in the acquired sentence (S425). . As a result of the determination in S425, if parentheses are not present in the acquired sentence (S425: NO), the sentence information corresponding to the acquired sentence indicates that the acquired sentence is a local sentence. "Type 3" is given as identification information of the sentence structure type (S430). Thereafter, the process proceeds to S460 described later in detail.

  If parentheses are present in the acquired sentence as a result of the determination in S425 (S425: YES), morphological analysis of the acquired sentence is executed (S435). For this morphological analysis, a known method may be used.

Subsequently, as a result of the morphological analysis in S435, it is determined whether or not the word immediately after the “close bracket” in the acquired sentence (hereinafter referred to as “determination target word”) is an attached word (S440). In addition,
The adjunct here is an adjunct to an independent word, and specifically includes at least one of a particle and an auxiliary verb.

  As a result of the determination in S440, if the determination target word is an attached word (S440: YES), “type 2” indicates that the acquired sentence is a mixed sentence in the sentence information corresponding to the acquired sentence. "Is added as sentence structure type identification information (S445). Thereafter, the process proceeds to S460 described later in detail.

  On the other hand, as a result of the determination in S440, if the determination target word is not an adjunct word (S440: NO), a new sentence with the sentence structure type identification information “type 1” for the sentence in parentheses in the acquired sentence Information is generated (S450). Further, new sentence information is generated with the sentence structure type identification information “type 3” for the sentence outside the parentheses in the acquired sentence (S455). Thereafter, the process proceeds to S460.

  That is, in S450 and S455, the acquired sentence is divided into a sentence inside parentheses and a sentence outside parentheses, and new sentence information is generated. At the same time, in S450 and S455, the sentence information corresponding to the sentence in parentheses is given the identification information of the sentence structure type “type 1”, and the sentence information corresponding to the sentence outside the parentheses is “type 3”. "Is added to the sentence structure type identification information.

  In S460, it is determined whether there is a next sentence along the sentence flow in the designated sentence data (S460). As a result of the determination in S460, if the next sentence exists (S460: YES), the next sentence is acquired (S465), and the process returns to S415.

On the other hand, as a result of the determination in S460, if the next sentence does not exist (S460: NO), the body structure determination process is terminated, and the process proceeds to S135 of the text reading process.
That is, in the sentence structure determination process, it is specified whether the structure of the sentence included in the designated sentence data is a conversational sentence, a local sentence, or a mixed sentence. Then, as shown in FIG. 11B, a sentence type structure indicating whether it is a conversational sentence, a local sentence or a mixed sentence is given to sentence information corresponding to each sentence.
<Facial expression identification processing>
When the facial expression specifying process activated in S135 of the text-to-speech process is activated as shown in FIG. 6, the sentence in parentheses and parentheses in the mixed sentence are based on the result of the sentence structure determination process executed previously. A sentence type specifying process for assigning a sentence structure type to another sentence is executed (S510).

  Specifically, as shown in FIG. 7, the sentence type specifying process acquires sentence information corresponding to the first sentence along the flow of sentences in the designated sentence data (S610). Subsequently, the sentence structure type in the sentence information acquired in S610 is acquired (S615), and it is determined whether or not the acquired sentence structure type is “type 2” (S620).

  As a result of the determination in S620, if the sentence structure type is “type 2” (S620: YES), the portion in parentheses in the sentence corresponding to the sentence information is referred to as a conversation sentence (hereinafter referred to as “parenthesized conversation sentence”). (S625). Further, the part outside the parentheses in the sentence corresponding to the sentence information is set as a background sentence (hereinafter referred to as “a sentence outside the parentheses”) (S630).

  That is, in S625 and S630, as shown in FIG. 12A, as the detailed information of the sentence structure type corresponding to the mixed sentence, information indicating that it is a conversation sentence with respect to the part in the parentheses in the mixed sentence. Is added to the portion outside the parentheses in the mixed sentence, and information indicating that it is a ground sentence.

Thereafter, the process proceeds to S635.
On the other hand, as a result of the determination in S620, if the sentence structure type is not “type 2” (S620: NO), the process proceeds to S635 without executing S625 and S630.

  In S635, it is determined whether or not there is a next sentence along the sentence flow in the designated sentence data (S635). As a result of the determination in S635, if the next sentence exists (S635: YES), the process returns to S610 to acquire sentence information corresponding to the next sentence (S610).

On the other hand, as a result of the determination in S635, if the next sentence does not exist (S635: NO), the body type specifying process is terminated.
Then, in the facial expression identification process, the expression frequency representing the high possibility of the facial expression expressed in the specific conversation sentence that is one of the conversation sentences, the explanation object that is the local sentence explaining the specific conversation sentence A facial expression assignment process derived based on the sentence is executed (S520).

  In the facial expression assignment process, specifically, as shown in FIG. 8, the sentence structure type is identified as “type 2” in the previous sentence type identification process, and the sentence flow in the designated sentence data is followed. A character string constituting the first sentence (hereinafter referred to as “estimation target sentence”) is acquired (S715). Subsequently, a character string (hereinafter referred to as “estimated usage sentence”) set as a sentence outside parentheses in the estimation target sentence acquired in S715 is acquired (S720).

  Subsequently, a conversation sentence in parentheses (hereinafter referred to as “expression estimation target sentence”) in the estimation target sentence is set as a specific conversation sentence, and an expression for the specific conversation sentence is analyzed based on the result of analyzing the estimated usage sentence as an explanation target sentence. The facial expression estimation process for deriving the frequency is executed (S725).

  Specifically, in this facial expression estimation process, as shown in FIG. 9, the estimated usage sentence is morphologically analyzed (S910), and for each word specified by the morphological analysis, word facial expression information corresponding to each word is acquired. (S915). The word expression information here is information in which each word is associated with the expression represented by each word in advance, and is stored in the word expression database 100 in advance.

  Then, according to the word expression information acquired in S915, the appearance frequency of the expression representing the same content is totaled for each expression content to derive the expression frequency (S920), and the process returns to the expression assignment process.

  In the facial expression assignment process returned from S920, it is determined whether or not there is a next estimation target sentence along the sentence flow in the designated sentence data (S730). As a result of the determination in S730, if there is a next estimation target sentence (S730: YES), the process returns to S715 to acquire the next estimation target sentence.

On the other hand, as a result of the determination in S730, if the next sentence to be estimated does not exist (S730: NO), the facial expression assignment process is terminated, and the process proceeds to S530 of the facial expression specifying process.
In S530 of the facial expression identification process, a conversation sentence other than the parenthesized conversation sentence is used as a specific conversation sentence, and the facial expression frequency representing the high possibility of the facial expression expressed in the specific conversation sentence is based on the explanation target sentence. The facial expression candidate derivation process is executed to execute the facial expression assignment process derived in step S530.

  In the facial expression candidate derivation process, specifically, as shown in FIG. 10, all sentences included in the same paragraph, that is, all sentences associated with the same paragraph ID are acquired based on sentence information. (S810). Subsequently, it is determined whether or not the number of sentences acquired in S810 is plural (S815).

  As a result of the determination in S815, when the number of sentences included in the same paragraph is singular (S815: NO), the process proceeds to S865 described later in detail. On the other hand, as a result of the determination in S815, if there are a plurality of sentences included in the same paragraph (S815: YES), the sentence structure type is set as “sentence structure type” based on sentence information corresponding to each of the plurality of sentences. It is determined whether or not a sentence given “type 2” or “type 3” exists in the plurality of sentences (S820).

  As a result of the determination in S820, if there is no sentence assigned “type 2” or “type 3” as the sentence structure type (S820: NO), the process proceeds to S865 described later in detail. On the other hand, as a result of the determination in S820, if there is a sentence assigned “type 2” or “type 3” as the sentence structure type (S820: YES), “type 2” or “ One sentence to which “type 3” is assigned is acquired along the sentence flow in the designated sentence data (S825). One sentence acquired in S825 is referred to as a second estimated usage sentence.

  And the facial expression estimation process which derives the facial expression frequency with respect to a specific conversation sentence based on the result of having analyzed the 2nd presumed utilization sentence acquired in S825 as a description object sentence is performed (S830). Note that the facial expression estimation process executed in S830 has the same processing content as the facial expression estimation process executed in S725, and a detailed description thereof will be omitted here.

  Subsequently, it is determined whether or not a sentence exists immediately before the second estimated usage sentence (that is, immediately before) along the flow of the sentence in the designated sentence data (S835). As a result of the determination in S835, if there is a sentence immediately before the second estimated usage sentence (S835: YES), the conversation sentence is included in the sentence immediately before (just before) the second estimated usage sentence. It is determined whether or not (S840).

  As a result of the determination in S840, if a conversation sentence is included in the sentence immediately before the second estimated usage sentence (S840: YES), the conversation sentence is set as a specific conversation sentence, and the facial expression estimation process in S830 is performed. The result is added to the expression frequency for the specific conversation sentence (S845). Thereafter, the process proceeds to S850.

  On the other hand, as a result of the determination in S835, when there is no sentence immediately before the second estimated usage sentence (S835: NO), or as a result of the determination in S840, the sentence immediately before the second estimated usage sentence When the conversation sentence is not included (S840: NO), the process proceeds to S850 without executing S845.

  In S850, it is determined whether there is a sentence immediately after (ie, immediately after) the second estimated usage sentence. As a result of the determination in S850, if there is a sentence immediately after the second estimated usage sentence (S850: YES), the conversation sentence is included in the sentence immediately after the second estimated usage sentence (immediately after). It is determined whether or not (S855).

  As a result of the determination in S855, if a conversation sentence is included in the sentence immediately after the second estimated usage sentence (S855: YES), the conversation sentence is set as a specific conversation sentence, and the facial expression estimation process in S830 is performed. The result is added to the expression frequency for the specific conversation sentence (S860). Thereafter, the process proceeds to S820.

  On the other hand, as a result of the determination in S850, when there is no sentence immediately after the second estimated usage sentence (S850: NO), or as a result of the determination in S855, the conversation immediately after the second estimated usage sentence When the sentence is not included (S855: NO), the process proceeds to S820 without executing S860.

By the way, when the number of sentences included in one paragraph is singular (S815: NO),
If a sentence assigned “type 2” or “type 3” as the sentence structure type is not included in one paragraph (S820: NO), the process proceeds to S865. In S865, it is determined whether or not the next paragraph exists along the sentence flow in the designated sentence data.

  As a result of the determination in S865, if the next paragraph exists (S865: YES), the process returns to S810, and if the next paragraph does not exist (S865: NO), the facial expression candidate derivation process ends. Then, the process proceeds to S540 of the facial expression specifying process.

In S540 of the facial expression specifying process, the facial expression of each specific conversation sentence is specified based on the facial expression frequency corresponding to each specific conversation sentence. Specifically, in S540 of the present embodiment, the facial expression frequency corresponding to each specific conversation sentence is the result of totaling the number of appearances for each facial expression as shown in FIG. The facial expression having the most content is identified as the facial expression of the specific conversation sentence associated with the facial expression frequency. In S540 of the present embodiment, as shown in FIG. 12C, the content of the specified facial expression is assigned to each corresponding specific conversation sentence.
<Speech synthesis processing>
As shown in FIG. 13, the speech synthesis process activated in S145 of the text-to-speech process, when activated, acquires the first sentence along the text flow in the designated text data as an output text (S1010).

  Subsequently, representative tag data TD corresponding to each phoneme necessary for outputting the acquired output wording as a synthesized sound and most similar to the speaker characteristic information in the output property information acquired in the previous S140; The associated voice parameter PV is acquired from the information processing server 10 (S1015).

  Subsequently, it is determined whether a facial expression is assigned to the acquired output word (S1020). If no facial expression is assigned to the output wording as a result of this determination (S1020: NO), the process proceeds to S1035 described later in detail.

  On the other hand, as a result of determination in S1020, if a facial expression is assigned to the acquired output wording (S1020: YES), facial expression data ET including representative tag data TD most similar to the facial expression assigned to the acquired output wording Is acquired from the information processing server 10 (S1025). Then, the voice parameter PV acquired in S1015 is adjusted based on the facial expression data ET acquired in S1025 so that a synthesized sound corresponding to the acquired output wording is output (S1030).

  Subsequently, speech synthesis is performed based on the speech parameter PV adjusted in S1030 (S1035). The voice synthesis in S1035 uses a well-known voice synthesis technique by formant synthesis. Note that, as a result of the determination in S1020, in S1035 where the expression is not assigned to the output wording (S1020: NO), the formant based on the voice parameter PV acquired in S1015 without executing S1025 and S1030. Perform synthesis.

  Subsequently, it is determined whether or not there is a next sentence along the flow of the sentence in the designated sentence data (S1040). If the result of the determination is that the next sentence exists (S1040: YES), the next Is obtained as an output word (S1045), and the process returns to S1015.

On the other hand, as a result of the determination in S1040, if the next sentence does not exist (S1040: NO), the speech synthesis process and the sentence reading process are ended.
[Effect of the first embodiment]
As described above, according to the speech synthesis system 1, the facial expression of a specific conversation sentence can be estimated based on the target explanatory sentence, and the specific conversation is performed with respect to the synthesized sound read out from the specific conversation sentence by speech synthesis. Facial expressions suitable for sentences can be given.

That is, according to the speech synthesis system 1, when a synthesized sound obtained by reading out a conversation sentence by speech synthesis is output, an appropriate expression can be given to the synthesized sound for the conversation sentence.
By the way, in the mixed sentence in which the conversation sentence and the local sentence are included in one sentence, it is highly likely that the local sentence included in the mixed sentence explains the conversation sentence included in the mixed sentence.

  Therefore, according to the speech synthesis system 1 that estimates a facial expression in a specific conversation sentence with a conversation sentence included in the mixed sentence as a specific conversation sentence, and a local sentence included in the mixed sentence as a target explanatory sentence, it is included in the mixed sentence. Since the expression of the conversational sentence included in the mixed sentence is estimated based on the local sentence, the estimation accuracy is high.

That is, according to the speech synthesis system 1, an appropriate facial expression can be given to the conversational sentence included in the mixed sentence.
In addition, a sentence composed of a single sentence in a local area often explains a conversation sentence included in a sentence immediately before or after the local sentence.

  Therefore, according to the speech synthesis system 1 in which the conversation sentence included in the immediately preceding sentence and the immediately following sentence of the local sentence is a specific conversation sentence and the local sentence is the target explanatory sentence, only the conversation sentence is used. Even if it is one sentence which becomes, the description object sentence which has demonstrated the conversation sentence can be specified.

In addition, a sentence composed only of a local sentence is likely to explain a conversation sentence included in the same paragraph.
Therefore, according to the speech synthesis system 1 that estimates the facial expression of the specific conversation sentence using the local sentence in the same paragraph as the target explanatory sentence for one specific conversation sentence, the estimation accuracy of the facial expression is further improved. Can do.

  In particular, in the speech synthesis system 1 of the present embodiment, the expression frequency is defined as the result of totaling the number of appearances for each facial expression corresponding to each specific conversation sentence, and the facial expression having the highest count result (value) of the expression frequency is expressed as It is specified as an expression of a specific conversation sentence associated with the expression frequency.

For this reason, according to the speech synthesis system 1, a more appropriate facial expression can be estimated for each specific conversation sentence. Furthermore, according to the speech synthesis system 1, even when there are a plurality of target explanatory sentences for one conversation sentence, it is possible to estimate an optimal facial expression for the conversation sentence.
[Second Embodiment]
The speech synthesis system according to the second embodiment is different from the speech synthesis system 1 according to the first embodiment mainly in the processing contents of some processes constituting the text reading process. For this reason, in the present embodiment, the same components and processes as those in the first embodiment are denoted by the same reference numerals, description thereof is omitted, and a part of the text reading process different from that in the first embodiment is configured. The process will be mainly described.
<Text reading process>
When the text-to-speech process in the present embodiment is activated by an activation command input via the information receiving unit 62 of the voice output terminal 60, first, as shown in FIG. S110).

  Subsequently, the designated sentence data corresponding to the sentence designation information obtained in S110 is obtained from the storage unit 30 of the information processing server 10 (S115). Furthermore, the specified sentence data is analyzed, and a sentence dividing process for specifying each sentence constituting the sentence represented by the specified sentence data is executed (S125).

Then, each sentence specified in the sentence division process (S125) is analyzed, and a sentence structure determination process for specifying a sentence structure for each sentence is executed (S130).
Subsequently, according to the sentence structure of each sentence specified in the sentence structure determination process, the target explanatory sentence that is an explanatory sentence for the conversation sentence is specified, and the facial expression in the conversation sentence is specified based on the specified result. A facial expression specifying process is executed (S135).

  Further, the output property information input via the information receiving unit 62 is acquired (S140). Subsequently, a speech synthesis process for generating and outputting a synthesized speech obtained by reading out the contents of the designated sentence data by speech synthesis is executed (S145).

Thereafter, the text reading process is terminated.
That is, in the text reading process of this embodiment, the paragraph division process (S120) is omitted from the text reading process in the first embodiment.
<Sentence division processing>
For this reason, as shown in FIG. 15, when the sentence division processing of this embodiment is started, the first character symbol along the sentence flow in the designated sentence data is acquired (S310).

  If the acquired character symbol is not “open bracket” (S315: NO), the process proceeds to S335 described later. On the other hand, if the acquired character symbol is “open bracket” (S315: YES), the next character symbol along the flow of the sentence in the designated sentence data is acquired (S320).

  If the character symbol acquired in S320 is not “close bracket” (S325: NO), the process returns to S320. On the other hand, as a result of the determination in S325, if the character symbol is “close bracket” (S325: YES), it is determined whether or not the character symbol acquired in S320 is “line feed” (S330).

If it is determined in S330 that the character symbol is not “line feed” (S330: NO), the process proceeds to S335.
In S335, it is determined whether or not the acquired character symbol is a “punctuation point”. As a result of the determination in S335, the acquired character symbol is “punctuation” (S335: YES), or as a result of the determination in S330, the character symbol acquired in S320 is “line feed” (S320). : YES), the process proceeds to S340.

In S340, a string of character symbols acquired after the previous execution of S340 is regarded as one sentence itself and stored in association with the sentence ID. Thereafter, the process proceeds to S345.
That is, the sentence information in the present embodiment is information in which a sentence ID is associated with a string of character symbols constituting a sentence identified by the sentence ID.

If the acquired character / symbol is not “punctuation” as a result of the determination in S335 (S335: NO), the process proceeds to S345 without executing S340.
In S345, it is determined whether or not the next character / symbol along the sentence flow in the designated sentence data is included. If the result of determination in S345 is that the next character symbol is included (S345: YES), the next character symbol is acquired (S360), and the process returns to S315.

On the other hand, as a result of the determination in S345, if the next character symbol is not included (S345: NO), it is assumed that the processing from S315 to S340 has been completed up to the last character symbol of the sentence in the designated sentence data. The division process ends, and the process proceeds to S130 for the text reading process.

  That is, in the sentence dividing process of this embodiment, S350 and S355 are omitted from the sentence dividing process in the first embodiment. As a result, the sentence information generated in the sentence division process according to the present embodiment is information in which the sentence ID is associated with the character symbol string constituting the sentence identified by the sentence ID.

Then, when the sentence structure determination process (S130) of the present embodiment is executed, the sentence information includes the sentence structure of each sentence in the sentence corresponding to the sentence information, as shown in FIG. A type is given.
<Facial expression identification processing>
Next, as shown in FIG. 16, the facial expression specifying process according to the present embodiment, when activated, a sentence in parentheses and a sentence outside parentheses in a mixed sentence based on the result of the sentence structure determination process executed earlier. A sentence type specifying process for assigning a sentence structure type to each is executed (S510).

  The processing content of the sentence type specifying process in the present embodiment is the same as the processing content of the sentence type specifying process in the first embodiment. For this reason, also in the sentence type specifying process of the present embodiment, as shown in FIG. 17B, as the detailed information of the sentence structure type corresponding to the mixed sentence, the conversation within the parentheses in the mixed sentence is performed. Information indicating that it is a sentence is given, and information showing that it is a local sentence is given to the part outside the parentheses in the mixed sentence.

  Next, facial expression assignment processing is performed to derive the facial expression frequency that represents the high possibility of facial expression expressed in a specific conversation sentence based on the explanation sentence that is the local sentence that explains the specific conversation sentence. (S520).

Since the processing content of the facial expression assignment process in the present embodiment is the same as the processing content of the facial expression assignment process in the first embodiment, detailed description thereof is omitted here.
Then, the facial expression of each specific conversation sentence is identified based on the expression frequency corresponding to each specific conversation sentence (S540). Specifically, in S540 of the present embodiment, the facial expression having the highest count result (value) of the facial expression frequency corresponding to each specific conversation sentence in which the number of appearances for each facial expression is totaled is identified in association with the facial expression frequency. It is specified as the expression of the conversation sentence. Note that also in S540 of the present embodiment, as shown in FIG. 17C, the content of the specified facial expression is assigned to each corresponding specific conversation sentence.
[Effects of Second Embodiment]
Even in such a speech synthesis system according to the second embodiment, the facial expression of the specific conversation sentence can be estimated based on the target explanatory sentence, as in the speech synthesis system 1 according to the first embodiment, and the specific conversation can be performed by speech synthesis. A facial expression suitable for the specific conversation sentence can be given to the synthesized sound obtained by reading the sentence.

That is, also in the speech synthesis system of the present embodiment, an appropriate facial expression can be given to the synthesized sound for the conversation sentence when outputting the synthesized sound that is read out from the conversation sentence by speech synthesis. [Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

In the facial expression candidate derivation process of the first embodiment, the conversation sentence existing in both the immediately preceding sentence and the immediately following sentence of the ground sentence is the specific conversation sentence, and the local sentence is the target explanatory sentence. However, in the present invention, only the sentence immediately before the local sentence may be the specific conversation sentence, or only the sentence immediately before and the sentence after the local sentence may be the specific conversation sentence. That is, in the present invention, the conversation sentence included in at least one of the sentence immediately before or the sentence immediately after the ground sentence may be set as the specific conversation sentence.

  In addition, the aspect which abbreviate | omitted a part of structure of the said embodiment as long as the subject could be solved is also embodiment of this invention. Further, an aspect configured by appropriately combining the above embodiment and the modification is also an embodiment of the present invention. Moreover, all the aspects which can be considered in the limit which does not deviate from the essence of the invention specified by the wording described in the claims are the embodiments of the present invention.

The reference numerals used in the description of the above embodiments are also used in the claims as appropriate, but they are used for the purpose of facilitating the understanding of the invention according to each claim, and the technical aspects of the invention according to each claim. It is not intended to limit the scope.
[Correspondence between Embodiment and Claims]
The function obtained by executing S110 and S115 of the text-to-speech process of the above embodiment corresponds to the text acquisition means in the description of the claims, and the function obtained by executing S120 to S130 is claimed. This corresponds to the sentence structure specifying means in the description of the range.

  Further, the function obtained by executing S135 of the text-to-speech process of the above embodiment corresponds to the facial expression estimating means in the claims, and the function obtained by executing S140 and S145 is claimed. This corresponds to the speech synthesis means in the description of the range.

DESCRIPTION OF SYMBOLS 1 ... Speech synthesis system 10 ... Information processing server 12 ... Communication part 20 ... Control part 22 ... ROM 24 ... RAM 26 ... CPU 30 ... Memory | storage part 60 ... Voice output terminal 61 ... Communication part 62 ... Information reception part 63 ... Display part 64 ... Sound input unit 65 ... Sound output unit 66 ... Storage unit 70 ... Control unit 72 ... ROM 74 ... RAM 76 ... CPU 100 ... Word expression database

Claims (7)

Sentence acquisition means for acquiring sentence data representing a character string constituting the specified sentence;
Sentence structure specifying means for specifying whether at least a part of each sentence included in the sentence represented by the sentence data acquired by the sentence acquisition means is a conversational sentence or a local sentence;
One of the conversation sentences specified by the sentence structure specifying means is a specific conversation sentence, and the specific conversation is based on the result of analyzing the meaning of the target explanation sentence that is at least one local sentence explaining the specific conversation sentence. Facial expression estimation means for estimating facial expressions in sentences;
A voice output device comprising: voice synthesis means for outputting a synthesized voice synthesized by voice so that the facial expression estimated by the facial expression estimation means is reflected in the specific conversation sentence. The facial expression estimation means includes
As a result of the specification by the sentence structure specifying means, when there is a mixed sentence in which the conversation sentence and the local sentence are included in one sentence in the sentence data, the conversation sentence included in the mixed sentence is The voice output device according to claim 1, wherein a specific conversation sentence is used, and a local sentence included in the mixed sentence is the target explanation sentence. The facial expression estimation means includes
As a result of the specification by the sentence structure specifying means, if the conversation sentence is present in at least one of the previous sentence or the next sentence of the local sentence, the conversation sentence is The voice output device according to claim 1, wherein a specific conversation sentence is used, and a sentence in the place is the target explanation sentence. The facial expression estimation means includes
The facial expressions corresponding to the meanings of the words included in the target explanatory sentence are totaled, and as a result of the totaling, the facial expression corresponding to the most frequent value is used as the facial expression of the specific conversation sentence. The audio output device according to claim 3. The sentence structure specifying means includes:
The sentence included in the sentence data is divided into specific paragraphs that are paragraphs as structural units, and it is specified whether at least a part of each sentence included in each of the specific paragraphs is a conversational sentence or a local sentence And
The facial expression estimation means includes
The speech output apparatus according to any one of claims 1 to 4, wherein a local sentence included in the specific paragraph that is the same as the specific conversation sentence is used as the target explanatory sentence. A sentence acquisition procedure for acquiring sentence data representing a character string constituting the specified sentence;
A sentence structure identifying procedure for identifying whether at least a part of each sentence included in the sentence represented by the sentence data obtained in the sentence obtaining procedure is a conversational sentence or a local sentence;
One of the conversation sentences specified in the sentence structure specifying procedure is set as a specific conversation sentence, and the specific conversation is based on the result of analyzing the meaning of the target explanation sentence that is at least one local sentence explaining the specific conversation sentence. A facial expression estimation procedure for estimating a facial expression in a sentence;
A voice output method comprising: a voice synthesis procedure for outputting a synthesized voice that is voice-synthesized so that the facial expression estimated in the facial expression estimation procedure is reflected in the specific conversation sentence. A sentence acquisition procedure for acquiring sentence data representing a character string constituting the specified sentence;
A sentence structure identifying procedure for identifying whether at least a part of each sentence included in the sentence represented by the sentence data obtained in the sentence obtaining procedure is a conversational sentence or a local sentence;
One of the conversation sentences specified in the sentence structure specifying procedure is set as a specific conversation sentence, and the specific conversation is based on the result of analyzing the meaning of the target explanation sentence that is at least one local sentence explaining the specific conversation sentence. A facial expression estimation procedure for estimating a facial expression in a sentence;
A speech synthesis procedure for outputting a synthesized speech synthesized by speech so that the facial expression estimated in the facial expression estimation procedure is reflected in the specific conversation sentence,
A program characterized by being executed by a computer.

Images