JPH07334507A - Human body action and voice generation system from text - Google Patents

Abstract

PURPOSE:To input a text written in a natural language, to generate a human body action which is matched with the content of the text and to output the content of the text by means of voice. CONSTITUTION:A natural language analysis device 1 fetching a word from the text, a verb/action pattern dictionary 21 describing the corresponding relation of the word showing the action and a human body action pattern, an action pattern generation device 2 retrieving the verb/action pattern dictionary 21 and generating the human body pattern, a modifier/action degree dictionary 31 describing the corresponding relation of a modifier and the degree of the action, an action degree generation device 3 retrieving the modifier/action degree dictionary 31 and generating the action degree of the human body pattern, an action and voice synchronization device 4 outputting operation time data obtained by making the appearing position of a verb in the text corresponding to the start time of action generation, and outputting voice data, an action video generation device 5 generating time sequential video data of the human body action and video-outputting it and a voice synthesis device 6 outputting voice by a regular voice synthesis method are provided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for generating a human body motion voice from text, and more particularly to the generation of a human body motion and voice, and human motion and voice from a texta file written in a natural language using a computer system. Related to the system that creates the animation.

[0002]

2. Description of the Related Art As a conventional technique, a method of generating a human body motion using a natural language syntax analysis technique by a computer is disclosed in Japanese Patent Laid-Open No. 264972/1992. This is to generate a human body motion by using a natural language syntax analysis technique to generate a motion program using a specific word or the like that represents a motion analyzed from text.

As another conventional technique, Japanese Patent Laid-Open No. 2-234285 discloses a method of generating a change in human mouth shape by using phonemes from a speech synthesizer. This is to generate a change in the human mouth shape by controlling the parameters of the mouth shape feature corresponding to each phoneme by using the phonemes of the voice generated by the regular voice synthesizer from the texter file.

[0004]

The above-mentioned conventional technique has the following problems.

(1) In the former case, the action can be generated from the text, but the voice synchronized with the action cannot be output.

(2) Further, in the former, a motion program dedicated to animation for describing motion is generated, but since the motion program is described in the same format as the computer program, editing work by a general user who is not a programmer Not suitable for.

(3) In the latter case, only the change of the mouth shape, which is a part of the human motion, is generated from the voice output of the text, and the other such as the body for generating a more natural human motion image. It is difficult to generate part movement.

[0008]

In order to solve the problems described in the above items (1) and (3), the system for producing a human body motion voice from text according to the present invention can be used to convert verbs and adverbs from texts. A natural language analysis means for extracting words,
A verb / action pattern dictionary that describes the correspondence between words that represent actions such as verbs and human body action patterns, and the verb / action pattern dictionary that is extracted using the verb extracted by the natural language analysis means are used to retrieve the human body action pattern. The action pattern generating means for generating, a modifier / actor degree dictionary describing the correspondence between modifiers such as adverbs for modifying verbs and the degree of motion, and the modifier extracted by the natural language analyzing means A motion generation means for searching a modifier / motion dictionary and generating the motion of the human body motion pattern, and the motion generation start for the appearance position of the verb in the text for synchronizing the motion video output and the synthetic voice output An operation voice synchronization means for outputting action time data associated with time, and voice time data including a text reading time calculated from the length of the text; A motion image generation unit that inputs a motion generation command including the human motion pattern, the degree of motion and motion time data, generates time series video data of the human motion and outputs the time series video data to the display unit, and a voice including the text and the voice time data. And a voice synthesizing unit which outputs a voice by a regular voice synthesizing method with a generation command as an input.

Further, in order to solve the problem described in the above item (2), the motion generation output from the motion pattern generation means, the motion degree generation means and the motion voice synchronization means is added to the above configuration. An action voice generation command / text conversion unit for converting a command and the voice generation command into a sentence action description file which is a human-readable text, and a sentence action description file converted by the action voice generation command / text conversion unit are stored. A text / motion description file storage means and a text / motion that converts the text / motion description file from the text / motion description file storage means into the motion generation command and the voice generation command, and outputs the motion / motion command to the motion video generation means and the voice synthesis means. By adding voice generation command conversion means, you can modify the sentence behavior description file with an external editor. It is made possible.

[0010]

In the present invention, the input text is analyzed and divided into words, and words such as verbs and verb modifiers are extracted. Then, the action generation timing is determined based on the rule that the position appearing in the verb is used as the action start signal.

Further, the point is to determine a human body motion pattern corresponding to the type of verb, or to determine the degree of motion of a motion by using a modifier. With this, when text is given, a voice output and a smooth human motion synchronized with the voice can be automatically created.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a human-body motion voice generation system from text according to the present invention, and is a diagram showing a configuration of an embodiment of the first invention.

As shown in FIG. 1, the system for producing a human motion voice from text according to the present embodiment comprises an A device 100 and a B device 200, and the A device 100 extracts words such as verbs and adverbs from the text. Natural language analysis device 1,
Using the verb / action pattern dictionary 21 that describes the correspondence between words representing actions such as verbs and human action patterns, and the verb / action pattern dictionary 21 that is extracted by the natural language analysis device 1, the human action pattern is searched. A motion pattern generation device 2 for generating a motion pattern, a modifier / motion degree dictionary 31 for describing a correspondence relation between a modifier such as an adverb for modifying a verb and the degree of motion, and a modifier extracted by the natural language analysis device 1. The movement degree generation device 3 that searches the modifier / movement degree dictionary 31 using the movement degree generation apparatus 3 for generating the movement degree of the human body movement pattern, and the appearance position of the verb in the text for synchronizing the movement video output and the synthesized voice output Action voice synchronization that outputs action time data associated with the start time of action generation, and voice time data that includes the text reading time calculated from the length of the text Composed from the device 4.

Further, the device B 200 receives the action generation command including the human action pattern, the degree of action and the action time data, generates the time-series image data of the human action, and outputs it to the display device. A voice synthesis device 6 which receives a voice generation command including text and voice time data and outputs a voice by a regular voice synthesis method.

The natural language analysis device 1 extracts individual words from text input from the outside. The natural language analysis device 1 extracts a word from a sentence by using a conventional syntax analysis method. Here, a method of syntactic analysis using the CYK method (Ryoichi Sugimura, Koji Akasaka, Yukihiro Kubo: Logical Morphological Analysis LAX, Proc. Of the Logic).
Programming Conf. ICOT, 2
13-222, 1988) and the like. Since the natural language analysis device 1 uses the existing technology,
Detailed description is omitted here.

With this natural language analysis apparatus 1, for example, a sentence such as "he laughed pleasantly" can be decomposed into individual independent phrases "he wa ... pleasantly laughed". Further, as shown below, the grammatical meaning of each word is obtained in the sentence.

He: Subject is: Particles Comfortable: Modifier Laughing: Verb The verb output from the natural language analysis device 1 is input to the motion pattern generation device 2 to generate a human body motion pattern. Specifically, the verb / motion pattern dictionary 21 that stores the correspondence between verbs and human motion patterns is searched, and a motion pattern corresponding to the input verb is generated and output.

Next, details of the process for generating the human body motion pattern will be described.

In the verb / motion pattern dictionary 21, a plurality of human motion patterns are associated with one verb, and each human motion pattern is given a priority. Therefore, when retrieving a human body movement pattern from a verb, the human body movement pattern is output in descending order of priority.

FIG. 5 is a block diagram showing details of the operation pattern generation device 2 of FIG. In FIG. 5, the motion pattern constraint condition dictionary 26 stores information as to whether or not two ordered human body motion patterns before and after are appropriate. For example, a motion pattern of swinging the head back and forth with the head tilted to the left is defined as unnatural and not appropriate. Knowledge of whether or not the human body motion pattern currently generated from the preceding human body motion pattern is appropriate is defined in the motion pattern constraint condition dictionary 26.

FIG. 6 is a flow chart showing the flow of processing for the motion pattern generation device 2 of FIG. 5 to generate a human motion pattern.

First, the verb obtained from the natural language analysis device 1 is input to the motion pattern search unit 22. At this time,
The count generator 23 adds "1" to the current count value (initial value is "0"), and sends the count value to the operation pattern search unit 22.

The operation of the operation pattern generator 2 will be described below separately for the case of count value = 1 and the case of count value> 1. When the count value = 1: First, the motion pattern search unit 2
Verb / action pattern dictionary 2 based on the verb 2 entered
The human body motion pattern with the highest priority is extracted from 1.
Then, at the same time that the extracted human body motion pattern is output, the human body motion pattern and the count value are stored together in the history storage unit 24. When the count value> 1, the action pattern search unit 22 extracts the human body action pattern (MP _i ) having the highest priority from the verb / action pattern dictionary 21 based on the input verb. Next, from the history storage unit 24, the current count value of 1
The previous human body motion pattern (MP _i-1 ) is taken out. Then, the human body motion patterns (MP _i ) and (MP _i-1 ) are sent to the motion pattern matching unit 25.

The motion pattern matching unit 25 refers to the motion pattern constraint condition dictionary 26, and compares the current human motion pattern (MP _i-1 ) with the previous human motion pattern (MP _i-1 ).
Determine if _i ) is appropriate.

When it is judged to be appropriate, the current human body motion pattern (MP _i ) is output, and the current count value and the current human body motion pattern (MP _i ) are stored in the history storage unit 24. . When it is determined that it is not appropriate, the motion pattern search unit 22 extracts the human body motion pattern (MP ' _i-1 ) having the next highest priority again.

Next, this human body motion pattern (MP '
_i-1 ) and the previous human body movement pattern (MP _i-1 ) are used to collate the human body movement pattern, and the human body movement pattern search / collation processing is repeated until it is determined to be appropriate. If it is determined that all searched human body movement patterns are inappropriate, the human body movement pattern having the highest priority is output.

Further, the modifier output from the natural language analysis device 1 is used as an input to the motion degree generation device 3 to generate motion degree data describing the degree of a human body motion pattern. Specifically, the modifier / motion degree dictionary 31 that stores the correspondence between the modifier and the motion degree of the human body motion pattern is searched, and the motion degree corresponding to the input modifier is generated and output. Here, numerical data can be used to express the degree of motion.

The motion voice synchronization device 4 has a function of matching the timing of motion generation and the output of synthetic voice based on the words and phrases obtained from the natural language analysis device 1. In the present invention, a delimited text sandwiched by punctuation marks from the natural language analysis device 1 is considered as a basic unit of motion generation and speech synthesis, and the position where a verb representing a motion appears in this text is the start of motion generation. Position. The specific processing will be described below.

FIG. 4 is a diagram showing an example of time data of motion / voice output from the motion voice synchronization device 4. In FIG. 4, first, the total time for outputting one word is calculated based on the voice output speed of the voice synthesizer 6. For example, if the time required for the speech synthesizer 6 to output one character is t seconds, then the output time T _{S of a} word consisting of n characters is Ts = n × t seconds. Then, when the voice start time is t _S0 seconds (relative time is 0 seconds) corresponding to the first word of the phrase, the voice end time is t _S0 + T _S seconds. If the position where the verb appears in the phrase is the m-th character, the motion generation start time t _m0 = t _S0 + m × t seconds The length of the operation time T _m = (n−m) × t seconds Becomes FIG. 4 shows the operation time data and the voice time data calculated in this way.

Next, the motion picture generation device 5 generates a motion including the human body motion pattern from the motion pattern generation device 2, the motion level from the motion level generation device 3, and the motion time data from the motion voice synchronization device 4. A command is input, and a time series image of a human body motion is output to a display device, a VTR, or the like. In this motion picture generation apparatus 5, a method of generating a human motion pattern by synthesizing a plurality of motion modules (eg, Luyama, Shun Yoshisaka, Hitoshi Miyai: “Proposal of human motion generation system”, IPSJ The 47th National Convention Lecture Collection (2), 345-346, 1993) is used.

Next, with respect to the speech synthesizer 6, the existing speech rule synthesizing method can be used (Seiichi Yamamoto, Yoshio Higuchi, Tohru Shimizu: "Prototype of speech rule synthesizer with text editing function", IEICE Technical Report SP
87-137, March 1988). Then, it inputs a voice output command including the phrase from the natural language analysis device 1 and the time data from the motion voice synchronization device 4 and synthesizes and outputs a voice.

FIG. 2 is a block diagram showing the configuration of an embodiment of the second invention. In this embodiment, as shown in FIG.
A motion voice generation command / text conversion device 7 for converting a motion generation command and a voice generation command output from the A device 100 into a text motion description file which is a human-readable text, and the motion voice generation command / text conversion device 7. A text action description file storage device 8 for storing the converted text action description file, a text action description file from this text action description file storage device 8 is converted into a motion generation command and a voice generation command, and the B device 200 is operated. It is composed of an output text / action voice generation command conversion device 9.

The device A 100 includes a motion pattern generator 2, a motion degree generator 3 and a motion voice synchronizer 4, and the device B 200 includes a motion video generator 5 and a voice synthesizer 6. However, these A device 100 and B device 200 have already been described in the embodiment of the first invention, and are omitted to avoid duplication,
Another part of FIG. 2 will be described.

In this embodiment, the motion generation command and the voice generation command output from the A-apparatus 100 are converted into the motion voice generation command.
The text conversion device 7 converts the text action description file into a text file suitable for the format.

FIG. 3 is a diagram showing an example of this format. In FIG. 3, an underline mark is added to the text sentence written in the text file at the same position as the verb included in the text, and
Under the underline mark, the human body motion pattern name generated by the device A 100, the parameter p of the motion level of the human body motion pattern, and the length t of the motion time are described.

Next, the sentence action description file converted by the action voice generation command / text converter 7 is stored in the sentence action description file storage device 8 composed of an external storage device such as a magnetic disk device. Since the stored text action description file is in the form of a readable text file, it is possible to easily correct the action using a commercially available text editor.

Text / motion voice generation command conversion device 9
Contrary to the motion voice generation instruction / text conversion device 7, reads the human motion pattern, the motion degree of the human motion pattern, and the motion time data written in the text motion description file, and the motion video generation device 5 in the device B 200 is read. It is converted into the action generation command of input. Then, the text written in the text action description file is read, and the text for voice output and voice time data are generated by using the voice time data generation method used in the motion voice synchronization device 4. The sound is output by inputting it to the synthesizer 6.

[0039]

As described above, the system for producing a human motion voice from text according to the present invention outputs the input natural language text as a synthetic voice and automatically performs the human motion synchronized with the voice. Can be generated.

By creating a motion voice description file having a shape close to the original text and editing the file with an ordinary text editor, the finally generated human body motion can be adjusted.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of a first invention.

FIG. 2 is a block diagram showing the configuration of an embodiment of the second invention.

FIG. 3 is a diagram showing an example of a format of a text action description file.

FIG. 4 is a diagram showing an example of operation / voice time data output by the operation voice synchronization apparatus of the embodiment.

5 is a block diagram showing details of the operation pattern generation device in FIG. 1. FIG.

FIG. 6 is a flowchart showing a flow of processing for generating a human body motion pattern by the motion pattern generation device of FIG.

[Explanation of symbols]

 1 Natural Language Analyzer 2 Behavior Pattern Generator 3 Behavior Degree Generator 4 Behavior Voice Synchronizer 5 Behavior Video Generator 6 Voice Synthesizer 7 Behavior Voice Generation Command / Text Converter 8 Sentence Behavior Description File Storage 9 Text / Action Speech generation command conversion device 21 Verb / action pattern dictionary 22 Action pattern search unit 23 Count generator 24 History storage unit 25 Action pattern matching unit 26 Action pattern constraint dictionary 31 Modifier / action degree dictionary 100 A device 200 B device

Claims (3)

[Claims] 1. A natural language analysis means for extracting words such as verbs and adverbs from texts, a verb / motion pattern dictionary for describing correspondence between words representing motions such as verbs and human motion patterns, and the natural language analysis means. A motion pattern generating means for searching the verb / motion pattern dictionary by using the verb extracted in 1. to generate a human motion pattern, and a modifier for describing the correspondence between a modifier such as an adverb that modifies the verb and the degree of motion. A motion degree dictionary, a motion degree generation means for searching the modifier / motion degree dictionary by using the modifiers extracted by the natural language analysis means, and generating a motion degree of the human body motion pattern; and a motion video output. Calculates the appearance position of the verb in the text to synchronize with the synthetic speech output from the action time data associated with the start time of action generation and the length of the text Operation voice synchronization means for outputting voice time data including the reading time of the text to be read, and time-series video data of the human body action generated and displayed by inputting the action generation command including the human body action pattern, the degree of action and the action time data A human body motion from a text, comprising: a motion video generation means for outputting to the means; and a voice synthesis means for inputting a voice generation command including the text and the voice time data and outputting a voice by a regular voice synthesis method. Voice generation system. 2. The human body motion voice generation system from text according to claim 1, wherein the motion generation command and the voice generation output from the motion pattern generation means, the motion degree generation means and the motion voice synchronization means. An action voice generation command / text conversion means for converting an instruction and a sentence action description file, a sentence action description file storage means for storing the sentence action description file converted by the action voice generation command / text conversion means, and the sentence A text / motion voice generation command conversion unit for converting the text motion description file from the motion description file storage unit into the motion generation command and the voice generation command and outputting to the motion video generation unit and the voice synthesis unit; The text that is characterized in that the text behavior description file can be modified with the editor of Human motion sound generation system et al. 3. The human-body motion voice generation system from text according to claim 2, wherein the text-motion description file is human-readable text.