JP2005216087A - Electronic mail reception device and electronic mail transmission device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To express feeling expression of an electronic mail transmitter by face image output and synthesized voice output on a reception side. <P>SOLUTION: This electronic mail reception device 14 has: a feeling expression data extraction part 12 extracting feeling expression data from a mail sentence when the feeling expression data showing feeling are inserted inside the received mail sentence and when simultaneously received attached data are base face data showing a position and a size of each portion showing a characteristic of a person face, and generating a feeling parameter according to the kind of the feeling shown thereby; a face data reproduction part 13 supplied with the base face data and the feeling parameter, performing a change on the basis of the base face data and the feeling parameter, and outputting an image as a face image; and a voice synthesis output part 11 supplied with the feeling parameter, and imparting variation to intonation in voice synthesis on the basis thereof to perform the voice synthesis. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an e-mail receiving apparatus and an e-mail transmitting apparatus that can read out a mail sentence with synthesized speech. In particular, the present invention also transmits and receives data related to the emotional expression of the sender, and the receiving side is based on this emotional expression data The present invention relates to an e-mail receiving apparatus and an e-mail transmitting apparatus capable of displaying a face image and expressing emotions in synthesized speech.

  In recent years, e-mail has been widely used as a communication means for exchanging requirements with the other party. However, unlike the telephone, this has a drawback that it is difficult to read the other party's feelings. In order to convey more detailed and diverse information, digitalized photos (still image information), moving image information, or audio information can be sent as e-mail attachments as file data. Generally done.

Further, Patent Document 1 discloses an e-mail receiving apparatus and an e-mail system that can read out a mail sentence accompanied by emotions such as joy, anger, and sadness by voice synthesis to convey various emotion expressions of the sender to the other party. Has been.
Further, for example, Japanese Patent Application Laid-Open No. 2003-228867 discloses a composite image display system that enables display as intended by the sender of the mail text when displaying the synthesized speech or the face composite image based on the mail text on the e-mail receiving side. It is disclosed.
JP 2001-34280 A Japanese Patent No. 32999797

When the sender wants to accurately convey his / her state to the communication partner, as described above, it is conceivable that the still image information or the moving image information and the audio information are attached to the mail and transmitted.
However, when still image information and audio information are attached, the information cannot be temporally linked. When moving image information and audio information are attached, the two can be linked but the data capacity is extremely large. However, when the transmission / reception line is a telecommunication network such as the Internet, problems such as a heavy load on the traffic become remarkable, which is not preferable in terms of cost and practical use.
This has the same problem in the means for attaching still image information or moving image information by e-mail using the invention disclosed in Patent Document 1.

  In the invention disclosed in Patent Document 2, face image information can be transmitted as a parameter. In this respect, the problem as in Patent Document 1 is improved. However, in the case of the present invention, there is a problem that it is not possible to take a flexible measure such as instantly changing the facial expression when the emotion changes in the mail sentence.

  The present invention has been made in order to solve the above-described problems, and as a means for the sender to convey the feelings about the mail sentence to the receiver, the face image representing the human face is not represented by the face image data itself. Send only information about the part that expresses the characteristics of the facial expression, and insert an identifier indicating the type of emotion in the email text of the sender, inserting it in the text of the email text, that is, where you want to express the emotion. Thus, when receiving a voice message by voice synthesis while displaying a face image on the receiving side, it is possible to output a synthesized voice and face image reflecting emotions at the timing intended by the sender An object of the present invention is to provide a device and an electronic mail transmission device.

  The present invention comprises the following means (1) and (2) in order to solve the above-mentioned problems.

(1) When text data as a mail sentence and basic face data consisting of parameters indicating the characteristics of a human face attached to the mail sentence are received, the received text data is subjected to speech synthesis processing and Comparing with a predetermined character string determined as an emotion expression to be extracted in advance from a sentence of the received text data, an e-mail receiving device comprising a speech synthesis output unit for outputting synthesized data by voice The emotion expression data of the sender of the e-mail sentence is extracted, and an emotion expression data extraction unit that generates an emotion parameter according to the type of emotion indicated by the emotion expression data, and the received basic face data is generated A face data reproducing unit that changes the expression according to the emotion of the sender based on the emotion parameter and outputs the changed face image as an image. Both the voice synthesizing output unit, electronic mail receiving apparatus, wherein said by being configured to provide a change in intonation based on the supplied emotion parameters from emotional expression data extraction unit when the speech synthesis process.

(2) An e-mail transmitting apparatus for transmitting to the e-mail receiving apparatus according to claim 1, wherein basic face data including parameters of size and arrangement position of each part for determining a facial expression of a human face is generated. An e-mail transmission apparatus comprising a basic face data creation unit and configured to transmit the generated basic face data attached to the text data.

According to the first aspect of the present invention, the e-mail sender creates basic face data representing, for example, his / her face, and inserts emotion expression data for expressing a predetermined emotion in the mail sentence. When the e-mail is transmitted together with the basic face data, the recipient of the e-mail browses the received e-mail text and based on the type of emotion indicated in the emotion expression data inserted in the e-mail text. In addition, it is possible to change emotions in reading out a mail sentence by voice synthesis, and to appropriately display a face image expressing each emotion.
As a result, the recipient of the e-mail can obtain a special effect that the emotion that the mail sender wants to express can be richly reproduced by the face image and the synthesized voice.

  According to the second aspect of the present invention, as described above, the e-mail sender creates basic face data, and inserts emotion expression data for expressing a predetermined emotion in the sentence of the mail sentence, and As long as the e-mail is sent together with the basic face data, the basic face data to be attached to the e-mail is not an image itself but a predetermined parameter, and emotion expression data is also inserted into the sentence. In addition to this, it is possible to extremely reduce the amount of data to be transmitted and to achieve an e-mail transmission apparatus with a very small transmission load on the telecommunication line.

  Further, as described above, the creation of an e-mail has the ease of operation on the sender side that a desired emotion can be expressed at a desired location simply by inserting emotion expression data into the location where the emotion is to be expressed.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, FIG. 1 is a block diagram illustrating functions of an electronic mail system according to an embodiment of the present invention. In the figure, 1 is a text input unit for creating a mail sentence while the sender specifies emotion expression data to be described later, and 2 is a basic face data related to a face image transmitted together with the mail sentence. A basic face data creation unit 3 for creating, a basic face data storage unit 3 for storing the basic face data created by the basic face data creation unit 2, and a mail sentence created by the text input unit 1 A mail transmission unit 5 for attaching the basic face data stored in the basic face data storage unit 3 to enable transmission as an electronic mail, and 5 for outputting the transmission enabled electronic mail to a communication line described later. The first communication interface (I / F) unit, and the configuration according to 1 to 5 is the e-mail transmission unit 6.

  Reference numeral 7 denotes a second communication I / F unit for receiving the e-mail transmitted from the first communication I / F unit 5 of the e-mail transmission unit 6 via the communication line, and 8 denotes a second communication I / F unit. A mail receiving unit for extracting the mail text and basic face data from the electronic mail received by the communication I / F unit 7, and 9 is a mail for storing the mail text and basic face data extracted by the mail receiving unit 8. The storage unit 10 selects the mail text and basic face data stored in the mail storage unit 9 by the receiver, reads them out, and supplies them to a speech synthesis output unit and a face data playback unit, which will be described later. The mail text can be browsed by the recipient with respect to the mail text, 11 is supplied with the text read by the mail browsing section 10 and is subjected to speech synthesis processing. Expression The emotion parameter described below is fed back from the emotion expression data extraction unit and reflected in the voice synthesis process of the mail text supplied from the mail browsing unit 10 to output voice with emotion. The voice synthesis output unit 12 extracts the emotion expression data inserted in the sentence while analyzing the mail sentence supplied from the voice synthesis output unit 11, and voice-synthesizes the emotion parameter based on the emotion expression data. An emotion expression data extracting unit 13 for supplying to the output unit 11 and a face data reproducing unit to be described later, 13 is the basic face data supplied from the mail browsing unit 10 based on the emotion parameters supplied from the emotion expression data extracting unit 12. A face data reproduction unit for outputting the basic face data while changing the basic face data. 4.

  Further, in the figure, reference numeral 15 denotes a first communication I / F unit 5 and a second communication I / F unit 7 in order for the e-mail receiving unit 14 to receive the e-mail transmitted from the e-mail transmitting unit 6. A communication line to be connected.

  As described above, in the electronic mail system, the electronic mail transmitting unit 6 and the electronic mail receiving unit 14 are connected via the communication line 15.

  FIG. 2 is a block diagram illustrating the electronic mail system of FIG. 1 as a specific hardware configuration. In the figure, in the electronic mail system, an electronic mail transmitting device 21 and an electronic mail receiving device 22 are connected via an electric communication line 23.

  First, in the electronic mail transmitting device 21, 201 is an input unit for a sender to input a mail text or operate the electronic mail transmitting device 21, 202 is a control unit for controlling the entire electronic mail transmitting device 21, 202a is a CPU that is included in the control unit 202 and is controlled by a transmission control program stored in a storage unit 203, which will be described later, 203 is a storage unit that stores data processed by the control unit 202 and the transmission control program, and 204 Is a display unit for displaying the processing result of the transmission control program controlled by the control unit 202 and the mail text input by the sender at the input unit 201, and 205 is an electronic communication line for sending the email supplied from the control unit 202 23 is a communication unit for outputting to the terminal 23.

  In the electronic mail receiving apparatus 22, 206 is a communication unit for receiving electronic mail supplied from the electronic mail transmitting apparatus 21 via the electric communication line 23, and 207 is for controlling the entire electronic mail receiving apparatus 22. A control unit 207a is included in the control unit 207 and is controlled by a reception control program stored in a storage unit 208, which will be described later, and 208 is a memory that stores data processed by the control unit 207 and the reception control program. 209, a display unit for displaying the processing result of the reception control program controlled by the control unit 207 and the content of the e-mail received by the communication unit 206, and 210 for operating the e-mail receiving device 22. An input unit 211 is a sound output unit for generating sound.

  As described above, the relationship between the functional block and the hardware block from FIGS. 1 and 2 is as follows. That is, the hardware corresponding to the functional blocks of the e-mail transmission unit 6, the e-mail reception unit 14, and the communication line 15 are the e-mail transmission device 21, the e-mail reception device 22, and the telecommunication line 23, respectively.

  The text input unit 1 and the basic face data creation unit 2 in the e-mail transmission unit 6 are realized by the control of the transmission control program by the CPU 202a of the control unit 202 based on the input from the input unit 201, and the basic face data storage The unit 3 is realized by the storage unit 203, the mail transmission unit 4 is realized by the control of the transmission control program by the CPU 202 a of the control unit 202, and the first communication I / F unit 5 is realized by the communication unit 205.

  The second communication I / F unit 7 in the e-mail receiving unit 14 is realized by the storage unit 208, and the mail receiving unit 8, the mail browsing unit 10, and the emotion expression data extracting unit 12 are executed by the CPU 207 a of the control unit 207. The mail storage unit 9 is realized by the storage unit 208, and the voice synthesis output unit 11 is realized by the control of the reception control program by the CPU 207a of the control unit 207 and the voice output unit 211, and is realized by the control of the reception control program. The reproduction unit 13 is realized by the control of the reception control program by the CPU 207 a of the control unit 207 and the display unit 209.

  According to the configuration example shown in FIG. 2, the e-mail transmission device 22 and the e-mail reception device 23 may be configured by a personal computer having the transmission control program or the reception control program operable on an operating system. Alternatively, each configuration of FIG. 2 may be configured as an electronic mail terminal configured with dedicated hardware and a program.

The operation of the e-mail system having the above-described configuration will be described with reference to the drawing, divided into the e-mail transmission unit 6 and the e-mail reception unit 14.
First, the operation of the e-mail transmission unit 6 will be described. The sender needs to create basic face data by the basic face data creation unit 2 in advance. This basic face data is a numerical value indicating the size of each part of the face, such as the outline of the human face, eyes, mouth, position of placement, etc., for example, the sender has seen on his face Basic face data can be created.

  FIG. 3 shows a data configuration example of the basic face data. This figure shows the contours, hairstyle, eyebrows, eyes, and mouth that make up the face, and the size (width and height or thickness) of each part and the two-dimensional distance from the center of the face. The data structure which digitized the arrangement position by is shown. In the figure, for example, the mouth is positioned below the center position of the face by an amount indicated by my, the thickness is mh, and the width is mw. Similarly, for the eyebrows and eyes, these arrangement positions and sizes are indicated by numerical values.

  For the face outline, for example, a spline curve is formed, and fr indicates where the relay point is.

  For hairstyles, there are a wide variety of hairstyles and it is difficult to quantify them even if you want to set the shape according to your taste. You may make it select arbitrarily.

  Further, attribute data indicating colors and patterns may be set for the contour, hairstyle, eyebrows, eyes, and mouth, and the basic face data may be added to the size and arrangement position data.

  The sender can display a basic face image on the display unit 204 in order to generate the basic face data, and can determine a numerical value related to each part by operating the input unit 201. For example, a basic face data creation screen as shown in FIG. 4 is displayed on the display unit 204 as a GUI (Graphical User Interface) by the control of the control unit 202 by the transmission control program, and the GUI operation from the input unit 201 is performed. While looking at the face image, the size and position of each part can be changed, and a hairstyle can be selected. In the figure, the left side is the face image that the sender confirms, and the right side displays a slide bar for changing the size and arrangement position of each part, hairstyle candidate options, and the like.

The sender can generate desired basic face data while confirming that the result of the operation is reflected in the face image on the left side of the figure by operating these with the input unit 201.
The basic face data generated in this way is stored in the basic face data storage unit 3. As a result, once generated basic face data can be repeatedly used in mail transmission.

  The basic face data once stored in the basic face data storage unit 3 is read again by the basic face data creation unit 2 according to the designation of the sender, and the size and the size of each part by the GUI of the basic face data creation screen are displayed. The arrangement position is corrected, and can be stored again in the basic face data storage unit 3 as data replacement or stored as new data.

  Next, creation of the emotion expression data will be described. In an e-mail sentence created by a sender, the emotion held by the sender usually changes with the contents of the sentence. Therefore, it is desirable that the emotion expression data is designated each time according to the contents of the e-mail sentence. . In the present embodiment example, the emotion expression data is inserted into the mail sentence as a predetermined character string determined according to the emotion. In other words, when creating a mail text, the sender inserts a predetermined character string determined according to the emotion into a location where the emotion is desired to be expressed or a location where the emotion is to be changed so as to indicate the desired emotion at the exact location. To.

  The predetermined character string is, for example, a character string as shown in FIG. The character string shown in the figure is a so-called “emoticon”, and the entire shape of the character string looks like a face expressing human emotions. These predetermined character strings may be special character strings and symbols determined in advance in addition to the emoticons. In this case, although the details will be described later, the voice text is output by voice synthesis processing by the voice synthesis output unit 11 of the e-mail receiving unit 14, but the predetermined character string is pronounced as an unknown word by the voice synthesis processing. For example, a character string that cannot be read aloud is set.

  After the e-mail text and the basic face data into which the emotion expression data is inserted are prepared as described above, the e-mail transmission unit 4 attaches the basic face data to the e-mail text and sends the first communication I as an e-mail. This is sent to the communication line 15 through the / F unit 5. The e-mail sent to the communication line 15 is received by the e-mail receiving unit 14 as a transmission destination via a mail server, a provider, etc. (not shown) connected to the communication line 15.

  Next, the operation of the e-mail receiving unit 14 that has received the e-mail transmitted from the e-mail transmitting unit 6 will be described. The e-mail receiving unit 14 reads the e-mail into the mail receiving unit 8 through the second communication I / F unit 7, extracts the e-mail text and the basic face data attached thereto, and stores the e-mail storage unit. 9 is recorded.

  Thereafter, the mail text and the basic face data recorded in the mail storage section 9 are read out to the mail browsing section 10 by the operation of the recipient, and the mail text can be displayed and browsed.

  Next, speech synthesis processing of the mail sentence and processing of the emotion expression data inserted into the mail sentence will be described. The mail browsing unit 10 supplies the read mail text to the speech synthesis output unit 11 and supplies the read basic face data to the face data reproduction unit 13.

  The voice synthesizing output unit 11 outputs the supplied mail text while converting the supplied mail sentence into voice data by voice synthesis processing. In this case, the voice synthesis process can convert the text data of the mail text into digital voice data with a well-known technique. For example, this function is realized as voice synthesis software as a specific mode. This is realized by converting the mail text into digital voice data under the control of the CPU 207 a and supplying the digital voice data to the voice output unit 211.

  Note that, as described above, when the emotional expression data is extracted from the sentence during the voice synthesis process of the mail sentence in the voice synthesis output unit 11, this data is set as a character string that cannot be pronounced. The voice expression data itself is not pronounced by the voice output unit 211.

  The mail text that is voice-synthesized by the voice synthesis output unit 11 and output as a voice is supplied to the emotion expression data extraction unit 12 in synchronization with the voice synthesis process. The emotional expression data extraction unit 12 searches the emotional expression data while sequentially reading the mail text supplied from the speech synthesis output unit 11. This is because the emotion expression data is composed of a predetermined character string, and the emotion expression data can be extracted by examining the character code of the read character.

  When the emotion expression data extraction unit 12 extracts the emotion expression data, the emotion expression data extraction unit 12 supplies parameters corresponding to the type of emotion to the speech synthesis output unit 11 and the face data reproduction unit 13. In this case, when the emotion expression data shown in FIG. 5 is used as an example of the parameter corresponding to the type of emotion, “joy” is 0, “anger” is 1, “sorrow” is 2, and “easy”. Is a definition of the type of emotion numerically, such as 3.

  The speech synthesis output unit 11 supplied with the emotion parameter from the emotion expression data extraction unit 12 changes the intonation such as voice inflection, tone, and tone to express the emotion indicated by the incoming parameter value. Is input to the voice synthesis process, and the emotion shown above can be included in the voice data to be generated.

  Further, as described above, the face data reproduction unit 13 is supplied with the basic face data from the mail browsing unit 10 and displays it as a face image. Then, when the emotion parameter is supplied from the emotion expression data extraction unit 12, the basic face data is changed and re-displayed based on the incoming parameter value so as to obtain a face image that expresses the emotion indicated by the parameter value. . FIG. 6 shows examples of the face images corresponding to “joy”, “anger”, “sorrow”, and “easy”, respectively.

As described above in detail, the e-mail receiving apparatus according to the embodiment of the present invention creates basic face data such that the mail sender represents, for example, his / her face, and expresses a predetermined emotion in the mail sentence. When the emotional expression data for expression is inserted and the e-mail is transmitted together with the basic face data, the recipient of this mail browses the received mail sentence and the emotional expression data inserted in the mail sentence Based on the types of emotions shown in the above, it is possible to cause emotional changes to be read out by mail synthesis by speech synthesis, and to appropriately display face images expressing each emotion.
Thus, the recipient of the e-mail can reproduce and receive the emotion that the mail sender wants to express with the face image and the synthesized voice.

  In addition, as described above, with respect to the e-mail transmission device, the e-mail sender creates basic face data, and inserts emotion expression data for expressing a predetermined emotion in the sentence of the e-mail sentence. As long as an email is sent together with the data, the basic face data attached to the email is not an image itself but a predetermined parameter, and emotion expression data is also inserted into the text. The amount of data to be performed can be extremely reduced, and the transmission load applied to the telecommunication line can be extremely reduced.

  In addition, as described above, the creation of an e-mail has the ease of operation on the sender side that a desired emotion can be expressed at a desired location simply by inserting emotion expression data into the location where the emotion is to be expressed.

  In the configuration example shown in FIG. 2, the e-mail transmission device 22 and the e-mail reception device 23 may be configured by a personal computer having the transmission control program or the reception control program operable on an operating system. 2 or may be configured as an e-mail terminal configured with dedicated hardware and a program. However, the e-mail transmitting device 22 and the e-mail receiving device 23 may have a mail transmitting function and a mail. The electronic mail transmission / reception apparatus may be configured to have both reception functions. This is sufficiently possible without needing to explain technical details in view of this configuration.

  In addition, as an example of the configuration of emotion expression data, “emoticon” as shown in FIG. 5 is given as an example, but in addition to means for setting and inserting such a special character string, an actual mail You may make it make the emotion expression data extraction part 12 recognize the expression regarding the emotion which appears in a sentence as emotion expression data. For example, when words indicating emotions such as “fun”, “happy”, “crying”, “angry” are used in the mail text, the emotion expression data extraction unit 12 reserves these in advance. You may make it control so that an emotion parameter may be output by collating and extracting a word. In this case, the sender need not bother to insert a predetermined character string such as “emoticon” into the mail text when composing the mail.

As described above, the e-mail receiving apparatus and e-mail transmitting apparatus according to the present invention can easily transmit emotions related to the sender's mail text to the receiver side by outputting a face image and reading aloud. The present invention is useful for information communication such as a video phone and an electronic bulletin board of a Web site using this.
Moreover, since pronunciation and an image can be changed according to a sentence, it is useful also in educational equipment for infants, for example.

It is the block diagram which illustrated the function of the electronic mail receiving device and electronic mail transmitting device concerning an embodiment of the invention. FIG. 2 is a block diagram illustrating the e-mail receiving device and the e-mail transmitting device of FIG. 1 as a specific hardware configuration. It is the figure which showed the data structural example of the basic face data. It is the example of a display which displayed the basic face data creation screen as GUI. The character string showing emotion expression data is illustrated. The example of a display of the facial image showing an emotion is shown.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Text input part 2 Basic data creation part 3 Basic face data memory | storage part 4 Mail transmission part 5 1st communication I / F part 6 E-mail transmission part 7 2nd communication I / F part 8 Mail receiving part 9 Mail storage part DESCRIPTION OF SYMBOLS 10 Mail browsing part 11 Speech synthesis output part 12 Emotion expression data extraction part 13 Face data reproduction part 14 E-mail reception part 15 Communication line

Claims (2)

When text data as mail text and basic face data consisting of parameters indicating the characteristics of a human face attached to the mail text are received, the received text data is subjected to speech synthesis processing and this synthesis processing is performed. An e-mail receiving device having a voice synthesis output unit for outputting the voice data
The sentiment expression data of the sender of the e-mail sentence is extracted by comparison with a predetermined character string determined as an emotion expression to be extracted in advance from the sentence of the received text data, and the emotion type indicated by the emotion expression data An emotion expression data extraction unit that generates emotion parameters according to
The received basic face data is changed to an expression according to the sentiment of the sender based on the generated emotion parameter, and includes a face data reproduction unit that outputs the changed face image, and
The e-mail receiving apparatus, wherein the voice synthesis output unit is configured to change intonation based on an emotion parameter supplied from the emotion expression data extraction unit when the voice synthesis process is performed. An e-mail transmitting device for transmitting to the e-mail receiving device according to claim 1,
It has a basic face data creation unit that generates basic face data including parameters of size and arrangement position of each part that determines the facial expression of a human face, and transmits the generated basic face data attached to the text data. An electronic mail transmitting device characterized by having a configuration.

Images