JP4829696B2 - Conversation support system - Google Patents

Description

  The present invention relates to a conversation support system.

  In order to attract attention and convey information efficiently and effectively in a conversation hall, the speaker needs presentation and conversation skills. Therefore, when the speaker does not have sufficient conversation skills, information cannot be effectively transmitted, and the progress of the conversation is hindered.

  2. Description of the Related Art Conventionally, a device has been developed that closely communicates with each other by matching the line of sight with a remote party (see Patent Document 1).

JP-A-5-219501

  In order to activate the conversation with the participant in the conversation area, it is effective to direct the line of sight of the participant in the conversation area to the speaker. However, in the technique disclosed in Patent Document 1, By attracting the attention of participants or diverting attention, it is not possible to induce speech promotion or deterrence in the conversation area.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a conversation support system that can guide a participant's line of sight in order to activate conversation with a participant in a conversation hall. To do.

According to the first invention, the utterance state of each of the plurality of participants in the conversation area is acquired based on information from the sound collecting device, and the line- of- sight direction of each participant is acquired based on the camera image from the imaging device. , A conversation field state detecting means for detecting a conversation field state including a speech state and a line-of-sight direction of each participant ; And searching for the database on the condition that the conversation field state detected by the conversation field state detecting means and the database are stored in association with each other, to guide the gaze of the participant corresponding to the detected conversation field state A recommended gaze direction calculating means for calculating recommended gaze direction information indicating a gaze direction to the target, and a gaze direction indicated by the recommended gaze direction information calculated by the recommended gaze direction calculation means Gaze direction control means for controlling a predetermined device so that the gaze of the participant in the conversation field is guided to the conversation field, a conversation field state parameter defined by a part of the conversation field state, and the detected conversation field state A conversation field state parameter match detection unit that detects that the information included in the history matches, and a conversation that is detected to match when detected by the conversation field state parameter match detection unit The direction of the line of sight of the participant included in the field state is evaluated, and based on the evaluation result, the line of sight of the participant is used as the recommended line-of-sight direction information and associated with the conversation field state parameter detected to match, A conversation support system comprising database update means for storing in the database .
According to a second invention, in the first invention, the conversation field state detection means is configured to transmit a conversation field state including a participant's speech state and a participant's line-of-sight direction in a remote conversation field via a network. A conversation support system characterized by including receiving.
According to a third invention, in the first invention or the second invention, the gaze direction control means participates so that the gaze direction of the participant is in the gaze direction indicated by the calculated recommended gaze direction information. A conversation support system comprising means for rotating a person's chair.
According to a fourth invention, in the first invention or the second invention, the line-of-sight direction control means is configured so that the participant's line of sight faces the line-of-sight direction indicated by the calculated recommended line-of-sight direction information. A conversation support system characterized by guiding a line of sight by applying a spotlight or projecting an image of an arrow.
According to a fifth invention, in any one of the first to third inventions, the display means for displaying a plurality of participants and the recommended gaze direction information detected by the recommended gaze direction detection means. A conversation support system, further comprising: a process of darkening an image of a participant displayed on the display unit other than the participant in the viewing direction shown.

  ADVANTAGE OF THE INVENTION According to this invention, in order to activate the conversation with the participant of a conversation place, the conversation assistance system which can guide | inspect a participant's eyes | visual_axis can be provided.

Hereinafter, a conversation support system according to an embodiment of the present invention will be described with reference to the drawings.
1 and 2 are diagrams for explaining a use environment of a conversation support system according to an embodiment of the present invention.
FIG. 1 shows a state of a conversation field detected by a sensing unit including a sound collection device such as a microphone and an image pickup device 2 such as a camera, and a video projection device 3 such as a projector based on the detected state of the conversation field. The arrow-type recommended line-of-sight direction 6 that guides the line of sight of the participants 5-2 to 5-4 other than the participant 5-1 who is the speaker on the table 4 in the conversation area is directed to the speaker 5-1. It is a figure for demonstrating the case where it displays.

  FIG. 2 shows the state of a conversation field detected by a sensing unit including a sound collection device such as a microphone and an imaging device 2 such as a camera, and a video projection device 3 such as a spotlight based on the detected state of the conversation field. Thus, by applying the spotlight 7 to the speaker 5-1 who is a participant, the line of sight of the participants 5-2 to 5-4 other than the participant 5-1 is guided to the speaker 5-1 Is.

FIG. 3 is a diagram showing a configuration for realizing the conversation support system according to the embodiment of the present invention.
The sound collection device 1 is, for example, a microphone array and a plurality of microphones, and is attached to a conversation place, and collects speech of each participant in the conversation place.
The imaging device 2 is a camera and images each participant in the conversation hall.

  The other sensing device 11 acquires the conversation field state by a method other than the sound collection device 1 and the imaging device 2. For example, thermometer, hygrometer, sound level meter, illuminance meter, seismic intensity meter, odor measuring device using multiple gas sensors, and heart rate meter, respiratory rate measuring device, published measuring device, eyeball to acquire the psychological state of participants It is a motion measurement device, a gaze detection device for each participant, an electroencephalogram detection device, a myoelectric detection device, a sphygmomanometer, a blood component detection device, a walking or seating balance detection device.

  The computer 21 includes a conversation field state detection unit 22, a recommended gaze direction calculation unit 26, a parameter input unit 31 for a prescribed conversation field state, a match determination unit 32 for the prescribed conversation field state and the conversation field state, a conversation field state-recommended gaze direction database. An update unit 33 and a conversation field state-recommended line-of-sight direction database 34 are provided.

The conversation field state detection unit 22 includes the utterance state of each participant in the conversation field and the gaze direction of the participant from at least one information from the sound collection device 1, the imaging device 2, and other conversation field state detection devices 11. Detect the conversation area state.
The sound collecting device 1, the imaging device 2, and the other conversation field state detection device 11 may be attached to the same conversation place or may be attached to remote conversation places. Information from the imaging device 2 and other conversation field state detection device 11 is input to the computer 21 via the network.

Specifically, the conversation field state detection unit 22 includes an utterance state detection unit 23, a gaze direction detection unit 24, and other conversation field state detection units 25.
The utterance state detection unit 23 detects the utterance state of the participants in the conversation hall. For example, based on information from the sound collection device 1, it is always sensed which participant is speaking. This detects who started speaking, stopped speaking, and who is speaking at the present time. Moreover, the utterance state may be detected more accurately from the movement of each participant's mouth by using the camera video information from the imaging device 2.

  The line-of-sight direction detection unit 24 detects the line-of-sight direction of each participant from the camera image from the imaging device 2 based on the direction of the head of each participant in the camera image. When the “eye” region of the participant is captured in the camera image, the participant's line-of-sight direction may be detected more accurately.

The other conversation field state detection unit 25 detects a conversation field state other than the speech state detected by the speech state detection unit 23 and the conversation field state in the gaze direction of the participant detected by the gaze direction detection unit 24.
For example, when guessing the right to speak in a conversation hall where raising hands is regulated, the utterance “Yes” accompanying raising hands is collected from the camera image from the imaging device 2 when the right of raising the hand is estimated. A raised hand state detection unit that estimates the right to speak by detecting voice information from the sound device 1 is provided.

  The recommended gaze direction calculation unit 26 searches the conversation field state-recommended gaze direction database 34 on the condition of the conversation field state detected by the conversation field state detection unit 22, thereby participating corresponding to the detected conversation field state. The recommended line-of-sight direction information indicating the line-of-sight direction toward the subject to which the person's line of sight is to be guided is calculated.

Further, the recommended gaze direction calculation unit 26 is based on the recommended gaze direction information indicating the calculated gaze direction, so that the gaze of the participant in the conversation field is directed to the recommended gaze direction. The chair rotation device 12 for rotating the chair on which the person is sitting is controlled.
As a method for guiding a participant's line of sight, there are roughly a method for indirectly guiding the user's line of sight and a method for moving the user's line of sight directly. The former includes a method in which only a speaker is visually focused by applying a spotlight with a projector, and a method in which a line of sight is guided in a direction to be focused by projecting an arrow image. In the latter case, there is a method in which the line of sight is forcibly focused by rotating the chair on which the participant is sitting. Needless to say, these methods may be used in combination.

  You can also use both methods, such as a method of forcibly moving the direction of the body by presenting “the chair being forcibly pulled by the leash” by the rotation of the chair and the movement of the leash effect. Good. Furthermore, in addition to information presentation by a projector, more effective guidance is possible by reproducing sound image localization by a plurality of speakers.

The parameter input unit 31 in the prescribed conversation place state is an interface for inputting a parameter indicating at least a part of the prescribed conversation place state input by the user.
The coincidence determination unit 32 between the prescribed conversation place state and the conversation place state includes the parameters of the prescribed conversation place state input by the parameter input unit 31 of the prescribed conversation place state and the conversation place state detected by the conversation place state detection unit 21. It detects that the information contained in the history matches.

  The conversation field state-recommended line-of-sight direction database update unit 33 evaluates the line of sight of the participants included in the conversation field state that is detected to match. For example, the direction of the line of sight of the participant included in the conversation field state where the matching is detected is evaluated by the data string for measuring the effect of the line of sight presentation.

  Based on the evaluation result, the participant's line-of-sight direction is set as the recommended line-of-sight direction and is stored in the conversation field state-recommended line-of-sight direction database 34 in association with the conversation field state parameter detected to match. For example, the line-of-sight direction with the highest evaluation is set as the recommended line-of-sight direction, and is stored in the conversation field state-recommended line-of-sight direction database 34 in association with the parameter of the prescribed conversation field state.

Next, the operation of the conversation support system according to the embodiment of the present invention will be described with reference to the flowchart of FIG.
The conversation support system according to the present embodiment operates by detecting a situation in which someone starts speaking or a situation in which the user is not focused. For example, it can be used for speech in brainstorming and meetings (the order of utterances is kept to some extent).

From the at least one information from the sound collecting device 1, the imaging device 2, and the other conversation field state detection device 11, the conversation field state detection unit 22 includes the utterance state of each participant in the conversation field and the gaze direction of the participant. A conversation place state is detected (S1).
Next, the recommended gaze direction calculation unit 26 searches the conversation field state-recommended gaze direction database 34 on the condition of the conversation field state detected by the conversation field state detection unit 22 (S2), thereby detecting the detected conversation. The recommended line-of-sight direction information indicating the line-of-sight direction to the target for which the line-of-sight of the participant corresponding to the field condition is to be collected is calculated (S3).

  Thereafter, the recommended gaze direction calculation unit 26, based on the recommended gaze direction information indicating the detected gaze direction, the video projector 3 or the participation of the conversation field so that the gaze of the participant in the conversation field is directed to the recommended gaze direction. The chair rotating device 12 for rotating the chair on which the person is sitting is controlled (S4).

For example, when there is only one speaker, recommended gaze direction information that causes the speaker to be noted is calculated. In particular, the strength of the control of the gaze direction may be changed according to the degree that the gaze of the other participant is not facing the speaker direction.
For a speaker's collision, if participants who speak at the same time are detected, recommended gaze direction information that determines the speaker who started speaking the earliest as the speaker and focuses on the speaker direction May be calculated.

When a person who desires to speak is hesitant to speak, recommended line-of-sight direction information that causes the speaker to pay attention may be calculated. In this way, by specifying a speaker who has the right to speak, it is possible to suppress utterances other than that speaker.
In the above-described example, the case where control is performed so as to draw attention to the participant's line of sight has been described, but control may be performed so as to suppress speech by diverting attention.

For example, a situation where one of the participants continues to speak and the other participants' utterances are excluded is detected and operated.
Specifically, it can be used in cases where the right to speak does not move despite brainstorming, or where the speech of another participant is always interrupted by one participant. In such a case, a case where the continuous utterance time of the speaker is long or a case where the total utterance time is long is detected, and recommended gaze direction information that diverts attention from the speaker is calculated. In this case, the strength of the control in the line-of-sight direction may be changed according to the ratio of the utterance in the utterance amount of all participants.

  In the above-described embodiment, the case where the visual line of the participant is guided using the video projection device 3 and the chair rotation device 12 is shown, but a display for displaying a plurality of participants participating in the conversation hall is provided, You may make it induce | guide | derive a participant's gaze by performing the process which darkens the image of the participant displayed on displays other than the participant of the gaze direction shown by the calculated recommended gaze direction.

Next, a database construction method for the conversation support system according to the embodiment of the present invention will be described with reference to the flowchart of FIG.
When a parameter for the specified conversation field state is input via the parameter input unit 31 for the specified conversation field state (S11), the match determination unit 32 between the specified conversation field state and the conversation field state inputs the parameter for the specified conversation field state. It is determined whether or not the parameters of the specified conversation field state input by the unit 31 and the information included in the conversation field state history detected by the conversation field state detection unit 21 match. Then, the line of sight of the participant included in the conversation area detected to be matched is evaluated (S13). On the other hand, if they do not match, the process returns to S11.

Then, based on the evaluation result, the gaze direction of the participant is set as the recommended gaze direction, and is stored in the conversation field state-recommended gaze direction database 34 in association with the conversation field state parameter detected to match (S14). .
As the parameter of the prescribed conversation place state, for example, a state in which the voice is not interrupted (a state in which the voice input is 3 seconds or longer in the conversation place) is defined as a parameter in a state where the conversation place is excited. Note that, by defining parameters for other states, it is possible to construct an ideal gaze direction database corresponding to a plurality of conversation place states.

  For example, if the conversation of one participant in a conversation hall continues continuously and there is little whisper of other participants, it is defined that the audience feels bored, and the speech of one participant is It can be defined that the audience understands the user's utterance if it continues continuously and the frequency of whispering of all other participants is high. That is, the parameters of the prescribed conversation place state can be set as appropriate according to the place of conversation to be used.

Therefore, according to the embodiment of the present invention, it is possible to guide the line of sight of the participant in the conversation place according to the state of the conversation place, so that the conversation with the participant in the conversation place can be activated. .
Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the method described in the embodiment is, for example, a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD) as programs (software means) that can be executed by a computer (computer). , MO, etc.), a semiconductor memory (ROM, RAM, flash memory, etc.) or the like, or can be transmitted and distributed via a communication medium. The program stored on the medium side includes a setting program that configures software means (including not only the execution program but also a table and data structure) in the computer. A computer that implements this apparatus reads a program recorded on a recording medium, constructs software means by a setting program as the case may be, and executes the above-described processing by controlling the operation by this software means. The recording medium referred to in this specification is not limited to distribution, but includes a storage medium such as a magnetic disk or a semiconductor memory provided in a computer or a device connected via a network.

It is a figure for demonstrating the utilization environment of the conversation assistance system which concerns on embodiment of this invention. It is a figure for demonstrating the other utilization environment of the conversation assistance system which concerns on embodiment of this invention. It is a figure which shows the structure which implement | achieves the conversation assistance system which concerns on embodiment of this invention. It is a flowchart for demonstrating operation | movement of the conversation assistance system which concerns on embodiment of this invention. It is a flowchart for demonstrating the update method of the database of the conversation assistance system which concerns on embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 22 ... Conversation place state detection part, 26 ... Recommended gaze direction calculation part, 31 ... Parameter input part of prescription | regulation conversation place state, 32 ... Match determination part of prescription | regulation conversation place state and present conversation place state, 33 ... Conversation place state-recommended Gaze direction database update unit.

Claims (7)

The utterance state of each participant in the conversation area is acquired based on information from the sound collection device , the line- of- sight direction of each participant is acquired based on the camera image from the imaging device, and the utterance state of each participant A conversation field state detection means for detecting a conversation field state including a gaze direction ;
A database that stores the conversation field state and recommended line-of-sight direction information that indicates the line-of-sight direction to the subject to which the participant's line of sight is to be guided,
A recommended line of sight indicating a line-of-sight direction to a target to which a participant's line of sight corresponding to the detected conversation field state is to be guided by searching the database on the condition of the conversation field state detected by the conversation field state detection unit Recommended gaze direction calculation means for calculating direction information;
Gaze direction control means for controlling a predetermined device so that the gaze of the participant in the conversation field is guided in the gaze direction indicated by the recommended gaze direction information calculated by the recommended gaze direction calculation means ;
A conversation field state parameter match detection means for detecting that a conversation field state parameter defined by a part of the conversation field state matches information included in the history of the detected conversation field state;
When matching is detected by the conversation field state parameter match detection means, the direction of the line of sight of the participant included in the conversation field state detected to match is evaluated, and based on the evaluation result, A conversation support system, comprising: database update means for storing in the database in association with a conversation field state parameter that is detected to be coincident with the participant's line-of-sight direction as recommended line-of-sight direction information .   2. The conversation field state detection means includes receiving a conversation field state including a speech state of a participant in a remote conversation field and a direction of the line of sight of the participant via a network. Conversation support system.   The line-of-sight direction control means comprises means for rotating the participant's chair so that the line-of-sight of the participant is directed to the line-of-sight direction indicated by the calculated recommended line-of-sight direction information. The conversation support system according to claim 2.   The line-of-sight direction control means directs the line of sight by applying a spotlight with a projector or projecting an image of an arrow so that the line of sight of the participant is directed in the line-of-sight direction indicated by the calculated recommended line-of-sight direction information. 3. The conversation support system according to claim 1, wherein guidance is provided.   Display means for displaying a plurality of participants;
  And a process of darkening an image of a participant displayed on the display unit other than the participant in the line-of-sight direction indicated by the recommended line-of-sight direction information detected by the recommended line-of-sight direction detecting unit. The conversation support system according to any one of claims 1 to 4.   In a conversation support method in a conversation support system comprising a database that stores a conversation field state and recommended line-of-sight direction information indicating a line-of-sight direction to a subject to which a participant's line of sight is to be guided is stored.
  The utterance state of a plurality of participants in the conversation area is acquired based on information from the sound collector, and the sight line direction of each participant is acquired based on the camera image from the imaging device, and the utterance state and sight line of each participant Detecting the conversation state including the direction,
  By calculating the recommended line-of-sight direction information indicating the line-of-sight direction to the target to be guided to the participant's line of sight corresponding to the detected conversation place state by searching the database on the condition of the detected conversation place state,
  Controlling a predetermined device so that the gaze of the participant in the conversation field is guided in the gaze direction indicated by the calculated recommended gaze direction information,
  Detecting that a conversation field state parameter defined by a part of the conversation field state matches information included in the history of the detected conversation field state;
  When a match is detected, the direction of the gaze of the participant included in the conversation field state where the match is detected is evaluated, and the gaze direction of the participant is determined based on the evaluation result. Information is stored in the database in association with a conversation field state parameter that is detected to match
A conversation support method characterized by that.   In a conversation support program used in a conversation support system including a database that stores a conversation field state and recommended line-of-sight direction information indicating a line-of-sight direction to a subject to which a participant's line of sight is to be guided is stored.
  The conversation support program includes the conversation support system,
  The utterance state of each participant in the conversation area is acquired based on information from the sound collector, the direction of each participant's line of sight is acquired based on the camera image from the imaging device, and the utterance state of each participant And the conversation field state including the gaze direction
  By searching the database on the condition of the detected conversation field state, to calculate recommended line-of-sight direction information indicating the line-of-sight direction to the target to be guided to the participant's line of sight corresponding to the detected conversation field state,
  A predetermined device is controlled such that the gaze of the participant in the conversation field is guided in the gaze direction indicated by the calculated recommended gaze direction information;
  Detecting that a conversation field state parameter defined by a part of the conversation field state matches information included in the history of the detected conversation field state;
  When a match is detected, the direction of the gaze of the participant included in the conversation field state where the match is detected is evaluated, and the gaze direction of the participant is determined based on the evaluation result. A conversation support program characterized in that the information is stored in the database in association with a conversation field state parameter detected to match.

Images