JP7393000B2 - Teleconferencing devices, systems, methods and programs - Google Patents

Description

The present invention relates to a remote conference device, system, method, and program.

2. Description of the Related Art A system is known in which a plurality of remote conference terminal devices are connected via a communication network to conduct a remote conference. In recent years, opportunities to hold remote conferences have increased due to requests to reduce various costs associated with moving between locations and an increase in remote work. In a remote conference system, a conference is generally held by connecting a plurality of bases, and information such as audio and data related to the conference is transmitted and received between each base via a communication network.

By the way, some participants are late for the meeting and attend the meeting in the middle of the meeting, but in this case, the mid-participant needs to understand the progress of the meeting up to that point and join the subsequent meeting. Patent Document 1 discloses a technology related to a system that can accommodate late participants in such a remote conference system when only some participants join the conference midway through.

Japanese Patent Application Publication No. 2003-339033

The technology disclosed in Patent Document 1 creates a summary of the conference content when a mid-participant is detected, and sends the summary to the mid-participant in audio or text in order to shorten the time it takes for mid-participants to understand the content of the meeting. It is sent by . However, in the remote conference system according to this technology, a summary is not created unless late participants are detected, so those who were unable to participate in the conference cannot grasp the content of what was said in the conference. Therefore, there is a problem in that although participants who are midway through a meeting can grasp the content of what is said in the meeting in a short time, those who do not participate are unable to grasp the content of what is being said.

The present disclosure aims to provide a remote conference device, system, method, and program that solve these problems.

A remote conference device according to the present disclosure includes a receiving unit that receives the utterances of a conference participant from a communication terminal used by the conference participants, a recording unit that records the utterances as audio data, and a recording unit that records character data from the utterances. a character generation unit that generates a character, a summarization unit that creates a summary of the audio data and the text data when a conference participant participates, and delivers at least one of the audio data and text data of the summary to the communication terminal. and a transmitting section.

A remote conference system according to the present disclosure includes a communication terminal used by a conference participant, and a remote conference device communicably connected to the communication terminal, and the remote conference device connects a communication terminal used by the conference participant to a remote conference device. a receiving unit that receives the utterances of the conference participants; a recording unit that records the utterances as audio data; a character data generation unit that generates character data from the utterances; and a summarization unit that creates a summary of the text data, and a transmission unit that delivers the summary to the communication terminal as at least one of audio data and text data.

A remote conference method according to the present disclosure includes the steps of: receiving the utterances of the conference participants from a communication terminal used by the conference participants; recording the utterances as audio data; and generating character data from the utterances. and, when a conference participant participates, creating a summary of the audio data and the text data, and delivering at least one of the audio data and the text data of the summary to the communication terminal. .

A program according to the present disclosure includes a process of receiving a statement from a conference participant from a communication terminal used by the conference participant, a process of recording the statement as audio data, and a process of generating character data from the statement. , when a conference participant participates, the information processing device performs a process of creating a summary of the audio data and the text data, and a process of delivering at least one of the audio data and the text data of the summary to the communication terminal. It is to be executed by

According to the present disclosure, it is possible to provide a remote conference device, a system, a method, and a program that allow non-participants and mid-participants to grasp the contents of the conference in a short time.

1 is a schematic configuration diagram of a remote conference device according to Embodiment 1 of the present disclosure. FIG. 2 is a schematic diagram of a remote conference system according to Embodiment 2 of the present disclosure. FIG. 2 is a schematic configuration diagram of a communication terminal used in a remote conference system according to Embodiment 2 of the present disclosure. FIG. 2 is a schematic configuration diagram of a remote conference server used in a remote conference system according to Embodiment 2 of the present disclosure. FIG. 7 is a diagram showing a display image of a communication device used in a remote conference system according to Embodiment 2 of the present disclosure. FIG. 2 is a diagram showing a data configuration of a server used in a remote conference system according to Embodiment 2 of the present disclosure. FIG. 2 is a sequence diagram of communication start of the remote conference system according to Embodiment 2 of the present disclosure. FIG. 3 is a flow diagram of voice group communication of the remote conference system according to Embodiment 2 of the present disclosure. FIG. 2 is a sequence diagram during communication of the remote conference system according to Embodiment 2 of the present disclosure. FIG. 7 is a flowchart at the time of gist registration in the remote conference system according to Embodiment 2 of the present disclosure. FIG. 2 is a sequence diagram when a gist is registered in the remote conference system according to Embodiment 2 of the present disclosure. FIG. 3 is a flow diagram of mid-participation in the remote conference system according to Embodiment 2 of the present disclosure. FIG. 3 is a sequence diagram of mid-participation in the remote conference system according to Embodiment 2 of the present disclosure. FIG. 3 is a flow diagram of summary notification of the remote conference system according to Embodiment 2 of the present disclosure. FIG. 2 is a sequence diagram of summary notification of the remote conference system according to Embodiment 2 of the present disclosure.

Embodiments will be described below with reference to the drawings. Note that since the drawings are simplified, the technical scope of the embodiments should not be interpreted narrowly based on the descriptions in the drawings. In addition, the same elements are given the same reference numerals, and redundant explanations will be omitted.

In the following embodiments, the description will be divided into a plurality of sections or embodiments when necessary for convenience. However, unless otherwise specified, they are not unrelated to each other, and one is a modification, application, detailed explanation, supplementary explanation, etc. of a part or all of the other. In addition, in the following embodiments, when referring to the number of elements (including number, numerical value, amount, range, etc.), when it is specifically specified, when it is clearly limited to a specific number in principle, etc. It is not limited to the specific number except for , and may be greater than or less than the specific number.

Furthermore, in the following embodiments, the constituent elements (including operational steps, etc.) are not necessarily essential, unless specifically stated or when it is considered to be clearly essential in principle. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc. of components, etc., the shape, positional relationship, etc. of components, etc. are referred to, unless specifically stated or when it is considered that it is clearly not possible in principle. This shall include things that approximate or are similar to, etc. This also applies to the above-mentioned numbers (including numbers, numerical values, amounts, ranges, etc.).

<Embodiment 1>
Embodiment 1 will be described using FIG. 1. FIG. 1 is a schematic diagram of a remote conference device 1 according to this embodiment.

A remote conference device 1 according to the present embodiment is a device that connects to a plurality of communication terminals 2 for remote conferences via a communication network and performs a remote conference. The remote conference device 1 is communicably connected to a communication terminal 2 used by conference participants.

The remote conference device 1 includes a receiving section 3, a recording section 4, a character generating section 5, a summarizing section 6, and a transmitting section 7. The receiving unit 3 receives comments from conference participants from the communication terminals 2 used by the conference participants. Moreover, the recording unit 4 records the speeches of conference participants as audio data. The character generation unit 5 generates character data from the utterances of conference participants.

The summary unit 6 creates a summary of the voice data recorded by the recording unit 4 and the character data generated by the character generation unit 5. The summary unit 6 starts creating a summary when even one person participates in the meeting. The transmitting unit 7 distributes at least one of voice data and text data of the summary to the communication terminals 2 used by conference participants.

Note that the remote conference device 1 may further include a storage unit (not shown) that stores audio data and text data of the gist of the conference participants' statements. Furthermore, a detection unit (not shown) may be further provided to detect late participants and absentees based on notifications from the communication terminals 2 used by late participants and absentees. At this time, the transmitting unit 7 of the remote conference device 1 acquires from the storage unit the summary that the late participants and absentees have not been confirmed since the detection unit has detected the late participants and absentees. Then, the transmitter 7 can deliver at least one of the audio data and text data of the summary to the communication terminal 2 used by the late participants and absentees.

By using the remote conference device 1 in this embodiment, those who are absent from the meeting or those who join in the middle of the meeting can grasp the content of the conference participants' statements in a short time.

<Embodiment 2>
Embodiment 2 will be described with reference to FIG. 2. FIG. 2 is a schematic diagram of the remote conference system 10 according to this embodiment.

The remote conference system 10 in this embodiment includes communication terminals 100a to 100c used by conference participants, and a remote conference server 120 connected via networks 110 and 111.

The communication terminal 100a is a terminal that cannot display text information but is capable of voice communication, and is, for example, a landline telephone. The communication terminals 100b and 100c are communication terminals capable of displaying text information and making voice calls, and are, for example, communication terminals such as mobile phones and smartphones. Note that unless there is a particular need to distinguish between the communication terminals 100a to 100c, they will be simply referred to as the communication terminal 100 hereinafter.

Communication terminal 100 and remote conference server 120 are connected via networks 110 and 111. In this embodiment, the communication terminal 100a is connected to a remote conference server 120 via a network 110 used for analog communication. Furthermore, the communication terminals 100b and 100c are connected to a remote conference server 120 via a network 111 used for digital communication. Note that the form in which the communication terminal 100 and the remote conference server 120 are connected is not limited to the networks 110 and 111 described above, and may be one, three or more.

The configuration of communication terminal 100 will be explained using FIG. 3. FIG. 3 is a schematic configuration diagram of the communication terminal 100 used in the remote conference system 10 according to the present embodiment.

The communication terminal 100a includes a voice communication section 101 and an operation section 102. The audio communication unit 101 performs audio communication during a remote conference using the remote conference system 10. The operation unit 102 includes a group of keys that can be pressed to issue operational instructions such as starting and ending voice communication. The communication terminal 100a can issue operation instructions from the operation unit 102 to the remote conference server 120 via the network 110. Further, line connection and voice communication can be performed from the remote conference server 120 to the communication terminal 100a.

The communication terminals 100b and 100c include a voice communication section 103, an operation section 104, a display section 105, and a control section 106. The voice communication unit 103 and the operation unit 104 have the same functions as the communication terminal 100a, so a description thereof will be omitted. The display unit 105 displays text information regarding the remote conference. The control unit 106 transmits and receives text information regarding a remote conference. The communication terminals 100b and 100c can issue operation instructions from the operation unit 104 to the remote conference server 120 via the network 111. The communication terminals 100b and 100c may instruct the remote conference server 120 to operate using text from the control unit 106, and display the operation status on the display unit 105. Further, line connection and voice communication can be performed from the remote conference server 120 to the communication terminals 100b and 100c.

The configuration of the remote conference server 120 will be explained using FIG. 4. FIG. 4 is a schematic configuration diagram of the remote conference server 120 used in the remote conference system 10 according to this embodiment.

The remote conference server 120 includes a voice communication section 121, a recording section 122, a character generation section 123, a storage section 124, and a control section 125. The voice communication unit 121 performs voice communication with the communication terminal 100. The recording unit 122 records comments made at the meeting as audio data. The character generation unit 123 generates character data from the voice of the utterance. The storage unit 124 stores voice data and character data. The control unit 125 transmits and receives character data to and from the communication terminal 100.

The remote conference server 120 is capable of voice group communication with the communication terminal 100. Further, the remote conference server 120 manages conference participants and groups of conference participants who connect to the remote conference server 120 using the communication terminals 100 that it owns. Line connection and voice communication can be performed from the remote conference server 120 to the communication terminal 100.

(Operation of remote conference system 10 in Embodiment 2)
The operation of the remote conference system 10 in this embodiment will be described below.

First, FIG. 5 shows a screen displayed on the display unit 105 of the communication terminals 100b and 100c. FIG. 5 is a diagram showing a display image of the display unit 105 of the communication terminals 100b and 100c used in the remote conference system 10 according to the present embodiment.

As shown in FIG. 5, an "abstract list screen" and an "abstract registration screen" are displayed on the display units 105 of the communication terminals 100b and 100c. As will be explained later, in the remote conference system 10 of this embodiment, the gist of what conference participants say is created using audio data and text data. By creating a gist of the audio data, conference participants can hear the contents of the conference in a short time. In addition, by creating a summary of text data, conference participants can visually grasp the content of statements made at the conference in a short time.

The summary list screen of the display unit 105 of the communication terminals 100b and 100c displays a list of the summaries of the conference participants' comments. The display unit 105 may also include a playback button for playing back audio and a confirmation button for notifying that the gist has been confirmed.

The abstract list screen displays the following items: holding group, abstract type, conversation user, conversation content, and user whose abstract has not been confirmed. Information identifying a group of conference participants, including those scheduled to attend the conference, is written in the host group item. In the abstract type item, descriptions indicating the contents of the abstract, such as "purpose," "main points," and "decisions," are made. The conversation user item contains personal information of the conference participant who spoke. The conversation content item describes the content of the statement. In the item of users who have not confirmed the abstract, personal information of those who are scheduled to participate in the conference, such as late participants and absentees, who have not confirmed the abstract, is recorded.

By providing an item for users who have not confirmed the abstract, conference participants can easily understand whether or not other conference participants have confirmed the abstract. When using the communication terminals 100b and 100c, it may be possible to select whether to display the summary in text data or to reproduce audio data.

On the abstract registration screen of the display unit 105 of the communication terminals 100b and 100c, the following items are displayed: abstract type, conversation user, and conversation content. Additionally, a registration button is provided for registering the gist of the conference participants' comments.

The abstract types shown on the abstract registration screen of the display unit 105 include, for example, "purpose," "main points," and "decision matters," but are not limited thereto. By displaying the abstract type, conference participants can quickly check what other participants are saying.

Further, a screen displayed on the display unit of the remote conference server 120 is shown in FIG. FIG. 6 is a diagram showing a data structure in the storage unit 124 of the remote conference server 120 used in the remote conference system 10 according to the present embodiment. As shown in FIG. 6, the storage unit 124 stores "group information," "conversation information," and "group status information."

"Group information" in the storage unit 124 of the remote conference server 120 includes information regarding a group made up of conference participants, conference participants belonging to the group, and communication terminals 100 owned by the conference participants. One conference participant may be registered in multiple groups. The above information is registered in the remote conference server 120 in advance. That is, the above information may be directly input from the remote conference server 120 or may be registered in the "group information" using a device that can communicate with the remote conference server 120.

The group column in "Group Information" indicates a group formed by conference participants. Further, the user column in "Group Information" indicates conference participants who belong to a group, and conference participants may be registered in multiple groups. The communication terminal column in "Group Information" indicates communication terminals 100 owned by conference participants.

"Conversation information" in the storage unit 124 of the remote conference server 120 manages the audio transmitted from each communication terminal 100 for each utterance when conference participants belonging to each group hold a conference using the communication terminals 100. This is a list for.

The holding group column in "conversation information" indicates a group made up of conference participants. The conversation user column in "conversation information" indicates the conference participant who made a statement among the conference participants belonging to the group. The audio data column in "conversation information" indicates audio data obtained by recording audio transmitted from each communication terminal 100. The text information column in "conversation information" indicates text data generated from audio data transmitted from each communication terminal 100. The gist type column in "conversation information" indicates whether or not there is a gist of audio data and text data.

“Group status information” in the storage unit 124 of the remote conference server 120 indicates for each conference participant whether or not each conference participant has confirmed the gist using the communication terminal 100 in a group made up of conference participants. This is a list for management.

The holding group column in "Group Status Information" indicates a group made up of conference participants. The user column in "group status information" indicates conference participants who belong to the group. The summary confirmation status column in "group status information" indicates whether late participants and absentees have confirmed the summary.

The flow of data when starting voice group communication using the remote conference system 10 in this embodiment will be explained using FIG. 7. FIG. 7 is a sequence diagram of communication start of the remote conference system 10 according to this embodiment.

First, the operation when a conference participant who owns the communication terminal 100b participates in a conference will be described. A conference participant uses the communication terminal 100b to transmit a group information request to the remote conference server 120 (step 301). The remote conference server 120 returns information about the group to which the requesting conference participant belongs from the "group status" in the storage unit 124 to the communication terminal 100b as a group information response (step 302). The group information may be confirmed and designated on the display section 105 by displaying the group information received by the communication terminal 100b on the display section 105 included in the communication terminal 100b.

Next, the operation when a conference participant who owns the communication terminal 100a participates in a conference will be described. The conference participant adds group information and transmits a group communication start instruction to the remote conference server 120 by operating the operation unit 102 included in the communication terminal 100a (step 303). When the remote conference server 120 receives the group communication start instruction, it acquires user information and communication terminal information belonging to the specified group from the "group information" in the storage unit 124. The remote conference server 120 sets the acquired group information and user information in the “group status information” of the storage unit 124. After that, remote conference server 120 transmits a group communication instruction to communication terminal 100a and communication terminal 100b (step 304). Voice group communication is started when communication terminal 100a and communication terminal 100b respond to the group communication instruction.

The flow of data during voice group communication using the remote conference system 10 in this embodiment will be explained using FIGS. 8 and 9. FIG. 8 is a flow diagram of voice group communication in the remote conference system 10 according to the present embodiment. Further, FIG. 9 is a sequence diagram during communication of the remote conference system 10 according to the present embodiment.

First, the outline of the data flow during voice group communication will be explained using FIG. A conference participant uses the communication terminal 100 to transmit audio to the remote conference server 120 (step 211). When speaking has started (“start” in step 212), the remote conference server 120 writes the holding group information and the conversation user information in the “conversation information” of the storage unit 124. Furthermore, the remote conference server 120 starts storing the received audio as audio data in the "conversation information" of the storage unit 124 (step 213). When the speech is finished (“end” in step 212), the remote conference server 120 finishes accumulating the audio data, and writes the text information obtained by converting the audio data into text data in the character generation unit 123 (step 214). . The remote conference server 120 adds conversation user information and text information and transmits a text information notification to the communication terminal 100b (step 215).

Next, a sequence during communication of the remote conference system 10 will be described using FIG. 9. The remote conference server 120 performs recording of speech during voice group communication, transcribing it into text, and mediating it to a communication terminal. When a conference participant starts speaking, the remote conference server 120 transmits the voice in real time to the communication terminals 100 of conference participants other than the speaker in the group every time it receives voice from the communication terminal 100 (step 311 ).

In parallel with the process in step 311, the remote conference server 120 writes the holding group information and the conversation user information in the "conversation information" of the storage unit 124 at the start of speaking, at the time of changing the speaker, and at the break of the audio. Additionally, the remote conference server 120 starts storing the received audio as audio data in the "conversation information" of the storage unit 124, and ends the storage of audio data when the speech ends, when the speaker changes, and when the audio breaks. The remote conference server 120 writes the character information obtained by converting the audio data into character data in the character generation unit 123 (step 312).

The remote conference server 120 adds conversational user information and text information and transmits a text information notification to the communication terminal 100b (step 313).

Through the above-described processing, voice group communication can be performed using the communication terminal 100a that cannot display text information and the communication terminal 100b that can display text information, and the received voice can be played back in real time. Further, by displaying the gist of the conference participant's comments as text information in the list of abstracts shown in FIG. 5, the conference participant can confirm the comments of other conference participants in a short time.

Next, the flow of registering the summaries of conference participants' comments using the remote conference system 10 in this embodiment will be explained using FIGS. 10 and 11. FIG. 10 is a flowchart at the time of abstract registration in the remote conference system 10 according to this embodiment. Further, FIG. 11 is a sequence diagram at the time of abstract registration of the remote conference system 10 according to the present embodiment.

First, the outline of the flow of registering the gist of comments made by conference participants will be explained using FIG. 10. As shown in the flowchart of FIG. 10, the process flow differs depending on whether the communication terminal 100 used by the conference participant is the communication terminal 100a that cannot display text information, or the communication terminal 100b or 100c that can display text information. different.

If the communication terminal 100 used by the conference participant is the communication terminal 100a that cannot display text information (YES in step 221), the remote conference server 120 performs the same process as the voice group communication process. That is, each time remote conference server 120 receives audio from communication terminal 100a, it transmits the audio to other communication terminals 100 in real time, and writes holding group information and conversation user information in storage unit 124. Further, the remote conference server 120 stores the received audio as audio data in the storage unit 124, and writes text information obtained by converting the audio data into text data (step 222). After that, the remote conference server 120 writes the gist type information in the gist type column of "conversation information" in the storage unit 124 (step 223). The remote conference server 120 transmits voice data or text data of the summary to the communication terminals 100 of the conference participants in the group. At this time, the remote conference server 120 transmits a gist information notification to which gist confirmation status information is added in addition to the gist (step 224).

If the communication terminal 100 used by the conference participant is the communication terminal 100b capable of displaying text information (NO in step 221), the remote conference server 120 extracts a record that matches the text data generated from the received utterance. do. The communication terminal 100b may add abstract type information, which is the type of abstract, and transmit it to the remote conference server 120, and the remote conference server 120 may write the received abstract type information in the abstract field of the record (step 225). The remote conference server 120 transmits voice data or text data of the summary to the communication terminals 100 of the conference participants in the group. At this time, the remote conference server 120 transmits a gist information notification to which gist confirmation status information is added in addition to the gist (step 224).

Here, the gist registration sequence of the remote conference system 10 will be explained using FIG. 11. First, the operation when a conference participant who owns the communication terminal 100a participates in a conference will be described.

First, a case where the communication terminal 100a is used will be explained. When using the communication terminal 100a, when registering the abstract, the user operates the operation unit 102 included in the communication terminal 100a to specify the abstract type and send an instruction to start the abstract to the remote conference server 120 (step 321). The abstract types used include "purpose" and "decision matters," but are not limited to these. After receiving the gist start instruction from the communication terminal 100a, the remote conference server 120 receives the voices of the conference participants (step 322).

At this time, the remote conference server 120 performs recording of the speech, transcribing it into text, and mediating it to the communication terminal, in the same way as during group communication. In addition, the remote conference server 120 writes gist type information in the gist type column 215 of “conversation information” in the storage unit 124 . In addition, in the gist confirmation status column of "group status information" in the storage unit 124, the column of a conference participant who is participating in group communication is written as gist confirmed. In addition, the column for those who have not participated in the group communication, such as mid-participants and absentees, is entered as unconfirmed abstract. The end of registration of the gist may be determined at the end of the speech, by changing the speaker and at the end of the voice, or by the operation of the operation unit 102 included in the communication terminal 100a.

The remote conference server 120 transmits a gist information notification to which audio data or text data of the gist and gist confirmation status information are added to the communication terminals 100 of the conference participants in the group (step 323).

Next, a case where the communication terminal 100b is used will be explained. When using the communication terminal 100b, when registering the abstract, the user operates the operation unit 102 included in the communication terminal 100a to specify the abstract type and send an instruction to start the abstract to the remote conference server 120 (step 325). At this time, the remote conference server 120 writes in the gist type column of "conversation information" and the gist confirmation status column of "group status information" of the storage unit 124, as in the case of using the communication terminal 100a. The remote conference server 120 transmits a gist information notification to which audio data or text data of the gist and gist confirmation status information are added to the communication terminals 100 of the conference participants in the group (step 326).

When the communication terminal 100a receives the gist information notification, it is possible to reproduce the audio data of the gist. When the communication terminal 100b receives the abstract information notification, by displaying the text data and reproducing the audio data of the abstract as shown in the list of abstracts shown in FIG. 5, conference participants participating in group communication can be notified whether the abstract has been confirmed or the abstract has not been confirmed. It becomes possible to confirm the identity of the person.

When an operation is performed from the communication terminal 100 to instruct reconfirmation of the gist, the gist reproduction instruction may be transmitted to the server by a playback operation using the operation units 102 and 104 of the communication terminal 100. At this time, the remote conference server 120 may transmit audio data of the corresponding gist type.

Next, the flow of registering the summaries of conference participants' comments using the remote conference system 10 in this embodiment will be explained using FIGS. 12 and 13. FIG. 12 is a flowchart of mid-participation in the remote conference system 10 according to this embodiment. Further, FIG. 13 is a sequence diagram of mid-participation by the remote conference system 10 according to the present embodiment.

First, an outline of the flow of midway participation in a conference will be explained using FIG. 12. The remote conference server 120 acquires the gist that the communication terminal 100 used by the participant in the middle of the conference is unconfirmed from the "group status information" in the storage unit 124, and acquires audio data or text data that matches the gist (step 231). When midway participants use the communication terminals 100b and 100c, they may be able to select whether to display the summary in text data or to reproduce audio data. If the midway participant uses the communication terminal 100a (YES in step 232), audio data of the summary is delivered to the communication terminal 100a (step 233). If the midway participant uses the communication terminal 100b (NO in step 232), audio data or text data of the summary is delivered to the communication terminal 100b (step 234). Thereafter, the remote conference server 120 updates the gist confirmation status of the midway participant in the "group status information" of the storage unit 124 to "confirmed" (step 235).

Here, the sequence of mid-participation processing of the remote conference system 10 will be described using FIG. 13. The communication terminal 100 transmits a group communication participation instruction to the remote conference server 120 by operating the operation units 102 and 104 (step 331). When the remote conference server 120 receives a group communication participation instruction from the communication terminal 100, it acquires the gist that mid-participants are not confirmed from the gist confirmation status column of "group status information" in the storage unit 124. Thereafter, the remote conference server 120 acquires audio data and character data that match the acquired gist from the “conversation information” in the storage unit 124.

The remote conference server 120 distributes audio data of the summary if a participant uses the communication terminal 100a during the conference (step 332). Further, the remote conference server 120 distributes audio data or text data of the summary if a participant in the middle of the conference uses the communication terminal 100b (steps 333, 334). Thereafter, the remote conference server 120 updates the "group status" of the storage unit 124 and the gist confirmation status of the midway participant to "confirmed".

In this way, the communication terminal 100a can reproduce the audio of the received gist. The communication terminal 100b can reproduce the audio of the received summary or confirm the text information by using the operation buttons on the display screen of the display unit 105 shown in FIG.

Finally, the flow of notification of abstracts to absentee persons whose abstracts have not been confirmed using the remote conference system 10 in this embodiment will be explained using FIGS. 14 and 15. FIG. 14 is a flowchart of summary notification in the remote conference system 10 according to the present embodiment. Further, FIG. 15 is a sequence diagram of summary notification of the remote conference system 10 according to the present embodiment.

First, the outline of the flow of summary notification will be explained using FIG. 14. When the voice group communication due to the conference ends, the remote conference server 120 acquires the voice or text data of those whose abstracts have not been confirmed, such as those who are absent from the conference, and whose abstracts have not been confirmed (step 241). Then, the remote conference server 120 notifies the communication terminal 100 used by the unconfirmed person of the summary notification (step 242). When a person whose abstract has not been confirmed uses the communication terminal 100a (YES in step 243), audio data of the abstract is distributed to the communication terminal 100a (step 244). When a person whose abstract has not been confirmed uses the communication terminal 100b (NO in step 243), audio data or text data of the abstract is distributed to the communication terminal 100b (step 245). When the absentee uses the communication terminals 100b and 100c, it may be possible to select whether to display the summary in text data or to reproduce audio data. The remote conference server 120 updates the gist confirmation status of the person whose gist has not yet been confirmed in the "group status information" of the storage unit 124 to "confirmed" (step 246).

Here, the flow sequence of notification of abstracts to absentee persons whose abstracts have not been confirmed will be explained using FIG. 15. When the voice group communication by the conference ends, the remote conference server 120 acquires information of users whose gist has not been confirmed from the gist confirmation status column of "group status information" in the storage unit 124. Thereafter, the remote conference server 120 acquires audio data and character data that match the acquired gist from the “conversation information” in the storage unit 124.

The remote conference server 120 notifies the communication terminal 100 used by the person whose abstract has not been confirmed (step 341). The remote conference server 120 delivers audio data of the summary when delivering the summary to the communication terminal 100a (step 342), and delivers audio data or text data of the summary when delivering the summary to the communication terminal 100b (step 342). Steps 343, 344).

The communication terminal 100 uses the operation units 102 and 104 to repeat the audio of the received summary, to display the text information of the summary, or by pressing a confirmation button on the display unit 105, so that the communication terminal 100 can connect to the remote conference server. 120 may be notified of abstract confirmation (step 345). At this time, the remote conference server 120 updates the gist confirmation status of the person whose gist has not been confirmed in the "group status information" of the storage unit 124 to "confirmed".

In this way, those who are absent from the meeting can grasp the gist of what was said at the meeting. Further, other conference participants can know who has not confirmed the gist and whether or not the person who has not confirmed the gist has confirmed the gist.

When the voice group communication due to the conference ends, the remote conference server 120 acquires the conversation user information and text information that indicates that there is a gist in the gist type column of "conversation information" in the storage unit 124, and performs independent communication. The document data may also be created by By making the created document data in a format that can be referenced from other communication terminals 100, it can be shared within the event group. It can also be used to create meeting minutes from document data.

By using the remote conference system 10 in this embodiment, those who are absent from the meeting or those who join in the middle of the meeting can grasp the contents of the statements made by the meeting participants in a short time. This is because, by using the communication terminal 100, an absentee or the like can select whether to confirm the gist of the statement using voice data or text data. Furthermore, even when using the communication terminal 100a that cannot display text information, it is possible to understand the gist from the audio data.

Moreover, the remote conference system 10 in this embodiment creates the gist of each speech of a conference participant as audio data and text data. That is, since the summary is created without detecting mid-participants, not only mid-participants in the conference can confirm the summary during the absent period of the conference, but also absentees can confirm the summary.

<Other embodiments>
In the remote conference system 10 according to the second embodiment, the remote conference server 120 stores audio data for each conference participant's utterance in the "conversation information" of the storage unit 124. The utterances may be stored as audio data. In this case, the remote conference server 120 further provides a conversation start time field and a conversation time field for each conversation in the "conversation information" of the storage unit 124, and transmits conversation start time data at the time of conversation start from the communication terminal 100. It's okay.

Further, in the remote conference system 10 in the second embodiment, the character generation unit 123 of the remote conference server 120 generates character data from the voice of the utterance, but a separate device or interface may be provided to edit the generated character data. Good too. Furthermore, character data may be edited using the communication terminals 100b and 100c. In this case, even if character data cannot be created correctly from the audio of the meeting, the character data can be corrected.

In the embodiments described above, the present disclosure has been described as using hardware, but the present disclosure is not limited thereto. The remote conference device 1 according to the present disclosure includes, for example, an embodiment as a remote conference method. That is, the remote conference method includes the following steps: receiving the utterances of the conference participants from the communication terminals used by the conference participants; recording the utterances as audio data; and generating text data from the utterances. When a conference participant participates, the method may include the step of creating a summary of the audio data and the text data, and the step of delivering at least one of the audio data and the text data of the summary to the communication terminal.

In the above examples, the programs can be stored and delivered to the computer using various types of non-transitory computer readable media. Non-transitory computer-readable media includes various types of tangible storage media. Non-transitory computer-readable media include, for example, magnetic recording media, magneto-optical recording media, CD-ROMs (Read Only Memory), CD-Rs, CD-R/Ws, and semiconductor memories. Examples of the semiconductor memory include mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (Random Access Memory). The program may also be provided to the computer on various types of transitory computer readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can provide the program to the computer via wired communication channels, such as electrical wires and fiber optics, or wireless communication channels.

The above program includes a process of receiving the conference participants' comments from the communication terminals used by the conference participants, a process of recording the comments as audio data, a process of generating text data from the comments, and a process of participating in the conference. When a person participates, the information processing device executes a process of creating a summary of the voice data and the text data, and a process of distributing at least one of the voice data and the text data of the summary to the communication terminal. It is a program.

Although the embodiments of the present disclosure have been described above in detail with reference to the drawings, the specific configuration is not limited to that described above, and various design changes may be made without departing from the gist of the present disclosure. etc. are possible.

1 Remote conference device 3 Receiving unit 4, 122 Recording unit 5 Character generating unit 6 Summarizing unit 7 Transmitting unit 10 Remote conference system 2, 100a, 100b, 100c Communication terminals 101, 103 Voice communication unit 102, 104 Operation unit 105 Display unit 106 , 125 control units 110, 111 network 120 remote conference server 123 character generation unit 124 storage unit

Claims (8)

a receiving unit that receives the conference participant's comments from a communication terminal used by the conference participant;
a recording unit that records the utterance as audio data;
a character generation unit that generates character data from the utterance;
a summary unit that creates a summary of the audio data and the text data when a conference participant participates;
a transmitting unit that delivers the summary to the communication terminal as at least one of audio data and text data;
a storage unit that stores audio data and text data of the gist of the statement;
a detection unit that detects late participants and absentees based on notifications from the communication terminal;
Equipped with
When the detection unit detects a late participant or an absentee,
The storage unit includes:
storing personal information of a conference participant who gave the gist in association with information of a user whose gist has not been confirmed, who is a mid-participant or an absentee who has not confirmed the gist;
If the detection unit detects a mid-way participant,
When the transmission unit acquires a summary for which the late participant is unconfirmed from the storage unit, and delivers at least one of audio data and text data of the summary to a communication terminal used by the late participant,
The abstract confirmation status of the midway participant will be updated as confirmed,
If the person detected by the detection unit is an absentee,
The transmission unit acquires a summary in which the absentee is unconfirmed from the storage unit, delivers at least one of audio data and text data of the summary to a communication terminal used by the absentee, and transmits communication of the absentee. When you receive an abstract confirmation notification from your device,
The summary confirmation status of the absentee will be updated as confirmed.
Remote conference equipment. The remote conference device manages whether or not the summary has been distributed to each communication terminal.
The remote conference device according to claim 1. The remote conference device can cause the communication terminal to refer to at least one of audio data and text data of the summary;
The remote conference device according to claim 1 or 2. A communication terminal used by a conference participant, and a remote conference device according to any one of claims 1 to 3, which is communicably connected to the communication terminal.
A remote conference system equipped with The communication terminal displays whether the remote conference device has delivered the summary to the communication terminal used by each conference participant.
The remote conference system according to claim 4. The communication terminal can add abstract type information that is the type of the abstract and register it in the remote conference device.
The remote conference system according to claim 4 or 5. receiving the conference participant's remarks from a communication terminal used by the conference participant;
recording the utterance as audio data;
generating character data from the utterance;
When conference participants participate, creating a summary of the audio data and the text data;
delivering the summary as at least one of audio data and text data to the communication terminal;
storing audio data and text data of the gist of the statement;
Detecting mid-participants and absentees by notification from the communication terminal;
Equipped with
If the detecting step detects a late participant or an absentee,
The step of storing associates and stores the personal information of the conference participant who gave the gist and the information of the user whose gist has not been confirmed, who is a mid-participant or an absentee who has not confirmed the gist;
If the person detected in the detecting step is a mid-participant,
When the distributing step obtains a gist that has not been confirmed by the late participant from the storing step, and distributes at least one of audio data and text data of the gist to a communication terminal used by the late participant;
The abstract confirmation status of the midway participant will be updated as confirmed,
If the person detected in the detecting step is an absent person,
The distributing step acquires from the storing step a summary for which the absentee has not been confirmed, and distributes at least one of audio data and text data of the gist to a communication terminal used by the absentee, and the absentee When you receive an abstract confirmation notification from your communication terminal,
The summary confirmation status of the absentee will be updated as confirmed.
A remote conference method using a remote conference device. A process of receiving the conference participant's remarks from a communication terminal used by the conference participant;
a process of recording the utterance as audio data;
a process of generating character data from the utterance;
When a conference participant participates, a process of creating a summary of the audio data and the text data;
a process of distributing the summary as at least one of audio data and text data to the communication terminal;
a process of storing audio data and text data of the gist of the statement;
A process of detecting late participants and absentees by notification from the communication terminal;
Equipped with
If the detection process detects a late participant or an absentee,
The storing process associates and stores the personal information of the conference participant who gave the summary and the information of a user whose abstract has not been confirmed, who is a mid-participant or an absentee who has not confirmed the summary;
If the detection process detects a mid-participant,
When the distributing process obtains a gist that is unconfirmed by the late participant from the storing process, and distributes at least one of audio data and text data of the gist to a communication terminal used by the late participant;
The abstract confirmation status of the midway participant will be updated as confirmed,
If the person detected by the detection process is an absent person,
The distributing process obtains a summary for which the absentee is unconfirmed from the storing process, distributes at least one of audio data and text data of the gist to a communication terminal used by the absentee, and When you receive an abstract confirmation notification from your communication terminal,
The summary confirmation status of the absentee will be updated as confirmed.
A program that causes an information processing device to execute a process.

Images