JP4406295B2 - Application linkage system and application linkage method - Google Patents

Description

  The present invention provides an application cooperation system, a network terminal device, and the like that operate an application of a participant of a conference in conjunction with an operation of an application used by a presenter in a multimedia conference.

  Currently, various communication services for general consumers or corporate users are provided by various communication operators. As one of these communication services, a voice conference service is provided (for example, a chorus line service) in which a multi-point general telephone, a mobile phone, a PHS, etc. are connected to carry out a conversation at the same time.

  On the other hand, along with the expansion of the Internet market, text conferencing services and text chat services are also provided in which many users share data-type information such as character data, image data, and music on the Web (for example, Web conference room).

  In the network-provided conference service, which is the voice conference service and the text conference service as described above, real-time voice conference service and data system service should be interlinked with the background of the diversification of user needs and business forms. ing.

  In recent years, with the advancement of VoIP (Voice over IP) technology, an application called an Internet telephone that performs two-way real-time voice communication on an IP (Internet protocol) packet network such as the Internet has begun to spread.

Therefore, the conventional system for providing multimedia conferencing services includes an image information sharing unit, a whiteboard sharing unit, a speaker screen display unit, an arbitrary information search unit, an arbitrary information posting unit, and a meeting minutes creation. The real-time audio conference and the data system conference equipped with any one or more of the departments are realized. Also, in audio conferencing, a multimedia conferencing service is provided in which a circuit switching audio communication terminal such as a telephone terminal and a VoIP communication terminal such as an Internet telephone terminal are mixed (see, for example, Patent Document 1). .
JP 2001-292138 A (1-7, FIG. 2)

  In general, when communication is performed using a transmission medium sharing type network, the amount of information that can be transmitted at a certain time depends on the amount of information transmitted by another terminal at that moment. For this reason, when a certain terminal tries to transmit data, there is no guarantee that data can be transmitted without delay. Actually, in addition to the video and audio of the presenter, when trying to transmit the output video of the application displayed on the terminal device on the presenter side at the same time, the amount of data to be transmitted becomes enormous. There are problems such as a shift in image data and a delay in data transmission due to an increase in traffic volume. In other words, on the receiving side, the image is displayed with a delay with respect to the voice, or is interrupted, or the conversation is jerky if the presenter's voice (conversation) arrives clearly behind the utterance. .

  In particular, in the conventional technology, a plurality of users who are present at multiple points announce the video presented by the terminal device user of the transmission source to a plurality of terminal devices on the receiving side connected via the network. I was sharing the video of the material. At the same time, for example, the video of the presenter was transmitted as a moving image. For this reason, the video of the document must be transmitted to the receiving device every time the video of the material being presented is switched, and the video of the presenter and the material is interrupted due to an increase in traffic each time the video of the material is switched. was there. For such problems, IPv6 (Internet Protocol Version 6), which is a next-generation Internet protocol, is about to be implemented. Based on the current IPv4 (Internet Protocol Version 4), IPv6 is a network camera (or digital camera) that has been improved such as an increase in address space that can be managed, addition of security functions, and data transmission according to priority. ) Is preferentially transmitted / received.

  However, in particular, when the above-described multimedia conference service is provided, it is necessary to satisfy a resolution that can be satisfied by both a computer that is used in the vicinity of a screen that uses a lot of characters and a home TV that displays moving images away from the screen. Yes, a resolution of 2M pixels / frame is required. This corresponds to the HDTV 1100 shown in FIG. 11, that is, the resolution of a high definition (HD) level. In the future, when such high-definition quality becomes normal and a plurality of AV information is frequently transmitted and received, naturally, the traffic amount of the network also increases. For this reason, even if AV information is preferentially transmitted by adopting IPv6, it is considered that the above-mentioned problem relating to the amount of network traffic cannot be completely solved.

  Furthermore, a problem occurs in data processing on the receiving side. In other words, as the video at the source terminal device is switched, the receiving terminal device performs processing to switch from the video before the change to the video after the change, so the internal processing of the receiving terminal device increases, and the VoIP audio is sequentially processed. There was a problem that the sound was interrupted because it was impossible. In this respect as well, it is expected that it will become a more prominent problem due to the increase in the amount of data due to the high definition.

  In the present invention, an application linkage system and a network that can display and play video and audio of the presentation contents without interruption at many points away from the point where the announcement is made, and further reduce the processing load inside the apparatus. An object is to provide a terminal device or the like.

The present invention employs the following means in order to achieve the above object. That is, the present invention has the same function, and is an application that operates two or more applications in cooperation with each other, which is operated by each of a transmitting device connected via a network and one or more receiving devices. cooperation Ru system der. Here, the above-mentioned originating device, operating on the originating device, read application data to the first application to switch the output image in response to an instruction that does not contain video, its first application a first application operation unit to be operated Mase, to the first application running on the first application operation unit on, the first application, and in the first application and function the same application The instruction that is applied to any of the second applications that operate on the receiving-side apparatus, and the screen switching instruction that instructs switching of the output video in each application is sent to the user for the first application. Or pre-set conditions for the first application Given in accordance with, the a first first application controller that allowed switching the output image of the application, when the screen switching instruction is given to the first application, the receiving-side apparatus that screen switching instruction And a transmission unit for transmitting to. That is, the first application control unit transmits an instruction to an application operating on the own device, and the transmission unit transmits the instruction to the receiving side device. Then, each receiving side apparatus operates the second application by receiving an electronic file including the contents of the application data in advance and reading the application data having the same contents as the application data into the second application. A second application operating unit, a receiving unit that receives the screen switching instruction transmitted from the transmitting unit, and the second application that is operating on the second application operating unit. received by, giving the screen switching instruction does not include a video, an output image based on the application data at the second application, the same as the output image at the first application the second application control unit that allowed switching in conjunction with the Equipped with a.

  According to the above, since the presentation material can be displayed in conjunction with the reception side device without transmitting the video data of the presentation material output by the application of the transmission side device, the traffic volume of the network It is possible to reduce the burden of the reduction and the calculation processing of the receiving side device, and to output the video and audio of the presenter and the application output video of the presentation material without interruption at multiple points away from the point where the announcement is made.

Further, according to the present invention, at least one of the transmitting side device and each receiving side device has an application running on the other side device, together with either the file name or the file creation date / time information, or any one of them. applications species including version information, or can be input, the state of the application that the display size includes information such as it is within a predetermined range, and, with the application data used by the application, the self-device An application / data management unit that checks the identity of whether or not the application is the same and permits the cooperative operation when there is identity is provided.

In this configuration, possible to avoid the application data that is available in the presenter-side application, it is different from the application data that is available in the other conference participants application, a problem such as that the application work together is funny Can do.

Further, the transmission unit, the address information of the receiving apparatus, and a content which the second application uses operating on the receiving side apparatus, transmission instruction and the content to the effect to be transmitted to the receiving device sends a predetermined server, also the reception unit, the content of the server is transmitting to receiving the transmission instruction received as the electronic file, the operation content that the received on the receiving device There is also a configuration for handing over to the second application.

Furthermore, the transmission unit may transmit the content to the second application running on the receiving side apparatus is utilized in a predetermined server, and further sends the address information of the server to the receiver of the receiving device, The receiving unit may receive the content as the electronic file from the server based on the received address information, and deliver the received content to the second application operating on the receiving-side device. Good.

  In this way, the transmission side device can transmit the content to the server at an appropriate time before the content is required at a meeting or the like, or after the content to be used is stored in the server, Content can be received at an appropriate time according to the circumstances of each point.

Further, the originating device, the first synchronizing and the screen switching instruction output from the audio signal and the first application control unit which is input to the video signal and the audio input unit that is input to the image input unit further comprising a time control unit, the transmitting unit, the other the video signal and the audio signal of the screen switching instruction is transmitted to the receiving device, also, the receiving unit, in addition to the originating side of the screen switching instruction Receiving the video signal and the audio signal from the device, and the receiving-side device receives the video signal , the audio signal and the instruction received by the receiving unit , and receives the video, audio and instruction. There is a configuration including a second time control unit that outputs the signals in synchronization with each other.

  In this configuration, it is possible to alleviate problems such as a gap between audio data and image data and a delay in data transmission due to an increase in traffic volume by performing synchronization.

  Note that when the video signal input from the video input unit is a high-definition quality video signal, the amount of data occupied by the video becomes large, and a large amount of processing is also required in the receiving side apparatus. The effect by appears remarkably.

  Here, the application cooperation system and the network terminal device can be embodied using a computer. In that case, each unit described above is embodied by operating a program on a computer.

  According to the application cooperation system, the application cooperation method, and the network terminal device of the present invention, the copy data of the material announced by the user on the transmission device side is set on the reception device side and the application is operated. The video signal output from the application at each site is switched based on an application output switching instruction from the user of the transmission side device or a preset condition. As a result, the network traffic volume is reduced and the processing load on the receiving device is reduced, and the video output of the presenter and the application output video of the presentation material are interrupted at multiple points away from the point of presentation. It has the effect that it can output without.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a conceptual diagram for explaining an outline of an application cooperation system in a multimedia conference system using a network terminal device of the present invention.

  The network terminal devices 100, 200, and 300 are used as user-side terminals in the multimedia conference system. Further, it is assumed that the network terminal device 100 is installed on the presenter side for use by the presenter, and the network terminal devices 200 and 300 are installed at the bases of the participants participating in each conference. In the following, the term “participant” does not include the presenter.

  The network terminal devices 100, 200, and 300 at each site are connected to the projectors 400, 500, and 600, respectively, receive video signals output from the network terminal devices, and project the received videos on the screens 800, 900, and 1000. .

  In the present invention, the application data of the presentation material is output as a video signal to the projector 400 by the network terminal device 100 on the presenter side, and the video signal to be output to the projector 400 is switched based on the presenter's instruction or preset conditions. It is like that. Here, the presenter's instruction means that the user who is actually presenting using the image projected on the projector instructs the application outputting the presentation material to switch the screen. . Also, the preset condition here refers to the case where the output video is automatically switched by the user setting the timing of the instruction to switch the screen to the application in advance for the application outputting the presentation material. Refers to the switching timing set in the application.

  Here, the network terminal apparatus will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the application cooperation system.

  The network terminal device 100 in FIG. 2 can be realized by connecting the following units via a bus 112.

  The application operation unit 101 operates an application that reads presentation materials used for a conference, converts them into video data, and outputs them.

  The input instruction unit 110 is for a user to input an instruction to switch the video output by the application to the apparatus, and corresponds to a mouse, a keyboard, or the like.

  The application control unit 102 outputs an instruction for causing the application to perform a predetermined operation based on the presenter's instruction input from the input instruction unit 110 or a preset condition.

  When the instruction output from the application control unit 102 is transmitted from the network terminal device 100, the transmission unit 103 transmits to the network terminal device (for example, the network terminal device 200, 300) used by another user. Send.

  The receiving unit 104 receives an instruction when a user of another network terminal device (for example, the network terminal device 200, 300) gives an instruction to an application operating on the other network terminal device.

  The I / F processing unit 105 performs connection to a network for communicating with an external device of the network terminal device 100.

  The IP voice conversion unit 106 converts an IP packet signal and a voice signal for bidirectional real-time voice communication over an IP packet network such as the Internet.

  The speaker 107 outputs the audio signal converted by the IP audio conversion unit 106. Further, the microphone 108 inputs voice as an audio signal to the IP voice conversion unit 106.

  The video output unit 109 outputs a video signal output from the application running on the application operating unit 101 to a display unit connected to the network terminal device 100.

  The address setting unit 111 receives input of the address of the device and the address of the communication destination device, and stores the input address.

  The application / data management unit 115 checks the version and state of the application operating between the apparatuses, the consistency (identity) of the read application data, and the like.

  The network terminal devices 200 and 300 are network terminal devices having the same functions as the network terminal device 100 described above. Note that 201 to 211 of the network terminal device 200 or 301 to 311 of the network terminal device 300 are assigned numbers corresponding to 101 to 109 of the network terminal device 100 and operate in the same manner. Description is omitted.

  The projectors 400, 500, and 600 are connected to the network terminal devices 100, 200, and 300, respectively, and receive the video signals from the video signal output units 109, 209, and 309 (not shown in FIG. 2) on the screen. Project an image. That is, it functions as the display unit.

  Next, the operation of the application cooperation system will be described with reference to FIG. 1, FIG. 2, and FIG. FIG. 3 is a flowchart showing the flow of processing in the application cooperation system.

  Here, it is assumed that the network terminal device 100 shown in FIGS. 1 and 2 is installed at the point A, the network terminal device 200 is installed at the point B, and the network terminal device 300 is installed at the point C. The network terminal device 100 at point A, the network terminal device 200 at point B, and the network terminal device 300 at point C are communicably connected via a network 700. When one of the network terminal devices calls another and the user of the received network terminal device responds to the call, a two-way voice call is possible. Hereinafter, in order to simplify the description, a conference between two points of A point and B point will be described as an example.

  First, application data is read into the application running on the application operating unit 101 of the network terminal apparatus 100 as presentation material used in the presentation. Then, the application control unit 102 transmits an instruction to the application based on the presenter's instruction or preset conditions, and as a result, the application switches the video signal to be output to the projector 400.

  In addition, the presenter sends an electronic file as presentation material to the conference participants in advance, and the conference participants at each point (for example, point B) read the received electronic file into the application of the network terminal device 200. Then, the same condition as the presenter, that is, the standby state is set (FIG. 3: step S101). In addition, the said application utilized by A point and B point is the same.

  At this time, the application / data management units 115, 215, and 315 provided in each network terminal apparatus check whether or not the electronic files to be set in the standby state are the same as those used in the conference. May be. For example, the application / data management unit 213 is inquired of the application / data management unit 113 about information related to the presentation material such as the file name and file creation date and time used in the conference, and whether or not the information about the presentation material is the same. To check. In this way, it is possible to avoid problems such as the application linkage operation being strange because the presentation material activated by the application on the presenter side is different from the presentation material activated by the application of other conference participants. At the same time, the application / data management unit may check whether the application being used is the same including the version, or whether the state of the application is the same. The state mentioned here is a state in which the application is in an input enabled state (has focus) or whether the display size of the application is within a predetermined range. This check ensures the cooperative operation of the application.

  Next, the control mode of the network terminal device used at each point is set. For example, the network terminal device 100 used at the point A on the presenter side is set to a mode (normal control mode) in which an instruction (application control signal) is received from the input instruction unit. In the case of the network terminal device 200 used by other conference participants (for example, the B point), a mode (remote control mode) in which an instruction is received from the outside via the receiving unit is set. (FIG. 3: Step S102). This setting is performed using a control mode signal from the input instruction unit.

  Here, differences in operation of the application control units 102, 202, and 302 depending on the control mode will be described with reference to FIGS. FIG. 4 is a block diagram showing an outline of the internal configuration of the application control unit.

  The first application cooperation unit 102A receives an instruction for cooperatively operating an application external to the network terminal device 100 from the input instruction unit 110, and transmits the instruction to the transmission unit 103.

  The control instruction selection unit 102C selectively outputs the instruction from the input instruction unit 110 and the instruction received by the receiving unit 104 based on the preset control mode. That is, when the control mode is the normal control mode, the instruction input from the input instruction unit is output to the application operation unit, that is, the application. When the control mode is the remote control mode, the instruction received by the reception unit is output to the application.

  The second application cooperation unit 102B transmits the instruction received from the control instruction selection unit 102C to the application operating on the application operation unit.

  Note that 202A, 202B, and 202C of the network terminal device 200 or 302A, 302B, and 302C of the network terminal device 300 are assigned numbers corresponding to 102A, 102B, and 102C of the network terminal device 100, and operate similarly. It is. Therefore, description of 202A, 202B, 202C or 302A, 302B, 302C is omitted here.

  The processing by each of the above parts is described below as a series of flows.

  Each of the network terminal devices 100 and 200 includes address setting units 111 and 211. If address setting is performed on the main body side, bidirectional communication is possible only by executing connection to the network. That is, the user on the network terminal device 100 side at the point A designates a communication partner (here, the network terminal device 200 at the point B), and performs connection via the network. On the other hand, when the connection partner accepts the connection request, bidirectional communication becomes possible (FIG. 3: step S103 → step S104).

  In a state where bidirectional communication is possible, voice communication is possible between the user at point A and the user at another point (for example, point B). In this state, the network terminal device waits for whether there is a control instruction for the application (FIG. 3: Step S105).

  The presenter instructs the network terminal device 100 to switch the application output video from the input instruction unit 110 (or sets the switching timing in advance in the application and instructs the application output video to switch based on the switching timing setting. (FIG. 3: Step S106). The instruction to the application mainly required in the meeting place is a change of the output video of the application, but the “switching the application output video” mentioned here includes all the instructions for the application. That is, even if an instruction other than the switching of the output video is transmitted to the application. In other words, the instruction does not include the output video from the application.

  Upon receiving the presenter's instruction, the application control unit 102 of the network terminal device 100 transmits the instruction to the application operation unit 101. In addition, the application control unit 102 transmits the instruction to the network terminal device 200 used on the other conference participant (for example, the B point) side via the transmission unit 103 (FIG. 3: step S107).

  The network terminal device 200 receives the above instruction transmitted from the network terminal device 100 on the presenter side by the receiving unit 204 and transmits it to the application control unit 202 of the network terminal device 200 (FIG. 3: step S108).

  The application control unit 202 transmits a control instruction to switch the output video from the received application to the application (FIG. 3: step S109). Thereby, the video output from the application on the network terminal device 200 side is switched based on an instruction from the network terminal device 100 at the point A, and as a result, the point B is interlocked with the switching of the application output video at the point A. The application output video can be switched.

  As described above, according to the present invention, the copy data of the material published by the present network user at the presenter side is also set in the network terminal device at the receiving side and the application is operated, Since the video signal output from the application at each site is switched based on the application output switching instruction from the original network terminal device user, it is not necessary to transmit a large amount of image data to the receiving side. As a result, it is possible to reproduce the same video as the video of the presentation content without interruption and without causing a delay or the like at many points away from the point where the announcement is made. Further, since the data received with respect to the video is only an instruction to the application, it is possible to reduce the burden of arithmetic processing inside the apparatus. Naturally, the consistency of the application at each point, the state of the application, and the application data is ensured by communication between the application and data management units.

  The audio signal during the conference is taken into the apparatus from the microphone 108 (208, 308) of the network terminal device at each site and converted from the audio signal to the audio IP packet data by the IP audio conversion unit 106 (206, 306). Is done. Here, audio transmission will be described by taking as an example a case where an audio signal captured by the microphone 108 of the network terminal device 100 is transmitted to the network terminal devices 200 and 300 at other points. The voice IP packet data converted by the IP voice conversion unit 106 of the network terminal device 100 participates in a conference other than a device (for example, the network terminal device 100) that transmits the voice IP packet data from the transmission unit 103 via the network 700. To other network terminal devices (for example, network terminal devices 200 and 300). A device (for example, the network terminal device 200, 300) on the side that receives the voice packet data receives the voice packet data at a receiving unit (for example, 204, 304). The voice packet data received by the receiving unit is converted into a voice signal by an IP voice conversion unit (for example, 206, 306) and reproduced by a speaker (for example, 207, 307). Further, when a plurality of voice packet data is received by the receiving unit of one network terminal device 200, the voice packet data is converted into a voice signal by the IP voice converting unit 206 and then subjected to a synthesis process. Also good. In this way, the voices of a plurality of conference participants can be reproduced in real time. In addition to this, application data can be switched only by sending and receiving display switching instructions as described above, instead of sending display data as in the past, compared to conventional real-time telephone conferences. A multimedia conference can be realized efficiently and economically.

  In the above description, the application output video switching is taken as an example, and the operation on the network terminal device 100 side and the network terminal device 200 side cooperate to switch the output video. However, the present invention is limited to this. Instead of this, it is also possible to realize cooperation of other operations in the same manner by transmitting and receiving instructions for controlling the operation of the network terminal device at each point in the same manner.

(Embodiment 2)
In the second embodiment, an application cooperation system will be described in which content is stored in a server on the Internet, and when content is required, the terminal requests transmission of the content from one of the terminals.

  FIG. 5 is a conceptual diagram for explaining the outline of the application cooperation system according to the second embodiment of the present invention.

  FIG. 6 is a block diagram showing the configuration of the application cooperation system according to Embodiment 2 of the present invention.

  Network terminal apparatuses A100, A200, and A300 are network terminal apparatuses on the user side in the multimedia conference system. The network terminal device A100 is installed on the presenter side, and the network terminal devices A200 and A300 are installed at the bases of the conferences. Further, each of the network terminal devices A100, A200, A300 is configured in the same manner as the network terminal devices 100, 200, 300 of the first embodiment, and has the same function, and redundant description is omitted here. .

  The server A 800 receives a content transmission request from the network terminal device and transmits the content to each network terminal device. However, the server A 800 can be realized by connecting the following units via the bus 806. Note that the server A 800 in FIG. 6 shows only the components necessary for the second embodiment.

  The content storage unit 801 stores content received from the outside via the network 700 or a portable recording medium.

  The user information management unit 802 manages users who transmit content such as conference materials to the server or receive content from the server.

  The receiving unit 804 receives an instruction to receive content from a network terminal device outside the device or to receive the received content to a predetermined network terminal device outside the device.

  The transmission unit 803 transmits the content to a predetermined network terminal device outside the device based on the content distribution instruction received by the reception unit 804.

  The I / F processing unit 805 performs a connection process to the network in order to communicate with the external device of the server A800.

  Next, the operation of the application cooperation system according to the second embodiment will be described with reference to FIGS. FIG. 7 is a flowchart showing the flow of processing in the application cooperation system of the second embodiment. Here, processing different from that of the first embodiment will be mainly described. As in the first embodiment, network terminal device A100 shown in FIGS. 5 and 6 is installed at point A, network terminal device A200 is installed at point B, and network terminal device A300 is installed at point C. And The network terminal device A100 at the point A, the network terminal device A200 at the point B, and the network terminal device A300 at the point C are connected to be communicable via the network 700. Hereinafter, in order to simplify the description, a conference between two points of A point and B point will be described as an example.

  In the application running on the application operating unit 101 of the network terminal apparatus A100, materials used in the announcement are read in advance. And the application control part 102 transmits the instruction | indication output based on the instruction | indication of a presenter or the preset conditions to the said application. As a result, the video output from the application to the projector (not shown) is switched. Further, the transmission unit 103 of the network terminal device A100 transmits the content used in the announcement to the server A800 (FIG. 7: Step S201).

  Subsequently, the server A 800 receives the content from the network terminal device A 100 via the network 700, the I / F processing unit 805, and the receiving unit 804, and stores it in the content storage unit 801 (FIG. 7: step S202).

  Note that the content receiving method is not limited to receiving from a network terminal device via a network, and may be received via a portable medium, for example. Further, the content is not limited to the presenter's material on the network terminal device A100 side, and may be created by other than the network terminal devices A100, A200, and A300, for example. In this case, it goes without saying that use authority at each terminal is required for content created by other than each network terminal device.

  Now, when the network terminal apparatus A100 calls the other terminal A200 to enable two-way communication, the network terminal apparatus A100 transmits a transmission instruction for transmitting the content to the other terminal, together with the address information of the other terminal, to the server A800. (FIG. 7: Step S203).

  Server A 800 transmits the content to the other terminal based on the address information of the other terminal received from network terminal apparatus A 100 (FIG. 7: step S204).

  Each network terminal device receives the content from the server A 800 and reads the content with the application of each network terminal device (FIG. 7: step S205).

  When the content, that is, the application data used in the announcement, has been read into the network terminal device at each point, the mode of the application operating on the network terminal device at each point is set (FIG. 3: step S102).

  Thereafter, the processing is executed sequentially as in the first embodiment (FIG. 3: steps S103 to S109).

  According to the application cooperation system and the application cooperation method of the second embodiment described above, the content is transmitted from the transmission side device to the server before the content is used on the reception side device, and the reception side device passes through the server. Receive content from the originating device. Therefore, the transmission side apparatus can transmit the content to the server at an appropriate time before the content is required at a meeting or the like.

  In the second embodiment, when the network terminal device A100 calls the other terminal A200 to enable bidirectional communication, the content is transmitted together with the address of the other terminal so that the content is transmitted to the other terminal. Although an example in which an instruction is sent to the server A 800 has been described, the timing at which each network terminal device receives content from the server A 800 is not limited to this, and the presenter using the content actually uses the presenter on the network terminal device A 100 side. It is only necessary that the application at each point can be set in a state in which the cooperative operation can be performed before performing.

  For example, the content and the address of the distribution destination network terminal device are transmitted in advance from the network terminal device A100 to the server A800, and the server A800 stores the content in the content storage unit 801 and also stores the content of the distribution destination network terminal device. The address is registered in the user management unit. Further, the network terminal device A100 transmits content acquisition destination information (here, address information of the server A800) to other terminals. When other network terminal device A200 is connected to network 700, it issues a content transmission request to server A800. When the server A 800 receives a content transmission request from another network terminal device A 200, the server A 800 transmits the user address of the user information management unit and the content corresponding thereto.

  In this way, each network terminal device is connected to the network 700 and can receive content by sending a content transmission request to the server A 800 when communication is possible. After being stored in A800, content can be received at an appropriate time according to the circumstances of each point.

In the second embodiment, the network terminal devices A100, A200, and A300 have been described as having the same configuration and function as the network terminal devices 100, 200, and 300 in the first embodiment. However, the terminal device is a network terminal device. Even if the device is not specialized for a conference apparatus as described above, it may be a device having a configuration capable of realizing the functions described here, for example, a personal computer.
(Embodiment 3)
In the third embodiment, an application cooperation system that synchronizes video, audio, and application control will be described.

  FIG. 8 shows a block diagram of an application cooperation system according to Embodiment 3 of the present invention. FIG. 9 shows a block diagram of a time control unit according to Embodiment 3 of the present invention.

  The network terminal device B100 is installed on the presenter side, and the network terminal devices B200 and B300 are installed at the bases of the conferences. Each network terminal device B100, B200, B300 has the same function as the network terminal device 100, 200, 300 of the first embodiment, and time control units 113, 213, 313 are added to each terminal. ing. Here, redundant description of similar functions already described in the first embodiment is omitted.

  Hereinafter, the processing of the time control unit will be described with reference to FIG. 8, FIG. 9, and FIG.

  The time control units 113, 213, and 313 shown in FIGS. 8 and 9 include a multiplexed data generation unit 113A (213A and 313A) and a reproduction timing control unit 113B (213B and 313B). The multiplexed data generation unit 113A (213A, 313A) generates multiplexed packet data by synchronizing the video signal, audio signal, and application control signal with a synchronization signal having a predetermined period. The application control signal is a signal that transmits an instruction to the application. Further, the reproduction timing control unit 113B (213B, 313B) separates the video signal, the audio signal, and the application control signal from the packet data received from the transmission side terminal device so that the video, audio, and application control are synchronized. Control.

  In this example, the synchronization signal is also handled as one channel. However, as in the bit multiplexing method, a synchronization signal for indicating a frame delimiter is embedded at a fixed period when multiplexed data is generated, and this synchronization signal is detected on the receiving side. Thus, the synchronization signal and each bit may identify and synchronize the media corresponding to the video signal, the audio signal, and the application control signal.

  The video input units 114, 214, and 314 shown in FIG. 8 capture video from a digital camera or network camera (not shown) installed on the terminal device side into the terminal device.

  The multiplexed data generation units 113A, 213A, and 313A shown in FIG. 9 receive the video signal from the video input unit 114, the audio signal from the microphone 108, and the application control unit 102 when the terminal device operates as a transmission side terminal device. Packet data in which the output application control signal and the synchronization signal are multiplexed is generated and transmitted to the transmission units 103, 203, and 303.

  The reproduction timing control units 113B, 213B, and 313B in FIG. 9 receive video signals, audio signals, and application control signals from the packet data received by the reception units 104, 204, and 304 when the terminal device operates as a reception terminal device. The synchronization signal is separated and output while synchronizing the video, audio, and application data playback.

  FIG. 10 shows a timing diagram of a video signal, an audio signal, an application control signal, and a synchronization signal in the application cooperation system according to Embodiment 3 of the present invention. FIG. 10A is a timing diagram in the terminal device on the presenter side, and FIG. 10B is a timing diagram in terminal devices other than the presenter.

  The video signal A shown in FIG. 10A is input from a video camera, a still image, or the like input from a digital camera (not shown) connected to the video input unit 114 of the network terminal device B100 that is the presenter terminal device. . Audio signal A shown in FIG. 10A is input from microphone 108 of network terminal apparatus B100. An application control signal A shown in FIG. 10A is a signal for controlling an application inside the network terminal apparatus B100, and is output from the application control unit 102. A synchronization signal A shown in FIG. 10A is a reference signal for synchronizing the video signal A, the audio signal A, and the application control signal A inside the network terminal apparatus B100, and is generated inside the network terminal apparatus B100.

  The video signal B in FIG. 10B is a video or still image input from a digital camera or network camera (not shown) connected to the video input units 214 and 314 of the network terminal devices B200 and B300 which are terminal devices other than the presenter. Is input. Audio signal B shown in FIG. 10B is input from microphones 208 and 308 of network terminal apparatuses B200 and B300. A synchronization signal B shown in FIG. 10B is a reference signal for synchronizing the video signal B and the audio signal B inside the network terminal apparatuses B200 and B300, and is generated inside the network terminal apparatuses B200 and B300.

  First, the network terminal device B100 as the presenter-side terminal device generates packet data obtained by multiplexing each signal as shown in FIG. The transmission unit 103 transmits the information to the outside of the terminal device.

  The network terminal devices B200 and B300, which are terminal devices other than the presenter, receive the packet data and output (reproduce) it in synchronization with the reproduction timing control units 213B and 313B of the time control units 213 and 313. That is, the network terminal apparatuses B200 and B300 display the received video signal A on a part of the display unit (not shown), reproduce the audio signal A with the speaker 107, and instruct the application based on the application control signal. Send. As a result, the application that has received the instruction displays the application data on a part of the display unit (not shown).

  On the other hand, the network terminal devices B200 and B300, which are terminal devices other than the presenter, generate packet data obtained by synchronizing the signals as shown in FIG. Then, the transmission units 203 and 303 transmit the information to the outside of the terminal device. At this time, the network terminal devices B100 and B300 (or B200), which are terminal devices other than the network terminal device B200 (or B300) at the transmitting end device, receive the packet data and receive the time control units 113, 313 (or 213). The reproduction timing control units 113B and 313B (or 213B) output them in synchronization. That is, the network terminal apparatuses B100, B300 (or B200) display the received video signal B on a part of the display unit (not shown) and reproduce the audio signal B with the speakers 107, 307 (or 207).

  Thus, any terminal device other than the presenter can transmit the video signal B and the audio signal B of the terminal device to the other terminal device. In addition, it is preferable that the receiving side device is configured to be able to switch the reproduction of video and audio received from each terminal device. That is, for example, when a presentation is made by a presenter, the video displayed on the display unit (not shown) and the sound reproduced on the speakers 107 and 307 (or 207) are reproduced from the presenter terminal device. However, when there is an opinion or a question from another terminal device side, the playback timing control unit switches to video or audio on the other terminal device side.

  As described above, according to the third embodiment of the present invention, in addition to the video signal and the audio signal, the application control signal and the synchronization signal are transmitted / received between the terminal devices as multiplexed data. Therefore, the amount of data to be transmitted is reduced compared to the conventional method of sending the screen data of the application displayed on the terminal device on the presenter side to other terminal devices, and further the packet data reception processing in other terminal devices is reduced. Therefore, it is possible to control the application (switching of presentation materials) while playing back video and audio at the same timing as the present terminal device even in the terminal device other than the present terminal device. In addition, when the video is of high-definition quality, packet data transmission / reception processing in the terminal device can be particularly reduced, so that the effect on the video / audio gap and the like appears remarkably.

  In addition, since the playback timing control unit switches the video and / or audio according to the input packet, it is possible to grasp the status of other recipients in addition to the status of the presenter.

  The network terminal device, the application cooperation system, the application cooperation method, and the program according to the present invention can share a video in each place without sending a video on a display on the transmission side. Therefore, the present invention is useful as a terminal device of a multi-point multimedia conference system in which a plurality of users participate in a common conference via a network such as a general public telephone network, the Internet network, a DSL network, and a leased line network.

The conceptual diagram for demonstrating the outline | summary of the application cooperation system which concerns on this Embodiment 1 of this invention. The block diagram which shows the structure of the application cooperation system which concerns on this Embodiment 1 of this invention. The figure which shows the flow of a process of the application cooperation system which concerns on this Embodiment 1 of this invention. The block diagram which shows the outline | summary of the internal structure of the application control part of Embodiment 1 of this invention. The conceptual diagram for demonstrating the outline | summary of the application cooperation system which concerns on this Embodiment 2 of this invention. The block diagram which shows the structure of the application cooperation system which concerns on Embodiment 2 of this invention. The figure which shows the flow of a process in the application cooperation system of this Embodiment 2. The block diagram which shows the structure of the application cooperation system which concerns on this Embodiment 3 of this invention. The block diagram of the time control part which concerns on Embodiment 3 of this invention. Timing diagram of video signal, audio signal, application control signal, and synchronization signal in application linkage system Diagram showing the amount of computation and transmission speed required for digital video encoding / decoding processing

Explanation of symbols

100, 200, 300 Network terminal device 101 Application operation unit 102 Application control unit 102A First application cooperation unit 102B Second application cooperation unit 102C Control instruction selection unit 103 Transmission unit 104 Reception unit 105 I / F processing unit 106 IP voice Conversion unit 107 Speaker 108 Microphone 109 Video signal output unit 110 Input instruction unit 111 Address setting unit 112 Bus 113 Time control unit 114 Video input unit 115 Application / data management unit

Claims (7)

An application cooperation system that cooperates two or more applications that have the same function and are operated in each of a transmission-side device connected via a network and one or more reception-side devices,
On the sending device,
A first application operation unit that operates on the transmission side device and operates the first application that receives an instruction that does not include a video and switches the output video by causing the first application to read the application data. When,
The first application operating on the first application operation unit is the same as the first application and the second application operating on the receiving-side device, and having the same function as the first application. The above-mentioned instructions that are applied to any of the above applications, and a screen switching instruction that instructs switching of the output video in each application is set in advance for a user operation on the first application or the first application. A first application control unit that changes the output video of the first application,
A transmission unit that transmits the screen switching instruction to the receiving device when the screen switching instruction is given to the first application;
For each receiving device,
A second application operating unit that receives the electronic file including the content of the application data in advance and causes the second application to read the application data having the same content as the application data, and to operate the second application;
A receiving unit that receives the screen switching instruction transmitted from the transmitting unit;
The second application running on the second application operating unit is given the screen switching instruction received by the receiving unit and not including the video, and the second application A second application control unit that switches the output video based on the application data to the same video in conjunction with the switching of the output video in the first application;
With
At least one of the transmitting device or each receiving device,
Applications running in the other party of the device, the application type, checking of the application state, and with the application data used by the application, the application and the same whether the identity of the device itself And an application cooperation system provided with the application data management part which permits the said cooperation operation | movement when it has identity . The transmission unit includes address information of the receiving device, content used by the second application operating on the receiving device, and a transmission instruction indicating that the content should be transmitted to the receiving device. Send it to a given server,
The receiving unit receives the content transmitted by the server in response to the transmission instruction as the electronic file, and delivers the received content to the second application operating on the receiving device. 1. The application linkage system according to 1. The transmission section transmits the content to the second application running on the receiving side apparatus is utilized in a predetermined server, and further sends the address information of the server to the receiver of the receiving device,
The said receiving part receives the said content as the said electronic file from the said server based on the received said address information , The said received content is delivered to said 2nd application which operate | moves on the said receiving side apparatus. Application linkage system. The originating side apparatus, first synchronizing and the image switching instruction output from the audio signal inputted to the video signal and the audio input unit that is input to the movies image input unit and the first application control unit of further comprising a time control unit,
The receiving unit receives the video signal and the audio signal from the transmission side device in addition to the screen switching instruction,
The receiving device further comprises a second time control unit which received the video signal, the audio signal and synchronizes the the instructions and receives the video and audio to the display switching instruction output by the receiving unit The application linkage system according to claim 1. 5. The application linkage system according to claim 4 , wherein the video signal input from the video input unit is a high-definition quality video signal. An application cooperation method for operating two or more applications that have the same function and operate on each of a transmission side device and one or more reception side devices connected via a network,
A first application operation step that operates on the transmission-side apparatus and operates a first application that switches an output video in response to an instruction not including a video by causing the first application to read application data. When,
An application having the same contents as the application data by receiving an electronic file including the contents of the application data in advance as a second application operating on the receiving-side apparatus and having the same function as the first application. A second application operation step for causing the second application to read and operate the data;
In at least one of said originating unit or the receiving device, the application running at the other side of the device, the application type, the application state, and with the application data used by the application, An application / data management step that checks the identity of whether or not it is the same as the application of its own device, and permits the cooperative operation when there is identity ,
The first application running on the originating device is the same as the first application and the second application running on the receiving device and having the same function as the first application. These instructions apply to both, and a screen switching instruction for instructing switching of output video in each application is performed in accordance with a user operation on the first application or conditions preset in the first application. When the first application control step for switching the output video in the first application and the screen switching instruction is given to the first application, the transmitting side device receives the screen switching instruction. A transmission step of transmitting to the side device;
A receiving step in which the receiving side apparatus receives the screen switching instruction transmitted in the transmitting step;
The second application running on the receiving side device is given the screen switching instruction received in the receiving step and does not include video, and the application data in the second application And a second application control step for switching the output video based on the same video in conjunction with the switching of the output video in the first application. Said transmission step, addition to the screen switching instruction, transmits a voice signal inputted to the video signal and the audio input unit is input to the image input unit to the receiving device, the receiving step, the screen switching instruction In addition to receiving the video signal and the audio signal transmitted in the transmission step,
A first time control step for synchronizing the video signal, the audio signal, and the screen switching instruction output in the first application control step is preceded by the transmission step .
A second time control step of receiving the video signal , the audio signal, and the screen switching instruction received in the receiving step and outputting the video, audio, and screen switching instruction in synchronization with each other is the application control step. Before the
Further application cooperation method according to claim 6 that Yusuke.

Images