JP2020005107A - Communication terminal, data transmission method, and program - Google Patents

Abstract

To provide a communication terminal that prevents a deterioration in quality of video data.SOLUTION: A communication terminal 10 transmits video data picked up by imaging means to another communication terminal 10 connected via a network 2, and comprises: communication band acquisition means 25 that acquires a communication band during the transmission of the video data; pixel number determination means 26 that determines the number of pixels of the video data that can be transmitted in the communication band acquired by the communication band acquisition means; display control means 20 that displays, on a display, a frame indicating the number of pixels determined by the pixel number determination means superimposed on the video data picked up by the imaging means of the communication terminal; and transmission means 12 that transmits the video data in the frame to the another communication terminal.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a communication terminal, a data transmission method, and a program.

  A communication system that enables remote participants to communicate remotely, such as a conference, by transmitting images and voices to each other by a communication terminal at a remote location, displaying the images on a display, and outputting voices from speakers. Are known.

  Such a remote conference may be held at multiple locations, but the performance of each communication terminal tends to vary as the number of locations increases. For example, the network environment at each site and the equipment (camera, microphone, speaker, display, etc.) used by each communication terminal for communication are not the same at each site in most cases. A performance difference has occurred.

  When there is a difference in the performance of the communication terminals (for example, when the resolution of the transmission source camera is higher than the resolution of the transmission destination display), it may be felt that the image quality of the video data has deteriorated. That is, there is a possibility that the information that the participant wants to share cannot be sufficiently shared.

  In order to cope with such inconvenience, a technique for facilitating sharing of information to be shared by participants has been devised (for example, see Patent Document 1). In Patent Document 1, a frame in consideration of the reproduction display capability (for example, resolution, etc.) of each site is displayed on the transmitting side of the content data, and the communication terminal on the transmitting side performs communication using only the information in the displayed frame. A performing delivery system is disclosed.

  However, in the related art, when the resolution of the video data on the transmission side determined according to the reproduction display capability (for example, resolution or the like) on the reception side is larger than the resolution that can be transmitted without delay for the communication environment on the transmission side, the network There is a problem that the quality of video data may be degraded due to delay or interruption of the video data. For this reason, if the communication environment of the video data on the transmitting side is not good and the quality of the video data is deteriorated due to a network delay or interruption, the information to be shared by the participants will not be shared with sufficient quality. there is a possibility.

  In view of the above problems, an object of the present invention is to provide a communication terminal that suppresses deterioration of quality of video data.

  In view of the above problems, the present invention is a communication terminal that transmits video data captured by an imaging unit to another communication terminal connected via a network, and acquires a communication band at the time of transmitting the video data. A communication band acquisition unit, a pixel number determination unit that determines the number of pixels of the video data that can be transmitted in the communication band acquired by the communication band acquisition unit, and a frame indicating the pixel number determined by the pixel number determination unit Display control means for displaying the video data captured by the imaging means of the communication terminal on the display, and transmission means for transmitting the video data in the frame to the other communication terminal. I do.

  It is possible to provide a communication terminal that suppresses quality deterioration of video data.

It is a figure which shows an example of the screen during a meeting which a communication terminal displays on a display. FIG. 1 is an example of a schematic configuration diagram of a communication system. FIG. 2 is a conceptual diagram illustrating an outline of communication in a communication system. FIG. 2 is an example of a hardware configuration diagram of a communication terminal. FIG. 2 is an example of a hardware configuration diagram of a management system. FIG. 3 is a block diagram illustrating an example of a functional configuration of a communication terminal. It is an example of the block diagram which shows the function of a video encoding part in a block form. It is an example of the block diagram which shows the function of an encoding setting part in block form. FIG. 9 is an example of a sequence diagram illustrating a process until a communication terminal connects a session. FIG. 4 is an example of a sequence diagram illustrating a process of transmitting video data while maintaining a sense of resolution of video data. FIG. 3 is an example of a diagram illustrating imaging information provided by an imaging unit. FIG. 4 is a diagram illustrating an example of band information handled by a band information processing unit of a communication terminal. FIG. 11 is an example of a flowchart illustrating a procedure in which an encoding setting unit makes a determination regarding an encoding setting. FIG. 4 is a diagram illustrating an example of an encoding setting set by an encoding setting unit. It is a figure which shows an example of the screen displayed in a video data column and the screen during a meeting which a display control part displays on a display. FIG. 4 is an example of a sequence diagram mainly showing a procedure in which a video encoding unit encodes a video and transmits the video to a relay device. It is a figure explaining operation by a user of the frame displayed on the video data column of the screen during a meeting. It is a figure showing an example of input information. FIG. 4 is an example of a diagram illustrating image processing by a video encoding unit. It is an example of the schematic diagram of a communication system. It is an example of the schematic diagram of a communication system. FIG. 4 is an example of a sequence diagram illustrating a process in which a communication terminal logs in to a management system. FIG. 2 is an example of a functional block diagram illustrating functions of a relay device included in the communication system in a block shape.

  Hereinafter, a communication system, a communication terminal, and a data transmission method performed by the communication terminal will be described with reference to the drawings as an example of an embodiment for implementing the present invention.

<Outline of Operation of Communication Terminal of Present Embodiment>
First, an outline of the operation of the communication terminal 10 of the present embodiment will be described with reference to FIG. FIG. 1 shows an example of a conference screen 301 displayed on the display 11 by the communication terminal 10.
(1) The communication terminal 10 displays the video data of its own base and the video data transmitted from another base on the in-meeting screen 301. The video data at the own site is captured by the camera of the communication terminal 10, and the resolution of the captured image is known. In FIG. 1, it is assumed that the size is 1920 × 1080.
(2) The communication terminal 10 calculates the calculated resolution from the communication band at the time of transmitting the video data. The details of the calculation resolution will be described later, but the calculation resolution is video data that can be transmitted without causing deterioration in quality such as delay or interruption in the current communication band, and more preferably can be transmitted while maintaining a sense of resolution of the video data. Is the resolution. In FIG. 1, 768 × 432 is assumed.
(3) Next, the communication terminal 10 determines whether or not the calculated resolution is smaller than the imaging resolution. If the calculated resolution is smaller than the imaging resolution, transmitting video data at the imaging resolution may cause quality degradation such as delay or interruption of the video data.
(4) Then, the communication terminal 10 displays the frame 302 of the calculated resolution and accepts the arrangement (position change) of the frame 302. The calculated resolution frame 302 is at least the resolution (the number of pixels of the video data) of the video data that can be transmitted without deteriorating the quality of the video data. Therefore, the communication terminal 10 transmits the video data without deteriorating the quality of the video data. it can. Also, although the calculated resolution frame 302 is smaller than the imaging resolution, the conference participant (hereinafter, referred to as a communication terminal user) can change the position of the frame 302, so that the communication terminal 10 transmits information to be transmitted. Can be.

<About terms>
Transmission in the communication band acquired by the communication band acquisition means means that transmission is possible without deteriorating the quality of the video data, and more preferably, the resolution of the video data related to the visual impression is not impaired or impaired. This means that video data can be transmitted without deteriorating only to the extent that it is not noticeable.

  Deterioration of the quality of video data means that video data is delayed or interrupted. Even if the quality of the video data does not deteriorate, the resolution at the time of transmission may be impaired, but if the resolution is not impaired, the quality of the video data can be regarded as not deteriorating. Degradation of the quality of video data may be referred to as degradation of the quality of video data.

  The number of pixels is the number of pixels constituting video data. If the number of pixels is large, the size or the size of the image also becomes large, and thus may be referred to as the size or the size of the image data. The number of pixels is determined by the resolution.

  The resolution is a degree representing the fineness of the digital image, and the resolution is expressed by numerically expressing the fineness of each unit point (dot, pixel) constituting the digital image. In general, “dot” is used as a unit representing the resolution. In the case of a display, in many cases, the number is represented by a number arranged in the horizontal and vertical directions, such as “1024 × 768 dots”.

<System configuration example>
FIG. 2 is a schematic configuration diagram of the communication system 1 of the present embodiment, and FIG. 3 is a conceptual diagram illustrating an outline of communication in the communication system 1 of the present embodiment.

  As shown in FIG. 2, the communication system according to the present embodiment includes a plurality of communication terminals 10 and displays 11 installed at each base, a plurality of relay devices 30, a management system 40, a program providing server 50, A server 60.

  The communication system 1 includes a data providing system for transmitting content data in one direction from one communication terminal 10 to the other communication terminal 10 via the management system 40, and a plurality of communication terminals 10 via the management system 40. It includes a communication system for mutually transmitting information, emotions, and the like. This communication system is a system for mutually transmitting information, emotions, and the like between a plurality of communication communication terminals (corresponding to “communication terminal 10”) via a relay device 30, and includes a television (or video) conference system, An example is a videophone system.

  In the present embodiment, a communication system 1, a management system 40, and a communication terminal 10 will be described, assuming a video conference system as an example of a communication system and a communication terminal as an example of a communication communication terminal. That is, the communication terminal 10 and the management system 40 of the present embodiment are applied not only to a video conference system but also to various communication systems. Hereinafter, a video conference may be simply referred to as a “conference”. Further, the management system 40 can be grasped as a system like a “management device” or as a device (information processing device).

  The display 11 is a display device for displaying video data connected to the communication terminal 10 by wire or wirelessly. Note that the display 11 may have a configuration integrated with the communication terminal 10.

  The communication terminal 10 and the relay device 30 are connected to a router as, for example, a node of a LAN (Local Area Network). The router is a network device that selects a route for data transmission. In the example shown in FIG. 2, the router 70a in the LAN 2a, the router 70b in the LAN 2b, the router 70c in the LAN 2c, the router 70d in the LAN 2d, The router 70e is connected to the router 70a and the router 70b via the dedicated line 2e and connected to the Internet 2i, and the router 70f is connected to the router 70c and the router 70d via the dedicated line 2f and connected to the Internet 2i.

  It is assumed that the LANs 2a and 2b are constructed at different places in the area X, and the LANs 2c and 2d are constructed at different places in the area Y. For example, region X is Japan, region Y is the United States of America, LAN 2a is built in a business office in Tokyo, LAN 2b is built in a business office in Osaka, LAN 2c is built in a business office in New York, and LAN 2d is a building in Washington, DC. C. It is built in the office of. In the present embodiment, the communication network 2 is configured by the LAN 2a, LAN 2b, dedicated line 2e, Internet 2i, dedicated line 2f, LAN 2c, and LAN 2d. The communication network 2 may include not only a wire but also a portion where wireless communication such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark) is performed.

  In the communication system 1 of the present embodiment, transmission and reception of video data and audio data (video data and audio data or content data by adding material data to these data) between the plurality of communication terminals 10 via the relay device 30 are performed. Transmission and reception are also called transmission, and the communication system 1 may be called a transmission system.

  The communication terminal 10 may be a terminal dedicated to the communication system 1 or a general-purpose terminal. The general-purpose terminal is, for example, a PC (Personal Computer), a smartphone, a tablet terminal, or the like, and operates as the communication terminal 10 by executing application software dedicated to the communication system 1.

  As shown in FIG. 3, a management information session Sei for transmitting and receiving various types of management information is established between the plurality of communication terminals 10 via the management system 40. In addition, a data session Sed for transmitting and receiving video data and audio data is established between the plurality of communication terminals 10 via the relay device 30. Here, in particular, the video data transmitted and received in the data session Sed is scalable coded data, for example, high-quality video coded data, medium-quality video coded data, and low-quality video. Are transmitted and received on separate channels. The processing for establishing the management information session Sei or the data session Sed between the plurality of communication terminals 10 can be performed by a known technique, and therefore, detailed description is omitted in the present embodiment. If necessary, for example, the technique described in JP-A-2012-138893 can be used, but the technique is not limited to this technique.

  The management system 40 is a computer (information processing device) that manages the entire communication system 1 of the present embodiment. The management system 40 functions as a call control server that manages the status of the communication terminal 10 and performs call control such as starting a session with two or more communication terminals when an incoming call is received. Since the management system 40 is a server, it has the function of an information processing device. Call control refers to a series of processes for making and disconnecting a call, such as calling for starting communication, receiving the call, responding to the call, and disconnecting the communication. In addition, in many cases, authentication, search, and alive monitoring of the communication terminal 10 are also performed. Further, the management system 40 also manages a database required for controlling the communication system 1, such as a destination list of the communication terminal 10. The destination list is data in which communication terminals (or users) of destinations to which each communication terminal 10 can transmit a request for starting remote communication (make a call) are registered.

  The status is a state of the communication terminal 10 in the communication system 1. The status is mainly divided into whether the call can be received or not. In addition, the management system 40 performs status management of the relay device 30, login management of the user using the communication terminal 10, management of the data session Sed established between the plurality of communication terminals 10, and the like.

  The program providing server 50 is a computer that provides various programs to the communication terminal 10, the relay device 30, the management system 40, the maintenance server 60, and the like. For example, the program providing server 50 stores a communication terminal program for causing the communication terminal 10 to realize various functions, and can transmit the communication terminal program to the communication terminal 10. Further, the program providing server 50 stores a program for the relay device 30 for realizing various functions in the relay device 30, and can transmit the program for the relay device 30 to the relay device 30. Further, the program providing server 50 stores a management system program for causing the management system 40 to realize various functions, and can transmit the management system program to the management system 40. Further, the program providing server 50 stores a maintenance server program for causing the maintenance server 60 to realize various functions, and can transmit the maintenance server program to the maintenance server 60.

  The maintenance server 60 is a computer for maintaining, managing, or maintaining at least one of the communication terminal 10, the relay device 30, the management system 40, and the program providing server 50.

<Example of hardware configuration>
Next, the hardware configuration of the communication terminal 10, the relay device 30, the management system 40, the program providing server 50, and the maintenance server 60 in the communication system 1 of the present embodiment will be described.

<< Example of hardware configuration of communication terminal >>
FIG. 4 shows an example of a hardware configuration of the communication terminal 10, and FIG. 5 shows an example of a hardware configuration of the management system 40. Although the communication terminal 10 in FIG. 4 is assumed to be a dedicated terminal, it can be applied to a general-purpose terminal. The relay device 30, the program providing server 50, and the maintenance server 60 can adopt the same hardware configuration as the relay device 30, and thus the description is omitted.

  As shown in FIG. 4, the communication terminal 10 has a central processing unit (CPU) 101 that controls the entire operation of the communication terminal 10 and a ROM (Read) that stores a program used to drive the CPU 101, such as an IPL (Initial Program Loader). Only Memory) 102. Also, a random access memory (RAM) 103 used as a work area of the CPU 101, and a flash memory 104 for storing various data such as a communication terminal program, video data, and audio data are provided. An SSD (Solid State Drive) 105 that controls reading and writing of various data from and to the flash memory 104 under the control of the CPU 101, and a media drive 107 that controls reading and writing (storage) of data from and to a recording medium 106 such as a flash memory. Prepare. Further, an operation button 108 operated when selecting another communication terminal 10 as a communication partner, a power switch 109 for turning on / off the power of the communication terminal 10, and data transmission using the communication network 2. And a network I / F (Interface) 111 for performing the operation.

  Further, the communication terminal 10 includes a built-in camera 112 that obtains video data by capturing an image of a subject under the control of the CPU 101, an image sensor I / F 113 that controls driving of the camera 112, a built-in microphone 114 that inputs audio, A built-in speaker 115 for outputting sound is provided. In addition, an audio input / output I / F 116 that processes input / output of audio signals between the microphone 114 and the speaker 115 according to the control of the CPU 101 is provided. Further, a display I / F 117 for transmitting display image data to the display 11 under the control of the CPU 101, an external device connection I / F 118 for connecting various external devices, and an alarm lamp 119 for notifying abnormality of various functions of the communication terminal 10. And a bus line 110 such as an address bus or a data bus for electrically connecting the above-described components.

  Note that the camera 112, the microphone 114, and the speaker 115 do not necessarily need to be built in the communication terminal 10, and may be configured to be externally attached. Further, the display 11 may be configured to be built in the communication terminal 10. The display 11 is assumed to be a display device such as a liquid crystal panel, but is not limited thereto, and may be a projection device such as a projector. The hardware configuration of the communication terminal 10 illustrated in FIG. 4 is merely an example, and hardware other than the above may be added.

  The communication terminal program provided from the program providing server 50 described above is stored in, for example, the flash memory 104, read out onto the RAM 103 under the control of the CPU 101, and executed. The memory for storing the communication terminal program may be a non-volatile memory, and is not limited to the flash memory 104, and may be an EEPROM (Electrically Erasable and Programmable ROM) or the like. Further, the communication terminal program may be provided by being recorded on a recording medium such as the recording medium 106 which can be read by a computer in a file in an installable format or an executable format. Further, the communication terminal program may be provided as an embedded program stored in advance in the ROM 102 or the like.

<< Hardware configuration example of management system >>
As shown in FIG. 5, the management system 40 includes a CPU 201 that controls the operation of the entire management system 40, a ROM 202 that stores a program such as an IPL used for driving the CPU 201, a RAM 203 that is used as a work area of the CPU 201, and a relay device. An HD (Hard Disk) 204 for storing various data such as a program for 30 is provided. In addition, an HDD (HD Drive) 205 that controls reading and writing of various data to and from the HD 204 under the control of the CPU 201 and a media drive 207 that controls reading and writing (storage) of data to and from a recording medium 206 such as a flash memory are provided. In addition, a display 208 for displaying various information, a network I / F 209 for transmitting data using the communication network 2, a keyboard 211, and a mouse 212 are provided. Also, a CD-ROM drive 214 that controls reading or writing of various data from and to a CD-ROM (Compact Disc Read Only Memory) 213 as an example of a removable recording medium, and an electrical connection between the above-described components. Bus lines 210 such as an address bus and a data bus.

  The management system program provided from the program providing server 50 described above is stored in, for example, the HD 204 and is read out and executed on the RAM 203 under the control of the CPU 201. Further, the management system program may be provided by being recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 in a file in an installable or executable format. In addition, the management system program may be provided as an embedded program stored in advance in the ROM 202 or the like.

<Function>
FIG. 6 is a block diagram illustrating an example of a functional configuration of the communication terminal 10. As shown in FIG. 6, the communication terminal 10 includes a transmission / reception unit 12, an operation input reception unit 13, an imaging unit 14, an audio input unit 15, an audio output unit 16, a video encoding unit 17, a video decoding unit 18, a login request unit. 19, a display control unit 20, a storage / read processing unit 21, a volatile storage unit 22, a nonvolatile storage unit 23, a band information processing unit 25, and an encoding setting unit 26.

  The transmission / reception unit 12 transmits / receives various data (or information) to / from another communication terminal 10, the relay device 30, the management system 40, and the like via the communication network 2. The transmission / reception unit 12 is realized by, for example, the network I / F 111 and the CPU 101 illustrated in FIG.

  The operation input receiving unit 13 receives various input operations by a user who uses the communication terminal 10. The operation input receiving unit 13 is realized by, for example, the operation button 108, the power switch 109, and the CPU 101 illustrated in FIG.

  The imaging unit 14 captures an image of a site where the communication terminal 10 is installed and outputs image data. The imaging unit 14 is realized by, for example, the camera 112, the imaging device I / F 113, and the CPU 101 illustrated in FIG.

  The voice input unit 15 inputs voice of the base where the communication terminal 10 is installed and outputs voice data. The voice input unit 15 is realized by, for example, the microphone 114, the voice input / output I / F 116, and the CPU 101 illustrated in FIG.

  The audio output unit 16 reproduces and outputs audio data. The audio output unit 16 is realized by, for example, the speaker 115, the audio input / output I / F 116, and the CPU 101 illustrated in FIG.

  The video encoding unit 17 encodes video data output from the imaging unit 14 and audio data output from the audio input unit 15 to generate encoded data. For example, the video encoding unit 17 encodes video data according to H.264. The video data is scalably encoded according to the H.264 / SVC encoding format. The video encoding unit 17 is realized, for example, by the CPU 101 illustrated in FIG. 4 executing an encoding / decoding program (video / audio codec) included in the above-described communication terminal program. The details of the video encoding unit 17 will be described with reference to FIG.

  The video decoding unit 18 decodes encoded data transmitted from another communication terminal 10 via the relay device 30 and outputs video data and audio data before encoding. The video decoding unit 18 is realized, for example, by the CPU 101 illustrated in FIG. 4 executing an encoding / decoding program (video / audio codec) included in the above-described communication terminal program.

  The login request unit 19 transmits login information to the management system 40 together with a communication ID and a password so that the communication terminal 10 logs in to the management system 40. The login request unit 19 is realized, for example, by the CPU 101 shown in FIG. 4 executing a communication terminal program.

  The display control unit 20 performs control for appropriately converting the video data decoded by the video decoding unit 18 into a display video and displaying the video on the display 11 (reproducing and outputting). When the video data decoded by the video decoding unit 18 includes video data transmitted from a plurality of communication terminals 10 at a plurality of bases, the video data may be respectively set according to a predetermined layout setting or a layout setting specified by a user. Is generated in one screen. The display control unit 20 is realized by, for example, the display I / F 117 and the CPU 101 illustrated in FIG.

  The storage / read processing unit 21 performs processing for storing and reading various data in the volatile storage unit 22 and the nonvolatile storage unit 23. The storage / read processing unit 21 is realized by, for example, the SSD 105 and the CPU 101 illustrated in FIG. The volatile storage unit 22 is realized by, for example, the RAM 103 illustrated in FIG. The nonvolatile storage unit 23 is realized by, for example, the flash memory 104 illustrated in FIG. The non-volatile storage unit 23 stores a resolution bit rate conversion table storage unit 29. The information stored in the resolution bit rate conversion table storage unit 29 will be described with reference to Table 1.

  The band information processing unit 25 acquires information on the current band of the communication network 2. The band information used in the present embodiment is mainly the transmission speed of video data. The band information processing unit 25 is realized by, for example, the network I / F 111 and the CPU 101 illustrated in FIG.

  The encoding setting unit 26 calculates, from the bandwidth information and the imaging resolution of the communication network 2, a calculation resolution that can transmit the video data to the relay device 30 without deteriorating at least the quality of the video data and preferably maintaining a sense of resolution. I do. FIG. 8 shows details of the encoding setting unit 26.

<< Video encoding unit >>
FIG. 7 is an example of a block diagram showing the function of the video encoding unit 17 in a block shape. As shown in FIG. 7, the video encoding unit 17 includes a data receiving unit 17a, an image processing unit 17c, an encoding unit 17d, and an input information receiving unit 17b.

  The data receiving unit 17a receives video data to be encoded from the imaging unit 14, and audio data to be coded from the audio input unit 15. Further, when there is a meeting material shared between the communication terminals 10, the material data or the like is received.

  When the user arranges the frame 302 displayed in the video data column 303 at an appropriate position, the input information receiving unit 17b acquires information on the position (hereinafter, referred to as input information) from the operation input receiving unit 13.

  The image processing unit 17c performs image processing on video data based on input information. That is, the range (part) of the video data indicated by the arranged frame 302 is cut out. The encoding unit 17d encodes the video data, the audio data, and the material data, which have been subjected to the image processing by the image processing unit 17c, by an encoding method suitable for each.

<< Video decoding section >>
FIG. 8 is an example of a block diagram showing the function of the encoding setting unit 26 in a block shape. As shown in FIG. 8, the encoding setting unit 26 includes an imaging information receiving unit 26a, a band information receiving unit 26b, and an encoding setting determining unit 26c.

  The imaging information receiving unit 26a receives imaging information from the imaging unit 14. The imaging information includes the imaging resolution, and the imaging information used in the present embodiment is mainly this imaging resolution.

  The band information receiving unit 26b receives band information from the band information processing unit 25. The fact that the data communication speed is high is sometimes referred to as “broadband”, and in the case of digital communication, it is often used in the same sense as the transmission speed. The transmission speed is the amount of data that can be transferred within a certain time. A unit such as “bps (bit per second)” or “B / sec” indicating the amount of data that can be transmitted per second is used. 1kbps is 1000bps, 1Mbps is 1000kbps. 1B (byte) corresponds to 8 bits. In the present embodiment, the transmission speed is mainly acquired as band information at the time of transmitting video data.

  The encoding setting determination unit 26c determines whether or not at least video data having an imaging resolution can be transmitted without deterioration in quality based on the current band information. At the time of the determination, the encoding setting unit 26 refers to the resolution bit rate conversion table storage unit 29 and sets the encoding setting that can transmit the video data without deteriorating the quality of the video data at least and preferably maintaining the resolution. decide. One example of an encoding setting is a calculated resolution. If it is determined that transmission cannot be performed without quality deterioration, the encoding setting determination unit 26c determines that video data is to be transmitted at an encoding setting (calculated resolution) at which video data can be transmitted without deterioration.

表１は、解像度ビットレート変換テーブル記憶部２９に記憶されている解像度ビットレート変換テーブルの一例を示す。解像度ビットレート変換テーブルにはビットレートに対応付けて、解像感を維持して送信できる解像度が登録されている。通信システム１では映像を符号化する際のビットレートによって圧縮率が変わるため、見た目（解像感）も変化する。このため、綺麗に送信することができる映像の解像度はある程度決まっている。

Table 1 shows an example of the resolution bit rate conversion table stored in the resolution bit rate conversion table storage unit 29. In the resolution bit rate conversion table, the resolution that can be transmitted while maintaining the resolution is registered in association with the bit rate. In the communication system 1, since the compression ratio changes depending on the bit rate when encoding a video, the appearance (the sense of resolution) also changes. For this reason, the resolution of a video that can be transmitted neatly is determined to some extent.

Here, “beautiful” means that the video data can be transmitted without deteriorating the resolution of the video data related to the visual impression, or to the extent that even if the resolution is noticed, the video data is not deteriorated.
Thereby, the user of the other communication terminal 10 can see beautiful video data.

  In the example of Table 1, for example, when the bit rate is 4000 kbps, the resolution that can be transmitted without deteriorating the quality of video data is 1920 × 1080. For a bit rate not registered in the resolution bit rate conversion table, the encoding setting unit 26 uses the correspondence between the bit rate and the resolution in the resolution bit rate conversion table to at least not deteriorate the quality of the video data, and A calculation resolution for transmitting the video data while maintaining the resolution is calculated.

  Since the resolution in Table 1 has a constant aspect ratio, the calculated resolution calculated based on the resolution bit rate conversion table in Table 1 also has the same aspect ratio. Here, the aspect ratio indicates the ratio (W: H) of the number of horizontal pixels W to the number of vertical pixels H of the screen.

<Overall operation flow>
<< Flow until meeting start >>
FIG. 9 is an example of a sequence diagram illustrating a process until the communication terminal 10 connects a session. The connection method as shown in FIG. 9 is merely an example, and it is sufficient that the communication terminal 10 in a remote place can start communication. In FIG. 9, one communication terminal 10 is a communication terminal 10A and the other communication terminal 10 is a communication terminal 10B.

  S1: First, when the user of the communication terminal 10A turns on the power switch and performs an operation for logging in, a login request is transmitted to the management system 40. The login request includes the communication ID and the password of the communication terminal 10 that makes the login request. The communication ID is identification information for identifying or specifying a communication terminal, but also has a function of identifying or specifying a user who uses the communication terminal 10. Therefore, the communication ID may be regarded as the user ID. ID is an abbreviation of Identification and means an identifier or identification information. The ID refers to a name, a code, a character string, a numerical value, or a combination of at least one of these, which is used to uniquely distinguish a specific target from a plurality of targets. When the login request is received by the management system 40, the management system 40 can grasp the IP address of the communication terminal 10A on the transmission side.

  S2: The management system 40 authenticates the communication terminal 10 based on the communication ID and the password included in the login request. The authentication method is not limited to the method based on the communication ID and the password, and may use an IC card or biometric authentication information. Here, it is assumed that the authentication has been established.

  S3: Since the authentication is established, the management system 40 updates the status (online because of communication) in association with the communication ID of the communication terminal 10A that is the request source.

  S4: Next, the management system 40 extracts, from the database, a list of destinations (a list of other communication terminals 10 that are destination candidates) from which the communication terminal 10A that has made the login request can start communication. Thereby, the status of the communication terminal 10B as the destination candidate (whether the communication terminal 10B is in a state where communication can be accepted) can be determined.

  S5: Next, the management system 40 transmits the destination state information including the communication ID and the status of the communication terminal 10B of the destination candidate to the communication terminal 10A of the request source.

  S6: In addition, the management system 40 acquires the status of the communication terminal 10A as the request source from a database or the like that manages the status of the communication terminal 10.

  S7: The management system 40 transmits the destination state information of the communication terminal 10A to the communication terminal 10B having the requesting communication terminal 10A as a destination candidate. The destination state information includes the communication ID and the status of the requesting communication terminal 10A. As a result, communication from the communication terminal 10B to the communication terminal 10A can be started.

  S8: The display control unit 20 of the communication terminal 10A that has received the destination state information displays the destination list screen on the display 11. Since the communication terminal 10B and its status are displayed on the destination list screen, the user can call the communication terminal 10 of the destination by grasping the state of the destination and selecting the destination.

  S9: The transmission / reception unit 12 of the communication terminal 10A transmits a conference start request to the management system 40. The start request includes the communication ID of the requesting communication terminal 10A and the communication ID of the destination communication terminal 10B.

  S10: The management system 40 sends a start request to the destination communication terminal 10B. The display control unit 20 of the destination communication terminal 10B displays an incoming screen including the base name of the requesting communication terminal 10A. Since the start request includes the base name of the communication terminal 10A of the request source, the user operating the communication terminal 10B of the destination can determine whether or not to start the video conference.

  S11: When the user operating the destination communication terminal 10B performs an operation to respond, the operation input accepting unit 13 accepts the operation and the transmitting / receiving unit 12 sends a response to the management system 40 that the operation is acceptable. When receiving the response, the management system 40 grasps the IP address of the destination communication terminal 10.

  S12: The management system 40 registers the communication ID, the IP address, and the like of the communication terminal 10A of the request source and the communication terminal 10B of the destination in the relay device 30, and transmits the communication device 10A of the request source and the communication terminal 10B of the destination to the relay device 30. Notify the IP address and the like. The requesting communication terminal 10A and the destination communication terminal 10B request the relay device 30 for permission of communication or the like, and when this is permitted, a session is connected between the communication terminal 10A and the communication terminal 10B. The management system 40 also assigns a meeting ID for identifying the meeting, and associates the communication IDs of the requesting communication terminal 10A and the destination communication terminal 10B with the meeting ID. Thereby, the same content data is distributed to the communication terminals 10 participating in the same conference in the future.

  S13: The transmitting / receiving unit 12 of the requesting communication terminal 10A transmits the video data, the audio data, and the material data to the relay device 30. Similarly, the destination communication terminal 10 starts transmitting the video data, the audio data, and the material data to the relay device 30.

  S14: The relay device 30 transfers the content data to the destination communication terminal 10. As described above, the session is connected, and the communication terminal 10A and the communication terminal 10B can transmit and receive video data, audio data, and material data.

<< Transmission processing of video data in which quality deterioration of video data is suppressed >>
The overall operation flow of the communication terminal when transmitting content data will be described with reference to FIG. FIG. 10 is an example of a sequence diagram illustrating a process of transmitting video data while maintaining the resolution of the video data, and FIGS. 11 to 15 are diagrams illustrating the processes of FIG. 10 in detail.

  FIG. 10 is a diagram showing a resolution at which a user can communicate while maintaining a sense of resolution at the time when deterioration of video data quality such as delay or interruption due to poor communication environment may occur. A process of presenting a frame 302 is shown.

  S21: First, the imaging unit 14 of the communication terminal 10 sends imaging information (imaging resolution) to the encoding setting unit 26. Details of the imaging information will be described with reference to FIG.

  S22: Next, the band information processing unit 25 sends the band information to the encoding setting unit 26. Details of the band information will be described with reference to FIG.

  S23: The imaging information receiving unit 26a of the encoding setting unit 26 of the communication terminal 10 receives imaging information (imaging resolution) from the imaging unit 14 of the communication terminal 10.

  S24: Similarly, the band information receiving unit 26b of the encoding setting unit 26 receives band information from the band information processing unit 25 of the communication terminal 10.

  S25: Similarly, the encoding setting determining unit 26c of the encoding setting unit 26 determines the encoding setting (calculation resolution). Details of this processing will be described with reference to FIGS.

  S26: The display control unit 20 of the communication terminal 10 displays a frame 302 in the video data column 303 of the in-meeting screen 301 based on the obtained encoding setting. FIG. 15 shows an example of the frame 302 displayed in the video data column 303 of the in-meeting screen.

<< About imaging information (imaging resolution) >>
FIG. 11 is an example of a diagram illustrating imaging information provided by the imaging unit 14. The imaging information has an imaging resolution, a frame rate, and a data format currently managed by the imaging unit 14. In the example of FIG. 11, the imaging resolution is 1920 × 1080, the frame rate is 30 fps, and the data format is I420. The imaging resolution is the resolution of video data actually captured by the imaging unit 14. The frame rate is the number of images captured per second of video data. I420 is one of the types of methods for thinning out color difference components in the YUV color space.

  The imaging unit 14 collectively stores information such as settings when the imaging unit (camera) acquires a video as imaging information.

<< Band information (transmission speed of video data) >>
FIG. 12 is a diagram illustrating an example of band information handled by the band information processing unit 25 of the communication terminal 10. FIG. 12A shows the types of data transmitted and received between two communication terminals 10. Video data, audio data, and material data are transmitted and received. The relay device 30 determines the image quality of the video data according to the communication band and the like. In FIG. 12, high-quality and low-quality video data are transmitted and received. The image quality of the material data can also vary. Since the material data is not always transmitted, one line is used for transmission and reception (in FIG. 12A, the line is transmitted from the communication terminal 10A to the communication terminal 10B).

  FIG. 12B shows an example of band information of one communication terminal 10. The bandwidth information processing unit 25 measures video data, audio data, material data, and the total bandwidth from actual communication results of various data, and collectively creates these as bandwidth information. The actual communication result is the communication speed when the communication terminal 10 receives the video data to be displayed on the display 11 at the time of reception. In the case of transmission, this is the communication speed at the time when the communication terminal 10 transmits the image data captured.

  Taking the reception band information of FIG. 12B as an example, the reception band information is large, so it can be determined that the video data can be properly received. However, this does not apply if there is a sudden deterioration in the communication environment. Although the transmission speed is relatively high, there is a possibility that an appropriate coding setting is required at least to suppress the deterioration of the quality.

  In FIG. 12B, the actual communication speed is used as band information that can be used as it is. Among the actual communication speeds, the communication speed at the time of reception can be calculated by the band information processing unit 25 measuring how many bits (bps) are received per second for each of the video data, the audio data, and the material data. For example, measurement is performed ten times, and the average is calculated. As for the communication speed at the time of transmission, the communication terminal 10 may acquire the communication speed measured when the relay device 30 receives the video data from the communication terminal 10 from the relay device 30. Alternatively, when the communication terminal 10 and the relay device 30 are directly connected (when not connected via a network device such as a router), the transmission speed indicates how many bits (bps) the communication terminal 10 has transmitted per second. It may be.

  Instead of using the actually transmitted and received video data, audio data, and material data, the communication speed at the time of transmitting and receiving the dummy data may be used.

  Instead of the average, for example, the maximum communication speed within a certain fixed period may be obtained, and the value may be treated as a usable communication band.

  Further, the band information processing unit 25 may use a server that provides a speed test. In this case, the script is transmitted from the server to the communication terminal 10, and the communication speed at the time of reception of the script is determined from the difference between the time stamp when the reception of the dummy data transmitted by the server is completed and the time stamp transmitted by the server. calculate. Conversely, the transmission speed at the time of transmission may be such that the script transmits dummy data to the server.

  The delay time may be calculated by a generally obtained method. For example, the relay device 30 can send the time stamp to the communication terminal 10 together with the video data, and can calculate the bandwidth information by comparing the time when the receiving communication terminal 10 received the time stamp with the time stamp.

  Further, for example, it may be calculated based on an RTT (Round Trip Time) value. The RTT value is a time taken from transmitting a signal or data to a communication partner until a response returns. Therefore, the RTT is the sum of the reception and transmission delay times.

  Also, an index that is not the delay time itself but may be correlated with the delay time may be used. For example, when TCP communication is performed, the number of NACKs for a predetermined time may be used as a delay parameter. A TCP NACK means that the communication resulted in a negative result. Since the number of NACKs increases when the bandwidth is narrow, the number of NACKs may have information equivalent to the delay time.

<<< Process of Encoding Setting Unit >>>
FIG. 13 is an example of a flowchart illustrating a procedure in which the encoding setting unit 26 makes a determination regarding the encoding setting.

  First, the encoding setting determination unit 26c starts encoding setting based on the acquired imaging information (imaging resolution) and band information (S101).

  The encoding setting determining unit 26c refers to the resolution bit rate conversion table and calculates a calculation resolution from which the image quality (appearance) of the transmitted video data becomes clear from the band information (S102). An example of calculating the calculated resolution will be described with reference to the resolution bit rate conversion table in Table 1. First, it is assumed that the band information (transmission speed of video data) is 1200 kbps. If there is a resolution associated with 1200 kbps, it is used for the resolution bit rate conversion table.

If there is a resolution associated with 1200 kbps, it is calculated as follows if it is not in the resolution bit rate conversion table. In the resolution bit rate conversion table of Table 1, the bit rate closest to 1200 kbps is 1000 kbps. It is assumed that the correspondence between 1000 kbps and this bit rate resolution of 640 × 360 can also be applied to 1200 kbps and 1200 kbps resolutions.
640 × 360: calculated resolution = 1000: 1200
Therefore, the calculated resolution = (640 × 360) × 1200/1000
= 768 × 432
From the above, the calculated resolution is 768 × 432. In this way, it is possible to calculate a resolution that allows clear transmission even with a band value that is not registered in the resolution bit rate conversion table. In the above calculation method, the calculation is performed on the assumption that the aspect ratio of the video data is maintained.

  Next, the encoding setting determination unit 26c compares the calculated resolution with the imaging resolution and specifies the smaller one (S103).

  If the imaging resolution is smaller, the video data can be transmitted as it is, so the encoding setting determination unit 26c sets the imaging resolution to the encoding setting (S104). In this case, a frame 302 described later need not be displayed. This is because displaying the frame 302 having the same number of pixels as the video data column 303 has little significance. However, since the user can be notified that the entire video data in the video data column 303 has been transmitted, the frame 302 may be displayed in the video data column 303 also in step S140.

  If the calculated resolution is smaller, only the video data having the calculated resolution or less can be transmitted, so the encoding setting determination unit 26c sets the calculated resolution to the encoding setting (S105).

  As described above, the smaller of the calculation resolution and the imaging resolution can be determined as the number of pixels of the frame 302, and therefore, after calculating the calculation resolution, almost no processing load is required for determining the frame 302. In the above example, the calculated resolution may be 768 × 432 or less.

  In addition, the encoding setting determination unit 26c sets the pixel number change flag to "necessary" because image processing of the video data captured by the imaging unit 14 is required (S106). The pixel number change flag is included in the encoding setting, and is referred to by the video encoding unit 17 to determine whether to cut out a part of the video data.

  The encoding setting determination unit 26c ends the determination regarding the encoding setting (S107). Thereby, the encoding setting is completed. FIG. 14 shows an example of the encoding setting.

<< Example of encoding setting >>
FIG. 14 is a diagram illustrating an example of the encoding setting set by the encoding setting unit 26. The encoding settings handled by the encoding setting unit 26 include the imaging resolution acquired from the imaging unit 14, the calculated resolution at which the video data calculated by the encoding setting determination unit 26c can be transmitted neatly, the frame rate acquired from the imaging unit 14, and , The number of pixels needs to be changed. In the example of FIG. 14, since the pixel number change flag is necessary, the video encoding unit 17 changes the pixel number of the transmitted video data to the calculated resolution.

<< Frame displayed in the video data section of the meeting screen >>
FIG. 15 shows an example of a conference screen 301 displayed on the display 11 by the display control unit 20 and a frame 302 displayed in the video data column 303. When the display control unit 20 receives the encoding setting as shown in FIG. 14, it first checks whether or not the number of pixels needs to be changed.

  When the item of the number of pixels change is “necessary”, the display control unit 20 displays the video data column 303 on the in-meeting screen 301 in which the video data captured by the communication terminal 10 of the transmission source is displayed. A frame 302 having the same number of pixels as the calculation resolution of the encoding setting is displayed. That is, a frame 302 having a size (size) corresponding to the number of pixels is displayed. The frame 302 indicates a range of video data to be encoded by the video encoding unit 17. For example, the frame 302 in the initial state is preferably located at the center of the video data column 303, but the frame 302 may be arranged anywhere in the video data column 303. Alternatively, the display control unit 20 may perform face recognition, and arrange the frame 302 so as to include the face.

  The aspect ratio of the frame 302 may be the same as the aspect ratio of the video data column 303. However, if the number of pixels (area) is the same even if the aspect ratio changes, the communication terminal 10 can transmit the image data while suppressing the quality deterioration of the video data at least, and the aspect ratio of the frame 302 is the same as that of the video data column 303. It is not necessary.

<Process when transmitting video data>
As described above, the frame 302 is displayed in the video data column 303 of the in-meeting screen, so that the user can operate the frame 302, perform image processing, and encode video data. Hereinafter, the operation of the frame 302 by the user, image processing, and encoding of video data will be described with reference to FIGS. 16 to 19.

  FIG. 16 is an example of a sequence diagram mainly illustrating a procedure in which the video encoding unit 17 encodes a video and transmits the video to the relay device 30. The processing in FIG. 16 is executed following the processing in FIG. FIG. 16 is a diagram showing a case where the communication environment on the transmission side that can be used at that time is used when the communication environment on the transmitting side of the content is poor and quality deterioration such as delay or interruption due to the poor communication environment may occur. This is a continuation of the sequence of the process of “presenting the resolution frame 302 that can be transmitted while maintaining the feeling.” FIG. 16 describes a process of “using video data in the frame 302 for actual communication”.

  S31: First, the user operates the frame 302 displayed on the in-meeting screen 301 of the communication terminal 10. The operation input receiving unit 13 of the communication terminal 10 holds a result of an operation performed by the user as input information. This operation is mainly movement of the frame 302. The operation of the frame 302 by the user will be described in detail with reference to FIG. 17, and an example of the input information will be described with reference to FIG.

  S32: The imaging unit 14 of the communication terminal 10 sends the video data obtained by the imaging to the video encoding unit 17.

  S33: The data receiving unit 17a of the video encoding unit 17 of the communication terminal 10 acquires the video data from the imaging unit 14 of the communication terminal 10.

  S34: The input information receiving unit 17b of the video encoding unit 17 acquires the input information from the operation input receiving unit 13.

  S35: The image processing unit 17c of the video encoding unit 17 performs image processing on the video data based on the input information. The image processing will be described in detail with reference to FIG.

  S36: The video encoding unit 17 of the communication terminal 10 encodes the video data.

  S37: The transmission / reception unit 12 of the communication terminal 10 transmits the encoded video data to the relay device 30.

  S38: Since the relay device 30 relays the video data to the communication terminal 10B, the transmission / reception unit 12 of the communication terminal 10B receives the video data. Since the resolution of the video data is limited by the frame 302, the video data can be received while maintaining the resolution.

<< About user operation >>
FIG. 17 is a diagram illustrating an operation by the user of the frame 302 displayed in the video data column 303 of the in-meeting screen. FIG. 17A is an example of a diagram illustrating the movement of the frame 302 by the user. When the frame 302 is displayed in the video data column 303 of the in-meeting screen 301, the user can operate the communication terminal 10 to move the frame 302 arbitrarily. The movable range is a range that overlaps with the video data column 303 (the range in which the user's video is displayed). The operation input receiving unit 13 receives the operation of the user and informs the display control unit 20 of the position of the frame 302. Therefore, the display control unit 20 displays the frame 302 at the position moved by the user.

  As a means for the user to move the frame 302, a button on a hardware, a keyboard, an operation on a touch panel, or the like can be considered. For example, the user can move the frame 302 to the face so that the face portion is displayed on the communication terminal 10 of the other party. Further, the user can move the frame 302 to the material so that the material held by the user is displayed on the communication terminal 10 of the other party. Alternatively, the display control unit 20 may perform face recognition and cause the frame 302 to follow the face so as to include the face.

  As shown in FIG. 17B, the operation input receiving unit 13 may receive a change in the aspect ratio of the frame 302 while keeping the number of pixels of the frame 302 constant. This is because if the number of pixels is constant, at least quality deterioration at the time of transmitting video data is suppressed. For example, this is effective when it is desired to transmit a video of vertically long material data to another communication terminal 10.

  The operation input receiving unit 13 may receive a request to reduce the number of pixels of the frame 302. This is because there is no change in that the quality degradation during transmission is suppressed as the number of pixels of the video data decreases. For example, this is effective when a part of the video data is not desired to be transmitted.

  In addition, the user is prohibited (limited) from increasing the number of pixels of the frame 302. That is, the operation input receiving unit 13 does not receive an operation in which the frame 302 is larger than the calculated resolution. When the user performs an operation to increase the number of pixels of the frame 302, an error message is displayed, for example, as illustrated in FIG. In FIG. 17C, a message 450 indicating that “the frame cannot be larger than 768 × 432” is displayed. Accordingly, it is possible to prevent the frame 302 from being erroneously enlarged when the user moves the frame 302.

<< input information >>
FIG. 18 shows an example of the input information. The input information is information indicating at which position of the frame 302 the user has placed the frame 302. FIG. 18A shows the input information shown together with the frame 302 arranged in the video data column 303 of the in-meeting screen.

  In FIG. 18, the starting point S1 and the ending point E1 of the horizontal axis of the frame 302 and the starting point S2 and the ending point E2 of the vertical axis of the frame 302 are represented by ratios to the video data column 303, respectively, with the lower left corner of the video data column 303 as the origin O. . In FIG. 18A, the start point S1 on the horizontal axis is a position 40% from the origin, and the end point E1 is a position 80% from the origin O. The start point S2 on the vertical axis is a position 40% from the origin O, and the end point E2 is a position 95% from the origin O.

  FIG. 18B is a diagram showing the input information determined in this way by numerical values. Where the origin O is, the starting point S1 and the ending point E1 on the horizontal axis of the frame 302 and the starting point S2 and the ending point E2 on the vertical axis of the frame 302 are represented by numerical values.

<< Image processing >>
FIG. 19 is an example of a diagram illustrating image processing by the video encoding unit 17. When the video encoding unit 17 acquires the input information as shown in FIG. 18, the image processing unit 17c of the video encoding unit 17 cuts a part of the video data indicated by the frame 302 of the input information from the video data. Trimming image data may be referred to as trimming.

  FIG. 19A shows an in-meeting screen 301 in which the arrangement of the frame 302 is indicated by the input information, and FIG. 19B shows a part of the video data cut from the video data by the image processing unit 17c. Clipping a video corresponds to image processing.

  As described above, in the present embodiment, resizing processing (reducing or compressing) is not performed on the entire video data so as to have a target resolution, but the image processing unit 17c performs the clipping processing. Of the video data can be reduced without deteriorating the quality of the video data and preferably maintaining the resolution. The video encoding unit 17 encodes the video data after the image processing.

<Summary>
As described above, the communication system 1 of the present embodiment can expect the following effects by designating the target range by the user and transmitting only that range by image processing.
1. Even if the communication environment is different at each site, the reduction or reduction in the data size (the number of pixels) of the video data can avoid delay or interruption of the video data due to the communication environment.
2. Even if the communication environment deteriorates, the communication terminal 10A transmits the video data to another communication terminal 10B without changing the resolution in the range specified by the user, so that the quality of information that the user wants to communicate with can be maintained. .

  In this embodiment, a configuration example of a system configuration diagram and another example of a communication ID will be described.

<Another example of the system configuration diagram>
In FIG. 2 described in the first embodiment, the number of management systems is one, but a plurality of management systems 40 may exist in the communication system 1. FIG. 20 is a schematic diagram of the communication system 1 according to the present embodiment. In FIG. 20, a plurality of management systems 40 are connected to the communication network 2. As described above, the management system 40 included in the communication system 1 may be plural, and any management system 40 may have a function.

  In addition, the system configuration in which the communication terminal 10 and the management system 40 described in the present embodiment are connected is an example, and it goes without saying that there are various system configuration examples according to applications and purposes.

  As shown in FIG. 21, the management system 40 and the relay device 30 may exist as one server device 80. In FIG. 21, server devices 80 are arranged in regions X and Y, respectively. The server device 80 has both the function of the management system 40 and the function of the relay device 30. In addition, there is a form in which a plurality of management systems 40 exist. The functions related to the management system 40 of each server device 80 communicate with each other to synchronize the status of the communication terminal 10 and the like.

  The server device 80 may be connected to the Internet 2i, or may be only one in each of the regions X and Y, and the configuration shown is an example.

  The communication terminal 10 may communicate with another communication terminal 10 without relaying by the relay device 30.

<User identification by email address>
In the present embodiment, the communication terminal 10 is identified by the communication ID, but a user ID may be used as identification information for identifying the communication terminal 10. Also, a mail address may be used as the user ID. In this case, the user of the communication terminal 10 logs in to the management system 40 with the mail address.

  FIG. 22 is an example of a sequence diagram illustrating a process in which the communication terminal 10 logs in to the management system 40. In the description of FIG. 22, differences from FIG. 9 will be mainly described. As shown in FIG. 22, in steps S1, S5, S7, S9, and S10, not the communication ID but the mail address is transmitted. The management system 40 identifies the communication terminal 10 and the user with the mail address and the password.

<Other application examples>
As described above, the best mode for carrying out the present invention has been described using the embodiment. However, the present invention is not limited to the embodiment, and various modifications may be made without departing from the gist of the present invention. And substitutions can be made.

  For example, the processing performed by the communication terminal 10 may be performed by the relay device 30 or the management system 40. FIG. 23 is an example of a functional block diagram showing functions of the relay device 30 included in the communication system in a block shape. In FIG. 23, the relay device 30 includes a band information processing unit 25, an encoding setting unit 26, and a resolution bit rate conversion table storage unit 29.

  In the configuration of FIG. 23, the bandwidth information processing unit 25 of the relay device 30 measures bandwidth information of video data transmitted by the communication terminal 10. Further, the encoding setting unit 26 calculates the calculated resolution based on the band information with reference to the resolution bit rate conversion table. Since the calculated resolution is transmitted to the communication terminal 10, when the calculated resolution is smaller than the imaging resolution, the display control unit 20 displays a frame 302 having the calculated resolution or less, and the operation input receiving unit 13 receives the arrangement of the frame 302 by the user. . As described above, the calculation of the calculation resolution can be performed by the server on the network.

  In FIG. 23, the relay device 30 calculates the calculated resolution so that the relay device 30 relays the video data. However, when the management system 40 relays the video data, the management system 40 has the same function.

  6 are divided according to main functions in order to facilitate understanding of processing by the management system 40, the communication terminal 10, and the relay device 30. The present invention is not limited by the way of dividing the processing unit or the name. The processing of the management system 40, the communication terminal 10, and the relay device 30 can be divided into more processing units according to the processing content. Further, it is also possible to divide the processing unit so that one processing unit includes more processing.

  Each function of the embodiment described above can be realized by one or a plurality of processing circuits. Here, the “processing circuit” in the present specification refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), SOC (System on a Chip), GPU (Graphics Processing Unit), and devices such as conventional circuit modules.

  Note that the band information processing unit 25 is an example of a communication band acquiring unit, the encoding setting unit 26 is an example of a pixel number determining unit, the transmitting and receiving unit 12 is an example of a transmitting unit, and the display control unit 20 is a display control unit. The resolution bit rate conversion table is an example of the pixel number communication speed conversion information, the image processing unit 17c is an example of the image processing unit, and the operation input receiving unit 13 is an example of the operation receiving unit.

REFERENCE SIGNS LIST 1 communication system 10 communication terminal 11 display 30 relay device 40 management system 80 server device

WO 2015/045788

Claims (8)

A communication terminal that transmits video data captured by an imaging unit to another communication terminal connected via a network,
Communication band acquisition means for acquiring a communication band at the time of transmitting the video data,
A pixel number determining unit that determines the number of pixels of the video data that can be transmitted in the communication band obtained by the communication band obtaining unit,
Display control means for displaying a frame indicating the number of pixels determined by the number-of-pixels determination means on a display so as to overlap the video data imaged by the imaging means of the communication terminal,
Transmission means for transmitting the video data in the frame to the other communication terminal,
A communication terminal comprising: An operation receiving unit that receives the arrangement of the frame displayed by the display control unit superimposed on the video data,
Image processing means for cutting out the video data in the frame in which the operation receiving means has received the arrangement,
The communication terminal according to claim 1, wherein the transmission unit transmits the video data cut by the image processing unit to the another communication terminal. The number-of-pixels determining means determines the number of pixels associated with the communication band acquired by the communication band acquiring means based on a correspondence between the number of pixels registered in the pixel number communication speed conversion information and the communication band. By calculating
The communication terminal according to claim 1, wherein the number of pixels of the video data that can be transmitted in the communication band is determined.   The number-of-pixels determination unit is the communication band, the smaller of the number of pixels associated with the communication band acquired by the communication band acquisition unit and the number of pixels of the video data imaged by the imaging unit. The communication terminal according to claim 3, wherein the number of pixels of the video data that can be transmitted is determined. The operation accepting unit accepts a change in aspect ratio while maintaining the number of pixels of the frame superimposed on the video data,
The image processing unit cuts out the video data within the frame of the aspect ratio for which the operation receiving unit has received the change, and the transmitting unit transmits the video data cut out by the image processing unit to the other communication terminal The communication terminal according to claim 2, wherein:   The communication terminal according to claim 5, wherein the operation accepting unit restricts an operation of increasing a size of the frame displayed on the video data by a user. A data transmission method performed by a communication terminal that transmits video data captured by an imaging unit to another communication terminal connected via a network,
Communication band obtaining means for obtaining a communication band when transmitting the video data,
Pixel number determining means, determining the number of pixels of the video data that can be transmitted in the communication band acquired by the communication band acquisition means,
Display control means, a step of displaying a frame indicating the number of pixels determined by the pixel number determination means on the display over the video data imaged by the imaging means of the communication terminal,
Transmitting means for transmitting the video data in the frame to the other communication terminal,
A data transmission method, comprising: To another communication terminal connected via a network, a communication terminal that transmits video data captured by the imaging unit,
Communication band acquisition means for acquiring a communication band at the time of transmitting the video data,
A pixel number determining unit that determines the number of pixels of the video data that can be transmitted in the communication band obtained by the communication band obtaining unit,
Display control means for displaying a frame indicating the number of pixels determined by the number-of-pixels determination means on a display so as to overlap the video data imaged by the imaging means of the communication terminal,
Transmitting means for transmitting the video data in the frame to the other communication terminal,
Program to function as.

Images