JP2020107978A - Information processing apparatus, information processing system, control method, and program - Google Patents

Abstract

To provide a mechanism that can display a video with good quality to a user.SOLUTION: When a client terminal 101 enters a room and a sound is emitted to the client terminal 101, the client terminal distributes sound data of the sound to the room and uploads, to a data management server 104, the sound data and a still image obtained by photographing at a predetermined frame rate. Subsequently, when the client terminal 101 uploads, to the data management server 104, video data obtained by photographing at a frame rate higher than the predetermined frame rate, the data management server composites the video data and the uploaded sound data.SELECTED DRAWING: Figure 8

Description

The present invention relates to a communication method performed by exchanging data.

There is a system capable of bidirectionally transmitting and receiving moving images (video and audio) to communicate (see, for example, Patent Document 1).

JP, 2008-210417, A

Some of such systems have a function of automatically adjusting the image quality of video according to the condition of the communication line and performing bidirectional communication.

Therefore, when the data transfer rate of the communication line is low and the communication band is narrow, the image quality of the image is deteriorated, block noise occurs, and the image being exchanged may be difficult to see.

Therefore, the method of transmitting and receiving a moving image (video and audio) and the video and audio are separated and the video and audio are separately transmitted and received between the terminals, but the video is a high-quality still image at a low frame rate. There is a mechanism that is realized by selecting which of the transmitting and receiving methods.

In the case of wanting to communicate with high-quality video, the latter method can be used to communicate with high-quality video regardless of the status of the communication line.

However, in the latter method, the video and the audio are separated and the still image is exchanged, so that the recorded data has no audio. In addition, since information is exchanged at a low frame rate, there is little information as a video.

Therefore, there is a problem that it is difficult to confirm the content of communication.

Therefore, it is an object of the present invention to provide a mechanism that enables a user to display a high quality image.

The present invention for achieving the above object is an information processing system including an apparatus capable of communicating with a first information processing apparatus and a second information processing apparatus, wherein the first information processing apparatus is a photographing device. First transmitting means for transmitting image data of a predetermined frame rate to the second information processing apparatus, and second transmitting means for transmitting the acquired audio data to the second information processing apparatus and the apparatus A second transmission unit that transmits image data captured at a frame rate higher than the predetermined frame rate to the device, wherein the second information processing device is transmitted by the first transmission unit. The apparatus is provided with a first receiving means for receiving image data and a second receiving means for receiving the audio data transmitted by the second transmitting means, wherein the device is transmitted by the second transmitting means. It is characterized by further comprising: a generation unit configured to combine the voice data and the image data transmitted by the third transmission unit.

According to the present invention, it is possible to display an image with high image quality to the user.

It is a schematic block diagram of an information processing system. It is a block diagram showing an example of hardware constitutions. It is a functional block diagram of an information processing system. Communication is a flowchart showing a communication process. Communication is a flowchart showing a communication process. It is a flow chart which shows processing of operation at the time of starting recording. It is a flow chart which shows processing of operation at the time of end of recording. It is a flow chart which shows processing of operation at the time of composition of recorded data. It is a block diagram which shows an example of a structure of each API. It is a block diagram which shows an example of a structure of each command. It is a block diagram which shows an example of a structure of recorded data. It is a block diagram which shows the structure of a login screen. It is a block diagram which shows the structure of a communication screen. It is a block diagram which shows the structure of a login screen. It is a block diagram which shows the structure of a room selection screen. It is a block diagram which shows the structure of a communication screen. It is a block diagram which shows the structure of a recorded data confirmation screen.

A schematic configuration of the information processing system according to the present embodiment will be described with reference to FIG.

As an example of a schematic configuration of an information processing system according to the present invention, the information processing system includes a client terminal 100, a client terminal 101, a network 102, a communication server 103, and a data management server 104. Are connected so that they can communicate.

In the information processing system, the client terminal 100 displays a login screen 1200 (details described later) and a communication screen 1300 (details described later), and provides a client function for performing various input/output.

It should be noted that these screens can be provided using, for example, a Web browser, but the present invention is not limited to such an aspect and can be provided by a generated application.

In the information processing system, the client terminal 101 displays a login screen, a room selection screen, or a communication screen, and provides a client function for performing various input/output.

As described above, these screens can be provided by using, for example, a Web browser, but the present invention is not limited to such an aspect, and can be provided by a generated application.

The communication server 103 is an external platform, and provides a communication function using a technology (for example, WebRTC or the like) that enables real-time communication of voice and video.

The data management server 104 stores data such as video, and in some cases, synthesizes audio data (hereinafter, audio data) and video data (hereinafter, video data) and provides them to the client terminal.

The data management server 104 includes a recording device 105 and a storage device 106. The recording device 105 combines audio data and video data, and the storage device 106 combines video data and audio data and video data. Store the combined data.

Next, the schematic configuration of the hardware of the client terminal 100 and the client terminal 101 will be described using the flowchart shown in FIG.

As shown in FIG. 2, each device includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input controller 205, a video controller 206, and a memory via a system bus 204. A controller 207 and a communication I/F controller 208 are connected.

The CPU 201 centrally controls each device and controller connected to the system bus 204.

The ROM 202 or the external memory 211 is a control program executed by the CPU 201, such as a BIOS (Basic Input/Output System), an OS (Operating System), a computer-readable executable program for implementing this information processing method, and various necessary programs. Holds data (including data table).

The RAM 203 functions as a main memory, a work area, etc. of the CPU 201. The CPU 201 loads various programs necessary for executing the processing from the ROM 202 or the external memory 211 into the RAM 203, and executes the loaded programs to realize various operations.

The input controller 205 controls input from an input device 209 such as a keyboard or a pointing device such as a mouse (not shown).

When the input device is a touch panel, the user can give various instructions by pressing (touching with a finger or the like) in accordance with an icon, a cursor, or a button displayed on the touch panel.

Further, the touch panel may be a touch panel such as a multi-touch screen capable of detecting a position touched by a plurality of fingers.

The video controller 206 controls display on an external output device such as the display 210.

The display shall include the display of a laptop computer integrated with the main body.

The external output device is not limited to the display, and may be a projector, for example. An input device is also provided for the device that can accept the touch operation described above.

Note that the video controller 206 can control a video memory (VRAM) for performing display control, can use part of the RAM 203 as a video memory area, or can be provided with a dedicated video memory separately. Is also possible.

The memory controller 207 controls access to the external memory 211. The external memory is connected to an external storage device (hard disk), a flexible disk (FD), or a PCMCIA card slot that stores a boot program, various applications, font data, user files, edit files, and various data via an adapter. A compact flash (registered trademark) memory or the like can be used.

The communication I/F controller 208 connects and communicates with an external device via a network, and executes communication control processing on the network. For example, communication using TCP/IP, communication using a telephone line such as ISDN, and communication using a 3G line of a mobile phone are possible.

Note that the CPU 201 enables display on the display 210, for example, by executing outline font rasterization processing in the display information area in the RAM 203. Further, the CPU 201 enables a user instruction using a mouse cursor or the like (not shown) on the display 210.

The camera controller 212 controls shooting by the camera 214. The image obtained by shooting with the camera 214 is stored in a storage area such as the external memory 211 or the ROM 202.

The voice controller 213 controls sound generation and sound collection by the microphone/speaker 215. The sound emitted to the microphone/speaker 215 can be acquired and stored as sound data in a storage area such as the external memory 211 or the ROM 202, or can be produced by adjusting the volume.

Note that the communication server 103 and the data management server 104 also have substantially the same configuration, so description thereof will be omitted.

Next, an outline of each function in the information processing system will be described with reference to FIG. 3, and each function will be described together with a flowchart described later.

The client terminal 100 includes a login unit 300, a room management unit 301, a still image acquisition unit 302, a still image display unit 303, a recording start/end unit 304, and a transmission/reception unit 305.

The login unit 300 has a function of logging in to the information processing system in the client terminal 100, outputs a login screen 1200 (see FIG. 12), performs authentication using the entered user ID and password, and logs in the user. ..

The room management unit 301 has a function of entering and leaving a space (hereinafter referred to as a room) in which voice data, video data, and the like using the communication server 103 are exchanged in real time for communication.

Further, it has a function of delivering voice data such as a voice uttered by a user who has activated the client terminal 100 to a room in which the user has entered. Further, it has a function of generating a recorder activation API 900 (see FIG. 9) and transmitting it to the data management server 104.

The still image acquisition unit 302 has a function of generating a still image acquisition API 930 (see FIG. 9) and using the still image acquisition API 930 to acquire from the storage device 106 a still image captured and uploaded by the client terminal 101. ..

The still image display unit 303 has a function of outputting the still image acquired from the storage device 106 to the communication screen 1300 (see FIG. 13).

The recording start/end unit 304 has a function of generating a recording request command 1000 (see FIG. 10) and transmitting it to the client terminal 101.

The transmission/reception unit 305 has a function of exchanging various data with other devices.

The client terminal 101 includes a login unit 310, a room management unit 311, a still image upload unit 312, a recording start/end unit 313, a video data storage unit 314, a video data upload unit 315, and a transmission/reception unit 316.

The login unit 310 has a function of logging in to the information processing system in the client terminal 101, outputs a login screen 1400 (see FIG. 14), performs authentication using the input user ID and password, and logs in the user. ..

The room management unit 311 has a function of entering and leaving a room, and also has a function of delivering voice data such as a voice uttered by a user who has activated the client terminal 101 to the room.

The still image upload unit 312 generates a still image upload API 920 (see FIG. 9), and using the still image upload API 920, still image data captured by the camera of the device on which the client terminal 101 operates (hereinafter, still image (Data) is transmitted to the storage device 106.

The recording start/end unit 313 has a function of generating a recording start/recording end command 1001 (see FIG. 10) and transmitting it to the recording device 105.

The video data storage unit 314 has a function of storing video data captured by the camera of the device on which the client terminal 101 operates.

The video data upload unit 315 has a function of generating a video data upload API 950 (see FIG. 9) and using the video data upload API 950, transmitting the video data stored in the video data storage unit 314 to the storage device 106. ..

The transmission/reception unit 316 has a function of exchanging various data with other devices.

The recording device 105 includes a login unit 32, a room management unit 321, an audio data storage unit 322, an audio data upload unit 323, a video/audio data acquisition unit 324, a video/audio data synthesis unit 325, a synthetic data upload unit 326, and transmission/reception. The unit 327 is provided.

The login unit 320 has a function of logging in to the information processing system in the recording device 105.

The room management unit 321 has a function of entering and leaving the room, and the voice data storage unit 322 has a function of acquiring and storing voice data distributed to the room.

The voice data upload unit 323 has a function of generating a voice data upload API 940 (see FIG. 9) and using the voice data upload API 940, transmitting the voice data stored in the voice data storage unit 322 to the storage device 106. There is.

The video/audio data acquisition unit 324 has a function of acquiring the video data and audio data stored in the storage device 106 from the storage device 106.

The video/audio data synthesizing unit 325 has a function of synthesizing the video data and the audio data acquired by the video/audio data acquiring unit 324.

The combined data upload unit 326 has a function of generating a combined data upload API 970 (see FIG. 9) and using the combined data upload API 970, transmitting the combined data generated by the video/audio data combining unit 325 to the storage device 106. ..

The transmission/reception unit 327 has a function of exchanging various data with other devices.

The data management server 104 includes an activation unit 330, a data storage unit 331, a data combination instruction unit 332, and a transmission/reception unit 333.

Upon receiving the recorder activation API 900 transmitted by the room management unit 301 of the client terminal 100, the activation unit 330 generates a recording preparation API 910 (see FIG. 9) and has a function of transmitting the recording preparation API 910 to the recording device 105. I have it.

The data storage unit 331 has a function of storing still images, audio data, video data, and composite data generated in the information processing system.

The data synthesis instruction unit 332 detects that the video data upload unit 315 has stored the video data in the storage device 106, and generates the video data/audio data synthesis API 960 (see FIG. 9).

It has a function of transmitting the generated video/audio data composition API 960 to the recording device 105.

The transmission/reception unit 305 has a function of exchanging various data with other devices.

Communication communication processing in the present embodiment will be described using the flowcharts shown in FIGS. 4 and 5.

In step S400, the login unit 300 outputs the login screen 1200 (see FIG. 12), acquires the user information such as the user ID and password input to the login screen 1200, performs authentication, and is authenticated. Then, login to the information processing system is performed.

FIG. 12 shows the configuration of a login screen 1200 output to the client terminal 100. The login screen 1200 includes a user ID input field 1201 for inputting a user ID and a password input field for inputting a password. 1202 and a login button 1203 that is pressed to issue a login instruction.

When the user presses the login button 1203, the user authenticates using the user ID and the password input in the user ID input field 1201 and the password input field 1202, and when the authentication is performed, the user is logged in to the information processing system. ..

In step S401, the room management unit 301 outputs the communication screen 1300 (see FIG. 13).

FIG. 13 shows the configuration of a communication screen 1300 output to the client terminal 100. The communication screen 1300 includes a remote shooting button 1301, a recording start/end button 1302, a room selection list box 1303, and a room entry button 1304. , And a room exit button 1305.

When the remote shooting button 1301 is pressed, a shooting instruction is remotely given to the client terminal 101 of the communication partner.

When the recording start/end button 1302 is pressed, recording can be started, and when the recording starts, the recording start/end button 1302 changes to red.

On the other hand, the recording start/end button 1302 can be pressed while recording is in progress to end the recording, and when the recording ends, the button turns green.

In the room selection list box 1303, the room to be entered can be selected for output, the room entry button 1304 can be pressed to enter the selected room, and the room exit button 1305 can be pressed to enter the room. You can leave the room.

In this step, when it is detected that the room entry button 1304 is pressed while the room output to the room selection list box 1303 is selected, the user is allowed to enter the selected room.

In step S402, the room management unit 301 distributes the sound acquired by the client terminal 100 to the room.

In step S403, the room management unit 301 generates the recorder activation API 900, and the transmission/reception unit 305 transmits it to the data management server 104.

FIG. 9 shows the configuration of the recorder activation API 900. The recorder activation API 900 includes a user ID and a room ID for identifying a room. The room ID is set to the room ID of the room selected in step S401.

The recorder activation API 900 is an API for requesting activation of the recording device 105 from the client terminal 100 to the data management server 104.

In step S404, the transmission/reception unit 333 receives the recorder activation API 900 transmitted from the client terminal 100 in step S403.

In step S405, the activation unit 330 generates the recording preparation API 910, and the transmission/reception unit 333 transmits the recording preparation API 910 to the recording device 105.

FIG. 9 shows the configuration of the recording preparation API 910. The recording preparation API 910 includes a domain (domain of user ID,

The following character strings) and the room ID.

The recording preparation API 910 is an API for making a recording preparation request to the recording device 105, and the recording preparation indicates a process of logging in to the information processing system and entering the same room as the client terminal 100. The recording preparation process can be executed by using the information of the domain and the room ID.

In step S406, the transmission/reception unit 327 receives the recording preparation API 910 transmitted from the data management server 104 in step S405.

In step S407, the login unit 320 logs in to the information processing system using the parameters of the recording preparation API 910 received in step S406.

In step S408, the room management unit 321 causes the recording device 105 to enter the same room as the room entered in step S401.

In step S409, the room management unit 321 generates the recording preparation API response 911, and the transmission/reception unit 327 returns it to the data management server 104.

FIG. 9 shows the structure of the recording preparation API response 911. The recording preparation API response 911 is composed of a message indicating that recording preparation has been made.

In step S410, the transmission/reception unit 333 receives the recording preparation API response 911 transmitted from the recording device 105 in step S409.

In step S411, the transmission/reception unit 333 returns the recorder activation API response 901 to the client terminal 100.

FIG. 9 shows the configuration of the recorder activation API response 901. The recorder activation API response 901 is composed of a message indicating that recording preparation has been made.

In step S412, the transmission/reception unit 305 receives the recorder activation API response 901 transmitted from the data management server 104 in step S411.

Step S413 indicates that the processes of steps S414 to S519 are repeatedly performed at predetermined time intervals (for example, one second intervals).

In step S414, the still image acquisition unit 302 generates a still image acquisition API 930 (see FIG. 9), and the transmission/reception unit 305 transmits the still image acquisition API 930 to the data management server 104.

FIG. 9 shows the configuration of the still image acquisition API 930. The still image acquisition API 930 is a tenant ID (user ID domain,

Subsequent character strings), room ID, and file ID (details will be described later).

The still image acquisition API 930 is an API for making a still image acquisition request stored in the storage device 106 from the client terminal 100 to the data management server 104, and uses a tenant ID, a room ID, and a file ID. Then, a specific still image can be acquired from the storage device 106.

In step S415, the transmission/reception unit 333 receives the still image acquisition API 930 transmitted from the client terminal 100 in step S414.

In step S416, the still image acquisition unit 334 acquires a still image from the storage device 106 according to the parameters of the still image acquisition API 930 received in step S415.

In step S417, the transmission/reception unit 333 returns the still image acquired from the storage device 106 in step S416 to the client terminal 100 with the still image acquisition API response 931 (see FIG. 9).

FIG. 9 shows the configuration of the still image acquisition API response 931. The still image acquisition API response 931 includes a status code (error code in HTTP) and a still image acquisition URL.

When the client terminal 100 receives the still image acquisition API response 931, the client terminal 100 can acquire the still image by using the still image acquisition URL.

In step S418, the transmission/reception unit 305 receives the still image acquisition API response 931 returned in step S417, and in step S419, the still image display unit 303 is included in the still image acquisition API response 931 received in step S418. Display a still image on the screen.

In step S500, the login unit 310 outputs the login screen 1400 (see FIG. 14), acquires the user information such as the user ID and the password input to the login screen 1400, performs authentication, and performs authentication. Then, login to the information processing system is performed.

FIG. 14 shows the configuration of a login screen 1400 output to the client terminal 101. The login screen 1400 includes a user ID input field 1401 for inputting a user ID and a password input field for inputting a password. 1402 and a login button 1403 that is pressed to issue a login instruction.

When the user presses the login button 1403, the user authenticates using the user ID and the password input in the user ID input field 1401 and the password input field 1402, and when the authentication is performed, the user is logged in to the information processing system. ..

In step S501, the room management unit 311 outputs the room selection screen 1500 (see FIG. 15).

FIG. 15 shows the configuration of the room selection screen 1500. The room selection screen 1500 selects a room to enter in the room selection list box 1501 and presses the room entry button 1504 to enter the selected room. You can enter the room.

Also, the video data upload button 1503 can be pressed to upload the video data.

When the return button 1502 is pressed, the login screen 1400 displayed previously is displayed.

When entering the room, a communication screen 1600 (see FIG. 16) is displayed.

FIG. 16 shows the configuration of a communication screen 1600. The communication screen 1600 includes a room leaving button 1601 for pressing when leaving a room, a shooting button 1602 for taking a picture, and a recording start/end button 1603. When is pressed, recording can be started, and when the recording start/end button 1603 is pressed again, recording is ended.

In step S502, the room management unit 311 delivers the sound acquired by the client terminal 101 to the room.

In step S503, the video data storage unit 314 stores the video captured by the camera of the client terminal 101 in the buffer area at a predetermined frame rate (for example, 30 fps per second). The video stored here is used as video data for the subsequent recording process.

Generally, by saving at 30 fps per second, it can be left as high frame rate video data, and by saving in the buffer area of the client terminal 101, it can be left as high image quality video data.

Step S504 indicates that the processes of steps S415 to S530 are repeated at 1 second intervals.

In step S505, the still image upload unit 312 acquires one still image per second from the video data stored in step S503. The still image acquired here is used for communication with the client terminal 100.

In step S506, the still image upload unit 312 generates a still image upload API 920 (see FIG. 9), and the transmission/reception unit 316 uses the still image upload API 920 to acquire the still image acquired in step S505 from the data management server 104. Send to.

FIG. 9 shows the configuration of the still image upload API 920. The still image upload API 920 is composed of a tenant ID, a room ID, a file ID, and binary data of a still image.

The still image upload API 920 is an API for making a still image upload request to the storage device 106 from the client terminal 101 to the data management server 104, and a tenant ID, a room ID, and a file ID (communicate with the other party. Therefore, a still image stored in the storage device 106 can be uniquely managed by using a unique ID.).

In step S420, the transmitting/receiving unit 333 receives the still image upload API 920 transmitted from the client terminal 101 in step S506, and in step S421, the data storage unit 331 acquires the still image included in the still image upload API 920. It is stored in the storage device 106.

In step S422, the transmission/reception unit 333 returns the still image upload API response 921 (see FIG. 9) to the client terminal 101.

FIG. 9 shows the configuration of the still image upload API response 921. The still image upload API response 921 is composed of a status code and a message (a message indicating the processing result).

In step S430, the transmission/reception unit 316 receives the still image upload API response 921 returned in step S429.

The processing of the operation at the start of recording in the present embodiment will be described using the flowchart shown in FIG.

In step S600, when the recording start/end unit 304 accepts that the recording start/end button 1302 on the communication screen 1300 is pressed, it generates a recording request command 1000 (see FIG. 10), and the transmitting/receiving unit 305 causes It is transmitted to the client terminal 101 (the room in which the user has entered).

FIG. 10 shows the structure of the recording request command 1000. The recording request command 1000 includes a command for instructing start/end of recording and a recording flag (whether recording is started or stopped). (Shown) and a recording ID (the ID is issued so as to be unique in order to communicate with the other party). Here, instruction information relating to an instruction to start recording is included as a command.

The recording request command 1000 is a command from the client terminal 100 to the client terminal 101 to instruct start/end of recording.

The recording ID is information for uniquely identifying video data, audio data, and composite data generated by the recording process.

By using the recording ID as the file name of the video data, the audio data, and the composite data, the video data and the audio data can be associated with each other. As a result, even if the video data and the audio data are stored separately, it is possible to generate the composite data in which the video data and the audio data are integrated by the combining process.

In step S601, the transmission/reception unit 316 receives the recording request command 1000 transmitted in step S600.

In step S602, the recording start/end unit 313 acquires the recording start date and time (for example, system date and time).

In step S603, the recording start/end unit 313 generates a recording start/recording end command 1001 and transmits the recording start/recording end command 1001 (see FIG. 10) to the client terminal 100 (the room in which the user has entered). To do.

FIG. 10 shows the structure of the recording start/recording end command 1001. The recording start/recording end command 1001 is composed of a command, a recording flag, a recording ID, and a recording start date and time.

The recording start/recording end command 1001 is a command for instructing the recording device 105 from the client terminal 101 to start/end recording. Here, a command for instructing the start of recording is included.

In step S604, the transmission/reception unit 327 receives the recording start/recording end command 1001 transmitted from the client terminal 101 in step S603.

In step S605, the room management unit 321 acquires the recording ID and the recording start date and time from the recording start/recording end command 1001 received in step S604.

Step S606 shows that the process of step S607 is repeated until the recording start/recording end command 1001 is received again.

In step S607, the audio data storage unit 322 saves the audio data distributed to the room in which the user is entering, in the buffer area of the Web browser.

In step S608, the video data storage unit 314 generates a video data file name in the configuration of the video data 1101 (see FIG. 11).

FIG. 11 shows the structure of the video data 1101. The video data 1101 is video data generated by the client terminal 101, and the recording start date and time, which includes the file name in the recording start/record end command 1001. And the recording ID, and the file name is composed of the recording start date and time and the recording ID included in the recording start/recording end command 1001. Since the file name is composed of the recording ID and the recording start date and time, the video data and the audio data can be linked in the subsequent combining process.

Step S609 indicates that the process of step S610 is repeated until the recording request command 1000 is received again.

In step S610, the video data stored in the buffer area of the client terminal 101 in step S503 is written to a file.

The processing of the operation at the end of recording in the present embodiment will be described using the flowchart shown in FIG.

In step S700, the recording start/end unit 304 generates the recording request command 1000, and the transmission/reception unit 305 transmits the recording request command 1000 to the client terminal 101 (room in which the user is in). At this time, the command includes instruction information relating to the instruction to end the recording.

In step S701, the transmission/reception unit 316 receives the recording request command 1000, and in step S702, the recording start/end unit 313 generates the recording start/recording end command 1001 (see FIG. 11) and enters the room. A recording start/recording end command 1001 is transmitted to. Here, the command includes a command for instructing to end recording.

In step S703, the transmission/reception unit 327 receives the recording start/recording end command 1001 transmitted in step S702.

In step S704, the audio data saving unit 322 ends the audio data saving process started in step S607.

In step S705, the voice data upload unit 323 generates a voice data upload API 940 (see FIG. 9), and the transmission/reception unit 327 uses the voice data upload API 940 to transmit the stored voice data to the data management server 104. ..

FIG. 9 shows the configuration of the voice data upload API 940, and the voice data upload API 940 includes a tenant ID, a room ID, and a file ID.

The voice data upload API 940 is an API for making a voice data upload request from the recording device 105 to the data management server 104 to the storage device 106.

By using the tenant ID, the room ID, and the file ID, the voice data saved in the storage device 106 can be uniquely managed, and the file name of the voice data to be transmitted is the voice data 1100 (see FIG. 11). With this configuration, since the file name is composed of the recording ID and the recording start date and time, the video data and the audio data can be linked in the subsequent synthesizing process.

FIG. 11 shows the configuration of the audio data 1100. The audio data 1100 is audio data generated by the recording device 105, and the recording start date/time, which includes the file name in the recording start/recording end command 1001. And a recording ID.

In step S706, the transmission/reception unit 333 receives the audio data upload API 940 transmitted from the recording device 105 in step S705.

In step S707, the data storage unit 331 acquires the voice data from the voice data upload API received in step S706, and stores the voice data in the storage device 106.

In step S708, the data storage unit 331 generates the audio data upload API response 941 (see FIG. 9), and the transmission/reception unit 333 transmits it to the recording device 105.

FIG. 9 shows the configuration of the voice data upload API response 941. The voice data upload API response 941 is composed of a status code and a broad URL. The recording device 105 can upload the audio data by using the upload URL.

In step S709, the transmission/reception unit 327 receives the voice data upload API response 941 transmitted from the data management server 104 in step S708.

In step S710, the video data storage unit 314 ends the video data file writing process started in step S610, and in step S711, the video data storage unit 314 stores the written video data in the storage area.

Next, the processing of the operation when synthesizing the recorded data in this embodiment will be described using the flowchart shown in FIG.

In step S800, the video data upload unit 315 generates a video data upload API 950 (see FIG. 9).

FIG. 9 shows the configuration of the video data upload API 950, and the video data upload API 950 includes a tenant ID, a room ID, and a file ID.

The video data upload API 950 is an API for making a video data upload request from the client terminal 101 to the data management server 104 to the storage device 106. By using the tenant ID, the room ID, and the file ID, the video data stored in the storage device 106 can be uniquely managed.

Using the video data upload API 950, the video data stored in the storage area of the client terminal 101 in step S711 is transmitted to the data management server 104.

In step S801, the transmission/reception unit 333 receives the video data upload API 950 transmitted from the client terminal 101 in step S800.

In step S<b>802, the data storage unit 331 acquires the video data from the video data upload API 950 received in step S<b>801 and stores it in the storage device 106.

In step S803, the data storage unit 331 generates a video data upload API response 951 (see FIG. 9), and the transmission/reception unit 333 transmits the video data upload API response 951 to the client terminal 101.

The video data upload API response 951 includes a status code and an upload URL, and the client terminal 101 can upload the video data by using the upload URL.

In step S804, the transmission/reception unit 316 receives the video data upload API response 951 transmitted from the data management server 104 in step S803.

In step S805, the data synthesis instruction unit 332 generates the video/audio data synthesis API 960 (see FIG. 9), and the transmission/reception unit 333 transmits it to the recording device 105.

FIG. 9 shows the configuration of the video/audio data composition API 960. The video/audio data composition API 960 includes a video data storage path, an audio data storage path, and a composite data storage path.

The video/audio data composition API 960 is an API for requesting the composition of video data and audio data from the data management server 104 to the recording device 105.

By using the video data storage path and the audio data storage path, the recording device 105 can acquire the video data and audio data to be combined from the storage device 106.

In step S806, the transmission/reception unit 327 receives the video/audio data composition API 960 transmitted from the data management server 104 in step S805.

In step S807, the video/audio data acquisition unit 324 acquires video data from the storage device 106 based on the information of the video/audio data composition API 960 received in step S806.

In step S808, the video/audio data acquisition unit 324 acquires audio data from the storage device 106 based on the information of the video/audio data composition API 960 received in step S806.

In step S809, the video/audio data combining unit 325 performs a combining process of the video data and audio data acquired from the storage device 106 in steps S807 and S808.

In step S810, the video/audio data combining unit 325 saves the combined data 1102 obtained by combining in step S809 as a file.

FIG. 11 shows the structure of the composite data 1102. The composite data 1102 is the composite data generated by the recording device 105, and the recording start date/time of which the file name is included in the recording start/record end command 1001. And a recording ID.

In step S811, the combined data upload unit 326 generates a combined data upload API 970 (see FIG. 9), and the transmission/reception unit 327 transmits the combined data upload API 970 to the data management server 104.

FIG. 9 shows the configuration of the composite data upload API 970, and the composite data upload API 970 includes a tenant ID, a room ID, and a file ID.

The composite data upload API 970 is an API for making a composite data upload request from the recording device 105 to the data management server 104 to the storage device 106. By using the tenant ID, the room ID, and the file ID, the combined data stored in the storage device 106 can be uniquely managed. The composite data upload API 970 is used to transmit the composite data to the data management server 104.

In step S812, the transmission/reception unit 333 receives the combined data upload API 970 transmitted from the recording device 105 in step S811, and in step S813, the data combination instruction unit 332 acquires the combined data from the combined data upload API 970 and stores it. Data is stored in the device 106, and in step S814, the data composition instruction unit 332 generates a composite data upload API response 971 (see FIG. 9), and the transmission/reception unit 333 transmits it to the recording device 105.

FIG. 9 shows the configuration of the composite data upload API response 971. The composite data upload API response 971 includes a status code and an upload URL. The recording device 105 can upload the composite data by using the upload URL.

In step S815, the transmission/reception unit 327 receives the composite data upload API response 971 transmitted from the data management server 104 in step S814.

FIG. 17 shows the structure of a recorded data confirmation screen 1700 for reproducing the combined data stored in the data management server 104. The recorded data confirmation screen 1700 shows the combined data for identifying the combined data. By displaying a thumbnail image or the like and pressing the play button 1701, the synthesized data can be played.

Although the embodiment has been described above, the present invention can be implemented as a system, a method, a program, a recording medium, or the like.

Specifically, it may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of one device.

Further, the program in the present invention is a program that allows the computer to execute the processing method of the flowcharts shown in FIGS. 4 to 8, and the storage medium of the present invention stores the program in which the computer can execute the processing method shown in FIGS. Has been done. The program according to the present invention may be a program for each processing method of each device in FIGS.

As described above, the recording medium recording the program that realizes the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus executes the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, non-volatile memory card, ROM, EEPROM, silicon. A disk or the like can be used.

Further, not only the functions of the above-described embodiments are realized by executing the program read by the computer, but also the OS (operating system) running on the computer is actually executed based on the instructions of the program. It goes without saying that a case where a part or all of the processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

Furthermore, after the program read from the recording medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function expansion board is instructed based on the instruction of the program code. Needless to say, this also includes the case where the CPU or the like included in the function expansion unit performs some or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

Further, the present invention may be applied to a system including a plurality of devices or an apparatus including a single device.

Further, it goes without saying that the present invention can be applied to the case where it is achieved by supplying a program to a system or an apparatus.

In this case, by reading the recording medium storing the program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

Furthermore, by downloading and reading a program for achieving the present invention from a server, a database or the like on a network using a communication program, the system or apparatus can enjoy the effects of the present invention.

It should be noted that the present invention also includes all configurations that combine the above-described embodiments and modifications thereof.

100 client terminal 100
101 Client Terminal 102 Network 103 Communication Server 104 Data Management Server

Claims (5)

An information processing system comprising a device capable of communicating with a first information processing device and a second information processing device,
The first information processing device is
First transmitting means for transmitting the image data of the captured predetermined frame rate to the second information processing device;
A second transmitting means for transmitting the acquired voice data to the second information processing device and the device;
Third transmitting means for transmitting image data captured at a frame rate higher than the predetermined frame rate to the device;
Equipped with
The second information processing device is
First receiving means for receiving the image data transmitted by the first transmitting means,
Second receiving means for receiving the voice data transmitted by the second transmitting means,
Equipped with
The device is
Generating means for synthesizing the audio data transmitted by the second transmitting means and the image data transmitted by the third transmitting means,
An information processing system comprising: A first information processing device that transmits image data captured at a predetermined frame rate to a second information processing device, and a device that can communicate with the second information processing device,
First receiving means for receiving the voice data acquired by the first information processing device;
Second receiving means for receiving image data captured at a frame rate higher than the predetermined frame rate;
Generating means for synthesizing the image data received by the first receiving means and the audio data received by the first receiving means;
An apparatus comprising: A method for controlling an information processing system comprising a device capable of communicating with a first information processing device and a second information processing device,
The first information processing device is
A first transmission step of transmitting the captured image data of a predetermined frame rate to the second information processing device;
A second transmitting step of transmitting the acquired voice data to the second information processing device and the device;
A third transmitting step of transmitting image data captured at a frame rate higher than the predetermined frame rate to the device;
Run
The second information processing device is
A first receiving step of receiving the image data transmitted by the first transmitting step,
A second receiving step of receiving the voice data transmitted by the second transmitting step;
Run
The device is
A generating step of synthesizing the voice data transmitted by the second transmitting step and the image data transmitted by the third transmitting means;
A method for controlling an information processing system, the method comprising: A method for controlling a first information processing device that transmits image data captured at a predetermined frame rate to a second information processing device and a device that can communicate with the second information processing device,
The device is
A first receiving step of receiving voice data acquired by the first information processing device;
A second receiving step of receiving image data captured at a frame rate higher than the predetermined frame rate;
A generating step of synthesizing the image data received by the first receiving step and the audio data received by the first receiving means;
A method for controlling an apparatus, comprising: Computer,
First receiving means for receiving image data captured at a predetermined frame rate, the first receiving means for transmitting the image data to the second information processing apparatus, and the voice data acquired by the first information processing apparatus
Second receiving means for receiving image data captured at a frame rate higher than the predetermined frame rate;
Generating means for synthesizing the image data received by the first receiving means and the audio data received by the first receiving means;
A program to make it work.

Images