JP5895921B2 - Captured video processing apparatus, control method, and program - Google Patents

Description

  The present invention relates to a photographed video processing apparatus, a control method, and a program equipped with a function of replaying a moving image photographed by a user.

  Patent Document 1 discloses a game device that replays a game play from a predetermined time before the occurrence of the event to the time of the occurrence of the event during the game play (A29 YES → A 31 → A 33 NO in FIG. 11). A13YES → A17YES → A19). On the replay screen of the game device, the virtual camera is photographed with a plurality of characters related to the generated event, and the captured moving images of the characters are arranged and proceed simultaneously (see FIG. 2 of Patent Document 1). FIG. 3 → FIG.

  Patent Document 2 discloses a game system in which a plurality of game terminals and monitor terminals installed in each store are connected to a common server on the Internet (FIG. 1 of Patent Document 2). In this game system, when a famous scene appears on a game terminal, game data necessary for replaying the famous scene is transmitted from the game terminal to the server via the Internet and stored in the server (FIG. 2). 30A22YES → A23 → A24 → A25). When there is a request for transmission of replay data from the monitor terminal, the server transmits the corresponding replay data to the monitor terminal (Patent Document 2 FIG. 34 E13 → E14 → E15), and the monitor terminal replays the famous scene received from the server. Data is stored (Patent Document 2 FIG. 34E16), the replay data is read as appropriate, and the name scene is replayed on the display (Patent Document 2 FIG. 35F13).

JP 2007-31001 A JP 2006-6853 A

  Different users have the same instructional video software and game software, and each user performs exercises such as dance and gymnastics while watching those videos on his / her video recording device, and the motion is recorded by the imaging device It is possible to do. In that case, each user may want to refer to how other users are exercising.

Patent Document 1 reproduces a game play of a predetermined period until the event occurrence with respect to the event occurrence of the game play, and arranges a plurality of images obtained by capturing a plurality of objects in the game space with a plurality of virtual cameras, and simultaneously proceeds. It is only displayed, and no specific means for disclosing the user's video of another device for the same teaching materials or games is disclosed. Patent Document 2 uploads name scene data appearing at each game terminal to a server on the Internet, accumulates it in a database of the server, and the server distributes name scene data to each monitor terminal. It only discloses that the scenes can be viewed individually, and does not disclose a specific method for allowing other users' videos of the same teaching materials or games to be viewed on the own machine.

  An object of the present invention is to provide a photographed video processing apparatus, a control method, and a program that can effectively use a photographed moving image of a behavior performed by another user on a predetermined basic moving image.

  According to the present invention, the photographed video processing apparatus can be used by a user of another device who has performed a predetermined software moving image (the software moving image includes not only game software and DVD software moving images but also TV program moving images). Data of the moving image taken by the behavior is acquired via the network, and the other user's captured video is displayed on the display unit in contrast to the basic moving image and the own user's captured video.

The captured image processing apparatus of the present invention includes the following.
Own user photographing data storage means for storing own user photographing data including information of the basic video in which the own user during a period of displaying the basic video on the display is stored in a storage device;
Upload unit for uploading the own apparatus user captured data via the communication means to directly or photographic data management server to another machine, and the recording the other machine user periods displaying the basic image in another machine display other machine user capturing data including information of the basic video, the basic search and on the basis of the video information, retrieved other machine user captured data directly from another machine or photographed other machines to get through communication means from the data management server User photographing data acquisition means,
The other device user-captured video during the period of displaying the basic video based on the basic video and the searched other-device user captured data, the first frame of the basic video or the frame of a predetermined event as a reference frame, Composite video generation means for generating a composite video in which display timings of the frames of the other-device user video corresponding to a reference frame are aligned, and display control means for displaying the composite video generated by the composite video generation means on the display .

The captured image processing apparatus control method of the present invention includes the following steps.
Own device user image data storing step for storing own device user image data including information of the basic image in which the own device user is recorded during a period during which the basic image is displayed on the display;
An uploading step of uploading the user's own user photographing data directly to another device via a communication means or to a photographing data management server;
Other device user image data including information on the basic image in which the other device user is recorded during the period in which the basic image is displayed on the display device of the other device is searched based on the information on the basic image, and the other device user who has searched is searched Other device user image data acquisition step for acquiring image data directly from another device or from the image data management server via communication means,
The other device user-captured video during the period of displaying the basic video based on the basic video and the searched other-device user captured data, the first frame of the basic video or the frame of a predetermined event as a reference frame, A composite video generation step of generating a composite video in which display timings of the frames of the other-user user captured video corresponding to the reference frame are aligned; and
A display control step of displaying the composite video generated by the composite video generation step on the display;

  The program of the present invention causes a computer to function as each means of the photographed video processing apparatus.

  According to the present invention, the user can compare the captured moving image of the behavior performed by the other device user with respect to the same basic moving image as the basic moving image with the basic moving image or the own captured moving image.

It is a schematic block diagram of a game system. It is a block diagram of a game device. It is a flowchart of the 1st video recording method. It is a figure which shows the specific example of game information. It is a flowchart of the 2nd video recording method. It is a flowchart of the 3rd recording method. It is a flowchart of a data upload method. It is a figure which shows the game information already registered in the game information database of a server. It is a flowchart of the 1st contrast moving image replay method. It is a figure which shows the synthesized image of a game image | video. It is a flowchart of the 2nd contrast moving image replay method. It is a block diagram of a picked-up image processing device. It is a flowchart of a picked-up image processing device control method.

  FIG. 1 is a schematic configuration diagram of the game system 10. The game system 10 includes the Internet 11, a plurality of game devices 12 and 13, and a server 14. The plurality of game apparatuses 12 and 13 and the server 14 are connected to the Internet 11 and can exchange data with each other. In FIG. 1, for convenience of illustration, only two game devices 12 and 13 are illustrated, but in an actual game system 10, a large number of game devices are connected to the Internet 11. The game devices 12 and 13 and the server 14 can be connected by a dedicated line instead of the Internet 11. Further, in the present embodiment, the game apparatuses 12 and 13 are supposed to indirectly exchange data via the server 14 instead of directly.

  In the present embodiment, the game device 12, the AV output device 19, and the imaging device 20 are deployed at the home of the user 18. The user 18 can also use the game apparatus 12, the AV output apparatus 19, and the imaging apparatus 20 that are deployed in a physical education facility or a game center other than home. The AV output device 19 is equipped with a display 21 and left and right speakers 22.

  The user 18 sets instructional software and various game software on the game device 12, then stands in front of the AV output device 19, and remotely controls the game device 12, the AV output device 19, and the imaging device 20 with a remote controller (not shown). Operate and give instructions to play or start the software. Along with this, the moving image and sound of the software content are output from the display 21 and the speaker 22, respectively. Note that the user 18 can obtain the various software from a predetermined server via a communication line such as the Internet as well as a DVD or other recording medium storing the software. In the present embodiment, the content is a model performance such as beauty gymnastics or dance, and the model performance of the teacher 23 is displayed on the display 21. The user 18 performs in accordance with the exemplary performance of the teacher 23 on the display 21. Further, the imaging device 20 captures the action of the user 18 and outputs a video signal to the game device 12. Note that the imaging device 20 may be of a built-in microphone type, and the voice of the user 18 and surrounding sounds can be collected by the microphone, and the sound signal can also be output to the game device 12. The game device 12 stores the video signal and / or audio signal data from the imaging device 20 in the second compression unit 32 (FIG. 2) in a predetermined compression standard format.

  The game apparatus 13 is provided at a location different from the game apparatus 12 and is used by a user different from the user 18. Similarly to the game device 12, the game device 13 is connected to the AV output device and the imaging device at the place where the game device 13 is provided, and is used by other users in the same manner as the game device 12.

  FIG. 2 is a block diagram of the game apparatus 12. The game apparatus 12 includes a game recording medium 25, a video signal input unit 26, an audio signal input unit 27, a video signal output unit 28, an audio signal output unit 29, a communication unit 30, a first recording unit 31, and a second recording unit. 32, a control unit 33, a playback unit 34, and an operation input unit 35. These game recording medium 25, video signal input unit 26, audio signal input unit 27, video signal output unit 28, audio signal output unit 29, and communication unit 30. The first recording unit 31, the second recording unit 32, the control unit 33, the reproduction unit 34, and the operation input unit 35 exchange data with each other via a common bus.

  The game recording medium 25 is a recording medium such as a CD-ROM or DVD, and stores instructional data, game AV (Audio Visual) data, instructional software, and game software programs. The video signal input unit 26 and the audio signal input unit 27 are input with the video signal and the audio signal from the imaging device 20 and the built-in microphone, respectively. The video signal output unit 28 and the audio signal output unit 29 output the video signal and the audio signal to the AV output device 19, respectively.

  The communication unit 30 includes a communication module for connecting to a wired LAN, a wireless LAN, Bluetooth, a mobile phone network, and the like. The communication unit 30 is connected to the Internet 11 via the wired LAN and transmits / receives data to / from the server 14. .

  The first recording unit 31 includes a detachable flash memory such as a USB memory or an SD card, for example, and allows game information (FIG. 4) to be readable and writable. The second recording unit 32 includes, for example, a hard disk device, a DVD-R, or a DVD-RW, and records compressed data related to the video signal and the audio signal from the imaging device 20 for reading and writing. The control unit 33 is composed of a CPU and controls each element in the game apparatus 12.

  The playback unit 34 reads out the content data of the game recording medium 25 and the first and second recording units 31 and 32 in accordance with an instruction from the user 18, the progress of the game, and the like, and generates video and audio playback signals. The video signal is output to the video signal output unit 28 and the audio signal output unit 29. The operation input unit 35 is a user interface of the game apparatus 12. The operation input unit 35 is typically a game controller, but receives all user operations from the user 18 on the game device 12, the AV output device 19, and the imaging device 20 in a lump.

  FIG. 3 is a flowchart of the first recording method 45. The game apparatus 12 can employ the first recording method 45 (FIG. 3), the second recording method 60 (FIG. 5), or the third recording method 68 (FIG. 6) as a recording method.

  In S <b> 46, the user 18 operates the operation input unit 35 to instruct game recording. Along with this, game software (including instructional video software) of the game recording medium 25 is activated, and the imaging device 20 starts outputting a shooting signal. Even when output from the imaging device 20 is started, recording of the output is not started.

  In S47, it is determined whether or not the imaging device 20 has recognized the user 18. Specifically, user recognition is achieved by mounting a known face recognition technique, human sensor, or the like on the imaging apparatus. As soon as the determination in S47 becomes positive, the process proceeds to S48. Note that S47 may be omitted. In S <b> 48, recording of the video signal from the imaging device 20 is started in the second recording unit 32. In the signal recording to the second recording unit 32, the audio signal from the built-in microphone of the imaging device 20 can be recorded in parallel. The video signal data and the audio signal data may be recorded as separate files or as the same file.

  In S49, recording of the game information to the first recording unit 31 is started. FIG. 4 shows a specific example of game information. Each row in FIG. 4 corresponds to the game information, and the user 18 executes a plurality of software in the game device 12, so there is game information for a plurality of game titles. As game information items, there are a game title, a game progress, a shooting start date and time, and a shooting end date and time.

  The game progress status is specified from the game start position at the shooting start date and time and the game end position at the shooting end date and time. In the example of FIG. 4, the game progress status is not expressed by two pieces of information of the game start position and the game end position, but is indicated by a game range from the game start position to the game end position. The game range is “2-1-3” and “2-1-4-27” in FIG. 4, and the left number indicates a larger break. For example, “2-1-3” of “Yoga Master” in FIG. 4 indicates the game range of Chapter 2 Section 1 Section 3 of “Yoga Master”. "-3" from the beginning to the end means the time from the photographing start time to the photographing end time.

  The game information is recorded in the first recording unit 31 in S52 described later. The game information before recording in the first recording unit 31 is temporarily stored in the RAM of the control unit 33. In S49, the game title, the start position of the game progress status, and the shooting start date and time are determined in the game information. The end position of the game progress status and the shooting end date and time are determined at the time of execution of S52 described later.

  In S50, it is determined whether or not there is an instruction to end shooting from the user 18, and as soon as the determination becomes positive, the process proceeds to S51. The user 18 gives an instruction to end photographing by operating a key on the operation input unit 35. In S51, the process of recording the video signal from the imaging device 20 in the second recording unit 32 ends. In S52, the process of recording the game information in the first recording unit 31 is terminated. That is, the game information is recorded as a single file in the first recording unit 31.

  FIG. 5 is a flowchart of the second recording method 60. In the second recording method 60, the same steps as the steps of the first recording method 45 (FIG. 3) are denoted by the same numbers as the corresponding steps of the first recording method 45, and the description thereof is omitted. Only will be described.

  S61 of the second recording method 60 is executed in place of S50 of the first recording method 45. In S61, it is determined whether or not a predetermined event in the game has ended, and the process proceeds to S51 as soon as the determination becomes positive. Specifically, when a certain stage of the game is cleared, when one instructional unit (such as one yoga pose) is completed in the instructional content, or when scoring in a sports game (baseball home run or soccer The goal scene is determined to be at the end of the predetermined event. Note that event end may be understood as the occurrence of an end event. That is, an event is defined by being divided into a start event (S69 in FIG. 6) and an end event, and event end means the occurrence of an end event. The end event is, for example, a home run in a baseball game or a goal in a soccer game.

  In S51 and S52 of the second recording method 60, the recording of the photographed video and the game information is finished in agreement with the end of the event of S61. If the event end is a scoring scene, the game information prior to the start time is recorded with the video and game information recording start time of the imaging device 20 as a time that is traced back for a predetermined period from the event end time. It may be deleted.

  FIG. 6 is a flowchart of the third recording method 68. In the third recording method 68, the same step as the step of the second recording method 60 (FIG. 5) is denoted by the same number as the corresponding step of the second recording method 60, and the description is omitted. Only will be described.

  S69 is inserted between S47 and S48. In S69, it is determined whether or not a game event is started, and the process proceeds to S48 as soon as the determination becomes positive. The start of the event can also be regarded as the occurrence of the start event. Examples of the start event include the start of a certain stage in the game, the start of one instructional unit in the instructional content, the start of the game in the sports game, the first half and the second As well as attacks and baseball changes in baseball games.

  S70 is executed after S52. In S70, it is determined whether or not an instruction to end the game has been issued by the user 18, and if the determination is positive, the third recording method 68 is terminated, and if the determination is not, the process returns to S69. Specifically, the power operation of the game apparatus 12, the depression of the reset button, the selection of another game, and the like can be given. A game over that occurs as a game event may be determined as a game end instruction.

  As a result of returning to S69 in S70, as a result of returning to S69, in the third recording method 68, a file of photographed video data and a file of game information are created for each predetermined progress period portion of the game play period.

  FIG. 7 is a flowchart of the data upload method 75. The data upload method 75 is executed after the recorded video and the game information are recorded by performing the first recording method 45, the second recording method 60, or the third recording method 68.

  In S <b> 76, the user 18 is caused to select a captured video of the user 18 to be uploaded to the server 14 from among the captured video already recorded in the second recording unit 32. Specifically, the user 18 displays (a) a thumbnail in association with each captured video that has been recorded in the second recording unit 32, selects one or more desired thumbnails, and (b) ) After searching by a specific term, recording date and time, etc., further sorting is performed to display a word / phrase associated with a desired captured video and select it.

  In S77, it is determined whether there is game information associated with the related game information, that is, the captured video of the user 18 selected in S76. As an example when there is no related game information, there is a case where the user 18 intentionally or mistakenly deletes the game information recorded in the recording process of S52 (FIG. 3, FIG. 5 or FIG. 6). If there is game information, the process proceeds to S78; otherwise, the process proceeds to S85. In S78, the captured video and game information are uploaded to the server 14. The server 14 temporarily stores the captured video and game information uploaded from the game device 12.

  In S79, it is determined whether or not game information that can be associated with the uploaded game information has been registered in the game information database of the server 14, and if the determination is positive, the process proceeds to S81. If not, the process proceeds to S80. FIG. 8 shows game information registered in the game information database of the server 14. Of the items of the game information (FIG. 4) of the game apparatus 12, only the game title and the game progress status are the items of the game information in the game information database of the server 14. If the game information uploaded from the game device 12 to the server 14 has already been registered in the game information database of the server 14 if the pair of the game title and the game progress status matches, the determination in S79 is positive. .

  In S80, it is determined whether or not the user 18 has selected game information on the server 14 associated with the captured video uploaded in S78. If the determination is positive, the process proceeds to S81, and if not, the process proceeds to S89. That is, if the game information associated with the uploaded video image is not registered in the game information database of the server 14, it is not determined where in the server 14 the video image should be registered. The user 18 is made to select the registration location. The user 18 registers the captured video in the corresponding location in association with the game title, event, and progress. Note that the server device may send a game title, an event, a video image of the game according to the progress status, and audio information in response to a request from the user, and confirm by reproducing the video on the user terminal. . If the user 18 sets a game title to be associated with the uploaded photographed video in S80, for the same game information, the determination in S79 becomes positive and the repeated execution of S80 is avoided.

  In S81, the captured video temporarily stored in the server 14 is officially stored in the storage device of the server 14 in association with the game information on the server 14 side. On the other hand, in S89, game information is excluded from the captured video and game information uploaded and temporarily stored in S78, and only the captured video is stored in the storage device of the server 14. In the storage of the captured video in S89, basic information such as a user ID, a registration date, and a recording date is registered in association with the captured video so that the server 14 can search for a desired captured video later.

  In S85, it is displayed that there is no game information. In S86, it is determined whether or not the user 18 has selected game information on the server 14 to be associated with the video to be uploaded. If the determination is positive, the process proceeds to S87, and if not, the process proceeds to S88. . The user operation in S86 is the same as the user operation in S80 described above.

  In S87, the user 18 uploads the captured video and the game information of the own device in association with the game information selected in S86. The uploaded game information and photographed video are temporarily stored in the server 14, and then, with the execution of S <b> 81, the photographed video temporarily stored in the server 14 is stored in the server 14 associated with the game information on the server 14 side. It is formally stored in the storage location of the device. In S88, only the captured video is uploaded to the server 14, and S89 is executed after the execution of S88 (S89 has already been described in S80).

  If NO in S77, that is, if there is no game information associated with the captured video to be uploaded in the game apparatus 12, the user 18 selects game information on the server 14 side to be associated with the captured video to be uploaded. (S86 is correct), the game information and the shot video are uploaded to the server 14 (S87). First, the game information and the shot video are uploaded to the server 14, and then the server 14 side associates with the uploaded shot video. It is also possible to cause the user 18 to select game information and then perform S81.

  FIG. 9 is a flowchart of the first contrast moving image replay method 100. In S101, the server 14 is accessed to select another user's video to be displayed. Specifically, the user 18 selects a desired event from a user ID, a game name, an event name during the game, and the like regarding the captured video registered in the video database of the server 14.

  In S102, it is determined whether or not the second recording unit 32 has its own captured video related to (specifically corresponding to) the captured video of the other user selected by the user 18, and the determination is positive. If not, the process proceeds to S103, and if not, the process proceeds to S105. In S <b> 103, it is determined whether the user 18 has selected whether or not to display the related own captured video in the second recording unit 32 along with the selected other user's captured video on the display 21. If display is selected, the process proceeds to S104, and if display is not selected, the process proceeds to S105.

  In S <b> 104, settings are made to play back the captured video of another user and the captured video of the second recording unit 32. In S105, a setting is made to reproduce only the captured video of another user.

  In S106, it is determined whether or not there is a game video related (specifically corresponding) to the game information of the captured video set in S104 or S105. If the determination is positive, the process proceeds to S107. If so, the process proceeds to S109. Specifically, the determination in S106 is performed based on whether or not there is a related game image in the game recording medium 25. Note that if it is possible to download a related game video from the server 14 instead of the game recording medium 25, the determination in S106 may be positive.

  In S107, the user 18 is selected to display or not display the game video, and if it is selected to display, the process proceeds to S108, and if not displayed, the process proceeds to S109. In S108, display of the game video is set. In S109, a setting is made not to display the game video.

  In S110, based on the information set in S104, S105, S108, and S109, a moving image obtained by synthesizing another user's captured video, own captured video, and / or game video is output to the display 21.

  FIG. 10 shows a composite image of a captured video of another user, a captured video of the user, and / or a game video. G1, G2, and G3 correspond to a game video, a video shot by the user, and a video shot by another user, respectively. Such a composite image can be realized by using a known picture-in-picture or screen division technique. In FIG. 10, the pose direction of the G3 person (game video: teacher) is opposite to the pose direction of the G1 and G2 persons (other machine user and own machine user: student). This is because the student stands at the position facing the display 21 and imitates the game image displayed on the display 21 in a left-right relationship as seen from the pose ("triangular pose" in the illustrated example). This is because photographing is performed by the imaging device 20 on the output device 19 side. In the composite image of G1, G2, and G3 on the display 21, if it is unsightly that the left and right sides of G2 and G3 with respect to G1 are reversed, a mirror image inversion process is performed only for the G1 image, and a composite image is generated. It may be displayed on the display 21.

Of course, the display timing between the game video G1, the video shot G3 of the other user and / or the video shot G2 of the other user in the synthesized image of the video shot of the other user, the video shot of the user, and / or the game video is naturally. Must be consistent. In order to match the display timing, for example, one or more predetermined frames (for example, the first frame or a predetermined event occurrence frame) of the game video are selected as the reference frames, and the reference frames are displayed on the AV output device 19. A composite image in which information identifying the frame of the captured video at the time is recorded separately from the captured video, and the frame of the captured video associated with the reference frame is displayed at the same time with reference to the specific information Create a frame.

  FIG. 11 is a flowchart of the second contrast moving image replay method 130. In the second contrast video replay method 130, the same step as the step of the first contrast video replay method 100 (FIG. 9) has the same number as the corresponding step of the first contrast video replay method 100. A description will be omitted, and only the differences will be described.

  In S <b> 131, the user 18 is caused to select his / her own captured video recorded in the second recording unit 32 of the game apparatus 12. In S132, the server 14 is accessed, and the captured video database of the server 14 is searched based on the game information recorded in the first recording unit 31 in association with the captured video selected in S131. It is determined whether or not there is an image. If the determination is positive, the process proceeds to S134, and if not, the process proceeds to S133.

  In S133, it is displayed that there is no other user's captured video, and the second contrast moving image replay method 130 is terminated. In S134, the user 18 is made to select whether or not to play back the captured video of another user, and when the user 18 selects not to play back, the second contrast moving image replay method 130 is terminated and playback is selected. If so, the process proceeds to S135.

  In S135, if the user 18 selects whether or not to reproduce a plurality (two or more) of the other user's captured images as the corresponding captured images in S132, and the user 18 selects not to reproduce a plurality of images, The process proceeds to S137. In S136, the user 18 is caused to select a plurality of corresponding captured videos of other users, and a setting of a plurality of reproductions is performed. In step S137, the user 18 is caused to select one corresponding captured video of another user, and playback is set. After execution of S136 or S137, the process proceeds to S106.

  In the second contrast moving image replay method 130, when S137 and S108 are executed, the screen of FIG. 10 is displayed on the display 21 of the AV output device 19 after the execution of S110. Note that G2 (video of the own device) may not be displayed from the screen of FIG. 10, and video of G1 (video of the game) and G3 (video of the other device) may be displayed. When S136 and S108 are executed, after the execution of S110, the display 21 of the AV output device 19 displays images of one or more users other than the user of G3 on the screen of FIG. An image is displayed.

  In addition, S103-S105 of the 1st contrast moving image replay method 100 (FIG. 9) is inserted between S136 or S137, and S106 of the 2nd contrast moving image replay method 130, and also the picked-up image of the user 18 is also included. It can also be displayed on the display 21 along with one or more other users' captured images.

  FIG. 12 is a block diagram of the captured video processing device 150. An example of the captured video processing device 150 is the game device 12 (FIG. 1). The captured video processing apparatus 150 includes a display control unit 151, an other device user captured data acquisition unit 152, and a comparison video generation unit 153. The video is a moving image and / or a still image.

  The “basic video”, “contrast video”, and “partial video” used below are defined. The “basic video” is, for example, a soft video generated from instructional video software or game software, or a video of content such as a gymnastic program for television broadcasting. In that case, it is necessary to record the content and generate a contrast video having the playback recording as a partial video. The basic video is typically a basic video of software installed in the captured video processing device 150, but may be a basic video downloaded from a server. “Partial video” means a video obtained by partially capturing videos that are compared with each other. The “contrast video” is an entire video for comparison including a plurality of partial videos.

  The display control unit 151 displays the basic video on the display 161. The other device shooting data acquisition unit 152 acquires the other device user shooting data in which the other device user is recorded as a video while the basic image is displayed on the display of the other device via the communication unit 162. The comparison video generation unit 153 compares the partial video with the basic video and the other device user-captured video based on the other device user shooting data as a partial video, and matches the display timings of the corresponding frames of the partial videos. For video. The display control unit 151 displays the comparison video generated by the comparison video generation unit on the display 161.

  A specific example of one scene of the contrast video is the display screen of the display 21 in FIG. The display screen includes a display of the own user, but the own user may be excluded. A specific example of the communication means 162 is the Internet 11 (FIG. 1). The communication means 162 may be a LAN or other network provided in one facility with a plurality of captured video processing devices 150 connected. The other device user shooting data acquisition unit 152 acquires the other device user shooting data typically through the shooting data management server 164, but may acquire it directly from the other device without using the shooting data management server 164. And

  The number of captured images of other device users included as a partial image in the comparison image is not limited to one. It may be 2 or more (for example, S136 of the second contrast moving image replay method 130 in FIG. 11).

  In order to adjust the time axis of each partial video in the contrast video, for example, association information is recorded for each frame of each partial video displayed simultaneously. The association information does not need to be set for all frames, but only the frames of each partial video corresponding to a specific frame of the contrast video or basic video (for example, a frame at the start of display of a soft video or a predetermined event). Can be set about.

  In this way, the user can refer to the captured video that other users behaved on other devices with respect to the basic video.

  Preferably, captured image processing apparatus 150 further includes own user captured data storage means 154. The user's own user shooting data storage unit 154 stores, in the storage device 163, the user's own user shooting data in which the user's own user is recorded as a video while the basic video is displayed on the display 161. The comparison video generation unit 153 generates a comparison video including the user's own user captured video based on the user's own user captured data read from the storage device 163 as a partial video.

  A specific example of the storage device 163 is the second recording unit 32 (FIG. 2). A specific example of one scene of the comparison image including the user's own user-captured image as a partial image is the display screen of the display 21 in FIG. In this way, the user of the captured video processing device 150 can compare the behavior of himself / herself and others with respect to the same basic video.

  The photographed image processing apparatus 150 can further include an upload unit 155. The upload unit 155 uploads the user's own user shooting data to the shooting data management server 164 on the network. The other device user image data acquisition unit 152 acquires other device user image data from the image data management server 164. A specific example of the imaging data management server 164 is the server 14 (FIG. 1).

  The comparison video generation unit 153 may generate the comparison video excluding the partial video of the basic video. A specific example of the scene of the comparison video is that the partial image G3 is deleted from the display screen of the display 21 in FIG. 10 and only the partial images G1 and G2 are left.

  The captured video processing apparatus 150 can further include a partial display period setting unit 156. The partial display period setting unit 156 sets one or more partial display periods in the entire display period based on the occurrence of an event in the entire display period of the basic video. The user's own user shooting data storage unit 154 stores the user's own user shooting data in which the user's own user in each partial display period is recorded in video in association with each partial display period in the storage device 163. As the event, an appropriate event is set as one that defines a group of captured images (e.g., a change of a progress scene and a goal).

  In this way, the user can perform efficient visual recognition by viewing the captured video only during the priority period of the basic video. Note that the photographed video processing apparatus 150 can output not only the contrast video but also audio. The audio may be recorded audio for one or a plurality of partial videos (eg, G1, G2, and / or G3 in FIG. 10) included in the contrast video. Typically, the recorded data of the photographed video and its sound are stored in the same file, and at the time of reproduction, the photographed video and its sound are reproduced at the same reproduction timing.

  In the preferred captured image processing apparatus 150, the other device user captured data acquisition unit 152 further acquires the other device user captured image associated with each partial display period via the communication unit 162. The comparison video generation unit 153 generates a comparison video for each partial display period.

  FIG. 13 is a flowchart of the captured image processing apparatus control method 180. The captured video processing device control method 180 is applied to the captured video processing device 150.

  In S181, the basic image is displayed on the display 161. In step S <b> 182, the user's own user shooting data recording the user's own user during the period in which the basic video is displayed on the display 161 is stored in the storage device 163. In S183, the user's own user photographing data is uploaded directly to the other device via the communication means 162 or to the photographing data management server 164.

  The processing of S181 to S183 corresponds to the functions of the other-device user shooting data acquisition means 152, the comparison video generation means 153, and the display control means 151, respectively, of the shooting video processing device 150 (FIG. 12). The specific modes described above for each means of the captured video processing device 150 can also be applied as specific modes of corresponding steps in the captured video processing device control method 180. In the captured image processing apparatus control method 180, steps corresponding to the functions of the user's own user captured data storage unit 154, the upload unit 155, and the partial display period setting unit 156 can be added. The order or insertion position of the additional steps is arranged so that the steps corresponding to the output side means are ahead of the steps corresponding to the input side means according to the input / output relationship between the means.

  The program to which the present invention is applied causes a computer to function as each unit of the captured video processing device 150. Another program to which the present invention is applied causes a computer to execute each step of the captured image processing apparatus control method 180.

  Needless to say, the present invention is not limited to the above-described embodiments, and various modifications (including addition and deletion) are possible without departing from the scope of the present invention.

  This specification discloses various ranges and levels of the invention. In addition to various devices and methods of various technical scopes and specific levels described in the present specification, the present invention has operations and effects independent of each device and each method within the scope of expansion or generalization. Extracting one or more elements, changing one or more elements within the scope of expansion or generalization, and combining one or more elements between devices and methods Includes replacements.

150: Captured video processing device, 151: Display control means, 152: Other device user captured data acquisition means, 153: Comparison video generation means, 154: Own user captured data storage means, 155: Upload means, 156: Partial display Period setting means, 161: display, 162: network, 163: storage device, 164: shooting data management server, 180: shooting video processing device control method.

Claims (4)

Own user photographing data storage means for storing own user photographing data including information of the basic video in which the own user during a period of displaying the basic video on the display is stored in a storage device;
Upload unit for uploading the own apparatus user captured data via the communication means to directly or photographic data management server to another machine, and the recording the other machine user periods displaying the basic image in another machine display other machine user capturing data including information of the basic video, the basic search and on the basis of the video information, retrieved other machine user captured data directly from another machine or photographed other machines to get through communication means from the data management server User photographing data acquisition means,
The other device user-captured video during the period of displaying the basic video based on the basic video and the searched other-device user captured data, the first frame of the basic video or the frame of a predetermined event as a reference frame, Composite video generation means for generating a composite video in which display timings of the frames of the other-device user video corresponding to a reference frame are aligned, and display control means for displaying the composite video generated by the composite video generation means on the display ,
A photographed image processing apparatus comprising: The composite video generation means including, in the composite video, own user user captured video based on the own user user captured data read from the storage device;
The photographed image processing apparatus according to claim 1, further comprising: Own device user image data storing step for storing own device user image data including information of the basic image in which the own device user is recorded during a period during which the basic image is displayed on the display;
An uploading step of uploading the user's own user photographing data directly to another device via a communication means or to a photographing data management server;
Other device user image data including information on the basic image in which the other device user is recorded during the period in which the basic image is displayed on the display device of the other device is searched based on the information on the basic image, and the other device user who has searched is searched Other device user image data acquisition step for acquiring image data directly from another device or from the image data management server via communication means,
The other device user-captured video during the period of displaying the basic video based on the basic video and the searched other-device user captured data, the first frame of the basic video or the frame of a predetermined event as a reference frame, A composite video generation step of generating a composite video in which display timings of the frames of the other-user user captured video corresponding to the reference frame are aligned; and
A display control step for displaying the composite video generated by the composite video generation step on the display;
A method for controlling a photographed video processing apparatus, comprising: The program which makes a computer function as each means of the picked-up image processing apparatus of any one of Claims 1-2.

Images