JP2005286378A - Moving picture reproduction system and moving picture reproduction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving picture reproduction system for reproducing and displaying a desired photographing scene of a moving picture comprising a plurality of photographing scenes at a prescribed reproduction speed as a moving picture. <P>SOLUTION: The moving picture reproduction system includes: a photographing information acquisition means provided in a photographing means for acquiring a moving picture to acquire the photographing information including at least the photographing position information and the photographing direction information of the photographing means when each frame image of the moving picture is acquired; a photographing scene specifying means for specifying a desired photographing scene by using the photographing information; and a moving picture reproduction display means for displaying the specified and desired photographing scene of the moving picture at a prescribed speed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving image playback system that plays back and displays a desired shooting scene of a moving image composed of a plurality of shooting scenes as a moving image at a predetermined playback speed.

  Currently, moving image playback devices such as video players and DVD players that play back and display moving images stored in a predetermined storage medium are widely used in general households, and many people watch various moving images. Yes. Also, not only playback by dedicated playback devices such as video players and DVD players, but in recent years, with the advancement of digital image technology, moving images have been connected to the Internet due to the increase in capacity of Internet lines and enhancement of digital broadcasting. It is frequently stored in a personal computer HDD or the like and reproduced.

When a viewer wants to view a moving image with an emphasis only on a specific scene in which a specific subject is captured in the entire moving image, the viewer desires a specific scene desired by the viewer Conventionally, a method for shortening the reproduction time of the entire moving image by reproducing fast-forwarding scenes other than (highly interested scenes) and reproducing only this specific scene at a normal speed has been generally performed.
In such a playback method, the viewer has to check the content of the scene while watching the moving image being fast-forwarded, and give a normal playback instruction when the desired scene content is reached. In such a method, an instruction for normal playback is issued at the timing when the fast-forward image becomes the desired scene content, so the beginning of the desired scene is not normally played back, and the entire desired scene is viewed in normal playback. However, it is necessary to rewind the moving image once for a predetermined time, which is troublesome. In addition, viewers need to check the contents of the scene by continuously concentrating on fast-changing screens and confirming the contents of the scene, which is subject to time constraints for observing unnecessary scenes and increases fatigue. was there.

A video summarizing method for confirming the contents of a moving image in a short time when reproducing and displaying the moving image is described in Patent Document 1 below. In the first video summarization method described in Patent Document 1, by detecting an image feature amount of a moving image and detecting this change, a time interval of an image including an object moving at high speed is detected, and this interval is detected at high speed. It is regarded as an operation interval, and a time interval in which similar images continue for a certain time or more is detected, and this time interval is regarded as a long-time similarity interval. The high speed operation section is played back at a lower speed than the time sections other than the high speed operation section, and the long time similar section is played back at a higher speed than the time sections other than the long time similar section. As a result, there is no subject moving at high speed in the scene of the moving image, and the “subjectively fast reproduction speed” felt by the viewer of the moving image is eliminated. In addition, since the video changes one after another in a short time, the “subjective slow playback speed” felt by the moving image viewer is also eliminated, which makes it easier for the viewer to check the content of the moving image. It is mitigating. In Patent Document 1, since the entire video is reproduced by such a method so that the reproduction speed subjectively perceived by the viewer falls within the allowable range, the subject in the moving image is shortened while reducing the reproduction time of the entire moving image. It is possible to grasp the contents centering on the operation of As a result, the viewer can observe the entire moving image while confirming the contents in a short time as a whole.
JP-A-8-251540

However, in such a video summarization method described in Patent Document 1, a high-speed operation section and a long-time similar section are detected using image feature amounts, and a long-time similar section is compared with a time section other than a long-time similar section. It is only playing fast. In such a method, when the screen change is small, the moving image is reproduced and displayed at a high reproduction speed even in a scene in which the viewer is highly interested. For this reason, for example, even in a scene where a subject of interest is shown, there is a problem that if the amount of image change of the image of the subject is small, the scene cannot be slowly viewed.
The present invention has been made in order to solve the above-described problems. From a predetermined moving image, a scene in which a viewer is interested, particularly a scene in which a specific subject is captured in a specific state. The object is to provide a moving image playback system that plays back at a slow playback speed with emphasis.

In order to solve the above problems, the present invention is a moving image reproduction system for reproducing and displaying a desired shooting scene of a moving image composed of a plurality of shooting scenes as a moving image at a predetermined reproduction speed,
An imaging unit that acquires the moving image, and an imaging unit that is provided in the imaging unit and that acquires imaging information including at least imaging position information and imaging direction information of the imaging unit when each frame image of the moving image is acquired. An information acquisition unit; a shooting scene specifying unit that specifies the desired shooting scene using the shooting information; and a moving image reproduction display unit that displays a moving image of the specified desired shooting scene at a predetermined reproduction speed. The moving image reproduction display unit reproduces and displays a moving image of the desired shooting scene at a reproduction speed different from that before and after the desired shooting scene in the reproduction of the moving image. provide.

  As the predetermined reproduction speed, for example, a normal reproduction speed is set. By setting the specified playback speed as the normal playback speed and performing fast forward playback of the entire moving image, the playback time of the entire moving image is shortened, and the desired shooting scene portion of the moving image is displayed at the normal playback speed. Can do.

  The desired shooting scene is an appearance scene of a specific subject, and further includes subject information acquisition means for acquiring at least subject position information of the specific subject, wherein the shooting scene specifying means includes the shooting information, the subject information, It is preferable that the appearance scene of the specific subject is specified using.

  The desired shooting scene of the present invention is not limited to a scene in which a specific subject appears, and may be a scene of a specific event that is a shooting scene shot at a specific event. For example, it may be a shooting scene specified by an event at which the shooting scene was shot, such as a scene shot at a fireworks display or a scene shot at an amusement park.

  The specific subject may be a person, a building such as a building, or a natural structure such as a mountain or a river. When the specific subject is a person, examples of the subject information acquisition unit include a portable device that is carried or attached to the specific subject and acquires the position information detected by the position detection unit that detects the position of the specific subject. . Examples of the position detection means include a portable device with a function for detecting the position of the specific subject, a GPS device carried by the specific subject, a position sensor attached to the specific subject, and a position using an IC tag. A detection system may be used. Further, when the specific subject is a building or a natural structure, a database for detecting at least position information of the specific subject from map data corresponding to a real space photographed by the photographing means and a search means for this database are exemplified.

  The shooting information acquisition unit further acquires shooting time information of the shooting unit when each frame image of the moving image is acquired, and the subject information acquisition unit further detects position information of the specific subject. Detection time information is obtained, and the shooting scene specifying means specifies the scene of the specific subject using the subject information having the detection time information that matches or substantially matches the shooting time information. preferable.

  The desired shooting scene is a scene in which the specific subject appears in a specific posture, and the subject information acquired by the subject information further includes information on the posture of the specific subject, and the shooting Preferably, the scene specifying means specifies the appearance scene of the specific subject using the subject information, and further specifies a scene with a specific posture among the appearance scenes of the specific subject.

  In this case, as the subject information acquisition unit that detects information related to the posture of the specific subject, the above-described portable device including the posture related information acquisition unit that further detects information related to the posture of the specific subject is exemplified. The posture-related information acquisition means is fixedly attached to the specific subject and detects an orientation sensor that detects information including the frontal posture of the specific subject, the moment that the specific subject moves, the moment that the specific subject moves, or the moving at a predetermined speed An accelerometer attached to a specific subject or a mobile phone with an accelerometer that is carried by a specific subject that detects that the user is moving, stationary, or moving at a specified speed Examples thereof include a pressure sensor attached to a shoe sole of a specific subject, and a device that detects a moment of movement or landing for obtaining information on a movement start posture including a moment of movement of the specific subject.

  Furthermore, it comprises storage means for storing identification information for identifying a desired shooting scene in a moving image as ancillary information in association with the frame image together with the moving image, and the moving image reproduction display means stores the moving image stored in the storage means. It is preferable that a moving image is displayed at a reproduction speed corresponding to the attached information.

  The attached information preferably includes identification information for identifying a specific subject appearing in the shooting scene, preferably includes information related to the appearance area of the specific subject in the shooting scene, and further includes the specific subject of the shooting scene. It is preferable to include information on the posture.

  Furthermore, a playback condition specifying means for specifying a playback condition is provided, wherein the specifying means extracts the attached information according to the playback condition specified by the playback condition specifying means, and the moving image playback display means is extracted. Preferably, the shooting scene is displayed as a moving image at a playback speed determined according to the attached information.

  In the present invention, there is further provided a moving image reproduction method for reproducing and displaying a desired shooting scene of a moving image composed of a plurality of shooting scenes as a moving image at a predetermined reproduction speed, and a shooting step for acquiring the moving image; A shooting information acquisition step for acquiring shooting information including at least shooting position information and shooting direction information when each frame image of the moving image is acquired, and a shooting scene for specifying the desired shooting scene using the shooting information A moving image reproduction display step of displaying a moving image of the identified desired shooting scene at the predetermined reproduction speed, wherein the predetermined reproduction speed is the desired shooting in the reproduction of the moving image. There is also provided a moving image reproducing method characterized by a speed different from that of the front and rear portions of the scene.

  Further, in the present invention, there is provided a moving image reproduction method for reproducing and displaying a desired shooting scene of a moving image composed of a plurality of shooting scenes as a moving image at a predetermined reproduction speed, and a shooting step for acquiring the moving image; A shooting information acquisition step for acquiring shooting information including at least shooting position information and shooting direction information when each frame image of the moving image is acquired, and a shooting scene for specifying the desired shooting scene using the shooting information A specifying step; a storing step of storing identification information for identifying the specified desired shooting scene in association with a frame image as attached information; and storing the moving image together with a moving image; a playback condition specifying step for specifying playback conditions; and the playback conditions An extraction step for extracting the attached information according to the reproduction condition designated by the designation means, and a reproduction speed determined according to the extracted attached information. Moving image reproduction method characterized by having a moving image playback display step of displaying a moving image photographed scene also provides together.

  According to the moving image playback system of the present invention, a desired shooting scene of a moving image composed of a plurality of shooting scenes can be played back and displayed as a moving image at a predetermined playback speed. As a result, the desired shooting scene in the moving image can be reproduced relatively slowly, and the other scenes can be reproduced at a relatively high reproduction speed. Thereby, it is possible to view a shooting scene in which the viewer is interested at a slow reproduction speed while shortening the viewing time of the entire moving image. In addition, it is possible to concentrate and view only the desired scene with less fatigue without the viewer concentrating and checking the playback screen during fast-forward playback.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a moving image reproduction system and a moving image reproduction method of the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

  FIG. 1 is a configuration diagram showing an outline of a moving image reproduction system 10 which is an embodiment of the moving image reproduction system of the present invention. The moving image reproduction system 10 of the first embodiment (hereinafter simply referred to as the system 10) is an arbitrary one that is present on the spot by a cameraman 12 at an arbitrary shooting place such as a sightseeing spot, a theme park, an event venue, or a park. A moving image is acquired by taking a picture of a person as a subject, or taking a picture of a contestant of a sports competition performed at a stadium. The moving image reproduction system 10 according to the first embodiment is a portion of a scene in a specific state of a specific subject (a scene in which a specific subject appears in a specific state) from the entire moving image captured in the above shooting. Of the whole moving image other than the scene with the specific posture of the specific subject is fast-forwarded and reproduced, and the scene portion with the specific posture of the specific subject (hereinafter referred to as a specific subject shot). It is a system that enables only the normal speed to be reproduced and displayed.

  The system 10 includes a photographing unit 14 corresponding to the photographing unit of the present invention, which captures a moving image by photographing a subject 16 (a person subject 16a and a structural subject 16b) that is a specific subject carried by a cameraman 12, and a photographing unit. 14, the apparatus sensor unit 15 corresponding to the photographing information acquisition means of the present invention provided in 14, the moving image obtained by photographing by the photographing means 14 and various information described later are stored, and the moving image and various information are processed. The moving image photographed by the photographing means 14 is reproduced and displayed while adjusting the reproduction speed based on the data processing apparatus 20 that performs the above and the later-described additional information attached to the moving image according to the reproduction condition of the predetermined moving image. And a playback device 50.

  The data processing device 20 acquires the moving image obtained by photographing by the photographing unit 14, the photographing information obtained by the photographing information obtaining unit 15, and the subject information obtained by the portable device 18, and the obtained moving image. In addition, various types of information and map data in the imaging space range described later are processed. Then, the subject 16 in each scene represented by each frame image of the moving image photographed by the photographing unit 14 by data processing is recognized, and in-scene subject recognition information (information on which subject 16 is captured for each scene). In addition to detection, the subject area information in the scene (information on the position and size of the area of the subject 16 in the scene) in each scene represented by each frame image of the moving image, and the subject posture information related in the scene (the subject 16 in the scene) Information on the posture of the image) and the information is stored in a predetermined storage medium together with the moving image as attached information of each frame image of the moving image. Hereinafter, the in-scene subject recognition information, the in-scene subject area information, and the in-scene subject posture information are combined and used as in-scene subject information.

  The subjects photographed in the system 10 include a human subject 16a and a structural subject 16b. The human subject 16a is a subject that freely moves around in a predetermined photographing space (the above-mentioned theme park, stadium, etc.) The structure subject 16b is a subject that is always located at the same place in the photographing space. The human subject 16a has a GPS function (subject position information detection function 18a) that constantly detects subject position information using a GPS system, a subject posture information detection function 18b that detects information about the posture of the subject (subject posture information), The portable device 18 is equipped with a timekeeping function (subject time information detection function 18c) for detecting subject time information corresponding to each piece of information detected by these detection functions. The subject 16a can detect the position information (subject position information), posture information (subject posture information), and subject time information of the human subject 16a at the time of photographing by the photographing means 14 by such a portable device. The subject position information, subject posture information, and subject time information acquired by the portable device 18 are stored in a subject information DB 30B of the data processing device 20 described later.

  Here, the posture-related information of the human subject 16a is information including the body orientation of the human subject 16a (the direction in which the front of the body faces), the moment when the subject 14 moves, the moment when the subject 14 moves, or the movement including moving at a predetermined speed. Is information including at least one of information on the posture and information on a movement start posture including a moment when the human subject 16a starts to move.

  As an example of the posture-related information detector 18b, when detecting information on the body direction of the human subject 16a (the direction in which the front of the body is facing) as posture-related information of the human subject 16a, An orientation sensor installed on an accessory whose position relative to a person subject 16a such as clothes, glasses, and clothes 16a is fixed. In addition, when detecting information on the posture of a motion including the moment when the person subject 16a moves, the moment when the person subject 16a moves, or moving at a predetermined speed, as the subject posture information, as an example of the posture related information acquisition unit 18b, for example, the subject 16a There is an acceleration sensor that measures the acceleration. In this case, by setting the acceleration condition of a specific subject in the scene as an image extraction condition, for example, the moment the subject 16a starts moving, the moment it stops, or moving at a speed of N / m per second, etc. It is possible to automatically extract image data of a scene having a posture at a specific operation timing.

  In addition, when detecting information related to the movement start posture including the moment when the subject 16a starts to move as the posture related information of the subject, examples of the posture related information acquisition unit 18b are, for example, installed on the shoe sole of the subject, and the human body from the ground There is a pressure sensor for detecting the pressure received. In this case, by setting the condition of the pressure that the subject's foot receives from the ground (subject pressure information) as the condition for image extraction, for example, the moment when the foot is stepped on the ground with the effort to move the specific subject, It is possible to automatically extract image data of a scene having a posture at a specific operation timing of a specific subject, such as a moment when a jumping subject has landed.

  Further, for example, when a subject 16a extracts a posture image in an operation performed by applying pressure to an object with a predetermined tool, such as baseball or tennis, for example, a baseball bat, a tennis racket, or a ball is used. It is possible to extract the image data of the scene at the moment when the bat, racket or the like collides with the ball by installing a pressure sensor on the surface. The posture related information of the human subject in the image photographing method of the present invention is not particularly limited as long as it is information related to the posture of a specific human subject. Hereinafter, an example in which information on the body direction of the subject is detected as the posture-related information of the subject will be described.

  Further, the position information and posture information of the structure subject 16b in the photographing space range in the real space photographed by the system 10 are previously stored as map data in the moving image information DB 30B in the HDD 30 of the data processing device 20 described later. It is remembered. Here, the information on the posture of the structure object refers to information on the direction of the structure. The map data stored in the map information DB 30B is three-dimensional map data including a shooting space range, and reproduces the shape of structures such as buildings and natural objects (such as mountains and forests) within the shooting space range. Three-dimensional information. This map information is not limited to three-dimensional information, and may be two-dimensional map information that reproduces only the position information of buildings and natural objects within the imaging space range. However, when a moving image is reproduced while adjusting the reproduction speed under more detailed conditions (that is, when the reproduction speed is adjusted based on more detailed subject information), the map information is preferably three-dimensional information. . Hereinafter, the subject position information, the subject posture information, and the subject time information of the person subject 16a and the structure subject 16b are collectively referred to as subject information.

The photographing means 14 for acquiring a moving image reproduced and displayed by the system 10 is possessed by the cameraman 12, and the cameraman 12 photographs a subject 16 (a person subject 16a and a structure subject 16b) to obtain a moving image. It is a camera. The photographing means 14 uses a GPS system to detect current position information (photographing position information) at the time of photographing, and a photographing direction (that is, a photographing lens of the photographing means 14 at the time of photographing). The photographing direction corresponding to the photographing information acquisition means of the present invention is provided with a photographing direction detecting function 15b for detecting the direction << direction, and further elevation angle >> and a photographing time information detecting function 15c for detecting a photographing time (photographing time information). The apparatus sensor 15 is mounted, and the shooting position information, the shooting direction information, and the shooting date / time information are collectively referred to as shooting information.
An example of the shooting direction information detection function 15b is a function in which an orientation sensor is installed at a predetermined position of the shooting unit 14. As this azimuth sensor, for example, a known sensor such as an azimuth sensor that detects azimuth using geomagnetism or an azimuth sensor that detects each inclination by detecting an angular velocity around a predetermined axis using a gyro can be used. It is.

  While the cameraman 12 is shooting a moving image, constantly shot image data and shooting information are transmitted from the shooting means 14 and the shooting device sensor 15 to the data processing device 20 via the communication control means 26. The image data sent to the data processing device 20 and taken by the photographing means 14 and the shooting information corresponding to the image data are associated with each other and stored in the shooting information DB 30A. The transmission of the image data and the photographing information from the photographing unit 14 to the data processing device 20 may be wireless communication or wired communication using an electric cable or an optical cable. Further, the image data and the shooting information are temporarily recorded on the recording medium 32 (for example, smart media (R), compact flash (R), memory stick (R), etc.) by the shooting means 14, via the media driver 28, The recording medium 32 may be recorded on a predetermined part of the data processing device 20.

  The data processing device 20 is composed of a computer such as a workstation, and a CPU 20A, ROM 20B, RAM 20C, and input / output port 20D are connected to each other via a bus 20E, and a communication control device (for example, a modem or a router) 26 is connected to the input / output port 20D. A driver 28 and a hard disk device (HDD) 30 are connected to each other. An image extraction program for executing the above-described data processing by the CPU 20A is preinstalled in the HDD 30 of the data processing device 20, and the data processing device 20 is configured by the CPU 20A executing the in-scene subject information detection program. The computer executes the in-scene subject information detection step described later.

  The hard disk device (HDD) 30 stores image data of an image captured by the imaging unit 14 and imaging information corresponding to image data acquired by the imaging unit 14 in association with each other. A subject information database 30B for storing the information database 30A (hereinafter referred to as the photographing information DB 30A), the subject information of the human subject 16a, and the map data corresponding to the structure subject 16b acquired by the portable device 18. (Hereinafter referred to as subject information DB 30B) and in-scene subject information corresponding to each image data obtained by data processing by the data processing device 20 are stored in association with each image data as attached information of each image. Moving image information database 30C (hereinafter referred to as moving image information DB 30C) It is equipped with a door. Each of these databases may be an individual information storage medium, or may correspond to each of a plurality of storage areas of one storage medium. Note that such map data is not limited to being stored in the HDD 30 of the data processing device 20, but can be retrieved from the data processing device 20 via a network such as the Internet. It may be acquired from a predetermined map information database that is connected (directly connected or indirectly connected via the Internet or the like).

  In the data processing device 20, the in-scene subject information is detected as follows. FIGS. 2A and 2B show the positions of the photographing means 14 and the subject 16 (personal subject 16a and structure subject 16b) and the moving images in the photographing of moving images (acquisition of moving images) by the photographing means 14. FIG. FIG. 2A is a diagram for explaining a scene of a frame image. FIG. 2A shows a positional relationship between the photographing unit 14 and the subject 16 when a moving image is photographed by the photographing unit 14, and FIG. FIG. 6 is a schematic diagram showing a scene of a moving image obtained by photographing by the photographing means 14 in the example shown in FIG. FIG. 2B is a schematic diagram showing one scene of a plurality of frame images of a moving image obtained by shooting by the shooting unit 14, and the frames shot at the timing corresponding to the diagram shown in FIG. An image scene is shown.

  At the time of shooting by the shooting unit 14, the position information and shooting direction information of the shooting unit 14 are continuously acquired by the shooting information acquisition unit 15 as described above, and associated with each frame image of the moving image, thereby moving the moving image. Along with the image, it is stored in the shooting information DB 30A. In the data processing device 20, the real space (corresponding to the scene of each frame image of the moving image obtained by shooting by the shooting unit 14 from the shooting information associated with each frame image of the moving image ( Determine the shooting area in the shooting space.

  In the subject information DB 30B, three-dimensional map data corresponding to this photographing space is stored in advance, and the data processing device 20 is present in the photographing region based on this three-dimensional map data. A ferris wheel illustrated in b) or a building such as a building, and a natural structure such as a mountain range or forest illustrated in FIGS. 2A and 2B are recognized. Such 3D map data also includes information on the height and shape of mountains, forests, and buildings. By comparing such information with area information, Information on what area the subject is in the scene is detected.

  The subject information DB 30B includes subject information corresponding to the shooting information acquired by the shooting information acquisition unit 15 of the shooting unit 14 (that is, corresponding to each frame image of a moving image shot by the shooting unit 14). It is remembered. The data processing device 20 reads out subject information (subject region information, subject position information, and subject orientation information) corresponding to each frame image, and collates it with the above-described photographing region information and photographing direction information, thereby freeing the photographing space. The information on which region (in what direction) the human subject 16a moving around is captured in what region in a scene of a plurality of frame images is detected. In the data processing device 20, as shown in FIG. 2 (b), information indicating what subject is captured in what area in what area in the moving image scene (subject subject information). It is possible to detect.

In this way, the data processing device 20 acquires in-scene subject information for each scene of each frame image of the moving image, associates the in-scene subject information as attached information for each frame image, and moves the moving image information to the moving image information DB 30C. Store together with the image (ie, with each frame image).
Such in-scene attached information is not limited to attaching in-scene subject information to each frame image. In the moving image reproduction system of the present invention, scene information in each moving image may be collectively attached to a header portion of one moving image. For example, information indicating the appearance timing when a moving image for each subject is reproduced and displayed (that is, information indicating the order in the moving image of the frame images of the scene in which each subject appears) and the scene in the scene in which each subject appears Information indicating the inner subject information may be attached to the header portion of the moving image.

  The attached information attached to the moving image is not limited to the in-scene subject information. For example, event information for each scene of each frame image (information on the contents of the event photographed in the scene of each frame image) Etc. may be attached as attached information. For example, information indicating that it is an image of a scene taken at a fireworks display or a festival held in a predetermined area may be attached to the attached information. Such event information includes, for example, a predetermined event information database in which various event content information (event name and other information indicating event content) is stored in association with the date and time of the event, shooting information ( (Photographing position information, photographing direction information, and photographing date / time information) may be retrieved and acquired. Such an event information database may be provided in, for example, the HDD 30 of the data processing device 20, and is connected to the searchable data processing device 20 via a network such as the Internet (directly connected, Alternatively, a predetermined database (indirectly connected via the Internet or the like) may be used.

The playback device 50 is a device that reproduces and displays a specific subject shot portion desired to be viewed by a viewer at a normal playback speed and plays back and displays the other portions at a fast-forward playback speed during fast-forward playback of the moving image. Yes, a data reading unit 51 that reads out moving images and attached information stored in the moving image information DB 30C of the data processing device 20, and conditions of a specific subject that the viewer wants to view with emphasis (which subject is in which posture) A reproduction condition designating unit 52 that designates whether or not the scene being viewed is emphasized), and according to the conditions designated by the reproduction condition designating unit 52, the attached information of the moving image is detected and a specific subject is detected. Attached information representing the appearance in a specific posture is extracted, and a playback speed setting means 54 for setting a playback display speed based on the attached information, and a setting in the playback speed setting means 54 An image data output unit 56 for outputting a moving image in accordance with the reproduction speed which is configured and a display 58 for displaying a moving image output from the image data output unit 56.
The image reproducing device 50 may be a computer having a CPU and a memory, and each part may be operated by a program, or may be a dedicated device in which each part is configured with a dedicated device and a circuit.

  Note that the reading of the moving image and the attached information by the data reading unit 51 is performed by the data reading unit 51 of the reproducing device 50 directly accessing the moving image information DB 30C via the input / output port 20D of the data processing device 20. Alternatively, the moving image and the attached information of the moving image may be recorded by the data processing device 20 on the recording medium 60 (for example, smart media (R), compact flash (R), memory stick (R), CD-R, DVD -R or the like) and the moving image recorded on the recording medium 60 and the attached information of the moving image may be read out.

  The reproduction condition instructing means 52 sets the reproduction speed condition of the moving image by the reproduction device 50 by designating the classification of the important scene portion and the reproduction speed of the important scene portion at the time of reproduction by the reproduction device 50. . Examples of classification of scene parts that are important for moving images include, for example, the subject in the scene, the location in the scene (or the location where the image was taken), and the general noun of the event in the scene And proper nouns. In specifying the playback instruction condition, not only the scene classification to be emphasized is specified, but also a priority is set for each specified scene classification, and a condition for changing the playback speed according to the priority is set. You can set it. In the example described later, a condition for reproducing and displaying a scene portion of a specific subject in a specific posture (referred to as a specific subject shot) at a normal speed is specified.

  The reproduction speed setting unit 54 searches the attached information (the in-scene subject information) read by the data reading unit 51 based on the condition (the above-described example) specified by the reproduction condition instruction unit 52. The attached information corresponding to the designated condition is detected, and the output speed of the moving image to be sequentially output from the image data output means 56 to the display 58 is set according to the attached information. In the example described later, the moving image playback speed condition is set so that the specific subject shot is played back at the normal playback speed, and the other portions are played back at the fast forward speed.

The image data output means 56 sequentially outputs each frame image to the display 58 at a timing according to the reproduction speed set by the reproduction speed setting means 54, and sequentially displays each frame image of the moving image displayed on the display 58. Switch to output. The image data output means 56 makes the interval of the output timing of each frame image shorter than the timing of normal playback for the portion of the moving image for fast-forward playback or the portion of the portion of the moving image for fast-forward playback. By thinning out and outputting the frame images, the moving image of the fast-forward playback portion of the moving image is displayed on the display 58 in a shorter time than the playback at the normal playback speed. In the moving image playback system of the present invention, the playback device 50 may be integrated with the data processing device 20.
With such a configuration, the moving image reproduction system 10 of the present embodiment can normally reproduce only a specific subject shot and can perform fast-forward reproduction of the other portions in the moving image.

  Next, an embodiment of the moving image reproduction method of the present invention performed by the moving image reproduction system 10 of the first embodiment will be described. FIG. 3 is a flowchart of one embodiment of the moving image reproduction method of the present invention.

  First, by a photographer 12, an arbitrary person (person subject 16a), a mountain, a forest, a building, or the like (stationary structure) at an arbitrary shooting location such as a sightseeing spot, a theme park, an event venue, or a park. Shooting of a moving image using the object subject 16b) as a subject is started (step S100).

When shooting of a moving image is started by the cameraman 12, subject information is acquired (step S102). The subject information of the person subject 16a in the subject information is acquired by the portable device 18 carried by the person subject 16a as described above.
The subject information acquired by the portable device 18 is sent to the data processing device 20 as described above and stored in the subject information DB 30B (step S104). When there are a plurality of subjects 16 (especially when a plurality of subjects 16 are randomly photographed by the cameraman 12), when transmitting subject information, subject position information indicating the current position of each subject 16 and the subject's position In addition to the subject posture information indicating the posture (body direction) and the time information corresponding to the information, it is preferable that information for identifying the customer (subject ID information) is simultaneously notified. When the current position information and posture information of the customer are transmitted from the mobile device 18 in real time constantly while the image is being taken by the cameraman 12, if the data processing device 20 has a time measuring function, The time information measured by the timekeeping function may be stored in the subject information DB 30B in association with the subject information according to the timing at which the subject information is received from the portable device 18. Acquisition of subject time information according to the present invention may be performed in this manner.

  Note that the subject information of the stationary structure subject 16b in the subject information is acquired by searching the 3D map data corresponding to the imaging range as described above (step S102), and stored in the subject information DB 30B (step S102). And corresponding to step S104). The acquisition of the 3D map data and the storage in the subject information DB 30B may be performed prior to the acquisition of the in-scene subject information (step S110 described later) by the data processing device 20, and the 3D map to the subject information DB 30B. The timing of data storage is not particularly limited.

The subject information of the person subject 16a is obtained by always performing the above process while the photographer is shooting, so that the subject information DB 30B has subject information (at a certain date and time) in units of individual person subjects 16a. A plurality of sets of subject time information, subject position information, and subject posture information representing the subject position information and posture information are stored. By referring to the subject information group stored in the subject information DB, individual information is stored. It is possible to determine the movement history of the person's subject 16a.
Note that this subject information is not limited to being notified from the portable device 18 to the data processing device 20 in real time. For example, as described above, during photographing, the portable device 18 records the subject information on a predetermined recording medium, and the photographing is finished. After that, the data processing device 20 may be notified.

On the other hand, when the photographer 12 starts photographing each subject (person subject 16a and stationary structure subject 16b) by the photographing unit 14, photographing information is acquired together with the image data as described above (step S106). As shooting information, shooting position information, shooting direction information, and shooting time information are acquired in time series continuously during shooting of a moving image at predetermined time intervals.
The shooting information acquired by the shooting unit 14 is stored in the shooting information DB 30A in association with the corresponding frame image. That is, information on when each frame image was shot (corresponding to the shooting time information), from which position (corresponding to the shooting position information), and in which direction the shooting lens of the shooting unit 14 was pointed (corresponding to the shooting direction information). Are associated with each frame image and stored in the photographing information DB 30A.

In addition, when there are a plurality of photographing means 14 (cameramen 12), it is preferable that information (camera ID information) for specifying the photographing means 14 is transmitted at the same time when photographing image data and photographing information are transmitted.
Note that the data processing device 20 has a timekeeping function when the current shooting position information and shooting direction information of the shooting unit 14 are always transmitted in real time from the shooting unit 14 while shooting by the cameraman 12 is being performed. If this is the case, the time information measured by the timekeeping function is stored in the shooting information DB 30A in association with each frame image and shooting information in accordance with the timing at which each frame image and shooting information is received from the shooting means 14. Good. Acquisition of each frame image and shooting information may be performed in this way.

By constantly performing the above processing while the photographer is photographing, the photographing information DB 30A captures each frame image of the moving image photographed by the individual photographing means 14 in units of the individual photographing means 14. A plurality of pieces of information (a set of shooting time information, shooting position information, and shooting direction information) are stored, and by referring to the shooting information group stored in the shooting information DB, each frame image is displayed at which position. Therefore, it can be seen in which direction the image is taken with the taking lens directed.
Note that this shooting information is not limited to being notified to the data processing device 20 in real time from the shooting unit 14, and for example, as described above, during shooting, the shooting unit 14 records the information on a predetermined recording medium, and the shooting ends. After that, the data processing device 20 may be notified.

  By a program installed in advance in the HDD 30 of the data processing device 20, a computer constituting the data processing device 20 obtains the shooting information from the shooting information DB 30A, and the three-dimensional data related to the person subject information and the structure subject 16b from the person subject information DB 30B. By reading out the map data information and collating them, the in-scene subject information for each scene of the moving image is detected as described above (step S110). The detected in-scene subject information is associated with the corresponding frame image and stored in the moving image information DB 30C together with the corresponding frame image as attached information of the frame image (step S112).

  When the detection and storage of the in-scene subject information in the data processing device 20 is completed, the moving image and the auxiliary information stored in the moving image information DB 30C are read by the data reading means 51 of the playback device 50 (step S113). The read moving image is sent to the image data output unit 56, and the attached information is sent to the reproduction speed setting means 54.

  Next, an important playback condition, which is a scene condition to be emphasized during moving image playback, is designated by the playback condition instruction means 52 of the playback device 50 (step S114). As described above, the reproduction condition includes the subject in the scene (which person subject is photographed or which stationary structure subject is photographed) and the posture of the person subject (posture information in the scene). Indicate conditions. Also, the playback importance may be specified for each specified condition. For example, when the subject 16 is a person and there are a plurality of subjects, the extraction condition is who (which subject 16) is photographed in what kind of body (an image photographed in the front direction). Alternatively, it may be a playback condition that places emphasis on whether the image is taken horizontally or whether it is taken backwards. Further, the type of event information described above may be designated as a condition to be emphasized during reproduction. Such designation of the importance of reproduction may be performed in advance before the start of shooting by the cameraman 12, for example, and if it is performed prior to the scene extraction described later, the timing of inputting the scene extraction condition is There is no particular limitation.

  When the reproduction condition is designated, the reproduction speed setting means 54 sets the reproduction speed of the moving image by setting the output speed of the moving image that is sequentially output from the image data output means 56 to the display 58 (step S115). In the present embodiment, as described above, the moving image reproduction speed condition is set so that the specific subject shot is reproduced at the normal reproduction speed and the other portions are reproduced at the fast-forward speed.

  When the playback speed condition is set, the image data output means 56 of the playback device 50 sequentially outputs each frame image to the display 58 at a timing corresponding to the playback speed set by the playback speed setting means 54. The frame images are sequentially switched and output (step S116). As described above, the image data output means 56 makes the interval for outputting each frame image shorter than the normal reproduction for the portion for fast-forward reproduction in the moving image or the portion for fast-forward reproduction in the moving image. By thinning out and outputting the frame images, the moving image is displayed on the display 58 in a shorter time than the normal reproduction for the portion of the moving image to be fast-forward reproduced. The moving image reproduction method of the present invention is performed in this way.

  The moving image reproduced in this way has in-scene subject information attached to each frame image. By using this attached information, the moving image can be reproduced and displayed in various modes. For example, each frame image of a moving image can be image-processed based on attached information, the image of each frame image can be divided into a plurality of regions, and can be reproduced and displayed at different speeds based on the attached information.

  For example, the above-described in-scene subject area information for each frame image of the moving image may be used, the scene of each frame image may be divided for each subject, and the playback speed may be individually controlled for each subject. . An example of such a playback method (hereinafter referred to as special mode playback) will be described below. In the following example, the image data output unit of the moving image reproduction system 10 includes predetermined image editing means (which is a computer), and each frame image of the moving image is subjected to image processing by a predetermined program of the image editing means. Is done.

  First, in the moving image, the viewer designates a frame for starting the special mode reproduction. The designation of the frame is performed by the viewer inputting an instruction to the image editing unit using an input unit (not shown) while looking at the display 58. The image editing means extracts a subject area (corresponding to the above-described in-scene subject area) in this frame. At this time, the subject region is set to a wider range than the subject in the scene, and this wide range region is extracted from the frame image. This subject may be a subject previously designated by the viewer using the reproduction condition designating means, or may be designated by the viewer inputting an instruction to the image editing means while viewing this frame. The above-described image editing means (that is, the image processing apparatus 50) is provided with instruction input means such as a keyboard and a mouse. The above-described instruction input is performed by a so-called GUI (Graphical User Interface) in which the viewer looks at the keyboard and mouse while looking at the display 58.

After that, the N-second frame (a group of frame images corresponding to a N-second scene; N is arbitrary) of the original moving image from which the subject area is cut out is once decoded, and each object area within the N-second frame is used for each purpose. It is applied to each frame image so as to achieve the reproduction speed as follows, and temporarily recorded in a buffer (a predetermined storage area in the image editing means).
That is, the image of each frame image is edited for each subject area within the frame for N seconds, and the image of each frame image is edited so that each subject area is displayed at an arbitrary playback speed when the frame is reproduced for N seconds. Then, the edited frame images are temporarily stored in a predetermined storage area in the image editing means.

By outputting the moving image stored in the predetermined storage area to the display 58 in this way, the scene of each frame image is divided for each subject, and the playback speed is individually controlled for each subject for display. Can do.
FIG. 4 is a diagram for explaining the subject area and the playback speed for each subject in such a special playback mode. In such special mode playback, for example, in the above-mentioned N-second frame in the moving image, the moving image can be played so that the human region is a moving image at a normal playback speed and the background region is a still image ( Pattern A) in FIG. In addition, it is possible to reproduce a moving image so that a person region is a still image and a background region is a moving image at a normal reproduction speed (pattern B in FIG. 4). Further, the present invention is not limited to moving images and still images at normal playback speed, and various playback speeds can be set for each subject in the scene. A moving image obtained by editing each frame image in this manner (moving image reproduced in the special mode) can be obtained by repeatedly reproducing and displaying the above-described N-second frame moving image, for example, It can be suitably used for a screen saver.

  In addition, the attached information in the moving image reproduction method of the present invention is not limited to the above-described example. For example, when reproducing a moving image such as a movie, subtitles representing words corresponding to the respective subjects are scenes related to the respective subjects. It may be used as internal subject information. That is, the scene portion where the subtitles are displayed during the playback display of the moving image may be played back at the normal playback speed, and the exception portion may be played back at the fast forward speed. By adopting such an aspect, it is possible to shorten the reproduction time of the entire moving image while the viewer slowly confirms the content of the word of the subject in a moving image such as a movie.

  Also, for example, a small camera is installed on a display on which a moving image is displayed, information on the movement of the viewer's line of sight is acquired, and the playback speed of the moving image is controlled according to the movement of the line of sight. Also good. When subtitles, which are information related to the subject, are displayed on the screen, the viewer's line of sight generally moves faster in order to check the moving image while checking the content of the subtitles. By installing a small camera on the display on which a moving image is displayed and acquiring information on the movement of the viewer's line of sight, the viewer's viewing state (settled It is possible to reproduce and display a moving image at a reproduction speed according to whether the user is watching or gazing while moving various lines of sight.

Further, as ancillary information in the moving image reproduction method of the present invention, for example, audio data of a specific subject, an image feature amount, or the like may be used. When using the feature amount of face image data of a specific person or the feature amount of audio data as attached information, the feature amount of face image data or the feature amount of audio data of a specific subject is acquired in advance, and The degree of similarity can be determined by comparing a frame image or sound with a feature amount acquired in advance. That is, it is possible to determine how much the face image in the scene of each frame image is similar to the face image of which subject, and how much the sound data related to each frame image is similar to the sound data of which subject. In this way, with respect to a specific subject, only a scene portion within a predetermined range including the scene portion of the scene portion of the frame image in which the similarity of the feature amount of the face image or the audio data is equal to or higher than the predetermined similarity is normally set. By performing playback and making the other parts fast-forward playback, even during fast-forward playback, the scene portion in which the specific subject is captured can be played back and displayed at the normal playback speed.
As described above, in the moving image playback system and the playback method of the present invention, the type of attached information attached to a moving image is not particularly limited with respect to the attached state.

  The moving image playback system and the moving image playback method of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various improvements and modifications can be made without departing from the gist of the present invention. Of course.

It is a block diagram which shows the outline of an example of the moving image reproduction system of this invention. (A) And (b) is a figure explaining the response | compatibility with the scene relationship of the imaging | photography means and each subject in the moving image reproduction system of this invention, and the frame image acquired by the imaging | photography means. It is a flowchart figure which shows the flow of an example of the moving image reproduction method of this invention. It is a figure explaining the to-be-photographed area | region and the reproduction speed for every object in special reproduction mode of the moving image reproduction method of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Moving image reproduction system 12 Photographer 14 Photographing means 15 Photographing information acquisition means 16 Subject 18 Portable device 20 Data processing device 26 Communication control device 28 Media drive 30 Hard disk device (HDD)
32 Recording medium 50 Playback device 52 Playback condition specifying means 54 Playback speed setting means 56 Image data output means 58 Display 60 Recording medium

Claims (11)

A moving image reproduction system for reproducing and displaying a desired shooting scene of a moving image composed of a plurality of shooting scenes as a moving image at a predetermined reproduction speed,
Photographing means for acquiring the moving image;
Shooting information acquisition means that is provided in the shooting means and acquires shooting information including at least shooting position information and shooting direction information of the shooting means when each frame image of the moving image is acquired;
A shooting scene specifying means for specifying the desired shooting scene using the shooting information;
A moving image reproduction display means for displaying the specified desired shooting scene as a moving image at a predetermined reproduction speed;
The moving image reproduction display unit reproduces and displays a moving image of the desired shooting scene at a reproduction speed different from that before and after the desired shooting scene in the reproduction of the moving image. The desired shooting scene is an appearance scene of a specific subject,
Subject information acquisition means for acquiring subject information including at least subject position information of the specific subject;
The moving image reproduction system according to claim 1, wherein the shooting scene specifying unit specifies an appearance scene of the specific subject using the shooting information and the subject information. The shooting information acquisition means further acquires shooting time information of the shooting means when each frame image of the moving image is acquired,
The subject information acquisition unit further acquires detection time information when detecting position information of the specific subject,
3. The moving image reproduction system according to claim 2, wherein the shooting scene specifying unit specifies the scene of the specific subject using the subject information having the detection time information that matches or substantially matches the shooting time information. The desired shooting scene is a scene in which the specific subject appears in a specific posture,
The subject information further includes information on the posture of the specific subject,
The photographic scene specifying unit uses the subject information to specify an appearance scene of the specific subject, and further specifies a scene with a specific posture among the appearance scenes of the specific subject. 4. The moving image reproduction system according to 3. Further, the storage device stores identification information for identifying the desired shooting scene in the moving image, associated with the frame image as auxiliary information, and stored together with the moving image.
The moving image according to claim 1, wherein the moving image reproduction display unit displays the moving image stored in the storage unit at a reproduction speed according to the attached information. Image reproduction system.   6. The moving image reproduction system according to claim 5, wherein the attached information includes identification information for identifying a specific subject appearing in the shooting scene.   The moving image reproduction system according to claim 5 or 6, wherein the auxiliary information includes information regarding an appearance area of a specific subject in the shooting scene.   The moving image reproduction system according to claim 5, wherein the attached information includes information related to a posture of a specific subject in the shooting scene. Furthermore, a playback condition specifying means for specifying playback conditions is provided,
The shooting scene specifying unit extracts the attached information according to the playback condition specified by the playback condition specifying unit,
9. The moving image reproduction system according to claim 5, wherein the moving image reproduction display means displays the desired shooting scene as a moving image at a reproduction speed determined according to the extracted attached information. . A moving image reproduction method for reproducing and displaying a desired shooting scene of a moving image composed of a plurality of shooting scenes as a moving image at a predetermined reproduction speed,
A shooting step for acquiring the moving image;
A shooting information acquisition step of acquiring shooting information including at least shooting position information and shooting direction information when each frame image of the moving image is acquired;
A shooting scene specifying step for specifying the desired shooting scene using the shooting information;
A moving image reproduction display step of displaying the specified desired shooting scene at the predetermined reproduction speed as a moving image;
A moving image reproduction method characterized in that, in the moving image reproduction display step, a moving image of the desired shooting scene is reproduced and displayed at a reproduction speed different from that before and after the desired shooting scene in the reproduction of the moving image. A moving image reproduction method for reproducing and displaying a desired shooting scene of a moving image composed of a plurality of shooting scenes as a moving image at a predetermined reproduction speed,
A shooting step for acquiring the moving image;
A shooting information acquisition step of acquiring shooting information including at least shooting position information and shooting direction information when each frame image of the moving image is acquired;
A shooting scene specifying step for specifying the desired shooting scene using the shooting information;
A storage step of storing identification information for identifying the specified desired shooting scene in association with a frame image as attached information and storing it together with a moving image;
A playback condition specifying step for specifying playback conditions;
An extraction step of extracting the attached information according to the reproduction condition;
A moving image reproduction display step of displaying a moving image of the desired shooting scene at a reproduction speed determined according to the extracted attached information.

Images