JP2005252803A - Apparatus, method, and program for image processing and recording medium with the program stored - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable extracting a moving image in which an object desired by the user is photographed from a plurality of photographed moving images. <P>SOLUTION: Moving images photographed in time series are acquired as one unit moving image (S10), and a target static image formed by connecting predetermined frame images is stored from among frame images forming the unit moving image (S30). Next, the stored target static image is displayed on a display unit (S50), and the user views the displayed target static image, to designate a desired image position. Then, a frame image, including the designated position in its image region, is extracted from the displayed target static image, and extracts a unit moving image to which the extracted frame image belongs, from the stored unit moving image (S60). Accordingly, since a plurality of photographed moving images can be extracted by designating the image position of one displayed static image, the user can easily select a moving image, in which the object desired by the user is photographed from among a plurality of photographed moving images. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing technique for storing or reproducing a moving image captured by an imaging unit.

  Conventionally, an image in which a plurality of panoramic images generated from images captured by a plurality of imaging devices is displayed, and one of the panoramic images is selected by selecting one of the panoramic images is displayed. A processing method is disclosed (for example, Patent Document 1). As one of computer application programs, an image processing method for displaying thumbnails by reducing the first frame images of a plurality of stored moving images is also known as a well-known technique. By using these techniques, the user can easily select a plurality of captured moving images by displaying them on one screen and designating a desired moving image.

JP 2000-59682 A

  In recent years, athletic meet and travel, shooting with moving images is often performed instead of still photos. In this case, there are many various shooting scenes such as pan and tilt in the captured moving image. The moving images taken of the respective shooting scenes are usually discontinuous in shooting time or have different locations of subjects to be shot. Even if a moving image captured in this way is displayed on one screen by the conventional image processing technique described above, the positional relationship of the captured subject is not displayed. It was difficult to understand the location of the place. In particular, when the first frame image is displayed as a thumbnail for a moving image showing each shooting scene, only a part of the shot subject is displayed, so it is difficult to know where the shooting scene was shot. It was difficult to determine the positional relationship of the subject between the images. For this reason, it is not easy to select a moving image in which a subject desired by the user is captured from the displayed images.

  The present invention was made to solve at least a part of such problems, and for a plurality of captured moving images, an image is displayed so that the positional relationship of the subject captured in each moving image can be understood, An object of the present invention is to provide an image processing apparatus, an image processing method, an image processing program, or a recording medium on which the program is recorded, which can easily extract a moving image obtained by capturing a desired subject from a displayed image. .

  An image processing apparatus of the present invention that achieves the above object is an image processing apparatus that stores or reproduces a moving image picked up by an image pickup means, and a moving image picked up in time series as one unit moving image. A moving image storage unit that acquires and stores, a still image generation unit that generates and stores a target still image obtained by connecting predetermined frame images among frame images constituting the unit moving image, and the still image generation unit An image display unit that displays the generated target still image on the display unit, and a frame image that includes the image position designated from outside in the image region is extracted from the displayed target still image, and the extracted frame image And a moving image extraction unit that extracts a unit moving image to which the image belongs from the stored unit moving image.

  According to such an image processing apparatus, a subject captured in a plurality of captured moving images is displayed as one target still image using a predetermined frame image. Then, a frame image including the image position specified by the user in the image area is extracted from the displayed target still images, and a moving image to which the extracted frame image belongs is extracted. Therefore, the user can easily select a moving image in which a desired subject is captured from a plurality of captured moving images.

  Here, the still image generation unit sets a predetermined one of the frame images constituting the unit moving image as a reference frame image, and extracts a frame image that overlaps the set reference frame image. Thereafter, an image selection unit that selects a frame image that has the smallest overlapping portion with the reference frame image from among the extracted frame images, and when the image selection unit extracts a frame image, the image selection unit has already performed a reference frame. An image excluding unit that excludes a frame image set as an image and an already extracted frame image from an extraction target of the frame image, and generates a target still image by joining the frame images selected by the image selecting unit It is good also as what to do.

  In this way, it is possible to generate one target still image obtained by connecting a small number of frame images from one moving image. As a result, it is possible to display almost the entire image as an image obtained by joining a small number of frame images for the subject captured by the moving image, and to reduce the processing load for generating the target still image.

  Further, the still image generation unit calculates a relative positional relationship with respect to the target still image for the frame image used for generating the target still image, and attaches the calculated relative position data to the frame image. May be stored. In this way, when extracting the frame image including the image position designated by the user in the target still image in the image area, the corresponding frame image can be easily extracted by referring to the relative position data of the frame image. .

  Furthermore, the still image generation unit may store the frame image in association with the unit moving image to which the frame image belongs when storing the frame image. In this way, a moving image to which the frame image belongs can be immediately extracted from the extracted frame image.

  Further, when there are a plurality of target still images generated, the still image generation unit may newly generate an image obtained by joining the plurality of target still images as a target still image. In this way, it is possible to display each subject in a plurality of captured moving images as one still image including the same screen.

  The still image generation unit calculates relative positional relationship data between the frame images used for generating the target still image, and the plurality of target still images are calculated based on the calculated relative position data. It is good also as what connects. In this way, the relative position relationship between the target still images is determined in order to check the overlap between the target still images for each frame image used for generating each target still image. The load of data calculation processing can be reduced.

  Here, the image display unit of the present invention includes, for the frame image used for generating the target still image, an image area of at least one frame image for each unit moving image within the image area of the target still image. The identification display may be performed by a predetermined method. In this way, the user can easily search for a moving image in which a desired subject is captured from an image area in the displayed target still image.

  Furthermore, the image display unit may reproduce and display the unit moving image extracted by the moving image extraction unit on the display unit using a predetermined display method. In this way, the user can easily confirm whether or not the displayed moving image is a moving image obtained by capturing a desired subject.

  The image processing method of the present invention is an image processing method for storing or reproducing a moving image captured by an imaging means, and acquiring and storing a moving image captured in time series as one unit moving image A step of generating and storing a target still image obtained by joining predetermined frame images among the frame images constituting the unit moving image, a step of displaying the target still image generated by the still image generation unit, A frame image including an image position input from the input device in an image area is extracted from the displayed target still image, and a unit moving image to which the extracted frame image belongs is extracted from the stored unit moving image. And a process.

  Further, the present invention may be a computer program or a recording medium recording the program. That is, a computer program for storing or reproducing a moving image captured by an imaging unit, wherein a moving image captured continuously in time series is set as one unit moving image, and image data of the unit moving image is stored in a storage unit. A function of storing, a function of generating a target still image obtained by connecting predetermined frame images among frame images constituting the unit moving image, and storing the generated target still image as image data in a storage unit; A function of displaying the generated target still image on a display unit and a frame image including an image position input from an input device in an image region are extracted by reading out image data of the displayed target still image. A function of extracting the unit moving image to which the extracted frame image belongs by reading out the unit moving image data stored in the storage unit , And summarized in that to realize the computer. As a recording medium for the program, various computer-readable media such as a flexible disk, a CD-ROM, an IC card, and a punch card can be used.

  Next, an embodiment of the present invention will be described.

  FIG. 1 is an explanatory diagram showing a schematic configuration of an image processing apparatus as an embodiment of the present invention. This image processing apparatus is configured using a general-purpose computer, and includes a keyboard 110 and a mouse 120 as apparatuses for inputting information to the computer 100, and a display 130 and a video projector 180 as apparatuses for outputting information. . The digital video camera 160 is a device for inputting moving images to the computer 100, and a CD-R / RW drive 140 and a DVD-R / RW drive 150 are devices for inputting moving images and application programs to the computer. . In addition, the computer 100 may be provided with a drive device that can read data from a storage medium that stores a moving image or an application program, if necessary.

  The computer 100 executes an application program for generating and displaying a still image using image data of a frame image acquired from a stored moving image under a predetermined operating system, and thereby a flowchart described later. It functions as an image processing apparatus that performs the processing shown in FIG. 2, FIG. 3, FIG. 7, and FIG. The application program is supplied from the outside by a computer-readable recording medium such as a CD-ROM, for example, and a semiconductor memory such as a hard disk or RAM provided in the computer 100 via the CD-R / RW drive 140 (both shown in FIG. (Not shown). Of course, data may be stored by accessing a server or the like that supplies an application program via network means such as the Internet and downloading data.

  By executing the application program, the computer 100 functions as a moving image storage unit 102, a still image generation unit 103, an image display unit 104, and a moving image extraction unit 105, particularly as shown in FIG.

  Each unit mainly performs the following processing. The moving image storage unit 102 acquires a moving image input to the computer 100 by storing a series of moving images continuous in time series as a unit moving image in a storage unit such as a hard disk. The still image generation unit 103 generates one target still image from the frame images constituting the acquired moving image. The image display unit 104 displays the generated target still image on the display 130 or the video projector 180 serving as a display unit. The moving image extraction unit 105 extracts a moving image specified at an image position designated by the user from among the displayed target still images.

  Next, processing performed by the image processing apparatus of the present embodiment will be described with reference to the flowchart of FIG. In each processing step shown in the flowchart of FIG. 2, the image storage unit 102 performs step S10, the still image generation unit 103 performs steps S20 to S40, the image display unit 104 performs step S50, and the moving image extraction unit 105 performs step S60. And S70 are executed.

  When the process shown in FIG. 2 is started, first, in step S10, a unit moving image is acquired from the input moving image. Here, in the present embodiment, a series of moving images that are continuous in time series are acquired as unit moving images. For example, when time data at the time of imaging is recorded with a captured moving image, the image position of the moving image where the time data is discontinuous is set as a dividing point, and one dividing point is changed to the next dividing point. The moving images up to are stored in the storage means as unit moving images. Alternatively, when the time data is not recorded, the switching position of the shooting scene is detected, and the shooting scene from one detected switching position to the next switching position is used as a unit moving image. It may be memorized. The shooting scene switching position may be a well-known shooting scene detection method related to image processing techniques, such as a method of detecting a position where the average luminance data of the entire screen changes suddenly from the image data of a moving image.

  In this embodiment, the storage method is stored using a known moving image file format such as AVI (Audio Video Interleaved) that operates under the operating system of the computer 100. Thereby, the stored unit moving image can be extracted by designating the file name. The storage method is not limited to this, and any storage method may be used as long as it can extract the specified moving image in the moving image extraction process described later.

  Next, in step S20, a frame image selection process for generating a target still image is performed from the obtained unit moving images. This process will be described in detail below with reference to the flowchart of FIG. 3 and the schematic diagrams of FIGS. The purpose of the processing performed in step S20 is to generate a target still image having a substantially equal range of image areas to be displayed with a small number of frame images with respect to the entire image area of the subject captured by the unit moving image. This is performed in order to extract a predetermined frame image from the obtained frame image. By doing so, it is possible to reduce the load related to the process of generating the target still image.

  When the processing of FIG. 3 is started, first, frame images constituting a unit moving image are acquired in step S201. The acquisition method is performed by capturing from a unit moving image and storing the identification number in a storage unit such as a hard disk provided in the computer 100 with an identification number. Of course, when the unit moving image is filed in step S10 (FIG. 2), the frame image may be stored simultaneously.

  Next, in step S202, the calculation processing of the relative positional relationship data between all the frame images is performed for the plurality of frame images acquired by storing in the computer 100. This process is performed in advance in order to examine the overlap of the image area with the reference frame image, which is performed when the frame image is extracted in step S204 described later. This is because the overlap amount can be easily calculated if the positional relationship between the focused frame images is known.

  The relative positions of all the frame images are obtained by performing a positional relationship calculation process between two frame images, which will be described below, on a plurality of stored frame images. In this embodiment, two frame images continuous in time series are sequentially selected while being shifted one by one to calculate the relative position, and the relative position of each frame image is calculated.

  The relative position calculation process between two frame images will be described with reference to the explanatory diagram of FIG. FIG. 4A and FIG. 4B respectively show a frame image G1 and a frame image G2 for calculating a relative position between the frame images. This example assumes a frame image acquired from a moving image obtained by panning a mountain landscape against the sunset from the upper left to the lower right. There are usually translational shifts in the vertical and horizontal directions, rotational shifts that cause the screen to tilt, and image magnification differences due to zooming between the captured images. Two frame images obtained by such panning were assumed. In this case, it can be assumed that the inclination of the screen is extremely small between the two, and therefore there is no rotation shift. Also, there is no difference in image magnification between the two, so there is only a translational deviation. This translational deviation can be obtained as a translation vector.

  The translation vector can be expressed by the number of pixels in the horizontal direction and the vertical direction, and can be generated by a known processing technique such as image pattern matching or feature point tracking. If the relative position data calculated from the translation vector thus generated is the number of pixels X2 in the horizontal (X) direction of the screen and the number of pixels Y2 in the vertical (Y) direction of the screen, the upper left corner C1 of the screen of the frame image G1 is the reference point. When (0, 0) is set, the upper left corner C2 of the frame image G2 has coordinates (X2, Y2). This state is shown in FIG. Thus, the relative position of the frame image G2 with respect to the frame image G1 is calculated.

  Further, when there is a rotation shift or image magnification is different between the frame image G1 and the frame image G2, the relative position can be calculated by, for example, a combination of optical flow estimation and pattern matching. There are various optical flow estimation methods. For example, the gradient-based method is based on the assumption that the luminance of an object is invariable between observed frame images, and the density distribution space of the object in the image. Using the relationship between the gradient and the time gradient, the movement of the subject relative to the photographing apparatus is estimated. Based on the result of this optical flow estimation, roughly the translational movement and rotation of the image, and the change in the zoom magnification are estimated, and the frame image G1 and the frame image G2 are corrected based on the estimation result with the same rotational offset. Image processing is performed to obtain a frame image with a magnification. Then, the frame position after the image processing is newly set as the frame image G1 and the frame image G2, and the relative position is calculated by performing pattern matching between the images.

  By performing the relative position calculation process between the two frame images described above for all the frame images acquired in step S201, the relative positional relationship of all the frame images is calculated as coordinate data, and the processing step End S202 and proceed to the next processing step S203. In the present embodiment, as described above, the relative positional relationship data between two frame images is calculated from the degree of overlapping of the images, so that two consecutive frame images are sequentially selected. . This is because it is assumed that there is usually a large overlap between time-series continuous frame images. Of course, at least two frame images that are supposed to overlap each other may be sequentially selected to calculate the relative positions of all the frame images.

  In step S203, a process of setting a predetermined frame image among the acquired frame images as a reference frame image is performed. The reference frame image to be set serves as a reference for selecting a frame image (hereinafter referred to as “selected frame image”) for generating a target still image to be described later. At the same time, the reference frame image is also the first selected frame image.

  In this embodiment, in order to speed up the selection process, the first frame image in the acquired time series order is automatically set as a reference frame image by default. Of course, the user may set the reference frame image by designating the identification number stored along with the keyboard 110 or the like.

  In step S204, a frame image that overlaps the set reference frame image is extracted. Processing subsequent to step S204 will be described with reference to FIG.

  FIG. 5A illustrates the frame image acquired in step S201. Four consecutive frame images acquired in time series from a moving image obtained by panning from the left to the right of the screen and capturing a mountain landscape. It is the schematic diagram which showed typically P1-P4 (P1-P4 is an identification number of a frame image). Here, in order to simplify the explanation, it is assumed that four frame images P1 to P4 having the same magnification and no screen tilt are acquired. In the example of FIG. 5A, each frame image P2 calculated from the positional relationship data calculated in step S202 using X and Y coordinates with the upper left corner of the screen of the frame image P1 as the origin (0, 0). , P3, and P4 have a relative position of coordinates (X2, Y2), (X3, Y3), and (X4, Y4), respectively. In addition, about this embodiment, step S204 and the process after it are demonstrated on the assumption that the frame image shown by the schematic diagram of Fig.5 (a) was acquired.

  In step S205, it is determined whether there is a frame image that overlaps the reference frame image set by the process in step S203. Usually, since there is an overlap between a series of consecutive frame images in time series, as a result of the determination, there is at least one overlapping frame image (step S205: YES). In the example shown in FIG. 5A, the frame image P2 and the frame image P3 are extracted as frame images that overlap the frame image P1 that is the reference frame image.

  In step S206, it is determined whether or not there are a plurality of extracted frame images. If the result is one (step S206: NO), the extracted frame image is selected as the selected frame image. On the other hand, if there are a plurality of extracted frame images as a result of the determination (step S206: YES), the process proceeds to step S208, and the frame image with the smallest overlapping area with the reference frame image is selected as the selected frame image. The processing from step S203 to S208 described above corresponds to the image selection unit described in the claims. FIG. 5B shows a state in which a frame image P3 having a smaller overlapping area is selected as a selected frame image from two frame images P2 and P3 that overlap the reference frame image P1.

  As described with reference to FIG. 5, the overlapping area can be easily calculated from numerical values representing the upper left corner coordinate position of each frame image and the screen size of each frame image by adding and subtracting those numerical values. Therefore, in this embodiment, the description about the overlapping area calculation method is omitted.

  In the next step S209, the reference frame image and the extracted frame image that overlaps with the reference frame image are excluded from the extraction target of the frame image when extracting the frame image that overlaps the newly set reference frame image. Perform the process. By doing so, it is possible to avoid the same frame image from being extracted many times and to speed up the extraction process of the selected frame image. This process corresponds to the image exclusion unit recited in the claims.

  In the next step S210, the selected frame image, that is, the selected frame image is newly set as a reference frame image, and the process returns to step S204, and a frame image overlapping the new reference frame image is extracted again.

  FIG. 5C shows a state in which an overlapping frame image P4 is extracted from the frame image P3 that is a new reference frame image. The frame images P1 and P2 indicated by broken lines in the figure are frame images that overlap the frame image P3, but the frame image P1 is the previous reference frame image, and the frame image P2 is extracted when the reference frame image is the frame image P1. As described above, the frame image is excluded from the extraction target frame image. Therefore, they are not extracted as overlapping frame images.

  In step S205, it is determined whether there is a frame image that overlaps with the newly set reference frame image. If the result of determination is that there is no frame image (NO), the processing routine here is terminated and processing returns to step S30 (FIG. 2). If the frame image that overlaps the newly set reference frame image is only the frame image that has already been extracted by the previous process, the processing routine here ends.

  On the other hand, if the result of determination is that there is a frame image, the processing from step S206 is executed again, the selected frame image is set again as a new reference frame image, and the processing up to step S205 is executed in step S205. Repeat until no overlapping frame images exist. Since the processing from step S206 to step S205 is basically the same as the processing described above, description thereof will be omitted.

  In this way, after the processing up to step S205 is repeatedly performed, if the extracted frame image does not exist as a result of the determination in step S205 (step S205: NO), the processing in FIG. 3 is terminated, and step S30 (FIG. Return to 2). In the example of FIG. 5, the frame images P1, P3, and P4 are finally selected as the selected frame images.

  Next, in step S30, the target still image is stored. The storage method is performed by storing as management data in a semiconductor memory (not shown) such as a hard disk or RAM provided in the computer 100. The management data includes at least the identification number and relative position data of the selected frame image, and unit moving image file data from which the selected frame image is acquired, and is stored in association with the target still image.

  FIG. 6 illustrates the management data of the target still image stored in the present embodiment, and represents the management data of the target still image generated for the unit moving image shown in FIG. The management data includes the name of the target still image “target still image A”, the number of unit moving images “1” included in the target still image, the file name “FILE1.AVI” of the unit moving image as the configuration data, the selected frame The number of images “3”, the selected frame image number “selected frame image P1, selected frame image P3, selected frame image P4” and relative position coordinates “(0, 0), (X3, Y3) between the selected frame images , (X4, Y4) ”is stored. By storing the target still image as such management data, the processing in step S60 (extraction processing of a unit moving image corresponding to the image position designated from the outside) can be facilitated.

  Next, in step S40 (FIG. 2), processing for joining a plurality of target still images is performed. Usually, an image taken by a user has a plurality of continuous images in time series, that is, a plurality of unit moving images. By connecting these unit moving images and generating them as one target still image, it is possible to display the subject image captured by each unit moving image as one target still image.

  At the same time, by recalculating the positional relationship of the selected frame image selected from the unit moving image as the relative positional relationship data with respect to the generated target still image, the position of each selected frame image becomes one target. The coordinate data common to the still images is provided.

  The processing for joining the target still images will be described in detail with reference to the flowchart of FIG. 7 and FIGS.

  When this process is started, first, two target still images A and B to be joined are selected in step S401. In the present embodiment, one selected target still image A is the image illustrated in FIG. 5C, and the other selected target still image B is the image shown in the schematic diagram of FIG. Image joining processing will be described.

  Prior to the description of the joining process, the target still image B shown in FIG. 8 will be described. FIG. 8 shows a unit moving image that starts from the vicinity of the subject shown in the image of the frame image P4 shown in FIG. 5 and captures the landscape by tilting downward. FIG. 8A shows four consecutive frame images T1 to T4 (T1 to T4 are identification numbers of frame images) acquired in time series from this unit moving image by the same process as in step S201 (FIG. 3) described above. ) Is a schematic diagram schematically showing.

  Here, in order to simplify the description, it is assumed that four frame images T1 to T4 having the same magnification and no screen tilt are acquired. In the example of FIG. 8A, the upper left corner of the screen of the frame image T1 is set to the origin (0, 0) from the positional relationship data calculated by the same process as that described in step S202 (FIG. 3). The relative positions of the frame images T2, T3, and T4 calculated using the X and Y coordinates are coordinates (X2b, Y2b), (X3b, Y3b), and (X4b, Y4b), respectively.

  Next, FIG. 8B shows a state in which the selected frame images T1, T3, and T4 are selected by the same process as in step S20 of FIG. Since the selection process is basically the same as the process described above, a description thereof will be omitted. As a result, the target still image B is stored in the storage device of the computer 100 using the same management data as in FIG. 6, with the name of the target still image “target still image B” and the number of unit moving images included in the target still image “1”. , The file name “FILE2.AVI” of the unit moving image as the configuration data, the number of selected frame images “3”, the selected frame image number “selected frame image T1, selected frame image T3, selected frame image T4” and the respective selected frames Relative position coordinates “(X1B, Y1B), (X3B, Y3B), (X4B, Y4B)” between images are stored (not shown).

  Now, a process for joining the target still images A and B described above will be described. In the next step S402, the selected frame images constituting the target still image A are sequentially selected one by one. Here, it is assumed that the selection frame image P1 is first selected in time series order, and the selection frame image P4 is sequentially selected.

  Next, in step S403, the selected frame images constituting the target still image B are sequentially selected one by one. Also in this case, it is assumed that the selected frame image T1 is first selected in chronological order, and the selection frame image T4 is sequentially selected.

  Next, in step S404, the relative position between the selected frame images each selected from the target still images A and B is calculated. In this embodiment, the relative position calculation method is performed by the relative position calculation process between two frame images described with reference to FIG. If there is an overlapping portion between the two selected frame images, the positional relationship between the two images is a translation vector, that is, the horizontal direction (X) by a known processing technique such as image pattern matching or feature point tracking as described above. And the difference value between the number of pixels in the vertical direction (Y).

  In the image pattern matching process, it is usually determined whether or not matching is performed using a threshold value. Therefore, it is preferable to set an appropriate threshold value as a reference for determining whether or not there is an overlap between the two selected frame images. For example, the threshold value of the matching process set between two frame images that are known to be overlapped, such as the selected frame images T1 and T2, may be set as the threshold value of the matching process performed in step S404. In this way, it is possible to accurately determine the overlap of images between the two selected frame images.

  Next, in step S405, it is determined whether or not there are selection frame images that overlap each other, that is, selection frame images for which the relative positions between the selection frame images are calculated. If the selected frame image does not exist as a result of the determination (NO), the target still image A and the target still image B do not overlap and cannot be connected to each other on the display 130 (not shown). The process in is terminated.

  On the other hand, if there is a selection frame image where the images overlap as a result of the determination (step S405: YES), the relative position between the target still images A and B is calculated based on the calculated relative position between the selection frame images. (Step S406). Then, in accordance with the calculation result of the relative position, processing for joining the target still images A and B is performed (step S407), and the joined target still image is newly set as the target still image A (step S408).

  In step S409, the relative position of the selected frame image constituting the new target still image A is corrected and stored as management data. That is, the process of correcting the relative position of the selected frame image to the X and Y coordinate values based on one frame image is performed. In this way, the relative positions of the selected selected frame images are displayed by a common X and Y coordinate system, and the position of each selected frame image can be determined with respect to a new target still image A. In the present embodiment, the X and Y coordinate values of each selected frame image are corrected using the selected frame image P1 as a reference. Of course, any one selected frame image may be used as a reference.

  FIG. 9A illustrates a new target still image A after the processing for joining the target still images A and B is performed. The new target still image A is composed of selected frame images P1, P3, P4, T1, T3, and T4 selected from two unit moving images. FIG. 9B illustrates management data of the new target still image A illustrated in FIG. In the management data, the name of the target still image “target still image A”, the number of unit moving images included in the target still image “2”, and the file name “FILE1.AVI” of the unit moving image as the configuration data 1 are selected. The number of frame images “3”, the selected frame image number “selected frame image P1, selected frame image P3, selected frame image P4” and their relative position coordinates “(0, 0), (X3, Y3), (X4, Y4) ”, the file name“ FILE2.AVI ”of the unit moving image as the configuration data 2, the number of selected frame images“ 3 ”, the selected frame image number“ selected frame image T1, selected frame image T3, selected frame image T4 ”. And the corrected relative position coordinates “(X1B, Y1B), (X3B, Y3B), (X4B, Y4B)” are stored.

  Thus, the process for joining the target still images in FIG. 7 ends, and the process returns to step S50. If there are more target still images to be joined together, the target still images are selected from the existing target still images in order, and the above-described joining processing of the two target still images is repeatedly performed. All target still images to be joined are processed.

  In the next step S50, a process of displaying the connected target still images A on the display unit is performed. In this embodiment, it is displayed on the display 130. Of course, it may be displayed on a display unit provided outside the video projector 180 or the computer 100. By displaying the target still image A, the entire image of the photographed subject is displayed on one screen, and the user views the entire image and designates the image in which the desired subject is shown. can do.

  FIG. 10 is a schematic view illustrating the target still image A displayed on the display 130. In the present embodiment, the smallest rectangular still image including all the target still images A is displayed on the display 130. Therefore, an image area N indicated by a shaded portion in the figure is an image area where no target still image exists. In the present embodiment, black is displayed for the image region N, and image data of a rectangular still image is generated using the image data of the selected frame image, the relative position coordinates, and the black image data. Of course, the display color of the image area N may be any color that can be distinguished from the display image area of the target still image.

  Next, in determination step S60, it is determined whether or not an image position is designated from the outside. As a result of the determination, if there is no image position designated from the outside (NO), all the processes are terminated. As a result of the determination, if there is an externally designated image position (YES), the process proceeds to the next step S70.

  In the present embodiment, it is assumed that the user performs designation from the outside. When the user looks at the target still image A displayed on the display 130 and wants to view a unit moving image in which a desired subject is captured, the position of the image in which the subject is captured using a mouse 120 with a pointer (not shown). Click the mouse to specify the image position. On the other hand, when the image position is not specified, such as when viewing is not performed, the user clicks on an icon provided on the display 130 and inputs that there is no image position to be specified.

  Next, in step S60, unit moving image extraction processing corresponding to the image position designated from the outside is performed. This process will be described with reference to the flowchart of FIG. 11 and the schematic diagram of FIG.

  When the process is started, first, in step S701, an externally designated image position is acquired. In the present embodiment, the click position is changed to the X and Y coordinate values on the management data of the target still image A from the image data of the displayed rectangular still image and the image data corresponding to the image position of the pointer at the time of clicking. Get by replacing. FIG. 12A illustrates a state where the coordinates of the image position S designated by the user in the image region of the target still image A are (Xs, Ys) in the target still image.

  Next, in step S702, a process of extracting a selected frame image including the designated image position S (Xs, Ys) in the image area is performed. With reference to the management data (FIG. 9B) of the target still image A stored in step S409 (FIG. 7), the X and Y coordinate values of the selected frame image and the X and Y coordinates of the designated image position S The corresponding selected frame image is extracted using the value.

Specifically, when the horizontal length of the selected frame image is W and the vertical length of the screen is H, the selected frame image in which the X coordinate value and the Y coordinate value satisfy the following expressions (1) and (2) To extract.
Xs−W ≦ X ≦ Xs (1)
Ys−H ≦ Y ≦ Ys (2)
FIG. 12B shows a state in which the extracted selection frame images are two of the selection frame image P4 and the selection frame image T1.

  In the next step S703, a process of extracting a unit moving image to which the extracted selected frame image belongs is performed. Similar to the previous step S702, referring to the management data of the target still image A, the unit moving image file having the selected frame image as the configuration data is specified and extracted. In the example of FIG. 12B, the unit moving image file FILE1. AVI and FILE2. AVI is extracted.

  Next, it is determined whether or not the unit moving image extracted in step S704 exists. If no unit moving image exists as a result of the determination (NO), the absence of unit moving image is displayed on the display 130 (step S705). This process assumes that the user clicks outside the image area of the target still image A (image area N in FIG. 10).

  If the unit moving image exists as a result of the determination (step S704: YES), it is determined in the next determination step S706 whether there are a plurality of unit moving images. As a result of the determination, if there is one unit moving image instead of a plurality (NO), the processing here is terminated and the process proceeds to step S70 (FIG. 2). On the other hand, if it is determined that there are a plurality of unit moving images (step S706: YES), the process proceeds to step S707.

  In step S707, unit moving image selection processing is performed. Specifically, the unit moving image identification display and the user-selected unit moving image recognition process are performed.

  With the start of this process, an identification display that identifies a plurality of unit moving images is displayed on the display 130, and the user uses the keyboard 110 or the mouse 120 to select a desired unit moving image according to the identification display. . As the identification display, for example, when the above-mentioned image position S is clicked, the unit moving image file name may be displayed in a pop-up menu (not shown). Alternatively, as shown in FIG. 12B, by displaying the selected frame image including the image position S in the image area as a “frame”, the existence of a plurality of unit moving images is displayed, and this “frame” is displayed. It may be clicked. In this case, using different frame colors makes it easier to identify. By performing the above processing, the unit moving image extraction processing routine is completed, and the process returns to step S80 (FIG. 2).

  In the next step S80, a process of reproducing the unit moving image obtained by imaging the subject desired by the user thus extracted is performed. Reproduction is performed by reading out image data from the stored unit moving image file, converting the signal into an image signal suitable for a display device that displays the unit moving image, and outputting the image signal. The signal conversion is performed by a signal conversion device (both not shown) built in or externally connected to the computer 100.

  In the present embodiment, the unit moving image is reproduced and displayed at a predetermined position on the display 130. The display position of the unit moving image may be the position of the selected frame image extracted in step S702 (FIG. 11) or may be a display area for reproduction provided on the display 130. Of course, the display area may be created by the user by dragging the mouse 120.

  By performing the above processing, the processing of the image processing apparatus in this embodiment is completed.

  As described above, according to the present embodiment, a plurality of captured unit moving images are displayed as one target still image, and the corresponding unit moving image is associated with each image position of the displayed target still image. It can be stored as an image file. Therefore, the user can also store and save a plurality of captured unit moving images as one image file. For example, by making a unit moving image that captures the entire conference landscape and the speech scenes of the conference participants into one image file according to the present embodiment, clicking on the image location of the conference participants from the entire conference landscape image It is possible to use such that a comment scene is reproduced and displayed as a moving image at a position.

  As mentioned above, although one embodiment of the present invention has been described, the present invention is not limited to such an embodiment, and can of course be implemented in various forms without departing from the spirit of the present invention. is there.

  As a first modification, in the embodiment, the frame image selection process is performed to generate the target still image (FIG. 2, step S20), but another method may be used. For example, the target still image may be generated using all the frame images constituting the unit moving image. In this way, although the image processing load associated with the generation of the target still image increases, all the captured images of the subject can be displayed on the target still image. Alternatively, in order to reduce the load of the frame image selection process, the frame images may be selected at predetermined time intervals in time-series order, or the frame images may be selected at predetermined number intervals.

  As a second modification, in the embodiment, only the target still image is displayed in the target still image display process (FIG. 2, step S50), but a predetermined selection is selected from the selection frame images constituting the target still image. A frame image may also be identified and displayed. In this way, the user can designate the image position S by looking at the selected frame image identified and displayed among the target still images. As a result, it is possible to appropriately extract a unit moving image obtained by imaging a subject desired by the user.

  The predetermined selection frame images to be identified and displayed are, for example, as shown in FIG. 13A, the first selection frame images belonging to each unit moving image in the time series order, that is, the selection frame image P1 and the selection frame image T1. Also good. Of course, it may be the last selected frame image in chronological order, or may be the substantially selected frame image in chronological order. Alternatively, as illustrated in FIG. 13B, all selected frame images belonging to each unit moving image may be identified and displayed. In this way, the user can easily confirm the image range of the subject captured by one unit moving image among the displayed target still images.

  Furthermore, as a third modification, when the selected frame image is identified and displayed, predetermined information accompanying the unit moving image to which the selected frame image belongs may be displayed. As the predetermined information, for example, as illustrated in FIG. 13C, the file name of the unit moving image may be used. Or it is good also as the information about the imaging methods, such as the time information when the unit moving image was imaged, pan, tilt, zoom, fixation. In this way, the user can appropriately select a unit moving image in which a desired subject is captured from the selected and displayed selection frame image.

  As a fourth modification, the image processing apparatus in the present embodiment is configured by a general-purpose computer. However, the present invention is not limited to this, and is configured by a mobile computer, a workstation, or the like. It may be. Alternatively, the image processing apparatus of the present invention may be configured in various devices such as a digital camera having a function as a computer, a video camera, a DVD player, a video projector, and a mobile phone.

1 is an explanatory diagram illustrating a schematic configuration of an image processing apparatus as an embodiment of the present invention. 6 is a flowchart of processing performed by the image processing apparatus according to the present embodiment. The flowchart of the selection process of the frame image for producing | generating a target still image. Explanatory drawing for demonstrating the calculation method of the relative position between frame images. Explanatory drawing explaining the selection method of the frame image which produces | generates a target still image. Explanatory drawing for demonstrating the management data of a target still image. The flowchart of the joining process of an object still image. Explanatory drawing about another object still image to connect. Explanatory drawing for demonstrating the target still image connected. Explanatory drawing explaining the state which displayed the connected target still image. FIG. 10 is a flowchart of unit video extraction processing corresponding to a specified image position. FIG. Explanatory drawing for demonstrating the designated image position. Explanatory drawing which shows the example of a display which identifies and displays the selection frame image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Computer, 102 ... Moving image memory | storage part, 103 ... Still image generation part, 104 ... Image display part, 105 ... Moving image extraction part, 110 ... Keyboard, 120 ... Mouse | mouth, 130 ... Display as a display part, 140 ... CD -R / RW drive, 150 ... DVD-R / RW drive, 160 ... digital video camera, 180 ... video projector as a display unit, P1 to P4 ... frame images acquired from unit moving images, T1 to T4 ... others Frame image acquired from unit moving image, G1 ... frame image, G2 ... frame image, A ... target still image, N ... image region other than frame image in target still image, S ... designated image position.

Claims (11)

An image processing apparatus for storing or reproducing a moving image captured by an imaging unit,
A moving image storage unit that acquires and stores moving images captured continuously in time series as one unit moving image;
Among the frame images constituting the unit moving image, a still image generation unit that generates and stores a target still image obtained by connecting predetermined frame images;
An image display unit for displaying a target still image generated by the still image generation unit on a display unit;
A frame image including an image position input from the input device in an image region is extracted from the displayed target still image, and a unit moving image to which the extracted frame image belongs is extracted from the stored unit moving image. A moving image extraction unit to
An image processing apparatus. The image processing apparatus according to claim 1,
The still image generation unit
A predetermined one of the frame images constituting the unit moving image is set as a reference frame image, a frame image that overlaps the set reference frame image is extracted, and then the extracted frame image An image selection unit that selects a frame image with the smallest overlapping portion with the reference frame image;
When the image selection unit extracts a frame image, an image exclusion unit that excludes the frame image that has already been set as a reference frame image by the image selection unit and the frame image that has already been extracted from the extraction target of the frame image. Prepared,
An image processing apparatus that generates and stores a target still image obtained by joining frame images selected by the image selection unit. The image processing apparatus according to claim 1, wherein:
The still image generation unit calculates a relative positional relationship with respect to the target still image for the frame image used for generation of the target still image, and stores the frame image accompanied with the calculated relative position data. An image processing apparatus. The image processing apparatus according to claim 3,
The still image generation unit is an image processing apparatus that stores the frame image in association with the unit moving image to which the frame image belongs when the frame image is stored. An image processing apparatus according to any one of claims 1 to 4,
When there are a plurality of generated target still images, the still image generation unit is an image processing apparatus that newly generates an image obtained by joining the plurality of target still images as a target still image. The image processing apparatus according to claim 5,
The still image generation unit calculates relative positional relationship data between frame images used for generating the target still image, and connects the plurality of target still images based on the calculated relative position data. Image processing device. The image processing apparatus according to any one of claims 1 to 6,
The image display unit identifies and displays an image area of at least one frame image for each unit moving image in a predetermined method within the image area of the target still image for the frame image used for generating the target still image. An image processing apparatus. An image processing apparatus according to any one of claims 1 to 7,
The image display device is an image processing device that reproduces and displays a unit moving image extracted by the moving image extraction unit on a display unit using a predetermined display method. An image processing method for storing or reproducing a moving image captured by an imaging unit,
Acquiring and storing a moving image continuously captured in time series as one unit moving image;
Generating and storing a target still image obtained by joining predetermined frame images among the frame images constituting the unit moving image;
Displaying the target still image generated by the still image generation unit;
A frame image including an image position input from the input device in an image region is extracted from the displayed target still image, and a unit moving image to which the extracted frame image belongs is extracted from the stored unit moving image. And a process of
An image processing method comprising: A computer program for storing or reproducing a moving image captured by an imaging means,
A function of storing, as a unit moving image, a moving image captured continuously in time series, and storing image data of the unit moving image in a storage unit;
A function of generating a target still image obtained by connecting predetermined frame images among frame images constituting the unit moving image, and storing the generated target still image as image data in a storage unit;
A function of displaying the generated target still image on a display unit;
A frame image including an image position input from an input device in an image region is extracted by reading out image data of the displayed target still image, and a unit moving image to which the extracted frame image belongs is stored in the memory A function of extracting by reading from the unit moving image data stored in the unit,
An image processing program for realizing a computer. A computer-readable recording medium on which the image processing program according to claim 10 is recorded.

Images