JP2011040860A - Image processor and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem not to select a scene considering desirability of the scene even if detecting a similar scene based on change of a feature quantity. <P>SOLUTION: An image processor includes: a feature quantity detection part 22 which detects feature quantities from respective frame images of continuous photographing image data groups or moving image data; an event detection part 23 which detects an event based on change of feature quantities accompanying time change; a transition evaluation part 24 which calculates a transition evaluation value based on a predetermined evaluation index for event detection frame images which is frame images corresponding to the detected event, and frame image groups of a predetermined time following it; and a selection part 25 which selects a scene by the frame image groups including the event detection frame images based on the calculated transition evaluation value. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Some recent digital cameras have not only a still image single shooting function but also a continuous shooting function and a moving image shooting function. A still image or a moving image shot with this digital camera may include a plurality of similar shooting scenes. For example, if you take a series of shots every time you circulate a person on a carousel from the same shooting location, or if you shoot a scene in which a person turns, the same shooting scene will be recorded multiple times. become. In such a case, the photographer selects and records a well-recorded continuous shooting group or movie from the continuous shooting image data group or movie data of the same shooting scene recorded immediately after shooting or at a later date. It is necessary to delete the other items. Such work is troublesome, and a function that is automatically selected by objective evaluation and determination is desired.

  2. Description of the Related Art A moving image data processing apparatus is known for specifying a break position corresponding to a break of moving image content (for example, a taken production process break) from moving image data (see, for example, Patent Document 1). The moving image data processing device extracts a plurality of types of feature amount information that is time-series data from the moving image data, and selects specific feature amount information based on the magnitude of the feature amount change. Then, the time series position having the largest feature amount change in the selected feature amount information is specified as the delimiter position. This moving image data processing apparatus is known to be used in a production management apparatus that discriminates a manufacturing process based on a captured moving image, a moving image editing apparatus that performs chapter classification of moving image data, generation of index information, and the like. It has been.

  By using the above-described conventional technology, it is possible to detect a frame image corresponding to a time-series position having a large feature amount change from a continuous shot image data group or moving image data. Conventionally, a time-series position at which a specific feature amount greatly changes is regarded as an event occurrence position from continuous image data groups and image data including similar shooting scenes, and a frame image corresponding to the event occurrence position is captured. It was detected and estimated as a main image in a shooting scene similar to each other.

JP 2008-9938 A

  According to the above-described prior art, it is possible to detect main frame images in mutually similar shooting scenes from a continuous shot image data group or moving image data including similar shooting scenes. Then, by selecting a time-series continuous-shot image data group or moving image data including the detected frame image, it is possible to extract a continuous-shot image data group or moving image data that is estimated to have high similarity. is there.

  However, it is not possible to determine which continuous-shot image data group or moving image data is the most preferable shot from among the continuous-shot image data group or moving image data extracted in this way. The determination of the preference of a scene is left to human subjective judgment.

  Therefore, the present invention has been made in view of the above circumstances, and image processing that can detect a similar scene and select a desired scene based on an objective evaluation criterion for continuous shot image data groups and moving image data. An object is to provide an apparatus and an image processing program.

  The present invention has been made in order to solve the above-described problem, and includes feature amount detection means for detecting a feature amount from each frame image of a continuous shot image data group or moving image data, and the feature amount accompanying a time change. A predetermined evaluation index for event detection means for detecting an event based on the change of the event, an event detection frame image that is a frame image corresponding to the detected event, and a frame image group for a predetermined time following the time series Evaluation means for calculating an evaluation value based on the above and a selection means for selecting a scene based on a group of frame images including an event detection frame image based on the evaluation value.

  According to the present invention, by detecting a significant event detection frame image, it can be estimated that a frame image group based on the event detection frame image is a similar scene. Then, a scene according to an objective evaluation criterion can be selected from those estimated to be similar scenes.

1 is a block diagram of an imaging apparatus to which an image processing apparatus according to first and second embodiments of the present invention is applied. It is the block diagram which showed the structure of the scene selection process part of the imaging device by 1st and 2nd embodiment of this invention, and the structure of the peripheral part. 3 is a flowchart showing processing of a scene selection function in the imaging apparatus according to the first embodiment of the present invention. It is a flowchart of the event detection process which the scene selection process part performs in the imaging device by 1st and 2nd embodiment of this invention. It is a graph of the face detection value by the feature-value detection part at the time of inputting the frame image sequentially in 1st Embodiment of this invention. It is a flowchart of the transition evaluation process which the scene selection process part performs in 1st and 2nd embodiment of this invention. It is a graph of the evaluation value of the degree of front direction at the time of inputting a frame image for a fixed period sequentially in a 1st embodiment of the present invention. It is a graph of the evaluation value of a center position degree at the time of inputting a frame image for a fixed period sequentially in a 1st embodiment of the present invention. It is a graph of the evaluation value of a blink detection value when a frame image for a certain period is sequentially input in the first embodiment of the present invention. It is a graph of the evaluation value of a smile degree at the time of inputting the frame image for a fixed period sequentially in 1st Embodiment of this invention. In the first embodiment of the present invention, the imaging device set to the scene selection function detects an event from each of the three data files, performs transition evaluation, extracts three similar scenes, and out of them. It is the figure which represented a mode that one scene was extracted. It is the figure which represented typically one frame image among the continuous shooting image | photographed continuous shooting image data groups in 2nd Embodiment of this invention. It is a flowchart which shows the process of the scene selection function in the imaging device by 2nd Embodiment of this invention. It is the figure which graphed and showed the transition of the coordinate value of the X direction of the gravity center position of the person in each frame image about each of two data file groups in 2nd Embodiment of this invention.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

[First Embodiment]
An example in which the image processing apparatus according to the first embodiment of the present invention is applied to an imaging apparatus will be described. This imaging device is, for example, a digital camera device. In addition to a single shooting function for shooting one still image by one shooting operation (pressing a shooting button), a continuous shooting function, a moving image shooting function, and a scene selection function It has a function. The continuous shooting function is a function for shooting a plurality of still images in time series for one shooting operation. The moving image shooting function is a function for shooting a moving image. The scene selection function is a function that detects and evaluates similar scenes from a group of continuous-shot image data obtained by continuous shooting or from moving image data obtained by moving-image shooting, and selects a scene that satisfies a predetermined evaluation criterion from these. is there. A scene is a group of continuous shot image data or moving image data for an arbitrary time.

  In this embodiment, this scene selection function will be mainly described. The imaging apparatus according to the present embodiment detects an event that is a feature change point by calculating a difference value of a feature amount between adjacent frame images with respect to one or a plurality of moving image data captured and recorded. If a plurality of events are detected as a result of the search, it is estimated that the frame images from which the events are detected are prominent images in similar scenes. Then, the imaging apparatus evaluates the transition of the image with a predetermined evaluation index for each frame image in which the event is detected and the frame image that continues in time series, and the most highly evaluated scene among similar scenes is evaluated. select. With this scene selection function, the imaging apparatus according to the present embodiment can rank scenes based on objective evaluation criteria when similar scenes are included in the recorded moving image data. , A scene having a desired order can be extracted. Hereinafter, an imaging apparatus having such characteristics will be described in detail.

  FIG. 1 is a block diagram of an imaging apparatus to which the image processing apparatus according to this embodiment is applied. In FIG. 1, an imaging apparatus 1 includes an optical system 11, an imaging unit 12, an operation unit 13, a control unit 14, a storage unit 15, a display unit 16, an input / output interface 17, and a recording medium 19. Consists of including. In the imaging device 1, the imaging unit 12, the control unit 14, the storage unit 15, the display unit 16, and the input / output interface 17 are connected to each other via a bus 18 inside the device. The imaging device 1 is composed of, for example, a digital camera.

  The optical system 11 includes a lens group, a diaphragm, and a focal length adjustment mechanism. The optical system 11 may be detachably attached to the main body of the imaging device 1. The imaging unit 12 forms an image of the light beam incident through the optical system 11 on the light receiving surface, and generates captured image data. Although not shown, the operation unit 13 receives a power button, a shooting button, a zoom button, a shooting mode for performing a shooting operation, and still image data, a continuous shot image data group, or moving image data recorded on the recording medium 19. A mode switching button for switching between a reproduction mode for reading and displaying on the display unit 16, and a transition to a setting mode for performing various settings of the imaging device 1 to display an operation menu on the display unit 16. Menu button, a selection button for operating the operation menu displayed on the display unit 16, and a determination button for determining an item selected on the operation menu by the operation of the selection button.

  The control unit 14 is a processing unit for controlling the entire imaging apparatus 1 and includes a CPU (Central Processing Unit). The control unit 14 includes a scene selection processing unit 20 and an image compression / decompression unit 21 as its functional configuration. The scene selection processing unit 20 will be described when referring to FIG. The image compression / decompression unit 21 compresses the captured image data generated by the imaging unit 12, and in the case of the single shooting function, the compressed still image data, in the case of the continuous shooting function, the compressed continuous image data group, and the moving image shooting In the case of a function, compressed moving image data is generated. The image compression / decompression unit 21 decompresses the compressed still image data, the compressed continuous-shot image data group, or the compressed moving image data read from the recording medium 19 to obtain the decompressed still image data and the decompressed continuous-shot image data. Group or decompression moving image data is generated.

  The storage unit 15 stores a ROM (Read Only Memory) that stores various control programs to be executed by the control unit 14, expands these control programs in a work area, and stores image data, operation menus, variables, and the like. RAM (Random Access Memory). The display unit 16 displays captured image data in an imaging state in the shooting mode, displays expanded still image data, extended continuous image data group, or expanded moving image data in the playback mode, and displays an operation menu or the like in the setting mode. A display device for displaying. The display device 16 includes a monitor having a liquid crystal display panel or an organic EL (Electro-Luminescence) panel. The input / output interface 17 is an interface unit including a connector for detachably connecting the recording medium 19. The recording medium 19 is a portable recording medium attached to the connector of the input / output interface 17. The recording medium 19 is a memory card, for example.

  Next, the scene selection processing unit 20 will be described. FIG. 2 is a block diagram showing the configuration of the scene selection processing unit 20 functionally included in the control unit 14 and the configuration of its peripheral part. In the figure, the scene selection processing unit 20 includes a feature amount detection unit 22, an event detection unit 23, a transition evaluation unit 24, a selection unit 25, a feature amount parameter setting unit 26, and an evaluation parameter setting unit 27. Consists of including.

  The feature amount detection unit 22 reads a desired compressed continuous shot image data group or compressed moving image data from the recording medium 19 and decompresses the image data by the image compression / decompression unit 21. Group or decompressed moving image data is input in units of frame images. The feature amount detection unit 22 detects a predetermined feature amount when a frame image is input. This feature amount is, for example, a quantification of visual features such as color, brightness, hue, and contrast in a frame image, image features such as the presence or absence of objects such as textures and human faces, and object positions. is there.

  The feature quantity parameter setting unit 26 sets one or more types of features to be detected by the feature quantity detection unit 22 and sets the priority order of feature quantity detection. The event detection unit 23 detects the same type of feature amount change between frame images at different times, and detects an event that is a feature change point when the change satisfies a predetermined criterion. Event detection will be described with a specific example in the operation description to be described later.

  The transition evaluation unit 24 performs transition evaluation, which is an evaluation of image transition based on a predetermined evaluation index, on the frame image in which the event is detected by the event detection unit 23 and the frame image that follows the time series. The evaluation index is, for example, an index that is referred to when evaluating a state change of an arbitrary evaluation target with the passage of time of an object whose image is recognized in a frame image. That is, the transition evaluation performed by the evaluation index is to evaluate the tendency of the frame image change by the evaluation index with the event detection time as a reference. The evaluation index will be described with a specific example in the operation description to be described later.

  The evaluation parameter setting unit 27 sets one type or a plurality of types of evaluation indexes to be evaluated by the transition evaluation unit 24, or sets the evaluation priority order based on a plurality of types of evaluation indexes.

  The selection unit 25 designates a frame image in which an event corresponding to the highest evaluation value among the evaluation values for each event evaluated by the transition evaluation unit 24 is detected, and a scene including the frame image The information of the start position and end position (scene position information) is output. When there is one event detected by the event detection unit 23, the selection unit 25 designates the frame image in which the event is detected, and outputs the position information of the scene including the frame image.

  Next, the operation in the scene selection function of the imaging apparatus 1 will be described. In the present embodiment, the imaging device 1 set to the moving image shooting function captures a scene (a turning scene) in which a person turning his back to the photographer turns to the photographer, and includes each turning scene. It is assumed that three compressed moving image data are converted into data files and recorded in advance on the recording medium 19. In the present embodiment, the imaging device 1 set to the scene selection function detects three turning scenes from the three compressed moving image data, and evaluates each scene with an evaluation index to obtain the highest evaluation scene. An example of selection will be described.

  In the present embodiment, the feature quantity parameter setting unit 26 indicates that the type of feature to be detected by the feature quantity detection unit 22 is human face detection, and the feature quantity is a determination value (face value) indicating whether or not a face has been detected. Detection value) is set in advance. In the evaluation parameter setting unit 27, the evaluation index to be evaluated by the transition evaluation unit 24 is set in advance as the front orientation degree, the eye center position degree from the image center, the blink detection value, and the smile degree. Yes. Furthermore, weighting is set in the evaluation parameter setting unit 27 for each evaluation index. For example, the order from the higher priority to the lower priority is matched with the order from the higher weighting to the lower priority. Specifically, when the descending order of priority is the blink detection value, the smile degree, the front direction degree, and the eye center position degree from the center of the image, the descending order of the weights is set to be the same order.

  FIG. 3 is a flowchart showing processing of the scene selection function in the imaging apparatus 1. In step S <b> 301, the control unit 14 selects three data files of compressed moving image data recorded on the recording medium 19 in accordance with the operation of the operation unit 13 by the operator of the imaging apparatus 1. The data file can be selected by the operator, for example, by operating the operation menu. That is, the control unit 14 causes the display unit 16 to display the operation menu when the operator presses the menu button of the operation unit 13. This operation menu has, for example, a hierarchical menu structure. A file selection menu is formed in any lower hierarchy, and the file name of the data file recorded on the recording medium 19 is displayed there. In accordance with the operation of the selection button on the operation unit 13 by the operator, the control unit 14 causes the display unit 16 to display a file selection menu. Then, the operator is made to select a desired data file.

  Next, in step S302, the scene selection processing unit 20 analyzes each of the three selected data files, ranks similar scenes based on objective evaluation criteria, and selects one scene with the highest rank. To obtain position information of the scene. This processing step will be described later in detail. Next, in step S303, the control unit 14 extracts moving image data corresponding to the scene from the data file recorded on the recording medium 19 based on the position information of the scene acquired in the process of step S302. Next, in step S304, the control unit 14 records the extracted moving image data on the recording medium 19. At this time, the control unit 14 may delete moving image data corresponding to two scenes not selected from three similar scenes, or a data file including these.

  Next, details of the processing in step S302 will be described. FIG. 4 is a flowchart of the event detection process executed by the scene selection processing unit 20. In step S401, the control unit 14 reads the compressed moving image data from the recording medium 19 in the order selected by the operator for the three data files selected in the process of step S301, and performs the decompression process in the image compression / decompression unit 21. Then, the data decompressed frame image is input to the feature amount detection unit 22. If no frame image is input in the process of step S401 (S401: NO), the process of this flowchart ends.

  On the other hand, when a frame image is input in the process of step S401 (S401: YES), in step S402, the feature amount detection unit 22 sets the feature type (person's face) set in the feature amount parameter setting unit 26. Detection) and feature amount detection processing based on the feature amount (face detection value). The face of a person as an object is detected, for example, by performing image recognition processing for recognizing the presence or absence of a person's eyes in a frame image. The face detection value, which is a feature amount, is expressed as a binary value indicating the presence or absence of a human face.

  FIG. 5 is a graph of face detection values by the feature amount detection unit 22 when frame images are sequentially input. According to the figure, the human eyes are not shown in the frame images sequentially input from the frame number # 1, or the face detection value is 0 level because the feature amount detection unit 22 does not detect the eyes. Subsequently, in the frame image of frame number #K, the feature amount detection unit 22 detects the eyes, so that the face detection value becomes a constant level value, indicating that a human face has been detected in this frame image. In the figure, it is shown that the feature quantity detection unit 22 continues to detect a person's face in frame images after frame number #K.

  Returning to the description of FIG. 4, in step S <b> 403, the feature quantity detection unit 22 stores the detected feature quantity in the storage unit 15 in association with the frame position. The frame position is a frame image number or a time stamp. Next, in step S404, the event detection unit 23 calculates a difference value between the latest feature amount stored in the storage unit 15 in the process of step S403 and the feature amount of the frame image one frame before, and the feature It is determined whether or not an event that is a change point is detected. For example, it is determined that an event has been detected when the difference value exceeds a predetermined reference value. In the example of the graph of FIG. 5, it is determined that an event has been detected in the frame image of frame number #K. That is, the frame image of frame number #K becomes a noticeable frame image.

  Next, when an event is detected (step S405: YES), in step S406, the event detection unit 23 stores the frame position of the frame image in which the event is detected in the storage unit 15. On the other hand, when an event is not detected (step S405: NO), the process returns to the process of step S401 and proceeds to the process of the next frame image.

  In this manner, the frame position of the frame image (the image with the saliency) whose face is detected in the person's turning scene is stored in the storage unit 15 from each of the three data files by the event detection process. The event detection unit 23 excludes the event determination for the first frame because the first frame image of each data file does not have a previous frame image and thus an event is apparently detected. Make sure you do.

  Next, FIG. 6 is a flowchart of the transition evaluation process executed by the scene selection processing unit 20. After the process by the event detection unit 23, the process of this flowchart is started. In step S <b> 601, the control unit 14 refers to the storage unit 15. When the frame position of the frame image in which the event is detected is read (S601: YES), the control unit 14 in step S602, the frame image corresponding to the read frame position, and a frame for a certain period following the frame image. The compressed moving image data corresponding to the image is read from the recording medium 19 and the image compression / decompression unit 21 performs decompression processing. Then, the control unit 14 sequentially supplies the data decompressed frame images to the transition evaluation unit 24. Then, the transition evaluation unit 24 performs transition evaluation.

  This transition evaluation is performed for a certain period of time for each of the four evaluation indexes set in the evaluation parameter setting unit 27, that is, the degree of front orientation, the degree of center position of the eye from the center of the image, the blink detection value, and the degree of smile. This is done for frame images. FIG. 7 to FIG. 10 show evaluation graphs for each evaluation index of the transition evaluation unit 24 when frame images for a certain period after the frame number #K in which an event is detected are sequentially input.

  FIG. 7 is a graph of the evaluation value of the degree of front orientation when frame images for a certain period after frame number #K are sequentially input. The front direction degree is an evaluation value of the degree of orientation of the person who is the subject with respect to the lens of the optical system 11 of the imaging apparatus 1, more specifically the photographer side. In other words, when the person's face is directly facing the lens, the front is set and the evaluation value is maximized. The degree of front orientation calculates an evaluation value based on the distance between both eyes of a person. That is, the evaluation value is calculated based on the fact that the distance between both eyes is small in the state where the person faces sideways, and the distance between both eyes increases as it goes in the front direction. This figure shows a state in which the evaluation value of the degree of front direction increases as the person turns after the frame image of frame number #K.

  FIG. 8 is a graph of the evaluation value of the degree of the center position when frame images for a certain period after frame number #K are sequentially input. The center position degree is an evaluation value for the position of a person's face with reference to the center position of the frame image. For the degree of the center position, the distance from the center position of the frame image to each eye of the person is calculated so that the smaller the distance, the higher the evaluation value. This figure shows a state in which a person is shown almost stably in the vicinity of the center of the frame after the frame image of frame number #K.

  FIG. 9 is a graph of the evaluation value of the blink detection value when frame images for a certain period after frame number #K are sequentially input. The blinking detection value is an evaluation value of presence / absence of blinking of a person. This evaluation value is binary, and it is 0 when it is detected that the eyes are open or when it is detected that the eyes are closed, and a constant level value when it is detected that the eyes are open. It becomes. In the figure, since the transition evaluation unit 24 could not detect the presence or absence of blinking for a short time after the frame image of frame number #K, it is 0 level, and thereafter it is detected that the eyes are open. Therefore, it shows a state where the level value is constant.

  FIG. 10 is a graph of the evaluation value of the smile level when frame images for a certain period after frame number #K are sequentially input. Smiles are estimated by analyzing data of these positions with respect to a person's eyes, eyebrows, mouth shape, and face outline, and performing comparison recognition processing with a smile pattern. The smile level is set to a higher evaluation value as the probability that the smile is estimated is higher. The figure shows that the degree of smile is higher at the back than when the face was first detected.

  Returning to the description of step S602 in FIG. 6, the transition evaluation unit 24 calculates, for each frame image, the front orientation degree, the eye center position degree from the image center, the blink detection value, and the smile degree evaluation value. Each evaluation value is weighted and added according to the weighting set in the evaluation parameter setting unit 27. Next, in step S603, the transition evaluation unit 24 stores the calculated evaluation value in the storage unit 15 in association with the frame position of the frame image in which the event is detected.

  On the other hand, in step S601, when the control unit 14 does not read the frame position of the frame image in which the event is detected, that is, when there is no event detection frame image to be referred to (S601: NO), in step S604. The selection unit 25 selects the frame position of the event detection frame image corresponding to the highest evaluation value among the evaluation values for each event stored in the storage unit 15. Next, in step S605, the selection unit 25 outputs the position information of the scene including the frame image at the detected frame position. For example, a start position a predetermined number of frames before the detected frame position and an end position a predetermined number of frames after the detected frame position are output as position information.

  FIG. 11 shows that the photographing apparatus 1 set to the scene selection function detects an event from each of three data files, performs transition evaluation, extracts three similar scenes, and selects one scene from them. It is the figure which represented the mode that it extracted. As shown in the figure, the imaging apparatus 1 according to the present embodiment analyzes the three data files by the event detection process described above, and detects an event from the frame image of each moving image data (event detection frame image f1). , F2, f3). Next, transition evaluation is performed on the event detection frame images f1, f2, and f3 and the frame images tf1, tf2, and tf3 for a certain period of time following the event detection frame images by the transition evaluation process. Then, a scene (start frame pf, end frame af) including the event detection frame image f2 is selected for the moving image data 2 that has the highest evaluation value.

  As described above, according to the present embodiment, three turning scenes can be detected from the three data files, and each scene can be evaluated and ranked according to an objective evaluation index. Furthermore, the scene with the highest evaluation can be selected from among them. Also, by evaluating by weighting, a scene preferable for the photographer, for example, a scene with no eyes blinking, smiling, facing the photographer and having a face in the center of the screen can be selected.

[Second Embodiment]
An example in which the image processing apparatus according to the second embodiment of the present invention is applied to an imaging apparatus will be described. Also in this embodiment, the scene selection function will be mainly described. The imaging apparatus according to the present embodiment calculates an event that is a feature change point by calculating a difference in feature amount between consecutive frame images for one or a plurality of continuous shot image data groups recorded by continuous shooting. To detect. If a plurality of events are detected as a result of the search, it is estimated that the frame images from which the events are detected are prominent images in similar scenes. Then, the imaging apparatus evaluates an image based on a predetermined evaluation index for each frame image in which an event is detected and a frame image that continues in time series, and selects a scene with the highest evaluation among similar scenes. . With this scene selection function, the imaging apparatus according to the present embodiment ranks scenes based on objective evaluation criteria when similar scenes are included in the recorded continuous shot image data group. And a scene of a desired order can be extracted. Hereinafter, an imaging apparatus having such characteristics will be described in detail.

  Since the imaging device in the present embodiment has the same configuration as that of the imaging device 1 according to the first embodiment described above, the same reference numerals are given and description of the configuration is omitted.

  Next, the operation in the scene selection function of the imaging apparatus 1 will be described. In the present embodiment, a compressed continuous shot image obtained by continuously shooting a person on a single-seat swing twice in a direction substantially perpendicular to the swinging direction by the imaging device 1 set to the continuous shooting function. It is assumed that data groups (two groups) are recorded on the recording medium 19 in advance. FIG. 12 is a diagram schematically showing one frame image in a continuously shot captured image data group. In the figure, one frame of a person moving on a swing and drawing an arc is shown.

  In the following description, as shown in the figure, the vertical direction is the X direction and the horizontal direction is the Y direction with the lower left corner of the frame image as the origin. When such a two-dimensional coordinate system is defined, the person performs a simple vibration motion in the X direction. Therefore, in the present embodiment, a frame image in a place where the position (for example, the center of gravity position) of a predetermined portion of the person in the X direction is the lowest is detected from the compressed continuous shot image data group, and the frame image group after the frame image is detected. An example in which the scene with the highest evaluation is selected by evaluating the evaluation index will be described.

  In the present embodiment, the feature amount parameter setting unit 26 indicates that the feature type to be detected by the feature amount detection unit 22 is a single vibration in the X direction in FIG. 12, and the feature amount is a coordinate value in the X direction of the single vibration. Is set in advance. In the evaluation parameter setting unit 27, it is preset that the evaluation index evaluated by the transition evaluation unit 24 is a difference (range) between the highest point position and the lowest point position of simple vibration.

  FIG. 13 is a flowchart illustrating processing of the scene selection function in the imaging apparatus 1. In step S <b> 131, the control unit 14 selects a data file group of two compressed continuous-shot image data groups recorded on the recording medium 19 in accordance with the operation of the operation unit 13 by the operator of the imaging apparatus 1. The selection of the data file group by the operator can be performed by operating the operation menu, for example. That is, the control unit 14 causes the display unit 16 to display the operation menu when the operator presses the menu button of the operation unit 13. This operation menu has, for example, a hierarchical menu structure. A file group selection menu is formed in any lower hierarchy, and the group names of the data file groups recorded on the recording medium 19 are displayed there. The control unit 14 causes the display unit 16 to display a file group selection menu according to the operation of the selection button on the operation unit 13 by the operator. Then, the operator is made to select a desired data file group.

  Next, in step S132, the scene selection processing unit 20 analyzes each of the two selected data file groups, ranks similar scenes based on objective evaluation criteria, and selects the one with the highest rank. A scene is selected and position information of the scene is acquired. This processing step will be described later in detail. Next, in step S133, the control unit 14 extracts a continuous shot image data group corresponding to the scene from the data file group recorded on the recording medium 19 based on the position information of the scene acquired in the process of step S132. . Next, in step S134, the control unit 14 records the extracted continuous-shot image data group on the recording medium 19. At this time, the control unit 14 may delete the continuous shot image data group corresponding to the unselected scene of the two similar scenes, or the data file group including these.

  Next, details of the processing in step S132 will be described. In this description, the flowchart of the event detection process and the flowchart of the transition evaluation process will be described with reference to FIGS. 4 and 6 used for the description of both processes in the first embodiment described above. Use the same number. In step S401, the control unit 14 reads out the compressed continuous-shot image data group from the recording medium 19 in the order selected by the operator for the two data file groups selected in the process of step S131, and the image compression / decompression unit 21 performs the processing. The decompression process is performed, and the frame image obtained by decompressing the data is input to the feature amount detection unit 22. If no frame image is input in the process of step S401 (S401: NO), the process of this flowchart ends.

  On the other hand, when a frame image is input in the process of step S401 (S401: YES), in step S402, the feature amount detection unit 22 sets the feature type (in the X direction) set in the feature amount parameter setting unit 26. A feature amount detection process is performed based on the simple vibration) and the feature amount (the coordinate value in the X direction). The target for obtaining the coordinate value may be the center position of a part such as a person's face or waist, or the center of gravity of the person may be recognized as an image.

  Next, in step S403, the feature amount detection unit 22 stores the detected feature amount in the storage unit 15 in association with the frame position. The frame position is a frame image number or a time stamp. Next, in step S404, the event detection unit 23 calculates the latest feature value stored in the storage unit 15 in the process of step S403, the feature value for the frame image one frame before, and the frame image two frames before. Based on the feature amount, it is determined whether or not an event that is a feature change point is detected. For example, it is determined that an event has been detected when a coordinate value Xi that satisfies Formula (1) is detected.

  Next, when an event is detected (step S405: YES), in step S406, the event detection unit 23 stores the frame position of the frame image in which the event is detected in the storage unit 15. On the other hand, when an event is not detected (step S405: NO), the process returns to the process of step S401 and proceeds to the process of the next frame image.

  In this way, the frame position of the frame image (significant image) positioned at the lowest point in the single vibration motion of the person in the X direction is stored in the storage unit 15 from each of the two data file groups by the event detection process. Is remembered. Note that the first and second frame images of each data file group cannot be calculated using Equation (1) because there are no past frame images. Therefore, the event detection unit 23 excludes the event determination for the first and second frames.

  Next, FIG. 6 is a flowchart of the transition evaluation process executed by the scene selection processing unit 20. After the process by the event detection unit 23, the process of this flowchart is started. In step S <b> 601, the control unit 14 refers to the storage unit 15. When the frame position of the frame image in which the event is detected is read (S601: YES), the control unit 14 in step S602, the frame image corresponding to the read frame position, and a frame for a certain period following the frame image. The compressed moving image data corresponding to the image is read from the recording medium 19 and the image compression / decompression unit 21 performs decompression processing. Then, the control unit 14 sequentially supplies the data decompressed frame images to the transition evaluation unit 24. Then, the transition evaluation unit 24 performs transition evaluation.

  This transition evaluation is performed for a certain period of time with respect to a range Rx that is set in the evaluation parameter setting unit 27 and is represented by the difference between the highest point position Xmax and the lowest point position Xmin of simple vibration in the X direction that is an evaluation index. This is done for frame images. That is, the evaluation value is calculated by the equation (2).

  Next, in step S603, the transition evaluation unit 24 stores the calculated evaluation value range Rx in the storage unit 15 in association with the frame position of the frame image in which the event is detected.

  On the other hand, in step S601, when the control unit 14 does not read the frame position of the frame image in which the event is detected, that is, when there is no event detection frame image to be referred to (S601: NO), in step S604. The selection unit 25 selects the frame position of the event detection frame image corresponding to the highest evaluation value among the evaluation values for each event stored in the storage unit 15. Next, in step S605, the selection unit 25 outputs the position information of the scene including the frame image at the detected frame position. For example, a start position and an end position that cover the time of a half cycle in the Y direction including the detected frame position are output as position information.

  FIG. 14 is a graph showing the transition of the coordinate value in the X direction of the center of gravity position of the person in each frame image for each of the two data file groups (groups 1 and 2). FIG. 4A is a graph for group 1, and FIG. 4B is a graph for group 2. FIG. As is apparent from the graph of FIG. 8, an event is detected in the frame image of frame number # 3 in both groups. It can also be seen that group 2 has a wider range than group 1. Therefore, it can be evaluated that the group 2 continuous shooting is capturing a dynamic image than the group 1 continuous shooting.

  As described above, according to the present embodiment, two scenes where the center of gravity of the person is at the lowest point are detected from two data file groups, and each scene is evaluated using an objective evaluation index. Can be ranked. Therefore, the scene with the highest evaluation can be selected from among them. This makes it possible to select a scene that is preferable for the photographer, for example, a scene that moves the swing most dynamically and dynamically.

  In the first and second embodiments described above, the type of feature quantity and the combination of the feature quantity and the evaluation index for transition evaluation may be stored in association with each other. At that time, it is preferable to rank the combinations. For example, if combinations that are frequently used by the operator or combinations that are determined to be preferable by the operator are stored in a higher order, the combinations can be displayed on the operation menu according to the order.

  In the first and second embodiments described above, an example in which one event is included in one moving image data file or one continuous shooting group has been described. Other than this, there are multiple similar scenes in one moving image data file or one group, ranking the evaluation of those scenes, or selecting the scene with the highest evaluation value You can also.

  Moreover, you may make it implement | achieve a part of image processing apparatus which is each embodiment mentioned above, for example, the function of a control part with a computer. In this case, the image processing program for realizing the control function is recorded on a computer-readable recording medium, and the image processing program recorded on the recording medium is read by the computer system and executed. Also good. Here, the “computer system” includes an OS (Operating System) and hardware of peripheral devices. The “computer-readable recording medium” refers to a portable recording medium such as a flexible disk, a magneto-optical disk, an optical disk, and a memory card, and a storage device such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case may be included and a program that holds a program for a certain period of time may be included. Further, the above program may be for realizing a part of the functions described above, or may be realized by a combination with the program already recorded in the computer system. .

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

22 feature amount detection unit 23 event detection unit 24 transition evaluation unit 25 selection unit

Claims (5)

A feature amount detecting means for detecting a feature amount from each frame image of the continuous shot image data group or moving image data;
Event detecting means for detecting an event based on a change in the feature amount accompanying a time change;
An evaluation means for calculating an evaluation value based on a predetermined evaluation index for an event detection frame image that is a frame image corresponding to the detected event and a frame image group for a predetermined time following the time series;
Selection means for selecting a scene by a frame image group including an event detection frame image based on the evaluation value;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The image processing apparatus, wherein the evaluation unit detects a feature amount of a type different from the feature amount in the feature amount detection unit from each frame image and calculates the evaluation value. The image processing apparatus according to claim 1.
The evaluation means detects a feature quantity of the same type as the feature quantity in the feature quantity detection means from each frame image, and calculates the evaluation value based on the detected feature quantity . The image processing apparatus according to claim 1.
The image processing apparatus, wherein the evaluation unit calculates an evaluation value by weighting the predetermined evaluation index. On the computer,
A feature amount detection step of detecting a feature amount from each frame image of the continuous shot image data group or the moving image data; and
An event detection step of detecting an event based on a change in the feature amount accompanying a time change;
An evaluation value calculating step for calculating an evaluation value based on a predetermined evaluation index for an event detection frame image that is a frame image corresponding to the detected event and a frame image group for a predetermined time that follows the event detection frame image;
A selection step of selecting a scene by a frame image group including an event detection frame image based on the evaluation value;
An image processing program for executing

Images