JP2014027372A - Image processor, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To specify and extract a moving image in a desired period even if a special device is not used.SOLUTION: An image processor 1 includes: an image acquiring section 42; an analysis section; a cueing image specifying section 43; and a reproduced image extracting section 44. The image acquiring section 42 acquires a moving image where a subject for performing a series of operations is captured. The analysis section analyzes a series of operations in the moving image acquired by the image acquiring section 42. The cueing image specifying section 43 specifies a cueing image in response to an analysis result. The reproduced image extracting section 44 extracts a specified image from the moving image in response to the specified cueing image.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

2. Description of the Related Art Conventionally, an apparatus is known that extracts moving image data for a part of a period including an impact time from moving image data obtained by shooting a subject performing a series of swing operations (see Patent Document 1).
Specifically, the apparatus measures the ball speed by a Doppler sensor while storing the moving image data in the ring buffer in an endless manner, and based on the processing time taken to measure the ball speed, the frame image at the time of impact Are extracted as moving image data including a subject performing a series of swing motions.

JP 2011-30669 A

  However, in the apparatus described in Patent Document 1, a Doppler sensor is used to extract moving image data for a certain period such as a moving image including a subject performing a series of swing operations from moving image data. A special device such as was required.

  The present invention has been made in view of such a situation, and an object thereof is to extract a desired image without using a special device.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention includes:
Acquisition means for acquiring a moving image in which a subject performing a series of operations is captured;
Analyzing means for analyzing the series of operations in the moving image acquired by the acquiring means;
A specifying unit for specifying a predetermined unit image based on an analysis result by the analyzing unit;
Extracting means for extracting a specific image from the moving image based on the predetermined unit image specified by the specifying means.

  According to the present invention, a desired image can be extracted without using a special device.

It is a block diagram which shows the structure of the hardware of the image processing apparatus which concerns on 1st Embodiment and 2nd Embodiment of this invention. FIG. 2 is a functional block diagram illustrating a functional configuration for executing a cue reproduction process among the functional configurations of the image processing apparatus of FIG. 1. It is a figure which shows the specific example which divides a captured image into a some image area. It is the figure which showed transition of a head image when a shot is made in a series of operations of a golf swing. It is the figure which showed a series of captured images in the golf swing operation | movement of a to-be-photographed object. 3 is a flowchart illustrating an example of a flow of cueing reproduction processing executed by an image processing apparatus having the functional configuration of FIG. 2. It is a functional block diagram which shows the functional structure for performing the cue selection process among the functional structures of the image processing apparatus shown in FIG. It is a figure which shows the specific example of a mask image. It is a figure which shows the difference capacity | capacitance between each captured image, and the change rate of difference capacity | capacitance. FIG. 10 is a flowchart showing an example of the cue selection process executed by the image processing apparatus having the functional configuration shown in FIG. It is a flowchart which shows the imaging | photography process of step S21 of the cue selection process of FIG. It is a flowchart which shows the moving image reproduction process of step S22 of the cue selection process of FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
Hereinafter, in the description of the first embodiment, the present embodiment refers to the first embodiment.
[Constitution]
FIG. 1 is a block diagram showing a hardware configuration of the image processing apparatus according to the first embodiment of the present invention.
The image processing apparatus 1 is configured as a digital camera, for example.

  The image processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an imaging unit 16, and an input unit. 17, an output unit 18, a storage unit 19, a communication unit 20, and a drive 21.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. An imaging unit 16, an input unit 17, an output unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 15.

  Although not shown, the imaging unit 16 includes an optical lens unit and an image sensor.

The optical lens unit is configured by a lens that collects light, for example, a focus lens or a zoom lens, in order to photograph a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. The zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

The image sensor includes a photoelectric conversion element, AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.

The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. Through various signal processing, a digital signal is generated and output as an output signal of the imaging unit 16.
Hereinafter, the output signal of the imaging unit 16 is referred to as “captured image data”. Data of the captured image is appropriately supplied to the CPU 11, RAM 13, and the like.

The input unit 17 includes various buttons and the like, and inputs various types of information according to user instruction operations.
The output unit 18 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 19 is composed of a hard disk or a DRAM (Dynamic Random Access Memory) or the like, and stores various image data.
The communication unit 20 controls communication performed with other devices (not shown) via a network including the Internet.

  A removable medium 31 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 31 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19.

FIG. 2 is a functional block diagram showing a functional configuration for executing the cue reproduction processing among the functional configurations of such an image processing apparatus 1.
Cue playback processing specifies and specifies a captured image (hereinafter referred to as “cue image”) at the time of starting playback from a plurality of captured images obtained by capturing a subject performing a series of operations. This is a process of starting reproduction from the cued image and sequentially reproducing the subsequent captured images.
Here, the target of the cue reproduction processing includes a plurality of still images obtained by continuous shooting, but in the present embodiment, it is assumed to be a moving image. Therefore, the above-mentioned “plurality of captured images” means “a plurality of unit images (frame images and field images) constituting a moving image” hereinafter. In other words, hereinafter, unless otherwise specified, the “captured image” means a “unit image”.
Specifically, the process of identifying the captured image including the subject at the time of starting the motion as a cue image and starting the reproduction from the identified cue image in a series of motions of the subject is a cue reproduction. It is processing.
Here, an example of the golf swing of the player imaged by the image processing apparatus 1 will be described. A golf swing is one of a series of movements of a golfer who sets a club, starts swinging, swings the club head up to the top position, and then swings the head down. The operation until the ball is shot.
If the time from the start of imaging to the start of the golf swing is long, a user who wants to view the moving image after the swing takes time and effort to fast-forward the moving image until the start of the swing. Therefore, the image processing apparatus 1 according to the present embodiment specifies the captured image at the start of the swing as a cue image, and starts reproduction from the specified cue image.

  When the cue reproduction processing is executed, as shown in FIG. 2, in the CPU 11, the imaging control unit 41, the image acquisition unit 42, the cue image specifying unit 43, the reproduction image extraction unit 44, and the reproduction are performed. The unit 45 functions. In the cueing image specifying unit 43, an image dividing unit 51 and a passage determining unit 52 function.

When the cue reproduction process is executed, an acquired image storage unit 61 provided as an area of the storage unit 19 is used.
In the acquired image storage unit 61, data of moving images (a plurality of captured images) output from the imaging unit 16 and acquired by the image acquisition unit 42 is stored.

The imaging control unit 41 sets various imaging conditions and controls the imaging operation in the imaging unit 16.
In this embodiment, in order to image a subject that performs a series of operations, the imaging control unit 41 causes the imaging unit 16 to continuously image the subject and output it as moving image data.

  The image acquisition unit 42 acquires moving image data output from the imaging unit 16. Thereafter, the image acquisition unit 42 encodes moving image data and stores the encoded data in the acquired image storage unit 61.

The cue image specifying unit 43 specifies a cue image based on the moving image data stored in the acquired image storage unit 61. That is, the cueing image specifying unit 43 sequentially decodes the moving image data stored in the acquired image storage unit 61 in units of captured images (unit images), and the decoded series of captured image data in the RAM 13. The cue image is specified from the series of captured images.
A specific method for specifying the cue image will be described below.

  The image dividing unit 51 divides the captured image into a plurality of image areas 71 to 78 for each of the series of captured image data. The division method is not particularly limited, but in this embodiment, as shown in FIG. 3, it is assumed that a method of dividing a unit image into 9 regions of 3 × 3 regions is adopted. In the following description, the center area in the lower stage is referred to as an image area 71, and image areas 72, 73,. In the present embodiment, these image areas 71 to 78 are image areas through which a golf club head image 81 to be described later may pass.

  In FIG. 3, the circled numbers drawn in each of the image areas 71 to 78 are not reflected but are symbols for visually distinguishing from other image areas. It is assumed that the relationship between each of the image regions 71 to 78 and each symbol (circle numeral) is common in the drawings after FIG.

In response to the image dividing unit 51 dividing each of the series of captured images into a plurality of image areas 71 to 78, the passage determining unit 52 determines that the head image 81 of the golf club is an image area 71, 72, 78. It is determined whether or not each has been passed in a specific order. Here, as the specific order, the order of the image regions 72, 71, 78 is determined in advance based on the actual movement trajectory of the golf club head. Further, the head image 81 refers to an image region indicating a golf club head in the captured image.
That is, the passage determination unit 52 determines whether or not the head image 81 has changed in a specific order within a specific range (image regions 71, 72, and 78), thereby performing a series of golf swing operations. It is then determined whether or not a shot has been made.

FIG. 4 is a diagram showing the transition of the head image 81 when a shot is made during a series of golf swing operations.
FIG. 4 shows the following state as a series of operations of golf. That is, first, the golf club is swung down from the top position, and the head image 81 passes through the image area 72 (see the left side of FIG. 4). Next, the head image 81 passes through the image area 71 and a shot is made, and the ball image 82 moves in the shot direction (see the center diagram in FIG. 4). Then, the head image 81 passes through the image area 78 (see the right side of FIG. 4).
The fact that the head image 81 has passed in the order of the image areas 72, 71, 78 indicates that a shot has been performed in a series of golf swing operations.
The passage determining unit 52 uses a known method such as template matching or moving object detection when determining that the passage has been made in the order.

When the head image 81 passes through the image areas 71, 72, and 78 in the order of the image areas 72, 71, and 78, the cueing image specifying unit 43 determines that the moving image Identify the cue image from the inside.
FIG. 5 is a diagram showing an example of a series of captured images showing a subject performing a golf swing motion.
According to FIG. 5, the series of captured images is composed of 16 consecutive captured images of a first captured image, a second captured image,... Note that the number of the series of captured images is 16 in this embodiment for convenience of explanation, but is not particularly limited to this, and in fact, there are often many more.
Further, the item “buffer” shown in FIG. 5 indicates, for each of the series of captured images, an image area to which the head image 81 of the golf club belongs and a storage address of the captured image in the acquired image storage unit 61. It shows that the information is associated and stored in the buffer area of the RAM 13. For example, for the first captured image, the golf club head image 81 belongs to the image area 71, and information indicating that the data of the first captured image is stored at the first address is stored in the buffer area of the RAM 13. Has been.

Hereinafter, a method for specifying a cueing image will be described.
When the head image 81 passes through the image areas 71, 72, and 78 in a specific order (the order of the image areas 72, 71, and 78), the cue image specifying unit is determined according to the determination by the passage determining unit 52. 43, in a series of captured images of a subject that has performed a golf swing motion, a group of captured images before passing in the order of the image areas 72, 71, and 78 are searched in a direction opposite to the reproduction direction, Specify the source image.
Specifically, the cue image specifying unit 43 searches the first captured image to the eleventh captured image from the eleventh captured image in the direction opposite to the reproduction direction, and the head image 81 first enters the image area 71. The third captured image to which it belongs is specified as a cue image.

  In response to the cueing image specifying unit 43 specifying the cueing image, the reproduction image extracting unit 44 sequentially extracts each data of a moving image for reproduction (a series of captured images) from the acquired image storage unit 61. . Specifically, the reproduction image extraction unit 44 sequentially extracts each data from the third captured image that is a cue image to the last sixteenth captured image from the acquired image storage unit 61 as reproduction image data.

In response to the reproduction image extraction unit 44 obtaining the reproduction image data, the reproduction unit 45 causes the output unit 18 to output the extracted reproduction image data.
Therefore, even if the moving image is not fast-forwarded until the swing start time, the captured image at the swing start time is automatically cued, and a series of subsequent captured images can be reproduced.

[Cue playback processing]
Next, a cue reproduction process executed by the image processing apparatus 1 will be described.
FIG. 6 is a flowchart showing an example of a cueing reproduction process executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.

  In FIG. 6, when the cue reproduction process is started when the input unit 17 receives an input of an instruction operation for executing the cue reproduction process, the following processes of steps S1 to S8 are executed. Is done.

In step S <b> 1, the cue image specifying unit 43 acquires the data of the first captured image (hereinafter referred to as “first image”) from the moving image data from the acquired image storage unit 61 and decodes it.
In step S <b> 2, the cue image specifying unit 43 stores the decoded first image data in the buffer area of the RAM 13. Note that step S2 is also executed after the decoding process in step S6 described later.

  In step S <b> 3, the image dividing unit 51 divides the captured image data stored in the buffer area of the RAM 13 by the cueing image specifying unit 43 into a plurality of image areas (here, 3 × 3 image areas).

  In step S <b> 4, the cue image specifying unit 43 sets the image area to which the head image 81 belongs and the storage address of the captured image in the acquired image storage unit 61 for the captured image data divided into a plurality of image areas. Is stored in the buffer area of the RAM 13 in association with each other. For example, in the example of FIG. 5, when the first captured image is a processing target, the cue image specifying unit 43 stores information indicating the image area 71 to which the head image 81 of the golf club belongs and the first captured image. The information indicating the first address is associated with and stored in the buffer area of the RAM 13.

In step S5, the passage determination unit 52 determines whether or not the head image 81 has passed through a specific image region group in a specific order.
Here, in this example, the specific image area group is the image areas 71, 72, 78, and the specific order is the order of the image areas 72, 71, 78. If it does not pass in the specific order, it is determined as NO in step S5, and the process proceeds to step S6.

In step S <b> 6, the cue image specifying unit 43 determines the captured image that is the determination target in the previous step S <b> 5 for the moving image data stored in the acquired image storage unit 61 based on a preset thinning rate. After thinning out the data of several captured images after the next, the data of the next captured image is acquired from the acquired image storage unit 61 and decoded. Note that the thinning rate is set at such a ratio that the passage determination unit 52 can detect that the head image 81 passes in the order of the image areas 72, 71, and 78.
When the process of step S6 ends, the process returns to step S2, and the subsequent processes are repeated. That is, the captured image data decoded in step S6 is stored in the buffer area of the RAM 13 (step S2), divided into a plurality of image areas (step S3), and the storage of the image area to which the club head image belongs and the captured image. Information indicating each of the addresses is stored (step S4), and it is determined whether or not the head image 81 has passed a specific image region group in a specific order (step S5).
Such a loop process of steps S2 to S5 is performed for each captured image constituting the thinned moving image every time it is determined that the head image 81 does not pass the specific image region group in the specific order. It is repeated for each piece of data.
If it is determined in step S5 regarding the data of the predetermined captured image that the head image 81 has passed through the specific image region group in a specific order, the process proceeds to step S7.

  In step S <b> 7, the cue image specifying unit 43 specifies a cue image. Specifically, the cue image specifying unit 43 searches the data of the captured image group before passing in the specific order in the decoded series of captured image data in a direction opposite to the reproduction direction. Identify the cue image. For example, in the example of FIG. 5, as described above, the cue image specifying unit 43 searches each data of the first captured image to the 11th captured image from the 11th captured image in the direction opposite to the reproduction direction. Thus, the third captured image in which the head image 81 first belongs to the image area 71 is specified as the cue image.

In step S <b> 8, the reproduced image extraction unit 44 extracts a series of captured image data for reproduction from the acquired image storage unit 61. Further, the reproduction unit 45 causes the output unit 18 to output the extracted series of captured image data for reproduction. Specifically, the reproduction image extraction unit 44 sequentially extracts each piece of data of a series of captured images for reproduction from the acquired image storage unit 61 using a sequence from the cue image to the last captured image as a series of captured images for reproduction. To do. Further, the playback unit 45 causes the output unit 18 to output each piece of extracted series of captured images for playback.
Therefore, the user can cue from the captured image at the time of starting the swing and view a series of captured images thereafter without fast-forwarding the moving image to the image at the time of starting the swing.

The first embodiment of the present invention has been described above.
The image processing apparatus 1 that performs the above-described cue reproduction processing includes an image acquisition unit 42, a cue image specifying unit 43, and a reproduction image extraction unit 44.
The image acquisition unit 42 acquires a moving image in which a subject that performs a series of operations is captured.
The cue image specifying unit 43 analyzes a series of operations in the moving image acquired by the image acquisition unit 42, and specifies a cue image based on the analysis result.
The reproduction image extraction unit 44 extracts a specific image from the moving image based on the specified cue image.
Therefore, such an image processing apparatus 1 can identify and extract a moving image of a desired period without using a special device.

The cueing image specifying unit 43 includes an image dividing unit 51 and a passage determining unit 52.
The image dividing unit 51 divides each of the unit images constituting the moving image into a plurality of image areas 71 to 78.
The passage determining unit 52 determines whether or not the head image 81 has passed in a specific order in the image regions 71, 72, and 78 of the plurality of image regions 71 to 78 divided by the image dividing unit 51. .
When the passage determining unit 52 detects that the head image 81 has passed in the order of the image areas 72, 71, and 78, the cueing image specifying unit 43 detects a moving image (first image) before the detection is performed. To 11th image) are searched in the reverse playback order, and the cueing image is specified based on the image at the time when the head image 81 belongs to the image area 71.
Therefore, such an image processing apparatus 1 can specify the image at the start of the golf swing as a cue image without causing the user to fast-forward.

Further, when the passage determination unit 52 searches for moving images in the reproduction order, the unit images are thinned out from the moving images at a predetermined thinning rate, and the head images 81 are arranged in a specific order in the reproduction order of the thinned moving images. It is determined whether or not it has passed.
Therefore, such an image processing apparatus 1 can improve the processing efficiency when specifying a cueing image.

The image processing apparatus 1 further includes a playback unit 45.
Based on the unit image extracted by the reproduction image extraction unit 44, the reproduction unit 45 reproduces data of the captured image group for a predetermined period.
Therefore, such an image processing apparatus 1 can reproduce a moving image of a desired period without using a special device.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
Hereinafter, in the description of the second embodiment, the present embodiment refers to the second embodiment.

  The image processing apparatus 1 according to the second embodiment of the present invention has the same hardware configuration as that of the first embodiment. That is, FIG. 1 also shows a hardware configuration of the image processing apparatus 1 according to the second embodiment. Since the hardware configuration in FIG. 1 has already been described, the description thereof is omitted here.

FIG. 7 is a functional block diagram showing a functional configuration for executing the cue selection process among the functional configurations of the image processing apparatus 1 shown in FIG.
Cue selection processing specifies multiple picked-up images (hereinafter referred to as “cue candidate images”) that are candidates for a cue image from a plurality of picked-up images of a subject that performs a series of operations. Then, it refers to a process of starting reproduction from a cue image using a cue image selected by the user from among a plurality of identified picked-up images as a cue image.
In the present embodiment, an example of a golf swing of a subject imaged by the image processing apparatus 1 will be described as in the first embodiment.
When the time from the start of imaging to the start of the golf swing is long, a user who wants to view the moving image after the swing has time and effort to fast-forward the moving image until the start of the swing. Therefore, the image processing apparatus 1 according to the present embodiment specifies a plurality of captured images for several seconds before and after the start of the swing as cue candidate images, and selects a captured image desired by the user by a user's selection operation. It is specified as a cue image, and reproduction is started from the specified cue image.

  When the cue selection process is executed, as shown in FIG. 7, in the CPU 11, the image pickup control unit 41, the image acquisition unit 42, the cue image specifying unit 43, the reproduction image extraction unit 44, and the reproduction The unit 45 functions. In the image acquisition unit 42, the mask processing unit 91 and the difference calculation unit 92 function, and in the cue image specifying unit 43, the change rate calculation unit 93 functions.

When the cue selection process is executed, an acquired image storage unit 61 and a mask image storage unit 62 provided as an area of the storage unit 19 are used.
In the acquired image storage unit 61, data of moving images (a plurality of captured images) output from the imaging unit 16 and acquired by the image acquisition unit 42 is stored.
The mask image storage unit 62 stores a mask image in which the area outside the locus of the head image 81 is a mask area 83 as shown in FIG. The mask image is superimposed on the captured image 84 by the mask processing unit 91, so that the captured image 84 is subjected to mask processing.
The mask image is generated based on the trajectory of the head image 81 calculated based on the data of a series of golf swing images stored in advance in the mask image storage unit 62.
Specifically, the outer image area through which the head image 81 does not pass in the reproduced moving image is the image area to be masked.

The imaging control unit 41 sets various imaging conditions and controls the imaging operation in the imaging unit 16.
In this embodiment, in order to image a subject that performs a series of operations, the imaging control unit 41 causes the imaging unit 16 to continuously image the subject and output it as moving image data.

The image acquisition unit 42 acquires moving image data output from the imaging unit 16. Thereafter, the image acquisition unit 42 encodes moving image data and stores the encoded data in the acquired image storage unit 61.
When the image acquisition unit 42 acquires the data of the moving image, the mask processing unit 91 performs a mask process on each of a series of captured images constituting the moving image. Specifically, the mask processing unit 91 performs a process of decoding the mask image stored in the mask image storage unit 62 together with the series of captured images and displaying the decoded images superimposed on each of the series of captured images. Apply. By performing the mask process on the captured image, the pixel value of the masked area is always constant.
Every time the series of captured images is acquired, the difference calculation unit 92 calculates the difference capacity of the captured image data from the previously acquired captured image. Specifically, the six captured images illustrated in FIG. 9 will be described as an example. The difference calculation unit 92 calculates the difference capacity of the captured image data between adjacent captured images. When the difference capacity is calculated, the captured image is masked. In FIG. 9, the frame number represents the imaging order of the captured images, and is reproduced in the order of the frame number at the time of reproduction. In addition, the difference calculation unit 92 stores the calculated difference capacity in the buffer area of the RAM 13. In the present embodiment, the stored differential capacity is distributed into three levels of large, medium, and small according to a predetermined threshold.
The image acquisition unit 42 causes the acquired image storage unit 61 to store moving image data in accordance with the difference capacity calculated by the difference calculation unit 92. At this time, the storage address in the acquired image storage unit 61 of the series of captured image data constituting the stored moving image data is stored in the buffer area of the RAM 13.

In response to completion of imaging by the imaging unit 16 and storage of all moving image data in the acquired image storage unit 61, the change rate calculation unit 93 included in the cueing image specifying unit 43 is performed by the difference calculation unit 92. The change rate of the difference capacity between the calculated captured images is calculated. For example, as illustrated in FIG. 9, when the change in the difference image capacity is small, such as “small → small”, “large → large”, or the like, the change rate of the difference capacity is “ In the case of “small”, “small → medium”, “medium → large”, and “large → medium”, the change rate of the differential capacity is “large” or “medium”.
The cue image specifying unit 43 specifies a plurality of cue candidate images based on the calculated change rate. Specifically, when the difference capacity is set to “large” or “medium”, the cue image specifying unit 43 has a smaller frame number among the two captured images that are the basis for calculating the difference capacity. This captured image is specified as a cue candidate image. Referring to FIG. 9 as an example, when the difference capacity is “Large”, the captured image with the frame number “3” is specified as a cue candidate image, and when the difference capacity is “Medium”, the frame number is “ The captured image of “4” is specified as a cue candidate image. Therefore, two captured images with frame numbers “3” and “4” are specified as the cue candidate images. At this time, for the two captured images with the frame numbers “3” and “4”, the storage address is stored in the buffer area of the RAM 13 as the storage address of the cue candidate image.

When the cue candidate image is specified, the reproducing unit 45 acquires moving image data from the acquired image storage unit 61, and decodes and reproduces the acquired moving image data.
When playback of the moving image is started by the playback unit 45, the playback image extraction unit 44 causes the output unit 18 to display the specified cue candidate image.
When the identified cue candidate image is displayed on the output unit 18, the reproduction image extraction unit 44 receives an input of an instruction operation for selecting a cue image from the user via the input unit 17, and finds the cue. Identify the image. At this time, the storage address of the identified cue image in the acquired image storage unit 61 is stored in the buffer area of the RAM 13 as the cue image storage address.

  In response to the identification of the cue image, the reproduction image extraction unit 44 sequentially extracts data of a moving image (a series of captured images) for reproduction from the acquired image storage unit 61. Specifically, the reproduction image extraction unit 44 extracts each data from the specified cue image to the last captured image from the acquired image storage unit 61 as image data for reproduction.

In response to the reproduction image extraction unit 44 obtaining the reproduction image data, the reproduction unit 45 causes the output unit 18 to output the extracted reproduction image data.
Therefore, when the user selects a desired image from among the cueing candidate images as a cueing image, even if there is no fast-forwarding of the moving image up to the cueing image, the user can automatically perform several seconds before and after the swing start time. One of the captured images is cued, and a series of subsequent captured images can be reproduced.

[Cue selection process]
Next, a cue selection process executed by the image processing apparatus 1 will be described.
FIG. 10 is a flowchart showing an example of the flow of cue selection processing (main flow) executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.

In FIG. 10, when the cue selection process is started when the input unit 17 receives an input of an instruction operation for executing the cue selection process, the image acquisition unit 42 in FIG. In step S22, a moving image reproduction process described later with reference to FIG. 12 is started, and the cue selection process ends.
The imaging process is a process in which the cue image specifying unit 43 or the like specifies a cue candidate image from captured image data. The moving image reproduction process is a process in which the reproduction image extraction unit 44 or the like identifies a cue image and starts reproduction from the identified cue image.

[Shooting process]
Next, a photographing process executed by the image processing apparatus 1 will be described.
FIG. 11 is a flowchart illustrating an example of a flow of photographing processing executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.

  In FIG. 11, when the shooting process is started, in step S <b> 31, the image acquisition unit 42 acquires moving image data output from the imaging unit 16 and stores the acquired data in the acquired image storage unit 61. Each time this step S31 is executed, a series of captured image data constituting the moving image is acquired one by one and stored in the acquired image storage unit 61.

  In step S <b> 32, the image acquisition unit 42 determines whether shooting has ended, that is, whether acquisition of moving image data has ended. This determination is executed based on, for example, whether or not the image acquisition unit 42 has received an instruction to end shooting by the user via the input unit 17. Until the image acquisition unit 42 determines that the shooting has ended, NO is determined in step S32, and the process proceeds to step S33.

  In step S33, the mask processing unit 91 included in the image acquisition unit 42 performs mask processing on the captured image data acquired in step S31. Specifically, the mask processing unit 91 performs mask processing by displaying the mask image stored in the mask image storage unit 62 so as to overlap the acquired image. By applying the mask to the captured image data, the pixel value of the masked area is always constant.

In step S34, the image acquisition unit 42 determines whether the data of the captured image acquired in step S31 is the data of the first captured image. In the case of the first captured image data, since there is only one target image for calculating the difference capacity of the captured image data, the difference capacity cannot be calculated. Therefore, YES is determined, and the process returns to step S31. The captured image data is acquired.
On the other hand, when it is not the data of the first captured image, it is determined as NO, and the process proceeds to step S35.

  In step S35, the difference calculation unit 92 included in the image acquisition unit 42 calculates the difference capacity between the captured image data acquired in step S31 and the captured image data acquired in the process of the previous step S31. Is calculated. Specifically, the six captured images illustrated in FIG. 9 will be described as an example. The difference calculation unit 92 calculates the difference capacity of the captured image data between adjacent captured images. In addition, the difference calculation unit 92 stores the calculated difference capacity in the buffer area of the RAM 13. In the present embodiment, the stored differential capacity is distributed into three levels of large, medium, and small according to a predetermined threshold. When the process of step S35 ends, the difference calculation unit 92 returns the process to step S31, and the data of the next captured image is acquired.

  In step S32, when the image acquisition unit 42 determines that the photographing has been completed, it is determined YES, and the process proceeds to step S36. Therefore, since the processes of steps S31 to S35 are repeatedly executed until the shooting is finished, the captured image data acquired in step S31 and the captured image acquired in step S31 executed immediately before are captured. The difference capacity with respect to the data is repeatedly calculated and stored in the buffer area of the RAM 13.

  In step S <b> 36, the change rate calculation unit 93 included in the cue image specifying unit 43 calculates the change rate of the difference capacity between the images calculated by the difference calculation unit 92. For example, as illustrated in FIG. 9, when the change in the difference image capacity is small, such as “small → small”, “large → large”, or the like, the change rate of the difference capacity is “ In the case of “small”, “small → medium”, “medium → large”, and “large → medium”, the change rate of the differential capacity is “large” or “medium”.

  In step S37, the cue image specifying unit 43 specifies a plurality of cue candidate images based on the calculated change rate. Specifically, when the difference capacity is set to “large” or “medium”, the cue image specifying unit 43 has a smaller frame number among the two captured images that are the basis for calculating the difference capacity. This captured image is specified as a cue candidate image.

[Video playback processing]
Next, the moving image reproduction process executed by the image processing apparatus 1 will be described.
FIG. 12 is a flowchart showing an example of the flow of a moving image reproduction process executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.

  In FIG. 12, when the moving image reproduction process is started, in step S41, the reproduction unit 45 acquires moving image data from the acquired image storage unit 61, and decodes and reproduces the acquired moving image data.

  In step S42, the reproduction image extraction unit 44 determines whether or not the cue button has been pressed by the user. Here, the cue button is a button that is pressed when the user wants to perform cue reproduction. Specifically, the reproduction image extraction unit 44 determines whether or not an input of an instruction operation for executing the cueing reproduction is accepted through the input unit 17 when the cue button is pressed. When the input of the instruction operation is not accepted, NO is determined in step S42, and the process proceeds to step S43.

  In step S43, the reproduction image extraction unit 44 determines whether or not reproduction of the moving image has ended. When the reproduction of the moving image is finished, YES is determined in step S43, and the moving image reproduction process is finished. On the other hand, if the reproduction of the moving image does not end, NO is determined in step S43, and the process returns to step S42. Therefore, if the cue button is not pressed, the reproduction of the moving image is finished as it is. On the other hand, if the cue button is pressed, YES is determined in step S42, and the process proceeds to step S44.

  In step S44, the reproduction image extraction unit 44 displays the specified cueing candidate image on the output unit 18, and in step S45, an input of an instruction operation for selecting a cueing image from the user is input via the input unit 17. To identify the cue image.

  In step S <b> 46, the reproduction image extraction unit 44 extracts data of a moving image (a series of captured images) for reproduction from the acquired image storage unit 61. Specifically, the reproduction image extracting unit 44 sequentially extracts each data from the cue image specified by the cue image specifying unit 43 to the last captured image from the acquired image storage unit 61 as reproduction image data. To do.

In step S47, the reproduction unit 45 causes the output unit 18 to output the extracted image data for reproduction.
Therefore, the user selects a desired image from among the cueing candidate images as a cueing image, so that the one image can be obtained without fast-forwarding the moving image to a captured image at a point in time for several seconds before and after the swing start point. It is possible to cue from the captured image at the time point and view a series of captured images thereafter.

  When the process of step S47 ends, the reproducing unit 45 returns the process to step S42. Therefore, the reproduction image extraction unit 44 determines again whether or not the cue button has been pressed by the user. Therefore, when the cue button is pressed again, the processes of steps S44 to S46 are executed again, and the cue playback is executed again. On the other hand, if the cue button is not pressed again, the processes in steps S42 and S43 are repeatedly executed, and the moving image reproduction process is ended as it is.

The second embodiment of the present invention has been described above.
The image processing apparatus 1 that executes the above-described cue selection processing includes an image acquisition unit 42, a cue image specifying unit 43, and a reproduction image extraction unit 44.
The image acquisition unit 42 acquires a moving image in which a subject that performs a series of operations is captured.
The cueing image specifying unit 43 analyzes a series of operations in the moving image acquired by the image acquiring unit 42, and specifies a cueing candidate image that is a unit image at the time of starting reproduction based on the analysis result.
The reproduction image extracting unit 44 extracts data of a captured image group for a predetermined period starting from one of the cue image candidate images specified by the cue image specifying unit 43 from the moving image data.
Therefore, such an image processing apparatus 1 can identify and extract a moving image of a desired period without using a special device.

The image acquisition unit 42 includes a difference calculation unit 92.
In addition, the cue image specifying unit 43 includes a change rate calculating unit 93.
The difference calculation unit 92 calculates the difference capacity of the data of each adjacent unit image in the moving image for each unit image.
The change rate calculation unit 93 calculates the change rate of each differential capacity calculated by the difference calculation unit 92.
The cue image specifying unit 43 specifies a cue candidate image based on each change rate calculated by the change rate calculating unit 93.
Therefore, such an image processing device 1 can specify an image at the start of the golf swing as a cue candidate image without causing the user to fast-forward.

The cue image specifying unit 43 specifies a plurality of cue images, selects an arbitrary cue image from the plurality of specified cue images,
The image processing apparatus according to appendix 4, wherein the reproduction image extraction unit 44 extracts data of a captured image group for a predetermined period starting from the selected cueing image.

The image processing apparatus 1 further includes a mask processing unit 91.
The mask processing unit 91 sets the pixel value of the mask area 83 that is an area outside the locus of the head image 81 to a constant value for each of the data of the plurality of unit images constituting the moving image.
Further, the difference calculation unit 92 calculates the difference capacity based on each of the unit images in which the mask processing unit 91 sets the pixel value of the mask region 83 to a constant value.
Therefore, such an image processing device 1 applies a mask to an image outside the track of the golf swing and calculates a differential capacity. Therefore, since the image area unrelated to the golf swing is not taken into account when calculating the differential capacity, it is possible to improve the accuracy when specifying the cue candidate image.

The image processing apparatus 1 further includes a playback unit 45.
Based on the unit image extracted by the reproduction image extraction unit 44, the reproduction unit 45 reproduces data of the captured image group for a predetermined period.
Therefore, such an image processing apparatus 1 can reproduce a moving image of a desired period without using a special device.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  In the above-described embodiment, the case where the present invention is applied has been described as an example for moving image data in which a golf swing is copied. However, the present invention can be applied to comparison of various operations. . For example, the present invention can be applied in the case of comparing operations such as a baseball bat swing, a tennis racket swing, or a dance choreography.

In the above-described embodiment, the image processing apparatus 1 to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto.
For example, the present invention can be applied to general electronic devices having an image processing function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configurations of FIGS. 2 and 7 are merely examples, and are not particularly limited. That is, it is sufficient that the image processing apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is particularly the example shown in FIGS. It is not limited to.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. The recording medium etc. provided in The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 19 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the order, but is not necessarily performed in chronological order, either in parallel or individually. The process to be executed is also included.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
Acquisition means for acquiring a moving image in which a subject performing a series of operations is captured;
Analyzing means for analyzing the series of operations in the moving image acquired by the acquiring means;
A specifying unit for specifying a predetermined unit image based on an analysis result by the analyzing unit;
Extracting means for extracting a specific image from the moving image based on the predetermined unit image specified by the specifying means;
An image processing apparatus comprising:
[Appendix 2]
Image dividing means for dividing each of the unit images constituting the moving image into a plurality of image regions;
Passage determining means for determining whether or not a predetermined portion of the subject has passed in a specific order within a predetermined image area group among a plurality of image areas divided by the image dividing means;
Further comprising
The analysis means analyzes the series of operations depending on whether the predetermined portion has passed in the specific order by the passage determination means,
The specifying unit specifies the predetermined unit image at the time when the predetermined part belongs to the specific image region based on the analysis result by the analyzing unit.
The image processing apparatus according to appendix 1, wherein:
[Appendix 3]
The passage determining means thins out unit images from the moving image at a predetermined thinning rate, and determines whether the predetermined portion has passed in a specific order in the reproduction order of the thinned moving images. The image processing apparatus according to Supplementary Note 2, which is a feature.
[Appendix 4]
A difference calculating means for calculating a difference capacity between adjacent unit images of the moving images for each unit image;
A rate of change calculation means for calculating a rate of change of each difference capacity calculated by the difference calculation means;
Further comprising
The analysis means analyzes the series of operations based on the change rates calculated by the change rate calculation means,
The image processing apparatus according to appendix 1, wherein the specifying unit specifies the predetermined unit image based on an analysis result by the analyzing unit.
[Appendix 5]
The specifying unit specifies a plurality of the predetermined unit images,
A selection unit that selects an arbitrary unit image from the plurality of predetermined unit images specified by the specifying unit;
The image processing apparatus according to appendix 4, wherein the extraction unit extracts the unit image selected by the selection unit as the specific image from the moving image.
[Appendix 6]
For each of the plurality of unit images constituting the moving image, further comprising pixel value control means for setting a pixel value of a partial region to a constant value,
The difference calculation means calculates the difference capacity based on each of the unit images in which pixel values of the partial areas are set to a constant value by the pixel value control means.
The image processing apparatus according to appendix 4 or 5, characterized in that:
[Appendix 7]
The image processing apparatus according to any one of appendices 1 to 6, further comprising a reproducing unit that reproduces an image group for a predetermined period based on the unit image extracted by the extracting unit.
[Appendix 8]
An image processing method in an image processing apparatus,
An acquisition step of acquiring a moving image in which a subject performing a series of operations is captured;
An analysis step for analyzing the series of operations in the moving image acquired by the acquisition step;
A specific step of specifying a predetermined unit image based on the analysis result of the analysis step;
An extracting step of extracting a specific image from the moving image based on the predetermined unit image specified by the specifying step;
An image processing method including:
[Appendix 9]
Computer
An acquisition means for acquiring a moving image in which a subject performing a series of operations is captured;
Analyzing means for analyzing the series of operations in the moving image acquired by the acquiring means;
A specifying unit for specifying a predetermined unit image based on an analysis result by the analyzing unit;
Extracting means for extracting a specific image from the moving image based on the predetermined unit image specified by the specifying means;
Program to function as.

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input / output interface, 16 ... Imaging part, 17 ... Input unit, 18 ... output unit, 19 ... storage unit, 20 ... communication unit, 21 ... drive, 31 ... removable media, 41 ... imaging control unit, 42 ... Image acquisition unit, 43 ... Cue image specifying unit, 44 ... Reproduction image extraction unit, 45 ... Reproduction unit, 51 ... Image division unit, 52 ... Passing determination unit, 61 ... Acquired image storage unit, 62 ... Mask image storage unit, 71 to 78 ... Image area, 81 ... Head image, 82 ... Ball image, 83 ... Mask area, 84 ... Captured image , 91 ... Mask part, 92 ... Difference calculation part, 93 ... Change rate calculation part

Claims (9)

Acquisition means for acquiring a moving image in which a subject performing a series of operations is captured;
Analyzing means for analyzing the series of operations in the moving image acquired by the acquiring means;
A specifying unit for specifying a predetermined unit image based on an analysis result by the analyzing unit;
Extracting means for extracting a specific image from the moving image based on the predetermined unit image specified by the specifying means;
An image processing apparatus comprising: Image dividing means for dividing each of the unit images constituting the moving image into a plurality of image regions;
Passage determining means for determining whether or not a predetermined portion of the subject has passed in a specific order within a predetermined image area group among a plurality of image areas divided by the image dividing means;
Further comprising
The analysis means analyzes the series of operations depending on whether the predetermined portion has passed in the specific order by the passage determination means,
The specifying unit specifies the predetermined unit image at the time when the predetermined part belongs to the specific image region based on the analysis result by the analyzing unit.
The image processing apparatus according to claim 1. The passage determining means thins out unit images from the moving image at a predetermined thinning rate, and determines whether the predetermined portion has passed in a specific order in the reproduction order of the thinned moving images. The image processing apparatus according to claim 2. A difference calculating means for calculating a difference capacity between adjacent unit images of the moving images for each unit image;
A rate of change calculation means for calculating a rate of change of each difference capacity calculated by the difference calculation means;
Further comprising
The analysis means analyzes the series of operations based on the change rates calculated by the change rate calculation means,
The image processing apparatus according to claim 1, wherein the specifying unit specifies the predetermined unit image based on an analysis result by the analyzing unit. The specifying unit specifies a plurality of the predetermined unit images,
A selection unit that selects an arbitrary unit image from the plurality of predetermined unit images specified by the specifying unit;
The image processing apparatus according to claim 4, wherein the extraction unit extracts the unit image selected by the selection unit from the moving image as the specific image. For each of the plurality of unit images constituting the moving image, further comprising pixel value control means for setting a pixel value of a partial region to a constant value,
The difference calculation means calculates the difference capacity based on each of the unit images in which pixel values of the partial areas are set to a constant value by the pixel value control means.
The image processing apparatus according to claim 4, wherein the image processing apparatus is an image processing apparatus.   The image processing apparatus according to claim 1, further comprising a reproducing unit that reproduces an image group for a predetermined period based on the unit image extracted by the extracting unit. An image processing method in an image processing apparatus,
An acquisition step of acquiring a moving image in which a subject performing a series of operations is captured;
An analysis step for analyzing the series of operations in the moving image acquired by the acquisition step;
A specific step of specifying a predetermined unit image based on the analysis result of the analysis step;
An extracting step of extracting a specific image from the moving image based on the predetermined unit image specified by the specifying step;
An image processing method including: Computer
An acquisition means for acquiring a moving image in which a subject performing a series of operations is captured;
Analyzing means for analyzing the series of operations in the moving image acquired by the acquiring means;
A specifying unit for specifying a predetermined unit image based on an analysis result by the analyzing unit;
Extracting means for extracting a specific image from the moving image based on the predetermined unit image specified by the specifying means;
Program to function as.

Images