JP2015126516A - Image processing apparatus, image processing method, program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus in which existence of a video related to a displayed still image can be confirmed.SOLUTION: To display an image for instructing playback of a video file related to a still image file corresponding to an image under display, when the file size of a video file related to a still image file corresponding to an image being displayed in a display matches the file size of a video file stored in the reference information contained in the video file.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  2. Description of the Related Art Conventionally, as an image processing apparatus, a digital camera that records image data obtained by imaging a subject on a recording medium as a still image or a moving image is known. Among these imaging apparatuses, for example, as disclosed in Patent Document 1, an imaging apparatus that captures a still image and a moving image and reproduces a recorded scene at the same time has been proposed.

Japanese Patent Laid-Open No. 10-224727

  However, Patent Document 1 is a method of recording related still image information in a moving image stream, and the moving image stream must be searched to identify the associated still image. The longer the processing time, the longer the processing time.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus capable of confirming the presence of a moving image related to a displayed still image.

  In order to achieve such an object, the image processing apparatus of the present invention corresponds to a reproduction unit that reproduces a moving image file and a still image file recorded on a recording medium, and a still image file reproduced by the reproduction unit. Stored in display control means for displaying an image on a display unit, file size in the recording medium of a moving image file related to a still image file corresponding to the image being displayed on the display unit, and reference information included in the moving image file The display control unit is controlled to display an image for instructing reproduction of the moving image file related to the still image file corresponding to the image being displayed when the file size of the moving image file matches And a control means.

  According to the present invention, the presence of a moving image related to a displayed still image can be confirmed.

1 is a diagram illustrating a configuration of an imaging apparatus according to an embodiment. The figure which shows operation | movement in the moving image still image shooting mode of a present Example. The figure which shows operation | movement in the moving image still image shooting mode of a present Example. The figure which shows the example of a display of the stop motion moving image of a present Example. The figure which shows the information recorded in the moving image still image shooting mode of a present Example. The figure which shows the additional information of the moving image of a present Example. The figure which shows operation | movement of the reproduction | regeneration mode of a present Example. The figure which shows the example of the display icon of a present Example.

  Hereinafter, examples of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments. The following embodiments do not limit the invention according to the claims, and all combinations of features described in the embodiments are not necessarily essential to the solution means of the invention.

  Note that each functional block described in the present embodiment is not necessarily separate hardware. That is, for example, the functions of some functional blocks may be executed by one piece of hardware. In addition, the function of one functional block or the functions of a plurality of functional blocks may be executed by some hardware linked operations. Further, the function of each functional block may be executed by a computer program developed on the memory by the CPU.

Example 1
In this embodiment, an imaging apparatus will be described as an example of a sound processing apparatus. However, any apparatus may be used as long as it can reproduce a moving image and a still image associated with the moving image. For example, a mobile phone, a smartphone, a tablet information terminal, a notebook information terminal, a computer, or the like may be used.

  The imaging apparatus according to the present embodiment can reproduce the moving image and the still image recorded on the recording medium, and when reproducing the still image associated with the moving image, there is a moving image associated with the still image. Can show. In addition, when a moving image is edited and the existence of a related moving image becomes uncertain, it is possible to prevent display indicating that there is a moving image related to the displayed still image.

  With such a configuration, the imaging apparatus according to the present embodiment can easily confirm the presence of a moving image related to the displayed still image. Further, when a moving image has been edited, it is possible to prevent erroneous reproduction by not displaying uncertain association information. Hereinafter, such an imaging apparatus will be described.

<Overall configuration>
First, the configuration of the imaging apparatus 100 of the present embodiment will be described with reference to FIG.

  As illustrated in FIG. 1, the imaging apparatus 100 according to the present exemplary embodiment includes a CPU 101, a RAM 102, a ROM 103, and an operation unit 104. In addition, the imaging apparatus 100 includes a first imaging unit 110, an image processing unit 111, a second imaging unit 112, a sound collection unit 120, and an audio processing unit 121. In addition, the imaging apparatus 100 includes a display unit 140, a display control unit 141, a recording / playback unit 150, a recording medium 151, and an attitude detection unit 160. CPU is an abbreviation for Central Processing Unit. RAM is an abbreviation for Random Access Memory. ROM is an abbreviation for Read Only Memory.

  In the imaging apparatus 100 according to the present exemplary embodiment, the CPU 101 develops various programs recorded in the ROM 103 using the RAM 102 as a work memory, and controls each block of the imaging apparatus 100 according to the program. The operation unit 104 includes switches for inputting various operations such as a power button, a record button, a zoom adjustment button, an autofocus button, a menu display button, a mode switch, and a determination button. Further, any type of operation element such as a cursor key, a pointing device, a touch panel, and a dial may be used. The operation unit 104 transmits an operation signal to the CPU 101 when these keys, buttons, and touch panel are operated by the user. Each operation member of the operation unit 104 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the display unit, and functions as various function buttons. Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, various setting menu screens are displayed on the display unit. The user can make various settings intuitively using the menu screen displayed on the display unit, and the four-way buttons and the SET button. Note that the operation unit 104 may be a touch panel that can detect contact with the display unit. The touch panel may be any of various types of touch panels such as a resistive film method, a capacitance method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method. .

  In the first imaging unit 110, the second imaging unit 112 converts the optical image of the subject captured by the lens into an image signal using an imaging element such as a CCD sensor or a CMOS sensor by controlling the amount of light using a diaphragm. In the first imaging unit 110, the second imaging unit 112 converts the obtained analog image signal into a digital image signal and temporarily stores it in the RAM 102. The image processing unit 111 processes image data stored in the RAM 102 during recording, and processes image data read from a recording medium 151 (to be described later) during reproduction. The image processing unit 111 performs various processes such as white balance adjustment, gain adjustment, and color enhancement processing of image data. The image processing unit 111 transmits the processed image data to the display control unit 141 or to the recording / playback unit 150 or the communication unit. In addition, the image processing unit 111 is an H.264. H.264 / AVC, Motion JPEG2000, H.264, etc. An image can be compressed as a moving image using a moving image compression technique such as H.265, or an image can be compressed as a still image using a still image compression technique such as JPEG or JPEG 2000. As the compression format, other methods may be used regardless of these formats.

  The audio processing unit 121 processes audio data obtained by collecting sound by the sound collecting unit 120 during recording, and processes compressed or uncompressed audio data read from a recording medium 151 described later during reproduction. . For example, audio data level adjustment processing, compressed audio data decompression processing, audio data compression processing, and the like are performed. The audio processing unit 121 transmits the processed audio data to the recording / reproducing unit 150 and the communication unit, or to the speaker during reproduction. Note that the sound collection unit 120 is, for example, a microphone that collects a plurality of omnidirectional sounds built in the housing of the imaging apparatus 100. Also, the audio processing unit 121 uses a known general audio compression method such as AC3, AAC, or the like as the audio compression method.

  The display unit 140 includes, for example, a liquid crystal display device or an organic EL display device, and displays an image under the control of the display control unit 141. The display unit 140 may be anything such as an LED display as long as it can provide an image to the user. The display control unit 141 displays an image on the display unit 140 based on the image data processed by the image processing unit 111. Further, the display control unit 141 may perform image signal processing such as matrix conversion, brightness adjustment, contrast adjustment, gamma adjustment, chroma gain adjustment, and sharpness adjustment on the video signal based on the digital image signal displayed on the display unit 140. Good.

  The recording / reproducing unit 150 records the image data and audio data processed by the image processing unit 111 on the recording medium 151 at the time of recording, and the image data and audio data recorded on the recording medium 151 at the time of reproduction. Reproduce. At the time of recording, the recording / reproducing unit 150 writes various types of information such as the shooting date, information related to compression, and settings of the imaging unit 110 to the recording medium 151 together with image data and audio data. The recording / reproducing unit 150 records image data and audio data on the recording medium 151 as a file in a form suitable for a file format such as FAT or exFAT, for example. Further, the recording medium 151 may be a recording medium built in the imaging apparatus or a removable recording medium. For example, the recording medium includes all types of recording media such as a hard disk, an optical disk, a magneto-optical disk, a CD-R, a DVD-R, a magnetic tape, a nonvolatile semiconductor memory, and a flash memory. When using removable recording media, the recording / reproducing unit 150 includes a mechanism for loading and unloading the removable recording media.

  The posture detection unit 160 detects the posture of the imaging apparatus 100 and is, for example, an angle sensor or an acceleration sensor. The posture detection unit 160 of the present embodiment detects whether the posture of the imaging device 100 is the normal position or the vertical position. For example, a rotation attitude angle (0 degree, 90 degrees, 180 degrees, 270 degrees) about the optical axis of the first imaging unit 110 of the imaging apparatus 100 can be detected. Here, 0 degrees is a so-called normal position (a state in which the upper surface of the imaging apparatus 100 faces the sky direction), and 180 degrees is a reverse position (a state in which the upper surface of the imaging apparatus 100 faces the ground direction). 90 degrees and 270 degrees are so-called vertical positions (a state in which either of the left and right surfaces of the imaging apparatus 100 faces the ground).

  Note that the first imaging unit 110 (first imaging unit) and the second imaging unit 112 (second imaging unit) of the present embodiment each capture subjects in different directions. Specifically, the second imaging unit 112 is a sub-camera for photographing a photographer (so-called cameraman), and the first imaging unit 110 is a subject (so-called subject) that the photographer intends to photograph. This is the main camera. Therefore, for example, the first image capturing unit 110 and the second image capturing unit 112 capture images in the reverse direction, but may not necessarily be in the reverse direction.

  Here, each of the image processing unit 111, the audio processing unit 121, the display control unit 141, and the recording / reproducing unit 150 may be a microcomputer equipped with a program for executing the above-described functions. Further, the program for executing the above-described processing recorded in the ROM 103 by the CPU 101 may be loaded into the RAM 102 and executed.

<Operation of Imaging Device 100>
Subsequently, the operation of the imaging apparatus 100 of the present embodiment will be described.

  The imaging apparatus 100 according to the present embodiment has a “moving image still image shooting mode” in addition to the “still image shooting mode”, the “moving image shooting mode”, and the “reproduction mode”.

  In each mode, the CPU 101 controls each block of the imaging apparatus 100 to perform the following operation.

  In the “still image shooting mode” and “moving image shooting mode”, only the image obtained by the first imaging unit 110 or the image obtained by the first imaging unit 110 and the first image until the shooting instruction is input. 2 The image obtained by combining the images obtained by the imaging unit 112 is displayed on the display unit 140. That is, an image obtained by each imaging unit is processed by the image processing unit 111, and the display control unit 141 is controlled to be displayed on the display unit 140. When a shooting instruction is input, the recording / reproducing unit 111 causes the image processing unit 111 to compress the image obtained by each imaging unit as a moving image or a still image, and sequentially record the compressed image on the recording medium 151. 150 is controlled. In the “still image shooting mode”, the image processing unit 111 is controlled so that one or a plurality of images obtained by each imaging unit are compressed using, for example, JPEG at a timing corresponding to a shooting instruction. Then, the recording / reproducing unit 150 is controlled to record the compressed still image data on the recording medium 151 as a still image file. On the other hand, in the “moving image shooting mode”, a moving image in which images sequentially obtained from images obtained by the respective imaging units at a timing corresponding to a shooting instruction are frame images is used, for example. The image processing unit 111 is controlled to perform compression using H.265. Then, the recording / playback unit 150 is controlled to record the compressed moving image data on the recording medium 151 as a moving image file. In the “moving image shooting mode”, this operation is continued until an instruction to end shooting is input. In the “still image shooting mode” and “moving image shooting mode”, a thumbnail image (still image data) based on the image shot in each mode is added to the still image file and the movie file and recorded on the recording medium 151. Is done.

  In “Movie Still Image Shooting Mode”, when a still image recording instruction is input, still image shooting and still image recording are performed, and moving image data for 4 seconds immediately before the still image recording instruction is input is recorded. Record on media. On the same day, when a plurality of still image recording instructions are input in the “moving picture still image shooting mode”, the moving image data for 4 seconds immediately before the input of the still image recording instruction is stored as one. Add to the movie file and record. By doing in this way, the moving image data just before the still image photography for 1 day can be stored in one moving image file, and browsing property can be improved. In the “moving picture still image shooting mode”, the CPU 101 performs only the image obtained by the first imaging unit 110 or the image obtained by the first imaging unit 110 and the second imaging until a shooting instruction is input. An image obtained by combining the images obtained by the unit 112 is displayed on the display unit 140. That is, an image obtained by each imaging unit is processed by the image processing unit 111, and the display control unit 141 is controlled to be displayed on the display unit 140. At this time, the image processing unit 111 combines only the image obtained by the first imaging unit 110 or an image obtained by combining the image obtained by the first imaging unit 110 and the image obtained by the second imaging unit 112. It is sequentially compressed as moving image data and temporarily stored in the RAM 102. The CPU 101 manages information in the RAM 102 so that the latest compressed video data for 4 seconds is always held in the RAM 102. For example, moving image data other than the latest 4 seconds may be discarded. Next, when a still image recording instruction is input, the image processing unit 111 is controlled to compress one or a plurality of images obtained by each imaging unit at a timing corresponding to the instruction using, for example, JPEG. To do. Then, the recording / reproducing unit 150 is controlled so as to record still image data compressed as a still image on the recording medium 151. On the other hand, in response to a still image recording instruction, the recording / reproducing unit 150 is controlled so that the latest 4-second compressed moving image data stored in the RAM 102 is also recorded on the recording medium 151. In addition, the image processing unit 111 is controlled so as to compress the image obtained by the still image recording instruction as a moving image (stop motion), and recorded on the recording medium 151 so as to be reproduced after the latest 4-second moving image data. To do. Thus, the recording / playback unit 150 includes the moving image data of the image obtained by the still image recording instruction in the same moving image file as the latest 4-second moving image data recorded on the recording medium.

  In the “playback mode”, the recording / playback unit 150 is controlled to read the file list and thumbnail images of the moving image file or still image file recorded on the recording medium 151. Then, the display control unit 141 is controlled so that the thumbnail image of each file reproduced by the recording / reproducing unit 150 is expanded by the image processing unit 111 and the expanded thumbnail image is displayed on the display unit 140. Then, the CPU 101 expands the still image file or the moving image file corresponding to the thumbnail image selected by the user operating the operation unit 104 to the image processing unit 111, and displays the obtained image on the display unit 140. The display control unit 141 is controlled.

  Here, the “moving image still image shooting mode” of the present embodiment will be described in detail with reference to FIG. FIG. 2 is a flowchart for explaining the operation of the imaging apparatus 100 in the “moving picture still image shooting mode”. The process of FIG. 2 is a process executed by the CPU 101 controlling each block of the imaging apparatus 100.

(S201)
First, the CPU 101 acquires the operation state of each block of the imaging device 100. That is, the operation state of each block of the imaging apparatus 100 before the still image recording instruction is input is acquired. For example, the posture of the imaging device 100 detected by the posture detection unit 160 is acquired. In addition, information on the focus operation in the first imaging unit 110 and information on the zoom operation are acquired. The information regarding the focus operation is information indicating whether or not the focus lens of the first imaging unit 110 is being driven. The information regarding the zoom operation is whether the focus lens of the first imaging unit 110 is being driven. This is information indicating whether or not. These pieces of information may be acquired from the imaging unit 110 by the CPU 101, or may hold a history of controlling the driving of the imaging unit 110 by the CPU 101 itself. At this time, the image signal obtained by the first imaging unit 110 is processed by the image processing unit 111. The CPU 101 analyzes the obtained image, determines whether or not a feature image such as a person or a face is included in the image, and further determines the “smile level” when a face is detected. The image processing unit 111 is controlled to make the determination. Note that the “degree of smile” is calculated by analyzing the image of the part corresponding to the eyes and mouth in the face image, the line of the line connecting the images corresponding to the eyes, and the edge and center of the image corresponding to the mouth. The smaller the angle formed by the connected lines, the higher the “smile level”, and the larger the angle formed, the lower the “smile level”. The determination method of “smile level” is not limited to this method. The CPU 101 holds the information acquired in this way for a predetermined time, for example, in units of 1 second and 0.5 seconds. These pieces of information are recorded as additional information in a still image file including still image data corresponding to a still image recording instruction by a process described later. This additional information is recorded, for example, as table data as shown in FIG.

(S202)
Next, the CPU 101 acquires setting information of the second imaging unit 112 stored in the RAM 102. The setting information of the second imaging unit 112 includes information indicating whether the imaging of the second imaging unit 112 is “valid” or “invalid” and an image acquired by the second imaging unit 112 by the first imaging unit 110. Information indicating a “display position” and a “display size” when the image is displayed superimposed on the acquired image is included.

(S203)
Next, the CPU 101 determines whether or not the shooting of the second imaging unit 112 is “valid” based on the setting information of the second imaging unit 112 acquired in S202.

(S204)
If the shooting of the second imaging unit 112 is “valid” in S203 (Yes in S203), the CPU 101 synthesizes the image obtained by the first imaging unit and the image obtained by the second imaging unit. The image processing unit 111 is controlled so as to generate the processed image.

(S205)
In S203, when the shooting of the second imaging unit 112 is not “valid” (No in S203), the CPU 101 controls the image processing unit 111 so as to generate an image obtained by the first imaging unit.

(S206)
After S204 and S205, the CPU 101 controls the image processing unit 111 to compress the generated image as moving image data. Thereby, moving image data in which the image generated in S204 or S205 is a frame image is generated. Then, the CPU 101 temporarily stores the compressed moving image data in the RAM 102. The same applies to the audio processing, and the CPU 101 causes the audio processing unit 121 to process the audio signal obtained by the sound collection unit 120 and temporarily stores the audio signal in the RAM 102 in association with the moving image data. At this time, the CPU 101 controls the image processing unit 111 to change the size of the image obtained in S204 or S205, adjust the gamma, and the like, and performs display control to display the obtained image on the display unit 140. The unit 141 is controlled. Thus, until a still image shooting instruction is input, an image obtained by each imaging unit can be compressed as moving image data and buffered in the RAM 102, and the image can be displayed on the display unit 140.

(S207)
Next, the CPU 101 determines whether an instruction to change to a mode other than the “moving image still image shooting mode” or to turn off the power is input via the operation unit 104.

(S208, S209)
If a mode change is input in S207 (Yes in S207), the CPU 101 controls the image processing unit 111 to stop the operation of compressing the image obtained by each imaging unit as a moving image. Further, the moving image data and audio data stored in the RAM 102 and the information on the operation state of each block of the imaging apparatus 100 are deleted, and the processing of the “moving image still image shooting mode” is ended.

(S210)
If no mode change is input in S207 (No in S207), the CPU 101 determines whether the operation state of each block of the imaging apparatus 100 has changed or whether the setting information of the second imaging unit 112 has changed. judge. Note that it may be determined whether there has been a change in only some of the operation states of the blocks of the imaging apparatus 100, or only some of the setting information in the second imaging unit 112 has changed. You may determine whether there was. In this embodiment, whether there is a change in the posture information, whether there is a change in the information indicating whether the second imaging unit 112 is “valid” or “invalid”, “display position” and “display size” It is determined whether there has been a change in the information indicating "."

(S211)
If the operation state of each block of the imaging apparatus 100 has changed in S210, or the setting information of the second imaging unit 112 has changed (Yes in S210). The CPU 101 erases the moving image data and audio data stored in the RAM 102 and the information on the operation state of each block of the imaging apparatus 100. Then, the process returns to S201. For example, such an operation is performed when the posture information of the imaging apparatus detected by the posture detection unit 160 changes from “0 degree” to “90 degrees”. Such an operation is also performed when the setting of the second imaging unit 112 is changed from “valid” to “invalid”.

(S212)
On the other hand, when the operation state of each block of the imaging apparatus 100 is not changed in S210 and the setting information of the second imaging unit 112 is not changed (No in S210). The CPU 101 determines whether SW1 of the shutter button of the operation unit 104 is turned on. Note that the shutter button of the image pickup apparatus 100 of the present embodiment is a two-stage switch, like the shutter button of a general image pickup apparatus. SW1 is turned on when the shutter button is half-pressed, and SW2 is turned on when the shutter button is fully pressed. That is, in S212, it is determined whether or not the shutter button has been half-pressed. If the SW1 of the shutter button of the operation unit 104 is not ON (No in S212), the process returns to S201. In this embodiment, a configuration in which a two-stage switch is used like a shutter button is shown as an example. However, SW1 and SW2 may be assigned to different buttons, or buttons corresponding to different UIs on the touch panel. May be assigned.

(S213)
On the other hand, when SW1 of the shutter button of the operation unit 104 is ON (Yes in S212), the CPU 101 causes the first imaging unit 110 to perform an operation for still image shooting. The processing includes, for example, autofocus (AF) processing, automatic exposure adjustment (AE) processing, auto white balance (AW) processing, AF auxiliary light emission processing, and the like. Note that when the SW1 is ON, the imaging apparatus 100 according to the present embodiment does not perform the determination of S210. For example, even if the attitude of the imaging apparatus 100 changes, moving image data, audio data, and imaging buffered in the RAM 102 The information on the operation state of each block of the apparatus 100 is not erased.

(S214)
Next, the CPU 101 determines whether SW2 of the shutter button of the operation unit 104 has been turned on. That is, in S214, it is determined whether or not the shutter button has been fully pressed. If the SW1 of the shutter button of the operation unit 104 is not ON (No in S214), the process returns to S201.

(S215)
On the other hand, when SW2 of the shutter button of the operation unit 104 is ON (Yes in S214), the CPU 101 acquires the first imaging unit 110 and / or the second imaging unit 112 so as to acquire a still image. To control. Then, the CPU 101 further controls the image processing unit 111 so as to stop the operation of compressing the image obtained by each imaging unit as a moving image, and the display control unit so as to black out the display of the display unit 140 once. 141 is controlled.

(S216)
Next, the CPU 101 executes processing after photographing. Processing after shooting will be described with reference to FIG. The process in FIG. 3 is a process executed by the CPU 101 controlling each block of the imaging apparatus 100.

(S301)
The CPU 101 controls the display control unit 141 to display the still image acquired from each imaging unit in step S215 on the display unit 140, and further controls the image processing unit 111 to compress the still image data. . Then, the CPU 101 controls the recording / reproducing unit 150 to record the still image data generated by the image processing unit 11 on the recording medium 151. The still image data is recorded on the recording medium 151 as a still image file. In the present embodiment, the images obtained by the first imaging unit 110 and the second imaging unit 112 are compressed as separate still image data. However, these images are combined into still image data. You may compress.

(S302)
Next, the CPU 101 determines whether the moving image data stored in the RAM 102 is moving image data for t0 seconds or more.

(S303)
When the moving image data stored in the RAM 102 is moving image data for t0 seconds or more (Yes in S302). The CPU 101 controls the recording / reproducing unit 150 to record the moving image data and audio data stored in the RAM 102 on the recording medium 151. Note that the moving image data and audio data are recorded on the recording medium 151 as a moving image file. At this time, the CPU 101 stores information indicating the operation state of each block of the imaging apparatus 100 shown in FIG. 6 stored in the RAM 102 as additional information in the header portion of the corresponding still image file. As a result, the operation state of each block of the imaging device 100 during shooting of the moving image data recorded on the recording medium 151 is recorded on the recording medium 151 as additional information of the corresponding still image file.

(S304)
On the other hand, when the moving image data stored in the RAM 102 is moving image data less than t0 seconds (No in S302). The CPU 101 erases the moving image data and audio data stored in the RAM 102 and information indicating the operation state of each block of the imaging apparatus 100 from the RAM 102.

(S305)
After the process of S303 or S304, the CPU 101 determines whether the setting of the second imaging unit 112 is “valid” or “invalid”. In the imaging apparatus 100 of the present embodiment, the generation method used when compressing the image obtained by the first imaging unit 110 as a moving image in response to a still image recording instruction (SW2 is ON) to be generated later is this. It depends on the setting. This moving image is a moving image in which images obtained by the first imaging unit 110 can be continuously displayed for a predetermined frame in response to the SW2 being turned on. In the present embodiment, this moving image is called a stop motion. . In the present embodiment, the second imaging unit is displayed when the image obtained by the first imaging unit 110 is displayed as a preview without using the stop motion for the image obtained by the second imaging unit 112. Assume that the images obtained in step 112 are sequentially synthesized. When the setting of the second imaging unit 112 is “valid”, a stop motion video is generated. If it is “invalid”, the moving image data stored in the RAM 102 before the SW2 is turned on is t0 or more, and if the stop motion setting is enabled, stop motion moving image data is generated. On the other hand, if it is “invalid”, if the moving image data stored in the RAM 102 before the SW2 is turned on is less than t0 or the stop motion setting is invalid, stop motion data is generated. do not do.

(S306)
If the setting of the second imaging unit 112 is “valid” in S305 (Yes in S305), the CPU 101 controls the image processing unit 111 to generate a stop motion video. In this case, as described above, the image obtained by the second imaging unit 112 when the image obtained by the first imaging unit 110 when the SW2 is turned ON is displayed on the display unit 140 as a preview. Composite to stop motion video. That is, as shown in FIG. 4A, the stop motion 401 in which the image obtained by the second imaging unit 112 is superimposed as a moving image as in the case of the stop motion 401 of the image obtained by the first imaging unit 110. Generate a video. Therefore, in S306, the image processing unit 111 is controlled to perform processing such as changing the image obtained by the first imaging unit 110 to the size for the stop motion moving image when the SW2 is turned on.

(S307)
Next, the CPU 101 controls the image processing unit 111 so as to generate an image obtained by combining the image obtained by the second imaging unit 112 with the image for stop motion generated in S306.

(S308)
Next, the CPU 101 controls the display control unit 141 to display the composite image generated in S307 on the display unit 140.

(S309)
Next, the CPU 101 controls the image processing unit 111 so as to compress the combined image generated in S307 into one frame of a moving image as moving image data.

(S310)
The CPU 101 continues the processing from S307 to S309 until the time of the stop motion moving image reaches t1 seconds. With this process, a stop motion video for t1 seconds is generated.

(S311)
On the other hand, if the setting of the second imaging unit 112 is “invalid” in S305 (No in S305), the CPU 101 determines whether or not the moving image data stored in the RAM 102 in S304 has been erased. If the moving image data has been erased (Yes in S311), the post-shooting process ends. That is, the processing ends without recording the moving image data immediately before the instruction to record the still image and the moving image data of the stop motion moving image.

(S312)
On the other hand, when the moving image data is not erased (No in S311), the CPU 101 determines whether or not the setting of the stop motion moving image is valid. If the stop motion video setting is not valid (No in S312), the process ends without generating the stop motion video.

(S313)
On the other hand, when the setting of the stop motion video is valid (Yes in S312), the CPU 101 controls the image processing unit 111 to generate the stop motion video. In S313, the image processing unit 111 is controlled so as to perform processing such as changing the image obtained by the first imaging unit 110 to the size for the stop motion moving image when the SW2 is turned on.

(S314)
Next, the CPU 101 controls the display control unit 141 so that the composite image generated in S313 is displayed on the display unit 140.

(S315)
Next, the CPU 101 controls the image processing unit 111 so as to compress the combined image generated in S313 as moving image data into one frame of the moving image.

(S316)
The CPU 101 continues the process of S315 until the time of the stop motion moving image reaches t2 seconds. With this process, a stop motion video for t2 seconds is generated.

(S317)
After S310 and S316, the CPU 101 controls the recording / reproducing unit 150 to record the stop motion moving image generated by the image processing unit 111 on the recording medium 151. In addition, when the moving image data for 4 seconds before SW2 is turned ON in S303 is recorded on the recording medium 151, the moving image data of the stop motion moving image is added to the moving image file storing the moving image data. .

  In this way, the process of S216 ends. Note that after the process of S216 ends, basically, the moving image still image shooting mode is not ended and the process continues from S201.

  In this embodiment, t1> t2. This is t2 when a stop motion video is simply displayed. T1 when a moving image that captures a change in the facial expression of the photographer when checking an image corresponding to the stop motion moving image is combined with the stop motion moving image. In this way, since the moving image capturing the change in the expression of the photographer is superimposed, t1 is made longer.

  Next, still images and moving images that are recorded in the moving image still image shooting mode will be described with reference to FIG. FIG. 5A is a schematic diagram showing the timing when SW2 is turned on in the moving image still image shooting mode, and the recorded still image and moving image. FIG. 5A shows the following situation. On October 10, 2013, SW2 is turned on at 15:16:20 (release 1), and moving image data M1 (501) and still image data P1 (502) are recorded. Further, SW2 is turned on at 15:30:45 (release 2), and moving image data M2 (503) and still image data P2 (504) are recorded. Further, SW2 is turned on at 15:40:30 (release 3), and moving image data M3 (505) and still image data P3 (506) are recorded.

  FIG. 5B is a schematic diagram showing the contents of a moving image file and a still image file recorded on the recording medium 151. In this embodiment, a file including data recorded on the same day is stored in one directory 511 (directory name: 100_1010). The directory name and each file name are determined in accordance with DCF (Design rule for Camera File System). Further, as described above, the moving image data recorded in the moving image still image shooting mode is added to one moving image file on the same day. In the moving image file 512 (moving image file name: MDG — 0001.MOV), moving image data corresponding to each of three shootings in the moving image still image shooting mode performed on the same day shown in FIG. A chapter is set for each moving image data. That is, the moving image file MDG_0001. The MOV includes moving image data corresponding to a plurality of scenes. Each scene is stored in a still image file IMG_0002. JPG, IMG_0003. JPG, IMG_0004. Each scene corresponds to a JPG. Note that a moving image shot in the normal moving image shooting mode is recorded as a separate moving image file.

  The moving image data corresponding to chapter 1 (514) is a still image file IMG_0002. The moving image data corresponding to the JPG shooting instruction is the moving image data M1 in FIG. The moving image data corresponding to chapter 2 (515) is a still image file IMG_0003. This is motion picture data corresponding to a JPG shooting instruction, and is motion picture data M2 in FIG. 5A and stop motion motion picture data generated from the same image as the still picture data P2. The moving image data corresponding to chapter 3 (516) is a still image file IMG_0003. This is moving image data corresponding to a JPG shooting instruction, and is stop motion moving image data generated from the same image as the still image data P2. The moving image data corresponding to the chapter 3 (516) does not include the moving image data M3 in FIG. 5A because the moving image data M3 is moving image data of t0 seconds or less.

  Movie file MDG_0001. In the MOV, “chapter reference information” of a moving image file including the plurality of moving image data is recorded in the header portion. This chapter information is updated every time moving image data is recorded on the recording medium 151 in the moving image still image shooting mode. The chapter reference information will be described with reference to FIG. The chapter reference information includes the file size (521) of the moving image file and the number of frames (522) of the moving image data included in the moving image file. Furthermore, the set chapter number (523), the shooting time of video data corresponding to the chapter (524), the number of the first frame of video data corresponding to the chapter (525), and the number of frames of video data corresponding to each chapter (526) is included. Furthermore, the composition position (527) of the image of the second imaging unit 112 and the position (528) of stop motion moving image data in the moving image data corresponding to the chapter are included.

  The chapter number (521) is added every time moving image data is recorded in the moving image still image shooting mode. The shooting time (524) of the moving image data is the time, year, month, and day when SW2 is turned on in the moving image still image shooting mode. That is, it is the same as the shooting time of the corresponding still image data recorded in the moving image still image shooting mode. Thus, by making the shooting time of each still image data the same as the shooting time of the moving image data of each chapter, it is easy to identify the corresponding chapter from the still image data.

  The first frame number (525) of each chapter is the frame number of the first frame of each chapter in the entire moving image data recorded in the moving image file 512. By using this, it is easy to cue and reproduce each moving image data. It becomes possible. The composite position (527) of the image of the second imaging unit 112 is superimposed when the upper left coordinate of the image of the first imaging unit 110 is (X, Y) = (0, 0). And the horizontal width W and the vertical width H of the image of the second imaging unit 112. In chapter 1, the values of the coordinates, the horizontal width, and the vertical width of the image of the second imaging unit 112 are zero. This indicates that the shooting setting of the second imaging unit 112 is “invalid”. By using the information on the composite position (527) of the image of the second imaging unit 112, it is possible to trim and reproduce only the image of the sub camera.

  The position (528) of the stop motion moving image data indicates the frame number of the stop motion moving image data portion when the first frame of the moving image data of each chapter is 0. In the moving image data of chapter 1, there is no stop motion moving image portion. In chapter 2, out of the number of frames 180 of the moving image data of chapter 2, the latter 90th frame to 179th frame are stop motion moving images. In chapter 3, all of the number of frames 180 of the moving image data of chapter 3 are stop motion moving images. By knowing the position of the stop motion moving image data, it becomes easy to reproduce only the portion of the stop motion moving image in the moving image data of each chapter, or to reproduce only the portion other than the stop motion moving image.

  In each still image file recorded in the moving image still image shooting mode, the operation state of each block of the imaging apparatus 100 stored in the RAM 102 in the moving image still image shooting mode before the still image recording instruction is input. Is stored (S303). The still image file also stores attribute information of the still image.

  The attribute information of the still image data specifically includes the manufacturer name and model name of the imaging device that recorded the still image data, the shooting date and time, the subject brightness that can determine overexposure and underexposure of the image, and focusing by autofocus Are the degree of focus indicating the accuracy of the image, setting information of the sub camera at the time of shooting, and the like. Also, information such as the degree of favorite indicating the priority of the image given by the user, information on whether or not the AF auxiliary light for AF was emitted at the time of shooting, and an arbitrary character string (event name) set by the user May be added and recorded as attribute information. Further, by analyzing the still image data by the image processing unit 109, information such as the number of subjects included in the still image data, the number of subjects, the position of the subject, and the size of the face may be recorded as attribute information. Other shooting settings and thumbnail images are still image data attribute information.

  On the other hand, the additional information indicating the operation state of each block of the imaging apparatus 100 before the still image recording instruction is input is information as illustrated in FIG. 6. As shown in FIG. 6, the operation state of each block of the imaging apparatus 100 before the still image recording instruction is input is acquired in predetermined time units, and additional information is generated based on the information. The time axis is set to 0 when SW2 is turned ON, and the operation state of each block of the imaging apparatus 100 before that is held for 4 seconds. FIG. 6 includes posture information of the camera 100, information on the presence / absence of an AF operation in the first imaging unit 110, information on the presence / absence of a zoom operation, and information on the degree of smile of the subject of the main camera as subject information. There may be items other than these as the operation state of each block of the imaging apparatus 100. For example, the frames included in the moving image data are analyzed by the image processing unit 111 so that the subject included in the moving image data, the position of the subject, the size of the face, and the like are the items of the operation state of each block of the imaging apparatus 100. May be included. Such information is included in the additional information.

  In FIG. 6, it can be seen that the posture information of the camera 100 was held in the horizontal position (0 degree) until the still image was taken. In addition, it can be seen that the AF operation only operates 2 seconds before still image shooting, and the zoom operation only operates 3 seconds before. It can also be seen that the smile level of the subject gradually increases from 3 seconds before still image shooting.

  Next, operations when a moving image file and a still image file recorded in the moving image still image shooting mode are reproduced will be described with reference to FIG. The process of FIG. 7 is executed by the CPU 101 controlling each block of the imaging apparatus 100.

  In the “playback mode”, as described above, the CPU 101 controls the recording / playback unit 150 to read the file list and thumbnail image of the moving image file or still image file recorded on the recording medium 151.

(S701)
The CPU 101 controls the recording / playback unit 150 so as to read still image data of a designated still image file, and causes the image processing unit 111 to expand the read still image data. Then, the display control unit 141 is controlled to display the expanded image on the display unit 140.

(S702)
Next, the CPU 101 searches for moving image data related to the displayed still image data. The CPU 101 confirms the shooting time of the displayed still image data based on the attribute information of the still image file. Next, whether the moving image file recorded in the moving image still image shooting mode on the same day is recorded on the recording medium 151. Determine. Since a moving image file recorded in the moving image still image shooting mode is recorded so that the identifier “MDG” is included in the file name, a moving image including “MDG” in the same shooting date as the still image file is recorded. It is determined whether the file is recorded on the recording medium 151. When a moving image file including “MDG” in the file name of the same shooting date as that of the still image file is recorded on the recording medium 151, the CPU 101 includes moving image data corresponding to the still image data being displayed in the moving image file. Determine if it is. The CPU 101 acquires “chapter reference information” (FIG. 5C) of the specified moving image file, and determines whether there is a chapter that matches the shooting time of the still image data being displayed in the shooting time 524. judge.

(S703)
Next, as a result of the search in S702, the CPU 101 determines whether there is a related moving image of the still image data being displayed.

(S704)
In S703, when there is a related moving image of the still image data being displayed (Yes in S703), the CPU 101 causes the display unit 140 to display an icon indicating the presence of the related moving image in the displayed still image data. 141 is controlled. This icon may not only indicate the presence of a related video, but may also be an icon for playing back a related video by designating the icon. This icon may be, for example, the related video full play icon 802 or the corresponding video play icon 803 in FIG. The related moving image all reproduction icon 802 is an icon corresponding to an instruction to reproduce all moving image data of a moving image file that is a related moving image of the still image data being displayed. The corresponding moving image reproduction icon 803 is an icon corresponding to an instruction to reproduce the moving image data of the chapter (scene) related to the still image data being displayed in the moving image file that is the moving image related to the still image data being displayed.

(S705)
Next, the CPU 101 determines whether the icon displayed in S704 has been selected.

(S706)
When the icon displayed in S704 is selected (Yes in S705), the CPU 101 reproduces the corresponding moving image data. When the related video all playback icon 802 is selected, the recording / playback unit 150 is controlled so that all video data of the video file that is the related video of the still image data being displayed is read in order from the top. The moving image data is sequentially expanded by the image processing unit 111. Then, the display control unit 141 is controlled to display the expanded image on the display unit 140. On the other hand, when the corresponding moving image playback icon 803 is selected, the moving image data of the chapter (scene) related to the still image data being displayed in the moving image file that is the moving image related to the still image data being displayed is read out. The recording / reproducing unit 150 is controlled. Then, the image processing unit 111 is controlled to sequentially expand the read moving image data, and the display control unit 141 is controlled to display the expanded image on the display unit 140. By this processing, the user can confirm the moving image data corresponding to all moving images or related chapters (scenes) of the related moving image file. When this process ends, the process returns to S704.

(S707)
Next, when it is determined in S703 that there is no related moving image of the still image data being displayed (No in S703), and in S705, the icon displayed in S704 is not selected (No in S705). The CPU 101 determines whether or not an instruction for changing the displayed still image to an image of another still image file is input. When an instruction to change the still image being displayed to an image of another still image file is input (Yes in S707), the process returns to S701.

(S708)
On the other hand, if an instruction to change the still image being displayed to an image of another still image file has not been input (No in S707), the CPU 101 issues an instruction to change to a mode other than the playback mode or other processing. Determine if it was entered. If there is no instruction for changing the mode or other processing, the CPU 101 returns the processing to S704, and if there is an instruction for changing the mode or other processing, this processing ends.

<About chapter reference information>
In this embodiment, the “chapter reference information” is updated when shooting is performed in the moving image still image shooting mode and additional recording is performed on the moving image file. Specifically, the information on the file size (521) of the moving image file and the information on the number of frames (522) of the moving image data included in the moving image file need to be changed every time additional recording is performed. Further, in the imaging device 100, for example, when a moving image of a specific chapter in the moving image file is deleted, the information on the corresponding chapter is deleted. Also during editing, the information on the file size (521) of the moving image file and the information on the number of frames (522) of the moving image data included in the moving image file are changed based on the edited file size and the number of frames.

  However, when a moving image file shot in the moving image still image shooting mode is edited by an external editing device, the chapter reference information may not be updated. Then, the chapter reference information cannot be used to specify the chapter (scene) movie data related to the still image data being displayed, and other movie data is played back, or there is no scene. May occur.

  Therefore, in step S702, the imaging apparatus 100 according to the present exemplary embodiment controls the recording / playback unit 150 so that the CPU 101 acquires the file size in the recording medium 151 of the moving image file that is the related moving image of the still image data being displayed. . The file size in the recording medium 151 is acquired from the file entry of the corresponding moving image file on the file system. Then, the file size information of the moving image file stored in the chapter reference information is compared with the file size in the recording medium 151 to determine whether they match. If they do not match, it is determined that there is no related video (No in S703), and the icon is not displayed (FIG. 8C). Further, only the related moving image reproduction icon 802 may be displayed and the corresponding moving image reproduction icon 803 may not be displayed (FIG. 8B). In this way, at least, there is no instruction to reproduce the moving image data of the chapter (scene) related to the still image data being displayed in the moving image file that is the moving image related to the still image data being displayed. Therefore, it is not possible to reproduce moving image data of unrelated chapters (scenes).

  As described above, the imaging apparatus 100 according to the present embodiment can confirm the presence of a moving image related to a displayed still image. In addition, when the existence of a moving image related to the displayed still image becomes uncertain, it is possible to prevent display indicating that there is a moving image related to the displayed still image.

  In the present embodiment, the image pickup apparatus has been described as an example. However, any apparatus may be used as long as it includes an internal microphone and can acquire an audio signal from an external microphone. For example, a mobile phone, a smartphone, a tablet information terminal, a notebook information terminal, a computer, or the like may be used.

(Other embodiments)
The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.). Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto. A computer program for realizing the above-described embodiment is supplied to a computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code. Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers. That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which the computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the user's computer Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example. Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer. Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

Claims (11)

Playback means for playing back a moving image file and a still image file recorded on a recording medium;
Display control means for displaying an image corresponding to the still image file reproduced by the reproduction means on a display unit;
The file size in the recording medium of the moving image file related to the still image file corresponding to the image currently displayed on the display unit matches the file size of the moving image file stored in the reference information included in the moving image file. And a control means for controlling the display control means so as to display an image for instructing reproduction of a moving image file associated with a still image file corresponding to the image being displayed. Processing equipment.   The image processing apparatus according to claim 1, wherein the control unit determines the related moving image file based on shooting time information of a still image file corresponding to the image being displayed.   The image for instructing the reproduction of the moving image file related to the still image file corresponding to the image being displayed is for instructing the reproduction of only the scene related to the still image file corresponding to the image being displayed. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image.   The image for instructing the reproduction of the moving image file related to the still image file corresponding to the image being displayed also instructs the reproduction of a scene other than the scene related to the still image file corresponding to the image being displayed. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image. The control means is configured to select a scene related to a still image file corresponding to the image being displayed.
The determination is based on the shooting time information of each scene stored in the reference information of the related moving image file and the shooting time information of the still image file corresponding to the image being displayed. The image processing apparatus according to 3. A playback step of playing back a moving image file and a still image file recorded on the recording medium;
A display control step of causing the display unit to display an image corresponding to the still image file reproduced by the reproduction step;
The display step includes: a file size of the moving image file related to the still image file corresponding to the image being displayed on the display unit in the recording medium; and the file of the moving image file stored in the reference information included in the moving image file. An image processing method comprising: displaying an image for instructing reproduction of a moving image file related to a still image file corresponding to the image being displayed when the sizes match.   The image processing method according to claim 6, wherein the related moving image file is determined based on shooting time information of a still image file corresponding to the image being displayed.   The image for instructing the reproduction of the moving image file related to the still image file corresponding to the image being displayed is for instructing the reproduction of only the scene related to the still image file corresponding to the image being displayed. The image processing method according to claim 6, wherein the image processing method is an image.   The image for instructing the reproduction of the moving image file related to the still image file corresponding to the image being displayed also instructs the reproduction of a scene other than the scene related to the still image file corresponding to the image being displayed. The image processing method according to claim 6, wherein the image processing method is an image. A scene related to a still image file corresponding to the image being displayed,
The determination is based on the shooting time information of each scene stored in the reference information of the related moving image file and the shooting time information of the still image file corresponding to the image being displayed. 8. The image processing method according to 7. A program for causing a computer to execute the image processing method according to any one of claims 6 to 10.

Images