JP2018195864A - Control device, imaging device, control method, program, and storage medium - Google Patents

Abstract

To provide a control device capable of performing correct shake correction when reproducing a video to which an intentional longitudinal vibration is added.SOLUTION: A control device (100) includes; shake amount calculation means (123, 124) for calculating shake amounts in a first direction and a second direction when capturing an image; recording control means (110, 125, 126) for performing recording control relating to the shake amount calculated by the shake amount calculating means; and shaking effect means (105) for adding a shaking effect in the first direction to the video. In accordance with whether the shaking effect means adds a shaking effect to the video or not, recording control on the shake amount in the first direction is changed by the recording control means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus that records and reproduces video with intentional pitching added.

  When video is recorded using a video recording device such as a video camera, the recorded video may be shaken due to shaking of the device due to camera shake or the like. In order to correct such shake at the time of video playback, Patent Document 1 discloses a technique for acquiring a motion vector amount representing video shake and selecting and enlarging a video partial region according to the motion vector amount. Has been. Japanese Patent Application Laid-Open No. 2004-151561 discloses a technique for detecting shake of the apparatus with an angular velocity sensor during video recording, recording shake information based on the angular velocity together with the video, and correcting the shake using the shake information during video playback. ing. Patent Document 3 discloses a technique for recording a video that is intentionally pitched as one of video expressions.

JP-A-7-143380 Japanese Patent Laid-Open No. 10-42233 JP 2013-145519 A

  However, there are cases where the image pickup apparatus of the prior art cannot determine a correct shake correction amount for an image with intentional pitching added. For example, there is a possibility that the image pickup apparatus erroneously recognizes the intentionally added pitching as a hand shake and corrects the shake that should not be corrected. As a result, unnatural field angle fluctuations occur.

  Therefore, an object of the present invention is to provide a control device, an imaging device, a control method, a program, and a storage medium capable of performing correct shake correction when reproducing an image with intentional pitching. To do.

  The control device according to one aspect of the present invention includes a shake amount calculation unit that calculates a shake amount in each of the first direction and the second direction when capturing an image, and the shake amount calculated by the shake amount calculation unit. Recording control means for performing recording control on the video, and shaking effect means for adding the shaking effect in the first direction to the video, wherein the shaking control means adds the shaking effect to the video. Depending on whether or not, the recording control relating to the shake amount in the first direction is changed.

  A control device according to another aspect of the present invention includes a shaking effect unit that adds a shaking effect in a first direction to a video, a shaking period acquisition unit that acquires a period during which the shaking effect is added to the video, Metadata setting means for setting metadata relating to a period, and recording control means for recording the video and the metadata on a recording medium.

  The control device according to another aspect of the present invention includes a shake amount calculation unit that calculates a shake amount in each of the first direction and the second direction when capturing an image, and the shake calculated by the shake amount calculation unit. Recording control means for performing recording control on the amount, shaking effect means for adding the shaking effect in the first direction to the video, and shaking for calculating the shaking amount of the shaking effect added to the video by the shaking effect means Amount recording means, and the recording control means changes recording control relating to the amount of shaking depending on whether or not the shaking effect means adds the shaking effect to the video.

  An imaging device as another aspect of the present invention includes an imaging device that photoelectrically converts an optical image formed through an imaging optical system and the control device.

  A control method according to another aspect of the present invention includes a step of calculating a shake amount in each of the first direction and the second direction when capturing an image, a step of performing recording control relating to the calculated shake amount, Adding a shaking effect in the first direction to the video, and performing the recording control in the step of performing the recording control, depending on whether or not the shaking effect is added to the video. Change the recording control related to the shake amount.

  A program according to another aspect of the present invention causes a computer to execute the control method.

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the invention are described in the following embodiments.

  According to the present invention, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium capable of performing correct shake correction when reproducing an image with intentional pitching. .

It is a block diagram of the imaging device in a 1st embodiment. It is a flowchart of the video recording process in 1st Embodiment. It is a conceptual diagram of the video recording process in 1st Embodiment. It is a block diagram of the imaging device in 2nd Embodiment. It is a flowchart of the video recording process in 2nd Embodiment. It is a conceptual diagram of the video recording process in 2nd Embodiment. It is explanatory drawing of the video recording process in 2nd Embodiment. It is a block diagram of the imaging device in 3rd and 4th embodiment. It is a flowchart of the video recording process in 3rd Embodiment. It is a conceptual diagram of the video recording process in 3rd Embodiment. It is a block diagram of the imaging device in 3rd and 4th embodiment. It is a flowchart of the video reproduction process in 3rd Embodiment. It is a flowchart of the video reproduction process in 4th Embodiment. It is a block diagram of the imaging device in 5th and 6th embodiment. It is a flowchart of the video recording process in 5th and 6th embodiment. It is a conceptual diagram of the video recording process in 5th and 6th embodiment. It is a block diagram of the imaging device in 5th and 6th embodiment. It is a flowchart of the video reproduction process in 5th Embodiment. It is a flowchart of the image | video reproduction | regeneration processing in 6th Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
First, the configuration of the imaging apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of an imaging apparatus 100 (video recording apparatus such as a video camera) in the present embodiment.

  The lens unit 101 (imaging optical system) forms (images) a subject image on the image sensor 102. The image sensor 102 includes a CCD sensor, a CMOS sensor, and the like, photoelectrically converts a subject image (optical image) formed via the lens unit 101, and outputs an imaging signal. The analog signal processing unit 103 performs a predetermined process on the imaging signal output from the imaging element 102 to generate an analog imaging signal. The analog signal processing unit 103 includes, for example, a CDS (Co-related Double Sampling) circuit, an AGC (Automatic Gain Control) circuit, and the like. The camera signal processing unit 104 includes an A / D converter, and generates a digital video signal from the analog imaging signal generated by the analog signal processing unit 103.

  The pitch effect generating unit 105 generates a predetermined pitch effect (pitch amount) for the digital video signal generated by the camera signal processing unit 104. The encoding unit 106 converts the digital video signal into MPEG2 format or H.264 format. It is encoded into video data in the H.264 format and output to the memory 107. The memory 107 is a volatile recording medium for storing video data, and each block can be used for work. The switch operation unit 108 includes a plurality of switches operated by the user. The switch operation unit 108 includes a switch for instructing start / end of video recording, and receives the instruction and inputs it to the microcomputer 120. In accordance with the instruction to start / end video recording, the microcomputer 120 outputs video and metadata to the recording medium 111 in accordance with the operation shown in FIG. The angular velocity sensor 109 detects the angular velocity of shake applied to the imaging device 100.

  The recording control unit 110 has interfaces with a pitching effect generation unit 105, an encoding unit 106, a memory 107, a recording medium 111, and a microcomputer 120, and controls data transfer. The recording medium 111 is a recording medium for video data and metadata, and includes a built-in memory such as a hard disk, SSD, or eMMC, or a removable external memory such as an SD card, CF card, or CFast card.

  Based on the output of the angular velocity sensor 109, the angular velocity shake amount acquisition unit 122 calculates an angular velocity shake amount (angular velocity information) that is a shake amount of the imaging device 100. Then, the angular velocity shake amount acquisition unit 122 outputs the vertical and horizontal angular velocity information (vibration information separated into the vertical direction and the horizontal direction) to the vertical shake amount calculation unit 123 and the horizontal shake amount calculation unit 124, respectively. The motion vector amount acquisition unit 121 acquires a motion vector amount from the video stored in the memory 107. Then, the motion vector amount acquisition unit 121 sends the vertical and horizontal motion vector amounts (information on the motion vectors separated into the vertical direction and the horizontal direction) to the vertical shake amount calculation unit 123 and the horizontal shake amount calculation unit 124, respectively. Output.

  The vertical shake amount calculation unit 123 performs vertical motion based on the vertical angular velocity shake amount (angular velocity information) output from the angular velocity shake amount acquisition unit 122 and the vertical motion vector amount output from the motion vector amount acquisition unit 121. Calculate the shake amount. The vertical shake amount recording unit 125 records the vertical shake amount calculated by the vertical shake amount calculation unit 123. The lateral shake amount calculation unit 124 generates a lateral motion based on the lateral angular velocity shake amount (angular velocity information) output from the angular velocity shake amount acquisition unit 122 and the horizontal motion vector amount output from the motion vector amount acquisition unit 121. Calculate the shake amount. The lateral shake amount recording unit 126 records the lateral shake amount calculated by the lateral shake amount calculating unit 123. The vertical shake amount recording unit 125 outputs metadata including the data set by the vertical shake amount calculation unit 123 to the memory 107 via the recording control unit 110. The lateral shake amount recording unit 126 outputs metadata including the data set by the lateral shake amount calculation unit 124 to the memory 107 via the recording control unit 110.

  Next, the operation (video recording process) of the imaging apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart of the video recording process in the present embodiment. Each step in FIG. 2 is mainly executed by the microcomputer 120 and the recording control unit 110.

  First, in step S201, the vertical shake amount calculation unit 123 and the horizontal shake amount calculation unit 124 calculate the vertical shake amount and the horizontal shake amount based on the angular velocity shake amount (angular velocity information) and the motion vector amount, respectively. Subsequently, in step S202, the recording control unit 110 determines whether a pitching effect is generated by the pitching effect generation unit 105, that is, whether there is a pitching amount. If the pitching effect has been generated, the process proceeds to step S203. On the other hand, if the pitching effect is not generated, the process proceeds to step S204. In step S203, the recording control unit 110 (or the microcomputer 120) sets the vertical shake amount to zero. In step S <b> 204, the recording control unit 110 outputs the video and metadata including the vertical shake amount and the horizontal shake amount to the recording medium 111.

  Subsequently, in step S <b> 205, the recording control unit 110 determines whether or not a video recording end instruction has been received from the microcomputer 120. If the end of video recording has not been instructed, the process returns to step S201. On the other hand, when the end of video recording is instructed, the process proceeds to step S206. In step S206, the recording control unit 110 closes the video and metadata output to the recording medium 111 as one video data.

  Next, the concept of the video recording process in this embodiment will be described with reference to FIG. FIG. 3 is a conceptual diagram of the video recording process. In FIG. 3, reference numeral 301 denotes video data (video and metadata) output to the recording medium 111. First, in the section T302, since no vertical shake is generated, the vertical shake amount set in step S201 in FIG. 2 is recorded as metadata. Subsequently, since vertical pitch is generated in the section T303, the vertical shake amount set in step S203 in FIG. 2 (that is, zero vertical shake amount) is recorded as metadata. Subsequently, in the section T304, as in the section T302, since no pitch is generated, the amount of vertical shake set in step S201 in FIG. 2 is recorded as metadata. Finally, the video and metadata output to the recording medium 111 are closed as one video data.

  As described above, in the present embodiment, the control device includes a shake amount calculation unit (vertical shake amount calculation unit 123, horizontal shake amount calculation unit 124), a recording control unit (vertical shake amount recording unit 125, lateral shake amount recording unit 126, A recording control unit 110) and a shaking effect means (pitch effect generating unit 105). The shake amount calculation means calculates the shake amount in each of the first direction (vertical direction) and the second direction (horizontal direction) when capturing an image. The recording control unit performs recording control related to the shake amount calculated by the shake amount calculation unit. The shaking effect means adds a shaking effect in the first direction to the video. The recording control means changes the recording control related to the shake amount in the first direction depending on whether the shaking effect means adds a shaking effect to the video.

  Preferably, the recording control means records the shake amount in the first direction calculated by the shake amount calculation means on the recording medium (memory 107) when the shake effect means does not add a shake effect to the video. On the other hand, when the shake effect means adds a shake effect to the video, the shake amount in the first direction calculated by the shake amount calculation means is reduced and recorded on the recording medium. More preferably, the recording control means records the amount of shake in the first direction calculated by the shake amount calculation means as 0 when the shake effect means adds a shake effect to the video.

  According to the present embodiment, by recording shake information (pitch amount) considering the generated pitch effect, correct shake correction can be performed when playing back an image.

<Second Embodiment>
Next, with reference to FIG. 4, the configuration of the imaging apparatus according to the second embodiment of the present invention will be described. FIG. 4 is a block diagram of the imaging apparatus 400 (video recording apparatus) in the present embodiment. The imaging apparatus 400 is different from the imaging apparatus 100 in the first embodiment in that the imaging apparatus 400 includes a microcomputer 420 having a pitching amount acquisition unit 401 instead of the microcomputer 120. Since the other configuration of the imaging apparatus 400 is the same as that of the imaging apparatus 100, description thereof is omitted.

  The pitch effect generation unit 105 generates a predetermined pitch effect (pitch amount) for the digital video signal, and outputs the generated pitch amount to the pitch amount acquisition unit 401. The pitching amount acquisition unit 401 outputs the pitching amount acquired from the pitching effect generation unit 105 to the pitching amount calculation unit 123. The vertical shake amount calculation unit 123 outputs the vertical angular velocity shake amount output from the angular velocity shake amount acquisition unit 122 and the vertical motion vector amount output from the motion vector amount acquisition unit 121 and the vertical shake amount acquisition unit 401. The vertical shake amount is calculated based on the vertical pitch amount (effect of the vertical shake). The vertical shake amount recording unit 125 records the vertical shake amount calculated by the vertical shake amount calculation unit 123.

  Next, with reference to FIG. 5, the operation (video recording process) of the imaging apparatus 400 in the present embodiment will be described. FIG. 5 is a flowchart of the video recording process in the present embodiment. Each step in FIG. 5 is mainly executed by the microcomputer 420 and the recording control unit 110.

  First, in step S501, the vertical shake amount calculation unit 123 and the horizontal shake amount calculation unit 124 calculate the vertical shake amount and the horizontal shake amount based on the angular velocity shake amount (angular velocity information) and the motion vector amount, respectively. Subsequently, in step S502, the recording control unit 110 determines whether a pitching effect is generated by the pitching effect generation unit 105, that is, whether there is a pitching amount. If the pitching effect has been generated, the process proceeds to step S503. On the other hand, when the pitching effect is not generated, the process proceeds to step S504. In step S503, the vertical shake amount calculation unit 123 sets the amount obtained by subtracting the vertical shake amount acquired by the vertical shake amount acquisition unit 401 from the vertical shake amount calculated in step S501 as the vertical shake amount. In step S <b> 504, the recording control unit 110 outputs, to the recording medium 111, the video that has been instructed to start recording and metadata that includes the vertical shake amount and the horizontal shake amount.

  Subsequently, in step S <b> 505, the recording control unit 110 determines whether or not a video recording end instruction has been received from the microcomputer 420. If the end of video recording has not been instructed, the process returns to step S501. On the other hand, if the end of video recording is instructed, the process proceeds to step S506. In step S506, the recording control unit 110 closes the video and metadata output to the recording medium 111 as one video data.

  Next, the concept of the video recording process in this embodiment will be described with reference to FIG. FIG. 6 is a conceptual diagram of the video recording process. In FIG. 6, reference numeral 601 denotes video data (video and metadata) output to the recording medium 111. First, in the section T602, since no vertical shake is generated, the vertical shake amount set in step S501 in FIG. 5 is recorded as metadata. Subsequently, since pitch is generated in the section T603, the value (vertical shake amount) calculated by subtracting the pitch amount from the vertical shake amount in step S503 in FIG. 5 is recorded as metadata. . Subsequently, in the section T604, as in the section T602, since the vertical shake is not generated, the vertical shake amount set in step S501 in FIG. 5 is recorded as metadata. Finally, the video and metadata output to the recording medium 111 are closed as one video data.

  Next, the video recording process (step S503 in FIG. 5) in the present embodiment will be described with reference to FIG. FIG. 7 is an explanatory diagram of the video recording process. Each of the vertical shake amount and the vertical shake amount is an amount indicating how many pixels the pixels of the current frame have moved in the vertical direction from the previous frame. By subtracting the vertical shake amount from the calculated vertical shake amount, the vertical shake amount recorded as metadata can be calculated.

  In the present embodiment, the amount of vertical shake recorded as metadata can be calculated as follows.

Vertical shake amount−Vertical shake amount = Vertical shake amount recorded as metadata In the example of C701 in FIG. 7, when the calculated vertical shake amount is −8 pixels and the vertical shake amount is −5 pixels, −8− ( Since −5) = − 3, the pitch amount to be recorded is minus 3 pixels. In this case, the image has moved minus 8 pixels in the vertical direction. For this reason, if reproduction anti-vibration is performed for minus 3 pixels, the video moves by minus 5 pixels, and the intended amount of pitching can be obtained.

  Similarly, in the example of C702, when the calculated vertical shake amount is −5 pixels and the vertical shake amount is −8 pixels, −5 − (− 8) = 3, so the vertical shake amount to be recorded is +3 pixels. It becomes. In the example of C703, when the calculated vertical shake amount is plus 8 pixels and the pitch amount is plus 5 pixels, 8−5 = 3, and thus the pitch amount to be recorded is plus 3 pixels. In the example of C704, when the calculated vertical shake amount is plus 5 pixels and the pitch amount is plus 8 pixels, since 5-8 = −3, the recorded pitch shake amount is minus 3 pixels. In the example of C705, when the calculated vertical shake amount is minus 8 pixels and the vertical shake amount is plus 5 pixels, −8−5 = −13, so the vertical shake amount to be recorded is minus 13 pixels. In the example of C706, when the calculated vertical shake amount is minus 5 pixels and the pitch amount is plus 8 pixels, −5−8 = −13, and thus the pitch amount to be recorded is minus 13 pixels. In the example of C707, when the calculated vertical shake amount is plus 8 pixels and the pitch shake amount is minus 5 pixels, since 8-(− 5) = 13, the pitch shake amount to be recorded is plus 13 pixels. In the example of C708, when the calculated vertical shake amount is plus 5 pixels and the pitch amount is minus 8 pixels, since 5-(− 8) = 13, the pitch amount to be recorded is plus 13 pixels.

  Thus, in this embodiment, in addition to the control device of the first embodiment, the control device acquires a shake amount acquisition unit (pitch amount acquisition) that acquires a shake amount of the shake effect added to the video by the shake effect unit. Part 401). The recording control means records the amount obtained by subtracting the shake amount from the shake amount in the first direction calculated by the shake amount calculation means on the recording medium when the shake effect means adds a shake effect to the video. To do.

  According to the present embodiment, by recording shake information (pitch amount) considering the generated pitch effect, correct shake correction can be performed when playing back an image.

<Third Embodiment>
Next, with reference to FIG. 8, the structure of the imaging device in the 3rd Embodiment of this invention is demonstrated. FIG. 8 is a block diagram of the imaging apparatus 800 (video recording apparatus) in the present embodiment. The imaging apparatus 800 is different from the imaging apparatus 100 in the first embodiment in that the angular velocity sensor 109 is not provided and that the microcomputer 820 is provided instead of the microcomputer 120. Since the other configuration of the imaging apparatus 800 is the same as that of the imaging apparatus 100, description thereof is omitted.

  The microcomputer 820 includes a pitch period acquisition unit 821 and a metadata setting unit 822. The pitch effect generating unit 105 generates a predetermined pitch effect (pitch amount) for the digital video signal. The pitching period acquisition unit 821 acquires a period (pitch period) during which a pitching effect (pitch amount) is generated from the pitching effect generation unit 105.

  The recording control unit 110 has interfaces with a pitching effect generation unit 105, an encoding unit 106, a memory 107, a metadata setting unit 822, and a recording medium 111, and exchanges data under the control of the microcomputer 820. Control. The memory 107 functions as a buffer that holds the video encoded by the encoding unit 106 for several seconds. The memory 107 also functions as a buffer that holds metadata output from a metadata setting unit 822 described later. The switch operation unit 108 is an operation unit including a plurality of switches operated by a user, includes a switch for instructing start / end of video recording, receives the instruction, and inputs the instruction to the microcomputer 112. In response to the instruction to start / end video recording, the microcomputer 820 outputs video and metadata to the recording medium 111 in accordance with a flowchart of FIG.

  The pitch period acquisition unit 821 acquires a period in which the pitch effect is applied to the digital video signal from the pitch effect generation unit 105 and outputs (sets) the period to the metadata setting unit 822. The metadata setting unit 822 outputs metadata including the set data to the memory 107 via the recording control unit 110.

  Next, with reference to FIG. 9, the operation (video recording process) of the imaging apparatus 800 in the present embodiment will be described. FIG. 9 is a flowchart of the video recording process in the present embodiment. Each step in FIG. 9 is mainly executed by the microcomputer 820 and the recording control unit 110.

  First, in step S <b> 901, the microcomputer 820 (pitch period acquisition unit 821 and metadata setting unit 822) determines whether or not pitching is intentionally applied by the pitching effect generation unit 105. If intentional shaking is applied, the process proceeds to step S902. In step S902, the microcomputer 820 sets the variable Flag to be recorded in the metadata as the pitch application period to “1”. On the other hand, if no intentional shaking is applied in step S901, the process proceeds to step S903. In step S903, the microcomputer 820 sets the variable Flag recorded in the metadata as the pitching imparting period to “0”.

  Subsequently, in step S <b> 904, the recording control unit 110 outputs the value of the variable Flag to the recording medium 111 as metadata indicating the video and the pitch application period.

  Subsequently, in step S <b> 905, the recording control unit 110 determines whether an instruction to end video recording has been received from the microcomputer 820. If the end of video recording has not been instructed, the process returns to step S901. On the other hand, if the end of video recording is instructed, the process proceeds to step S906. In step S906, the recording control unit 110 closes the video and metadata output to the recording medium 111 as one video data.

  Next, the concept of the video recording process in this embodiment will be described with reference to FIG. FIG. 10 is a conceptual diagram of the video recording process. In FIG. 10, reference numeral 1001 denotes video data (video and metadata) output to the recording medium 111. First, in section T1002, since there is no intentional pitching, the value 0 set in step S903 in FIG. 9 is recorded as metadata. Subsequently, in section T1003, since there is intentional pitching, the value 1 set in step S902 in FIG. 9 is recorded as metadata. Subsequently, in the section T1004, as in the section T1002, since there is no intentional pitching, the value 0 set in step S903 in FIG. 9 is recorded as metadata. Finally, the video and metadata output to the recording medium 111 are closed as one video data.

  Next, with reference to FIG. 11, the configuration of the video playback apparatus in the present embodiment will be described. FIG. 11 is a block diagram of the imaging apparatus 1100 (video playback apparatus) in the present embodiment. Video data to which the metadata described with reference to FIG. 8 is added is recorded on the recording medium 111. However, the recording format of the metadata and the video data is not limited to this. For example, the metadata may be recorded separately from the video data.

  The playback control unit 1110 acquires video and metadata from the recording medium 111. The decoding unit 1101 decodes the video data (in this embodiment, an H.264 stream) acquired by the reproduction control unit 1110, and outputs the decoded data to the memory 107 together with the decoded digital video signal metadata. The memory 107 stores the decoded digital video signal for several seconds. By storing a digital video signal for several seconds in the memory 107, it becomes possible to obtain information that is temporally ahead of the reproduced image (image being reproduced) output to the display unit 1103.

  The image processing unit 1102 reproduces the video data according to the cutout position and cutout size determined by the processing of the microcomputer 1120 described later. Specifically, the image processing unit 1102 performs electronic zoom on a predetermined image from a predetermined cutout position of the unit image for each of a plurality of unit images (field image or frame image) whose video data constitutes a display video. And output to the display unit 1103. The display unit 1103 displays video by sequentially displaying the output images.

  The joystick 1104, the play / pause button 1105, and the stop button 1106 are operation units, respectively, that receive a user operation and play a role of notifying the microcomputer 1120 of a control instruction. The joystick 1104 gives an instruction to move a cursor for selecting an item in the screen up, down, left, and right. The joystick 1104 can be pressed in the vertical direction, selects an item on which the cursor is placed on the screen, and gives instructions for confirming various setting operations. The playback / pause button 1105 can be started or paused by being pressed when playing back video. A stop button 1106 is used to end the video reproduction.

  The microcomputer 1120 includes a motion vector amount acquisition unit 121, a metadata reading unit 1122, a shake correction amount determination unit 1123, and a pitch amount setting unit 1124, and controls the processing shown in the flowchart of FIG. . The motion vector amount acquisition unit 121 acquires a motion vector amount from the digital image stored in the memory 107. The amount of motion vector serves as an index of image shake. The metadata reading unit 1122 reads metadata indicating the pitch application period from the memory 107. The pitch amount setting unit 1124 sets the pitch amount when Flag = 1 based on the metadata indicating the pitch grant period. Based on the motion vector amount acquired by the motion vector amount acquisition unit 121 and the vertical shake amount set by the vertical shake amount setting unit 1124, the shake correction amount determination unit 1123 cuts out to correct the video shake. Determine position and size. Then, the shake correction amount determination unit 1123 outputs (sets) the determined cutout and size to the image processing unit 1102.

  Next, with reference to FIG. 12, the video reproduction process of the present embodiment will be described. FIG. 12 is a flowchart of the video reproduction process. Each step in FIG. 12 is mainly executed by the microcomputer 1120 and the image processing unit 1102.

  First, in step S1201, the microcomputer 1120 causes the display unit 1103 to display a user interface (UI) for video playback processing. Subsequently, in step S1202, the microcomputer 1120 determines whether or not an instruction to start video playback has been received. If an instruction to start video playback is given, the encoded video recorded on the recording medium 111 is decoded and displayed on the display unit 1103, and the process advances to step S1203. On the other hand, if the start of video reproduction is not instructed, the process returns to step S1201.

  In step S1203, the microcomputer 1120 determines whether or not it is a pitching grant period based on the information regarding the pitching grant period read from the metadata reading unit 1122. If it is not the pitching period (Flag = 0), the process proceeds to step S1204. On the other hand, if it is the pitching period (Flag = 1), the process proceeds to step S1205.

  In step S <b> 1204, the shake correction amount determination unit 1123 determines the shake correction amount based on the motion vector amount acquired by the motion vector amount acquisition unit 121. In step S505, the shake correction amount determination unit 1123 determines the shake correction amount based on the motion vector amount acquired by the motion vector amount acquisition unit 121 and the pitch amount set by the pitch amount setting unit 1124. To do.

  Subsequently, in step S1206, the shake correction amount determination unit 1123 determines the cutout position and cutout size from the video based on the shake correction amount. Subsequently, in step S1207, the cut-out position and cut-out size of the playback frame in the video determined in step S1206 are set in the image processing unit 1102, and a video obtained by performing electronic zoom on the predetermined image from the cut-out position is displayed on the display unit. 1103.

  As described above, in the present embodiment, the control device includes a shaking effect unit (pitch effect generation unit 105), a shaking period acquisition unit (pitch period acquisition unit 821), a metadata setting unit (metadata setting unit 822), and And a recording control means (recording control unit 110). The shaking effect means adds a shaking effect in the first direction to the video. The shaking period acquisition unit acquires a period (pitch period) in which a shaking effect is added to the video. The metadata setting means sets metadata relating to the pitching period. The recording control means (recording control unit 110) records video and metadata on a recording medium (memory 107).

  Preferably, the control device further includes a motion vector acquisition unit (motion vector amount acquisition unit 121), a shake amount setting unit (pitch amount setting unit 1124), a determination unit (shake correction amount determination unit 1123), and image processing. Means (image processing unit 1102). The motion vector acquisition means acquires a motion vector amount from the video. The shaking amount setting means sets the shaking amount of the shaking effect on the video based on the metadata. The determining means determines a shake correction amount of the video based on the motion vector amount. The image processing means cuts out a part of the video from the video based on the shake correction amount (cuts out the video from the video data). The determining means determines the shake correction amount based on the motion vector amount and the shake amount set by the shake amount setting means during the period in which the shake effect is added to the video (S1205).

  According to the present embodiment, correct shake correction can be performed at the time of reproduction by adding the pitch amount in the shake correction process based on the information about the pitch period recorded in the metadata.

<Fourth Embodiment>
Next, with reference to FIG. 13, a video reproduction process according to the fourth embodiment of the present invention will be described. Note that the basic configuration and operation of the imaging apparatus of the present embodiment are the same as those of the imaging apparatus of the third embodiment. FIG. 13 is a flowchart of video playback processing in the present embodiment. Each step of FIG. 13 is mainly executed by the microcomputer 820 and the image processing unit 1102. Note that steps S1301 to S1303, S1309, S1310, S1312, and S1313 in FIG. 13 are the same as steps S1201 to S1207 in FIG.

  If the microcomputer 1120 determines that it is the pitch application period (Flag = 1) based on the information about the pitch application period read from the metadata reading unit 1122 in step S1303, the process proceeds to step S1304. In step S1304, the microcomputer 1120 determines whether or not the predetermined image cut out based on the shake correction amount of the previous frame image held in the memory 107 in step S1311 described later includes the upper end or the lower end of the frame image. Determine. When the predetermined image includes the upper end or the lower end of the frame image, the process proceeds to step S1306. On the other hand, if the predetermined image does not include either the upper end or the lower end of the frame image, the process proceeds to step S1305.

  In step 1305, the microcomputer 1120 sets a normal value (pitch amount: medium (first shake amount)) as the pitch amount. In step S <b> 1306, the microcomputer 1120 matches the longitudinal component of the motion vector amount acquired by the motion vector amount acquisition unit 121 with the pitch direction applied (added) by the pitch amount setting unit 1124. If yes, the process proceeds to step S1307. On the other hand, if these directions do not match each other, the process proceeds to step S1308.

  Subsequently, in step S1307, the microcomputer 1120 sets a value larger than the normal pitch amount (pitch amount: large (second swing amount)). On the other hand, in step S1308, the microcomputer 1120 sets a value smaller than the normal pitch amount (pitch amount: small (third swing amount)). In step S1310, a shake correction amount is determined based on the pitch amount obtained as described above and the motion vector amount obtained by the motion vector amount obtaining unit 121. Subsequently, in step S1311, the determined shake correction amount is held in the memory 107.

  As described above, in this embodiment, the swing amount setting means selectively sets the first swing amount, the second swing amount larger than the first swing amount, and the third swing amount smaller than the first swing amount. Is possible. For example, the shaking amount setting means sets the first shaking amount (pitch amount: medium) as the shaking amount when the image cut out by the image processing means includes the upper end or the lower end of the image before being cut out (S1305). . In addition, when the direction of the motion vector amount and the direction of the amount of shake coincide with each other, the shake amount setting means sets the second shake amount (pitch amount: large) as the shake amount (S1307). Further, when the direction of the motion vector amount and the direction of the swing amount do not match each other, the swing amount setting means sets a third swing amount (pitch amount: small) as the swing amount (S1308).

  According to the present embodiment, correct shake correction can be performed at the time of reproduction by adding the pitch amount in the shake correction process based on the information about the pitch period recorded in the metadata.

<Fifth Embodiment>
Next, with reference to FIG. 14, the structure of the imaging device in the 5th Embodiment of this invention is demonstrated. FIG. 14 is a block diagram of an imaging apparatus 1400 (video recording apparatus) in the present embodiment. The imaging apparatus 1400 is different from the imaging apparatus 100 in the first embodiment in that the imaging apparatus 1400 includes a microcomputer 1420 having a pitching amount calculation unit 1421 and a pitching amount recording unit 1422 instead of the microcomputer 120. Since other configurations of the imaging device 1400 are the same as those of the imaging device 100, descriptions thereof are omitted.

  The pitching amount calculation unit 1421 calculates the pitching amount based on the information output from the pitching effect generation unit 105. The pitch amount recording unit 1422 outputs metadata including data set by the pitch amount calculation unit 1421 to the memory 107 via the recording control unit 110. The microcomputer 1420 controls processing shown in a flowchart of FIG.

  Next, with reference to FIG. 15, the operation (video recording process) of the imaging apparatus 1400 in the present embodiment will be described. FIG. 15 is a flowchart of video recording processing in the present embodiment. Each step in FIG. 15 is mainly executed by the microcomputer 1420 and the recording control unit 110.

  First, in step S1501, the vertical shake amount calculation unit 123 and the horizontal shake amount calculation unit 124 calculate the vertical shake amount and the horizontal shake amount based on the angular velocity shake amount (angular velocity information) and the motion vector amount, respectively. In step S1502, the recording control unit 110 determines whether a pitching effect is generated by the pitching effect generation unit 105, that is, whether there is a pitching amount. If the pitching effect has been generated, the process proceeds to step S1503. On the other hand, if the pitching effect has not been generated, the process proceeds to step S1504. In step S <b> 1503, the recording control unit 110 (or the microcomputer 1420) outputs video and metadata including the vertical shake amount, the horizontal shake amount, and the vertical shake amount to the recording medium 111. In step S <b> 1504, the recording control unit 110 (or the microcomputer 1420) outputs the video and metadata including the vertical shake amount and the horizontal shake amount to the recording medium 111.

  In step S <b> 1505, the recording control unit 110 determines whether a video recording end instruction has been received from the microcomputer 1420. If the end of video recording has not been instructed, the process returns to step S1501. On the other hand, if the end of video recording is instructed, the process advances to step S1506. In step S1506, the recording control unit 110 closes the video and metadata output to the recording medium 111 as one video data.

  Next, the concept of the video recording process in this embodiment will be described with reference to FIG. FIG. 16 is a conceptual diagram of video recording processing. In FIG. 16, reference numeral 1601 denotes video data (video and metadata) output to the recording medium 111. First, in the section T1602, since no vertical shake is generated, the video image and the vertical shake amount and the horizontal shake amount set in step S1504 in FIG. 15 are recorded as metadata. Subsequently, since pitch is generated in the section T1603, the video, the vertical shake amount, the horizontal shake amount, and the vertical shake amount set in step S1503 in FIG. 15 are recorded as metadata. Subsequently, in the section T1604, as in the section T1602, no vertical shaking is generated, so the video, the vertical shake amount, and the horizontal shake amount set in step S1504 in FIG. 2 are recorded as metadata. . Finally, the video and metadata output to the recording medium 111 are closed as one video data.

  Next, with reference to FIG. 17, the configuration of the video reproduction device in the present embodiment will be described. FIG. 17 is a block diagram of an imaging apparatus 1700 (video playback apparatus) according to this embodiment. Video data with the metadata described with reference to FIG. 14 is recorded on the recording medium 111. However, the recording format of the metadata and the video data is not limited to this. For example, the metadata may be recorded separately from the video data.

  The decoding unit 1101 decodes the video data acquired by the reproduction control unit 1110 (in this embodiment, an H.264 stream), and outputs the decoded data to the memory 107 together with the metadata of the decoded digital video signal. The image processing unit 1102 reproduces video data according to the cutout position and cutout size determined by the processing of the microcomputer 1720 described later. Specifically, the image processing unit 1102 performs electronic zoom on a predetermined image from a predetermined cutout position of the unit image for each of a plurality of unit images (field image or frame image) whose video data constitutes a display video. And output to the display unit 1103. The display unit 1103 displays video by sequentially displaying the images output from the image processing unit 1102.

The reproduction control unit 1110 has interfaces with the recording medium 111, the decoding unit 1101, the image processing unit 1102, the display unit 1103, the memory 107, and the microcomputer 1720, and controls data transfer. The memory 107 stores the decoded digital video signal for several seconds. By storing a digital video signal for several seconds in the memory 107, it becomes possible to obtain information that is temporally ahead of the reproduced image (image being reproduced) output to the display unit 1103. The switch operation unit 108 is an operation unit including a plurality of switches operated by the user, includes a switch for instructing start / end of video reproduction, receives the instruction, and inputs the instruction to the microcomputer 1720. In accordance with the instruction to start / end video playback, the microcomputer 1720 plays back video and metadata in accordance with the operation of FIG.
The microcomputer 1720 includes a motion vector amount acquisition unit 121, a shake correction amount determination unit 1123, a vertical shake amount acquisition unit 1721, a lateral shake amount acquisition unit 1722, and a vertical shake amount acquisition unit 1723. The flowchart of FIG. The process shown in FIG. The motion vector amount acquisition unit 121 acquires a motion vector amount from the digital image stored in the memory 107. The amount of motion vector serves as an index of image shake. The vertical shake amount acquisition unit 1721 reads metadata indicating the vertical shake amount from the memory 107. The lateral shake amount acquisition unit 1722 reads metadata indicating the lateral shake amount from the memory 107. The pitch amount acquisition unit 1723 reads metadata indicating the pitch amount from the memory 107.

  The shake correction amount determination unit 1123 includes the motion vector amount acquired by the motion vector amount acquisition unit 121, the vertical shake amount acquired by the vertical shake amount acquisition unit 1721, the horizontal shake amount acquired by the horizontal shake amount acquisition unit 1722, and the vertical shake amount. The pitching amount acquired by the shaking amount acquisition unit 1723 is output. The shake correction amount determination unit 1123 determines the cutout position and size so as to correct the shake of the video based on the motion vector amount, the vertical shake amount, the horizontal shake amount, and the vertical shake amount, and these are determined as the image processing unit. Output (set) to 1102.

  Next, with reference to FIG. 18, the video reproduction process of the present embodiment will be described. FIG. 18 is a flowchart of the video reproduction process. Each step of FIG. 18 is mainly executed by the microcomputer 1720, the reproduction control unit 1110, and the image processing unit 1102.

  First, in step S1801, the playback control unit 1110 uses the decoding unit 1101 to decode the encoded video recorded on the recording medium 111. The motion vector amount acquisition unit 121 acquires a motion vector amount. Subsequently, in step S1802, the microcomputer 1720 determines whether or not pitching is recorded based on the pitching amount acquired by the pitching amount acquisition unit 1723. If it is determined that the pitch is recorded, the process advances to step S1803. On the other hand, if it is determined that no pitch is recorded, the process advances to step S1804.

  In step S1803, the shake correction amount determination unit 1123 determines the shake correction amount based on the horizontal shake amount acquired by the horizontal shake amount acquisition unit 1722. In step S1804, the shake correction amount determination unit 1123 determines the shake correction amount based on the vertical shake amount acquired by the vertical shake amount acquisition unit 1721 and the horizontal shake amount acquired by the horizontal shake amount acquisition unit 1722. To do.

  Subsequently, in step S1805, the shake correction amount determination unit 1123 determines the cut-out position and size from the video based on the motion vector amount acquired in step S1801 and the shake correction amount determined in step S1803 or step S1804. To decide. The determined cutout position and size are output to the image processing unit 1102 and set. In step S1806, the image processing unit 1102 displays on the display unit 1103 a video obtained by electronically zooming a predetermined image based on the cutout position and cutout size of the playback frame in the video determined in step S1805. In step S1807, the microcomputer 1720 determines an instruction to end the reproduction of the video. When an instruction to end video playback is given, the processing of this flow ends.

  As described above, in the present embodiment, the control device includes a shake amount calculation unit (vertical shake amount calculation unit 123, lateral shake amount calculation unit 124), and a recording control unit (vertical shake amount recording unit 125, lateral shake amount recording unit). 126, a recording control unit 110). In addition, the control device includes a shaking effect unit (pitch effect generation unit 105) and a shaking amount calculation unit (pitch amount calculation unit 1421). The shake amount calculation means calculates the shake amount in each of the first direction (vertical direction) and the second direction (horizontal direction) when capturing an image. The recording control unit performs recording control related to the shake amount calculated by the shake amount calculation unit. The shaking effect means adds a shaking effect in the first direction to the video. The shaking amount calculating means calculates the shaking amount of the shaking effect added to the video by the shaking effect means. The recording control means changes the recording control related to the shaking amount depending on whether or not the shaking effect means adds a shaking effect to the video.

  Preferably, the recording control means does not record the shaking amount on the recording medium (memory 107) when the shaking effect means does not add a shaking effect to the video. On the other hand, the recording control means records the amount of shaking on the recording medium when the shaking effect means adds a shaking effect to the video. More preferably, the recording control means records the video and the respective shake amounts in the first direction and the second direction on the recording medium when the shaking effect means does not add the shaking effect to the video. On the other hand, when the shaking effect means adds a shaking effect to the video, the recording control means records the video, the shake amounts in the first direction and the second direction, and the shaking amount on the recording medium.

  Preferably, the control device includes a motion vector acquisition unit (motion vector amount acquisition unit 121) and a shake amount acquisition unit (vertical shake amount acquisition unit 1721, lateral shake amount acquisition unit 1722). The control device also includes a shake amount acquisition unit (pitch amount acquisition unit 1723), a determination unit (shake correction amount determination unit 1123), and an image processing unit (image processing unit 1102). The motion vector acquisition means acquires a motion vector amount from the video. The shake amount acquisition unit acquires the shake amounts in the first direction and the second direction from the recording medium. The shaking amount acquisition means acquires the shaking amount from the recording medium. The determining means determines a shake correction amount for correcting the shake of the video based on the motion vector amount, the shake amounts in the first direction and the second direction, and the shake amount. The image processing means cuts out a part of the video from the video based on the shake correction amount. The determining means changes the shake correction amount determining method according to whether or not the shake amount acquiring means has acquired the shake amount. More preferably, the determination unit determines the shake correction amount based on the motion vector amount and each shake amount in the first direction and the second direction when the shake amount acquisition unit does not acquire the shake amount (S1803). ). On the other hand, when the shake amount acquisition unit acquires the shake amount, the determination unit determines the shake correction amount based on the motion vector amount and the shake amount in the second direction (S1804).

  According to the present embodiment, by calculating the shake correction amount based on the vertical shake amount recorded in the metadata, correct shake correction can be performed during reproduction.

<Sixth Embodiment>
Next, with reference to FIG. 19, a video reproduction process according to the sixth embodiment of the present invention will be described. Note that the basic configuration and operation of the imaging apparatus of the present embodiment are the same as those of the imaging apparatus of the fifth embodiment. FIG. 19 is a flowchart of video playback processing in the present embodiment. Each step of FIG. 19 is mainly executed by the microcomputer 1720, the reproduction control unit 1110, and the image processing unit 1102. Note that steps S1901, S1902, and S1904 to S1907 in FIG. 13 are the same as steps S1801, S1802, and S1804 to S1807 in FIG.

  In step S1903, the shake correction amount determination unit 1123 determines the vertical shake amount acquired by the vertical shake amount acquisition unit 422, the horizontal shake amount acquired by the horizontal shake amount acquisition unit 423, and the vertical shake amount acquisition unit 424. The shake correction amount is determined on the basis of the pitch amount acquired in step (1). Here, when the vertical shake amount and the vertical shake amount are in the same direction, the respective amounts are added. On the other hand, when the vertical shake amount and the vertical shake amount are opposite to each other, a value obtained by subtracting the other amount from one amount is set as the vertical shake correction amount.

  As described above, in the present embodiment, the determination unit (the shake correction amount determination unit 1123) determines the motion vector amount, the first direction (vertical direction), and the second direction (when the shake amount acquisition unit does not acquire the shake amount). A shake correction amount is determined based on each shake amount in the horizontal direction (S1904). On the other hand, when the shake amount acquisition means acquires the shake amount, the determination means determines the shake correction amount based on the motion vector amount, the shake amounts in the first direction and the second direction, and the shake amount ( S1903).

  According to the present embodiment, by calculating the shake correction amount based on the vertical shake amount recorded in the metadata, correct shake correction can be performed during reproduction.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  According to each embodiment, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium capable of performing correct shake correction when playing a video with intentional pitching. it can.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

105 Pitch effect generator (swing effect means)
110 Recording control unit (recording control means)
120 Microcomputer (control device)
123 Longitudinal shake amount calculation unit (runout amount calculation means)
124 Lateral shake amount calculation unit (shake amount calculation means)
125 Vertical shake amount recording unit (recording control means)
126 Lateral shake amount recording unit (recording control means)

Claims (19)

A shake amount calculating means for calculating a shake amount in each of the first direction and the second direction when capturing an image;
Recording control means for performing recording control relating to the shake amount calculated by the shake amount calculation means;
Shaking effect means for adding a shaking effect in the first direction to the video,
The control apparatus according to claim 1, wherein the recording control unit changes recording control related to the shake amount in the first direction according to whether or not the shaking effect unit adds the shaking effect to the video. The recording control means includes
When the shaking effect means does not add the shaking effect to the video, the shaking amount in the first direction calculated by the shaking amount calculation means is recorded on a recording medium,
When the shake effect means adds the shake effect to the video, the shake amount in the first direction calculated by the shake amount calculation means is reduced and recorded on the recording medium. The control device according to claim 1.   The recording control means records the shake amount in the first direction calculated by the shake amount calculation means as 0 when the shake effect means adds the shake effect to the video on the recording medium. The control device according to claim 2. Further comprising a shake amount obtaining means for obtaining a shake amount of the shake effect added to the video by the shake effect means;
The recording control means is obtained by subtracting the shake amount from the shake amount in the first direction calculated by the shake amount calculation means when the shake effect means adds the shake effect to the video. The control device according to claim 2, wherein the recorded amount is recorded on the recording medium. A shaking effect means for adding a shaking effect in the first direction to the image;
A shaking period acquisition means for acquiring a period in which the shaking effect is added to the video;
Metadata setting means for setting metadata relating to the period;
And a recording control means for recording the video and the metadata on a recording medium. Motion vector acquisition means for acquiring a motion vector amount from the video;
A shaking amount setting means for setting a shaking amount of the shaking effect on the video based on the metadata;
Determining means for determining a shake correction amount of the video based on the motion vector amount;
Image processing means for cutting out a part of the video from the video based on the shake correction amount;
The determining means determines the shake correction amount based on the motion vector amount and the shake amount set by the shake amount setting means in the period when the shake effect is added to the video. The control device according to claim 5, characterized in that:   The shaking amount setting means can selectively set a first shaking amount, a second shaking amount larger than the first shaking amount, and a third shaking amount smaller than the first shaking amount. The control apparatus according to claim 6, wherein the control apparatus is provided. The shaking amount setting means includes:
When including the upper end or the lower end of the video before being cut out by the image processing means, the first shake amount is set as the shake amount,
When the direction of the motion vector amount and the direction of the amount of shaking coincide with each other, the second amount of shaking is set as the amount of shaking,
The control device according to claim 7, wherein when the direction of the motion vector amount and the direction of the amount of swing do not match each other, the third amount of swing is set as the amount of swing. A shake amount calculating means for calculating a shake amount in each of the first direction and the second direction when capturing an image;
Recording control means for performing recording control relating to the shake amount calculated by the shake amount calculation means;
A shaking effect means for adding a shaking effect in the first direction to the video;
A shaking amount calculating means for calculating a shaking amount of the shaking effect added to the video by the shaking effect means,
The control apparatus according to claim 1, wherein the recording control unit changes recording control related to the amount of shaking depending on whether or not the shaking effect unit adds the shaking effect to the video. The recording control means includes
When the shaking effect means does not add the shaking effect to the video, the amount of shaking is not recorded on a recording medium,
The control apparatus according to claim 9, wherein when the shaking effect unit adds the shaking effect to the video, the shaking amount is recorded on the recording medium. The recording control means includes
When the shaking effect means does not add the shaking effect to the video, the video and the shake amounts of the first direction and the second direction are recorded on a recording medium,
When the shaking effect means adds the shaking effect to the video, the video, the shaking amount in the first direction and the second direction, and the shaking amount are recorded on the recording medium. The control device according to claim 10. Motion vector acquisition means for acquiring a motion vector amount from the video;
A shake amount acquisition means for acquiring the shake amounts of the first direction and the second direction from a recording medium;
A shaking amount obtaining means for obtaining the shaking amount from the recording medium;
Determining means for determining a shake correction amount for processing a shake of the video based on the motion vector amount, the shake amount in each of the first direction and the second direction, and the shake amount;
Image processing means for cutting out a part of the video from the video based on the shake correction amount;
12. The method according to claim 9, wherein the determination unit changes a determination method of the shake correction amount according to whether or not the shake amount acquisition unit has acquired the shake amount. 13. The control device described. The determining means includes
When the shake amount acquisition means does not acquire the shake amount, the shake correction amount is determined based on the motion vector amount and the shake amounts of the first direction and the second direction,
13. The control apparatus according to claim 12, wherein when the shake amount acquisition unit acquires the shake amount, the shake correction amount is determined based on the motion vector amount and the shake amount in the second direction. . The determining means includes
When the shake amount acquisition means does not acquire the shake amount, the shake correction amount is determined based on the motion vector amount and the shake amounts of the first direction and the second direction,
When the shake amount acquisition means acquires the shake amount, the shake correction amount is determined based on the motion vector amount, the shake amounts in the first direction and the second direction, and the shake amount. The control device according to claim 12, wherein:   The control device according to claim 1, wherein the first direction is a vertical direction and the second direction is a horizontal direction. An image sensor that photoelectrically converts an optical image formed via the imaging optical system;
An imaging device comprising: the control device according to claim 1. Calculating each shake amount in the first direction and the second direction when capturing an image;
Performing recording control relating to the calculated shake amount;
Adding a shaking effect in the first direction to the video,
The control method characterized in that in the step of performing the recording control, the recording control related to the shake amount in the first direction is changed according to whether or not the shaking effect is added to the video. Calculating each shake amount in the first direction and the second direction when capturing an image;
Performing recording control relating to the calculated shake amount;
Adding a shaking effect in the first direction to the video, and causing a computer to execute the program,
In the step of performing the recording control, the recording control related to the shake amount in the first direction is changed according to whether or not the shaking effect is added to the video.   A storage medium storing the program according to claim 18.