JP2013168822A - Image pickup device, control method of the same, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which: much work is required to cut out a part of an image picked up by using an image pickup device, which leads to a result that a user inevitably has to perform complicated operations; and therefore, an image pickup device that can cut out a specific area with a simple operation is desired.SOLUTION: An image pickup device includes: an image pickup unit that picks up an image; a movement detection unit that detects movement of the image pickup device; image composition unit that creates a composite image from a plurality of photographic images picked up by the image pickup unit; a trimming range calculation unit that calculates a trimming range in cutting out a part of the composite image on the basis of the movement of the image pickup device; and an image cutting-out unit that cuts out an image from the composite image on the basis of the trimming range.

Description

  The present invention relates to an image capturing apparatus, a control method thereof, and a program.

  In recent years, in many electronic devices such as digital cameras, mobile phones, smartphones, tablet PCs (Personal Computers), notebook PCs, etc., devices equipped with cameras are increasing. For this reason, there are increasing opportunities for users who own these electronic devices to handle images.

  Here, Patent Document 1 discloses an image integration method in which a plurality of partial images are input and these images are combined to be integrated into a single image.

  Patent Document 2 discloses a technique for generating a panoramic image output from a plurality of captured still pictures and video frames.

  Further, Patent Document 3 discloses a technique for enabling a trimmed image to be imaged from the beginning when it is planned to trim and use a captured image.

Japanese Patent No. 4340722 JP 2010-252312 A Japanese Patent No. 3993457

  Each disclosure of the above prior art document is incorporated herein by reference. The following analysis has been made from the viewpoint of the present invention.

  As described above, with the increase in electronic devices equipped with cameras, the opportunities for users to handle images are increasing. Furthermore, in recent years, with the development of network technology, captured images are often disclosed to an unspecified number of people. At that time, there are cases where information that is not suitable for publishing the photographed image is included. Alternatively, there is a case where an image in which a part of a captured image is emphasized is disclosed.

  In such a case, it is necessary to cut out only a necessary image from the photographed image. That is, if a necessary image is cut out, an unnecessary image is not released. In addition, if an image is cut out like a star shape or a heart shape, a part of the image can be emphasized.

  However, such work is not always easy (free). For example, in the case of a computer equipped with a camera, the captured image is stored in a storage device. Thereafter, the image is read by an application capable of image conversion, and the photographed image is processed. Further, an electronic device equipped with a camera cannot always use such an application. In such a case, it is necessary to perform additional operations such as storing the captured image in a removable medium and moving the captured image to a computer.

  As described above, in order to cut out a part of the captured image, a lot of work is required, resulting in a complicated operation for the user. Therefore, an image pickup apparatus that can cut out a specific region by a simple operation, a control method thereof, and a program are desired.

  Patent Documents 1 and 2 merely disclose a method for synthesizing images. Further, Patent Document 3 discloses a technique in which a user selects a trimming form and performs signal processing so that an image fits in the range of the trimming form. Therefore, even if the technique of Patent Document 3 is used, an image after trimming cannot be automatically acquired.

  According to a first aspect of the present invention, there is provided an image capturing apparatus including an image capturing unit that captures an image, a movement detecting unit that detects movement of the image capturing apparatus, and a plurality of images captured by the image capturing unit. An image composition unit that generates a composite image from the captured image, a trimming range calculation unit that calculates a trimming range when a part of the composite image is cut out based on the movement of the image capturing device, and the trimming range. There is provided an image capturing device including an image cutout unit that cuts out an image from the composite image.

  According to a second aspect of the present invention, there is provided a method for controlling an image capturing apparatus including an image capturing unit that captures an image, a movement detecting step for detecting movement of the image capturing apparatus, A trimming range calculating step for calculating a trimming range for cutting out a part of the composite image based on movement, an image combining step for generating a composite image from a plurality of captured images taken by the image capturing unit, and the trimming range And a method of controlling the image capturing apparatus, including an image cutting-out step of cutting out an image from the composite image.

  According to a third aspect of the present invention, there is provided a program that is executed by a computer that controls an image capturing apparatus including an image capturing unit that captures an image, and a movement detection process that detects movement of the image capturing apparatus; A trimming range calculation process for calculating a trimming range for cutting out a part of the composite image based on the movement of the image capturing apparatus, and an image composition process for generating a composite image from a plurality of captured images captured by the image capturing unit. And an image cutout process for cutting out an image from the composite image based on the trimming range. This program can be recorded on a computer-readable storage medium. The storage medium can be non-transient such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium, or the like. The present invention can also be embodied as a computer program product.

  According to each viewpoint of the present invention, an image capturing apparatus capable of cutting out a specific region by a simple operation, a control method thereof, and a program are provided.

It is a figure for demonstrating the outline | summary of one Embodiment of this invention. 1 is a plan view illustrating an example of an overall configuration of an image capturing apparatus 1 according to a first embodiment of the present invention. It is a figure which shows an example of the internal structure of the imaging device 1 which concerns on the 1st Embodiment of this invention. 3 is a diagram illustrating an example of a background where a user intends to take a picture using the image pickup apparatus 1. FIG. It is a figure for demonstrating operation of the image pick-up device 1 by a user. FIG. 6 is a diagram for explaining an image displayed on the display unit 20. 4 is a diagram for describing trajectory information displayed on the image capturing apparatus 1. FIG. 3 is a flowchart showing an example of the operation of the image capturing apparatus 1.

  First, an outline of an embodiment will be described with reference to FIG. Note that the reference numerals of the drawings attached to this summary are attached to the respective elements for convenience as an example for facilitating understanding, and are not intended to limit the present invention to the illustrated embodiment.

  As described above, in order to cut out a part of a captured image, a lot of work is required, resulting in a complicated operation for the user. Therefore, an image pickup apparatus that can cut out a specific region by a simple operation is desired.

  Therefore, as an example, an image capturing apparatus 200 shown in FIG. 1 is provided. The image capturing apparatus 200 includes an image capturing unit 201 that captures an image, a movement detection unit 202 that detects movement of the image capturing apparatus, and an image combining unit that generates a composite image from a plurality of captured images captured by the image capturing unit. 203, a trimming range calculation unit 204 that calculates a trimming range when a part of the composite image is cut out based on the movement of the image capturing device, an image cutout unit 205 that cuts out an image from the composite image based on the trimming range, Is provided.

  When a user who uses the image capturing apparatus 200 starts continuous shooting of still images, he / she holds the image capturing apparatus 200 so as to surround an object to be cut out from an image displayed on the liquid crystal panel or the like of the image capturing apparatus 200. Manipulate. As an operation in this case, an operation of slightly tilting the image capturing apparatus 200 or slightly moving it up, down, left and right can be considered. That is, it can be said that the image capturing apparatus 200 is moved by the user so as to surround an object to be cut out from an image displayed on a liquid crystal panel or the like. At that time, the trimming range calculation unit 204 calculates a range formed by operating the image capturing apparatus 200 as a trimming range. On the other hand, the image composition unit 203 generates a composite image from a plurality of images obtained from the image capturing unit 201. Further, the image cutout unit 205 cuts out an image included in the trimming range from the composite image.

  As described above, the image capturing apparatus 200 obtains a plurality of images and calculates a trimming range when a part of the composite image is cut out. Thereafter, a composite image is generated from the plurality of images, and the composite image that is included in the trimming range is cut out. During this time, the user only operates the image capturing apparatus 200 so as to surround the object projected by the liquid crystal panel or the like. Therefore, it is possible to provide an image pickup apparatus that can cut out a specific region by a simple operation.

  Hereinafter, specific embodiments will be described in more detail with reference to the drawings.

[First Embodiment]
The first embodiment of the present invention will be described in more detail with reference to the drawings.

  FIG. 2 is a plan view illustrating an example of the overall configuration of the image capturing apparatus 1 according to the present embodiment.

  The image capturing apparatus 1 includes an operation button 10 and a display unit 20. The user operates the image capturing apparatus 1 using the operation button 10. The display unit 20 displays information necessary when the user operates the image capturing apparatus 1, a captured image, and the like. The display unit 20 is a device such as a liquid crystal panel or an organic EL (Organic Electro Luminescence). Although not shown in FIG. 2, it is also possible to operate the image pickup apparatus 1 with a device such as a touch panel. Further, the image pickup apparatus 1 includes a camera (lens), which is not shown in FIG. 2 because the lens is attached to the side facing the display unit 20.

  Next, the internal configuration of the image capturing apparatus 1 will be described.

  FIG. 3 is a diagram illustrating an example of an internal configuration of the image capturing apparatus 1 according to the present embodiment.

  The image capturing apparatus 1 includes an operation button 10, a display unit 20, a display control unit 30, a camera module 40, an acceleration sensor 50, a gyro sensor 60, a control unit 70, a storage unit 80, and a memory 90. , Is composed of. In FIG. 3, only the modules related to the image capturing apparatus 1 according to the present embodiment are illustrated for simplicity. Since the operation button 10 and the display unit 20 are as described above, further description is omitted.

  The display control unit 30 is connected to the display unit 20 and plays a role as a driver for setting and driving the display unit 20. The display control unit 30 generates a signal for driving the display unit 20 from the screen display information transmitted by the control unit 70.

  The camera module 40 includes a lens, an optical sensor, and a bus that transmits information from the sensor. The camera module 40 can exchange data with the control unit 70, the storage unit 80, and the memory 90.

  The acceleration sensor 50 detects the acceleration of the image capturing apparatus 1, converts the acceleration data into speed data, and then outputs the speed data to the control unit 70. The gyro sensor 60 measures the angular velocity of the image capturing apparatus 1 and outputs the information to the control unit 70. The acceleration sensor 50 and the gyro sensor 60 correspond to the above-described movement detection unit.

  The control unit 70 is connected to each module included in the image capturing apparatus 1 via a bus. By controlling these modules, the functions of the image capturing apparatus 1 are realized. The control unit 70 is realized by a CPU (Central Processing Unit). Further, the control unit 70 includes a trimming range calculation unit 71, an image composition unit 72, and an image cutout unit 73.

  The storage unit 80 is a device that stores fixed data such as a program executed in the control unit 70. The memory 90 is an area for temporarily storing data when the control unit 70 executes a program. Further, the memory 90 is also an area for storing an image photographed by the camera module 40.

  Next, the operation of the image capturing apparatus 1 will be described.

  FIG. 4 is a diagram illustrating an example of a background in which a user attempts to take a picture using the image capturing apparatus 1. Here, an operation of cutting out the rectangular cutout area 100 in the background shown in FIG. 4 will be described. The user operates the image capturing apparatus 1 along an image displayed on the display unit 20 and desired to be cut out. As the operation at that time, as described above, an operation of slightly tilting the image capturing apparatus 1 or slightly moving it up, down, left and right can be considered.

  FIG. 5 is a diagram for explaining the operation of the image capturing apparatus 1 by the user. For example, initially, it is assumed that the user holds the imaging device 1 perpendicular to the ground and points the lens toward the object. At this time, an example of an image displayed on the display unit 20 is shown in FIG. Thereafter, it is assumed that the user operates the image capturing apparatus 1 to tilt slightly upward. Then, the image displayed on the display unit 20 of the image capturing apparatus 1 moves upward by a distance corresponding to the distance L1 in FIG. The distance L1 is determined depending on the distance L2 between the image pickup device 1 and the object and the tilted angle θ (see FIG. 5). Similarly, if the user swings the image pickup device 1 slightly to the left or right, the image displayed on the display unit 20 moves according to the swing angle. Furthermore, even when the user moves the image capturing apparatus 1 up, down, left, and right, the image displayed on the display unit 20 moves. However, the moving distance of the image displayed on the display unit 20 in this case matches the distance that the user moved the image capturing apparatus 1. That is, when the user moves the image capturing apparatus 1 to the distance L3 (not shown in FIG. 5), up, down, left, and right, the image displayed on the display unit 20 also moves by a distance corresponding to the distance L3.

  FIG. 6 is a diagram for explaining an image displayed on the display unit 20. The area surrounded by each dotted line in FIG. 6 indicates a display area displayed on the display unit 20 at each point when the user moves the image capturing apparatus 1. FIG. 6 shows six regions. In addition, the center points of the display areas are denoted as C01 to C06, respectively.

  First, the user operates the image capturing apparatus 1 so that the center point C01 displayed on the display unit 20 moves toward the center point C02. Thereafter, the center points C02 to C06 reflected on the display unit 20 are operated so as to move sequentially. Finally, the image capturing apparatus 1 is operated so as to move from the center point C06 toward the center point C01. In this way, the user operates the image capturing device 1 so as to be a closed region when connecting the center points of the images displayed by the image capturing device 1. Further, the display unit 20 displays trajectory information for assisting the movement of the image capturing device 1 by the user.

  FIG. 7 is a diagram for explaining the trajectory information displayed on the image capturing apparatus 1. Assume that the user initially operates the image capturing apparatus 1 so that the image displayed on the display unit 20 moves from top to bottom as shown in FIG. In this case, a trajectory connecting the center points before and after the movement is displayed as trajectory information 101. Thereafter, it is assumed that the user operates the image capturing apparatus 1 so that the image displayed on the display unit 20 moves from left to right as shown in FIG. In that case, the trajectory information 101 in FIG. 7A is updated so as to extend to the right side. As described above, the image capturing apparatus 1 displays the trajectory information on the display unit 20 to assist the user in the image cutting operation. As described above, since the user operates the image capturing apparatus 1 so as to surround the image to be cut out, the trajectory information draws a closed region.

  FIG. 8 is a flowchart illustrating an example of the operation of the image capturing apparatus 1.

  In step S <b> 01, the control unit 70 activates the camera module 40 and displays an image obtained from the optical sensor included in the camera module 40 on the display unit 20. At that time, the control unit 70 displays a cross or a round point on the display unit 20 as a pixel position used for determining a locus when the user operates the image capturing apparatus 1. For example, the pixel position is displayed at the center of the screen as in the center points C01 to C06 shown in FIG.

  In step S <b> 02, the control unit 70 acquires an operation corresponding to an instruction to start image shooting by the user. For example, if the operation button 10 is to function as a shutter, image shooting is started when the operation button 10 is pressed.

  In step S <b> 03, the control unit 70 stores the start point coordinates of the trajectory information in the memory 90. More specifically, the coordinates on the display unit 20 and the pixel position (for example, the center of the screen) when the user presses the operation button 10 are stored as the start point coordinates. Note that the above-described closed region is formed from the start point coordinates, the end point coordinates after the movement of the image capturing apparatus 1, and the intermediate coordinates until the start point coordinates and the end point coordinates are measured.

  In the following description, it will be described that a plurality of images are acquired and stored in the memory 90. At that time, the timing for acquiring each image is expressed by an integer i (i is an integer of 0 or more). For example, the timing for acquiring a still image when the user presses the operation button 10 is i = 0, and the next timing for acquiring a still image is i = 1. Note that the timing for acquiring a still image is determined by a predetermined sampling time. That is, after acquiring a still image and a predetermined time has elapsed, the control unit 70 increments the integer i to create a timing for acquiring the still image. Further, the coordinates, still image, etc. at that time are expressed by attaching an integer i in parentheses. For example, a still image acquired when i = 0 is expressed as a still image (0), and coordinates relating to the pixel position at that time are expressed as coordinates (0).

  In this step, the control unit 70 sets the determination flag to 0. The determination flag is a flag for determining whether or not the distance between the start point coordinates and the end point coordinates of the trajectory information has once exceeded a preset threshold value. In other words, if the distance between these coordinates is too short, it is reasonable to consider it as an effect of user shake. Therefore, it should be determined that the image capturing apparatus 1 is operated by the user's intention only when the distance between the start point coordinates and the end point coordinates of the trajectory information exceeds the threshold value. At the same time, by using the determination flag, it is possible to prevent the trajectory information representing the movement of the image capturing apparatus 1 from being an extremely narrow region. The control unit 70 sets a determination flag to 1 when the distance between the start point coordinates and the end point coordinates of the trajectory information exceeds a threshold value, and uses it to determine whether the trajectory information is a closed region.

  In step S <b> 04, the control unit 70 acquires the still image (i) and stores it in the memory 90.

  In step S <b> 05, the control unit 70 acquires information obtained from the acceleration sensor 50 and the gyro sensor 60 (hereinafter referred to as sensor information) and stores it in the memory 90. The acquired sensor information is used to generate trajectory information. In other words, the direction in which the image capturing apparatus 1 has moved is specified by the rotation angle obtained from the gyro sensor 60, the acceleration is detected from the acceleration sensor 50, and the image capturing apparatus 1 is detected by the speed value converted from the acceleration data to the speed data. Specifies the distance traveled by.

  In step S06, the control unit 70 determines whether or not the integer i is 0. If the integer i is 0, it can be determined that an operation for moving the image displayed on the display unit 20 by the user has not been started, so there is no need to determine whether or not the trajectory information is a closed region. . Therefore, if the integer i is not 0, the process proceeds to step S07. However, if the integer i is 0, this step is repeated and the process waits until the integer i is updated.

  In step S07, the trimming range calculation unit 71 calculates coordinates (i). Specifically, the current coordinate (i) is calculated from the sensor information (i), the immediately preceding sensor information (i-1), and the immediately preceding coordinate (i-1).

  In step S08, the trimming range calculation unit 71 determines whether the determination flag is 0 or not. Further, the trimming range calculation unit 71 compares the distance between the start point coordinates and the end point coordinates of the trajectory information with a predetermined threshold value (step S09). If the determinations in steps S08 and S09 are both YES, the determination flag is set to 1 (step S10). If any of the determinations in steps S08 and S09 is NO, the determination flag is not changed and the process proceeds to step S11.

  In step S11, the trimming range calculation unit 71 calculates pixels (trajectory information) corresponding to coordinates (0) to coordinates (i) on the still image (i). For example, the trajectory information 101 shown in FIG. 7 is calculated.

  In step S <b> 12, the control unit 70 displays the still image (i) and the trajectory information (i) on the display unit 20.

  In step S13, the control unit 70 determines whether a condition necessary for trimming an image is satisfied. The first condition is when the determination flag is 1 and the distance between the start point coordinates and the end point coordinates of the trajectory information is smaller than the threshold value. The second condition is when a certain time has elapsed since the user pressed the operation button 10. If any one of these conditions is satisfied, the control unit 70 proceeds to step S14. If neither condition is satisfied, the control unit 70 proceeds to step S04 and continues the process.

  As described above, the trajectory information that assists the operation of moving the image displayed on the display unit 20 is displayed on the display unit 20, and a plurality of images are acquired until the trajectory information forms a closed region. To do. In many cases, the start point coordinates and end point coordinates of the trajectory information after acquiring a plurality of images do not match. This is a phenomenon often seen when the threshold value for determining whether or not the trajectory information forms a closed region is increased. In such a case, it is desirable to interpolate the acquired trajectory information with a straight line or the like. Alternatively, it is conceivable to use multi-order curve interpolation such as Lagrangian interpolation or spline interpolation. The control unit 70 stores the trajectory information in the memory 90 and uses it when trimming the composite image.

  In step S <b> 14, the control unit 70 determines a blank area and copes with it. Here, even when the user operates the image capturing apparatus 1 and acquires a plurality of images, there may be a case where not all necessary images are acquired. If the range in which the image is to be cut out is too large, the image near the center may not be acquired. That is, when the user draws a huge trajectory with respect to the shooting range of the image, a donut-shaped blank area may be generated during image synthesis.

  In order to prevent such a situation, the control unit 70, when there is an image (undefined pixel information) that cannot be acquired within the range in which trimming is scheduled, the stage in which the trajectory information forms a closed region. Then, processing for prompting an operation for acquiring such an image is performed. Specifically, the control unit 70 prompts the user to point the image capturing apparatus 1 in the direction of the arrow by displaying an arrow or the like in the direction in which the blank area exists on the display unit 20.

  The processing in this step is performed as follows. First, the control unit 70 determines whether or not there is a blank area as follows. Note that there is a possibility that a blank area may exist when the imaging range fluctuates by taking two vertical axes such as up and down and left and right of the display area in the image pickup apparatus 1 and exceeding 1/2 of the angle of view of both axes. . Therefore, when the control unit 70 determines that there is a blank area in the trimming range, the controller 70 estimates the range and direction of the blank area based on the sensor information measured in steps S04 to S13 and the angle of view information. Furthermore, the control unit 70 displays on the display unit 20 a display that prompts the user to change the direction in which the image capturing apparatus 1 is facing. After the user operates the image pickup apparatus 1 toward the blank area, an image is added and acquired. Such processing is repeated, and when the blank area disappears, the processing in this step is terminated.

  In step S <b> 15, the image composition unit 72 obtains a composite image using the plurality of images obtained up to the previous step. When compositing images, reference is made to the movement information of the preceding and following images, an overlapping area is determined, and a composite image is generated. Note that such an image composition method is an example, and is not intended to limit the image composition method. Of course, other methods can be applied to the generation of the composite image. Further, when the subject is moving, an afterimage may occur when the images are combined. In such a case, a rule for preferentially acquiring an image is determined in advance so that no afterimage remains in the composite image.

  In step S16, the image composition unit 72 reconstructs the trajectory information obtained up to step S13 on the composite image. At this time, the synthesized image and the trajectory information are synchronized based on the first image center point of the first image used at the time of synthesis and the start point coordinates of the trajectory information for trimming. That is, the trajectory information generated by the control unit 70 is reflected on a composite image generated from a plurality of images (a closed region is formed on the composite image). In the determination using the first image, the error may be large. Therefore, the entire image is corrected with reference to the start point coordinates, the end point coordinates, and the coordinates at these intermediate points. May be.

  In step S <b> 17, the image cutout unit 73 performs trimming of the composite image after the trimming range in the composite image is determined, stores the trimmed image, and ends the process.

  As described above, the image capturing apparatus 1 according to the present embodiment acquires a plurality of images and collects trajectory information for determining the trimming range. Thereafter, the presence of a blank area within the trimming range is determined, and if there is a blank area, a process for prompting the user to fill the blank area is performed. Furthermore, a composite image is generated from a plurality of images, and trajectory information is reconstructed on the composite image, thereby determining a trimming range. If the trimming range is determined, the range is cut out from the composite image and stored.

  When the trimming range is not smooth, it is desirable that the trimming range calculation unit 71 converts the shape into a smooth shape by calculating an approximate curve. The trimming range calculation unit 71 can also use a predetermined trajectory as a template by comparing a preset trajectory with a curve formed by the obtained trimming range and matching the two. Trimming may be performed with a preset shape after the entire acquired trimming range is included. For example, a quadrilateral including a selected line, an ellipse including a selected range, and a heart shape including a selected line.

  It is also conceivable that the image composition unit 72 performs a blurring process on a region outside the trimming range of the composite image and stores the composite image in a form that emphasizes only the trimming range. The image composition unit 72 may cut out an image from the composite image based on the feature amount closest to the trimmed curve line. For example, if the trajectory information is composed of five straight lines and five angular displacement points, a pentagonal image is cut out from the composite image, and if it is composed only of curves, a circular image is extracted from the composite image. cut. Alternatively, the candidates may be selected by sorting the candidates in the descending order of the feature amount of the trajectory information. When the image composition unit 72 performs image composition using a plurality of images, it may perform super-resolution processing at the same time to improve visibility.

  Furthermore, when the image composition unit 72 stores the trimmed composite image, a large image obtained by moving the image capturing apparatus 1 may be stored. Further, the image cutout unit 73 may perform the storage in a state where information related to the trimming range is added to the composite image. In order to reduce the processing load of the control unit 70, the movement range at the time of trimming may be limited.

  In addition, the image composition processing in the image composition unit 72 may be omitted, and a specification that allows a trimming range to be specified in a pseudo manner with respect to the first captured image may be used. When the trimmed image is other than a quadrilateral, the trimming range calculation unit 71 may store the trimmed image by superimposing the trimmed image on a quadrilateral of a specific color having a set image size. Alternatively, the image cutout unit 73 may set a transparent color outside the trimming range.

  As described above, the image capturing apparatus 1 according to the present embodiment acquires a plurality of images and acquires a range in which the user desires trimming as trajectory information. A composite image is generated from the acquired plurality of images, and the trimming range is determined by associating the composite image with the trajectory information. Thereafter, a trimming range is cut out from the composite image.

  As a result, the user can cut out only a specific area by a simple operation at the time of shooting, and there is no need to perform a work (trimming operation) using a dedicated application. Furthermore, since it is possible to acquire only a necessary area at the image capturing stage, no special work is required when publishing the image on the network. Accordingly, it is possible to shorten the time from image shooting to release. Furthermore, it is possible to store an image in a star shape, a heart shape, or the like by a simple operation, and it is possible to increase the user's opportunity to use the image.

  A part or all of the above embodiments can be described as in the following supplementary notes, but is not limited thereto.

  (Additional remark 1) It is an image imaging device provided with the image imaging part which image | photographs an image, Comprising: A synthetic | combination image from the movement detection part which detects the movement of the said image imaging device, and the several picked-up image image | photographed by the said image imaging part An image composition unit to be generated, a trimming range calculation unit that calculates a trimming range when a part of the composite image is cut out based on the movement of the image capturing device, and an image from the composite image based on the trimming range. An image pickup device comprising: an image cutout unit to cut out.

  (Additional remark 2) Furthermore, the said image imaging device is provided with the display part which displays the image imaged by the said image imaging part, The said trimming range calculation part is a coordinate on the image which the said display part displays, An image imaging apparatus that calculates the trimming range based on trajectory information formed by connecting a plurality of coordinates calculated based on movement of the image imaging apparatus detected by a movement detection unit.

  (Additional remark 3) The said display part is an image imaging device which superimposes and displays the said locus | trajectory information on the image imaged by the said image imaging part.

  (Supplementary Note 4) The trimming range calculation unit is a coordinate on the image displayed by the display unit, and a start point coordinate at which the image imaging device starts to move, and an intermediate coordinate when the image imaging device moves. An image capturing apparatus that calculates a closed region formed by connecting the end point coordinates at which the image capturing apparatus has finished moving as the trimming range.

  (Additional remark 5) The said trimming range calculation part is an image imaging device which starts calculation of the said locus | trajectory information, when the distance between the said starting point coordinate and the said intermediate coordinate exceeds the predetermined threshold value.

  (Additional remark 6) The said trimming range calculation part is an image imaging device which determines the said locus | trajectory information, when the distance between the said start point coordinate and the said end point coordinate becomes smaller than the predetermined threshold value.

  (Supplementary note 7) The image capturing apparatus further includes: a blank area determination unit that determines a blank area in which pixels of an image included in the trimming range are undetermined; and if the blank area exists, On the other hand, an image capturing apparatus comprising: a blank area acquisition unit that instructs acquisition of an image corresponding to the blank area.

  (Additional remark 8) The said image pick-up device is further provided with the memory | storage part which memorize | stores an image, The said image cut-out part respectively cuts out the image cut out from the said synthesized image and the said synthesized image based on the said trimming range, respectively. Image capturing device to be stored in

  (Additional remark 9) It is a control method of an image pick-up apparatus provided with the image pick-up part which image | photographs an image, Comprising: From the movement detection process which detects the movement of the said image pick-up apparatus, and the some picked-up image image | photographed by the said image pick-up part. An image composition step for generating a composite image, a trimming range calculation step for calculating a trimming range for extracting a part of the composite image based on the movement of the image capturing device, and the composite image based on the trimming range And a method for controlling the image capturing apparatus, including an image cut-out step of cutting out an image from the image.

  (Additional remark 10) The said imaging device is provided with the display part which displays the image which the said imaging part imaged, The said trimming range calculation process is a coordinate on the image which the said display part displays, Comprising: The said movement detection process A control method for an image capturing apparatus, which calculates the trimming range based on trajectory information formed by connecting a plurality of coordinates calculated based on the movement of the image capturing apparatus detected by.

  (Additional remark 11) The control method of the said image pick-up device further includes the process of superimposing and displaying the said locus | trajectory information on the image imaged by the said image pick-up part.

  (Supplementary Note 12) The trimming range calculation step includes coordinates on an image displayed by the display unit, starting point coordinates at which the image capturing apparatus starts moving, and intermediate coordinates at which the image capturing apparatus moves. A control method for an image capturing apparatus that calculates a closed region formed by connecting the end point coordinates at which the image capturing apparatus has finished moving as the trimming range.

  (Additional remark 13) The said trimming range calculation process is a control method of the imaging device which starts calculation of the said locus | trajectory information, when the distance between the said starting point coordinate and the said intermediate coordinate exceeds the predetermined threshold value.

  (Additional remark 14) The said trimming range calculation process is a control method of the image pick-up device which determines the said locus | trajectory information, when the distance between the said start point coordinate and the said end point coordinate becomes smaller than the predetermined threshold value.

  (Supplementary Note 15) The control method of the image pickup apparatus may further include a blank area determination step for determining a blank area in which pixels of an image included in the trimming range are undetermined, and when the blank area exists And a blank area obtaining step for instructing the user to obtain an image corresponding to the blank area.

  (Supplementary Note 16) A program that is executed by a computer that controls an image capturing apparatus including an image capturing unit that captures an image, the motion detecting process for detecting movement of the image capturing apparatus, and the image captured by the image capturing unit Based on an image composition process for generating a composite image from a plurality of captured images, a trimming range calculation process for calculating a trimming range when a part of the composite image is cut out based on the movement of the image capturing device, and the trimming range And an image cutout process for cutting out an image from the composite image.

  (Additional remark 17) The said image imaging device is provided with the display part which displays the image imaged by the said image imaging part, The said trimming range calculation process is a coordinate on the image which the said display part displays, Comprising: The said movement detection process A program for calculating the trimming range based on trajectory information formed by connecting a plurality of coordinates calculated based on the movement of the image capturing apparatus detected by the computer.

  (Additional remark 18) The said program is a program which performs the process which further superimposes and displays the said locus | trajectory information on the image imaged by the said image imaging part.

  (Supplementary Note 19) The trimming range calculation process includes coordinates on an image displayed by the display unit, starting point coordinates at which the image capturing apparatus starts moving, and intermediate coordinates when the image capturing apparatus moves. A program for calculating, as the trimming range, a closed region formed by connecting an end point coordinate at which the image capturing apparatus has finished moving.

  (Additional remark 20) The said trimming range calculation process is a program which starts calculation of the said locus | trajectory information, when the distance between the said starting point coordinate and the said intermediate coordinate exceeds the predetermined threshold value.

  (Additional remark 21) The said trimming range calculation process is a program which determines the said locus | trajectory information, when the distance between the said start point coordinate and the said end point coordinate becomes smaller than a predetermined threshold value.

  (Supplementary note 22) The program further includes a blank area determination process for determining a blank area in which pixels of an image included in the trimming range are undetermined, and if the blank area exists, A blank area acquisition process for instructing acquisition of an image corresponding to the blank area.

  Each disclosure of the cited patent documents and the like cited above is incorporated herein by reference. Within the scope of the entire disclosure (including claims) of the present invention, the embodiments and examples can be changed and adjusted based on the basic technical concept. Various disclosed elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) within the scope of the claims of the present invention, Selection is possible. That is, the present invention of course includes various variations and modifications that could be made by those skilled in the art according to the entire disclosure including the claims and the technical idea. For example, any algorithm for specifying a coordinate position using an acceleration sensor or the like or an algorithm for image synthesis may be used.

DESCRIPTION OF SYMBOLS 1,200 Image pick-up device 10 Operation button 20 Display part 30 Display control part 40 Camera module 50 Acceleration sensor 60 Gyro sensor 70 Control part 71, 204 Trimming range calculation part 72, 203 Image composition part 73, 205 Image clipping part 80 Storage part 90 Memory 100 Cutout Area 101 Trajectory Information 201 Image Imaging Unit 202 Movement Detection Unit

Claims (10)

An image capturing apparatus including an image capturing unit that captures an image,
A movement detector for detecting movement of the image pickup device;
An image compositing unit that generates a composite image from a plurality of captured images captured by the image capturing unit;
A trimming range calculation unit that calculates a trimming range when cutting out a part of the composite image based on the movement of the image capturing device;
An image cutout unit that cuts out an image from the composite image based on the trimming range;
An image pickup apparatus comprising: And a display unit that displays an image captured by the image capturing unit.
The trimming range calculation unit is a coordinate on the image displayed by the display unit, and the trajectory information formed by connecting a plurality of coordinates calculated based on the movement of the image pickup device detected by the movement detection unit. The image capturing apparatus according to claim 1, wherein the trimming range is calculated based on the image capturing apparatus.   The image capturing apparatus according to claim 2, wherein the display unit displays the trajectory information so as to overlap the image captured by the image capturing unit.   The trimming range calculation unit is coordinates on an image displayed by the display unit, and is a start point coordinate at which the image capturing device starts moving, an intermediate coordinate when the image capturing device is moved, and the image capturing The image capturing apparatus according to claim 2, wherein a closed area formed by connecting the end point coordinates at which the apparatus has finished moving is calculated as the trimming range.   The image capturing apparatus according to claim 4, wherein the trimming range calculation unit starts calculating the trajectory information when a distance between the start point coordinates and the intermediate coordinates exceeds a predetermined threshold.   The image capturing apparatus according to claim 4 or 5, wherein the trimming range calculation unit determines the trajectory information when a distance between the start point coordinates and the end point coordinates becomes smaller than a predetermined threshold. Furthermore, a blank area determination unit that determines a blank area in which pixels of an image included in the trimming range are uncertain,
When the blank area exists, a blank area acquisition unit that instructs the user to acquire an image corresponding to the blank area;
An image capturing apparatus according to any one of claims 1 to 6. Furthermore, a storage unit for storing images is provided,
The image capturing apparatus according to claim 1, wherein the image cutout unit stores the composite image and an image cut out from the composite image based on the trimming range in the storage unit. A method for controlling an image capturing apparatus including an image capturing unit that captures an image,
A movement detecting step for detecting movement of the image pickup device;
An image compositing step for generating a composite image from a plurality of captured images captured by the image capturing unit;
A trimming range calculation step of calculating a trimming range when cutting out a part of the composite image based on the movement of the image capturing device;
An image cutout step of cutting out an image from the composite image based on the trimming range;
A control method for an image pickup apparatus. A program that causes a computer that controls an image capturing apparatus including an image capturing unit to capture an image to be executed,
A movement detection process for detecting movement of the imaging device;
Image composition processing for generating a composite image from a plurality of photographed images photographed by the image capturing unit;
Trimming range calculation processing for calculating a trimming range when cutting out a part of the composite image based on the movement of the image capturing device;
Image cutout processing for cutting out an image from the composite image based on the trimming range;
A program that executes.

Images