JP2015156693A - Imaging apparatus and image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which analyzes photographing time information of a plurality of images and combines the plurality of images with one another.

SOLUTION: In an imaging apparatus, an imaging device 202 photographs an image through an optical system 201 (S701), and acquires photographing time information for each photographed image (S702). When multiple exposure photographing is finished (S703), the photographing time information acquired for each of the plurality of photographed images is analyzed and a combination ratio is calculated (S705), and the plurality of photographed images are combined in accordance with the calculated combination ratio (S707).

COPYRIGHT: (C)2015,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an image processing apparatus that perform image processing for superimposing a plurality of captured images such as multiple exposure shooting.

  As an image processing apparatus that creates one composite image from a plurality of captured images at the time of multiple exposure photographing, an apparatus that records information for restoring a photographed image before composition in tag information of the composite image has been proposed (see Patent Document 1). . In addition, a technique has been proposed in which each captured image is converted using shooting time information at the time of shooting each of a plurality of shot images, and the converted images are combined (see Patent Document 2).

  FIG. 8 is a diagram schematically illustrating the influence of exposure of each image in a composite image by multiple exposure. FIG. 8A shows a case where multiple exposure is performed with a captured image having the same exposure, and in this case, the influence of each captured image on the composite image is approximately the same. On the other hand, as shown in FIG. 8B, when multiple exposure is performed on one of the captured images using a captured image with low exposure, the influence of the captured image with low exposure on the composite image is relative. It becomes low.

  As described above, when a composite image is generated and recorded by multiple exposure, the effect on the composite image is weakened in a photographed image when taken dark and strong in a photographed image taken brightly. Therefore, it is possible to control the composition ratio of each captured image in multiple exposure by capturing a certain captured image darker or brighter than other captured images.

JP 2002-300372 A JP-A-10-271427

  However, in the above-described prior art, the composition ratio of each captured image is not controlled using the shooting time information of each captured image, and therefore, unintentionally as shown in FIG. Problems such as shooting will occur.

  An object of the present invention is to provide an imaging apparatus capable of performing more appropriate processing based on information at the time of photographing of each photographed image when combining a plurality of photographed images.

  An imaging apparatus according to the present invention includes an imaging unit that captures a subject image, an acquisition unit that acquires shooting time information for each captured image at the time of shooting by the imaging unit, and a plurality of captured images obtained by the imaging unit. Setting means for setting the composite ratio of the plurality of photographed images by analyzing the photographing time information obtained by the obtaining means for each of the plurality of photographed images taken when the multiple exposure is performed. Synthesizing means for synthesizing the plurality of captured images in accordance with the ratio.

  According to the present invention, when a plurality of captured images are combined, more appropriate processing is performed based on the shooting time information of each captured image, so that a more preferable combined image can be obtained.

It is a figure explaining the effect by selecting the order which performs exposure correction and multiple exposure at the time of multiple exposure photography. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to a first embodiment. It is a block diagram which shows schematic structure of another image processing apparatus which concerns on 1st Embodiment. 3 is a flowchart illustrating a flow when exposure correction is performed in multiple exposure shooting, which is executed by the image processing apparatus of FIG. 2. FIG. 5 is a diagram illustrating an example of a relationship between shooting time information and a composition ratio used in step S404 in FIG. 4. It is a figure which shows typically the change method of the synthetic | combination ratio of the some picked-up image performed with the image processing apparatus which concerns on 2nd Embodiment. 14 is a flowchart of development parameter synthesis processing executed by the image processing apparatus according to the third embodiment. It is a figure which illustrates typically the influence of exposure of each image in the composite image by multiple exposure.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<First Embodiment>
In the first embodiment, an image processing apparatus that switches an exposure correction method using information at the time of shooting of a plurality of shot images at the time of multiple exposure shooting will be described. FIG. 1 is a diagram for explaining the effect of selecting the order of performing exposure correction and multiple exposure (compositing of captured images) during multiple exposure shooting.

  Here, a case is considered where a photographic image 100 and a photographic image 101 having different exposures are obtained by two photographic operations, and exposure correction and multiple exposure are performed on these photographic images 100 and 101. As a method for correcting exposure of a captured image in multiple exposure shooting, as shown in FIG. 1A, a method of performing exposure correction on a captured image that is underexposed before performing multiple exposure, and FIG. As described above, a method of performing exposure correction after performing multiple exposure is conceivable.

  As shown in FIG. 1A, the exposure correction image 102 is formed once by performing exposure correction on the captured image 101, and then the multiple exposure is performed thereafter, so that the captured image 100 and the captured image 101 are at a similar rate. A combined composite image 103 is obtained. On the other hand, as shown in FIG. 1B, when the composite image 104 is obtained by the multiple exposure first from the captured images 100 and 101 and the exposure correction is performed thereafter, the captured image 101 is compared with the captured image 100. An exposure corrected image 105 having a low composition ratio is obtained.

  Therefore, when it is preferable that the composite ratio of the photographed image 101 is low, it is desirable to perform exposure correction after performing multiple exposure as shown in FIG. On the other hand, when it is preferable that the composite ratio of the photographed image 100 and the photographed image 101 is approximately the same, as shown in FIG. 1A, it is desirable to perform multiple exposure after performing exposure correction. Therefore, the determination of switching between these processes is performed using the shooting time information of each of the shot images 100 and 101. The details will be described next with reference to FIGS.

  FIG. 2 is a block diagram illustrating a schematic configuration of the image processing apparatus according to the first embodiment. The image processing apparatus 300 includes an imaging apparatus such as a digital camera or a digital video camera that captures a subject and obtains image data.

  In FIG. 2, an optical system 201 is composed of a lens, a shutter, and an aperture, and forms an image of light from a subject on the image sensor 202 with an appropriate amount and timing. The image sensor 202 converts light imaged through the optical system 201 into an image. The CPU 203 controls the operation of the entire image processing apparatus 200 by performing various calculations and controlling each part of the image processing apparatus 200 in accordance with an input signal from the operation unit 208 and a program.

  The primary storage device 204 is, for example, a RAM, stores temporary data, and is also used as a work area for the CPU 203. The secondary storage device 205 is a ROM, for example, and stores a program (firmware) for controlling the image processing device 200 and various setting information. The storage medium 206 is, for example, an HDD or a DVD-RAM, and stores captured image data and the like. The storage medium 206 can be attached to and detached from the main body of the image processing apparatus 200, and can be loaded (connected) to a personal computer (PC) or the like to read data. In the image processing apparatus 200, the CPU 203 has a function of accessing the storage medium 206 and reads / writes data to / from the storage medium 206.

  A display unit 207 performs display of a viewfinder image at the time of shooting, display of a shot image, character display for interactive operation, and the like. The operation unit 208 includes, for example, buttons, levers, a touch panel, and the like as means for receiving user operations. A communication device 209 connects the image processing device 200 and an external device, and enables transmission / reception of control commands and data. For example, PTP (Picture Transfer Protocol) is used as a protocol for establishing a connection and executing data communication.

  The communication device 209 is not limited to this, and may perform communication through a wired connection such as a USB (Universal Serial Bus) cable, or may perform communication through a wireless connection such as a wireless LAN. Further, communication may be performed by direct connection with an external device, or a method of connecting to an external device via a server or a method of connecting to an external device via a network such as the Internet may be used.

  FIG. 3 is a block diagram illustrating a schematic configuration of another image processing apparatus according to the first embodiment. In the image processing apparatus 300, the display unit 301 performs display of an image before image processing, display of an image after image processing, character display for interactive operation, and the like. The operation unit 302 includes a keyboard, a pointing device, and the like for receiving user operations. The CPU 303 controls the operation of the entire image processing apparatus 300 by performing various operations and controlling each part of the image processing apparatus 300 according to input signals and programs. The primary storage device 304 is, for example, a RAM, stores temporary data, and is also used as a work area for the CPU 303. The secondary storage device 305 is a ROM, for example, and stores a program for performing image processing and various setting information. The communication device 306 is connected to an external device and transmits / receives control commands and data. The communication device 306 can select the same form as the communication device 209 described with reference to FIG.

  FIG. 4 is a flowchart showing the flow when exposure correction is performed in multiple exposure shooting, which is executed by the image processing apparatus 200. In the processing according to this flowchart, the exposure correction method is switched using the shooting time information. In FIG. 4, the case where the exposure correction method is switched for a captured image by the image processing apparatus 200 will be described. However, the image processing apparatus 300 can perform the same processing on image data having shooting time information. Therefore, the image processing apparatus 300 can obtain the shooting time information when shooting the subject image, add the shooting time information to the shot image, and save the image data as image data. Can be performed.

  First, when an image is captured by the image sensor 202 through the optical system 201 (step S401), the CPU 203 acquires shooting time information for each captured image (step S402) and determines whether or not to end the multiple exposure shooting. Is determined (step S403). If multiple exposure shooting is to be continued (“NO” in S403), the process returns to step S401. If multiple exposure shooting is to be ended (“YES” in S403), the process proceeds to step S404.

  In step S404, the CPU 203 selects an exposure correction method. This is to select the order of exposure correction and multiple exposure (superposition of a plurality of photographed images) as shown in FIGS. 8 (a) and 8 (b). Here, an example of a method for selecting the order of exposure correction and multiple exposure will be described with reference to FIG. FIG. 5 is a diagram showing the relationship between the shooting time information and the composition ratio used in step S404.

  FIG. 5A is a diagram illustrating an example in which the order of exposure correction and multiple exposure is selected based on the aperture value, shutter speed, ISO sensitivity, and brightness of a captured image when the shooting mode is manual exposure. As described in the upper part of FIG. 5A, when the brightness of a certain captured image is captured darkly with respect to the brightness of another captured image by the user's settings such as aperture value, shutter speed, and ISO sensitivity, It is desirable to reduce the composite ratio of the captured images. Therefore, in this case, the CPU 203 proceeds from the step S404 to the multiple exposure in the step S405, further selects a route for proceeding to the exposure correction in the step S406, executes the process, and thereafter ends the process. .

  As described in the lower part of FIG. 5A, when the brightness of a certain captured image is photographed brighter than the brightness of another captured image by the user's settings such as aperture value, shutter speed, and ISO sensitivity, It is desirable to increase the composition ratio of the captured images. Therefore, also in this case, the CPU 203 proceeds from the step S404 to the multiple exposure in the step S405, selects the route for proceeding to the exposure correction in the step S406, executes the process, and thereafter ends the process.

  On the other hand, in the case of manual exposure, if the brightness of a certain captured image is dark or bright relative to the brightness of another captured image, the image is not captured dark or bright as set by the user. It is desirable to set the image composition ratio to the same level as other captured images. Therefore, in such a case, the CPU 203 proceeds from step S404 to exposure correction in step S407, further selects a route for proceeding to multiple exposure in step S408, executes the process, and then performs the process. Terminate.

  FIG. 5B is a diagram illustrating an example in which the order of exposure correction and multiple exposure is selected based on ISO sensitivity, exposure correction, and brightness of a captured image when the shooting mode is set to automatic exposure. As described in the upper part of FIG. 5B, when the brightness of a shot image is dark with respect to the brightness of another shot image by the user settings such as ISO sensitivity and exposure correction, the shot image is synthesized. It is desirable to reduce the ratio. Therefore, in this case, the CPU 203 proceeds from the step S404 to the multiple exposure in the step S405, further selects a route for proceeding to the exposure correction in the step S406, executes the process, and thereafter ends the process. .

  As described in the lower part of FIG. 5B, when the brightness of a shot image is shot brighter than the brightness of other shot images by user settings such as ISO sensitivity and exposure correction, the shot image is synthesized. It is desirable to increase the ratio. Therefore, in this case as well, the CPU 203 advances the multiple exposure from step S404 to step S405, selects the route that advances the process to exposure correction in step S406, executes the process, and then ends the process. .

  On the other hand, in the case of automatic exposure, it was not taken dark or bright by the setting by the user, but when the brightness of one shot image is dark or bright relative to the brightness of another shot image, It is desirable that the composite ratio of the photographed images be comparable to that of other photographed images. Therefore, in such a case, the CPU 203 proceeds from step S404 to exposure correction in step S407, further selects a route for proceeding to multiple exposure in step S408, executes the process, and then performs the process. Terminate.

  In addition to using the determination method shown in FIGS. 5A and 5B described above, an exposure meter or similar device is provided, and the order of exposure correction and multiple exposure is selected using photometric information obtained therefrom. It is also possible to perform. FIG. 5C is a diagram showing an example in which the order of exposure correction and multiple exposure is selected from photometric information.

  As shown in the upper part of FIG. 5C, when the photometric value (photometric information) at a certain photometric point is bright and the brightness (lightness of the photometric point) at the position of the photographed image corresponding to the photometric point is dark, It is desirable to reduce the composite ratio of the captured image. Further, as shown in the lower part of FIG. 5C, when the photometric value at a certain photometric point is dark and the brightness at the position of the image corresponding to the photometric point is bright, the composite ratio of the photographed image is increased. It is desirable. In any of these cases, the CPU 203 proceeds from the step S404 to the multiple exposure in the step S405, further selects the route for proceeding to the exposure correction in the step S406, executes the process, and thereafter ends the process. .

  On the other hand, when it does not correspond to either the upper stage or the lower stage in FIG. Therefore, in this case, the CPU 203 proceeds from step S404 to exposure correction in step S407, further selects a route for proceeding to multiple exposure in step S408, executes the process, and then ends the process. .

  Note that in the analysis of the shooting time information, it is not necessary to consider all of the plurality of elements listed in FIG. 5, and only a part of the plurality of elements may be used for the analysis. Further, in addition to the plurality of elements shown in FIG. 5, when another manual operation by the user is included, it is possible to proceed from step S <b> 404 to step S <b> 405 in FIG. 4. .

  As described above, according to the first embodiment, in the multiple exposure shooting, the exposure correction method is switched by analyzing the shooting information. As a result, a more preferable composite image, that is, a composite image reflecting the user's intention to shoot can be obtained.

<Second Embodiment>
In the second embodiment, an image processing apparatus will be described that changes the composition ratio of a plurality of photographed images to be composited using information at the time of photographing of each photographed image during multiple exposure photography. The image processing apparatus according to the second embodiment has the same hardware configuration as the image processing apparatus according to the first embodiment, but only the program executed by the CPU 203 is different. A description of the hardware configuration of the image processing apparatus is omitted.

  In the second embodiment, specifically, when the CPU 203 determines that a plurality of photographed images are photographed continuously in time from the photographing time information, the composite ratio of some photographed images is set. Make it higher than other captured images. In the second embodiment, when the CPU 203 determines from the shooting information that a plurality of captured images include a face, the composite ratio of the captured image including the face rather than the captured image not including the face. Be high. These will be described in detail with reference to FIG.

  FIG. 6 is a diagram schematically illustrating a method for changing a composition ratio of a plurality of captured images to be synthesized, which is executed by the image processing apparatus according to the second embodiment. FIG. 6A shows a plurality of similar captured images obtained by continuously capturing (continuous shooting) a moving object from the same position. In this case, if multiple exposure is performed on each captured image at the same composition ratio, a composite image shown in FIG. 6B is obtained. In such multiple exposure shooting, for example, the composite ratio of the first captured image is increased, the composite ratio of the last captured image is increased, or the composite ratio of the first and last captured images is increased. An expression method can be considered.

  FIG. 6C shows a composite image obtained when a method of increasing the composite ratio of the last photographed image is used. The composite image gives an impression greatly different from the composite image of FIG. 6B. It can be seen that an image is obtained. From this, by using the shooting time information, the shooting scene is determined. For example, when it is determined from the shooting time information that the continuous shooting mode is set, a process of increasing the composition ratio of a specific shot image is performed. You can see that it can be done.

  Then, next, consider image synthesis by multiple exposure of each shot image of a scene with a face and a scene without a face, as shown in FIG. In this case, when multiple exposure is performed with the same composition ratio of the captured images, a composite image as shown in FIG. 6E is obtained. However, in such a case, the main subject is often a face, and it is preferable to generate a composite image as shown in FIG. 6F by increasing the composition ratio of a scene with a face.

  Therefore, the face information is extracted from the shooting time information and scene determination is performed. When a face is detected in a shot image, the shot image in which the face is detected is combined with the shot image in which no face is detected. Process to increase the ratio. It is also possible to perform processing such as changing the composition ratio in consideration of the size, position, etc. of the face. For example, for a photographed image with a large face and a photographed image with a face in the center, this is a process of increasing the composition ratio.

  In this way, in multiple exposure shooting, a more preferable image can be obtained by changing the combination ratio of a plurality of shot images to be combined using the shooting time information of each shot image.

<Third Embodiment>
In the third embodiment, an image processing apparatus that combines parameters used for development using shooting time information of a plurality of shot images to be combined at the time of multiple exposure shooting will be described. The image processing apparatus according to the third embodiment has the same hardware configuration as the image processing apparatus according to the first embodiment, but only the program executed by the CPU 203 is different. A description of the hardware configuration of the image processing apparatus is omitted.

  In the image processing apparatus according to the third embodiment, when a composite image is obtained by multiple exposure, a plurality of RAW image data generated by a plurality of shootings are combined to generate combined RAW image data, and the generated combined RAW image Develop data. At this time, if only development parameters obtained from the shooting time information of the first shooting are used as parameters (development parameters) used for development such as the white balance setting value and the color conversion table, the shot image of the first shooting is used. Development parameters that are suitable only for the above are used. However, for example, when the main subject is included in the second and subsequent shooting and the shooting is performed so that the combination ratio of the first shooting is low, the development of the combined RAW image data is performed using such development parameters. Is not preferred. Therefore, in the third embodiment, parameters used for developing each RAW image data are synthesized using each shooting time information obtained by a plurality of times of shooting, and a composite development parameter used for developing the synthesized RAW image data is generated. .

  FIG. 7 is a flowchart of a development parameter composition process executed by the image processing apparatus according to the third embodiment. First, an image formed on the image sensor 202 through the optical system 201 is captured (step S701). Next, the CPU 203 acquires shooting time information (step S702), and determines whether or not the multiple exposure shooting has been completed (step S703).

  If multiple exposure shooting is to be continued (“NO” in S703), the process returns to step S701. If multiple exposure shooting is to be ended (“YES” in S703), the process proceeds to step S704. In step S <b> 704, the CPU 203 generates combined RAW image data by combining a plurality of RAW image data obtained by a plurality of shootings. Thereafter, the CPU 203 calculates a development parameter composition ratio from a plurality of pieces of shooting time information in a plurality of photographing operations (step S705), and performs parameter composition using the calculated composition ratio (step S706).

  The calculation of the composition ratio in step S705 can be determined by analyzing the shooting time information shown in FIG. 5 as described in the first embodiment. For example, the composite ratio of the development parameter suitable for the captured image is set to the development parameter suitable for the other captured image with respect to the captured image captured dark with respect to the brightness of the other captured image. A method such as lowering the synthesis ratio can be employed. As a result, in step S706, the CPU 203 synthesizes development parameters with a higher development parameter ratio suitable for a captured image having a higher synthesis ratio to obtain a composite development parameter.

  The CPU 203 develops the combined RAW image data obtained by combining the plurality of RAW image data generated by the plurality of shootings using the more preferable combined development parameter combined in Step S706 (Step S707). Thereby, a more preferable image can be obtained.

<Other embodiments>
Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

  For example, the shooting time information is not limited to the above-described aperture value (see FIG. 5), the presence / absence of a face in the shot image (see FIG. 6), the color conversion table setting value, etc. (third embodiment). As other shooting information, thumbnail information, exposure time, light adjustment correction, lens information, distance to the subject, flash emission state, custom setting information, and the like can be used.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to the image processing apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the image processing apparatus reads the program. It is a process to be executed.

100, 101 Captured image 103, 104 Composite image 105 Exposure correction image 201 Optical system 202 Image sensor 203 CPU

Claims (6)

Imaging means for capturing a subject image;
An acquisition unit that acquires shooting time information for each captured image at the time of shooting by the imaging unit;
When multiple exposure of a plurality of captured images obtained by the imaging unit is performed, the shooting time information acquired by the acquisition unit is analyzed for each of the plurality of captured images captured, and the plurality of captured images are synthesized. Setting means for setting the ratio;
An image pickup apparatus comprising: a combining unit that combines the plurality of captured images according to the combining ratio.   When the setting unit determines from the shooting time information that the scenes in which the plurality of captured images are similar are continuously captured, a combination ratio of a part of the captured images in the plurality of captured images The image pickup apparatus according to claim 1, wherein the image pickup device is set higher than other photographed images.   The imaging apparatus according to claim 2, wherein the part of the captured images is a first or last captured image of the continuous shooting.   The setting means, when judging from the shooting time information that there is an image including a face in the plurality of imaged images, increases the composition ratio of the imaged image including the face than the image captured without the face. The imaging apparatus according to claim 1.   The shooting information includes presence / absence of face, face position, face size, color conversion table setting value, white balance setting value, thumbnail information, exposure time, aperture value, shooting mode, ISO sensitivity, shutter speed, and light control. 5. The apparatus according to claim 1, wherein at least one of correction, exposure correction, lens information, distance to a subject, flash emission state, photometry information, and custom setting information is included. Imaging device. An image processing apparatus that generates a composite image by superimposing a plurality of image data in which the shooting time information is added to the captured image, by acquiring shooting time information for each captured image when shooting a subject image. And
Image processing characterized by comprising setting means for analyzing shooting time information added to each of the plurality of photographed images and setting a composite ratio of the photographed images when the plurality of photographed images are superimposed. apparatus.