JP2007251997A - Imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging unit capable of photographing while selecting an ordinary photographing mode during which image composition is not performed, or a wide dynamic range photographing mode during which image composition is performed. <P>SOLUTION: The imaging unit includes: an imaging means 1 including a solid-state imaging device which sequentially outputs image signals resulting from imaging an object; an image composition means 6 which composes a plurality of image signals sequentially output from the imaging means 1 to generate one synthetic image signal; a selection means 20 for selecting either one mode of the photographing mode during which image composition is not performed by the image composition means 6, or the photographing mode during which image composition is performed by the image composition means 6; and a control means 8 which allows ordinary image processing to be performed by obtaining image signals for one picture within one time of photographing when the photographing mode during which image composition is not performed, is selected by the selection means 20, or allows image composition to be performed by obtaining image signals for a plurality of pictures within one time of photographing when the photographing mode during which image composition is performed, is selected by the selection means 20. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus having a solid-state image pickup element such as an electronic still camera or a digital camera, and more particularly to an image pickup apparatus that aims to expand the dynamic range of the solid-state image pickup element.

  The imaging device having a solid-state imaging device has a narrower dynamic range than the silver salt film, so for example, when shooting a subject with a relatively bright background, the exposure time is adjusted so that the subject is appropriate. When the exposure time is adjusted so that the image signal is saturated and the image signal is saturated and the background is appropriate, the subject becomes underexposed and dark. Therefore, as a conventional imaging device of this type, a subject is photographed with different exposure amounts, and two image signals with different exposure amounts are added and combined, or two image signals with different exposure amounts are partially combined. There has been proposed a technique in which the dynamic range of a solid-state imaging device is expanded by switching and synthesizing them.

  However, in the case of combining the two image signals having different exposure amounts, such as the former, when the subject is moving or when a relative positional shift occurs between the images due to camera shake, There is a problem that the S / N is reduced in the portion having the relative positional deviation, and there is a problem that a false color or a pseudo contour is generated in the case of a color image.

  As a solution to such a problem, for example, a method is proposed in which a difference between two image signals having different exposure amounts is detected and the absolute value of the difference is subtracted from the added image signal as a correction amount. (For example, refer to Patent Document 1).

  However, in this case, since the deviation-corresponding portion is removed by subtraction correction, for example, when the main subject moves, the shape of the main subject in the composite image changes and is compared with a normal photographic image using a silver salt film. As a result, there is a problem that a sense of incongruity remains.

  Such a problem similarly occurs when two image signals having different exposure amounts are partially switched and combined as in the latter case. That is, as shown in FIG. 13, the image signal of the overexposed portion B (for example, the background portion) of the image A in the long-time exposure is changed to the image signal obtained by adjusting the gain of the corresponding portion D of the image C in the short-time exposure. When the main subject F moves from right to left in FIG. 13 between the short-time exposure and the long-time exposure when the composite image E is obtained by switching to, the short-time exposure corresponding to the movement amount The image signal of the main subject portion (portion G in the composite image E) is also synthesized, so that the main subject F is doubled and the composite image E remains uncomfortable.

JP-A-2-280585

  SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus capable of selecting and photographing a normal photographing mode in which image composition is not performed and a wide dynamic range photographing mode in which image composition is performed.

The invention of an imaging apparatus according to claim 1 that achieves the above object is as follows:
An imaging means including a solid-state imaging device that sequentially outputs image signals obtained by imaging a subject image;
Image synthesizing means for synthesizing the plurality of image signals sequentially output from the imaging means to generate one synthesized image signal;
A selection means for selecting one of a shooting mode in which the image composition is not performed by the image composition means and a photographing mode in which the image composition is performed by the image composition means;
When a shooting mode in which the image synthesis is not performed is selected by the selection unit, an image signal for one screen is obtained by one shooting and normal image processing is performed, and the image synthesis is performed by the selection unit. When the mode is selected, control means for obtaining image signals for a plurality of screens in one shooting and performing the image synthesis;
It is characterized by having.

The invention according to claim 2 is the imaging apparatus according to claim 1,
The selection means has an input key for selecting each mode manually.

The invention according to claim 3 is the imaging apparatus according to claim 1,
The selection means is configured to automatically select each mode based on an image signal output from the imaging means.

The invention according to claim 4 is the imaging apparatus according to claim 1,
The imaging means is configured to sequentially obtain image signals of subject images having different exposure amounts from the solid-state imaging device,
The image synthesizing unit is configured to generate one synthesized image signal by adding and synthesizing a plurality of image signals of the subject images having different exposure amounts sequentially output from the imaging unit. To do.

The invention according to claim 5 is the imaging apparatus according to claim 4,
The image synthesizing unit is configured to generate one synthesized image signal having a wide dynamic range by adding and synthesizing a plurality of image signals of the subject images having different exposure amounts sequentially output from the imaging unit. It is characterized by this.

The invention according to claim 6 is the imaging apparatus according to claim 5,
The selection means is configured to select a photographing mode in which the image composition output from the image pickup means is detected and the image composition is automatically performed.

  According to the present invention, since the selection means for selecting one of the photographing mode in which the image composition means does not perform the image composition and the photographing mode in which the image composition means performs the image composition is provided, the image composition is not performed normally. A shooting mode and a wide dynamic range shooting mode for combining images can be selected and shot.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a basic configuration of an electronic still camera as an imaging apparatus according to the present invention. This electronic still camera uses a single-plate color CCD image pickup device 1 having an electronic shutter function. A subject image is formed on the CCD image pickup device 1 through a lens 2 and an aperture / shutter mechanism 3. The subject image is photoelectrically converted and output as an image signal. The image signal of the subject image from the CCD image pickup device 1 is amplified by the amplifier 4 after being removed by a correlated double sampling circuit or the like (not shown) and further converted into a digital signal by the A / D converter 5. It is supplied to the camera signal processing circuit 6 where it is processed as image data.

  The output of the A / D converter 5 is also supplied to the AF, AE, and AWB detection circuit 7, where AF detection processing for extracting AF information for automatically controlling the focus prior to actual photographing. Then, AE detection processing for extracting AE information for automatically controlling exposure and AWB detection processing for extracting AWB information for automatically setting white balance are performed. The AF information, AE information, and AWB information from the AF, AE, AWB detection circuit 7 are supplied to the lens 2, the aperture / shutter mechanism 3 and the camera signal processing circuit 6 through the CPU 8, respectively.

  The camera signal processing circuit 6 and the CPU 8 are connected to a bus line 9, and a DRAM 11 used as a working memory when performing color processing of image data or the like is connected to the bus line 9 via a memory controller 10. In addition, a compression circuit (JPEG) 12 for compressing image data from the camera signal processing circuit 6 is connected. In addition, a memory card I / F 14 is connected to the bus line 9 in order to record the image data compressed by the compression circuit 12 in the memory card 13, and the image data recorded in the memory card 13 is stored in the bus line 9. A liquid crystal display element (LCD) 16 is connected via a display circuit 15 to read and display or to display a shooting state. Further, a PCI / F 18 for transferring image data recorded on the memory card 13 to a personal computer (PC) 17 is connected to the bus line 9.

  The CPU 8 is connected to a strobe device 19 that is controlled based on AE information from the AF, AE, and AWB detection circuit 7 and an input key 20 that sets various shooting modes, drives a trigger switch, and the like. Yes. The CCD image pickup device 1 is driven by a timing pulse from a timing generator (TG) 21 under the control of the CPU 8.

  In the imaging apparatus shown in FIG. 1, a normal shooting mode in which image synthesis is not performed and a wide dynamic range shooting mode in which image synthesis is performed are manually selected by operating the input key 20, or from the CCD imaging device 1. The overexposure of the image signal is detected and automatically selected by the CPU 8, and the photographing operation is controlled by the CPU 8 in accordance with the selected photographing mode. That is, when the normal shooting mode is selected, an image signal for one screen is obtained from the CCD image pickup device 1 by one shooting by a normal shooting operation, and when the wide dynamic range shooting mode is selected, The electronic shutter function of the CCD image sensor 1 or a combination of this electronic shutter function and the aperture / shutter mechanism 3 is a known method for a plurality of screens (in this case, two different exposure amounts from the CCD image sensor 1 in one shooting). Image signal) is obtained, and the camera signal processing circuit 6 performs image data processing corresponding to the shooting mode.

  FIG. 2 is a block diagram showing a configuration of an example of the camera signal processing circuit 6 shown in FIG. This camera signal processing circuit 6 is provided with an input side changeover switch 25 and an output side changeover switch 26, and one changeover contact of the input side changeover switch 25 is connected to one of the output side changeover switches 26 via the camera signal image processing circuit 27. Connect to the switching contact. Further, the other switching contact of the input side switching switch 25 is connected to the image switching switch 28, and one switching contact of the image switching switch 28 is passed through the image data buffer 29, the open / close switch 30 and the process circuit 31 to the image composition processing circuit. The other switching contact is connected to the other input terminal of the image composition processing circuit 32 via the image data buffer 33, the open / close switch 34 and the process circuit 35. Further, the output terminal of the image composition processing circuit 32 is connected to the input terminal of the image compression processing circuit 36, and the output terminal of the image compression processing circuit 36 is connected to the other switching contact of the output side changeover switch 26.

  The input side changeover switch 25 and the output side changeover switch 26 are controlled based on the photographing mode signal ii from the CPU 8, and the image changeover switch 28 and the open / close switches 30, 34 are applied to the image composition processing control signal jj from the CPU 8. Based on this, the switching control circuit 37 is used for control. The image composition processing circuit 32 is supplied with an exposure amount ratio data signal mm between two image signals having different exposure amounts from the CPU 8 in the wide dynamic range photographing mode.

  That is, in the normal shooting mode, the input side changeover switch 25 and the output side changeover switch 26 are connected to the camera signal image processing circuit 27 side by the shooting mode signal ii from the CPU 8, and the A / D converter 5 shown in FIG. Is supplied to the camera signal image processing circuit 27 through the input side changeover switch 25, where AWB processing based on the AWB information from the CPU 8 and color information interpolation based on the color filter structure of the CCD image pickup device 1 are performed. Normal image processing such as processing and contrast enhancement processing is performed, and the image signal hh by the normal photographing is output to the bus line 9 as the final output image signal kk through the output side changeover switch 26.

  On the other hand, in the wide dynamic range shooting mode, the input side changeover switch 25 is connected to the image changeover switch 28 side and the output side changeover switch 26 is connected to the image compression processing circuit 36 side according to the shooting mode signal ii from the CPU 8, respectively. The image changeover switch 28 and the open / close switches 30 and 34 are controlled via the changeover control circuit 37 based on the image composition processing control signal jj from the CPU 8, and two image signals having different exposure amounts are supplied to the image data buffers 29 and 33. Store.

  That is, first, the open / close switches 30 and 34 are opened, and the image changeover switch 28 is switched in synchronism with the fetching of two image signals having different exposure amounts, so that an image signal having a small exposure amount from the A / D converter 5 is obtained. The aa is stored in the image data buffer 29 and the image signal aa having a large exposure amount from the A / D converter 5 is stored in the image data buffer 33, respectively. Thereafter, the open / close switches 30 and 34 are simultaneously closed, and an image signal bb with a small exposure amount is supplied from the image data buffer 29 to the process circuit 31 through the open / close switch 30 and is exposed from the image data buffer 33 in synchronization therewith. A large amount of the image signal cc is supplied to the process circuit 35 via the open / close switch 34.

  The process circuits 31 and 35 perform image processing similar to that of the camera signal image processing circuit 27 to generate an image signal dd having a small exposure amount from the process circuit 31 and an image signal ee having a large exposure amount from the process circuit 35. The image composition processing circuit 32 supplies the image signal in synchronization with the image composition processing circuit 32. In the image composition processing circuit 32, the image signal dd and the image signal ee are partially switched and synthesized using the exposure amount ratio data signal mm from the CPU 8. The wide dynamic range synthesized image signal ff synthesized by the image synthesis processing circuit 32 is supplied to the image compression processing circuit 36 to perform compression processing so that the dynamic range becomes an appropriate exposure level, and the synthesized image compressed image signal. gg is output to the bus line 9 as the final output image signal kk through the output side changeover switch 26.

  In FIG. 2, two sets of image data buffers 29 and 33 and open / close switches 30 and 34 are provided. However, for example, the image data buffer 33 and open / close switch 34 are omitted, and the image data buffer 29 has an image signal with a small exposure amount. After storing aa, the open / close switch 30 is closed in synchronism with the switching of the image changeover switch 28, and the image signal bb having a small exposure amount from the image data buffer 29 and the exposure amount from the A / D converter 5 are stored. The large image signal aa can be supplied in synchronization with the corresponding process circuits 31 and 35.

  The purpose of the electronic still camera shown in FIG. 1 is to always obtain an image without a sense of incongruity when a plurality of image signals having different exposure amounts are partially switched and combined to obtain an image with a wide dynamic range.

  Therefore, in the first embodiment of the present invention, when the image synthesis processing circuit 32 in FIG. 2 synthesizes the image signals dd and ee having different exposure amounts, the level ratio between the image signals is calculated for each corresponding pixel. Then, a pixel whose level ratio is equal to or greater than a predetermined value compared with the exposure amount ratio data signal mm from the CPU 8 is detected as a composite incompatible pixel, and the pixel signal of the composite incompatible pixel is located in the vicinity of the composite incompatible pixel. In addition, correction is performed by interpolation based on the combined pixel signal of the pixels that are not detected as non-combining pixels.

  FIG. 3 is a block diagram showing a configuration of an example of the image composition processing circuit 32 in the first embodiment. In the image composition processing circuit 32, an image signal ee having a large exposure amount is supplied to one input terminal of the image composition unit 41 and also supplied to one input terminal of the image data comparison unit 42. The image signal dd having a small exposure amount is supplied to the other input terminal of the image data comparison unit 42 and also to the multiplier 43. The multiplier 43 performs gain adjustment for amplifying the image signal dd by the exposure amount ratio based on the exposure amount ratio data signal mm from the CPU 8, and outputs the image signal pp having a small exposure amount adjusted by the gain to the image composition unit 41. Supply to the other input terminal.

  The image data comparison unit 42 compares the level ratio of the image signals dd and ee in pixel units, that is, for each same pixel position, and supplies the image data comparison signal nn to the comparison calculation unit 44. The comparison operation unit 44 compares the image data comparison signal nn with the reference value based on the exposure amount ratio data signal mm from the CPU 8 and outputs a comparison result logical data signal oo. The comparison result logical data signal oo is combined with the image. Are supplied to the unit 41 and the composition incompatible pixel region extraction unit 45, respectively. Here, as the comparison result logical data signal oo, when each pixel of the image data comparison signal nn exceeds the reference value, the logic “1” is set as a non-compliance pixel, and when it is equal to or lower than the reference value, the logic “0” is set as a combination compatible pixel. Is output.

  Based on the logical value of the comparison result logical data signal oo, the image synthesizing unit 41, when the logic is “0”, that is, the composition-adapted pixel, the image signal pp with a small exposure amount adjusted for gain and the image signal ee with a large exposure amount The corresponding pixel signals are switched and combined as described with reference to FIG. 13, and when the logic is “1”, that is, the combination incompatible pixel, the corresponding pixel signals are not switched and combined. The synthesized image signal qq obtained by synthesizing pixels other than the non-compatibility pixels obtained from the image synthesizing unit 41 is supplied to the interpolation target pixel detecting unit 46 and the low-pass filter (LPF) 47, respectively.

  On the other hand, the combined nonconforming pixel region extraction unit 45 extracts the combined nonconforming pixel region for each line (horizontal direction) based on the logical value of the comparison result logical data signal oo, that is, based on the region where logic “1” continues. The region signal rr is supplied to the interpolation target pixel detection unit 46 and the synthesis incompatible pixel interpolation unit 48, respectively. In the interpolation target pixel detection unit 46, based on the synthesized image signal qq synthesized from the image synthesis unit 41 other than the synthesis non-conforming pixels and the region signal rr from the synthesis non-conforming pixel region extraction unit 45, A pixel to be interpolated (correction reference pixel) for interpolating the pixel signal is detected, and the correction reference pixel signal ss (including information on the composite image signal qq) is supplied to the composite nonconforming pixel interpolation unit 48.

  The synthesis nonconforming pixel interpolation unit 48 interpolates the pixel signal of the synthesis nonconforming pixel corresponding to the region signal rr from the synthesis nonconforming pixel region extraction unit 45 based on the correction reference pixel signal ss from the interpolation target pixel detection unit 46. The corrected combined image signal tt that has been corrected and interpolated is supplied to the low-pass filter 47. In the low pass filter 47, the composite corrected image signal tt is low pass filtered at least in the vertical direction using the composite image signal qq from the image compositing unit 41, and the composite image signal ff is sent to the image compression processing circuit 36 shown in FIG. To supply.

  FIG. 4 is a flowchart showing in detail the correction processing operation of the pixel signal of the non-conforming pixel by the non-conforming pixel region extracting unit 45, the interpolation target pixel detecting unit 46, the non-conforming pixel interpolating unit 48 and the low-pass filter 47 shown in FIG. It is. First, the combined nonconforming pixel area extracting unit 45 determines whether or not the reference pixel is a combined nonconforming pixel based on the comparison result logical data signal oo from the comparison operation unit 44 (step S1). In the case of logic “1”, combined nonconforming pixels that are continuous in the horizontal direction are integrated as one region (step S2), and the length (S) of the combined nonconforming pixel region is calculated (step S3). . The combined non-conforming pixel for this length is defined as a region signal rr.

  Thereafter, in the interpolation target pixel detection unit 46, pixels at a distance of S × β (where β is an arbitrary adjustment parameter; β> 0) are corrected reference pixels (LPx, RPx) from the left and right ends of the non-combination pixel region. (Step S4), the luminance signal (Y-LPx, Y-RPx) of each correction reference pixel is calculated (step S5), and the magnitude relationship is determined (step S6). If Y-LPx = Y-RPx, the color difference signals (Cr-LPx, Cr-RPx) and (Cb-LPx, Cb-RPx) of each correction reference pixel are calculated (step S7). In the case of -LPx> Y-RPx, the correction area is expanded by d pixels (where d ≦ S × β) from the left end of the non-combining pixel area (step S8), and then the process of step S7 is executed. In the case of -LPx <Y-RPx, the correction region is expanded by d pixels from the right end of the non-combining pixel region (step S9) and then the processing of step S7 is executed.

  Thereafter, the composite nonconforming pixel interpolation unit 48 uses the pixel signal of the correction reference pixel to smooth and interpolate the pixel signal of the correction region (composite nonconforming pixel region + extended portion) into luminance and color discrimination (step S10). When the interpolation processing for the correction region is completed, the low-pass filter 47 performs low-pass filter processing for luminance and color discrimination on the correction region (step S11), and converts the interpolated luminance and color difference signals into RGB signals. (Step S12) and output as a composite image signal ff.

  In the first embodiment, the combined incompatible pixel area is calculated in the horizontal direction and the pixel signal is corrected. However, the combined incompatible pixel area is calculated in the vertical direction in the same manner, and the pixel signal is calculated. It is also possible to perform correction based on the pixel signal of the correction reference pixel in the horizontal and vertical directions, and to perform low-pass filter processing in the horizontal and vertical two-dimensional directions. In this way, it is possible to obtain a composite image that is more comfortable with respect to the surroundings and the entire composite image. In the first embodiment, the composition nonconformity is detected and corrected in units of pixels. However, it may be configured to detect and correct in units of blocks having a predetermined number of pixels.

  In the second embodiment of the present invention, when the image synthesis processing circuit 32 in FIG. 2 synthesizes the image signals dd and ee having different exposure amounts, the motion vector is calculated based on the comparison of the image signals, and the motion A pixel having a vector size exceeding a predetermined value is detected as a composite nonconforming pixel, and the pixel signal at the start point of the motion vector is based on a composite pixel signal that is not a composite nonconforming part in a predetermined small block including the end point of the motion vector. And interpolating and correcting the pixel signal of the unsuitable portion of the remaining area including the end point of the motion vector based on the pixel signal of the corresponding area of the image signal dd.

  FIG. 5 is a block diagram showing a configuration of an example of the image composition processing circuit 32 in the second embodiment. In this image composition processing circuit 32, an image signal ee having a large exposure amount is supplied to one input terminal of the image composition unit 51, and an image signal dd having a small exposure amount is supplied from the CPU 8 to the exposure amount ratio data signal from the CPU 8. Based on mm, gain adjustment is performed to amplify by the exposure amount ratio, and an image signal pp having a small exposure amount after gain adjustment is supplied to the other input terminal of the image composition unit 51. The image signals pp and ee are supplied to the corresponding edge detection units 53 and 54 to detect the edges of the images, respectively, and the edge detection signals uu and vv are supplied to the corresponding block setting units 55 and 56, respectively. A block signal ww, xx is generated and supplied to the motion vector calculation unit 57. The motion vector calculation unit 57 calculates a motion vector between both images based on the block signal ww, xx, and the motion vector data signal yy Are supplied to the image composition unit 51, small block setting unit 58, small block average calculation unit 59, and vector pixel interpolation unit 60, respectively.

  The image compositing unit 51 compares the magnitude of the motion vector data signal yy with a predetermined threshold value, and when the magnitude of the motion vector data signal yy is equal to or smaller than the predetermined threshold value, the exposure adjustment amount adjusted for gain is used as a compositing compatible pixel. The pixel signals corresponding to the small image signal pp and the large exposure image signal ee are combined by switching as described with reference to FIG. 13, and when the magnitude of the motion vector data signal yy exceeds a predetermined threshold, the combination is performed. The pixel signal corresponding to the non-conforming pixel is not switched and combined. In this way, the synthesized image signal qq obtained by synthesizing other than the non-synthesizing pixels obtained from the image synthesizing unit 51 is supplied to the small block setting unit 58 and the low-pass filter (LPF) 61, respectively.

  On the other hand, the small block setting unit 58 compares the magnitude of the motion vector data signal yy starting from the reference pixel with a predetermined threshold, and only when the magnitude of the motion vector data signal yy exceeds the predetermined threshold As shown in FIG. 6 (A), a small block (here, 3 × 3 pixel block) centering on the end point of the motion vector is set from the synthesized image signal qq as a reference non-compliance pixel. The small block setting signal zz (including information of the composite image signal qq) is supplied to the small block average calculation unit 59, where color is generated using the composite pixel signal existing in the small block as shown in FIG. 6B. An average value (Rv, Gv, Bv) is calculated for each signal, and the small block average value signal aaa (including information of the composite image signal qq) is supplied to the vector pixel interpolation unit 60.

  The vector pixel interpolation unit 60 is also supplied with an image signal pp having a small exposure amount adjusted for gain, and as shown in FIG. 6C, the motion vector of the motion vector is returned so as to return the moving subject to the state before the movement. The pixel signal of the synthesis incompatibility pixel at the start point is interpolated by the small block average value signal aaa, and the pixel signal of the synthesis incompatibility portion of the remaining area including the end point of the motion vector corresponds to the image signal pp whose gain has been adjusted. Interpolation is performed using the pixel signal of the region, and the interpolated composite corrected image signal bbb is supplied to the low-pass filter 61. In the low-pass filter 61, the composite corrected image signal bbb is subjected to the low-pass filter processing in the same manner as in the first embodiment using the composite image signal qq obtained by combining the non-compatibility pixels from the image compositing unit 51, and the composite image signal get ff.

  FIG. 7 is a flowchart showing in detail the interpolation processing operation of the pixel signals of the nonconforming pixels by the small block setting unit 58, the small block average calculation unit 59, the vector pixel interpolation unit 60, and the low pass filter 61 shown in FIG. First, the small block setting unit 58 determines whether or not the motion vector starting from the reference pixel exceeds a predetermined threshold Th based on the motion vector data signal yy from the motion vector calculation unit 57 (step S21). If it exceeds, a small block of 3 × 3 pixels centering on the end point of the motion vector is set based on the composite image signal qq from the image composition unit 51 with the reference pixel as the composition non-conforming pixel (step S22). The block setting signal zz is supplied to the small block average calculation unit 59.

The small block average calculation unit 59 receives the small block setting signal zz, and initializes the color signal variables (R _B , G _B , B _B ) and the internal counter (Count) for counting the number of additions (step S23). While scanning the set small block (step S24), it is determined whether or not the composite pixel signal (R, G, B) exists in the small block (step S25), and the composite pixel signal exists. In step S26, the composite pixel signal is added to the corresponding variable and the counter is incremented. After that, when the scanning of the small block is completed (step S27), the average value (Rv, Gv, Bv) of the variables (R _B , G _B , B _B ) is calculated using the Count value (step S28). The average value is supplied to the vector pixel interpolation unit 60 as the small block average value signal aaa.

  The vector pixel interpolating unit 60 interpolates the pixel signal of the synthesis incompatible pixel at the start point of the motion vector with the average value signal aaa in the small block so as to return the moving subject to the state before the movement (step S29), and the motion The pixel signal of the unsuitable portion of the remaining area including the end point of the vector is interpolated with the pixel signal of the corresponding area of the image signal pp whose gain has been adjusted (step S30), and supplied to the low-pass filter 61 as the combined corrected image signal bbb. Then, low-pass filter processing for each color signal is executed on the pixel signal on the motion vector interpolated using the composite image signal qq (step S31), and a composite image signal ff is obtained.

  That is, in the second embodiment, in the wide dynamic range photographing mode, the image signal dd with a small exposure amount shown as the short-time exposure image in FIG. 8A from the process circuits 31 and 35 (see FIG. 2), When an image signal ee having a large exposure amount shown as a long-time exposure image in 8B is obtained, the level of the short-time exposure image is adjusted to the level of the long-time exposure image as shown in FIG. The gain adjustment is performed as described above, and the motion vector is detected from the short-exposure image and the long-exposure image that have been gain-adjusted, and the switching composition is performed except for pixels whose motion vector size exceeds a predetermined threshold. A synthesized image as shown in FIG. 8D is obtained (however, at this time, the start point of the motion vector shown in FIG. 8D is not replaced). For pixels corresponding to a motion vector, for each motion vector, the pixel signal of the pixel at the start point is the average of the synthesized pixel signals in the small block centered on the end point of the motion vector as described in FIG. The pixel signal of the remaining area including the end point of the motion vector is interpolated with the value (at this time, the start point of the motion vector shown in FIG. 8D is replaced) with a white frame in FIG. The corrected composite image shown in FIG. 8E is obtained by interpolating with the pixel signal of the corresponding region with the gain adjustment shown. Therefore, if the above processing is performed on a plurality of motion vectors shown in FIG. 9A, a corrected composite image as shown in FIG. 9B can be finally obtained.

  In the above description, the pixel signal at the boundary before and after the movement of the moving subject is corrected (interpolated), but the same applies to the case of correcting (interpolating) the background portion. In this case, first, similarly to the correction for the moving subject, the average value of the combined pixel signals in the small block set for the end point of the motion vector is calculated, and the average value is calculated as the pixel at the start point of the motion vector. Interpolate to signal. The average value in this case is mainly information on the moving subject. Next, pixel signals other than the start point of the motion vector are interpolated with pixel signals in the corresponding region of the short-time exposure image (corresponding to an image signal with a small exposure amount). At this time, since the information to be interpolated is the background portion, the gain of the short-time exposure image is adjusted by a factor of 1 so that the background portion is not overexposed. By performing the interpolation processing in this way, the motion vector is interpolated into the information of the background portion only with the moving subject having the moving subject as the main information, and the others. FIGS. 10A to 10E show an outline of the interpolation processing in this case. Details thereof are described above, the descriptions of FIGS. 8A to 8E, and FIG. 9A. ) And (B), it will be omitted because it is clear.

  In the second embodiment, the pixel signal at the start point of the motion vector is corrected by replacing it with the average value of the synthesized pixel signals in the small block centered on the end point, but is simply replaced with the pixel signal at the end point. It can also be corrected.

  In the third embodiment of the present invention, when the image synthesis processing circuit 32 in FIG. 2 synthesizes the image signals dd and ee having different exposure amounts, the difference between the image signals is calculated and the absolute value of the difference is calculated. A motion vector is calculated for a pixel region where the value of the motion vector is equal to or greater than a predetermined value, a pixel having a motion vector size exceeding a predetermined value is detected as a composition nonconforming pixel, and an end point of the motion vector centered on the start point of the motion vector is determined. Pixels lost due to the replacement by replacing the pixel signal of the nonconforming portion in the predetermined pixel block including the pixel signal of the nonconforming nonconforming portion in the same size pixel block centered on the end point of the motion vector The signal is corrected by the pixel signal in the corresponding region of the image signal with a small exposure amount obtained by adjusting the gain.

  FIG. 11 is a block diagram illustrating a configuration of an example of the image composition processing circuit 32 in the third embodiment. In this image composition processing circuit 32, an image signal ee having a large exposure amount is supplied to one input terminal of the image composition unit 70, and an image signal dd having a small exposure amount is supplied to the exposure amount ratio data signal from the CPU 8 in the multiplier 71. Based on mm, gain adjustment for amplification by the exposure amount ratio is performed, and the image signal pp having a small exposure amount that has been gain-adjusted is supplied to the other input terminal of the image composition unit. The image signals pp and ee are supplied to one input terminal of each of the corresponding AND gates 72 and 73 and also supplied to the image difference calculation unit 74. The image difference calculation unit 74 calculates the difference between the image signals pp and ee in pixel units, that is, for each same pixel position, compares the absolute value with a predetermined threshold value, and the absolute value of the difference exceeds the threshold value. Supplies the comparison result logical value data signal ccc of logic “1” to the other input terminals of the AND gates 72 and 73 when the logic is “1” or less than the threshold. This causes the AND gate 72 to output the image signal ddd in the region where the absolute value of the difference between the image signals pp and ee out of the image signal pp whose gain has been adjusted exceeds the threshold value. The image signal eee in the region where the absolute value of the difference between the image signals pp and ee exceeds the threshold is output.

  The image signals ddd and eee from the AND gates 72 and 73 are supplied to the corresponding edge detection units 75 and 76 to detect the edges of the images, respectively, and the edge detection signals fff and ggg are corresponding to the corresponding block setting units 77 and 77, respectively. 78 to generate block signals ww, xx and supply them to the motion vector calculation unit 79. The motion vector calculation unit 79 calculates motion vectors between the two images based on the block signals ww, xx, The motion vector data signal yy is supplied to the image composition unit 70, the corrected block setting unit 80, the corrected block replacement pixel determining unit 81, and the data correcting unit 82, respectively.

  As in the second embodiment, the image composition unit 70 compares the magnitude of the motion vector data signal yy with a predetermined threshold value, and if the magnitude of the motion vector data signal yy is equal to or smaller than the predetermined threshold value, As shown in FIG. 13, the corresponding pixel signals of the image signal pp having a small exposure amount adjusted for gain and the image signal ee having a large exposure amount are switched and combined as described with reference to FIG. 13, and the magnitude of the motion vector data signal yy is predetermined. When the threshold value is exceeded, switching / combination of the corresponding pixel signals as a composition incompatible pixel is not performed. In this way, the synthesized image signal qq obtained by synthesizing other than the non-synthesizing pixels obtained from the image synthesizing unit 70 is supplied to the corrected block setting unit 80 and the low-pass filter (LPF) 83, respectively.

  On the other hand, as shown in the flowchart in FIG. 12, the corrected block setting unit 80 compares the magnitude of the motion vector data signal yy starting from the reference pixel with a predetermined threshold Th (step S41), and the motion vector. Only when the magnitude of the data signal yy exceeds the predetermined threshold Th, the reference pixel is regarded as a non-compliance pixel, and the motion vector calculation calculation is performed in the same manner as the motion vector calculation calculation, with the motion vector start point (reference pixel) as the center. Then, the corrected block including the end point is set (step S42), and the corrected block setting signal hhh (including the information of the composite image signal qq) is supplied to the corrected block replacement pixel determining unit 81.

  The corrected block replacement pixel determining unit 81 sets a correction block having the same size as the corrected block centered on the end point of the motion vector based on the corrected block setting signal hhh and the motion vector data signal yy (step S43). ), And supplies the correction block setting signal iii (including information of the composite image signal qq) to the data correction unit 82.

  The data correction unit 82 is also supplied with the gain-adjusted image signal pp. Here, based on the correction block setting signal iii and the motion vector data signal yy, the pixel signal of the synthesis incompatible pixel in the correction target block is stored in the correction block. (Step S44), and the pixel signal of the region lost by the replacement is interpolated and corrected by the pixel signal of the corresponding region of the image signal pp with a small exposure amount adjusted for gain (step S45). ), The combined corrected image signal jjj subjected to the correction processing is supplied to the low-pass filter 83. The low-pass filter 83 performs low-pass filter processing for each color signal on the block to be corrected and the surrounding pixel signal corrected for the combined corrected image signal jjj (step S46), thereby obtaining a combined image signal ff.

  In the third embodiment, the pixel signal of the unsuitable portion in the corrected block is replaced with the combined pixel signal that is not the unsuitable portion in the correction block. However, interpolation is performed instead of the replacement. You can also. In this way, since there is no pixel signal lost in the correction block, interpolation processing of the pixel signal is not necessary, and the processing is simplified.

  In still another embodiment of the present invention, a program for executing the processing operation in the wide dynamic range shooting mode described in each of the above embodiments or modifications is recorded on a recording medium, and the recording apparatus stores the recording medium. The program recorded on the recording medium can be read and executed by the CPU 8 (see FIG. 1) of the imaging apparatus main body via the driver.

  According to each of the above-described embodiments, when a plurality of image signals having different exposure amounts are partially switched and combined to generate an image signal with a wide dynamic range, a non-compliance portion of the image signal is detected and the combination is detected. Since the pixel signal of the non-conforming part is corrected, for example, when the main subject moving with a bright background is photographed, the influence at the time of composition of the image signal can be minimized and compared with a normal photographic image. It is possible to obtain an image with a wide dynamic range without any discomfort.

It is a block diagram which shows the basic composition of the electronic still camera as an imaging device concerning this invention. It is a block diagram which shows the structure of an example of the camera signal processing circuit shown in FIG. FIG. 3 is a block diagram showing a configuration of an example of an image composition processing circuit shown in FIG. 2 in the first embodiment of the present invention. It is a flowchart for demonstrating operation | movement of the principal part in 1st Embodiment. It is a block diagram which shows the structure of an example of the image composition processing circuit shown in FIG. 2 in 2nd Embodiment of this invention. It is a figure for demonstrating the operation | movement. Similarly, it is a flowchart for demonstrating operation | movement of the principal part. Similarly, it is a figure for demonstrating operation | movement. Similarly, it is a figure for demonstrating operation | movement. Similarly, it is a figure for demonstrating operation | movement. It is a block diagram which shows the structure of an example of the image composition processing circuit shown in FIG. 2 in 3rd Embodiment of this invention. It is a flowchart for demonstrating operation | movement of the principal part. It is a figure for demonstrating the method to obtain an image with a wide dynamic range by partially switching two image signals from which exposure amount differs.

Explanation of symbols

1 Color CCD Image Sensor 2 Lens 3 Aperture / Shutter Mechanism 6 Camera Signal Processing Circuit 8 CPU
DESCRIPTION OF SYMBOLS 20 Input key 21 Timing generator 32 Image composition processing circuit 41 Image composition part 42 Image data comparison part 43 Multiplier 44 Comparison operation part 45 Composition nonconformity pixel area extraction part 46 Interpolation object pixel detection part 47 Low-pass filter (LPF)
48 synthesis incompatible pixel interpolation unit 51 image synthesis unit 52 multiplier 53, 54 edge detection unit 55, 56 block setting unit 57 motion vector calculation unit 58 small block setting unit 59 small block average calculation unit 60 vector pixel interpolation unit 61 low-pass filter ( LPF)
DESCRIPTION OF SYMBOLS 70 Image composition part 71 Multiplier 72,73 AND gate 74 Image difference calculating part 75,76 Edge detection part 77,78 Block setting part 79 Motion vector calculation part 80 Corrected block setting part 81 Corrected block replacement pixel determination part 82 Data Correction unit 83 Low-pass filter (LPF)

Claims (6)

An imaging means including a solid-state imaging device that sequentially outputs image signals obtained by imaging a subject image;
Image synthesizing means for synthesizing the plurality of image signals sequentially output from the imaging means to generate one synthesized image signal;
A selection means for selecting one of a shooting mode in which the image composition is not performed by the image composition means and a photographing mode in which the image composition is performed by the image composition means;
When a shooting mode in which the image synthesis is not performed is selected by the selection unit, an image signal for one screen is obtained by one shooting and normal image processing is performed, and the image synthesis is performed by the selection unit. When the mode is selected, control means for obtaining image signals for a plurality of screens in one shooting and performing the image synthesis;
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the selection unit includes an input key for manually selecting the modes.   The imaging apparatus according to claim 1, wherein the selection unit is configured to automatically select the modes based on an image signal output from the imaging unit. The imaging means is configured to sequentially obtain image signals of subject images having different exposure amounts from the solid-state imaging device,
The image synthesizing unit is configured to generate one synthesized image signal by adding and synthesizing a plurality of image signals of the subject images having different exposure amounts sequentially output from the imaging unit. The imaging device according to claim 1.   The image synthesizing unit is configured to generate one synthesized image signal having a wide dynamic range by adding and synthesizing a plurality of image signals of the subject images having different exposure amounts sequentially output from the imaging unit. The imaging apparatus according to claim 4.   The said selection means is comprised so that the overshoot of the image signal output from the said imaging means may be detected and the imaging | photography mode which performs the said image synthesis automatically may be selected. Imaging device.

Images