JP2001136434A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device that can extend a dynamic range of even an object with a motion such as a moving object and extend an optimum dynamic range in response to the object image. SOLUTION: This image pickup device is provided with an optical means 1 that forms the image of an object, a luminous quantity ratio variable means 2 that separates the same object image and revises a luminous quantity ratio to form a plurality of object images, a plurality of image pickup means 3A, 3B that respectively pick up a plurality of the object images formed by the luminous quantity ratio variable means to provide an output of an image signal and an image composition means that composes image signals outputted form the image pickup means 3A, 3B to generate one composite image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging device,
More specifically, the present invention relates to an imaging device used for a digital still camera using a solid-state imaging device such as a CCD.

[0002]

2. Description of the Related Art Conventionally, an imaging device using a solid-state imaging device such as a CCD as an imaging device has been widely used because of its low cost and high reliability. In recent years, it has attracted attention as a substitute for a silver halide photography device. ing. However, a solid-state imaging device such as a CCD has a small dynamic range with respect to incident light as compared with a silver halide film or the like. Dark subjects are crushed. Such a narrow dynamic range is less than that of a silver halide photography device.

Therefore, a method of expanding the dynamic range by devising a structure of a solid-state image pickup device different from that of a normal CCD has been devised, as disclosed in Japanese Patent Application Laid-Open Nos. 4-4680 and 5-22670. . In such a scheme,
Since the dynamic range of the image sensor itself is to be expanded, the structure of the semiconductor becomes complicated, and a high-density pixel image sensor having a large number of pixels cannot be manufactured. Also, the effect of widening the dynamic range cannot be expected. In addition, if the image sensor itself is compatible with a wide dynamic range, gradation cannot be properly expressed when photographing a subject having a small difference in luminance within the angle of view.

As disclosed in Japanese Patent Application Laid-Open No. 1-93967, a method of enlarging a dynamic range by photographing a plurality of times with different exposure amounts using a shutter / aperture and using a plurality of captured images has been devised. I have. As described above, an imaging device that synthesizes a plurality of image signals having different exposure times has been devised. However, since imaging is basically performed a plurality of times, it is impossible to capture a moving object or the like.

Further, even in Japanese Patent Application Laid-Open No. 7-131799, a plurality of photographed images having different exposure times are similarly synthesized, so that it is impossible to photograph a moving object or the like. Also, a method of changing the exposure amount using an aperture or the like is disclosed. However, if a plurality of images with different apertures are combined, normal image combination cannot be performed due to differences in the degree of background blur.

[0006] Further, as disclosed in Japanese Patent Application Laid-Open No. 5-64070, a method of synthesizing images with different exposure amounts using a beam splitter has been devised. In such a method, unlike the method in which the exposure amount is changed by using an aperture or a shutter, the above-described problem does not occur.

[0007]

However, the above-mentioned Japanese Patent Application Laid-Open No. 5-64070 has the following problems. First, since different exposures are formed by using a beam splitter, the ratio of the exposures of the two images becomes constant, and the dynamic range is constantly expanded at a constant rate, which corresponds to the shooting subject. However, there is a problem that the dynamic range cannot be expanded.
Second, when the ratio of the exposure difference is small, there is a problem that overexposure or crushing occurs when an image of a subject having a large luminance difference is taken. Thirdly, when the ratio of the exposure difference is large, noise accompanying the expansion of the dynamic range occurs in the image created by the signal on the high luminance side, and the intermediate luminance portion becomes discontinuous. For,
There is a problem that an appropriate combined image cannot be obtained. Fourth, when an image of a subject having a small luminance difference is taken, there is a problem that information of the image pickup device on one side becomes meaningless. Fifth, since the constant light amount is divided into the image sensors on one side, a loss of the light amount occurs, which causes a problem in low-luminance photographing.

The present invention has been made in view of the above, and it is possible to expand a dynamic range even for a moving subject such as a moving object, and further to expand an optimum dynamic range adapted to a subject image. It is an object to provide a simple imaging device.

[0009]

According to a first aspect of the present invention, there is provided an optical apparatus for forming a subject image, wherein the same subject image is separated and a light amount ratio is changed. Light amount ratio varying means for forming a plurality of subject images by
Generating a single image by synthesizing a plurality of image pickup units each of which picks up a plurality of subject images formed by the light amount ratio changing unit and outputs image signals, and an image signal output from each of the plurality of image pickup units; Image synthesizing means.

According to the invention, the light quantity ratio varying means separates the same subject image and changes the light quantity ratio to form a plurality of subject images. Are respectively imaged and an image signal is output, and the image synthesizing means synthesizes the image signals respectively output from the plurality of imaging means to generate one image.

The invention according to claim 2 is based on claim 1.
In the imaging device described in (1), the light amount ratio varying means forms a plurality of subject images having different spectral transmission characteristics. According to the present invention, when forming a plurality of subject images, the light quantity ratio varying means forms a plurality of subject images having different spectral transmission characteristics and outputs long-wavelength and short-wavelength subject images.

The invention according to claim 3 is based on claim 1.
In the imaging device described in (1), each of the plurality of imaging units has a different spectral sensitivity. According to the present invention, the plurality of image pickup units have different spectral sensitivities, and realize the expansion of the dynamic range for each color.

Further, the invention according to claim 4 is based on claim 1.
The light amount ratio of the light amount ratio variable means is determined based on image signals output from the plurality of image pickup means. According to the above invention, the light amount ratio is determined based on the image signals output from the plurality of imaging units.

[0014] The invention according to claim 5 is based on claim 1.
3. The image pickup apparatus according to claim 1, further comprising: a noise removal unit that removes a noise component of an image signal output from the plurality of imaging units based on the image signals of the subject images having a plurality of different light amounts formed by the light amount ratio varying unit. Means. According to the invention, the noise removing unit removes noise components of the image signals output from the plurality of imaging units, based on the image signals of the subject images having the different light amounts formed by the light amount ratio varying unit.

The invention according to claim 6 is based on claim 1.
5. The imaging apparatus according to claim 1, further comprising a relative position displacement unit that relatively displaces the plurality of imaging units and a subject image input by the optical input unit, wherein each of the imaging units includes the relative displacement. And outputs the plurality of image signals respectively. According to the present invention, high image quality is achieved by using the image shifting method together.

According to a seventh aspect of the present invention, there is provided an optical unit for forming an object image, and a light amount ratio varying unit for separating the same object image and changing the light amount ratio to form a plurality of object images. A plurality of image pickup means for respectively picking up a plurality of subject images formed by the light amount ratio varying means and outputting image signals; and combining one image signal outputted from each of the plurality of image pickup means into one image A first imaging mode for generating one image by synthesizing image signals respectively output from the plurality of imaging means;
A second imaging mode for generating an image based on an image signal output from one of the plurality of imaging units.

According to the invention, in the first imaging mode, one image is generated by synthesizing image signals respectively output from the plurality of imaging means, while in the second imaging mode, one image is produced. An image is generated based on the image signal output from.

The invention according to claim 8 is based on claim 7.
In the imaging device described in (1), the first imaging mode and the second imaging mode are switched based on an image signal. According to the above invention, the first signal is generated based on the image signal.
And the second imaging mode.

According to a ninth aspect of the present invention, there is provided an optical device for forming an image of a subject, an image capturing device for capturing an image of the subject and outputting an image signal, and an image capturing device configured to capture images with different exposure amounts by the image capturing device. And a noise removing unit for removing a noise component of the image signal output from the imaging unit based on a plurality of image signals having different exposure amounts obtained as described above.

According to the above invention, the noise removing unit removes a noise component of the image signal output from the image pickup unit based on a plurality of image signals having different exposure amounts obtained by imaging the image pickup unit with different exposure amounts. I do.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS With reference to the accompanying drawings, a preferred embodiment of an image pickup apparatus according to the present invention will be described below. Camera].

[Summary of the Invention (Principle)] FIG. 1 is a diagram showing a schematic configuration of an imaging apparatus according to the present invention. The imaging apparatus shown in FIG. 1 is based on an optical unit 1 for imaging light from a subject and a light amount ratio setting signal input from a control unit (not shown).
Light amount ratio varying means 2 for separating the formed object images and changing the light amount ratio to output two object images, and image pickup means 3A and 3B for respectively taking the input object images and outputting them as image signals. And a synthesizing unit 4 that synthesizes two image signals input from the imaging units 3A and 3B to generate one image.

The general operation of the imaging apparatus having the above configuration will be described. First, the optical means 1 forms a subject image, and the light quantity ratio varying means 2 separates the formed subject image and, in accordance with a light quantity ratio setting signal determined based on the luminance distribution of the subject, The ratio of the light amounts of the two subject images is changed and output. One subject image is input to the imaging unit 3A, and the other subject image is input to the imaging unit 3A. And the imaging means 3A, 3B
Outputs an image signal obtained by capturing each of the input subject images to the synthesizing means 4. The combining means 4 combines the two input image signals to create a wide dynamic range image.

Next, the method of expanding the dynamic range according to the present invention will be described with reference to FIGS. FIG. 2 is a diagram illustrating a schematic configuration of a conventional imaging device that does not expand a dynamic range. In the conventional image pickup apparatus shown in FIG. 2, a subject image formed by the optical means 1 is picked up by the image pickup means 3 and output as it is as an image signal, and the dynamic range for incident light is not expanded.

The relationship between the output signal of the image pickup means and the incident light will be described with reference to FIG. FIG. 3 is a diagram illustrating the relationship between the incident light and the output signal of the imaging unit. In the figure, the horizontal axis represents the incident light, and the vertical axis represents the output signal of the imaging means.
In FIG. 1, reference numeral 1 denotes a saturation signal region, 2 denotes a proper signal region, 3 denotes a noise region, 4 denotes a proper exposure region, and 5 denotes a saturation region.

As shown in FIG. 3, the output signal of the image pickup means is obtained in the proper exposure area 4 in proportion to the incident light, but the output signal is saturated in the saturation area 5 of higher brightness. A proper output signal cannot be obtained. In a noise region 3 which is a darker portion than the proper exposure region 4, a proper output signal cannot be obtained due to noise of the solid-state imaging device.

For this reason, in the conventional image pickup apparatus, the amount of exposure is controlled using a shutter, a diaphragm, and the like so that the amount of light incident on the image pickup means from the subject enters the proper exposure area 4. FIG. 4 is a diagram illustrating an example of a relationship between a subject image and incident light. In the figure, 1 indicates overexposure, 2 indicates proper reproduction, 3 indicates underexposure (noise portion), and 4 indicates a proper exposure area. As shown in FIG. 4, when the light amount distribution of the subject image incident on the image pickup means is so wide that it does not fall within the proper exposure range, even if the exposure amount is controlled, as shown in FIG. In areas brighter than the area, the output signal is saturated, so that the gray scale cannot be reproduced and the area flies white. Also,
A portion darker than the proper exposure area, such as the portion 3 in FIG. 4, is noise and the like, so that the image is rough and the noise is conspicuous. In the example shown in the figure, only the portion 2 within the proper exposure area 4 can be properly reproduced and a normal light and dark gradation can be obtained.

In view of the above, various methods for enlarging a proper exposure area have been conventionally devised. First, as the most basic method, there is a method of increasing the saturation region of the solid-state imaging device. However, it is necessary to increase the charge holding capacity of the photoelectric conversion unit and the capacity of the transfer unit of the solid-state imaging device. Yes. However, it is difficult with the increase in the number of pixels and the speed of the solid-state imaging device. Further, a method of suppressing noise or the like can be considered, but this method is also difficult for the same reason.

Therefore, the present invention proposes a method of expanding the dynamic range using a plurality of images of different exposure amounts by employing the configuration shown in FIG. FIG. 5 shows an output signal corresponding to the light amount of a subject imaged at a certain exposure amount E.
FIG. 5A shows an output signal corresponding to the amount of light of a subject obtained by capturing the same subject at half the exposure amount E / 2, and FIG. 5B is a diagram obtained by doubling the output signal of FIG. FIG. 5 (c) shows the case where it is superimposed on 5 (a).

In FIG. 3B, since the image is taken with half the exposure amount, the slope up to saturation is half that of FIG. Therefore, if the output signal of (b) is doubled and superimposed on the graph of (a), it overlaps as shown in (c).

In (c), the portion is a proper region of the output signal imaged at the exposure amount E, and the portion is the exposure amount E / 2.
3 shows an appropriate area of the output signal imaged. Therefore, by newly creating an image signal using the appropriate areas of the two image signals of the exposure amount E and the exposure amount E / 2, it is possible to obtain an appropriate exposure area wider than the appropriate exposure area obtained by a single exposure. it can.

FIG. 6 is a diagram for explaining the S / N ratio when the exposure amount is changed. In the figure, (a) is the S / N ratio when the exposure is performed at the exposure amount E, and (b) is the exposure amount E /
The S / N ratio when exposure is performed with K, and (c) indicates the S / N ratio when one image signal is created by combining two image signals obtained with the exposure amount E and the exposure amount E / K. FIG.

In FIG. 6, as shown in FIGS. 6A and 6B, the S / N ratio obtained by a single exposure does not change even when the exposure amount is changed, but as shown in FIG. When an image signal is created by combining two image signals of the amount E and the amount of exposure E / K, the dynamic range can be expanded to a K-times S / N ratio. However, if K takes a value larger than the single S / N ratio, the two appropriate exposure areas do not overlap, and an appropriate image signal cannot be created. That is, the ratio of the two exposure differences needs to be smaller than the S / N ratio of the solid-state imaging device at the maximum. However, the noise of the solid-state imaging device is not constant and varies depending on the temperature and the position of the pixel.

Next, the output signal of the portion where the appropriate exposure areas of the exposure amount E and the exposure amount E / K overlap will be described. As the output signal V0 at the exposure amount E, a value V0 = S + e0 obtained by adding the signal S proportional to the light amount of the subject and the noise e due to the noise of the solid-state imaging device is output. In addition, the exposure amount E / K
Is a value V1 obtained by adding an S / K signal and a noise e to a signal S proportional to the amount of light of a subject having an exposure amount E.
= S / K + e1 is output.

When V1 is multiplied by K in order to create a new imaging signal using these two imaging signals, V1 '= S + K.multidot.
e1. Here, if e0 ≒ e1, V1 ′ becomes V0
, An output signal containing K times noise is obtained. Therefore, it is necessary to create an image signal using the output signal V0 at the exposure amount E as much as possible in a portion where the proper exposure areas overlap. However, it is necessary to use the output signal V1 'of the exposure E / K in a region that is brighter than the proper exposure region of the exposure E.

Therefore, it is necessary to set the value of K as small as possible, and as shown in FIG. 7, it is necessary to take a plurality of images at an exposure ratio (light amount ratio) K which at least satisfies the distribution of the light amount of the subject. This makes it possible to minimize noise in the high-luminance part of the subject.

Next, a method of imaging with different exposure amounts will be described. As a method of changing the exposure amount of the image pickup device, there is a method of changing the exposure amount by using a mechanical shutter or a diaphragm in the photographing optical system and photographing a plurality of times by changing the exposure amount. The depth of field changes, so that a plurality of images are different, and an in-focus image and a blurred image are combined even in the same subject portion. As a result, an unsightly image is obtained depending on the subject.

Further, in the method of changing the exposure time using a mechanical shutter of a photographing optical system or an electronic shutter of a solid-state image pickup device, a different image is synthesized in a photographing in which a subject is moving, and cannot be synthesized normally. Also,
When the exposure is performed at a high shutter speed, the ratio of the exposure amount cannot be finely controlled.

Therefore, in the present invention, as described above, a method of changing the amount of light incident on the image pickup means 3A, 3B in FIG. 1 is employed. The light amount ratio varying means 2 forms two object images having different light amount ratios based on the light amount ratio setting signal from the incident light by the optical means 1. A method of forming subject images having different light intensity ratios will be described with reference to FIGS. 8A and 8B show a reflection plate, and FIG. 8A shows a 50% reflection plate (a portion that reflects light and a portion that transmits light are halved),
(B) shows a 33% reflector. FIG. 9 shows a case where two reflectors are stacked. FIG. 10 shows that the reflectance is 1
The superposition forms of the reflection plates of 00%, 75% and 50% are shown, respectively.

As shown in FIG. 9, two reflectors 2a each having a reflectance of 50% as shown in FIG. 8 (a) are superimposed, and one of the reflectors 2a is driven by a driving mechanism (not shown) based on a light amount ratio setting signal. By shifting 2a, as shown in FIG.
Change up to 50%. Also, as shown in FIG. 8 (b), two reflectors each having a reflection portion in a portion having an area of 33% are overlapped, and by shifting one of the reflection plates, the total reflectance is 33% to 66%. %. These are methods for mechanically changing the reflectance.

Next, a method of electrically changing the reflectance will be described with reference to FIG. FIG. 11 is a diagram showing a configuration of the light amount ratio varying means 2 when the reflectance is electrically changed. As shown in FIG. 11, the light quantity ratio varying means 2 includes an anisotropic reflection plate 2c having a different reflectance depending on the polarization direction of the light, and a liquid crystal plate 2b for changing the polarization of the light based on the light quantity ratio setting signal. .

In the light amount ratio varying means 2 having the above-described configuration, a voltage is applied to the liquid crystal plate 2b based on the light amount ratio setting signal to polarize the light and control the ratio of the P wave to the S wave. The light is reflected by the anisotropic reflection plate 2c based on the ratio between the P wave and the S wave, and the ratio between the outgoing light 1 and the outgoing light 2 changes.

As described above, various configurations of the light quantity ratio varying means are conceivable, but it is sufficient to use the light quantity ratio varying means most suitable for the target imaging apparatus. When the light amount ratio variable means sets the light amount ratio based on the light amount ratio setting signal, the light amount ratio detecting means detects the ratio of the signals of the appropriately exposed portions of the imaging outputs of the plurality of object images, and determines the light amount ratio of the object image. Can be detected, the light amount ratio setting signal may be set based on the imaging output. That is, even if the light amount ratio varying means does not change linearly with respect to the light amount ratio setting signal, an appropriate light amount ratio can be set by applying feedback based on the image pickup output.

In FIG. 1, the light quantity ratio varying means is arranged behind the optical means. However, the light quantity ratio varying means may be arranged on the front face of the optical means. FIG. 12 shows a configuration in which the light quantity ratio varying means 2 is disposed on the front surface of the optical means 1A, 1B.

As described above, by using the light amount ratio varying means, it is possible to obtain a subject image which differs only in the light amount, and further, by selecting the light amount ratio matching the light amount distribution of the object as described above, the noise is reduced. A composite image can be obtained.

Next, a method for reducing noise accompanying the expansion of the dynamic range will be described. When two image sensors are used, if the image signal of one image sensor is V0, the image signal of the other image sensor is V1, the true signal without noise is S, and the noise is e0, e1, Signal V
0 and V1 are V0 = S + e0 and V1 = S + e1.

Here, it is assumed that the values of e0 and e1 are not random noise that changes with time. The difference ΔV between the two image signals is ΔV = V0−V1 = e0−e1, and the noise difference is obtained. Next, the exposure amount of the imaging element on the V1 side is set to 1
/ K ', and the product of the output multiplied by K' is V1 ', then V1' = S + K'.e1 ', and ΔV = V0-V
1 ′ = e0−K ′ · e1 ′. FIG. 13 shows a graph of ΔV with respect to the K value of the measured value.

If the light amount ratio is set to K based on the light amount distribution of the subject, the outputs V0 and V1 of the same pixel of the two image pickup devices are detected at a plurality of light amount ratios, and the difference ΔV from the difference ΔV is used. ΔV value can be estimated. That is, the estimated ΔV ≒ e0
−K · E1. From V1 = S + K · E1, V
When ΔV is added to 1, V1 + ΔV ≒ S + e0.

That is, by calculating ΔV at a plurality of light quantity ratios and estimating ΔV at the target K value, V1 = S + K
E1 becomes V1 + ΔV ≒ S + e0, and the noise on the output signal does not increase even if K is increased. That is, it is possible to expand an ideal dynamic range in which the amount of noise does not change even if the dynamic range is expanded. However, in order to always perform such operations over all the pixels of the image sensor, it is necessary to adopt a configuration in which the light amount ratio can be arbitrarily changed.

Also, in this example, it is assumed that the amount of noise is not constant with respect to the incident light, but it is simpler when the amount is constant, and ΔV = e0−e1 and ΔVK = e0−K · e1.
Thus, e1 = (ΔVK−ΔV) / (K−1), e0 = Δ
V + e1, and both noises themselves can be obtained. However, the assumption that the amount of noise is not constant with respect to incident light is a fixed pattern noise, so there is another method for removing it. Here, since it is assumed that the amount of noise is not constant with respect to the incident light, the description thereof is omitted.

When random noise that changes with time is included, ΔV is obtained a plurality of times at the same light amount ratio, and by taking an average, the random noise portion can be removed from ΔV. Can be removed as described above. However, it is better to adopt a method of removing random noise on a circuit. As this method, there is a method in which when an output signal is detected, an A / D converter is used to oversample at a higher frequency than the output signal, and the obtained value is filtered below the frequency of the output signal. Simply, an average value may be used. As described above, by changing the light amount ratio K and processing the output signal from the obtained information, it is possible to expand an ideal dynamic range in which the noise amount does not change.

[Digital Still Camera to which the Imaging Apparatus of the Present Invention is Applied] Next, a digital still camera to which the imaging apparatus of the present invention is applied will be described with reference to FIGS. FIG. 14 is a block diagram showing a configuration of a digital still camera to which the imaging device of the present invention is applied. FIG. 15 is a diagram showing the relationship between the amount of light of the subject and the output signal of the image sensor.

In FIG. 14, reference numeral 100 denotes a digital still camera. As shown in FIG. 14, the digital still camera 100 includes a photographing lens 101 for forming an image of light from a subject, an aperture and a shutter for determining an exposure amount. 102,
Based on the light amount ratio setting signal, the light amount ratio changing means 103 separates the input object image to form two object images having different light amount ratios.
Area-type CCD image sensors 104A, 104A, which respectively capture two subject images having different light intensity ratios and output image signals.
4B and a CDS · A / D converter after performing a double correlation between the image signals output from the imaging elements 104A and 104B, respectively.
AD circuits 105A and 105B are provided.

The digital camera 100 has a signal processing circuit 106 for processing the A / D-converted signal and a DRA for temporarily storing image data for signal processing.
M107, a CPU 108 for controlling the overall operation of the digital still camera, a CARDRAM 109 for storing a finally created image, and a focus, aperture / shutter 102 and light quantity ratio varying means based on control signals from the CPU 108. Driving circuit 110 for driving 103
And an LCD monitor 111 for monitoring a captured image and displaying a stored image. In the above configuration, the driving circuit 110 is
A light amount ratio setting signal is generated in accordance with the control signal output from the controller to drive the light amount ratio varying unit 103.

The photographing operation of the digital still camera having the above configuration will be described. First, in a pre-imaging stage (during monitoring), the CPU 108 controls the output of the light amount ratio varying unit 103 via the drive circuit 110 so that a large amount of light is distributed to one CCD image sensor 104A. CCD in this case
The output signal of the image sensor 104A is as shown in FIG.

Next, the CPU 108 sets the aperture / shutter 102 or the electronic shutter speed via the drive circuit 110 based on the image information obtained from the CCD image pickup device 104A so that the exposure becomes almost appropriate. Set conditions. Subsequently, the CPU 108 performs focusing control,
The photographing lens 101 is driven via the drive circuit 110 so as to focus on the subject. After the focus is adjusted, C
The PU 108 calculates the light amount distribution of the subject by using the method described above.
The light amount ratio K value is determined by detecting from the image information obtained from the CD imaging devices 104A and 104B. And CPU1
In step 08, as described above, image signals having different light amounts are obtained, and noise removal data ΔVK at the light amount ratio K value of each pixel of the CCD image sensor is obtained and stored in the DRAM 107.

The CPU 108 drives the light amount ratio varying means 103 to control the light amount ratio to the K value. In this case, the output signals of the CCD image sensors 104A and 104B are as shown in FIGS. The above is the preparation stage for shooting,
Next, the process proceeds to the photographing stage.

When the light amount ratio is K, the output signals of the two CCD image sensors are as shown in FIGS. 16A and 16B, respectively, and the proper exposure range is as shown in FIG. The combined signal obtained by combining the output signals of the two CCD image sensors is shown in FIG.
As shown in FIG. 7, proper exposure can be performed within the range of the proper signal shown in FIG. For example, in the case of a subject image having the characteristics shown in FIG. 18, one output V0 of the CCD image sensor has the characteristics (including the wavy line) shown in FIG. The output V1 of the other CCD image sensor has characteristics as shown in FIG.

The two subject images formed at the light intensity ratio K by the light intensity ratio varying means 103 are
A, 104B, are converted into image signals, and are input to the signal processing circuit 106 via the subsequent CDS / AD circuits 105A, 105B, where noise is removed.

FIG. 20 is a diagram showing a configuration of a noise removing circuit in the signal processing circuit 106. The noise removal circuit shown in FIG. 20 reduces noise by performing the process of K · V1 + ΔVK on each pixel. As shown in FIG. 20, the noise elimination circuit includes a multiplication circuit 201 that multiplies V1 by a K value, and a K · V output from the multiplication circuit 201.
Adder 302 that adds ΔVK read from the DRAM 107 to K.1 and K · V1 +
It includes a comparator 202 for combining and outputting ΔVK and a component of V0 equal to or lower than VTH.

FIG. 21 shows that the signal having the characteristic shown in FIG.
It is a figure which shows the characteristic of the signal after performing the process of * V1 + (DELTA) VK. The signal processing circuit 106 combines the non-saturated portion (output portion below VTH) in FIG. 19A with the image information in FIG. 21 to obtain a dynamic range suitable for the light quantity distribution of the subject. Image information with reduced noise can be obtained.

The signal processing circuit 106 performs a color interpolation process on the image information, performs a gamma process with an appropriate gamma curve, and performs a compression process to perform a CARD process.
To memorize. Since a wide dynamic range image is obtained, the image may be stored without performing gamma processing, and may be subjected to gamma processing according to an output device when output by another display device or output device.

In the above description, the mode for expanding the dynamic range has been described. To expand the dynamic range by dividing the light of a single image pickup optical system means that the light is allocated to both CCD image pickup devices 104A and 104B. Therefore, as shown in 1 in FIG.
Light incident on both CCD image sensors 104A and 104B is smaller than when the light is assigned to the D image sensor.
Therefore, in the case of a dark subject, all light is
The first mode for expanding the dynamic range by allocating to the CD image sensor and providing a mode that does not expand the dynamic range
The user may be able to select the imaging mode described above and the second imaging mode that does not expand the dynamic range by using an operation unit (not shown).

Also, when a very bright subject is imaged, a small amount of light is allocated to one area CCD image sensor to facilitate exposure control, and a second image capturing mode that does not expand the dynamic range is operated by an unillustrated operation. The user may be allowed to make a selection in the section.

In addition, the light amount distribution of the subject is detected based on the image information obtained from the CCD image pickup device during the above-mentioned monitoring. As a result of the detection, if it is not necessary to expand the dynamic range, the second dynamic range is not expanded. When it is necessary to expand the dynamic range in the imaging mode, the configuration may be such that the mode is automatically switched to the first imaging mode in which the dynamic range is expanded.

By the way, conventionally, a method of obtaining a high-resolution image by a so-called spatial pixel shift method in which an image is taken by performing a pixel shift is also known. 22 and 23 are diagrams for explaining the spatial pixel shift method. FIG. 22 shows an example of a filter arrangement of the CCD image sensor. For example, in FIG. 22, an image is taken at the position of the CCD image sensor, and then,
FIG. 23 shows composite pixels when an image is captured at the position where the CCD image sensor has moved. In the spatial pixel shift method, a method of mechanically shifting an image sensor or a method of optically shifting a subject image is adopted.

The spatial pixel shifting method is highly effective when the light amount ratio is 1: 1. However, the effect can be obtained even when the light amount ratio is different. If the method of expanding the dynamic range according to the present invention and the method of shifting the spatial pixels are performed together, it is possible to obtain a higher quality image.

Next, a method of expanding the dynamic range according to the color of the subject will be described. Basically, the light quantity distribution of the subject does not determine which color subject is bright. In the conventional method, the dynamic range of each color is not expanded by merely focusing on the brightness of the subject. Therefore, when expanding the dynamic range of the blue sky or the like, the dynamic ranges of R and G that do not exceed the range are also expanded.

Strictly speaking, it is better to expand the dynamic range of only the colors that are displaced in normal photographing. Therefore, in the present invention, a method of expanding the dynamic range corresponding to the subject color while maintaining the above-described configuration will be described.

Description will be made by taking an example of backlight photography with a blue sky as a background as shown in FIG. In this case, the range over the proper exposure is the blue sky in the background, and the output of the color image sensor in the blue sky is as shown in FIG. Then, B
The dynamic range of the pixel in which the color filter is arranged may be expanded. Here, by making the reflectance or transmittance of the light quantity ratio varying means 103 different depending on the wavelength as shown in FIGS. 26 (a) and 26 (b), the short wavelength is increased as shown in FIG. It can be divided into the output side and the side that outputs a large amount of long wavelength.

When the light amount ratio of the light amount ratio varying means 103 is changed and the light amount on the side that outputs a large amount of short wavelength is increased, the light amount ratio of blue
And the red light amount ratio Kr decreases. Conversely, when the light amount on the side that outputs a large amount of short wavelengths is reduced, the blue light amount ratio Kb decreases and the red light amount ratio Kr increases. Therefore, Kr, Kg, and Kb can be set for each color in this manner.

That is, when it is desired to further expand the dynamic range of B as shown in FIG. 25, the light amount on the side that outputs a large number of short wavelengths is increased so as to increase the light amount ratio Kb. FIG. 28 shows the output obtained when Kb>Kg> Kr. By expanding the dynamic range for each color using this output, R and G, which do not need to be expanded, can be expanded to a small extent to expand the dynamic range in a well-balanced manner.

Although the configuration in which the spectral distribution of the light amount ratio varying means 103 is changed has been described here, the configuration may be such that the spectral sensitivity of the CCD image sensor is changed as shown in FIG. In this case, it can be realized only by changing the transmittance of the color filter of the CCD image sensor, and R, G, and B can be set finely. Further, a configuration for changing the spectral distribution of the light amount ratio varying means and a configuration for changing the transmittance of the color filter of the CCD image sensor may be combined.

As described above, in the present embodiment, the light quantity ratio varying means 103 separates the same subject image and forms a plurality of subject images by changing the light quantity ratio. , B each image a plurality of formed subject images and output image signals, and the signal processing circuit 106 combines the image signals output from the CCD image sensors 104A and 104B to generate one image. Therefore, the dynamic range can be expanded even for a moving object such as a moving object, and the optimum dynamic range adapted to the subject image can be expanded.

In this embodiment, since the light amount ratio varying means 103 forms a plurality of subject images having different spectral transmission characteristics, the dynamic range can be expanded according to the color of the subject. It becomes possible.

Further, in the present embodiment, claim 3
According to the imaging device of the first aspect, the CCD imaging devices 104A and 104B have different spectral sensitivities in the imaging device according to the first aspect, so that the dynamic range adapted to the color of the subject can be easily expanded. Is possible, and
RGB can be set independently.

In the present embodiment, the light amount ratio K
Is determined on the basis of the image signals output from the CCD image sensors 104A and 104B.
Even if 03 does not change linearly, the light amount ratio can be set accurately, and the dynamic range can be expanded in accordance with the brightness of the subject.

In this embodiment, ΔV is calculated based on image signals of subject images of different light amounts output from the CCD image sensors 104A and 104B, and the noise of the signal processing circuit 106 is calculated based on the ΔV. Since the noise component of the image signal is removed by the removal circuit, an image signal from which the noise component has been removed can be obtained even at a desired dynamic range expansion rate, and there is no noise even if the dynamic range is expanded. Natural images can be obtained.

Further, in the present embodiment, the combination of the expansion of the dynamic range and the imaging by shifting the pixels is combined, so that a high resolution can be achieved. Further, in the present embodiment, since the first imaging mode for expanding the dynamic range and the second imaging mode for not expanding the dynamic range are provided, the dynamic range is expanded at low luminance. No ordinary shooting can be performed, and unnecessary long exposure can be prevented.
Furthermore, since the first imaging mode and the second imaging mode can be switched by operating the operation unit, the usability of the user is improved.

When it is not necessary to photograph a moving object as in a document camera, it is not necessary to form a plurality of subject images. Therefore, the exposure amount is changed by an electronic shutter or the like in the configuration shown in FIG. , The noise information ΔV may be calculated as described above using the plurality of pieces of image information, and noise accompanying the expansion of the dynamic range may be removed.

The present invention is not limited to the above-described embodiment, and can be carried out with appropriate modifications without departing from the spirit of the invention. For example, in the present embodiment, the dynamic range is appropriately expanded according to the subject, and noise in the expanded portion is reduced.
Although a special combination process after the enlargement is unnecessary, the combination process may be performed in combination with another combination process algorithm. Also, although a general imaging device has been described, the invention can be applied to a photomultiplier tube such as a night vision device. Further, in the present embodiment, the area sensor has been described, but the noise removing function of the present embodiment may be applied to other scanners or the like.

[0082]

As described above, according to the imaging apparatus of the first aspect, the light quantity ratio varying means separates the same subject image and changes the light quantity ratio to form a plurality of subject images. , A plurality of imaging units each captures a plurality of formed subject images and outputs image signals, and the image combining unit combines image signals output from the plurality of imaging units to generate one image. Therefore, the dynamic range can be expanded even for a moving object such as a moving object, and the optimum dynamic range adapted to the subject image can be expanded.

According to the image pickup apparatus of the present invention, in the image pickup apparatus of the present invention, the light quantity ratio varying means forms a plurality of subject images having different spectral transmission characteristics. In addition to the effect of the invention described in the item 1, it becomes possible to expand the dynamic range adapted to the color of the subject.

According to the image pickup apparatus of the present invention, each of the plurality of image pickup means has a different spectral sensitivity. In addition to the effects of the invention, the dynamic range adapted to the color of the subject can be easily expanded, and RGB can be set independently.

According to the imaging apparatus of the fourth aspect, in the imaging apparatus of the first aspect, the light quantity ratio of the light quantity ratio varying means is determined based on image signals output from the plurality of imaging means. Therefore, in addition to the effect of the first aspect of the present invention, even when the light amount ratio variable means does not change linearly, the light amount ratio can be set accurately, and the dynamic range adapted to the brightness of the subject can be expanded. Can be performed.

According to the imaging apparatus of the fifth aspect, in the imaging apparatus of the first aspect, the noise removing means is based on image signals of a plurality of different light amounts of the subject image formed by the light amount ratio varying means. Therefore, the noise component of the image signal output from the plurality of image pickup means is removed. Therefore, in addition to the effect of the invention described in claim 1, a natural image without noise can be obtained even when the dynamic range is expanded. It is possible to obtain.

According to the imaging apparatus of the sixth aspect, in the imaging apparatus of the first aspect, the relative position displacing means relatively displaces the plurality of imaging means and the subject image input by the optical input means. And a plurality of image pickup means output a plurality of image signals relatively displaced from each other, so that in addition to the effect of the invention according to claim 1, high resolution by pixel shift is achieved. Becomes possible.

According to the imaging apparatus of the present invention, the light quantity ratio varying means separates the same subject image and changes the light quantity ratio to form a plurality of subject images, and the plurality of imaging means are formed. Each of the plurality of subject images is imaged and an image signal is output, and the image synthesizing unit synthesizes the image signals respectively output from the plurality of imaging units to generate one image. Generating one image by synthesizing the image signals respectively output from the image capturing means, and in the second image capturing mode, generating the image based on the image signal output from one of the plurality of image capturing means. Since it is generated, by performing normal shooting without expanding the dynamic range at low luminance, unnecessary long-time exposure can be prevented.

According to the imaging apparatus of the present invention, the first imaging mode and the second imaging mode are switched based on the image signal. In addition to the effects of the image pickup apparatus according to claim 7, the dynamic range is expanded only when it is necessary to expand the dynamic range, and when it is not necessary, normal photographing is performed to perform unnecessary long-time exposure. This can be prevented.

Further, according to the imaging apparatus of the ninth aspect, the imaging means captures a subject image and outputs an image signal.
The noise elimination means is based on a plurality of image signals having different exposure amounts obtained by imaging with different exposure amounts by the imaging means.
Since the noise component of the image signal output from the imaging means is removed, it is possible to obtain a natural image without noise even when the dynamic range is expanded in an imaging device such as a document camera which does not need to image a moving object. Becomes possible.

[Brief description of the drawings]

FIG. 1 is a diagram showing a schematic configuration of an imaging device of the present invention.

FIG. 2 is a diagram illustrating a schematic configuration of a conventional imaging device that does not expand a dynamic range.

FIG. 3 is a diagram illustrating a relationship between an output signal of an imaging unit and incident light.

FIG. 4 is a diagram illustrating an example of a relationship between a subject image and incident light.

FIG. 5 is an explanatory diagram for explaining a principle of expanding a dynamic range.

FIG. 6 is a diagram for explaining an S / N ratio when an exposure amount is changed.

FIG. 7 is a diagram for explaining an exposure ratio (light amount ratio) K.

FIG. 8 is a diagram showing the reflectance of a reflector.

FIG. 9 is a diagram illustrating a case where two reflectors are stacked.

FIG. 10 is a diagram illustrating a superimposition mode of reflectors having reflectivities of 100%, 75%, and 50%.

FIG. 11 is a diagram for explaining a case where the reflectance is changed electrically.

FIG. 12 is a diagram showing a configuration in which a light amount ratio varying unit is arranged on the front surface of an optical unit.

FIG. 13 is a graph showing a difference ΔV between image signals.

FIG. 14 is a block diagram illustrating a configuration of a digital still camera to which the imaging device of the present invention has been applied.

FIG. 15 is a diagram illustrating a relationship between a subject light amount and an output signal of an imaging element.

FIG. 16 is a diagram showing output signals of a CCD image sensor.

FIG. 17 is a diagram showing a combined signal obtained by combining output signals of two CCD imaging devices.

FIG. 18 is a diagram illustrating an example of output characteristics of a subject image.

19 is a diagram showing an output of the CCD image sensor of the subject image of FIG.

FIG. 20 is a diagram illustrating a configuration of a noise removal circuit in the signal processing circuit.

21 is a graph showing the relationship between the signal having the characteristic shown in FIG. 18 and K · V1 + Δ
FIG. 9 is a diagram illustrating characteristics of a signal after performing a VK process.

FIG. 22 is a diagram illustrating an example of a filter array of a CCD image sensor.

FIG. 23 is a diagram illustrating synthesized pixels when pixel shift is performed by the CCD image pickup device in FIG. 22;

FIG. 24 is a diagram illustrating an example of a subject.

25 is a diagram illustrating an output of an image sensor for the subject in FIG.

FIG. 26 is a diagram showing the reflectance or transmittance of the light amount ratio varying unit.

FIG. 27 is a diagram for explaining an output of the light amount ratio varying unit of FIG. 26;

FIG. 28 is a diagram illustrating a case where a dynamic range is expanded for each color.

FIG. 29 is a diagram illustrating an example of the spectral sensitivity of a CCD image sensor.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 optical means 2 light quantity ratio variable means 3 imaging means 4 synthesizing means 100 digital still camera 101 photographing lens 102 aperture / shutter 103 light quantity ratio variable means 104 CCD image sensor 105 CDS / AD circuit 106 signal processing circuit 107 DRAM 108 CPU 109 CARDRAM 110 Drive circuit 111 LCD monitor

Claims (9)

[Claims] An optical means for forming an image of a subject;
A light amount ratio varying unit that separates the same subject image and changes a light amount ratio to form a plurality of subject images; and a plurality of subject images formed by the light amount ratio varying unit, each of which captures an image and outputs an image signal. An imaging apparatus, comprising: a plurality of imaging units; and an image combining unit that combines image signals output from the plurality of imaging units to generate one image. 2. An imaging apparatus according to claim 1, wherein said light amount ratio varying means forms a plurality of subject images having different spectral transmission characteristics. 3. The imaging apparatus according to claim 1, wherein said plurality of imaging units have different spectral sensitivities. 4. The image pickup apparatus according to claim 1, wherein the light amount ratio of said light amount ratio variable means is determined based on image signals output from said plurality of image pickup means. 5. The image pickup apparatus according to claim 1, further comprising an image output from the plurality of image pickup units based on image signals of a plurality of object images of different light amounts formed by the light amount ratio variable unit. An imaging apparatus comprising a noise removing unit for removing a noise component of a signal. 6. The image pickup apparatus according to claim 1, further comprising a relative position displacement unit that relatively displaces the plurality of image pickup units and a subject image input by the optical input unit. The imaging device outputs the plurality of relatively displaced image signals. 7. Optical means for forming an image of a subject,
A light amount ratio varying unit that separates the same subject image and changes a light amount ratio to form a plurality of subject images; and a plurality of subject images formed by the light amount ratio varying unit, each of which captures an image and outputs an image signal. A plurality of image pickup means; and an image synthesizing means for synthesizing image signals respectively output from the plurality of image pickup means to generate one image, and synthesizing the image signals respectively output from the plurality of image pickup means. And a second imaging mode for generating an image based on an image signal output from one of the plurality of imaging units. Imaging device. 8. The imaging apparatus according to claim 7, wherein the first imaging mode and the second imaging mode are switched based on the image signal. 9. An optical means for forming a subject image,
An imaging unit that captures a subject image and outputs an image signal; and an image signal output from the imaging unit based on a plurality of image signals having different exposure amounts obtained by capturing the image at different exposure amounts by the imaging unit. An image pickup apparatus comprising: a noise removing unit that removes a noise component.