JPH11331691A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the image pickup device that obtains an excellent image even when areas with large luminance differences are in existence in an object field. SOLUTION: An image pickup optical system 1 has a lens consisting of an aperture, an ND filter, a mechanical shutter and a diffraction grating. A control section 3 corrects a photographing signal obtained by setting the aperture, the ND filter, the mechanical shutter or a charge storage time of an image pickup element 2 so as to photograph a major object in an object field at a proper lightness by means of a photographing signal obtained by setting the aperture or the like so that the major object is photographed darker than the proper lightness and the corrected signal is stored in a memory 4.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, and more particularly, to an image pickup apparatus having a lens formed of a diffraction grating in an image pickup optical system.

[0002]

2. Description of the Related Art A lens having a light condensing function by a diffraction grating
(Hereinafter, referred to as a “diffractive lens”) has useful features not found in conventionally known refractive lenses. For example, the following features are known. A: Since a diffractive lens can be attached to the surface of a normal refractive lens, one lens can have both a diffractive action and a refracting action. : Since the amount corresponding to the dispersion characteristic of the refractive lens has the opposite value in the diffractive lens, the chromatic aberration can be effectively corrected by the diffractive lens.

[0003]

The chromatic aberration correction is positive / negative 2
It is generally performed by a combination of two refractive lenses. However, if a diffractive lens having the above features is attached to the surface of the refractive lens, it is possible to perform chromatic aberration correction with one lens. However, the diffractive lens has the above-mentioned useful features, but also has a property that the diffraction efficiency of the diffraction grating depends on the wavelength. For example, in the case of a blazed diffraction grating, the diffraction efficiency is 1 for a specific wavelength, so that the focal length is single, but a plurality of diffracted lights are generated at wavelengths other than the specific wavelength, so that a plurality of focal lengths are generated. Will be. Therefore, when a diffractive lens is used in the imaging optical system, a single image is formed for a specific wavelength, but a plurality of images are generated at other wavelengths because the diffraction efficiency depends on the wavelength.

Japanese Patent Application Laid-Open No. 9-238357 has proposed an example in which an image pickup apparatus including a diffraction lens in an image pickup optical system eliminates a plurality of images generated by a diffraction grating.
This imaging apparatus adopts a configuration in which image processing is performed on a captured image so that one image is obtained by eliminating the plurality of images. However, a problem caused by the generation of a plurality of images by the diffractive lens is when the luminance difference of the subject is very large. Since a normal image sensor has a small luminance difference (that is, a dynamic range of luminance) that can be picked up, the image pickup apparatus cannot cope with an area having a large luminance difference in an object scene.

The present invention has been made in view of such circumstances, and has as its object to provide an image pickup apparatus capable of obtaining a good image even when there is an area having a large luminance difference in an object scene. .

[0006]

In order to achieve the above object, an image pickup apparatus according to a first aspect of the present invention is an image pickup apparatus having a lens composed of a diffraction grating in an image pickup optical system, wherein a main object in a field is a subject. Aperture so that images can be taken with appropriate brightness, N
An imaging signal obtained by setting a charge accumulation time of a D filter, a mechanical shutter, or an image sensor is squeezed so that the main subject is imaged darker than the appropriate brightness, an ND filter, a mechanical shutter, or an imaging device. The correction is performed using an image pickup signal obtained by setting the charge accumulation time of the element.

According to a second aspect of the present invention, in the configuration of the first aspect, the diffraction efficiency of the diffraction grating is used for correcting the image signal.

According to a third aspect of the present invention, in the imaging apparatus according to the first aspect, the imaging signal is corrected by using imaging information of the diffraction grating based on zero-order or second-order diffracted light. .

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an imaging apparatus embodying the present invention will be described with reference to the drawings. In the block diagram of FIG.
1 shows a schematic configuration of an imaging device according to the present embodiment. This imaging apparatus includes an imaging optical system 1, an imaging element 2, a control unit 3, and a memory 4. The imaging optical system 1 includes a diffraction lens; a diaphragm, an ND filter, a mechanical shutter, and the like, and forms a subject image on a light receiving surface of an imaging element {for example, a CCD (Charge Coupled Device)} 2. The imaging element 2 photoelectrically converts the formed subject image and outputs a corresponding imaging signal. The control unit 3 sets the aperture, the ND filter, the mechanical shutter, or the charge accumulation time of the image sensor 2 to a predetermined state,
The image pickup signal output from the image pickup device 2 is stored in the memory 4.
To save.

FIG. 1 shows the diffraction efficiency of the diffraction lens used in the image pickup optical system 1. On the image plane where the first-order diffracted light is formed, a point image blurred by the zero-order diffracted light and the second-order diffracted light is
The point image of the first-order diffracted light is largely covered, resulting in a flare. The degree of blurring due to the zero-order diffracted light and the second-order diffracted light increases as the power due to the diffraction action of the diffractive lens increases. The intensity of the blur caused by the zero-order diffracted light and the second-order diffracted light is proportional to the magnitude of the diffraction efficiency shown in FIG. It becomes bigger.

FIG. 2 shows the spectral sensitivity of the image sensor 2 for each color of BGR. The output intensity of the imaging device 2 due to the 0th-order diffraction light and the 2nd-order diffraction light is obtained by multiplying the diffraction efficiency of the 0th-order diffraction light and the 2nd-order diffraction light (FIG. 1) by the spectral sensitivity of the imaging device 2 (FIG. 2). is there. Table 1 shows the ratio of the output intensity of the image sensor 2 due to the zero-order diffracted light and the second-order diffracted light to the output intensity of the image sensor 2 due to the first-order diffracted light. As can be seen from Table 1, the value of the G imaging device is the smallest. That is, the influence of the 0th-order diffracted light and the 2nd-order diffracted light is the smallest in the G imaging device. Further, if the sensitivity wavelength width of the G image pickup device is made narrower, the influence of the 0th-order diffracted light and the 2nd-order diffracted light becomes smaller.

[0012]

[Table 1]

Table 2 shows the performance of the image sensor 2 (a luminance difference (namely, a dynamic range of luminance) that can be imaged by the image sensor 2, a maximum value of an image signal of the image sensor 2, and a signal resolution of the image sensor 2 at that time). Show. Note that the luminance difference used here
(EV) is an EV value obtained by taking a log based on 2 with respect to the light intensity ratio, and does not consider the visibility.

[0014]

[Table 2]

FIG. 3 shows an example of an object scene. In the object scene, a light source 7 which is a bright part with a dark part as a background 5 is present together with the main subject 6. FIG. 4 shows an image obtained when the subject (FIG. 3) is imaged under an imaging condition (C1 in FIG. 6) at an exposure level at which the main subject 6 has an appropriate brightness, and a line x in FIG. FIG. 6 shows the luminance (BV) of each part at the time. As can be seen from the light source image 8 in FIG. 4, flare is generated in the background 5 near the light source 7 due to the 0th-order diffracted light and the 2nd-order diffracted light of the light source 7. In FIG. 6, B5 is the luminance of the background 5 (background luminance = BV1), and B6 is the luminance of the main subject 6 (main subject luminance = BV1).
BV3) and B7 are the luminance of the light source 7 (light source luminance = BV11),
B8 is the luminance of the background 5 near the light source 7 (flare background luminance = B
V6) and A7 are image luminance (flare luminance) of the light source 7 due to the 0th-order diffracted light and the 2nd-order diffracted light.

Under the imaging condition C1, the main subject 6 is imaged with appropriate brightness by setting the aperture, ND filter, mechanical shutter, or charge accumulation time of the imaging device 2, but the light source luminance B7 and the flare background luminance B8 is the image sensor 2
, The image output corresponding to the luminances B7 and B8 cannot be measured.
The image sensor output of the background 5 near the light source 7 corresponds to the flare background luminance B8 which is the sum of the background luminance B5 and the flare luminance A7. Therefore, the flare luminance A7 is calculated using the data shown in Table 1, and the flare luminance A7 equivalent is subtracted from the image sensor output of the background 5 near the light source 7 (corresponding to the flare background luminance B8) to obtain the correct background luminance. An image output of B5 can be obtained. However, to obtain the flare luminance A7, it is necessary to measure the correct light source luminance B7, and to determine the size of the flare, it is necessary to measure the correct size of the light source 7.

Therefore, in this embodiment, the background 5 and the light source 7
Even if the brightness difference (B7-B5) from the image pickup device 2 is larger than the imageable brightness difference of the image sensor 2, the imaging conditions are set to C1 and C until the light source brightness B7 is obtained in order to obtain the light source brightness B7 and the like.
Instead of C2 and C3, the configuration is such that image sensor outputs (image signals) of three images having different exposure levels are obtained (details will be described later with reference to FIG. 8). By changing the aperture, the ND filter, the mechanical shutter, or the setting of the charge accumulation time of the image sensor 2, the second exposure is performed under the imaging condition C2 whose exposure level is lower than the imaging condition C1 (that is, the image is dark). The background luminance B8 can be obtained. Further, when the third imaging is performed under the imaging condition C3 in which the exposure level is lower than the imaging condition C2 (that is, the image is darker) by changing the setting of the aperture or the like, the light source luminance B7 can be obtained. FIG. 5 shows an image corresponding to the imaging signal obtained at this time.
The light source image 9 (not including flare) in FIG. 5 corresponds to an image formed by the first-order diffracted light of the light source 7. Note that the main subject 6 in FIG. 5 is out of the imageable luminance range of the image capturing condition C3, and thus cannot be actually imaged together with the background 5.

Table 3 shows a luminance difference that cannot be resolved by the signal resolution of the image pickup device 2 (that is, a luminance difference that does not affect the image pickup device output) even when the flare luminance A7 is added to the original image output (background luminance B5). In a region where there is a luminance difference that does not affect the subject photographing, the flare luminance A7 is too small, and there is no problem even if the element output of the flare background luminance B8 is used as the image output of the background 5 near the light source 7. Therefore, in the present embodiment, when the brightness difference between the B image sensor output and the R image sensor output is 1 EV or more, and when the brightness difference between the G image sensor output is 4 EV or more, the 0th order diffracted light and the 2nd order A process for removing the influence of folding light (hereinafter, referred to as “flare removal”) is performed. Details will be described later with reference to FIGS.

[0019]

[Table 3]

Table 4 shows that the flare luminance A7 is large.
The signal resolution of the image sensor 2 indicates a luminance difference at which a correct image output cannot be resolved. In an area where there is a luminance difference equal to or larger than the unresolvable luminance difference in the object field, the flare luminance A7 is too large, and the flare background luminance B8 indicates that the background 5 near the light source 7 cannot be obtained.
Cannot obtain a correct image output. B image sensor and R
With the image sensor, a correct image output cannot be obtained at a luminance difference of 8 EV or more, and with a G image sensor, a correct image output cannot be obtained at a luminance difference of 11 EV or more. Therefore, in the present embodiment, a luminance difference of 8 EV
In the above, using the G image sensor output, the B image sensor output and the R image sensor output are complemented (hereinafter, referred to as “image complement”). Details will be described later with reference to FIGS.

[0021]

[Table 4]

Unaffected luminance difference (Table 3) and unresolvable luminance difference
The G image sensor output having the maximum value shown in Table 4 is appropriate as a signal used as a reference for image signal correction in order to represent field information most accurately. Table 5 shows the relationship between the luminance difference (B7-B8) for the light source 7 and the background 5 near the light source 7 and the G image sensor output difference. Note that the data in Table 5 was obtained using the data in Table 1.

[0023]

[Table 5]

Next, the control performed by the control unit 3 will be described. However, the luminance difference in the following description corresponds to the output difference of the G image sensor. First, the shooting of the object scene and the taking of the image into the memory 4 will be described with reference to FIG. In step # 10, the aperture and the N are set so that the main subject 6 is imaged with appropriate brightness.
An image is captured by setting the D filter, the mechanical shutter, or the charge accumulation time of the image sensor 2, and the obtained image signal (output of the image sensor 2) is stored in the memory 4. The exposure level at which the main subject 6 has an appropriate brightness is set so that the main subject 6 is reproduced at an intermediate level of the dynamic range of the luminance of the image sensor 2. The setting of the exposure level corresponds to the above-described imaging condition C1, and the obtained imaging signal corresponds to the data of the above-described image (FIG. 4).

Next, it is determined whether or not there is an overflow area in the image sensor 2. If there is no overflowing image sensor area, the process proceeds to the next sequence (FIG. 9),
Step # 3 if there is an overflowing image sensor area
Go to 0. In step # 30, imaging is performed under imaging conditions (the above-described imaging conditions C2, C3,...) In which the main subject 6 is imaged darker than appropriate brightness, and only the imaging signal in the area where the previous imaging has overflowed is stored. 4 is updated. The data input in steps # 20 and # 30 is repeated until there is no overflow area, and then the process proceeds to the next sequence (FIG. 9).

In order to image the main subject 6 darker than the proper brightness, the aperture should be narrowed down by an imageable luminance difference, the ND filter density should be increased, the mechanical shutter speed should be increased, and the like. The exposure level may be reduced by shortening the charge storage time of the image sensor 2 or the like. When the exposure level is changed so that the main subject 6 is imaged darker than the appropriate brightness, the imageable luminance differences of the imaging conditions C1, C2, C3,... May overlap each other.

Next, processing of a memory image will be described with reference to FIG. In step # 40, the output of the image sensor of the adjacent bright part (light source luminance B7) and dark part (flare background luminance B8) is input from the memory 4. Next, in step # 50, it is determined whether or not the luminance difference (B7-B8) is equal to or larger than 1 EV. Brightness difference
If (B7-B8) is less than 1 EV, the process is terminated because the flare is negligible, and if it is 1 EV or more, it is determined in step # 60 whether or not the luminance difference (B7-B8) is 4 EV or more. .

In the determination of step # 60, the luminance difference (B7-
If B8) is less than 4 EV, the process proceeds to step # 70, where 4
If it is equal to or larger than EV, the luminance difference (B7-B8) is determined in step # 90.
Is 7EV or more. In step # 70,
It is determined whether or not the input image signal is a G image sensor output.
If the input image signal is the output of the G image sensor, the processing is terminated because the influence of flare is small as described above. If the input image signal is the output of the B image sensor or the output of the R image sensor, the process is terminated after performing flare removal (FIG. 10).

At step # 90, the luminance difference (B7-
If (B8) is less than 7 EV, flare removal (FIG. 10) is performed in step # 100, and the process is terminated. If it is 7 EV or more, it is determined in step # 110 whether the luminance difference (B7-B8) is 8 EV or more. I do. In the determination of step # 110, the luminance difference
If (B7-B8) is less than 8 EV, the process proceeds to step # 120. If (B7-B8) is 8 EV or more, it is determined in step # 150 whether or not the image signal is the G image sensor output.

After performing flare removal (FIG. 10) in step # 120, it is determined in step # 130 whether or not the image pickup signal is the output of the G image pickup device. If the input image signal is a G image sensor output, the process ends. If the input image signal is a B image sensor output or an R image sensor output, image complementation (FIG. 11) is performed, and then the process ends. If it is determined in step # 150 that the image signal is the B image sensor output or the R image sensor output, the process proceeds to step # 170. If it is determined in step # 150 that the image signal is the output of the G image sensor, step # 16
0, and replaces the stored image signal (G image sensor output) with a predetermined value.
Proceed to. After performing flare removal (FIG. 10) in step # 170 and image complementation (FIG. 11) in step # 180, the process ends.

Tables 6 to 10 show the relationship between each of the B, G, and R image sensors 2 in the sequence shown in FIG. 9 and flare removal and image complementation.

[0032]

[Table 6]

[0033]

[Table 7]

[0034]

[Table 8]

[0035]

[Table 9]

[0036]

[Table 10]

Next, flare removal will be described with reference to FIG. First, in step # 200, an image sensor output corresponding to the brightness (flare background brightness) B8 of the dark portion adjacent to the bright portion (light source 7) is input from the memory 4. As described above, the image sensor output corresponding to the flare background luminance B8 corresponds to an image signal of luminance obtained by adding flare luminance A7 to background luminance B5 which is a dark part. In step # 210, an image sensor output corresponding to the light source luminance (brightness of the bright part) B 7 is input from the memory 4. In step # 220, a correction process {Equations (8) to (1)
0), a flare removal map or the like}, and stores the image output in the memory 4 as a background image output near the corresponding light source 7.

The correction processing in step # 220 will be described in detail below. First, it is assumed that the imaging optical system 1 is configured as follows in order to determine the size of the flare.
It is assumed that the imaging optical system 1 is simplified as a thin lens having one lens, and a diffraction lens is provided on the surface of the thin lens. Then, the relationship between the power of the refractive lens and the power of the diffractive lens constituting the imaging optical system 1 is expressed by the following equation (1). φ = φr + φDOE = 1 (1) where φ: power of the imaging optical system (1 for simplicity), φr: power by the refraction of the refraction lens, and φDOE: power by the refraction of the diffraction lens. .

The condition for correcting the chromatic aberration is represented by the following equation (2). (φr / νd) + (φDOE / νDOE) = 0 (2) where νd is the Abbe number of the glass type of the refractive lens, and νDOE is the value corresponding to the Abbe number of the diffractive lens.

The power φ0 of the imaging optical system 1 for the 0th-order diffracted light is represented by the following equation (3), and the power φ2 of the imaging optical system 1 for the second-order diffracted light is represented by the following equation (4). φ0 = r = 1 + {νDOE / (νd−νDOE)} ≒ 1 + (νDOE / νd) (3) φ2 = r = 1− {νDOE / (νd−νDOE)} ≒ 1− (νDOE / νd) (( Four)

When the focal length is f ± Δ with respect to the image plane of the imaging optical system 1 whose focal length is f and the F number is F, the magnitude of blur on the previous image plane is as follows: It is represented by equation (5). Since f = 1 / φ, f ± Δ is represented by the following equation (6). Accordingly, the magnitude of blur caused by the 0th-order diffracted light and the 2nd-order diffracted light of the light source 7 (= flare magnitude)
Is represented by the following equation (7). For example, f = 5, F = 4,
When νd = 60 and νDOE = -3.45, the size of the flare = 70 (μ
m), the image sensor 2 has a pixel size of □ 5 (μm),
In the case of one pixel size of □ 10 (μm) of RGB (one pixel = two pixels vertically and horizontally), the size of the flare is equivalent to seven pixels. [Blur size] = Δ / F (5) f ± Δ ≒ 1 / (1 ± νDOE / νd) ≒ 1−νDOE / νd, 1 + νDOE / νd (6) [flare size] = − νDOE / Νd / F… (7)

The calculation of the flare removal is performed by the following equations (8) to (10).
Is performed for all the pixel areas of the bright part (light source luminance B7). Tables 11 to 1 show the flare removal maps in equations (8) to (10).
3 is shown. This flare removal map is an arithmetic filter obtained using the data of Table 1 as diffraction efficiency information and the imaging information by the 0th and 2nd order diffracted light, and acts more strongly in a pixel region with higher luminance. (B image output of dark part) = (B image sensor output of dark part) − (B image sensor flare removal map) × (B image luminance of bright part) (8) (G image output of dark part) = (G of dark part) (Imaging element output) − (G imaging element flare removal map) × (bright image G image luminance) (9) (R image output of dark part) = (R imaging element output of dark part) − (R imaging element flare removal map) ) × (R image brightness in bright area)… (10)

[0043]

[Table 11]

[0044]

[Table 12]

[0045]

[Table 13]

Next, image complement will be described with reference to FIG. First, in step # 300, the G image output from the G image sensor is input from the memory 4, and in step # 310, the B and R image outputs from the B and R image sensors are input from the memory 4.
In step # 320, the output of the image sensor from which the flare has been removed
Using (B image output, G image output, R image output), the following equation (1)
The correction processing represented by 1) and (12) is performed on the B and R image outputs,
The B and R image outputs are stored in the memory 4. (B image output) = k × (B image output) + (1−k) × (G image output) (11) (R image output) = k × (R image output) + (1−k) × ( (G image output) (12) where k = 0.1 to 0.8.

According to the above-described control, the image pickup signal corresponding to the image shown in FIG. 4 is corrected using the image pickup signal corresponding to the image shown in FIG. It is possible to obtain an imaging signal of an image. Therefore, even if there is a region having a large luminance difference in the object scene, a good image as shown in FIG. 3 can be obtained.

[0048]

As described above, according to the imaging apparatus of the present invention, even if there is an area having a large luminance difference in the object scene,
Since the correction is performed using the imaging signal obtained by setting the aperture and the like so that the main subject is imaged darker than appropriate brightness, a good image can be obtained.

[Brief description of the drawings]

FIG. 1 is a graph showing the diffraction efficiency of a diffraction lens used in an imaging optical system.

FIG. 2 is a graph showing the spectral sensitivity of an image sensor.

FIG. 3 is a diagram illustrating an example of an object scene.

FIG. 4 is a view showing an image when the main subject is imaged at an exposure level at which an appropriate brightness is obtained by the main subject in FIG. 3;

FIG. 5 is a view showing an image when the object scene of FIG. 3 is imaged at a lower exposure level than in the case of FIG. 4;

FIG. 6 is a graph showing the luminance of each part at line x in FIG. 4;

FIG. 7 is a block diagram illustrating a schematic configuration of an imaging device according to an embodiment.

FIG. 8 is a flowchart showing a sequence at the time of field imaging and memory.

FIG. 9 is a flowchart illustrating a processing sequence of a memory image.

FIG. 10 is a flowchart showing a flare removal sequence.

FIG. 11 is a flowchart showing a sequence of image complementation.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Imaging optical system 2 ... Image sensor 3 ... Control part 4 ... Memory 5 ... Background 6 ... Main subject 7 ... Light source 8 ... Light source image (including flare) 9 ... Light source image (not including flare)

Claims (3)

[Claims] 1. An image pickup apparatus having a lens formed of a diffraction grating in an image pickup optical system, wherein an aperture, an ND filter, a mechanical shutter, or an image pickup element is provided so that a main subject in an object scene is picked up with appropriate brightness. The image pickup signal obtained by setting the charge accumulation time is set so that the main subject is imaged darker than the appropriate brightness by setting the charge accumulation time of an ND filter, a mechanical shutter, or an image sensor. An imaging apparatus, wherein correction is performed using an obtained imaging signal. 2. The imaging apparatus according to claim 1, wherein the diffraction efficiency of the diffraction grating is used for correcting the imaging signal. 3. The method according to claim 1, wherein the correction of the imaging signal is performed by using the zero of the diffraction grating.
2. The imaging apparatus according to claim 1, wherein imaging information based on second-order or second-order diffracted light is used.