JPH0817456B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup device having a substantially wide dynamic range.

[Conventional technology]

The image pickup device is widely used as a video camera unit such as a VTR with a built-in camera and a still video camera. A video camera that uses an image pickup tube or a solid-state image sensor has a narrower dynamic range than a conventional silver halide photography system, and therefore, underexposure and underexposure (a common name for parts with extremely high or low brightness levels). And so on. In the conventional video camera, in such a case, the aperture is opened by about 2 apertures manually or by operating the backlight compensation button to adjust the light amount.

However, even when such backlight correction is appropriately performed, overexposure occurs in the background even if the main subject has an appropriate exposure amount, and the screen has only a white background. In other words, the narrow dynamic range of the image pickup device cannot be solved only by adjusting the light amount so that the exposure amount of the main subject becomes appropriate as in the conventional device. For example line
In a conventional imaging device that converts a still image into an electric signal using a scanner or the like, a configuration is considered in which one screen is composed from a plurality of screens obtained from the same subject and having different exposure amounts.

[Problems to be solved by the invention]

However, this conventional image pickup apparatus targets a still image and cannot obtain a moving image with a wide dynamic range.

In view of such a problem, it is an object of the present invention to provide an imaging device capable of obtaining a moving image with a substantially wide dynamic range.

[Means for solving problems]

An image pickup apparatus according to the present invention relates to an image pickup unit for converting an image of an object into an electric signal and an image pickup unit for the same subject by changing the accumulation time of the image pickup unit at a predetermined cycle while the image pickup unit picks up the same subject. An imaging control unit that continuously captures a plurality of images with different exposure amounts and outputs the images periodically, and a storage unit that holds the immediately preceding image of the plurality of images with different exposure amounts output from the imaging unit. ,
For the image held in the storage means, the level of the signal in the screen is compared with a predetermined reference value to determine whether the level is appropriate or not, and the signal in the screen determined to be incorrect is displayed on another screen. It is characterized by having a control means for forming a composite screen in which the level of the signal of the entire screen is optimized in the predetermined cycle by replacing the signal at the corresponding position with the calculated result.

[Action]

By the control means, it is possible to form a composite screen in which each part of the screen has an appropriate signal level. This composite screen has no overexposure even in a backlit state, for example. This substantially widens the dynamic range.

〔Example〕

An embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram of the entire configuration when the present invention is applied to a camera-integrated VTR.

In FIG. 1, 100 is a camera unit, 200 is a processing unit, and 300.
Is a recording unit. In the camera unit 100, the amount of light rays incident from the optical system 101 is limited by the diaphragm 102, and the image pickup device 1
Image on 03. The image pickup device 103 is composed of an image pickup tube and a semiconductor image pickup device such as a MOS or CCD. The focus drive circuit 107, the diaphragm drive circuit 106, and the image pickup device drive circuit 105 are camera control circuits.
Under the control of 108, the optical system 101, the diaphragm 102, and the image sensor 103 are driven, respectively. The camera signal processing circuit 104 is a well-known circuit that performs γ correction and other processing similar to the signal processing circuit of a normal video camera.

The video signal output from the camera unit 100 is output by the processing unit 200.
It is converted to a digital signal by the A / D converter 201, and the arithmetic circuit 20
The pixel data conversion described later in 2 is performed, and the D / A converter 203
Is converted back into an analog signal and supplied to the recording unit 300. 204
Is an image memory for calculation in the arithmetic circuit 202, and 205 is its addressing circuit. Addressing circuit 20
Reference numeral 5 outputs a write / read address control signal for the image memory 204 in response to a timing signal from the control circuit 108 of the camera section 100.

In the recording unit 300, the analog signal from the D / A converter 203 is recorded in the VTR recorder 301 by a known method.

Next, the operation of the image sensor 103 will be described. 2 is a more detailed block diagram of the camera section 100, and FIG. 3 is an NTS.
Taking the C signal as an example, the timing of the camera unit 100
A chart is shown. Field index (FI) signals are odd (ODD) fields and even (E)
This is a signal to distinguish it from the VEN) field. V _BLK
The signal is a vertical blanking signal, the H (high) period corresponds to the effective screen, and the L (low) portion corresponds to the vertical blanking period. T _pulse is a signal for controlling the charge accumulation time of the image sensor 103, and is a pulse for reading pixel output to the vertical transfer CCD in the case of a CCD image sensor, for example. The iris gate signal is a signal that specifies whether to use a 1/1000 second accumulated signal or a 1/60 second accumulated signal as a video signal that serves as a reference for automatic exposure, which will be described later.

In the illustrated example, 1/1000 second accumulation is performed during the vertical blanking period, and the 1/1000 second accumulation signal is output during the next effective screen period. Then, the charge is accumulated for substantially 1/60 seconds in the effective screen period immediately after the 1/1000 second accumulation period, and the 1/60 second accumulation signal is output in the effective screen period of the next field. In this way, there are two types (1/1000 seconds and 1/60 seconds) for each field.
Seconds) light intensity signals are output alternately.

In FIG. 2, 20 is a camera signal processing circuit 104.
A known AE control circuit that receives a signal (for example, a video signal) from the controller and calculates a control signal for exposure control, 22 is a known AF control circuit that outputs a control signal for focus control, and 24 is a vertical It is a 1/2 divider circuit that _{divides the} blanking signal V _BLK by 2. 26 and 27 are sample and hold circuits, 28 is an inverter, and 29 and 30 are switches for selecting which of the output of the 1/2 frequency divider circuit 24 and its inverted signal from the inverter 28 determines the sampling timing. The outputs of the sample and hold circuits 26 and 27 are applied to the diaphragm drive circuit 106 and the focus drive circuit 107, respectively, and automatic exposure control and automatic focus adjustment are executed.

In the above embodiment, a combination of 1/1000 sec and 1/60 second, since the light quantity change of about 4 stages (2 ^4x), for example, a CCD
In the case of a camera using an image sensor, 1/60 in the EVEN field
When the exposure is adjusted to the main subject based on the accumulation time of 2 seconds, overexposure is likely to occur in the background in the EVEN field, whereas black shadow often occurs in the main subject in the ODD field where the four-step light amount is reduced. . It should be noted that this example assumes the case where the exposure is adjusted to the background side at the time of backlight correction, and of course, it may be set to a value other than 1/1000 seconds depending on the situation at the spot.

According to the present invention, such overexposure and / or underexposure in one field is positively utilized to improve the screen. In other words, the portion where whiteout or blackout occurs is replaced by the corresponding portion of another field (blackout or whiteout does not occur because the exposure is different), and the signals of both fields are combined to produce the final image. Signal. The basic idea will be described with reference to FIG. In FIG. 4, the main subject is schematically shown by a vertically long rectangle. In FIG. 4, a through (T) image means a direct output of the image sensor 103, and a memory (M) image or a memory output means an image memory 204.
The signal of the immediately preceding field once stored in. In the through image, the main subject at the time of backlighting is blacked out in each ODD field, and the background is overexposed in each EVEN field. Also, since the memory image is composed of signals delayed by one field period, overexposure and underexposure occur in a field different from that of the through image.

Therefore, if the through image and the memory image are properly combined, a good image without overexposure and underexposure can be obtained. That is, the signals of the through image and the memory image are compared with a predetermined threshold value for each field, and if the value is larger than the threshold value, the value is set to 1, and if it is smaller than the threshold value, 0 is determined for each pixel. FIG. 6 shows the threshold value and the brightness value of the pixel,
Indicates the relationship with the field. In FIG. 6A, the horizontal axis represents the brightness level and the vertical axis represents the appearance frequency of each brightness level in one screen. As shown in FIG. 6 (a), the threshold value Th1 is set so as to be able to determine blackout, and the threshold value Th2 is set so as to be able to determine whiteout. That is, it is determined that Th1 or less is blackout, and threshold Th2 or more is overexposure. FIG. 6B shows the relationship between each field and the threshold value. As described above, overexposure and underexposure alternate between the ODD field and the EVEN field, so the threshold value for the determination is also changed for each field.

In this way, it is possible to determine which pixel portion of which field is blackout or overexposure, and thus the determination result can be used to select a pixel signal with an appropriate exposure amount for a through image and a memory image. For example, the logical product of the determination A and the determination B is taken, and in the ODD field, the through image signal is selected for the pixel having the logical product 1, and the memory image is selected for the pixel having the logical seat 0. By selecting a signal and setting it in the opposite relationship in the EVEN field, a selection flag as shown in FIG. 4 is obtained. The picture at the bottom of FIG. 4 shows a composite image according to the selection flag. In this figure, the effect of the time axis shift on the image was confirmed on the assumption that the main subject moves at a constant velocity, but it can be seen that the moving image can be practically sufficient.

FIG. 5 shows the above threshold value in the arithmetic circuit 202 of the processing unit 200.
FIG. 3 is a detailed block diagram showing the configuration of a circuit part for comparing with Th1 and Th2 and forming a selection flag. The Th switching control signal is a signal in which “H” and “L” are inverted for each field like the FI signal, and is applied to the second threshold generation circuit 52 via the threshold generation circuit 53 and the inverter 51. It Threshold generator circuit 52,5
3 is the threshold value T of the relationship in FIG. 6 (b) according to the switching signal.
Generates h1 or Th2. The comparison circuits 54 and 55 compare the memory image and the through image with the threshold values from the threshold value generation circuits 52 and 53, respectively, and output the A signal and the B signal. The AND gate 56 calculates the logical product of the A signal and the B signal and outputs a selection flag signal. The switch 57 switches according to the selection flag signal and selects a signal of a memory image or a through image.

FIG. 7 shows a gradation characteristic diagram. The solid line in (a) is the characteristic diagram of a normal video camera, the input and output are linear up to 100%, and the slope called the KNEE characteristic for inputs higher than that (100 to 400%). Has a loose relationship. If this change point is P1, then this change point moves to the position of P2 during high-speed shuttering. However, it is assumed that P1 is 1/60 second and P2 is 1/250 second of the change of the exposure amount in two steps. As described above, in the case of the difference of 4 steps, the relationships (1) and (5) of FIG. 7D are obtained. By the way, (1) of Fig. 7 (d) is
1/60 seconds, (2) 1/125 seconds, (3) 1/250 seconds, (4)
1/500 seconds, (5) is a characteristic diagram when 1/1000 seconds. A characteristic having a desired curve is synthesized from two characteristics having different inclinations. 7 (b) and 7 (c) are examples of the synthesis.

A method of synthesizing gradation characteristics will be specifically described. As an example, control is performed so that either the change point P1 (for low shutter speed) or P2 (for high shutter speed) from the linear part to the KNEE characteristic part becomes the 100% point. To take. As shown in FIG. 6, a threshold value is set to determine whiteout and blackout on the screen, but gradation characteristics change depending on the setting of the threshold value. The state of the change is shown in FIG. 7 (b). The characteristics (1) to (3) have threshold Th2 for overexposure determination.
When the value is changed from a low value to a high value in sequence, the position where the high-speed shutter side is selected is changed to the high-luminance side by the pixel switching by the switch 57.

In FIG. 5, one pixel signal is selected by the switch 57, but data of two corresponding pixels may be arithmetically processed to obtain a target signal. For example, FIG. 7 (c) shows characteristic changes when various calculation methods are selected based on the characteristic (2) of FIG. 7 (b). The characteristic (2) of FIG. 7 (c) is the same as the characteristic (2) of FIG. 7 (b). Seventh
The characteristic (1) in FIG. 10C shows a case where the average value processing is performed using the data of the pixel for which the “out-of-white highlight” determination has been made and the data of the corresponding pixel on another screen. Characteristic (3) shows the case where the subtraction processing is performed. For example, letting D ₁ and D ₂ be the data of the corresponding two pixels that are determined to be “whiteout” and the processing result to be D, D = (D ₁ + D ₂ ) / 2 in the average value processing, In the subtraction process, D = D ₁ −k (D ₁ −D ₂ ). However,
k is changed according to the set threshold. In the characteristic (3) of FIG. 7 (c), k is about 1.88. In addition to such average value processing and subtraction processing, so-called offset processing such as constant addition and constant subtraction may be used, or these may be used together.

Next, another detailed example of the control circuit 108 is shown in FIG. The master clock generator 40 generates a master clock for the control circuit 108 according to an external reference signal. The 1/1000 shutter clock generator 41 generates a high speed clock according to its master clock, and the 1/60 shutter clock generator 42 generates a low speed clock according to its master clock. The switch 45 switches for each field, and alternately applies the outputs of the clock generators 41 and 42 to the drive circuit 105. AE control signal generator 43
Generates an AE control signal for aperture control based on the video signal from the camera signal processing circuit 104. The control signal holding circuit 44 holds the control signal for one field. The switch 46 switches for each field, and the AE control signal generator
The output of 43 and the holding signal from the control signal holding circuit 44 are alternately applied to the aperture control circuit 106. The switching signal generator 47 is
It controls the switching of the switches 45 and 46. The switches 45 and 46 switch in synchronization.

In this embodiment, a clock generator is provided for each of low speed and high speed, and the clock is switched by the output signal of the switching signal generator that generates a signal for each field, so that the circuit configuration and operation are simplified. The effect is that it is especially suitable for movies.

〔The invention's effect〕

As can be easily understood from the above description, according to the present invention, the dynamic range of the image pickup device can be substantially widened. Specifically, it is possible to take an image such that not only the main subject but also the background is properly exposed even when the subject is backlit. Furthermore, when the main subject is properly exposed, the degree of background reflection (degree of overexposure, etc.) can be adjusted as desired, and more natural depiction and intentional drawing are possible. Of course, conversely, the background can be properly exposed to adjust the degree of blackout of the main subject. The range of adjusting the degree of reflection can obtain a myriad of wide-range characteristics by combining the shutter speed in addition to the pixel data calculation method.

[Brief description of drawings]

FIG. 1 is a block diagram of a camera-integrated VTR using an embodiment of the present invention, and FIG. 2 is a control circuit of the camera section shown in FIG.
108 is a specific block diagram of the configuration, FIG. 3 is an operation timing chart of the image pickup device, FIG. 4 is a conceptual diagram of image processing according to the present invention, and FIG. 5 is a specific configuration block diagram of the arithmetic circuit 202 of FIG. FIG. 6 is a diagram for explaining a method of determining a threshold value for determining overexposed areas and underexposed areas, FIG. 7 is a gradation characteristic diagram, and FIG. 8 is another block diagram of the control circuit 108. 100 …… Camera section, 200 …… Processing section, 300 …… Recording section

Front page continued (72) Inventor Toshiyuki Masui, 770 Shimonoge, Takatsu-ku, Kawasaki-shi, Kanagawa Canon Inc., Tamagawa Plant (72) Takashi Kobayashi, 770, Shimonoge, Takatsu-ku, Kawasaki-shi, Kanagawa Canon Inc.

Claims (1)

[Claims] 1. An image pickup means for converting a subject image into an electric signal, and an exposure amount of the same subject is changed by changing a storage time of the image pickup means at a predetermined cycle while picking up the same subject by the image pickup means. An imaging control unit that continuously captures a plurality of different images and outputs the images periodically, a storage unit that holds the immediately preceding image of the plurality of images with different exposure amounts output from the imaging unit, and the storage unit. By comparing the level of the signal in the screen of the image held by the means with a predetermined reference value, it is determined whether the level is appropriate or not, and the signal in the screen determined to be incorrect is associated with another screen. And a control means for forming a composite screen in which the signal level of the entire screen is optimized in the predetermined cycle by replacing the position signal with the calculated result. Imaging device.