JP2012235534A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform imaging sensitivity control without changing a dynamic range.SOLUTION: An imaging apparatus includes: an imaging element which can individually output, in the form of analog signals, imaging signals of different photoelectric conversion characteristics being a linear characteristic and a logarithm characteristic; A/D conversion means for converting the analog signals into digital signals; noise cancel means for linear characteristic, to which the digital signal corresponding to the imaging signal of the linear characteristic is inputted among the digital signals and which removes fixed pattern noise peculiar to imaging in the linear characteristic; noise cancel means for logarithm characteristic, to which the digital signal corresponding to the imaging signal of the logarithm characteristic is inputted among the digital signals and which removes the fixed pattern noise peculiar to the imaging in the logarithm characteristic; and image generating means synthesizing the imaging signal from which the noise is removed by the noise cancel means for linear characteristic and the imaging signal from which the noise is removed by the noise cancel means for logarithm characteristic and generating an image.

Description

  The present invention relates to an imaging apparatus using an imaging sensor having photoelectric conversion characteristics different from linear characteristics and logarithmic characteristics, and more particularly to an imaging apparatus capable of performing sensitivity control processing on an imaging signal of each characteristic by the imaging sensor. It is.

  2. Description of the Related Art Conventionally, in an imaging apparatus such as a digital camera, in response to a demand for higher image quality, one of the major themes is to expand the luminance range of a subject that can be handled by an imaging sensor, that is, the dynamic range (DR). . Regarding the enlargement of the DR, the linear characteristic in which the electrical signal is linearly converted with respect to the incident light quantity and output is different from the logarithmic characteristic in which the electrical signal is converted logarithmically with respect to the incident light quantity. An imaging sensor having photoelectric conversion characteristics is known (this imaging sensor is referred to as a “linear log sensor”). This imaging sensor outputs a linear characteristic imaging signal on the low luminance side and a logarithmic imaging signal on the high luminance side. By the way, generally, there is sensitivity correction (ISO sensitivity correction) as one of various correction processes for a photographed image with a silver halide camera. When this sensitivity correction is performed in a digital camera, for example, when increasing the sensitivity of imaging (increasing sensitivity), multiplication processing is performed on linear characteristic imaging signals, and addition processing is performed on logarithmic imaging signals ( For example, see Patent Document 1).

JP 2006-352804 A

  However, the above conventional technique has a problem that the dynamic range changes when the sensitivity is increased in an analog signal state.

  The present invention has been made in view of the above problems, and an object thereof is to provide an imaging apparatus capable of performing imaging sensitivity control (sensitivity correction) without changing the dynamic range.

  An image pickup apparatus according to the present invention has photoelectric conversion characteristics having different linear characteristics and logarithmic characteristics, and an image pickup element configured to individually output an image pickup signal of each photoelectric conversion characteristic as an analog signal, and the image pickup element A / D conversion means for converting an analog signal output from the digital signal into a digital signal, and among the digital signals converted by the A / D conversion means, a digital signal corresponding to the imaging signal having the linear characteristic is input and input Of the digital signal converted by the A / D converter, the noise canceling means for linear characteristic that removes fixed pattern noise peculiar to imaging with the linear characteristic in the digital signal that has been converted to the logarithmic characteristic imaging signal A logarithmic characteristic that eliminates fixed pattern noise specific to imaging with the logarithmic characteristic of the input digital signal when the corresponding digital signal is input An imaging signal from which fixed pattern noise peculiar to imaging with the linear characteristic is removed by the noise canceling unit and the linear characteristic noise canceling unit, and a fixed characteristic peculiar to the imaging with the logarithmic characteristic by the logarithmic characteristic noise canceling unit. Image generating means for generating an image by synthesizing the imaging signal from which the pattern noise has been removed is provided.

  Further, in the above configuration, an amplifying unit for amplifying the imaging signal of each photoelectric conversion characteristic, and an amplification factor control for individually controlling the amplification factor for the imaging signal of each photoelectric conversion characteristic in the amplifying unit for each photoelectric conversion characteristic And a means.

  According to the above configuration, the imaging device has photoelectric conversion characteristics having different linear characteristics and logarithmic characteristics, and an imaging element configured to be able to individually output an imaging signal of each photoelectric conversion characteristic as an analog signal; and Amplifying means for amplifying the imaging signal of photoelectric conversion characteristics, amplification rate control means for individually controlling the amplification factor for each imaging signal of photoelectric conversion characteristics in the amplifying means for each photoelectric conversion characteristic, and each photoelectric signal amplified by the amplifying means An image generation means for generating an image by synthesizing imaging signals having conversion characteristics is provided, and amplification factor control is performed for each photoelectric conversion characteristic using the amplification factor calculated by the amplification factor control means. Therefore, it is possible to individually (with different values) set the amplification factor of the image pickup signal of each photoelectric conversion characteristic, that is, to perform amplification processing separately (separately) for each of the image pickup signals having different photoelectric conversion characteristics. Therefore, for example, based on an evaluation value obtained only from an imaging signal having a linear characteristic with higher sensitivity than the logarithmic characteristic, the amplification factor of the imaging signal having the linear characteristic is controlled, that is, only the imaging signal having the linear characteristic is amplified. The amplification factor control can be performed, and the amplified photoelectric conversion characteristic (linear characteristic) imaging signal and the other non-amplified logarithmic characteristic imaging signal are combined by the image generation means to generate an image. By doing so, it is possible to perform imaging sensitivity control (sensitivity correction) without changing the dynamic range.

  In the above configuration, the amplification factor control means calculates an amplification factor for the imaging signal having the linear characteristic based on the imaging signal having the linear characteristic, and uses the calculated amplification factor to generate the linearity after the image generation. You may make it control the amplification factor of only the imaging signal of a characteristic.

  According to this, the amplification factor for the linear characteristic imaging signal is calculated based on the linear characteristic imaging signal by the amplification factor control means, and the lowest luminance image is generated after the image is generated using the calculated amplification factor. Since the amplification factor of only the imaging signal representing the image is controlled, the imaging signal representing the image with the lowest luminance after this image generation, i.e., the linear characteristic imaging signal and the other non-amplified photoelectric conversion characteristics, i.e., logarithmic imaging By synthesizing the signal and generating an image, it is possible to control the imaging sensitivity without changing the dynamic range.

  Further, in the above configuration, the image generation unit includes the first imaging signal that is the linear characteristic imaging signal in which the amplification factor is controlled, and the amplification factor that the logarithmic imaging signal is calculated by the amplification factor control unit. The first imaging signal having a predetermined signal level or lower and the second imaging signal having a level higher than the signal level are selected from the second imaging signal obtained by data conversion according to the rate, and the selected first and first An image is generated by combining two imaging signals.

  According to this, the first image pickup signal having the gain-controlled characteristics and the image pickup signal having the characteristics other than the gain-controlled characteristics are subjected to data conversion according to the calculated gain. From the two image pickup signals, a first image pickup signal having a predetermined signal level or less and a second image pickup signal having a level higher than the signal level are selected, and the selected first and second image pickup signals are combined to generate an image. Therefore, it is possible to easily generate an image by synthesizing the imaging signals of the respective characteristics without changing the dynamic range, for example, by setting the predetermined signal level as the inflection point level of each photoelectric conversion characteristic. .

  According to the imaging apparatus of the present invention, it is possible to perform imaging sensitivity control (sensitivity correction) without changing the dynamic range (to increase the sensitivity of the imaging signal while maintaining the dynamic range).

1 is a schematic block configuration diagram mainly relating to imaging processing of a digital camera which is an example of an imaging apparatus according to a first embodiment. FIG. It is a graph which shows an example of the photoelectric conversion characteristic of the imaging sensor used for the said digital camera. It is a figure which shows an example of the circuit structure of the said imaging sensor. It is a functional block diagram which shows an example of the imaging sensor and image processing part of the said digital camera. It is a functional block diagram of the above-mentioned image sensor when different photoelectric conversion characteristics are a linear characteristic and a logarithmic characteristic. It is a schematic diagram for demonstrating the timing of exposure and imaging signal output in the said linear characteristic and logarithmic characteristic. It is a functional block diagram of the said image processing part in case a different photoelectric conversion characteristic is a linear characteristic and a logarithmic characteristic. It is a graph for demonstrating selection operation | movement with respect to the imaging signal of each characteristic in the image processing part (selection part) shown in the said FIG. It is a graph for demonstrating the position of the inflection point level at the time of exposure by a linear characteristic and a logarithmic characteristic. It is a functional block diagram which shows an example of the imaging sensor and image processing part of the digital camera in 2nd Embodiment. FIG. 11 is a functional block diagram of the imaging sensor shown in FIG. 10 when different photoelectric conversion characteristics are a linear characteristic and a logarithmic characteristic. It is a schematic diagram for demonstrating the timing of exposure and imaging signal output in the said linear characteristic and logarithmic characteristic in 2nd Embodiment. FIG. 11 is a functional block diagram of the image processing unit shown in FIG. 10 when different photoelectric conversion characteristics are a linear characteristic and a logarithmic characteristic. Each of the characteristics in the image processing unit (selection unit) when the photoelectric conversion characteristics are composed of two different linear characteristics (first linear characteristics / second linear characteristics), which is a modification of the first and second embodiments. It is a graph for demonstrating selection operation | movement with respect to the imaging signal of. A modification of the first and second embodiments, and a selection operation for an imaging signal of each characteristic in the image processing unit (selection unit) when the photoelectric conversion characteristic is composed of three or more different linear characteristics will be described. FIG.

(Embodiment 1)
FIG. 1 is a schematic block diagram illustrating mainly an imaging process of a digital camera which is an example of an imaging apparatus according to the first embodiment. The digital camera 1 includes a lens unit 2, an image sensor 3, an A / D conversion unit 4, an image processing unit 5, and a control unit 6. The lens unit 2 constitutes an optical lens system for guiding subject light to the imaging sensor 3. The lens unit 2 is provided with a diaphragm and a shutter (both not shown) for adjusting the amount of transmitted light, and the drive of the lens unit 2 is controlled by the control unit 6.

  The imaging sensor 3 performs photoelectric conversion to an image signal of each of R, G, and B components according to the amount of subject light guided by the lens unit 2 and outputs the image signal. In the present embodiment, as the image sensor 3, as shown in FIG. 2, when the sensor surface illuminance (sensor incident luminance) is low (in the dark), an electric signal (an output electric signal generated by photoelectric conversion) is linearly obtained. A photoelectric conversion characteristic consisting of a linear characteristic region that is converted and output, and a logarithmic characteristic region that is output by logarithmically converting (logarithmically compressing) an electrical signal when the sensor surface illuminance is high (during light), in other words For example, a logarithmic conversion type solid-state imaging device having a photoelectric conversion characteristic having a linear characteristic on the low luminance side and a logarithmic characteristic on the high luminance side is used. The switching point (inflection point; Vth) between the linear characteristic and logarithmic characteristic of the photoelectric conversion characteristic can be arbitrarily controlled by a predetermined control signal for each pixel circuit of the image sensor 3.

The imaging sensor 3 includes, for example, a plurality of pixels including a photoelectric conversion element such as a photodiode, a P-type (or N-type) MOSFET (Metal Oxide Semiconductor Field Effect Transistor), or an FD (Floating Diffusion) in a pixel circuit. It is a MOS type solid-state imaging device arranged in a matrix, and is a so-called XY address type CMOS image sensor. In this configuration, the image sensor 3 changes, for example, the transfer gate potential of the transfer transistor in the MOSFET, the reset gate potential of the reset transistor, and the like, by changing the pixel signal having linear characteristics (linear characteristic output) from the FD, and the logarithmic characteristics A pixel signal (logarithmic characteristic output) is output. However, the present invention is not limited to this, and a VMIS image sensor, a CCD image sensor, or the like may be used.

  Specifically, the imaging sensor 3 has a circuit configuration shown in FIG. 3, for example. That is, the imaging sensor 3 includes a pixel unit 31, a vertical scanning circuit 32, a first horizontal scanning circuit 33, a second horizontal scanning circuit 34, a first readout circuit 35 and a second readout circuit 36. The pixel unit 31 includes a plurality of pixels 311 (pixels), and photoelectrically converts the image signals to the linear characteristics and logarithmic characteristics according to the amount of light of the subject light image and outputs the image signals. The pixel unit 31 includes a load circuit 312 having a load transistor Qa that functions as a so-called electronic load by applying a load voltage (signal VD) to the gate.

  The vertical scanning circuit 32 is a vertical shift register that performs vertical scanning on each pixel 311 (all pixel rows) of the pixel unit 31 and sequentially scans each row selection signal line 313. Each of the first horizontal scanning circuit 33 and the second horizontal scanning circuit 34 is a horizontal shift register that performs horizontal scanning (column selection operation) on each pixel 311, and each vertical signal line 314 (column selection signal line; output signal line). ) Are sequentially scanned. However, the first horizontal scanning circuit 33 performs horizontal scanning on the linear characteristic output from each pixel 311, while the second horizontal scanning circuit 34 performs horizontal scanning on the logarithmic characteristic output (logarithmic compression characteristic output) from each pixel 311. Do.

The first readout circuit 35 outputs an image signal based on a linear characteristic output derived from each pixel 311 of the pixel row scanned by the vertical scanning circuit 32 to the vertical signal line 314 for each pixel according to the horizontal scanning by the first horizontal scanning circuit 33. It is a circuit for reading sequentially. Similarly, the second readout circuit 36 is a circuit for sequentially reading out the image signal by the logarithmic characteristic output derived to the vertical signal line 314 for each pixel according to the horizontal scanning by the second horizontal scanning circuit 34. As described above, in the present embodiment, a dedicated (individual) readout circuit is provided for each of the different characteristics in the photoelectric conversion characteristics of each pixel 311, that is, for each of the linear characteristic output and the logarithmic characteristic output. Each of the first and second readout circuits 35 and 36 functions as a so-called sample hold circuit that samples and holds an image signal and a noise signal by the linear characteristic output and the logarithmic characteristic output.

  That is, the first readout circuit 35 includes a signal sample hold circuit 351 including a signal sample hold switch Sa1 and a signal sample hold capacitor Ca1, and a noise sample hold circuit 352 including a noise sample hold switch Sa2 and a noise sample hold capacitor Ca2. A sample-and-hold circuit 353 for outputting a linear characteristic has a sampling (sampling) signal (image signal) and noise (noise signal) as an input analog signal, and temporarily holds this value. Similarly, the second readout circuit 36 includes a signal sample hold circuit 361 composed of a signal sample hold switch Sb1 and a signal sample hold capacitor Cb1, and a noise sample hold circuit 362 composed of a noise sample hold switch Sb2 and a noise sample hold capacitor Cb2. A sample-and-hold circuit 363 for logarithmic characteristic output, and samples and holds signals and noise.

  The first and second readout circuits 35 and 36 include amplifiers 354 and 364 (analog amplifiers; differential amplifiers), respectively, and noise components due to sensitivity variations of the pixels 311 are removed by these amplifiers. At the same time, the signal of each pixel 311 is amplified in an analog state, that is, it is amplified by applying an analog gain. Image data (image signal) having linear characteristics and logarithmic characteristics subjected to noise removal and gain adjustment by the amplifiers 354 and 364 is transmitted to the processing unit (A / D conversion unit 4) at the subsequent stage. Note that a circuit including one signal sample hold circuit 351 (361) and noise sample hold circuit 352 (362) in the sample hold circuit 353 (363) is expressed as a vertical output circuit 350 (360).

  The A / D converter 4 converts the analog image signal (analog signal) amplified by the amplifiers 354 and 364 in the image sensor 3 into a digital image signal (digital signal). The image processing unit 5 performs various image processing (digital signal processing) on the image signal having each characteristic input from the A / D conversion unit 4. The main control unit 6 includes a ROM that stores each control program, a RAM that temporarily stores data, and a control program that is read from the ROM and executed (central processing unit: CPU). It is responsible for overall operation control. The main control unit 6 controls the imaging operation of the imaging sensor 3 and other image processing parameters based on the image evaluation result obtained from the image signal in the image processing unit 5.

  By the way, the digital camera 1 of the present embodiment is configured to be able to perform individual image processing on each image signal of each characteristic obtained by imaging with photoelectric conversion characteristics having different characteristics. FIG. 4 is a functional block diagram of the image sensor 3 and the image processing unit 5 relating to this configuration. As shown in the figure, the imaging sensor 3 includes a pixel unit 31, a first amplifier 371 and a second amplifier 372 connected to the pixel unit 31, and a first amplification factor control unit 381 and a second amplification factor control unit 382. ing. The image processing unit 5 includes a first characteristic image processing unit 51, a second characteristic image processing unit 52, a first characteristic amplification factor calculation unit 53, a second characteristic amplification factor calculation unit 54, and a synthesis unit 55. Yes. Here, it is assumed that there are two types of the different photoelectric conversion characteristics, which are expressed as a first characteristic and a second characteristic, respectively.

The first amplifier 371 and the second amplifier 372 respectively amplify the image signal of each characteristic by the pixel unit 31, that is, the first characteristic imaging signal and the second characteristic imaging signal by applying a predetermined analog gain, and then the imaging signal Are output to the image processing unit 5. However, these image pickup signals, which are converted from analog signals to digital signals by an A / D conversion unit 4 (not shown), are input to the image processing unit 5. The first and second characteristic image processing units 51 and 52 perform individual image processing on the first and second characteristic imaging signals, respectively. The first characteristic amplification factor calculation unit 53 calculates the amplification factor for the first characteristic (first characteristic amplification factor) based on the processing parameter (evaluation value) in the first characteristic image processing unit 51. The calculated first characteristic gain is transmitted to the first gain control unit 381 as a control signal. The second characteristic amplification factor calculation unit 54 calculates an amplification factor for the second characteristic (second characteristic amplification factor) based on the processing parameters in the second characteristic image processing unit 52. Similarly, the second characteristic gain is transmitted to the second gain control unit 382 as a control signal. The first and second gain control units 381 and 382 in the image sensor 3 are based on the information on the first and second characteristic gains from the image processing unit 5, respectively. It controls the amplification operation of 372. The synthesizing unit 55 performs a predetermined synthesizing process on the first and second characteristic image data subjected to various image processes by the first and second characteristic image processing units 51 and 52. The combined image after the combining process is output to the subsequent processing unit.

  As described above, the imaging sensor 3 and the image processing unit 5 output imaging signals (analog signals) having respective characteristics from the imaging sensor 3, and the image processing unit 5 receives these imaging signals (digital signals) to perform predetermined image processing. In addition, the characteristic image is synthesized, the amplification factor for each characteristic is calculated, and the amplification factor information is fed back to the imaging sensor 3. In the imaging sensor 3, the imaging signal of each characteristic is based on the amplification factor information. Of the image signal (analog amplification), so-called loop control related to analog amplification of the imaging signal is possible.

  In the first embodiment, sensitivity control (sensitivity correction) processing using linear characteristic and logarithmic characteristic imaging signals obtained by the imaging sensor 3 is performed using the configuration shown in FIG. This will be described in detail below. FIG. 5 is a block configuration diagram of the imaging sensor 3 in which the imaging sensor 3 shown in FIG. 4 is associated with the configuration of the actual imaging sensor 3 shown in FIG. The imaging sensor 3 illustrated in FIG. 5 includes the pixel unit 31 illustrated in FIG. 3, a vertical scanning unit 320 corresponding to the vertical scanning circuit 32, a first horizontal scanning circuit 33, and a first readout circuit 35 (an amplifier 354 is not included). ) Corresponding to the first horizontal scanning section 330, the second horizontal scanning circuit 34 and the second readout circuit 36 (not including the amplifier 364), amplifiers 354 and 364, and FIG. The sensor control part 380 containing the 1st and 2nd gain control part 381 and 382 shown is provided.

  The imaging sensor 3 according to the present embodiment performs exposure with linear characteristics and logarithmic characteristics for each pixel row (one horizontal line) in the pixel unit 31 (all pixels 311), and the imaging signals with these characteristics differ for each characteristic. Output from an output circuit (read circuit or output terminals (amplifiers 354 and 364)). The imaging sensor 3 performs an imaging operation in response to control signals from the outside, that is, the image processing unit 5 or the main control unit 6 (first and second amplification factors (amplification factor control signals), timing signals, etc.). Outputs characteristic imaging signals. The imaging signal obtained by the photoelectric conversion in the pixel unit 31 is subjected to the first horizontal scanning for each characteristic by the vertical scanning unit 320 for the imaging signal for one horizontal line (the pixel signal of each pixel 311 in the pixel row of the horizontal one line). Is output to the unit 330 or the second horizontal scanning unit 340 and is held by these horizontal scanning units. The imaging signals having the respective characteristics held in the first and second horizontal scanning units 330 and 340 are amplified by the amplifiers 354 and 364 for each pixel, and the amplifier 354 receives linear characteristics corresponding to the first characteristic imaging signal. An imaging signal is output from the amplifier 364 as a logarithmic characteristic imaging signal corresponding to the second characteristic imaging signal. The amplification factors of the amplifiers 354 and 364 are set by the sensor control unit 380 based on the control signal from the outside. The sensor control unit 380 controls amplification processing of the amplifiers 354 and 364 in response to the control signal from the outside, and performs photoelectric conversion operation of the pixel unit 31, vertical scanning operation of the vertical scanning unit 320, and The horizontal scanning operation of the first and second horizontal scanning units 330 and 340 is controlled.

Here, FIG. 6 shows the timing for the exposure and the output (reading) of the imaging signal at each photoelectric conversion characteristic for each horizontal line. For example, for three horizontal lines, first, in one horizontal line (line L1), exposure with a linear characteristic is started, and when a predetermined exposure time ends, an imaging signal obtained by exposure with this linear characteristic. Are output to and held by the first horizontal scanning unit 330 (t1 to t3; time from time t1 to t3). Next, when the exposure with the logarithmic characteristic is started and the predetermined exposure time is completed, the imaging signal obtained by the exposure with the logarithmic characteristic is output to the second horizontal scanning unit 340 and held (t3 to t4). . Thereafter, the imaging signals having the respective characteristics are read out from the first horizontal scanning unit 330 and the second horizontal scanning unit 340, that is, are amplified and output by the amplifiers 354 and 364 at a predetermined amplification rate (t4 to t6). . The next horizontal line (line L2) and the next horizontal line (line L3) are similarly subjected to exposure and readout control, and the line L2 and line L3 are sequentially imaged following the image signal output of the line L1. Signals are output (sequentially output from t6 to t8 of line 2 and t8 to t9 of line 3 following the imaging signal output times t4 to t6 of line L1). As described above, the exposure and output operations of the linear characteristics and logarithmic characteristics for each line are performed over all horizontal lines (all pixel rows) of the pixel unit 31.

  FIG. 7 shows the main processing in the image processing unit 5 in the case of performing sensitivity control (sensitivity correction) processing using imaging signals of various characteristics using the configuration shown in FIG. 4 as with the imaging sensor 3 shown in FIG. It is a block block diagram about the function part regarding this sensitivity control. As shown in the figure, the image processing unit 5 includes a linear characteristic noise cancellation unit 511, an evaluation value calculation unit 512, a logarithmic characteristic noise cancellation unit 521, a characteristic conversion unit 522, an amplification factor calculation unit 541, a selection unit 551, and A comparison unit 552 is provided. However, the linear characteristic noise cancellation unit 511 and the evaluation value calculation unit 512 correspond to the first characteristic image processing unit 51 shown in the image processing unit 5 of FIG. 4, and the amplification factor calculation unit 541 calculates the first characteristic amplification factor. The logarithmic characteristic noise canceling unit 521 and the characteristic converting unit 522 correspond to the second characteristic image processing unit 52, and the selecting unit 551 and the comparing unit 552 correspond to the synthesizing unit 55.

The linear characteristic noise canceling unit 511 is a fixed pattern noise (FPN) that is specific to the linear characteristic imaging signal that is input after being converted into a digital signal by the A / D conversion unit 4, that is, the linear characteristic imaging. ) Is removed (cancelled).
The FPN in this case is a variation in dark current (offset variation) that differs for each pixel generated in the pixel 311.

  The evaluation value calculation unit 512 is a linear characteristic imaging signal from which FPN has been removed by the linear characteristic noise canceling unit 511, and the entire linear characteristic imaging signal (linear characteristic data) over one imaging screen (one frame). A predetermined evaluation value is calculated from Here, the average value of the entire linear characteristic data is calculated as the evaluation value. The calculated evaluation value is output to the amplification factor calculation unit 541. However, as for the evaluation value, not only the average value (addition average) of the entire screen but also the average value in the main subject region, the maximum or minimum value, or the weighted average value or histogram based on the spatial arrangement of pixels, etc. Various values such as dispersion of pixel values based on the above can be adopted.

  The amplification factor calculation unit 541 calculates an amplification factor (linear characteristic amplification factor) for the linear characteristic imaging signal based on the input evaluation value (average value). This linear characteristic amplification factor is calculated by a comparison process (division process) between the evaluation value and the reference evaluation value. This reference evaluation value is stored in advance in the amplification factor calculation unit 541.

Note that the gain of the imaging signal in the imaging apparatus of the present embodiment can be set with different values for the gain of the linear characteristic imaging signal and the logarithmic characteristic imaging signal (see FIG. 4). In particular, in the sensitivity control process here, the amplification factor of the linear characteristic imaging signal is variable based on the evaluation value (average value) obtained from the same linear characteristic imaging signal. That is, since the amplification factor is calculated only for the linear characteristic imaging signal, FIG. 7 does not show an amplification factor calculation unit corresponding to the second characteristic amplification factor calculation unit 54 shown in FIG. The evaluation value may be not only the average value over the entire screen (frame) but also the weighted average value based on the spatial arrangement of pixels, the distribution of pixel values based on a histogram, and the like.

Similarly, the logarithmic characteristic noise canceling unit 521 removes (cancels) fixed pattern noise (FPN) in the logarithmic characteristic imaging signal that has been converted into a digital signal by the A / D conversion unit 4 and input. Is. The FPN in this case is a variation in threshold value at which a different logarithmic characteristic is generated for each pixel generated in the pixel 311. The characteristic conversion unit 522 converts the characteristic of the logarithmic characteristic imaging signal (linear characteristic data) from which the FPN has been removed by the logarithmic characteristic noise canceling unit 521, from the logarithmic characteristic to the linear characteristic, that is, from the logarithmic characteristic to the linear characteristic. To do. This characteristic conversion is performed using an LUT (Look Up Table) stored in the characteristic conversion unit 522 in advance. This LUT (conversion characteristic) is changed according to the amplification factor of the linear characteristic imaging signal, that is, the linear characteristic amplification factor calculated by the amplification factor calculation unit 541 (the imaging sensor 3 amplifies using this linear characteristic amplification factor). It can be said that it is changed based on the output linear characteristic imaging signal). In this case, for example, information on various LUTs corresponding to each linear characteristic amplification factor is stored in the characteristic conversion unit 522, and from these various LUTs according to the value of the linear amplification factor given from the amplification factor calculation unit 541. The configuration may be such that the corresponding LUT is selected. In this embodiment, amplification processing (amplification control) is not performed on the logarithmic characteristic image pickup signal. At this time, the amplification factor of the amplifier 364 (FIG. 5) or the second amplifier 372 is “1”. It may be considered that amplification is performed. In this case, for example, an amplification factor control signal (logarithmic characteristic amplification factor) may be transmitted from the characteristic conversion unit 522 to the amplifier 364 or the second amplifier 372 so that the amplification factor is “1”.

  The selection unit 551 sets the level of the linear characteristic imaging signal (signal level) from the linear characteristic imaging signal output from the evaluation value calculation unit 512 and the linear characteristic imaging signal after characteristic conversion output from the characteristic conversion unit 522. In response, the image pickup signal is selected (the selected image pickup signals are combined or the linear characteristic image data and the logarithmic characteristic image data are selectively combined). The selected imaging signal is output to a subsequent processing unit. The comparison unit 552 compares, for example, a predetermined inflection point level given by the main control unit 6 and a linear characteristic imaging signal (an inclination of a graph described later; or an evaluation value) output from the evaluation value calculation unit 512. The information of the inflection point position (coordinate position determined by the inflection point level and the inflection point illuminance described later) obtained from this comparison result is given to the selection unit 551. The inflection point level shown in FIG. 8 may be any level as long as it is below the saturation level of the linear characteristic imaging signal and the logarithmic characteristic imaging signal, that is, the sensor output (the same applies to FIGS. 14 and 15 described later). ).

  With such a configuration, the amplification factor of the imaging signal in the digital camera 1 of the present embodiment controls (sets) the amplification factor of the linear characteristic imaging signal and the amplification factor of the logarithmic characteristic imaging signal with different values. In particular, the amplification factor of the linear characteristic imaging signal can be controlled based on an evaluation value (average value) obtained only from the linear characteristic imaging signal.

FIG. 8 is a graph for explaining the selection process in the selection unit 551. In FIG. 8, the imaging signal indicated by reference numeral 601 (range from the origin to the saturation level in FIG. 8) is amplified by the amplifier 354 of the imaging sensor 3 using a certain linear characteristic amplification factor calculated by the amplification factor calculation unit 541. The linear characteristic imaging signal (linear characteristic imaging signal 601) transmitted to the selection unit 551 through the linear characteristic noise cancellation unit 511 and the evaluation value calculation unit 512. On the other hand, the imaging signal indicated by reference numeral 602 (range from the origin to the saturation level) is amplified by the amplifier 354 using a linear characteristic amplification factor that is larger than the linear characteristic amplification factor in the case of the linear characteristic imaging signal 601. Similarly, a linear characteristic imaging signal (linear characteristic imaging signal 602) transmitted to the selection unit 551 via the linear characteristic noise canceling unit 511 and the evaluation value calculating unit 512. By the way, when the imaging signal level with respect to the sensor surface illuminance of the imaging sensor 3, that is, the output level with respect to the input of the subject luminance, is the “sensitivity” (corresponding to ISO sensitivity) of the digital camera 1, the amplification factor (linear characteristic amplification factor) ) Are different, it can be said that the linear characteristic imaging signal 601 is a linear characteristic imaging signal at low sensitivity, and the linear characteristic imaging signal 602 is a linear characteristic imaging signal at high sensitivity. The linear characteristic gain at the time of low sensitivity is expressed as a low sensitivity gain, and the linear characteristic gain at the time of high sensitivity is expressed as a high sensitivity gain. Further, the saturation level in the figure is a level at which the output of the imaging sensor 3 (linear characteristic imaging signal) is saturated (for example, the maximum value of the output level of the imaging sensor 3). Not the saturation level of processing). An imaging signal denoted by reference numeral 603 is a logarithmic characteristic imaging signal (logarithmic characteristic imaging signal 603) output from the imaging sensor 3 (amplifier 364) and transmitted to the characteristic converting unit 522 via the logarithmic characteristic noise canceling unit 521. Show.

  Further, the inflection point level shown in FIG. 8 is a level (output level information) given to the comparison unit 552 and given as a level equal to or lower than the saturation level. However, as shown in FIG. 9, at the time of exposure with a linear characteristic, the operation of the image sensor 3 is controlled so that the inflection point (inflection point level) is equal to or higher than the saturation level of the sensor output (the output is thereby performed). All signals have linear characteristics). Further, at the time of exposure with logarithmic characteristics, the operation of the image sensor 3 is controlled so that the inflection point is below the minimum level of the sensor output (thus all output signals have logarithmic characteristics).

  The logarithmic characteristic imaging signal 603 is converted into a linear characteristic imaging signal corresponding to the magnitude of the linear characteristic amplification factor (given from the amplification factor calculation unit 541) (linear having a slope on the graph corresponding to the amplification factor) in the characteristic conversion unit 522. Characteristic). That is, a logarithmic characteristic imaging signal 603 whose characteristic is converted according to the high sensitivity amplification factor is a linear characteristic imaging signal 604 (this is referred to as a converted linear characteristic imaging signal 604), and a logarithmic characteristic imaging signal 603 is the low sensitivity amplification. What has undergone characteristic conversion according to the rate is a post-conversion linear characteristic imaging signal 605.

  The selection unit 551 selects an imaging signal imaged with a linear characteristic until the output level of the imaging signal is equal to or lower than the inflection point level, and a level (output value) higher than the inflection point level is imaged with a logarithmic characteristic. Select a signal. That is, when the sensitivity is low, the linear characteristic imaging signal 601 is selected below the inflection point level, and the converted linear characteristic imaging signal 605 is selected at a level higher than the inflection point level. In other words, at the time of low sensitivity, from the inflection point level in the linear characteristic imaging signal 601 and the converted linear characteristic imaging signal 605 from the inflection point level (low sensitivity inflection point illuminance) to the maximum value of DR. Image signal 6051 is selected. Similarly, at the time of high sensitivity, the linear characteristic imaging signal 602 and the converted linear characteristic imaging signal 604 are selected, that is, the linear characteristic imaging signal 602 at the inflection point level (high sensitivity inflection point illuminance). And the imaging signal 6041 from the inflection point level to the maximum DR value in the converted linear characteristic imaging signal 604 is selected. The imaging signals having the selected characteristics are synthesized by the selection unit 551 and output as one image (image generation (image synthesis)). However, the above inflection point illuminance, which is the level of switching between linear and logarithmic characteristics with respect to imaging illuminance, varies depending on the amplification factor of the linear characteristic imaging signal, and when high sensitivity (high amplification factor), low sensitivity (low amplification factor) Change from linear to logarithmic characteristics at lower illuminance.

  With the above configuration, the amplification factor of the photoelectric conversion characteristic (linear characteristic) for imaging at least the low luminance side is calculated using the evaluation value obtained only from the imaging signal of the photoelectric conversion characteristic of the low luminance side, and this amplification factor is calculated. The amplification process is performed only on the image pickup signal representing the image with the lowest luminance after image generation, that is, on the image pickup signal having photoelectric conversion characteristics on the low luminance side, for example. By combining the amplified linear imaging signal and the other non-amplified photoelectric conversion characteristic, that is, logarithmic imaging signal, by the selection unit 551 as described above, imaging sensitivity control is performed without changing the imaging DR. (To increase sensitivity). Further, since amplification is performed in the imaging sensor 3 in the previous stage of the A / D conversion unit 4, that is, amplification processing (analog amplification) can be performed on an analog imaging signal that does not include noise generated in the A / D conversion unit 4. , The SN ratio becomes high (when the SN ratio is high, the imaging sensitivity control with DR maintained can be performed). Incidentally, it is the dark image portion in the image (the image obtained by the image generation), that is, the low-luminance image, that the amplification process (sensitivity increase) is performed on the imaging signal of the photoelectric conversion characteristic (linear characteristic) on the low-luminance side. By trying to make the part look better (to improve the gradation).

(Embodiment 2)
In the first embodiment, individual image processing for each image signal of each characteristic is performed using the configuration of the imaging sensor 3 and the image processing unit 5 shown in FIG. Now, the configuration of the image sensor 3a and the image processing unit 5a shown in FIG. 10 is used. In the case of FIG. 4, the first and second characteristic imaging signals are individually output from the corresponding output terminals (the first amplifier 371 and the second amplifier 372), but in the case of FIG. The second characteristic imaging signal is output from the same output terminal (amplifier 373) at different timings. That is, the imaging sensor 3a includes an amplification factor controller 383 instead of the first and second amplification factor controllers 381 and 382 shown in FIG. 4, and an amplifier 373 instead of the first amplifier 371 and the second amplifier 372. The image processing unit 5a includes a switching unit 56.

  The amplifier 373 amplifies the first characteristic imaging signal and the second characteristic imaging signal output from the pixel unit 31 by applying a predetermined analog gain, and outputs these imaging signals to the image processing unit 5a. The image processing unit 5a receives the first and second characteristic imaging signals, performs image processing for each characteristic in the first and second characteristic image processing units 51 and 52, and uses the result of the image processing to synthesize. A predetermined synthesizing process is performed by the unit 55, and first and second characteristic amplification factors are calculated by the first and second characteristic amplification factor calculation units 53 and 54 using processing parameters (evaluation values) at the time of image processing. (These processes are the same as in the image processing unit 5). Information on the calculated first and second characteristic amplification factors is transmitted to the switching unit 56. The switching unit 56 switches between the signal with the first characteristic gain and the signal with the second characteristic gain, and transmits the signal to the gain control unit 383. The amplification factor control unit 383 controls the amplification operation for the first characteristic imaging signal based on the transmitted information on the first characteristic amplification factor, and amplifies the second characteristic imaging signal based on the information on the second characteristic amplification factor. Control the behavior.

  In the present embodiment, sensitivity control (sensitivity correction) processing using imaging signals having linear characteristics and logarithmic characteristics is performed using the configuration shown in FIG. FIG. 11 shows the configuration of this embodiment with respect to the configuration of FIG. 5 in the first embodiment. The imaging sensor 3a differs from the imaging sensor 3 shown in FIG. 5 in a main control unit 380a, a horizontal scanning unit 330a, and an amplifier 373. The main control unit 380a is a control unit including the amplification factor control unit 383 shown in FIG. The horizontal scanning unit 330 a reads out an image pickup signal having linear characteristics and logarithmic characteristics from the pixel unit 31. The imaging sensor 3a performs exposure with linear and logarithmic characteristics for each frame, and outputs an imaging signal with these characteristics from the same output terminal (amplifier 373) as described above (1 frame for linear characteristics and 1 for logarithmic characteristics). Exposure is performed over a total of two frames, and these imaging signals are output). An imaging signal obtained by photoelectric conversion in the pixel unit 31 is converted into an image signal for one horizontal line (pixel signal of each pixel 311 in the pixel row of the horizontal one line) by the vertical scanning unit 320 for each characteristic. Is output and held. The image pickup signal having each characteristic held in the horizontal scanning unit 330a is amplified by an amplifier 373 (amplifier) for each pixel and output. However, the amplification factor of the amplifier 373 is configured to be variable for each frame. The amplification factor of the amplifier 373 is set by the sensor control unit 380a based on the control signal from the outside. The sensor control unit 380a controls the amplification process of the amplifier 373 in response to an external control signal, and controls various operations of the pixel unit 31, the vertical scanning unit 320, and the horizontal scanning unit 330a.

Here, FIG. 12 shows the timing for exposure and output (reading) of the imaging signal at each photoelectric conversion characteristic for each horizontal line in the present embodiment. For example, for three horizontal lines, first, in one horizontal line (line L1), exposure with a linear characteristic is started, and when a predetermined exposure time ends, an imaging signal obtained by exposure with this linear characteristic. The (linear characteristic imaging signal) is output to the horizontal scanning unit 330a and held (read) (t11 to t12). The linear characteristic imaging signal of the line L1 stored in the horizontal scanning unit 330a is amplified by the amplifier 373 at a predetermined amplification factor and output (t12 to t13). Similarly, the exposure control and the readout control are performed for the lines L2 and L3, and the linear characteristic imaging signals of the lines L2 and L3 are sequentially output following the output of the linear characteristic imaging signal of the line L1 (linear characteristic imaging signal). Are continuously output from t12 to t13 of the line L1, t13 to t14 of the line 2, and t14 to t16 of the line 3).

  When the exposure with the linear characteristic in the lines L1 to L3 is completed, the exposure with the logarithmic characteristic in the line 1 is started, and the horizontal scanning unit of the logarithmic characteristic imaging signal obtained by this exposure is started as in the case of the linear characteristic. Output to 330a is performed (t15 to t16), and the amplifier 373 performs amplification processing at a predetermined amplification factor and outputs (t16 to t18). Similarly, the line L2 and the line L3 are subjected to exposure and readout control, and the logarithmic characteristic imaging signals of the line L2 and the line L3 are sequentially output following the output of the logarithmic characteristic imaging signal of the line L1 (logarithmic characteristic imaging signal). Are continuously output from t16 to t18 on line L1, t18 to t20 on line 2, and t20 to t21 on line 3). In this way, exposure and output operations with linear characteristics for each line are performed over all horizontal lines (all pixel rows) of the pixel unit 31.

  FIG. 13 is a block configuration of a functional unit mainly related to sensitivity control in the image processing unit 5a when performing sensitivity control (sensitivity correction) processing using imaging signals of various characteristics using the configuration shown in FIG. FIG. The image processing unit 5 a differs from the image processing unit 5 mainly in that it includes a memory 513. That is, since the linear characteristic imaging signal and the logarithmic characteristic imaging signal are alternately input for each frame, the image processing unit 5a changes processing according to the imaging signal having each characteristic input for each frame. At the time of input of the linear characteristic imaging signal, noise is removed by the linear characteristic noise canceling unit 511, and the linear characteristic amplification factor is calculated based on the evaluation value by the evaluation value calculation unit 512 and the amplification factor calculation unit 541. The linear characteristic imaging signal is stored in the memory 513. When the logarithmic characteristic imaging signal is input, the linear characteristic imaging signal stored in the memory 513 is read out, and transmitted through the read linear characteristic imaging signal, the logarithmic characteristic noise canceling unit 521 and the characteristic converting unit 522. The logarithmic characteristic imaging signal that has been subjected to noise removal and characteristic conversion is input to the selection unit 551, and the selection unit 551 performs imaging based on the level of the linear characteristic imaging signal from these linear characteristic imaging signal and logarithmic characteristic imaging signal. A signal is selected. Note that the selection processing in the selection unit 551 in the present embodiment is the same as that in the selection unit 551 in the first embodiment (the selection method described in FIG. 8 above).

  Similarly in the case of the present embodiment, the gain of the imaging signal in the digital camera 1 having the above-described configuration is to control (set) the amplification factor of the linear characteristic imaging signal and the amplification factor of the logarithmic characteristic imaging signal with different values. In particular, the amplification factor of the linear characteristic imaging signal can be controlled based on the evaluation value (average value) obtained only from the linear characteristic imaging signal.

By the way, in each of the above-described embodiments, the linear log sensor (imaging sensor 3 (3a)) having a linear characteristic on the low luminance side and a logarithmic characteristic on the high luminance side is used. And a photoelectric conversion characteristic that is a linear characteristic on the high luminance side, that is, a sensor having a photoelectric conversion characteristic composed of a first linear characteristic and a second linear characteristic that have different slopes on the graph (different exposure times, etc.) (linear linear sensor ) May be used. However, the first and second linear characteristics correspond to the linear characteristic and the logarithmic characteristic, respectively, and it is assumed that the first linear characteristic has a larger slope than the second linear characteristic. In this case, the second linear characteristic is photographed with a lower sensitivity (shorter exposure time) than imaging with the first linear characteristic, that is, the first linear characteristic is “high sensitivity (long exposure)”. The “linear photoelectric conversion characteristics” and the second linear characteristics are “low sensitivity (short-time exposure) linear photoelectric conversion characteristics”.

  As shown in FIG. 14, the imaging signals corresponding to the first linear characteristic are the first linear characteristic imaging signal 701 and the first linear characteristic imaging signal 702, and the imaging signal corresponding to the second linear characteristic is the second linear characteristic. This is a characteristic imaging signal 703. Also, the converted linear characteristic imaging signal 704 and the second linear characteristic imaging signal 703 are characteristic-converted according to the high-sensitivity amplification factor when the second linear characteristic imaging signal 703 is characteristic-converted according to the low-sensitivity amplification factor. What is the converted linear characteristic imaging signal 705.

  In this case, as in FIG. 8, at the time of low sensitivity, the inflection point level in the first linear characteristic imaging signal 701 and the converted linear characteristic imaging signal 705 with the inflection point level (low sensitivity inflection point illuminance) as a boundary. To the maximum value of DR is selected. Similarly, at the time of high sensitivity, the maximum value of DR from the inflection point level in the first linear characteristic imaging signal 702 and the converted linear characteristic imaging signal 704 with the inflection point level (high sensitivity inflection point illuminance) as a boundary. Up to the imaging signal 7041 is selected. The selected imaging signals are synthesized by the selection unit 551 and output. Also in this case, the amplification factor of the photoelectric conversion characteristic (first linear characteristic) for imaging at least the low luminance side is calculated using the evaluation value obtained only from the imaging signal of the photoelectric conversion characteristic of the low luminance side, and this amplification Using the rate, the amplification process may be performed only on the image signal representing the image with the lowest luminance after image generation, that is, on the image signal having the photoelectric conversion characteristic on the low luminance side, for example.

  As described above, according to the imaging apparatus (digital camera 1) of the present embodiment, the imaging apparatus has two or more different photoelectric conversion characteristics and can individually output imaging signals having the photoelectric conversion characteristics. The configured imaging device (imaging sensors 3, 3a), amplification means (first and second amplifiers 371, 372, amplifier 373, amplifiers 354, 364) for amplifying the imaging signals of the respective photoelectric conversion characteristics, and amplification means Amplification rate control means (first and second characteristic image processing units 51 and 52, first and second characteristic gain calculation units 53 and 54) for controlling the amplification rate, and each photoelectric conversion characteristic amplified by the amplification means Image generation means (synthesizer 55, selection part 551, and comparison part 552) for synthesizing the image pickup signals of the two, and amplifying the photoelectric conversion characteristics of the image pickup signals by the amplification factor control means. The rate of each photoelectric It is calculated based on 換特 of imaging signals, the amplification factor control is performed for each photoelectric conversion characteristic with the amplification factor issued the calculated. Therefore, it is possible to individually (with different values) set the amplification factor of the image pickup signal of each photoelectric conversion characteristic, that is, to perform amplification processing separately (separately) for each of the image pickup signals having different photoelectric conversion characteristics. Therefore, for example, the amplification factor of the linear characteristic imaging signal is controlled based on the evaluation value obtained only from the linear characteristic imaging signal having higher sensitivity than the logarithmic characteristic, that is, amplification processing is performed only on the linear characteristic imaging signal. The gain control can be performed, and the image signal with the amplified photoelectric conversion characteristic (linear characteristic) and the image signal with the other non-amplified photoelectric conversion characteristic are combined by the image generation means to generate an image. By doing so, it is possible to perform imaging sensitivity control (sensitivity correction) without changing DR.

  In addition, since the amplification factor control is performed by using the amplification factor of the imaging signal having at least the most sensitive photoelectric conversion characteristic in two or more different photoelectric conversion characteristics, the amplification factor having the highest sensitivity is amplified. It is possible to easily and reliably perform imaging sensitivity control without changing DR by synthesizing an imaging signal with conversion characteristics and an imaging signal with other photoelectric conversion characteristics that are not amplified to generate an image. become.

  In addition, since the different photoelectric conversion characteristics of the image pickup device are a linear characteristic and a logarithmic characteristic, an image pickup signal having a linear characteristic is amplified as the photoelectric conversion characteristic having the highest sensitivity and combined with an image signal having an unamplified logarithmic characteristic. As a result, when a solid-state imaging device (linear log sensor) having the linear / logarithmic characteristics is used, imaging sensitivity control can be performed without changing DR.

  Further, the amplification factor for the imaging signal having the linear characteristic is calculated based on the imaging signal having the linear characteristic by the amplification factor control unit, and the image having the lowest luminance after the image generation is represented by using the calculated amplification factor. Since the amplification factor of only the imaging signal is controlled, the imaging signal representing the image with the lowest luminance after this image generation, i.e., the imaging signal having a linear characteristic, for example, and the other non-amplified photoelectric conversion characteristic, i.e., the logarithmic imaging signal By synthesizing and generating an image, it is possible to control the imaging sensitivity without changing the DR.

  In addition, since the different photoelectric conversion characteristics of the imaging device are two or more linear characteristics having different sensitivities, the imaging signal having the linear characteristics which is the highest sensitivity is amplified, and other imaging signals having a linear characteristic which is not amplified. In the case of using a solid-state imaging device having two or more linear characteristics having different sensitivities, the imaging sensitivity control can be performed without changing the DR.

  Further, the amplification factor is calculated based on the imaging signal having the high sensitivity linear characteristic based on the imaging signal having the high sensitivity linear characteristic, by the amplification factor control means, and the amplification factor for the imaging signal having the highest sensitivity among the two or more linear characteristics is calculated. Since the amplification factor is used to control the amplification factor of only the imaging signal that represents the image with the lowest luminance after the image generation, the imaging signal that represents the image with the lowest luminance after the generation of the amplified image, for example, high sensitivity. By synthesizing an imaging signal having a linear characteristic and an imaging signal having a non-amplified linear characteristic to generate an image, the imaging sensitivity can be controlled without changing the DR. Since all the different photoelectric conversion characteristics are linear characteristics, for example, a process of converting logarithmic characteristics to linear characteristics is not necessary when an image is generated by synthesizing image pickup signals of the respective characteristics, and thus DR is changed. Therefore, it is possible to easily control the imaging sensitivity.

  Furthermore, the first image pickup signal having the gain-controlled characteristics and the image pickup signal having the characteristics other than the gain-controlled characteristics are subjected to data conversion according to the calculated gain by the image generation means. Therefore, the first imaging signal having a predetermined signal level or less and the second imaging signal having a level higher than the signal level are selected, and the selected first and second imaging signals are combined to generate an image. By setting the predetermined signal level as the inflection point level of each photoelectric conversion characteristic, it is possible to easily generate an image by synthesizing the imaging signals of each characteristic without changing the DR. In addition, this invention can take the following aspects.

  (A) In each of the above embodiments, amplification (sensitivity increase) is performed by analog processing (first and second amplifiers 371 and 372 or amplifier 373), but digital processing (for example, digital data in the image processing unit 5). It is also possible to carry out in the process. In this case, the S / N ratio may be deteriorated as compared with analog processing using an imaging signal that does not include noise generated in the A / D converter, but the analog amplifier for the analog amplification and the amplification factor are controlled. Since it is not necessary to provide an amplification factor control unit or the like, the configuration can be simplified.

  (B) In each of the above embodiments, two different photoelectric conversion characteristics (linear characteristics / logarithmic characteristics, or first linear characteristics / second linear characteristics) are used, but even if there are three or more different photoelectric conversion characteristics. Good. In this case, as shown in FIG. 15, for example, the selection unit 55 for imaging signals having three types of photoelectric conversion characteristics, that is, the first linear characteristic imaging signal 801, the second linear characteristic imaging signal 802, and the third linear characteristic imaging signal 803. The selection operation may be performed. Specifically, the sensitivity of the imaging signal having the highest sensitivity among these three imaging signals 801 to 803 is controlled (amplification factor is changed), and the remaining two imaging signals are amplified by the sensitivity control. Data conversion (LUT conversion; characteristic conversion) is performed so as to match the rate (gradient on the graph), and a selection operation is performed on these imaging signals.

The first linear characteristic imaging signal 804 is obtained by changing the amplification factor of the first linear characteristic imaging signal 801 having the highest sensitivity among the three to further increase the sensitivity. The converted linear characteristic imaging signal 806 is an imaging signal obtained by characteristic-converting the second linear characteristic imaging signal 802 and the third linear characteristic imaging signal 803 according to the linear amplification factor of the first linear characteristic imaging signal 801. The converted linear characteristic imaging signal 805 is an imaging signal obtained by performing characteristic conversion on the second linear characteristic imaging signal 802 and the third linear characteristic imaging signal 803 according to the linear amplification factor of the first linear characteristic imaging signal 804. It is.

  Similarly, in this case, when the sensitivity is low, DR is determined from the inflection point level in the first linear characteristic imaging signal 801 and the converted linear characteristic imaging signal 806 with the inflection point level (low sensitivity inflection point illuminance) as a boundary. The imaging signal up to the maximum value is selected. At the time of high sensitivity, imaging from the inflection point level to the maximum value of DR in the first linear characteristic imaging signal 804 and the converted linear characteristic imaging signal 805 with the inflection point level (high sensitivity inflection point illuminance) as a boundary. Signal is selected. The selected imaging signals are synthesized by the selection unit 551 and output. When this is applied to the first embodiment, three horizontal scanning units or three amplifiers are required corresponding to the three photoelectric conversion characteristics. On the other hand, when applied to the second embodiment, three frames are required. Therefore, it is necessary to have a configuration in which exposure with different characteristics and output of imaging signals are performed. Similarly, when there are four or more different photoelectric conversion characteristics, the amplification factor is changed only for the imaging signal having the highest sensitivity photoelectric conversion characteristic, and the remaining photoelectric conversion characteristics are the amplification factors of the highest sensitivity photoelectric conversion characteristic. The data is converted in accordance with the image signal, and the selection operation for these imaging signals may be performed.

  (C) Imaging with three or more different photoelectric conversion characteristics, for example, three types of linear characteristics, that is, first to third linear characteristics (see FIG. 15) as shown in the modification (B), and calculating a predetermined amplification factor Imaging signals other than the imaging signal (the third linear characteristic imaging signal having the lowest sensitivity) representing the image with the highest luminance after image generation using the linear amplification factor calculated by the unit (two linear characteristic imaging signals on the low luminance side) The first linear characteristic imaging signal and the second linear characteristic imaging signal having higher sensitivity than the third linear characteristic imaging signal may be controlled. However, in this case, the linear amplification factor is calculated based on the first linear characteristic imaging signal and / or the second linear characteristic imaging signal. Further, in this case, the third linear characteristic imaging signal is data-converted so as to coincide with the amplification factors of the first linear characteristic imaging signal and the second linear characteristic imaging signal for which the amplification factor control is performed as described above. .

  As described above, the imaging signal representing the image with the highest luminance after the image generation using the amplification factor for the imaging signal of each photoelectric conversion characteristic calculated based on the imaging signal of each photoelectric conversion characteristic by the amplification factor control unit. Since the amplification factor of the other imaging signal is controlled, the imaging signal other than the imaging signal representing the image with the highest luminance after the generation of the amplified image (the imaging signal on the low luminance side) and the other are not amplified. By combining the imaging signal (the imaging signal with the highest luminance or the lowest sensitivity) to generate an image, the imaging sensitivity can be controlled without changing the DR.

  (D) In each of the above embodiments, the amplifier is provided in the image sensor 3 (3a), but may be provided outside the image sensor 3. In each of the above embodiments, the A / D conversion unit 4 is provided outside the imaging sensor 3, but may be provided in the imaging sensor 3 (for example, in the vertical output circuits 350 and 360).

1 Digital camera (imaging device)
3, 3a Image sensor (image sensor)
31 pixel unit 311 pixel 32 vertical scanning circuit 33 first horizontal scanning circuit 34 second horizontal scanning circuit 35 first readout circuit 36 second readout circuit 330 first horizontal scanning unit 340 second horizontal scanning unit 330a horizontal scanning unit 350, 360 Vertical output circuit 354, 364 Amplifier (amplifying means)
371 First amplifier (amplifying means)
372 Second amplifier (amplifying means)
373 Amplifier (amplification means)
380, 380a Sensor control unit 383 Amplification rate control unit 4 A / D conversion unit 5, 5a Image processing unit 51 Image processing unit for first characteristic (amplification rate control means)
52 Image processing unit for second characteristic (amplification rate control means)
53 1st characteristic gain calculation part (gain control means)
54 Second characteristic gain calculation unit (gain control means)
55 Synthesizer (image generation means)
512 Evaluation Value Calculation Unit (Amplification Factor Control Unit)
513 Memory 522 Characteristic conversion unit 541 Amplification rate calculation unit (amplification rate control means)
551 selection unit (image generation means)
552 comparison unit (image generation means)
6 Main control unit

Claims (4)

An image sensor having a photoelectric conversion characteristic different from a linear characteristic and a logarithmic characteristic, and configured to individually output an image signal of each photoelectric conversion characteristic as an analog signal;
A / D conversion means for converting an analog signal output from the image sensor into a digital signal;
Among the digital signals converted by the A / D conversion means, a digital signal corresponding to the imaging signal having the linear characteristic is input, and fixed pattern noise peculiar to imaging with the linear characteristic in the input digital signal is removed. Noise canceling means for linear characteristics,
Among the digital signals converted by the A / D conversion means, a digital signal corresponding to the logarithmic characteristic imaging signal is input, and fixed pattern noise peculiar to imaging with the logarithmic characteristic in the input digital signal is removed. Logarithmic characteristic noise canceling means,
The fixed signal noise peculiar to the imaging with the logarithmic characteristic is removed by the imaging signal from which the fixed pattern noise peculiar to the imaging with the linear characteristic has been removed by the noise canceling means for the linear characteristic and the noise canceling means for the logarithmic characteristic. An image pickup apparatus comprising: an image generation unit that generates an image by combining the image pickup signal. Amplifying means for amplifying the imaging signal of each photoelectric conversion characteristic;
The imaging apparatus according to claim 1, further comprising: an amplification factor control unit that individually controls an amplification factor for the imaging signal of each photoelectric conversion characteristic in the amplification unit for each photoelectric conversion characteristic. The amplification factor control means includes
An amplification factor for the imaging signal having the linear characteristic is calculated based on the imaging signal having the linear characteristic, and the amplification factor of only the imaging signal having the linear characteristic is controlled after the image generation using the calculated amplification factor. The imaging apparatus according to claim 2, characterized in that: The image generating means includes
A first imaging signal that is the imaging signal having the linear characteristic with the amplification factor controlled, and a second imaging that is obtained by data conversion of the imaging signal having the logarithmic characteristic according to the amplification factor calculated by the amplification factor control means. Selecting a first imaging signal having a predetermined signal level or less and a second imaging signal having a level higher than the signal level from a signal, and generating the image by combining the selected first and second imaging signals The imaging apparatus according to claim 3.

Images