JP6366325B2 - Imaging system - Google Patents

Description

  The present invention relates to an imaging system.

  In recent years, there has been proposed a method of calculating a subject distance by projecting infrared light onto a subject and receiving the reflected light with an imaging device. The speed of light is 3 × 10 m / s. Since this is known, the pulse light is emitted from the light source toward the target, the reflected light bounced back from the target is received, and the distance to the target is measured by measuring the delay time of the pulsed light. Can do. The TOF (Time-Of-Flight) method is a method for measuring the distance to an object by measuring the time of flight of this pulsed light. The distance measurement range with respect to the delay time measurement range can be read. For example, if the delay time measurement range is 1 μs and the delay time measurement resolution is 1 ns, the 150 m range can be measured with a resolution of 15 cm. It can be used as a distance sensor.

  By applying this principle, a technique for acquiring a two-dimensional distance image using a CMOS solid-state imaging device having a charge distribution type pixel structure has been proposed (for example, see Patent Document 1). Specifically, the irradiation pulse light is reflected by the object, and the signal component corresponding to the preceding portion of the reflected pulse light that arrives late and the signal component corresponding to the following portion are distributed by the switch. The distance information for each pixel can be obtained by detecting these signals for each pixel and obtaining the ratio of the preceding part and the succeeding part.

  Further, as a method of not reducing the aperture ratio of the photodiode by the TOF method, a method of obtaining distance information using different pixel outputs by changing transfer timing between even rows and odd rows has been proposed (for example, Patent Documents). 2).

JP 2004-294420 A JP 2010-213231 A

  In general, in a solid-state imaging device for generating an image, red (R), green (G), and blue (B) color filters are arranged in a Bayer shape on pixels. In Patent Document 2, distance information is obtained by using even row pixel signals and odd row pixel signals. However, in this case, pixel outputs of different colors are used, and there is a problem that the processing circuit at the subsequent stage becomes complicated in order to obtain accurate distance information. Further, since one floating diffusion portion is shared by even-numbered row pixels and odd-numbered row pixels, a memory for holding signals that are transferred almost simultaneously is necessary. In addition, a normal imaging system has a means for generating focus detection information by phase difference detection using a passive dedicated sensor, but there is a problem that accuracy is generally lowered in a dark situation.

  An object of the present invention is to provide an imaging system capable of generating a pixel signal for calculating a distance to a subject with a simple configuration.

The imaging system of the present invention includes a first photoelectric conversion element and a first transfer switch that transfers the charge of the first photoelectric conversion element, and outputs a signal corresponding to the charge of the first photoelectric conversion element. A first pixel, a second photoelectric conversion element, and a second transfer switch that transfers the charge of the second photoelectric conversion element, and outputs a signal corresponding to the charge of the second photoelectric conversion element. It is arranged in a plurality of a matrix state and a second pixel, the first pixel and the second pixel and the solid-state image sensor disposed adjacent to each other provided with a same color of the color filter, the photographic material A light emitting element that projects light onto the light source , a distance calculation unit that calculates a distance to the subject based on output signals of the first pixel and the second pixel, and any one of the first mode and the second mode Switch to some mode Control means for controlling, and in the first mode, the control means drives the solid-state imaging device so as to simultaneously turn on the first transfer switch and the second transfer switch. Controlling, in the second mode, to turn on the first transfer switch, and then start projecting light by the light emitting element, and then turn off the first transfer switch, and the second transfer The switch is turned on, and then the light emitting element finishes projecting light, and then the solid-state imaging device and the light emitting element are driven to turn off the second transfer switch, and the distance calculation unit Control is performed so as to calculate the distance to the subject .

It is possible to obtain a highly accurate pixel signal for calculating the distance to the subject in easy single configuration. As a result, the distance to the subject can be calculated with high accuracy.

It is a figure which shows the structural example of a pixel. It is a figure which shows the structural example of a solid-state imaging device. It is a figure which shows the structural example of the read-out part. It is a layout diagram of a pixel. 6 is a timing chart for explaining read driving. It is a figure for demonstrating the imaging system which can produce | generate object distance information. It is a timing chart which shows the drive method in TOF method. It is a figure for demonstrating pixel output. It is a timing chart which shows the drive method in TOF method. It is a figure for demonstrating pixel output. It is a timing chart which shows the drive method in TOF method. It is a figure for demonstrating pixel output. It is a figure which shows the structural example of the read-out part.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration example of a pixel 100 according to the first embodiment of the present invention. The photodiode 101 is a photoelectric conversion element that generates charges by photoelectric conversion, and has an anode grounded. The cathode of the photodiode 101 is connected to the floating diffusion Cfd and the gate of the amplification MOS transistor 104 via a transfer MOS transistor (transfer switch) 102. The gate of the amplification MOS transistor 104 is connected to the source of the reset MOS transistor 103 for resetting the floating diffusion Cfd. The drain of the reset MOS transistor 103 is connected to the node of the power supply voltage VDD. The amplification MOS transistor 104 has a drain connected to the node of the power supply voltage VDD and a source connected to the drain of the selection MOS transistor 105.

  A signal PTX is input to the gate of the transfer MOS transistor 102. The transfer MOS transistor 102 transfers the charge of the photodiode 101 to the floating diffusion Cfd and the gate of the amplification MOS transistor 104 according to the signal PTX. A signal PRES is input to the gate of the reset MOS transistor 103. The reset MOS transistor 103 resets the charges of the floating diffusion Cfd and the photodiode 101 in response to the signal PRES. A signal PSEL is input to the gate of the selection MOS transistor 105. The selection MOS transistor 105 connects the source of the amplification MOS transistor 104 to the terminal OUT according to the signal PSEL. The terminal OUT is connected to the vertical output line 301 in FIG. The amplification MOS transistor 104 functions as a source follower amplifier by being connected to the load current source 302 of the vertical output line 301 of FIG. 3 via the selection MOS transistor 105. The signals PRES and PSEL are respectively generated by the vertical scanning unit 202 in FIG.

  FIG. 2 is a diagram illustrating a configuration example of the CMOS solid-state imaging device 200. The solid-state imaging device 200 includes a pixel unit 201, a vertical scanning unit 202, a reading unit 203, a horizontal scanning unit 204, and a reading amplifier 150. The pixel unit 201 includes a plurality of pixels 100 (FIG. 1) arranged in a two-dimensional matrix, and receives an optical image formed by the optical system. The vertical scanning unit 202 sequentially selects the rows of the pixels 100 in the pixel unit 201 based on the signals PTX, PSEL, and PTX. The horizontal scanning unit 204 sequentially selects the pixel columns in the pixel unit 201. Thereby, the plurality of pixels in the pixel unit 201 are selected in order. The readout unit 203 reads out the signal of the pixel 100 selected by the vertical scanning unit 202 and the horizontal scanning unit 204 to the readout amplifier 150. Note that the solid-state imaging device 200 may include a timing generator that provides a timing signal to each of the above circuits.

  3 is a diagram illustrating a configuration example of a part of the pixel 100 in FIG. 1 and the readout unit 203 in FIG. The pixel 100_00 is the pixel 100 in the 0th row and the 0th column. The pixel 100_01 is the pixel 100 in the 0th row and the first column. The pixel 100_02 is the pixel 100 in the 0th row and the second column. The pixel 100_03 is the pixel 100 in the 0th row and the third column. The pixel 100_10 is the pixel 100 in the first row and the 0th column. The pixel 100_11 is the pixel 100 in the first row and first column. The pixel 100_12 is the pixel 100 in the first row and the second column. The pixel 100_13 is the pixel 100 in the first row and the third column. The eight pixels 100_00 to 100_13 are arranged in a two-dimensional matrix in the pixel portion 201. Although an example of eight pixels 100_00 to 100_13 will be described, a larger number of pixels 100 are actually arranged. The output terminal OUT of the pixel 100 in each column is commonly connected to the vertical output line 301 and the current source 302 arranged for each column. The current source 302 is a load of the vertical output line 301.

  The signals PTX_00 to PTX_13 are signals PTX input to the gates of the transfer MOS transistors 102 of the pixels 100_00 to 100_13, respectively. The signal PSEL_0 is a signal PSEL input to the gates of the selection MOS transistors 105 of the pixels 100_00, 100_01, 100_02, and 100_03 in the 0th row.

  The amplification amplifier 303 amplifies the signal read out to the vertical output line 301. The holding capacitor 306 holds the output signal of the amplification amplifier 303 via the switch 304. The holding capacitor 307 holds the output signal of the amplification amplifier 303 via the switch 305. The switch 304 is driven by the signal PTN in FIG. 5, and the holding capacitor 306 holds a signal (N signal) in the reset state of the pixel 100. Further, the switch 305 is driven by the signal PTS in FIG. 5, and the holding capacitor 307 holds a signal (S signal) based on the photoelectric conversion of the pixel 100. The signals in the columns of the storage capacitors 306 and 307 are sequentially transferred to the input terminal of the read amplifier 150 by the horizontal scanning unit 204. The read amplifier 150 outputs the difference between the S signal of the storage capacitor 307 and the N signal of the storage capacitor 306.

  FIG. 4 is a layout diagram of the pixels 100_00 to 100_13. Each of the pixels 100_00 to 100_13 includes a photodiode 101, a transfer MOS transistor 102, and a floating diffusion Cfd in a semiconductor layer, and a microlens 401 is disposed immediately above the photodiode 101. A color filter layer is arranged on the photodiode 101 in a Bayer array. A red (R) color filter is disposed in each of the pixels 100_00 and 100_02. A green (Gr) color filter is disposed in each of the pixels 100_01 and 100_03. A green (Gb) color filter is disposed in each of the pixels 100_10 and 100_12. A blue (B) color filter is disposed in each of the pixels 100_11 and 100_13.

  FIG. 5 is a timing chart showing a driving method in the first mode of the solid-state imaging device 200, and shows a driving method for reading out signals stored in the photodiode 101 for one screen in order to image a subject. The horizontal axis indicates the passage of time, and the time is indicated by T1 to T13. As the signal HD changes from the low level to the high level, the pixel selection row by the vertical scanning unit 202 is switched. The phase of the signal HD indicates the phase of the signal PSEL in the selected row. Each pulse signal turns on the corresponding transistor at a high level. Further, the transfer MOS transistors 102 in the same row of the 0th row are driven by signals PTX_00 and PTX_01 and signals PTX_02 and PTX_03 every two columns. The transfer MOS transistors 102 in the same row of the first row are driven by signals PTX_10, PTX_11 and signals PTX_12, PTX_13 every two columns. Prior to time T1, the photodiode 101 accumulates charges generated by photoelectric conversion.

  At time T1, the signals HD, PTS, and PTN are at a low level, and the signal PRES is at a high level. Next, at time T2, the signal HD (PSEL) changes from the low level to the high level, so that the vertical scanning unit 202 selects the 0th row. Specifically, the signal PSEL_0 in the 0th row changes from the low level to the high level, the selection MOS transistor 105 in the 0th row is turned on, and the amplification MOS transistor 104 in the 0th row is connected to the vertical output line 301. At this time, the signal PRES is at a high level, the reset MOS transistors 103 of all the pixels 100 are turned on, and the floating diffusion Cfd is in a reset state.

  Next, at times T3 to T4, the signals PTS and PTN are at a high level, the switches 304 and 305 are turned on, and the storage capacitors 306 and 307 are reset.

  Next, at time T5, the signal PRES goes low, the reset MOS transistors 103 of all the pixels 100 are turned off, and the reset state of the floating diffusion Cfd is released.

  Next, at times T <b> 6 to T <b> 7, the signal PTN becomes high level again, the switch 304 is turned on, and the output signal (N signal) after the reset state of the pixel 100 is released is held in the holding capacitor 306.

  Next, at times T8 to T9, the signals PTX_00 and PTX_01 and the signals PTX_02 and PTX_03 are at a high level. Thereby, in all the pixels 100_00 to 100_03 in the 0th row, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd. In addition, at time T8 to T10, the signal PTS becomes high level again, the switch 305 is turned on, and an output signal (S signal) based on photoelectric conversion of the pixels 100_00 to 100_03 in the 0th row is held in the holding capacitor 307. .

  Next, during the period from time T <b> 10 to T <b> 11, the signals held in the holding capacitors 306 and 307 are sequentially transferred to the input terminal of the read amplifier 150 by the horizontal scanning unit 204. The read amplifier 150 outputs the difference between the S signal of the storage capacitor 307 and the N signal of the storage capacitor 306.

  Next, at time T11, the signal PRES goes high, the reset MOS transistors 103 of all the pixels 100 are turned on, and the floating diffusion Cfd is reset.

  At times T11 to T12, the signals HD and PSEL_0 are at a low level, the 0th row selection MOS transistor 105 is turned off, and the 0th row amplification MOS transistor 104 is disconnected from the vertical output line 301.

  Next, at time T12, the signals HD and PSEL_1 change from low level to high level, the first row selection MOS transistors 105 are turned on, and the first row amplification MOS transistors 104 are disconnected from the vertical output line 301. Similarly to the times T2 to T12, the pixels 100_10 to 100_13 in the first row are read. At this time, instead of the signals PTX_00 to PTX_03 in the 0th row, the signals PTX_10 to PTX_13 in the 1st row become high level, and the transfer MOS transistors 102 of the pixels 100_10 to 100_13 in the 1st row are turned on. The above processing is sequentially performed for all rows, and signals of all pixels 100 for one screen can be read. Thereby, the pixel signal for one screen for imaging a subject is obtained.

  FIG. 6 is a diagram illustrating a configuration example of an imaging system (camera) that can generate subject distance information (distance image) by the TOF (Time-Of-Flight) method using the solid-state imaging device 200 in the second mode. is there. The photographing lens 601 collects light from the object scene onto the solid-state imaging device 200. The light emitting element (light projecting unit) 602 projects (irradiates) pulsed light of “R color”, “G color”, and “B color” to the object scene. The timing generation circuit 603 drives the solid-state imaging device 200 and the light emitting element 602. The light emitting element 602 projects “R color” light onto the object scene when the signal LED_R_PLS becomes a high level. In addition, the light emitting element 602 projects “G color” light onto the object scene when the signal LED_G_PLS becomes a high level. In addition, the light emitting element 602 projects “B color” light onto the object scene when the signal LED_B_PLS becomes a high level. The distance calculation unit 604 measures distance information (distance image) from the imaging system to the subject by measuring the time until the light projected from the light emitting element 602 is reflected by the subject and received by the solid-state imaging device 200. calculate. The light emitting element 602 can selectively project red, green, and blue light. The color filters of the plurality of pixels 100_00 to 100_13 in FIG. 4 include color filters having substantially the same color as red, green, and blue light projected by the light-emitting element 602.

  FIG. 7 is a timing chart showing a driving method in the second mode of the solid-state imaging device 200, and shows a driving method for generating subject distance information for one screen. In this timing chart, the light emitting element 602 projects light of “R color” and the reflected light is received by the pixel 100 having the filter of “R color”, or the light emitting element 602 receives light of “B color”. The projected light is received by a pixel 100 having a “B color” filter. Thus, the distance to the subject for one screen is measured.

  First, at time T1, the signals HD, PTS, and PTN are at a low level, and the signal PRES is at a high level. Next, at time T <b> 2, the signals HD and PSEL_ 0 are at a high level, the 0th row selection MOS transistor 105 is turned on, and the 0th row amplification MOS transistor 104 is connected to the vertical output line 301. At this time, the signal PRES is at a high level, the reset MOS transistors 103 of all the pixels 100 are turned on, and the floating diffusion Cfd is in a reset state. At times T2 to T3, the signals PTX_00 and PTX_01 and the signals PTX_02 and PTX_03 are at a high level, and the transfer MOS transistor 102 is turned on in all the pixels 100_00 to 100_03 in the 0th row, and the charge of the photodiode 101 is reset. The

  Next, at times T4 to T5, the signals PTS and PTN are at a high level, the switches 304 and 305 are turned on, and the storage capacitor 306 and the storage capacitor 307 are reset.

  Next, at time T6, the signal PRES goes low, the reset MOS transistors 103 of all the pixels 100 are turned off, and the reset state of the floating diffusion Cfd is released.

  Next, at time T <b> 7 to T <b> 8, the signal PTN goes high again, the switch 304 is turned on, and the output signal (N signal) after the reset state of the pixel 100 is released is held in the holding capacitor 306.

  Next, in the period from time T9 to T11, the signals PTX_00 and PTX_01 are at a high level. Thereby, in the pixels 100_00 and 100_01, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  Next, in the period of time T11 to T13, the signals PTX_02 and PTX_03 are at a high level. Thereby, in the pixels 100_02 and 100_03, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  In the period from time T10 to time T12, the signal LED_R_PLS is at a high level, and the light emitting element 602 projects “R color” light toward the object scene.

  After that, at time T14 to T15, the signal PTS becomes high level again, the switch 305 is turned on, and the output signal (S signal) based on the photoelectric conversion of the pixels 100_00 to 100_03 in the 0th row is held in the holding capacitor 307. .

  Next, during the period from time T15 to time T16, the signals held in the holding capacitors 306 and 307 are sequentially transferred to the input terminal of the read amplifier 150 by the horizontal scanning unit 204. The read amplifier 150 outputs the difference between the S signal of the storage capacitor 307 and the N signal of the storage capacitor 306.

  Here, the pixels in a part of time T9 to T11 in the period in which the light emitting element 602 projects “R color” light toward the object field and the reflected light is received by the solid-state imaging device 200. Photodiodes 101 of 100_00 and 100_01 receive light. In addition, the photodiodes 101 of the pixels 100_02 and 100_03 receive light during a part of the period from time T11 to time T13. As shown in FIG. 4, the pixels 100_00 and 100_02 have “R color” color filters arranged on the photodiode 101. Therefore, when the projection light is “R color”, the signal intensity is very high. Become. On the other hand, in the pixels 100_01 and 100_03, since the “G color” color filter is arranged on the photodiode 101, the photoelectric conversion efficiency is low.

  That is, a predetermined calculation is performed on the output signals of the photodiodes 101 of the pixels 100_00 and 100_02 among the differential signals of the S signal and the N signal read from the pixels 100_00 to 100_03 in the 0th row. Accordingly, the subject distance can be calculated by efficiently using the reflected light of the light projected in “R color”.

  A method for generating subject distance information by the TOF (Time-Of-Flight) method will be described. During the time T9 to T13, when light is projected onto the object scene and the reflected light is received, the charges of the photodiodes 101 of the two pixels 100_00 and 100_02 are taken out in a time-sharing manner. The distance calculation unit 604 (FIG. 6) in the imaging system can calculate the distance from the imaging system to the subject by comparing the output signals of the two pixels 100_00 and 100_02.

  Next, at time T16, the signal PRES goes high, the reset MOS transistors 103 of all the pixels 100 are turned on, and the floating diffusion Cfd is reset.

  At times T16 to T17, the signals HD and PSEL_0 are at a low level, the 0th row selection MOS transistor 105 is turned off, and the 0th row amplification MOS transistor 104 is disconnected from the vertical output line 301.

  Next, at time T17, the signals HD and PSEL_1 change from the low level to the high level, the selection MOS transistor 105 in the first row is turned on, and the amplification MOS transistor 104 in the first row is connected to the vertical output line 301.

  At time T17, the signals PTX_10 and PTX_11 and the signals PTX_12 and PTX_13 are at a high level, the transfer MOS transistors 102 are turned on in all the pixels 100_10 to 100_13 in the first row, and the charge of the photodiode 101 is reset.

  Next, as in the period of time T4 to T5, the signals PTS and PTN are at a high level. Next, as at time T6, the signal PRES goes low. Next, as in the period of time T7 to T8, the signal PTN goes high. Next, at time T18, the signal PTN goes low.

  Next, during the period of time T19 to T21, the signals PTX_10 and PTX_11 are at a high level. As a result, in the pixels 100_10 and 100_11, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  Next, during the period of time T21 to T23, the signals PTX_12 and PTX_13 are at a high level. Thereby, in the pixels 100_12 and 100_13, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  In the period from time T20 to time T22, the signal LED_B_PLS is at a high level, and the light emitting element 602 projects “B color” light toward the object scene.

  After that, at time T <b> 24, the signal PTS becomes high level again, the switch 305 is turned on, and the output signal (S signal) based on the photoelectric conversion of the pixels 100_10 to 100_13 in the first row is held in the holding capacitor 307. Next, the signals held in the holding capacitors 306 and 307 are sequentially transferred to the input terminal of the read amplifier 150 by the horizontal scanning unit 204. The read amplifier 150 outputs the difference between the S signal of the storage capacitor 307 and the N signal of the storage capacitor 306.

  Here, the pixels in a part of time T19 to T21 in the period in which the light emitting element 602 projects light of “B color” toward the object field and the reflected light is received by the solid-state imaging device 200. Photodiodes 101 of 100_10 and 100_11 receive light. In addition, the photodiodes 101 of the pixels 100_12 and 100_13 receive light during a part of the period from time T21 to T23. As shown in FIG. 4, the pixels 100_11 and 100_13 have the “B color” color filter disposed on the photodiode 101. Therefore, when the projection light is “B color”, the signal intensity is very high. Become. On the other hand, since the “G color” color filter is arranged on the photodiode 101 in the pixels 100_10 and 100_12, the efficiency of photoelectric conversion is poor.

  That is, a predetermined calculation is performed on the output signals of the photodiodes 101 of the pixels 100_11 and 100_13 among the differential signals of the S signal and the N signal read from the pixels 100_10 to 100_13 in the first row. Accordingly, the subject distance can be calculated by efficiently using the reflected light of the light projected with “B color”.

  After finishing the processing of the first row, the processing of the second row is performed in the same manner as the processing of the zeroth row. Thereafter, the third line process is performed in the same manner as the first line process. The even-numbered row processing and the odd-numbered row processing are sequentially performed for all rows, and all pixels 100 for one screen are read. Subject distance information for one screen can be obtained.

  FIGS. 8A and 8B are diagrams showing an image of the output signal of the pixel when driven by the timing chart of FIG. 7, and the photodiode 101 of the two pixels 100 shown by hatching for each row. The subject distance is calculated using this signal. For example, in FIG. 8A, the photodiode 101 of the “R color” color filter used when the “R color” light is projected is indicated by hatching. In FIG. 8B, the photodiode 101 of the “B color” color filter used when the “B color” light is projected is hatched. That is, the output signal of the pixels 100 in two columns of the same color in four columns is used in one of the two rows. In this way, one distance information (distance image) can be obtained for each 1 × 4 pixel in one screen.

  FIG. 9 corresponds to FIG. 7 and is a timing chart showing a driving method in the second mode of the solid-state imaging device 200, and shows a driving method for generating subject distance information for one screen. This timing chart shows a case where the light emitting element 602 measures the distance to the subject by projecting “G color” light and receiving the reflected light by the pixel 100 having the “G color” filter. The difference between FIG. 9 and FIG. 7 is that the signal LED_B_PLS is at a high level in the period from time T10 to time T12 and in the period from time T20 to time T22, and the light emitting element 602 emits “G color” light in the object scene. It is the point which is projecting toward. Hereinafter, the points of FIG. 9 different from FIG. 7 will be described.

  In the period from time T10 to time T12, the light emitting element 602 projects “G color” light toward the object scene, and the reflected light is received by the solid-state imaging device 200. In some periods, the photodiodes 101 of the pixels 100_00 and 100_01 receive light. In addition, the photodiodes 100 of the pixels 100_02 and 100_03 receive light during a part of the period from time T11 to time T13. As shown in FIG. 4, the pixels 100_01 and 100_03 have a “G color” color filter disposed on the photodiode 101. Therefore, when the projected light is “G color”, the signal intensity is very high. Become. On the other hand, since the “R color” color filter is arranged on the photodiode 101 in the pixels 100_00 and 100_02, the photoelectric conversion efficiency is low.

  That is, a predetermined calculation is performed on the output signals of the pixels 100_01 and 100_03 among the differential signals of the S and N signals read from the pixels 100_00 to 100_03 in the 0th row. Accordingly, the subject distance can be calculated by efficiently using the reflected light of the light projected in “G color”.

  Similarly, during the period from time T20 to time T22, “G color” light is projected toward the object scene, and a part of the time T19 to T21 in the period in which the reflected light is received by the solid-state imaging device 200. During this period, the photodiodes 101 of the pixels 100_10 and 100_11 receive light. In addition, the photodiodes 101 of the pixels 100_12 and 100_13 receive light during a part of the period from time T21 to T23. As shown in FIG. 4, the pixels 100_10 and 100_12 have the “G color” color filter disposed on the photodiode 101. Therefore, when the projection light is “G color”, the signal intensity is very high. Become. On the other hand, in the pixels 100_11 and 100_13, since the “B color” color filter is disposed on the photodiode 101, the photoelectric conversion efficiency is low.

  That is, a predetermined calculation is performed on the output signals of the pixels 100_10 and 100_12 among the differential signals of the S signal and the N signal read from the pixels 100_10 to 100_13 in the first row. Accordingly, the subject distance can be calculated by efficiently using the reflected light of the light projected in “G color”.

  After finishing the processing of the first row, the processing of the second row is performed in the same manner as the processing of the zeroth row. Thereafter, the third line process is performed in the same manner as the first line process. The even-numbered row processing and the odd-numbered row processing are sequentially performed for all rows, and all pixels 100 for one screen are read.

  FIG. 10 is a diagram showing an image of pixel output when driven by the timing chart of FIG. 9. When the light emitting element 602 projects “G color” light, two rows shown by hatching for each row are shown. The subject distance is calculated using the signal of the photodiode of the pixel. That is, the output signals of the pixels 100 in two columns of the same color in four columns in each row are used. Thus, one distance information (distance image) can be obtained for every 1 × 4 pixels in one screen.

  As described above, according to the first embodiment, a pixel signal for one screen for capturing an image of a subject can be obtained, and reflected light can be used efficiently according to the color of the projected light. One distance information (distance image) can be obtained for every 1 × 4 pixels on the screen.

  Next, an application example of the first embodiment will be described. In an imaging system using the solid-state imaging device 200, the photographing lens 601 is normally driven while generating focus detection information by phase difference detection with a passive dedicated sensor, and the subject is focused, and the timing chart of FIG. The subject is imaged by driving with.

  When the surrounding environment is dark and it is difficult to detect the focus by phase difference detection using a passive type dedicated sensor, distance information (distance image) is acquired by the TOF method described with reference to FIGS. 6 to 10 and the photographing lens 601 is driven. To focus on the subject. At this time, the subject is imaged by driving according to the timing chart of FIG.

  Furthermore, when acquiring distance information (distance image) by the TOF method, the distance information can be efficiently acquired by changing the projection light color according to the color of the subject and driving with the corresponding driving method. Can do. By performing such an operation, it is possible to widen the range of brightness that can be focused.

(Second Embodiment)
FIG. 13 corresponds to FIG. 3 and is a diagram illustrating a configuration example of a part of the pixel 100 and the readout unit 203 according to the second embodiment of the present invention. Here, 2 × 2 pixels 100_00, 100_01, 100_10, and 100_11 will be described as an example. In practice, more pixels 100 are arranged. Hereinafter, the points of FIG. 13 different from FIG. 3 will be described. Two vertical output lines 301a and 301b are provided for the pixels 100_00 and 100_10 in the 0th column, the pixels 100_10 in the odd rows are connected to the vertical output lines 301a, and the pixels 100_00 in the even rows are connected to the vertical output lines 301b. Is done. Similarly, two vertical output lines 301a and 301b are provided for the pixels 100_01 and 100_11 in the first column, the pixels 100_11 in the odd-numbered rows are connected to the vertical output lines 301a, and the pixels 100_01 in the even-numbered rows are connected to the vertical output lines. 301b. As in FIG. 3, the vertical output lines 301 a and 301 b are each connected to the amplification amplifier 303. The storage capacitor 306 is connected to the output terminal of the amplification amplifier 303 via the switch 304. The storage capacitor 307 is connected to the output terminal of the amplification amplifier 303 via the switch 305. That is, in this embodiment, the output signals of the pixels 100 for two rows can be obtained by the operation for one row in the timing charts of FIGS.

  FIG. 11 corresponds to FIG. 9 and is a timing chart showing a driving method in the second mode of the solid-state imaging device 200, and shows a driving method for generating subject distance information for one screen. This timing chart shows a case where the light emitting element 602 measures the distance to the subject by projecting “G color” light and receiving the reflected light by the pixel 100 having the “G color” filter.

  Hereinafter, the points of FIG. 11 different from FIG. 9 will be described. The pixels 100 for two rows are read from time T2 to time T17, and the transfer MOS transistors 102 in the same row are driven at the same timing.

  At times T2 to T3, the signals PTX_00 to PTX_13 are at a high level, the transfer MOS transistors 102 are turned on in all the pixels 100_00 to 100_13, and the charge of the photodiode 101 is reset. At times T2 to T16, the signals PSEL_0 and PSEL_1 are at a high level, and the selection MOS transistor 105 is turned on in the pixels 100_00 to 100_02 in the 0th row and the pixels 100_10 to 100_13 in the 1st row.

  At times T7 to T8, the signal PTN of the 0th row and the 1st row becomes a high level, and the output signal (N signal) after the reset state of the pixels 100_00 to 100_13 of the 0th row and the 1st row is released is the storage capacitor 306. Hold on.

  In the period from time T9 to T11, the signals PTX_00 and PTX_01 and the signals PTX_10 and PTX_11 are at a high level. Thereby, in the pixels 100_00, 100_01, 100_10, and 100_11, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  In the period from time T11 to T13, the signals PTX_02 and PTX_03 and the signals PTX_12 and PTX_13 are at a high level. Thereby, in the pixels 100_02, 100_03, 100_12, and 100_13, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  In the period from time T10 to time T12, the signal LED_G_PLS is at a high level, and the light emitting element 602 projects “G color” light toward the object scene.

  After that, at time T14 to T15, the signal PTS becomes a high level, and an output signal (S signal) based on photoelectric conversion of the pixels 100_00 to 100_13 in the 0th row and the 1st row is held in the holding capacitor 307.

  Thereafter, during the period from time T15 to time T16, the signals of the storage capacitors 306 and 307 are sequentially transferred to the input terminal of the read amplifier 150 by the horizontal scanning unit 204. The read amplifier 150 outputs a difference between the S signal and the N signal of the 0th and 1st rows.

  After finishing the processing of the 0th row and the 1st row, after the time T17, the processing of the 2nd row and the 3rd row is performed similarly to the processing of the 0th row and the 1st row.

  At time T17, the signals PTX_00 to PTX_13 become high level, the transfer MOS transistor 102 is turned on in all the pixels, and the charge of the photodiode 101 is reset. At time T17, the signals PSEL_2 and PSEL_3 are at a high level, and the selection MOS transistor 105 is turned on in the second row of pixels 100 and the third row of pixels.

  Prior to time T18, the signals PTN in the second and third rows are at a high level, and the output signal (N signal) after the reset state of the pixels in the second and third rows is released is held in the holding capacitor 306. .

  In the period from time T19 to T21, the signals PTX_20 and PTX_21 and the signals PTX_30 and PTX_31 are at a high level. Thereby, in the pixels 100_20, 100_21, 100_30, and 100_31, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  During the period from time T21 to T23, the signals PTX_22 and PTX_23 and the signals PTX_32 and PTX_33 are at a high level. Thereby, in the pixels 100_22, 100_23, 100_32, and 100_33, the transfer MOS transistor 102 is turned on, and the charge of the photodiode 101 is transferred to the floating diffusion Cfd.

  In the period from time T20 to T22, the signal LED_G_PLS is at a high level, and the light emitting element 602 projects “G color” light toward the object scene.

  Thereafter, at time T <b> 24, the signal PTS goes high, and the output signal (S signal) based on the photoelectric conversion of the pixels 100_20 to 100_33 in the second row and the third row is held in the holding capacitor 307.

  Thereafter, the signals of the storage capacitors 306 and 307 are sequentially transferred to the input terminal of the read amplifier 150 by the horizontal scanning unit 204. The read amplifier 150 outputs the difference between the S and N signals in the second and third rows.

  The above processing is sequentially performed for all rows in units of two rows, and signals of all pixels 100 for one screen are read out.

  FIG. 12 is a diagram illustrating an image of an output signal of a pixel when driven by the timing chart of FIG. 11. When the light emitting element 602 projects “G color” light, 2 shown by hatching for each row. The subject distance is calculated using the signal of the photodiode 101 of each pixel 100. That is, the output of two pixels of the same color in 2 rows and 2 columns is used. In this way, one distance information (distance image) can be obtained for every 2 × 2 pixels in one screen.

  As described above, according to the second embodiment, a pixel signal for one screen for imaging a subject can be obtained, and reflected light can be used efficiently according to the color of the projected light. One distance information (distance image) can be obtained for every 2 × 2 pixels in one screen. Note that an application example of the first embodiment can also be applied to the present embodiment.

  According to the first and second embodiments, in the second mode (FIG. 7 and the like), the transfer MOS transistor 102 is turned on at a timing that overlaps with the period during which the light emitting element 602 projects light. The on / off state varies depending on the color of light projected by the light emitting element 602. In the first mode, the transfer MOS transistor 102 has the same on / off state in the same row, and in the second mode, the on / off state differs depending on the column even in the same row.

  The matrix photodiode 101 includes a first photodiode 101 and a second photodiode 101 provided with a color filter of the same color adjacent to the first photodiode 101. The matrix-shaped transfer MOS transistor 102 includes a first transfer MOS transistor 102 that transfers the charge of the first photodiode 101 and a second transfer MOS transistor 102 that transfers the charge of the second photodiode 101. In the second mode, the first transfer MOS transistor 102 is turned on, and then the light emitting element 602 starts projecting light, and then the first transfer MOS transistor 102 is turned off and the second transfer MOS transistor 102 is turned on. Turns on. Thereafter, the light emitting element 602 finishes projecting light, and then the second transfer MOS transistor 102 is turned off. The distance calculation unit 604 calculates the distance to the subject according to the charge of the first photodiode 101 and the charge of the second photodiode 101. In the first mode (FIG. 5), the first transfer MOS transistor 102 and the second transfer MOS transistor 102 are turned on simultaneously.

  In the first embodiment (FIG. 3), a plurality of vertical output lines 301 are provided corresponding to each column of the plurality of photodiodes 101. A signal corresponding to the charge of the photodiode 101 in each column among the plurality of photodiodes 101 can be output to the same vertical output line 301. A signal corresponding to the charges of the plurality of photodiodes 101 is output to the vertical output line 301 in units of one row of the plurality of photodiodes 101.

  In the second embodiment (FIG. 13), the plurality of vertical output lines 301a and 301b correspond to each column of the plurality of photodiodes 101, and are provided separately in odd and even rows. Signals corresponding to the charges of the plurality of photodiodes 101 are output to the vertical output lines 301a and 301b in units of two rows of the plurality of photodiodes 101.

  According to the first and second embodiments, a pixel signal for one screen for obtaining a captured image of a subject can be obtained, and a distance calculation is performed by efficiently using reflected light according to the color of the projected light. A distance image with improved accuracy can be obtained. In addition, since the photographing lens 601 can be driven and focused on the subject based on distance information (distance image) acquired by the TOF method, it is possible to widen the range of brightness that can be focused.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

101 photodiode, 102, 103, 104, 105 MOS transistor, 602 light emitting element

Claims (6)

A first pixel that includes a first transfer switch that transfers the charge of the first photoelectric conversion element and the first photoelectric conversion element, and that outputs a signal corresponding to the charge of the first photoelectric conversion element; A plurality of second photoelectric conversion elements and a second transfer switch that transfers a charge of the second photoelectric conversion element, and a second pixel that outputs a signal corresponding to the charge of the second photoelectric conversion element. Are arranged in a matrix, and the first pixel and the second pixel include color filters of the same color and are disposed adjacent to each other;
A light emitting element for projecting light on the subject,
A distance calculation unit that calculates a distance to the subject based on output signals of the first pixel and the second pixel;
Control means for controlling to switch to one of the first mode and the second mode,
The control means controls to drive the solid-state imaging device so as to simultaneously turn on the first transfer switch and the second transfer switch in the first mode, and in the second mode, Turn on the first transfer switch, then start projecting light by the light emitting element, then turn off the first transfer switch, turn on the second transfer switch, and then turn on the light emitting element Finishes projecting light, and then drives the solid-state imaging device and the light emitting device to turn off the second transfer switch, and controls the distance calculation unit to calculate the distance to the subject. An imaging system characterized by: Furthermore, it has a plurality of output lines provided corresponding to each column of the plurality of pixels ,
The signals of the pixels in each column of the plurality of pixels can be output to the same output line,
Signals of the plurality of pixels, one line at a time of the plurality of pixels, the imaging system according to claim 1, wherein the output to the output line. Furthermore, each of the plurality of pixels has a plurality of output lines separately provided in odd rows and even rows, corresponding to each column,
Signals of the plurality of pixels, in units of two rows of the plurality of pixels, the imaging system according to claim 1, wherein the output to the output line. The light emitting element can selectively project red, green and blue light,
Wherein the first pixel and the second color filter having pixels comprising the any one of claims 1 to 3, wherein the light emitting element is characterized in that substantially the same color and any color of light to be projected The imaging system described in 1. 5. The on / off state of the first transfer switch and the second transfer switch differs according to the color of a color filter included in the first pixel and the second pixel. The imaging system described. 6. The imaging system according to claim 4 , wherein the plurality of pixels are arranged so that colors of color filters provided in each of the plurality of pixels are arranged in a Bayer array.

Images