JP7142658B2 - Imaging device - Google Patents

Description

The present invention relates to an imaging device.

A so-called rolling shutter system is known for an imaging device having a plurality of pixels arranged in a matrix. In the rolling shutter method, an operation of selecting pixels in the same row and accumulating and reading out pixel signals is sequentially repeated for each row. Furthermore, it is known that a global shutter is used in place of the rolling shutter method to prevent rolling distortion that occurs when an image of a moving object is captured (see, for example, Patent Document 1).
Patent Document 1 Republished 2010/023903

However, the global shutter has a problem that the wiring layout becomes complicated.

A first aspect of the present invention is an imaging device, comprising: an imaging device in which unit blocks each having a plurality of pixels arranged side by side in the row direction and the column direction are respectively arranged in the row direction and the column direction; , in order to sequentially read out signals in the column direction for each pixel arranged in the row direction in each of the first unit block and the second unit block arranged side by side in at least the column direction among the plurality of unit blocks. and a correlation calculator for calculating an evaluation value indicating the correlation between signals read from the first pixel of the first unit block and the second pixel of the second unit block, which are arranged adjacent to each other in the column direction. and the evaluation value calculated by the correlation calculation unit, the correlation is increased for one of the signals read from the pixels of the first unit block and the pixels of the second unit block. and a correcting unit for correcting.

It should be noted that the above summary of the invention does not list all the necessary features of the invention. Subcombinations of these feature groups can also be inventions.

1 is a cross-sectional view of a back-illuminated imaging device according to an embodiment; FIG. It is a figure explaining the pixel arrangement|sequence of an imaging chip, and a unit block. It is a circuit diagram corresponding to a pixel. A unit block, its peripheral circuits, and an outline of their connection relationship are shown. 1 is a block diagram showing the configuration of an imaging device according to this embodiment; FIG. 3 is a block diagram showing a specific configuration of a driving section; FIG. 3 shows functional blocks of an arithmetic circuit; FIG. 4 shows a timing chart of operations such as charge accumulation and readout in each row of a unit block; FIG. An example of a subject image incident on an imaging device is shown. 4 shows a captured image before correction. 4 is a flow chart showing the operation of an arithmetic circuit; 4 shows a captured image after correction.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution of the invention.

FIG. 1 is a cross-sectional view of a back-illuminated imaging device 100 according to this embodiment. The imaging device 100 includes an imaging chip 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes pixel signals, and a memory chip 112 that stores pixel signals. These imaging chip 113, signal processing chip 111 and memory chip 112 are stacked and electrically connected to each other by conductive bumps 109 such as Cu.

Incidentally, as shown in the figure, the incident light is mainly incident in the Z-axis plus direction indicated by the white arrow. In the present embodiment, the surface of the imaging chip 113 on which incident light is incident is referred to as the rear surface. Also, as shown in the coordinate axes, the right direction on the paper surface perpendicular to the Z-axis is the positive X-axis direction, and the frontward direction on the paper surface perpendicular to the Z-axis and the X-axis is the positive Y-axis direction. In the following several figures, the coordinate axes are displayed with reference to the coordinate axes of FIG. 1 so that the direction of each figure can be understood.

An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer is arranged on the back side of the wiring layer 108 . The PD layer 106 has a plurality of PDs (photodiodes) 104 arranged two-dimensionally and transistors 105 provided corresponding to the PDs 104 .

A color filter 102 is provided on the incident light side of the PD layer 106 with a passivation film 103 interposed therebetween. The color filters 102 have a plurality of types that transmit different wavelength regions, and have specific arrangements corresponding to the respective PDs 104 . The arrangement of the color filters 102 will be described later. A set of color filter 102, PD 104 and transistor 105 forms one pixel.

A microlens 101 is provided on the incident light side of the color filter 102 so as to correspond to each pixel. The microlens 101 collects incident light toward the corresponding PD 104 .

The wiring layer 108 has wiring 107 for transmitting pixel signals from the PD layer 106 to the signal processing chip 111 . The wiring 107 may be multi-layered and may be provided with passive and active elements.

A plurality of bumps 109 are arranged on the surface of the wiring layer 108 . The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surface of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined together and electrically connected.

Similarly, a plurality of bumps 109 are arranged on surfaces of the signal processing chip 111 and the memory chip 112 facing each other. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are bonded and electrically connected.

The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and may be micro bump bonding by solder melting. Also, about one bump 109 may be provided for one output wiring described later, for example. Therefore, the size of the bumps 109 may be larger than the pitch of the PDs 104 . Also, bumps larger than the bumps 109 corresponding to the pixel regions may be provided in peripheral regions other than the pixel regions where the pixels are arranged.

The signal processing chip 111 has TSVs (through silicon vias) 110 that connect circuits provided on the front and rear surfaces thereof. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112 .

FIG. 2 is a diagram for explaining the pixel array of the imaging chip 113 and the unit block 131. As shown in FIG. In particular, a state of observing the imaging chip 113 from the back side is shown. The imaging chip 113 has an imaging section in which more than 20 million pixels are arranged in a matrix. In the example of FIG. 2, 16 pixels of adjacent 4×4 pixels form one unit block 131 . The grid lines in the figure illustrate the concept of grouping adjacent pixels to form a unit block 131 .

As shown in the partial enlarged view of the imaging unit, the unit block 131 includes four so-called Bayer arrays each including four pixels of green pixels Gb and Gr, blue pixels B and red pixels R, vertically and horizontally. The green pixels Gb and Gr each have a green filter as the color filter 102 and receive light in the green wavelength band among incident light. Similarly, the blue pixel B has a blue filter as the color filter 102 and receives light in the blue wavelength band, and the red pixel R has a red filter as the color filter 102 and receives light in the red wavelength band. .

In FIG. 2, for the purpose of simplification of explanation, an example in which the unit block 131 consists of 16 pixels of 4×4 pixels has been explained. The number of rows and the number of columns are not particularly limited, but when the total number of pixels of the imaging unit is about 20 million pixels, the number is, for example, 64 rows and 32 columns. Although there is no limit to the number of rows and the number of columns of the unit blocks 131 included in the imaging unit, for example, they are arranged in 48 rows and 114 columns.

FIG. 3 is a circuit diagram corresponding to the pixel 150. As shown in FIG. In FIG. 3 , a rectangle encircled by dotted lines typically represents a circuit corresponding to one pixel 150 . Note that at least part of each transistor described below corresponds to the transistor 105 in FIG.

PD 104 is connected to transfer transistor 154 . A gate of the transfer transistor 154 is connected to a wiring Tx_j to which a transfer pulse is supplied. A subscript j is a serial number within the unit block 131 that identifies the row number within the unit block 131 .

The drain of transfer transistor 154 is connected to the source of reset transistor 152 . As a result, a so-called FD (Floating Diffusion) 156 is formed between the drain of the transfer transistor 154 and the source of the reset transistor 152 . The drain of the reset transistor 152 is connected to the wiring Vdd to which the power supply voltage is supplied, and the gate is connected to the wiring Rst_j to which the reset pulse is supplied.

One end of FD 156 is further connected to the gate of amplification transistor 162 . A drain of the amplification transistor 162 is connected to a wiring Vdd to which a power supply voltage is supplied. The source of amplification transistor 162 is connected to the drain of corresponding selection transistor 164 . A gate of the selection transistor 164 is connected to a wiring Sel_j to which a selection pulse is supplied.

The source of select transistor 164 is connected to column transmission line 170 . Load current source 166 supplies current to column transmission line 170 . That is, the column transmission line 170 for the selection transistor 164 is formed by a source follower.

FIG. 4 schematically shows the unit block 131, its peripheral circuit 133, and their connection relationship. In the unit block 131 of FIG. 4, a total of (P×L) pixels 150 are arranged in L rows and P columns.

A wiring Rst_l (where l is an integer from 1 to L) is connected to the row control unit 200 and is commonly connected to the P pixels 150 in the l-th row in the unit block 131 . Similarly, the wiring Tx_l and the wiring Sel_l are also connected to the row control unit 200 and are commonly connected to the P pixels 150 in the l-th row in the unit block 131 .

The row control unit 200 is also called a row selection unit, a vertical scanning circuit, or the like. A row control unit 200 is provided for each unit block 131 . The row control unit 200 may be provided on the signal processing chip 111 side.

A column transmission line 170 is provided for each pixel 150 in the same column. These column transmission lines 170_p (where p is an integer from 1 to P) are commonly connected to the L pixels 150 of the p-th column in the unit block 131 . Thus, the column transmission line 170 is shared by the pixels 150 in the same column within the unit block 131, and transmits signals from the pixels 150 included in the column.

These column transmission lines 170_p are connected from the imaging chip 113 side to the peripheral circuit 133 provided on the signal processing chip 111 side via the bumps 109 . The peripheral circuit 133 is provided for each unit block 131 and is arranged so as to overlap the unit block 131 in the imaging chip 113 when viewed from the stacking direction.

Peripheral circuit 133 has CDS circuit 202 and A/D conversion circuit 204 connected in series for each column transmission line 170_p. In the example shown in FIG. 4, P sets of CDS circuits 202 and A/D conversion circuits 204 are provided per unit block 131 . The CDS circuit 202 removes noise from the pixel signal. The A/D conversion circuit 204 converts the pixel signal from which noise has been removed by the CDS circuit 202 into a digital signal.

The peripheral circuit 133 further has a shift register 206 arranged on the output side of the P A/D conversion circuits 204 . In the example of FIG. 4, one shift register 206 is arranged for each unit block 131 . The output of shift register 206 is connected to pixel memory 414 via column bus line 172 . The shift register 206 may also be called a horizontal scanning circuit, multiplexer, or the like.

FIG. 5 is a block diagram showing the configuration of the imaging device according to this embodiment. The image pickup apparatus 500 includes an image pickup lens 520 as an image pickup optical system, and the image pickup lens 520 guides subject light beams incident along the optical axis OA to the image pickup device 100 . The photographing lens 520 may be an interchangeable lens that can be attached to and detached from the imaging device 500 . The imaging device 500 mainly includes an imaging device 100 , a system control unit 501 , a driving unit 502 , a photometry unit 503 , a work memory 504 , a recording unit 505 and a display unit 506 .

The photographing lens 520 is composed of a plurality of optical lens groups, and forms an image of subject light flux from a scene in the vicinity of its focal plane. In FIG. 5, a single virtual lens arranged in the vicinity of the pupil is represented as a representative. The drive unit 502 executes charge accumulation control of the image sensor 100, readout control of pixel signals, and the like in accordance with instructions from the system control unit 501. FIG.

The image pickup device 100 transfers pixel signals to the image processing unit 511 of the system control unit 501 . An image processing unit 511 performs various image processing using the work memory 504 as a workspace to generate image data. For example, when generating image data in the JPEG file format, compression processing is performed after performing white balance processing, gamma processing, and the like. The generated image data is recorded in the recording unit 505, converted into a display signal, and displayed on the display unit 506 for a preset time.

A photometry unit 503 detects the luminance distribution of a scene prior to a series of shooting sequences for generating image data. The photometry unit 503 includes, for example, an AE sensor with approximately one million pixels. A calculation unit 512 of the system control unit 501 receives the output of the photometry unit 503 and calculates the brightness for each area of the scene. A calculation unit 512 determines the shutter speed, the aperture value, and the ISO sensitivity according to the calculated luminance distribution. Note that the pixels used in the AE sensor may be provided in the image sensor 100, and in this case, the photometry unit 503 separate from the image sensor 100 may not be provided.

FIG. 6 is a block diagram showing a specific configuration of the driving section 502. As shown in FIG. The drive unit 502 has a sensor control unit 441, a block control unit 442, a synchronization control unit 443, a signal control unit 444, a pixel memory 414, an arithmetic circuit 415 as shared control functions, and centrally controls these control units. and a drive control unit 420 that The drive unit 502 further includes an I/F circuit 418 between the drive control unit 420 and the system control unit 501 of the imaging apparatus 500 main body.

The drive control unit 420 refers to the timing memory 430, converts an instruction from the system control unit 501 into a control signal that can be executed by each control unit, and delivers it to each control unit. The timing memory 430 is formed by flash RAM or the like.

The sensor control unit 441 controls the transmission of control pulses related to charge accumulation and charge readout of each pixel, which are transmitted to the imaging chip 113 . Specifically, the sensor control unit 441 controls the start and end of charge accumulation of the target pixel by sending a reset pulse and a transfer pulse to the row control unit 200 of each unit block 131, and controls the readout pixel. Sending out the selection pulse causes the pixel signal to be output to the column transmission line 170 .

The block control unit 442 sends a specific pulse to the imaging chip 113 to specify the unit block 131 to be controlled. The transfer pulse or the like received by each pixel via the wiring Tx_j or the like is the logical product of each pulse sent by the sensor control section 441 and the specific pulse sent by the block control section 442 . In this way, each area can be controlled as a block independent of each other. When using synchronized pulses in a plurality of unit blocks 131, and when performing an operation across a plurality of unit blocks 131, the block control unit 442 generates a specific pulse for specifying each of the plurality of unit blocks. are sent at the same time.

The synchronization control unit 443 sends out a synchronization signal to the imaging chip 113 . Each pulse becomes active in the imaging chip 113 in synchronization with the synchronization signal. For example, by adjusting the synchronizing signal, random control, thinning control, etc., for controlling only specific pixels belonging to the same unit block 131 are realized. Also, the signal control unit 444 performs timing control for the CDS circuit 202 , the A/D conversion circuit 204 , and the shift register 206 .

The arithmetic circuit 415 calculates AE evaluation values and the like based on the pixel signals stored in the pixel memory 414 . The calculation circuit 415 outputs the calculation result to the drive control section 420 .

The pixel memory 414 has a memory space capable of storing pixel signals from the pixels 150 of the imaging unit, and stores the pixel signals read from each pixel and digitized. A pixel memory 414 is preferably provided corresponding to each unit block 131 .

The pixel memory 414 is provided with a data transfer interface for transmitting pixel signals according to delivery requests. The data transfer interface is connected to a data transfer line connected to the image processing unit 511 . The data transfer line is composed of, for example, a data bus among bus lines. In this case, the transfer request from the system control unit 501 to the drive control unit 420 is executed by addressing using the address bus.

The transmission of pixel signals by the data transfer interface is not limited to the addressing method, and various methods can be adopted. For example, it is possible to adopt a double data rate method in which data transfer is performed using both rising and falling edges of a clock signal used for synchronization of each circuit. In addition, a burst transfer method can be employed to transfer data at once by partially omitting procedures such as addressing, thereby increasing the speed. It is also possible to adopt a combination of a bus method using a line connecting the control unit, memory unit and input/output unit in parallel, and a serial method for serially transferring data bit by bit.

With this configuration, the image processing unit 511 can receive only necessary pixel signals, so that image processing can be completed at high speed especially when forming a low-resolution image. Note that the driving unit 502, the row control unit 200 in FIG. 4, and the peripheral circuit 133 function as a reading unit that sequentially reads pixel signals of the pixels 150 included in the imaging unit across a plurality of unit blocks 131. FIG.

FIG. 7 shows functional blocks of the arithmetic circuit 415 . The arithmetic circuit 415 has a correlation calculator 472 and a corrector 474 in addition to the functions described above. The correlation calculator 472 reads the pixel signals from the pixel memory 414 and calculates the correlation of the pixel signals between the boundary rows of the unit blocks 131 adjacent in the same column. The correction unit 474 corrects the pixel signals in the adjacent unit blocks 131 in the same column based on the correlation calculated by the correlation calculation unit 472 and writes them to the pixel memory 414 .

FIG. 8 shows a timing chart of operations such as charge accumulation and readout in each row of the unit block 131A. Within each unit block, the operation of selecting the pixels 150 in the same row and accumulating and reading out the pixel signals is sequentially repeated row by row, which is a so-called rolling shutter method.

The drive unit 502 starts charge accumulation in the pixels 150 in the first row by applying drive signals to the wirings Rst_1 and Tx_1 for the pixels 150 in the first row via the row control unit 200 . Further, after the accumulation period, the driving unit 502 starts reading pixel signals of the pixels 150 on the first row by applying a driving signal to the wiring Sel_1 for the pixels 150 on the first row via the row control unit 200 . The readout period is the time from when pixel signals are read from each pixel 150 in the first row, processed by the CDS circuit 202 and the A/D conversion circuit 204, and sequentially written from the shift register 206 to the pixel memory 414. include.

Similarly, driving signals are applied to the wirings Rst_2, Tx_2 and Sel_2 for the pixels 150 in the second row, and storage and readout are performed. The length of the accumulation period for the pixels 150 on the second row is the same as that for the pixels 150 on the first row, but is delayed by approximately the readout period in terms of time. A driving signal is applied to the wirings Rst_l, Tx_l, and Sel_l from the third row to the Lth row, and storage and readout are sequentially performed.

The order of rows selected is the same for both of the plurality of unit blocks 131A and 131B. In the example of FIG. 8, each row is selected from the row on the -Y side toward the row on the +Y side. Furthermore, the timing of storage and readout of the same row is synchronized between the plurality of unit blocks 131A and 131B. However, the timing of at least one of storage and readout of the same row may not be synchronized between the plurality of unit blocks 131A and 131B.

FIG. 9 shows an example of a subject image 300 incident on the imaging device 100. As shown in FIG. For the sake of explanation, image light immediately before entering the image sensor 100 is depicted as viewed from the +Z side.

In the example shown in FIG. 9, a subject image 300 includes two moving object images 302 and 306 that move in the X direction and are long in the Y direction, and one still object image 304 that is stationary and long in the Y direction. ing. Furthermore, moving object image 302 moves faster than moving object image 306 .

FIG. 10 shows a captured image before correction. For the sake of explanation, it is shown in the direction in which it is output, displayed, etc. as a captured image. Also, it is assumed that the subject image 300 in FIG. 9 is incident on four unit blocks 131A, 131B, 131C, and 131D.

As shown in the unit blocks 131A and 131B of FIG. 10, the moving object image 302 is obliquely distorted within the unit blocks. This distortion shifts the moving object image 302 in the +X direction, which is the movement direction, because the imaging time differs for each row by the A/D conversion time. This deviation is called rolling distortion. Similarly, in the unit block 131D as well, the moving object image 306 is distorted in the +X direction. However, since the speed of the moving object image 306 is smaller than the speed of the moving object image 302, the amount of distortion is also small. On the other hand, as indicated by the unit blocks 131C and 131D, the stationary image 304 is not distorted.

Further, attention is focused on the pixels in the boundary row between the adjacent unit blocks 131A and 131B in the same column. That is, focus on the 10th row of the unit block 131A and the 1st row of the unit block 131B. These two rows are the furthest apart in timing of the accumulation period. Reflecting this, the positions of the pixels corresponding to the moving object image 302 moving together in the +X direction are shifted to different column positions between these two rows. On the other hand, the column positions of the pixels corresponding to the still body image 304 are not aligned between the 10th row of the unit block 131C and the 1st row of the unit block 131D.

FIG. 11 is a flowchart showing the operation of the arithmetic circuit 415, and FIG. 12 shows a captured image after correction. The flowchart of FIG. 11 starts when the pixel signal before correction of FIG. 10 is acquired and stored in the pixel memory 414 .

The correlation calculator 472 reads the pixel signals of the boundary row between the adjacent unit blocks 131A, 131B, etc. in the same column from the pixel memory 414 (S100). For example, in the case of a pair of unit blocks 131A and 131B, the pixel signals of the 10th row of the unit block 131A and the pixel signals of the 1st row of the unit block 131B are read.

The correlation calculator 472 calculates the correlation of the boundary row (S102). It can also be said that the magnitude of correlation is the degree of matching between pixel signals in one row. As an example of an evaluation value for evaluating the correlation, the correlation calculation unit 472 squares the difference between the pixel signals of the pixels in the same column in the 10th row of the unit block 131A and the 1st row of the unit block 131B, and sums them. Calculate the sum of squared residuals. Note that there is a relationship that the smaller the residual sum of squares, the greater the correlation. An example using the residual sum of squares as the evaluation value will be described below.

The correlation calculator 472 determines whether the residual sum of squares is greater than a threshold (S104). If the magnitude of the residual sum of squares is equal to or less than the threshold (S104: No), the flow chart for the adjacent unit blocks 131A, 131B, etc. on the same column ends.

When the magnitude of the residual sum of squares is smaller than the threshold, for example, as shown in FIG. It is assumed that the displacement of the column position is small even though the readout has been performed. Therefore, in the unit blocks 131C and 131D adjacent in the same column, the unit block 131C having the 10th row stored and read out temporally later has no rolling distortion, or even if it has rolling distortion. presumed to be small. Therefore, the unit block 131C is temporarily excluded from the correction targets in FIG. However, due to the influence of the adjacent unit blocks 131A in the same column, correction may be performed to form the correction block 132C.

On the other hand, when the magnitude of the residual sum of squares is greater than the threshold (S104: Yes), the correlation calculator 472 compares one to the other in the 10th row of the unit block 131A and the 1st row of the unit block 131B. The residual sum of squares is calculated for the pixels shifted by one pixel in the row direction (S106), that is, the pixels shifted by one column (S108). The correlation calculation unit 472 repeats the above steps S106 and S108 for calculating the residual sum of squares after shifting by one pixel until the pixel number p in the column is reached (S110: No). In this case, the pixels are shifted in the +X direction and the -X direction.

After shifting by p pixels (S110: Yes), the correlation calculator 472 specifies the number of pixels to be used for correction (112). In this case, the correlation calculation unit 472 uses, for correction, the number of pixel deviations when the residual sum of squares is the smallest among the plurality of residual sums of squares calculated by repeating steps S106 to S110. number of pixels. If there are a plurality of smallest sums of squared residuals, it is preferable to set the number of pixels to be smaller than the number of pixel shifts.

In the example shown in FIG. 10, when the 10th row of the unit block 131A is shifted from the 1st row of the unit block 131B by 4 pixels in the -X direction, the residual sum of squares is minimized. Therefore, the correlation calculator 472 specifies "4" as the number of pixels to be used for correction.

The correction unit 474 corrects the pixel signal of the unit block 131A using the number of pixels specified in step S112 (S114). Here, among the unit blocks 131A and 131B adjacent in the same column, the pixel signal of the unit block 131A having the 10th row, which is accumulated and read later in time, is corrected. In this case, the correction unit 474 evenly allocates the shift amount to each row in the unit block 131A so that the 10th row is shifted from the 1st row by 4 pixels, that is, 4 columns. In the example of FIG. 10, two adjacent rows are treated as a set, and a shift of 1 pixel is assigned to the adjacent sets.

As shown in FIG. 12, the correction unit 474 fixes the 5th and 6th rows in the center of the unit block 131A with respect to the other unit blocks 131B, etc., and converts the pixel signals of the pixels in the other rows to the column positions. is replaced with the pixel signal of the shifted pixel to generate the correction block 132A. Similarly, correction blocks 132B and 132D are generated by shifting the column position of each row of unit blocks 131A and 132D.

The correction unit 474 further corrects the pixel signals at the boundary between the adjacent unit blocks 131A and 131C in the same row. As shown in FIG. 12, when the unit blocks 131A and 131C adjacent in the same row are corrected to generate corrected blocks 132A and 132C, blank pixel 140 areas and overlapping pixels 142 where pixels overlap each other are formed on the boundary between them. A region of The correction unit 474 allocates pixel signals from the periphery to the region of the blank pixels 140 . Furthermore, the correction unit 474 assigns the average value of the pixel signals of the respective pixels to the overlapping pixels 142 .

As described above, the correction block 132A and the like are generated. Here, the correction block 132C does not have pixels replaced by the correlation determination in step S104 with respect to the unit block 131C, but its boundary is corrected under the influence of the correction from the adjacent correction block 132A. The correction unit 474 outputs the pixel signal of the correction block 132A and the like to the pixel memory 414 (S116), and ends this flowchart. As described above, rolling distortion can be reduced with a simple configuration.

In step S102, instead of calculating the correlation between the boundary rows in the unit blocks 131A, 131B, etc. adjacent in the same column, the correlation between the first row and the last row in the same unit block 131A is calculated. may In each of steps S102 and 108, if the pixel signal has luminance and color signals, the correlation may be calculated using the luminance signal. Alternatively, the correlation may be calculated for pixels of the same color, such as green pixels. As the correlation evaluation value, the sum of the absolute values of the differences may be used instead of the residual sum of squares. Also, instead of repeating p pixels in step S110, a so-called hill-climbing method may be used, in which the repetition is stopped when a minimum value appears compared to calculating the residual sum of squares. .

When the unit block 131 has a color filter such as a Bayer array, the operation of FIG. 11 is preferably executed before interpolation processing. When the A/D-converted pixel signal is output as a captured image converted into a predetermined format such as JPEG, the operation of FIG. It is preferably done before converting. At least the correlation calculator 472 and the corrector 474 are preferably provided within the signal processing chip 111 .

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the scope of claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The execution order of each process such as actions, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly "before", "before etc., and it should be noted that they can be implemented in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if the description is made using "first," "next," etc. for the sake of convenience, it means that it is essential to carry out in this order. not a thing

100 image sensor, 101 microlens, 102 color filter, 103 passivation film, 104 PD, 105 transistor, 106 PD layer, 107 wiring, 108 wiring layer, 109 bump, 110 TSV, 111 signal processing chip, 112 memory chip, 113 imaging Chip, 131 unit block, 132 correction block, 133 peripheral circuit, 140 blank pixel, 142 duplicate pixel, 150 pixel, 152 reset transistor, 154 transfer transistor, 156 FD, 162 amplification transistor, 164 selection transistor, 166 load current source, 170 column transmission line, 172 column bus line, 200 row control unit, 202 CDS circuit, 204 A/D conversion circuit, 206 shift register, 300 object image, 302 moving object image, 304 still object image, 306 moving object image, 414 pixel memory, 415 arithmetic circuit, 418 I/F circuit, 420 drive control unit, 430 timing memory, 441 sensor control unit, 442 block control unit, 443 synchronization control unit, 444 signal control unit, 472 correlation calculation unit, 474 correction unit, 500 imaging Apparatus, 520 photographing lens, 501 system control unit, 502 drive unit, 503 photometry unit, 504 work memory, 505 recording unit, 506 display unit, 511 image processing unit, 512 calculation unit

Claims (13)

an imaging device in which unit blocks each having a plurality of pixels arranged side by side in the row direction and the column direction are arranged in the row direction and the column direction;
In each of the first unit block and the second unit block arranged side by side in at least the column direction among the plurality of unit blocks, a signal is generated sequentially in the column direction for each pixel arranged in the row direction. a driver for reading out the
a correlation calculation unit that calculates an evaluation value indicating a correlation between signals read from a first pixel of the first unit block and a second pixel of the second unit block, which are arranged adjacent to each other in the column direction; ,
According to the evaluation value calculated by the correlation calculation unit, the correlation increases with respect to one of the signals read from the pixels of the first unit block and the pixels of the second unit block. a correction unit that corrects such that
An imaging device comprising: The imaging device according to claim 1,
The imaging element includes a first row control section for reading out signals from the pixels included in the first unit block, and a second row control section for reading out signals from the pixels included in the second unit block. imaging device. In the imaging device according to claim 2,
The first row control unit outputs a drive signal to the pixels of the first unit block,
The imaging device, wherein the second row control section outputs a drive signal to the pixels of the second unit block. In the imaging device according to claim 2 or 3,
The imaging element includes an imaging chip having a semiconductor substrate on which the plurality of unit blocks are arranged, and a signal processing chip having a semiconductor substrate on which the first row control section and the second row control section are arranged. imaging device. In the imaging device according to claim 4,
The imaging device, wherein the imaging chip is stacked with the signal processing chip. In the imaging device according to claim 4 or claim 5,
the imaging element includes a first transmission path for transmitting signals from the pixels of the first unit block;
a second transmission path for transmitting signals from the pixels of the second unit block;
the first transmission line is connected to the plurality of pixels arranged side by side in the column direction in the first unit block;
The imaging device, wherein the second transmission path is connected to the plurality of pixels arranged side by side in the column direction in the second unit block. In the imaging device according to claim 6,
The imaging element is connected to the first transmission line, and is connected to a first conversion section for converting signals read from the pixels of the first unit block into digital signals, and the second transmission line, and the and a second converter that converts a signal read from the pixels of the second unit block into a digital signal. In the imaging device according to claim 7,
The first conversion section and the second conversion section are arranged on a semiconductor substrate included in the signal processing chip. The imaging device according to claim 1 ,
the imaging element includes a first transmission path for transmitting signals from the pixels of the first unit block;
a second transmission path for transmitting signals from the pixels of the second unit block;
the first transmission line is connected to the plurality of pixels arranged side by side in the column direction in the first unit block;
The imaging device, wherein the second transmission path is connected to the plurality of pixels arranged side by side in the column direction in the second unit block. In the imaging device according to claim 9 ,
The imaging element is connected to the first transmission line, and is connected to a first conversion section for converting signals read from the pixels of the first unit block into digital signals, and the second transmission line, and the and a second converter that converts a signal read from the pixels of the second unit block into a digital signal. In the imaging device according to claim 10,
The imaging device includes an imaging chip having a semiconductor substrate on which the plurality of unit blocks are arranged, and a signal processing chip having a semiconductor substrate on which the first conversion section and the second conversion section are arranged. Device. The imaging device according to claim 11, wherein
The imaging device, wherein the imaging chip is stacked with the signal processing chip. In the imaging device according to any one of claims 1 to 12 ,
the first pixel is included in pixels aligned in the row direction and read last in the first unit block among the plurality of pixels included in the first unit block;
The imaging device, wherein the second pixel is included in the plurality of pixels aligned in the row direction and read first in the second unit block among the pixels included in the second unit block.

Images