JP6010943B2 - Imaging device - Google Patents

Description

The present invention relates to an imaging equipment.

A digital camera, which is an example of an imaging apparatus, has a moving image shooting function in addition to a still image shooting function. This digital camera is equipped with a solid-state imaging device with high pixels in order to improve the quality of still images. As this solid-state imaging device, a single-plate color sensor including a Bayer array color filter array in which red (R), green (G), and blue (B) colors are arranged in a checkered pattern is used. When a still image is taken using such a digital camera, signals are read out from all the pixels arranged in the solid-state imaging device. On the other hand, when capturing a moving image, in order to correspond to the frame rate at the time of acquiring the moving image or the file format of the moving image, the same color pixels among the pixels arranged in the solid-state image sensor are thinned, for example. The signal is read using a method such as reading.
By the way, the signal of each pixel output from the solid-state imaging device composed of the single-plate color sensor described above is a signal of any color component of red (R), green (G), and blue (B). . For this reason, the output signal is subjected to color interpolation processing to generate signals of all color components of R, G, and B colors. For example, when a moving image is shot using a digital camera in which the imaging optical system is optimized for the resolution of still images, thinning readout is performed when the imaging optical system is close to focusing. If color interpolation processing is performed on the obtained signal, moire and false colors are caused. In order to prevent the occurrence of these moire and false colors, for example, it has also been proposed to provide the effect of a low-pass filter by widening the range of pixels to be read during addition thinning readout (see Patent Document 1).

Japanese Patent No. 3877695

  However, although the above-described invention of Patent Document 1 can suppress the generation of moire and false colors, there is a problem that the resolution in the obtained moving image is lowered.

The present invention, while suppressing the occurrence of moire and false color, and an object thereof is to provide an imaging equipment that make it possible to prevent a decrease in perceived resolution in the video.

In order to solve the above-described problem, an imaging apparatus according to the present invention includes a first light receiving unit that receives light of a first color and a first amplification unit that amplifies a signal from the first light receiving unit. A first pixel, a second pixel having a second light receiving part for receiving light of the second color, a second amplification part for amplifying a signal from the second light receiving part, and a first position Amplification control for setting the amplification factor of the first amplification unit and the amplification factor of the second amplification unit according to the distance from the first pixel to the first pixel and the distance from the first position to the second pixel And a circuit for adding signals from the plurality of first pixels and adding signals from the plurality of second pixels.

The imaging device of the present invention includes a first pixel having a first light receiving unit that receives light of a first color, a first amplification unit that amplifies a signal from the first light receiving unit, and a first pixel. A second pixel having a second light receiving portion for receiving light of two colors, a second amplification portion for amplifying a signal from the second light receiving portion, and a third pixel for receiving light of the third color. A third pixel having a third amplifying unit for amplifying a signal from the light receiving unit and the third light receiving unit, a distance from the first pixel to the first pixel, and the first pixel from the first position. The amplification factor of the first amplification unit, the amplification factor of the second amplification unit, and the amplification of the third amplification unit are determined according to the distance of the second pixel and the distance of the third pixel from the third position. An amplification control unit for setting a rate, signals from a plurality of the first pixels are added, signals from a plurality of the second pixels are added, and a plurality of the second images are added. Adding the signals from and a circuit.

  According to the present invention, it is possible to prevent a decrease in resolution in a moving image while suppressing generation of moire and false colors.

It is a functional block diagram which shows an example of an imaging device. It is a figure which shows the structure of the image pick-up element in 1st Embodiment. It is a figure which shows the structure of a pixel. It is a figure which shows the positional relationship of the pixel on an image sensor, and the pixel of the moving image read by addition thinning-out reading. It is a figure which shows the weight of each pixel contained in the read-out range. It is a figure explaining the read-out range which used R color pixel in coordinates (i, j) on an image sensor as a standard pixel. It is a figure explaining the read-out range which used Gr color pixel in coordinates (i + 3, j) on an image sensor as a standard pixel. It is a figure explaining the read-out range which used Gb color pixel in coordinates (i, j + 3) on an image sensor as a standard pixel. It is a figure explaining the reading range which used B color pixel in coordinates (i + 3, j + 3) on an image sensor as a standard pixel. It is a figure which shows the gravity center position of the color component in each pixel of the moving image which performed the addition thinning-out reading using simple addition averaging, and the gravity center position in the pixel on an image sensor. It is a figure which shows the structure of the image pick-up element in 3rd Embodiment. It is a figure which shows the positional relationship of the pixel on an image pick-up element and the pixel of the moving image read by addition thinning-out reading in 3rd Embodiment. It is a figure which shows the positional relationship of the pixel on an image pick-up element and the pixel of the moving image read by addition thinning-out reading in 4th Embodiment. It is a figure which shows the example of the weight of each pixel contained in the read-out range. It is a figure which shows the example of the weight of each pixel contained in the read-out range. It is a figure which shows the example of the weight of each pixel contained in the read-out range. It is a figure which shows an example of an image processing apparatus. It is a flowchart which shows an example of the process in an image processing program.

  Embodiments of the present invention will be described below. FIG. 1 is a functional block diagram illustrating an example of an imaging apparatus according to the present embodiment. The imaging apparatus 10 includes an imaging optical system 15, an imaging element 16, an operation member 17, a lens driving circuit 18, a photometric circuit 19, a CPU 20, an imaging element driving circuit 21, an A / D converter 22, an image processing circuit 23, and a liquid crystal monitor 24. A compression / decompression processing circuit 25, a display output circuit 26, and a storage medium 27.

  A subject image (not shown) is formed on the image pickup surface of the image pickup device 16 by the image pickup optical system 15. The imaging optical system 15 includes a plurality of lenses, and is configured to be able to adjust focus, zoom, and the like via the lens driving circuit 18 based on the operation of the operation member 17 and the like.

  For example, a CMOS image sensor is used as the image sensor 16. As this CMOS image sensor, for example, a single-plate color sensor including a Bayer array color filter array in which red (R), green (G), and blue (B) color filters are arranged in a checkered pattern is used. The image sensor 16 is an image sensor that can capture a still image as well as continuously capture a still image and a moving image. The image pickup device 16 is driven by the image pickup device drive circuit 21 under the control of the CPU 20 based on the subject photometry data obtained by the photometry circuit 19. The image signal read by the image sensor 16 is input to the A / D converter 22.

  The A / D converter 22 performs A / D conversion processing on the analog image signal read from the image sensor 16 and converts the analog image signal into a digital image signal. The digital image signal is input to the image processing circuit 23.

  The image processing circuit 23 performs image processing such as white balance processing, γ conversion processing, and edge enhancement processing on the input image signal. The image processing circuit 23 performs color space conversion processing on the input image signal as necessary.

  In addition, the image processing circuit 23 performs resolution (number of pixels) conversion processing for displaying on the liquid crystal monitor 24 and outputs it to the compression / decompression processing circuit 25 and the display output circuit 26. The display output circuit 26 performs predetermined signal processing on the image signal input from the image processing circuit 23 and outputs it to the liquid crystal monitor 24. The display output circuit 26 further performs a process of superimposing an overlay image signal such as a shooting menu or a cursor on the image signal output from the image processing circuit 23 as needed under the control of the CPU 20. Thus, the overlay image is superimposed on the subject image and displayed on the liquid crystal monitor 24.

  The compression / decompression processing circuit 25 performs compression processing on the input image signal and stores it in the storage medium 27. In addition to this, when the image signal stored in the storage medium 27 is read, the compression / decompression processing circuit 25 performs a decoding process on the read image signal, and the image processing circuit 23 and This is supplied to the liquid crystal monitor 24 via the display output circuit 26.

  The CPU 20 extracts an image signal of an area (AF area) set on the imaging screen based on an operation of a release button that constitutes a part of the operation member 17, and the contrast value of the area (or the area) A high spatial frequency component amount) is calculated, and a so-called contrast AF operation for adjusting the focus state of the subject image on the imaging surface of the image sensor 16 is performed based on the calculation result.

  Further, the CPU 20 drives the imaging optical system 15 to analyze sequentially obtained image signals for each subject in the image, and for each subject based on the lens position when the contrast value in the region becomes maximum. Get shooting distance information.

  Although the contrast AF operation is exemplified as the automatic focusing operation, the present invention is not limited to this, and a well-known pupil division type phase difference AF operation can also be used. Also in this case, the shooting distance information of each region can be obtained by the automatic focusing operation.

  The CPU 20 drives the imaging optical system 15 via the lens driving circuit 18 based on the operation of the zoom operation member that constitutes a part of the operation member 17, and the subject image formed on the imaging surface of the image sensor 16. An optical zoom operation for enlarging or reducing the image is performed. Further, the CPU 20 converts the image signal obtained by the imaging device 16 or the image signal stored in the storage medium 27 based on the operation of the zoom operation member constituting a part of the operation member 17 to the resolution by the image processing circuit 23. (Number of pixels) An electric zoom operation for enlarging or reducing by the conversion process is controlled.

  Next, a first implementation will be described in which the size of the moving image obtained by shooting (image size) is set to an image size that is 1/3 times the horizontal size and the vertical size of the still image. This will be described as a form.

<First Embodiment>
First, the configuration of the image sensor in the first embodiment will be described with reference to FIG. As shown in FIG. 2, the image pickup device 16 includes a pixel unit 41, precharge units 42a and 42b, switching units 43a and 43b, a vertical scanning circuit 44, and horizontal scanning circuits 45a and 45b.

  The pixel unit 41 is two-dimensionally arranged with a plurality of pixels in a horizontal row direction and a vertical column direction. In FIG. 2, “R” is a pixel in which a red (R) color filter is arranged, “G” is a pixel in which a green (G) color filter is arranged, and “B” is blue. (B) A pixel in which a color filter of color is arranged. In the following description, a pixel in which a red (R) color filter is arranged is an R color pixel, a pixel in which a green (G) color filter is arranged is a G pixel, and a blue (B) color filter is arranged. The pixel will be described as a B color pixel.

  In the image sensor 16, two vertical signal lines 47a and 47b are provided for pixels arranged in the same column. The color filter provided in each pixel has a configuration in which each color filter of R color, G color, and B color is arranged in a Bayer arrangement with a 2 × 2 arrangement pattern. For example, in a column (odd column) in which R color pixels and G color pixels are alternately arranged, the vertical signal line 47a is connected to the R color pixel, and the vertical signal line 47b is connected to the G color pixel. Further, in a column in which G color pixels and B color pixels are alternately arranged (even number columns), the vertical signal line 47a is connected to the B color pixel, and the vertical signal line 47b is connected to the G color pixel.

  The precharge units 42a and 42b include a precharge transistor PRE having a source electrode connected to each of the vertical signal lines 47a and 47b, and a row signal line for supplying a control signal φPRE to the gate voltage of the precharge transistor PRE. A precharge voltage Vpre is supplied to the drain electrode of each precharge transistor PRE. The vertical signal lines 47a and 47b provided in each column are precharged to a predetermined precharge voltage Vpre via a corresponding precharge transistor PRE before reading a voltage signal (pixel signal) from each pixel.

Switching unit 43a includes a capacitor CD1, adding transistors _{ADD 1, ADD 2, ADD 3} , ADD 4, ADD 5, ADD 6, the switching transistor SW1, the control signal _{φADD 1, φADD 2, φADD 3} , φADD 4, φADD 5, φADD ₆ and 10 row signal lines for supplying control signals φLINE1, φLINE2, φLINE3, and φLINE4. The capacitor CD1 holds a voltage signal output to the vertical signal line 47b from the G color pixel arranged in each column among the plurality of pixels provided in the pixel unit 41.

The addition transistor ADD ₁ is connected to the vertical signal line 47b in the 6R-5 (R = 1, 2, 3,...) Column and the vertical signal line 47b in the 6R-4 column. The addition transistor ADD ₁ is turned on when the control signal φADD ₁ is output, and the vertical signal line 47b in the 6R-5th column and the vertical signal line 47b in the 6R-4th column are short-circuited.

Summing transistor ADD ₂ are connected to each of the 6R-4 column of vertical signal lines 47b and 6R-3 column of the vertical signal line 47b. The addition transistor ADD ₂ is turned on when the control signal φADD ₂ is output, and short-circuits the vertical signal line 47b in the 6R-4th column and the vertical signal line 47b in the 6R-3th column.

Summing transistor ADD ₃ is connected to each of the 6R-3 column of the vertical signal line 47b and 6R-2 column of the vertical signal line 47b. The addition transistor ADD ₃ is turned on when the control signal φADD ₃ is output, and short-circuits the vertical signal line 47b in the 6R-3 column and the vertical signal line 47b in the 6R-2 column.

Summing transistor ADD ₄ is connected to each of the 6R-2 column of the vertical signal line 47b and 6R-1 column of the vertical signal line 47b. The addition transistor ADD ₄ is turned on when the control signal φADD ₄ is output, and the vertical signal line 47b in the 6R-2 column and the vertical signal line 47b in the 6R-1 column are short-circuited.

The addition transistor ADD ₅ is connected to each of the vertical signal line 47b in the 6R-1 column and the vertical signal line 47b in the 6R column. The addition transistor ADD ₅ is turned on when the control signal φADD ₅ is output, and short-circuits the vertical signal line 47b in the 6R-1 column and the vertical signal line 47b in the 6R column.

The addition transistor ADD ₆ is connected to each of the vertical signal line 47b in the 6R column and the vertical signal line 47b in the 6R + 1 column. The addition transistor ADD ₆ is turned on when the control signal φADD ₆ is output, and the vertical signal line 47b in the 6R column and the vertical signal line 47b in the 6R + 1 column are short-circuited.

  The switching transistor SW1 reads the voltage signal read for each column. For example, the signal φLINE1 is input to the gate electrode of the switch transistor SW1 arranged in the 4S-3 (S = 1, 2, 3,...) Column, and the gate of the switch transistor SW1 arranged in the 4S-2 column. A signal φLINE2 is input to the electrode. A signal φLINE3 is input to the gate electrode of the switch transistor SW1 arranged in the 4S-1 column, and a signal φLINE4 is input to the gate electrode of the switch transistor SW1 arranged in the 4S column. When the switching transistor SW1 is turned on, a voltage signal held in the capacitor CD1 of the corresponding column is output to the column amplifier CAMP1.

Switching unit 43b includes a capacitor CD2, adding transistor _{ADD 7, ADD 8, ADD 9} , ADD 10, a switching transistor SW2, the control signal _{φADD 7, φADD 8, φADD 9} , φADD 10, control signals φLINE5, φLINE6, φLINE7, the φLINE8 It consists of eight row signal lines to be supplied. The capacitor CD2 holds a voltage signal output from the R color pixel or the B color pixel arranged in each column to the vertical signal line 47a among the plurality of pixels provided in the pixel unit 41.

The addition transistor ADD ₇ is connected to the vertical signal line 47a in the 6R-5th column and the vertical signal line 47a in the 6R-3th column. The addition transistor ADD ₇ is turned on when the control signal φADD ₇ is output, and short-circuits the vertical signal line 47a in the 6R-5th column and the vertical signal line 47a in the 6R-3th column.

The addition transistor ADD ₈ is connected to the vertical signal line 47a in the 6R-4th column and the vertical signal line 47a in the 6R-2th column. The addition transistor ADD ₈ is turned on when the control signal φADD ₈ is output, and short-circuits the vertical signal line 47a in the 6R-4th column and the vertical signal line 47a in the 6R-2th column.

The addition transistor ADD ₉ is connected to the vertical signal line 47a in the 6R-2 column and the vertical signal line 47a in the 6R column. The addition transistor ADD ₉ is turned on when the control signal φADD ₉ is output, and short-circuits the 6R-2 column vertical signal line 47a and the 6R column vertical signal line 47a.

Summing transistor _{ADD 10} includes a 6R-1 column of the vertical signal line 47a, is connected to the 6R + 1 column of the vertical signal line 47a. The addition transistor ADD ₁₀ is turned on when the control signal φADD ₁₀ is output, and the vertical signal line 47a in the 6R-1 column and the vertical signal line 47a in the 6R + 1 column are short-circuited.

  The switching transistor SW2 reads the voltage signal read for each column. Here, the control signal φLINE5 is input to the gate electrode of the switch transistor SW2 arranged in the 4S-3 column, and the signal φLINE6 is input to the gate electrode of the switch transistor SW2 arranged in the 4S-2 column. A signal φLINE7 is input to the gate electrode of the switch transistor SW2 arranged in the 4S-1 column, and a signal φLINE8 is input to the gate electrode of the switch transistor SW2 arranged in the 4S column. When the switching transistor SW2 is turned on, a voltage signal held in the capacitor CD2 of the corresponding column is output to the column amplifier CAMP2.

  The column amplifier CAMP1 and the column amplifier CAMP2 receive the voltage signal output from each pixel connected to the corresponding vertical signal line. From each pixel, a voltage signal (optical signal) corresponding to the amount of charge obtained by photoelectric conversion when incident light is received, and a voltage signal (dark signal) when the charge generated by photoelectric conversion is reset Is output. These column amplifiers CAMP1 and CAMP2 sample and hold the optical signal and the dark signal, amplify and output a differential signal indicating the difference between the optical signal and the dark signal.

  The vertical scanning circuit 44 includes a signal generation circuit and a buffer circuit (not shown), and two-dimensionally supplies the control signal φTX, the control signal φRST, the control signal φSW3, the control signal φGAIN, and the power supply VCC to the pixel unit 41. A line-by-row supply is provided for each pixel to be arranged.

The horizontal scanning circuit 45a supplies a control signal φPRE and a power supply Vpre to the precharge unit 42a. The horizontal scanning circuit 45a, the control signal FaiADD ₁ with respect to the switching unit _{43a, φADD 2, φADD 3,} φADD 4, φADD 5, φADD 6, control signal φLINE1, φLINE2, φLINE3, supplies a respective signal of FaiLINE4.

Similarly, the horizontal scanning circuit 45b supplies the control signal φPRE and the power supply Vpre to the precharge unit 42b. The horizontal scanning circuit 45b supplies control signals φADD ₇ , φADD ₈ , φADD ₉ , φADD ₁₀ , and control signals φLINE5, φLINE6, φLINE7, and φLINE8 to the switching unit 43b.

  Next, the configuration of the pixels arranged in the pixel portion 41 of the image sensor 16 will be described with reference to FIG. The pixel P of the image sensor 16 includes a photodiode PD, a transfer transistor TX, a floating diffusion FD, an amplification transistor AMI, a reset transistor RST, a current source ISS, a switching transistor SW3, and an amplifier AMP.

  The photodiode PD generates charges according to the amount of incident light received by photoelectrically converting the incident light. The transfer transistor TX transfers the charge generated by the photodiode PD to the floating diffusion FD in response to the control signal φTX supplied to the gate electrode. The floating diffusion FD accumulates the charges transferred from the transfer transistor TX. The potential of the floating diffusion FD is determined according to the amount of accumulated charge.

  The amplification transistor AMI has its drain electrode connected to the power supply node VCC and its gate voltage connected to the floating diffusion FD. The source electrode of the amplification transistor AMI is connected to the current source ISS. As a result of the amplification transistor AMI forming a source follower, a voltage corresponding to the potential of the floating diffusion FD is generated at the source electrode of the amplification transistor AMI.

  Reset transistor RST is connected between power supply node VCC and floating diffusion FD. The reset transistor RST resets the potential of the floating diffusion FD in response to the control signal φRST supplied to the gate electrode.

  The amplifier AMP amplifies the input voltage by multiplying the voltage generated by the amplification transistor AMI by a predetermined gain. The gain multiplied by the input voltage will be described later. For example, the amplifier AMP is disposed in the image sensor 16 and the gain used when reading out all of the plurality of pixels disposed in the image sensor 16. The gain used when adding and reading out a plurality of pixels while thinning out can be switched. This gain switching is executed depending on whether or not the control signal φGAIN is output. The switching transistor SW3 is turned on by the output of the signal φSW3. When the switching transistor SW3 is turned on, the voltage signal amplified by the amplifier AMP is output to the vertical signal line 47.

  Next, the gain of the amplifier AMP in each pixel will be described. When a still image is taken, the image sensor 16 is controlled by reading out all pixels, so that the gain of each pixel with respect to the amplifier AMP is 1.

  On the other hand, when shooting a moving image, the image sensor 16 is controlled by addition thinning readout. As described above, in the first embodiment, the image size of the moving image obtained by shooting is reduced to 1/3 times in the horizontal direction and the vertical direction with respect to the image size (W × H) of the still image. , W / 3 × H / 3 image size is set.

  The position of each pixel in the image sensor 16 is represented by R (i, j), G (i + 1, j), G (i, j + 1), B (i + 1, j + 1), and the like. Note that i is a coordinate in the horizontal direction, and j is a coordinate in the vertical direction. On the other hand, the position of each pixel in the acquired moving image is represented by R ′ (x, y), G ′ (x, y), B ′ (x, y), or the like. Note that x is a horizontal coordinate, and y is a vertical coordinate.

  FIG. 4 shows the positional relationship between the pixels of the moving image read by the addition thinning readout and the pixels on the image sensor 16. In FIG. 4, each pixel arranged in the pixel portion 41 of the image sensor 16 is indicated by a thin line, and each pixel of the moving image is indicated by a thick line. The first embodiment weights and adds the signal values of the same color pixels included in the pixel range (hatching range) of 4 rows and 4 columns among the pixels arranged in the pixel unit 41 of the image sensor 16, The case where the signal value of each color component of the pixel of an image is produced | generated is shown. Hereinafter, the pixel range of 4 rows and 4 columns is referred to as a readout range, and reference numeral 60 is given.

  The weighted addition using the signal value of the same color pixel in the reading range 60 is executed so that the barycentric position of each color component in the pixel of the moving image matches the barycentric position in the reading range 60. In FIG. 4, the position indicated by the symbol “◯” is the barycentric position in the reading range 60.

  As a method of matching the barycentric position of each color component in the pixel generated by the addition thinning-out reading with the barycentric position in the reading range, setting the weighting coefficient of each pixel in the weighted addition based on a Gaussian distribution can be mentioned. FIG. 5 is an example of the weight for each pixel set when the pixel range of 4 rows and 4 columns is set as the readout range. The weight of each pixel included in the readout range is set based on a Gaussian distribution with the center of gravity of the readout range 60 as the center.

  As shown in FIG. 6, a case is considered in which a readout range 60 is defined as a pixel range (hatched region) of 4 rows and 4 columns based on the R color pixel in the upper left corner. In the readout range 60, an R color pixel at coordinates (i, j) on the image sensor 16 is set as a reference pixel (hereinafter referred to as a reference pixel). In this case, each signal value of the R color component, the G color component, and the B color component of the pixel located at the coordinates (x, y) in the moving image is expressed by Expression (1). In the 2 × 2 pixel Bayer array, since two G color pixels are included, the G pixel is hereinafter referred to as a Gr pixel and a Gb pixel.

  Here, R ′ (x, y), G ′ (x, y), and B ′ (x, y) indicate the pixels at the coordinates (x, y) in the moving image. In other words, in the case of the readout range 60 using the R color pixel located in the upper left corner as the reference pixel, the gain of the amplifier AMP of each pixel included in the readout range 60 is set so as to satisfy the above-described equation (1).

  Next, as shown in FIG. 7, a case is considered in which the readout range 60 is a pixel range (hatched region) of 4 rows and 4 columns based on the Gr color pixel at the upper left corner. As described above, the pixels of the respective color components arranged in the pixel portion 41 of the image sensor 16 are in a 2 × 2 pixel Bayer array. On the other hand, in the case of a moving image, considering that the size of the image is 1/3 times the reduction rate in each of the horizontal direction and the vertical direction with respect to the size of the image read by all-pixel reading, the reading range 60 is This is the hatched area shown in FIG. Here, among the pixels included in the readout range 60, when the Gr color pixel at the upper left corner, in other words, the Gr pixel at the coordinates (i + 3, j) is used as the reference pixel, the pixel arranged in the i + 3 column is again Read out. In this case, the signal values of the R color component, the G color component, and the B color component of the pixel located at the coordinates (x + 1, y) in the moving image can be expressed by Expression (2).

  That is, in the case of the readout range 60 using the Gr color pixel at the upper left corner as the reference pixel, the gain of the amplifier AMP of each pixel included in the readout range 60 is set so as to satisfy the above-described equation (2).

  Next, as shown in FIG. 8, consider a case where the readout range 60 is a 4 × 4 pixel range (hatched region) using the Gb color pixel in the upper left corner as a reference pixel. In this case, the read range 60 is a hatched area shown in FIG. Here, among the pixels included in the reading range 60, when the Gb pixel at the upper left corner, in other words, the Gb pixel at the coordinates (i, j + 3) is used as the reference pixel, the pixel arranged in the j + 3th row is read again. It is. In this case, the signal values of the R color component, the G color component, and the B color component of the pixel located at the coordinates (x, y + 1) in the moving image can be expressed by Expression (3).

  That is, in the case of the readout range 60 using the Gb color pixel in the upper left corner as the reference pixel, the gain of the amplifier AMP of each pixel included in the readout range 60 is set so as to satisfy the above-described equation (3).

  Finally, as shown in FIG. 9, a case is considered in which the readout range 60 is a 4 × 4 pixel range (hatched region) based on the B color pixel in the upper left corner. In this case, the read range 60 is a hatched area shown in FIG. Among the pixels included in the readout range 60, when the B pixel at the upper left corner, in other words, the B color pixel at the coordinates (i + 3, j + 3) is used as the reference pixel, the pixels arranged in the i + 3 column and the i + 3 row are It is read again. In this case, each signal value of the R color component, the G color component, and the B color component of the pixel located at the coordinates (x + 1, y + 1) in the moving image can be expressed by Expression (4).

  That is, in the case of the readout range 60 using the B color pixel in the upper left corner as the reference pixel, the gain of the amplifier AMP of each pixel included in the readout range 60 is set so that the above-described equation (4) is satisfied.

  Next, a process for reading an image signal from the image sensor 16 will be described. Still image capturing is performed by all-pixel readout, in which charges accumulated in each of a plurality of pixels arranged in the pixel unit 41 of the image sensor 16 are read out for each pixel. In this all-pixel readout, a plurality of pixels are read out row by row. In this all-pixel reading, the vertical scanning circuit 44 does not output the control signal φGAIN. As a result, the gain of the amplifier AMP provided in each pixel becomes 1. Further, the horizontal scanning circuit 45a and the horizontal scanning circuit 45b do not output a control signal for each addition transistor provided in the switching unit 43a. In this all-image reading, the image signal output from the image sensor 16 is an image signal in which the color component of each pixel is any one of R, G, and B color components.

  On the other hand, in moving image shooting, for example, a range of pixels of 4 rows and 4 columns among a plurality of pixels arranged in the pixel unit 41 of the image sensor 16 is read out by weighted addition for each color component as a reading range. Reading is performed.

  First, the vertical scanning circuit 44 outputs a control signal φGAIN to each pixel arranged in four rows included in the readout range. In response to this output, the gain of the amplifier AMP of each pixel is switched from 1 to a gain value at the time of addition thinning readout using weighted addition. Note that the value of the gain to be switched is a value that satisfies any of the above-described equations (1) to (4). Then, the vertical scanning circuit 44 outputs a control signal φSW3. As a result, the voltage signal amplified by the amplifier AMP is output from each pixel included in the readout range to the corresponding vertical signal line. The voltage signal output to the vertical signal line is accumulated in the capacitor CD1. Thereafter, the vertical scanning circuit 44 stops outputting the control signal φSW3.

The horizontal scanning circuit 45a outputs control signals φADD ₁ , φADD ₂ , and φADD ₃ . As a result, the addition transistors ADD ₁ , ADD ₂ , and ADD ₃ are turned on, and the vertical signal lines 47b are short-circuited. As a result, the charge amounts of the capacitors CD1 connected to the vertical signal lines 47b in the 6R-5th, 6R-4th, 6R-3th, and 6R-2th columns are the same. Thereafter, the horizontal scanning circuit 45a stops outputting the control signals φADD ₁ , φADD ₂ , and φADD ₃ . Then, the horizontal scanning circuit 45a outputs a control signal φLINE1. As a result, the switch transistor SW1 connected to the vertical signal line 47b in the 6R-5th column is turned on, and a voltage signal based on the amount of charge accumulated in the capacitor CD1 connected to the corresponding vertical signal line 47b is supplied to the column amplifier CAMP1. Is output. Here, the output voltage signal is, for example, the signal value of the G color component of the pixel at the coordinate (x, y) in the moving image.

The horizontal scanning circuit 45b outputs control signals φADD ₇ and φADD ₈ . When adding transistor ADD ₇ is turned on, 6R-5 column and 6R-4 column vertical signal line 47a is short-circuited. As a result, the charge amount of the capacitor CD2 connected to the vertical signal lines 47a in the 6R-5th column and the 6R-4th column becomes the same. The addition transistor ADD ₈ are the turned on, 6R-4 column and 6R-2 column of the vertical signal line 47a is short-circuited. As a result, the charge amounts of the capacitors CD2 connected to the vertical signal lines 47a in the 6R-4th column and the 6R-2th column become the same. Thereafter, the horizontal scanning circuit 45 b stops outputting the control signals φADD ₇ and φADD ₈ . Then, the horizontal scanning circuit 45b outputs control signals φLINE5 and φLINE6. As a result, the switch transistors SW2 connected to the vertical signal lines 47a in the 6R-5th column and the 6R-4th column are turned on, respectively, and the charge amount accumulated in the capacitor CD2 connected to the corresponding vertical signal line 47a is increased. A voltage signal based thereon is output to the column amplifier CAMP2. Here, the voltage signal output from the vertical signal line 47a in the 6R-5th column becomes, for example, the signal value of the R color component of the pixel at the coordinates (x, y) in the moving image. Further, the voltage signal output from the vertical signal line 47a in the 6R-4th column becomes, for example, the signal value of the B color component of the pixel at the coordinate (x, y) in the moving image.

After this reading, the vertical scanning circuit 44 outputs a control signal φSW3. Further, the horizontal scanning circuit 45a outputs control signals φADD ₄ , φADD ₅ and φADD ₆ . In response to this, the addition transistors ADD ₄ , ADD ₅ , and ADD ₆ are turned on, and the vertical signal lines 47b in the 6R-2, 6R-1, 6R, and 6R + 1 columns are short-circuited. As a result, the charge amount of the capacitor CD2 connected to these vertical signal lines 47a becomes the same. Thereafter, the horizontal scanning circuit 45a stops outputting the control signal control signals φADD ₄ , φADD ₅ , φADD ₆ . Then, the horizontal scanning circuit 45a outputs a control signal φLINE4. As a result, the switch transistor SW1 connected to the vertical signal line 47b in the 6R-2th column is turned on, and a voltage signal based on the amount of charge accumulated in the capacitor CD1 connected to the corresponding vertical signal line 47b is supplied to the column amplifier CAMP1. Is output. Here, the output voltage signal is, for example, the signal value of the G color component of the pixel at the coordinate (x + 1, y) in the moving image.

The horizontal scanning circuit 45b outputs control signals φADD ₉ and φADD ₁₀ . When the addition transistor ADD ₉ is turned on, the vertical signal lines 47a in the 6R-2 column and the 6R column are short-circuited. As a result, the charge amounts of the capacitors CD2 connected to the vertical signal lines 47a in the 6R-2 column and the 6R column become the same. The addition transistor _{ADD 10} is the turned on, 6R-1 column and 6R + 1 column of the vertical signal line 47a is short-circuited. As a result, the charge amounts of the capacitors CD2 connected to the vertical signal lines 47a in the 6R-1 and 6R + 1 columns are the same.

Thereafter, the horizontal scanning circuit 45b stops outputting the control signals φADD ₉ and φADD ₁₀ . Then, the horizontal scanning circuit 45b outputs control signals φLINE8 and φLINE5. As a result, the switch transistors SW2 connected to the vertical signal lines 47a of the 6R-2 column and the 6R-1 column are respectively turned on, and the charge amount accumulated in the capacitor CD2 connected to the corresponding vertical signal line 47a is increased. A voltage signal based thereon is output to the column amplifier CAMP2. Here, the voltage signal output from the vertical signal line 47a of the 6R-2th column becomes, for example, the signal value of the B color component of the pixel at the coordinate (x + 1, y) in the moving image. Further, the voltage signal output from the vertical signal line 47a in the 6R-1 column becomes, for example, the signal value of the R color component of the pixel at the coordinate (x + 1, y) in the moving image.

  By performing the above-described operation, the readout of the pixel signal is performed while using the range of 4 rows and 4 columns as the readout range and moving the readout range by 3 pixels in the column direction. Then, after the target row is read, the above-described reading is performed again by shifting the range of 4 rows and 4 columns by 3 pixels in the row direction.

  As described above, when a moving image is captured, addition thinning-out readout by weighted addition is performed for each identical color component included in the readout range, so that the image signal read out from the image sensor 16 is an R color component, G The image signal has all the color components of the color component and the B color component. In this case, the data amount of the read image signal is 1/3 × 1/3 × 3 = 1/3 times the data amount of the image signal read by all-pixel reading, and high-speed reading is performed. be able to.

  As described above, the image signal read from the image sensor 16 at the time of capturing a moving image is an image signal having all the color components of the R color component, the G color component, and the B color component. There is no need to obtain signal values of color components that are insufficient in the read image signal by color interpolation processing. In addition, by reading the signal values of pixels of the same color included in the readout range in a weighted state, it is possible to prevent the occurrence of moire and false colors in the readout image and impair the sense of resolution in the readout image. You do n’t have to.

  In the first embodiment described above, the center of gravity position of each color component of each pixel of the moving image is obtained by setting the gain of the amplifier AMP of the pixel included in the readout range to a value based on the weight set based on the Gaussian distribution. Can coincide with the position of the center of gravity in the reading range. However, it is not necessary to set the gain of the amplifier AMP of the pixel included in the readout range to a value based on the weight set based on the Gaussian distribution. For example, the weight for each pixel included in the readout range can be set to 1. Hereinafter, the case where the weight for the pixels included in the readout range is set to 1 will be described as the second embodiment.

Second Embodiment
In the second embodiment, as in the first embodiment, the image size of the moving image obtained by shooting is set to 1/3 times the horizontal size and the vertical direction of the still image size. The case where it will be described. In this case, since the configuration of the image sensor 16 is the same as that of the first embodiment, the configuration of the image sensor 16 is omitted here.

As described above, in the second embodiment, the gain of the amplifier AMP of each pixel included in the read range 60 is set with a weight of 1 for the pixels included in the read range 60 being set. For example, as shown in FIG. 6, the signal values of the R color component, G color component, and B color component of the pixel at the coordinate (x, y) in the moving image are at the coordinate (i, j) on the image sensor 16. It is obtained by weighted addition for each pixel of the same color using pixels included in the reference range 60 with a certain R color pixel as the upper left corner.
Here, when the R color pixel at the coordinates (i, j) on the image sensor 16 is used as the reference pixel, the R color component, the G color component of the pixel at the coordinates (x, y) in the moving image, Each signal value of the B color component can be expressed by equation (5).

ここで、（５）式に示すように、動画像において、座標（ｘ，ｙ）にある画素における各色成分の信号値は、図６に示す読出範囲６０に含まれる同色画素の信号値を単純加算平均した値となる。

Here, as shown in the equation (5), in the moving image, the signal value of each color component in the pixel at the coordinate (x, y) is simply the signal value of the same color pixel included in the readout range 60 shown in FIG. It is the value obtained by averaging.

  Further, when reading the signal values of the R color component, G color component, and B color component of the pixel at the coordinate (x, y) in the moving image, the target readout is performed so that the above-described equation (5) is satisfied. The gain of each amplifier AMP of the R color pixel, G color pixel, and B color pixel included in the range is set.

  Similarly, when reading the signal values of the R color component, G color component, and B color component of the pixel at the coordinates (x + 1, y) in the moving image, the same color pixel included in the readout range 60 shown in FIG. Read the signal value by addition thinning readout. Furthermore, it is also possible to read out the signal values of the R color component, G color component, and B color component of the pixel at the coordinate (x, y + 1) and the pixel at the coordinate (x + 1, y + 1) in the moving image. The signal value of the same color pixel included in the readout range to be read is read by addition thinning readout. Also in these cases, the signal value of each color component is read out by addition thinning-out readout using simple addition averaging of pixels of the same color included in the readout range 60.

  In this case, there arises a problem that the barycentric positions of the color components do not coincide with each other in each pixel in the moving image. FIG. 10 shows a case where the center of gravity of the R color component in each pixel of the moving image is “Δ”, the center of gravity of the G color component is “◯”, and the center of gravity of the B color component is “×”. In FIG. 10, thick lines indicate moving image pixels, solid lines indicate readout range 60, and thin lines indicate each pixel of the pixel portion 41 of the image sensor 16. As shown in FIG. 10, the center of gravity of the G color component is located at the center of each pixel of the moving image, whereas the center of gravity of the R color component and the center of gravity of the B color component are respectively the moving image. The center of gravity of the R color component and the center of gravity of the B color component are also shifted. In addition, the centroid position of the R color component and the centroid position of the B color component are different for each pixel. Such a shift in the position of the center of gravity appears as a false color in a still image and causes image deterioration.

  Here, in consideration of moving images, moving images obtained by shooting are generally stored by converting the color space format from the RGB color space to the YUV420 color space. That is, when this color space is converted, a centroid position when reading out a moving image is generated by generating a color difference signal in the YUV420 color space at a location where the centroid positions of the R color component, the G color component, and the B color component match. Correct the deviation.

  Hereinafter, a process of converting the color space format of the obtained moving image from the RGB color space to the YUV420 color space will be described. This process is executed as a color space conversion process in the image processing circuit 23.

  Here, the YUV420 color space is a color space that takes 4 pixels of luminance signals (Y signals) and 1 pixel of color difference signals (U signals and V signals) for 2 × 2 pixels. First, the image processing circuit 23 generates each signal value of the R color component, the G color component, and the B color component for conversion into the YUV color space using the equation (6).

そして、画像処理回路２３は、（６）式により求めた値を用いて色空間変換処理を実行する。ここで、輝度信号（Ｙ信号）については、動画像における座標（ｘ，ｙ）、座標（ｘ＋１，ｙ）、座標（ｘ，ｙ＋１）、座標（ｘ＋１，ｙ＋１）にある各画素のＲ’色成分、Ｇ’色成分、Ｂ’色成分の信号値を用いて色空間を変換する。色差信号（Ｕ信号及びＶ信号）については、（６）式にて求めたＲ”色成分、Ｇ”色成分、Ｂ”色成分を用いて色空間を変換する。なお、色空間を変換する式は、下記の（７）式となる。

Then, the image processing circuit 23 executes the color space conversion process using the value obtained by the equation (6). Here, regarding the luminance signal (Y signal), the R ′ color of each pixel at the coordinates (x, y), coordinates (x + 1, y), coordinates (x, y + 1), and coordinates (x + 1, y + 1) in the moving image. The color space is converted using the signal values of the component, G ′ color component, and B ′ color component. For the color difference signals (U signal and V signal), the color space is converted using the R ″ color component, the G ″ color component, and the B ″ color component obtained by the equation (6). The equation is the following equation (7).

  By performing this color space conversion processing, the barycentric position of the color component of each pixel of the moving image converted into the YUV420 color space becomes the position of the coordinates (i + 3, j + 3) of the image sensor 16 (moving image coordinates (x + 0. 5, y + 0.5)). Note that the position indicated by “☆” in FIG. 10 is the barycentric position of each pixel of the moving image converted into the YUV420 color space.

  Thus, in the case of addition thinning readout using simple addition averaging of pixels of the same color included in the readout range, the barycentric position of each color component is different in each pixel of the moving image to be read out. In such a case, during the color space conversion process executed by the image processing circuit 23, the color space format in the moving image is converted so that the gravity center positions of the respective color components coincide with each other, thereby preventing the displacement of the gravity center position. As a result, it is possible to prevent the occurrence of false colors in moving images.

  In the second embodiment, as in the first embodiment, the data amount of the image signal read from the image sensor 16 is 1/3 × 1/3 × 3 = 1/3 times the data amount of the still image. Therefore, it becomes possible to perform reading at high speed.

  In the second embodiment, the same effect as that of the first embodiment can be obtained by performing the color space conversion process by the image processing circuit 23 on the image signal read from the image sensor 16. Can do.

  In the first embodiment and the second embodiment described above, the image size of the acquired moving image is 1/3 times the horizontal size and 1/3 times the vertical size of the still image size. However, the present invention is not limited to this, and the image size of the moving image is an image size that is 1/4 times in the horizontal direction and 1/4 times in the vertical direction with respect to the image size of the still image. Also good.

  Hereinafter, the case where the image size of the moving image is 1/4 times the horizontal size and 1/4 times the vertical image size is described as the third embodiment. .

<Third Embodiment>
In the third embodiment, the configuration of a switching unit for reading G color pixels and a switching unit for reading R color pixels or B color pixels are different from those of the first embodiment. Hereinafter, the switching unit for reading the G color pixel will be denoted by reference numeral 43a ′, and the switching unit for reading the R color pixel or B color pixel will be denoted by reference numeral 43b ′. Since other configurations are the same as those in the first embodiment, the same reference numerals as those in the first embodiment are used for explanation.

As shown in FIG. 11, the switching unit 43a ′ includes the capacitor CD3, the addition transistors ADD ₁₁ , ADD ₁₂ , ADD ₁₃ and the switching transistor SW4, as well as the control signals φADD ₁₁ , φADD ₁₂ , φADD ₁₃ and the control signals φLINE9, φLINE10, It consists of seven row signal lines for supplying φLINE11 and φLINE12. Here, the addition transistor ADD ₁₁ is turned on when the control signal φADD ₁₁ is output, and the vertical signal line 47b in the 4T-3 (T = 1, 2, 3,...) Column and the 4T-2 column. The vertical signal line 47b is short-circuited. The addition transistor _{ADD 12} is turned on by the control signal FaiADD ₁₂ is outputted, 4T-2 column of the vertical signal line 47b and 4T-1 column of the vertical signal line 47b are short-circuited. The addition transistor ADD ₁₃ is turned on when the control signal φADD ₁₃ is output, and the vertical signal line 47b in the 4T-1 column and the vertical signal line 47b in the 4T column are short-circuited.

Switching unit 43 b 'includes a capacitor CD4, an addition transistor _ADD 14, _{ADD 15,} other switching transistor SW5, control signals φADD _14, φADD ₁₅ and the control signal φLINE13, φLINE14, φLINE15, 6 pieces of the row signal line for supplying a φLINE16 It consists of. Here, the addition transistor ADD ₁₄ is turned on when the control signal φADD ₁₄ is output, and the 4U-3 (U = 1, 2, 3,...) Column vertical signal line 47a and the 4U-1 column. The vertical signal line is short-circuited. The addition transistor ADD ₁₅ is turned on when the control signal φADD ₁₅ is output, and the vertical signal line 47a in the 4U-2 column and the vertical signal line in the 4U column are short-circuited.

  As described above, the third embodiment aims to set the image size of the moving image to 1/4 times the horizontal size and 1/4 times the vertical size relative to the still image size. ing. In other words, the third embodiment is the same as the first embodiment in that the pixel range of 4 rows and 4 columns is read out, but in the third embodiment, 4 pixels are arranged in the horizontal direction or the vertical direction. This is different from the first embodiment in that each color component of the pixel in the moving image is read out while shifting the pixel by pixel. FIG. 12 shows the relationship between the position of each pixel on the image sensor 16 and the position of each pixel in the moving image. Here, the range of 4 rows and 4 columns indicated by hatching is the read range 60. In addition, the readout range and the pixels of the moving image match. In this case, the arrangement of each pixel included in the readout range 60 is the same pixel arrangement.

  Also in this case, the weighting coefficient of each pixel in the weighted addition is set using a weight based on the Gaussian distribution shown in FIG. As described above, since the arrangement of each pixel included in the readout range is the same regardless of the readout position, the gain of the amplifier AMP of each pixel is set in consideration of the following equation (8). The

  Here, R ′ (x, y), G ′ (x, y), and B ′ (x, y) are signal values of the respective color components of the pixel at the coordinate (x, y) in the moving image.

  Next, processing when an image signal is read out from the image sensor 16 when capturing a moving image will be described. First, the vertical scanning circuit 44 outputs a control signal φGAIN to each pixel arranged in the four rows of interest. In response to this output, the gain of the amplifier AMP of each pixel is switched from 1 to a predetermined value. Then, by outputting the control signal φSW3, the voltage signal amplified by the amplifier AMP is output from each pixel to the corresponding vertical signal line. The voltage signal output to the vertical signal line is accumulated in the capacitor CD1. Thereafter, the vertical scanning circuit 44 stops outputting the control signal φSW3.

Thereafter, the horizontal scanning circuit 45a outputs control signals φADD ₁₁ , φADD ₁₂ , and φADD ₁₃ . As a result, the addition transistors ADD ₁₁ , ADD ₁₂ , and ADD ₁₃ are turned on, and the vertical signal lines 47b in the 4T-3rd, 4T-2th, 4T-1th, and 4Tth columns are short-circuited. As a result, the charge amounts of the capacitors CD3 connected to the vertical signal lines 47b in the 4T-3rd, 4T-2th, 4T-1th, and 4Tth columns become the same. Thereafter, the horizontal scanning circuit 45a stops outputting the control signals φADD ₁₁ , φADD ₁₂ , φADD ₁₃ . Then, the horizontal scanning circuit 45a outputs a control signal φLINE9. As a result, the switch transistor SW4 connected to the vertical signal line 47b in the 4T-3th column is turned on, and a voltage signal based on the amount of charge accumulated in the capacitor CD3 connected to the corresponding vertical signal line 47b is supplied to the column amplifier CAMP1. Is output. Here, the voltage signal output from the column amplifier CAMP1 becomes a signal of the G color component of the pixel located at the coordinates (x, y) in the moving image (see FIG. 11).

At the same time, the horizontal scanning circuit 45b outputs control signals φADD ₁₄ and φADD ₁₅ . As a result, the addition transistors ADD ₁₄ and ADD ₁₅ are turned on. When the addition transistor ADD ₁₄ is turned on, the vertical signal lines 47a in the 4U-3 column and the 4U-1 column are short-circuited. As a result, the charge amount of the capacitor CD4 connected to each vertical signal line 47b in the 4U-3 column and the 4U-1 column becomes the same. Further, when the addition transistor ADD ₁₄ is turned on, the vertical signal lines 47a in the 4U-2 column and the 4U column are short-circuited. As a result, the charge amount of the capacitor CD4 connected to each vertical signal line 47a in the 4U-2 column and the 4U column becomes the same. Thereafter, the horizontal scanning circuit 45a stops outputting the control signals φADD ₁₄ and φADD ₁₅ . The horizontal scanning circuit 45a outputs control signals φLINE13 and φLINE14.

  By outputting the control signal φLINE13, the switch transistor SW5 connected to the vertical signal line 47b in the 4U-3th column is turned on, and the charge amount accumulated in the capacitor CD4 connected to the corresponding vertical signal line 47b is increased. A voltage signal based thereon is output to the column amplifier CAMP1. Here, the output voltage signal is a signal of the R color component of the pixel at the coordinate (x, y) in the moving image.

  Further, the output of the control signal φLINE14 turns on the switch transistor SW4 connected to the vertical signal line 47b in the 4U-2 column, and the charge accumulated in the capacitor CD4 connected to the corresponding vertical signal line 47b. A voltage signal based on the quantity is output to the column amplifier CAMP1. Here, the output voltage signal is a signal of the B color component of the pixel at the coordinate (x, y) in the moving image.

  By performing the above-described processing, addition thinning-out readout using the signal value of each pixel arranged in the pixel unit 41 of the image sensor 16 is repeated. As described above, when a moving image is shot, the addition signal is read out for each same color pixel included in the reference range with the range of 4 rows and 4 columns as the reference range. Each pixel is an image signal having all the R, G, and B color components. In this case, the data amount of the read image signal is 1/4 × 1/4 × 3 = 3/16 times the data amount of the image signal read by all-pixel reading, and high-speed reading is performed. It becomes possible. In this case, the same effect as that of the first embodiment can be obtained.

  In the third embodiment, addition thinning readout is performed so that the image size of the still image is 1/4 times in the horizontal direction and 1/4 times in the vertical direction. Also in this case, the gain of the amplifier AMP of each pixel is set using a ratio based on a Gaussian distribution. In this case, since the full width at half maximum of the Gaussian distribution is smaller than the reduction magnification, aliasing distortion tends to occur in the read image. In order to suppress the occurrence of such aliasing distortion, it is possible to set the reading range when performing addition thinning-out reading to be larger than the pixel range of 4 rows and 4 columns. Hereinafter, the case where the reading range when performing the addition thinning-out reading is set to be larger than the pixel range of 4 rows and 4 columns will be described as a fourth embodiment.

<Fourth embodiment>
In the fourth embodiment, a case will be described in which the readout range when performing addition thinning readout is a 6 × 6 pixel range. FIG. 13 shows the relationship between each pixel in the image sensor 16 and each pixel in the moving image. In FIG. 13, a 6-by-6 pixel range (a range surrounded by a thick dotted line) where addition thinning readout is performed is indicated by a hatched range 61. Further, each pixel of the moving image is indicated by a thin line, and pixels arranged in the pixel portion 41 of the image sensor 16 are indicated by thin lines. Also in this case, the gain of the amplifier AMP of each pixel is set so that the barycentric position in the 6 × 6 pixel range where addition thinning readout is performed matches the barycentric position of each color component in each pixel of the read moving image. To do.

Also in this case, the value of the weight of each pixel included in the readout range is a value set based on the Gaussian distribution. FIG. 14 shows the weight of each pixel set based on the Gaussian distribution when the pixel range of 6 rows and 6 columns is set as the readout range. Based on this weight, the gain of the amplifier AMP of each pixel included in the readout range 61 is set.
For example, each signal value of the R color component, the G color component, and the B color component of the pixel located at the coordinates (x, y) in the moving image can be expressed by Expression (9).

  As shown in FIG. 13, the pixels arranged in the i + 4th column, the i + 5th column, the i + 8th column, the i + 9th column, and the pixels arranged in the j + 4th row, the j + 5th row, the i + 8th column, and the i + 9th column are , Read out redundantly. In addition, these pixels have different weighting coefficients used for reading. In this case, although not shown, the gain is adjusted according to the position of the pixel with respect to the readout range by using an amplifier capable of adjusting the amplifier AMP provided in each pixel to a plurality of gains. Alternatively, a plurality of amplifiers having different gains are provided for each pixel, and switching is made so that one of the amplifiers is used according to the position of the pixel with respect to the readout range.

  By performing such processing, the same effects as those of the third embodiment can be obtained. In addition, it is possible to suppress the occurrence of aliasing distortion by setting the reading range used for the addition thinning-out reading in advance to be larger than the pixel range based on the reduction ratio.

  In the fourth embodiment, the gain of the amplifier AMP provided in each pixel is set based on the weight of each pixel when a Gaussian distribution is applied to a 6 × 6 pixel range. The weight of each pixel in the readout range is not limited to the weight (horizontal / vertical direction ratio 1: 3: 4: 4: 3: 1) shown in FIG. 14, but the weight (horizontal / vertical direction) shown in FIG. Ratio 1: 4: 7: 7: 4: 1) and weights shown in FIG. 16 (horizontal / vertical direction ratio 1: 5: 10: 100: 5: 1) can also be used.

  In the third embodiment and the fourth embodiment described above, a case where the image size of the moving image is 1/4 times the horizontal size and 1/4 times the vertical size relative to the still image size. Show. However, for example, in an imaging apparatus equipped with an imaging element with an aspect ratio of 3: 2, when the image size satisfies the horizontal pixel number 1920 necessary for HD video, the number of pixels of the imaging element is horizontal. (1920 × 4) in the vertical direction (1920 × 2/3 × 4) is 37.5M in total. Here, when the number of pixels of the image sensor mounted on the imaging device does not satisfy 37.5 M pixels, the moving image read by the addition thinning readout does not satisfy the image size in the HD moving image. In such a case, it is possible to perform super-resolution processing on the obtained moving image.

  In the first and second embodiments described above, the image size of the moving image is set to an image size that is 1/3 times in the horizontal direction and 1/3 times in the vertical direction relative to the image size of the still image. It explains about. In the third and fourth embodiments, the image size of the moving image is set to 1/4 times the horizontal size and 1/4 times the vertical size relative to the still image size. It explains about. However, the reduction ratio in the horizontal direction and the reduction ratio in the vertical direction are just examples, and these reduction ratios may be set as appropriate. In each of the above-described embodiments, the horizontal reduction ratio and the vertical reduction ratio are set to the same reduction ratio. However, the present invention is not limited to this, and the horizontal reduction ratio and the vertical reduction ratio are not limited thereto. It is also possible to use different reduction ratios.

  In the first to fourth embodiments described above, the color filters arranged in each pixel of the image sensor are R color, G color, and B color. However, the present invention is not limited to this, and cyan (C) , Yellow (Y) and magenta (M) color filters.

  In the first to fourth embodiments described above, a Bayer array in which R color pixels, G color pixels, and B color pixels are arranged in 2 rows and 2 columns is used. However, the present invention is not limited to this, and other arrays may be used. It is also possible to arrange pixels of each color component.

  In the first to fourth embodiments described above, the description has been made on the assumption that addition thinning readout is performed at the time of moving image shooting. However, the present invention is not limited to this. It is also possible to use any method of the fourth embodiment.

  In the first to fourth embodiments described above, the image pickup apparatus has been described as an example. However, the present invention is not limited to this and may be an image processing apparatus. As illustrated in FIG. 17, the image processing device 60 includes a reduced image generation unit 61. For example, a moving image read from the image sensor by reading all pixels is input to the image processing apparatus 60 directly or via a buffer memory or a storage medium. The reduced image generation unit 61 generates a reduced image using the equations (1) to (4) for the input image. Alternatively, the reduced image generation unit 61 generates a reduced image using the expression (5) for the input image. Alternatively, a reduced image is generated using Expression (8s) or Expression (9). Note that when a reduced image is generated using Expression (5), processing for converting the color space format using Expression (6) and Expression (7) is also executed.

  In addition to the image processing apparatus, an image processing program may be used. In this case, any image processing program that can execute the processing of the flowchart shown in FIG.

  Hereinafter, a processing procedure in the image processing program will be described.

  Step S101 is a process for inputting a moving image. By performing this process, for example, the computer accepts a moving image read by all-pixel reading.

  Step S102 is a reduced image generation process. Reduced image generation processing is performed on the working image received in step S101. Note that this reduced image generation processing is executed using any one of Expressions (1) to (4), (5), (8), and (9). By using one of these equations, a reduced moving image is generated. In addition, when the expression (5) is used, the process of converting the color space format using the expressions (6) and (7) is also executed.

  Step S103 is a process of determining whether or not the reduced image generation process is completed. The CPU of the computer that executes the image processing program executes the determination process in step S103 depending on whether or not the reduced image generation process in step S102 is completed. If the reduced image generation process has been completed, the determination process in step S103 is Yes, and the process proceeds to step S104. On the other hand, if the reduced image generation process has not ended, the determination process in step S103 is No, and the process returns to step S102.

  Step S104 is processing for outputting a reduced moving image. The CPU of the computer described above outputs the reduced moving image to, for example, a storage medium.

  The image processing program is preferably stored in a computer-readable storage medium such as a memory card, a magnetic disk, or an optical disk.

DESCRIPTION OF SYMBOLS 10 ... Imaging device, 16 ... Imaging device, 41 ... Pixel part, 42a, 42b ... Precharge part, 43a, 43b ... Switching part, 44 ... Vertical scanning circuit, 45a, 45b ... Horizontal scanning circuit

Claims (6)

A first pixel having a first light receiving portion for receiving light of a first color and a first amplification portion for amplifying a signal from the first light receiving portion;
A second pixel having a second light receiving portion for receiving light of the second color and a second amplification portion for amplifying a signal from the second light receiving portion;
According to the distance from the first position to the first pixel and the distance from the first position to the second pixel, the amplification factor of the first amplification unit and the amplification factor of the second amplification unit are obtained. An amplification control unit to be set;
A circuit for adding signals from the plurality of first pixels and adding signals from the plurality of second pixels;
An imaging apparatus comprising: The imaging device according to claim 1,
The amplification control unit sets the amplification factor of the first amplification unit to be high when the distance from the first position to the first pixel is short, and the distance from the first position to the second pixel is set to be high. An imaging device that sets the amplification factor of the second amplifying unit high when the length is short . A first pixel having a first light receiving portion for receiving light of a first color and a first amplification portion for amplifying a signal from the first light receiving portion;
A second pixel having a second light receiving portion for receiving light of the second color and a second amplification portion for amplifying a signal from the second light receiving portion;
A third pixel having a third light receiving portion for receiving light of a third color and a third amplification portion for amplifying a signal from the third light receiving portion;
The distance between the first pixel from the first position, the distance from the first position to the second pixel, and the distance from the third position to the third pixel. An amplification control unit for setting an amplification factor, an amplification factor of the second amplification unit, and an amplification factor of the third amplification unit;
A circuit that adds signals from the plurality of first pixels, adds signals from the plurality of second pixels, and adds signals from the plurality of third pixels;
An imaging apparatus comprising: The imaging device according to claim 3 .
The amplification control unit sets the amplification factor of the first amplification unit to be high when the distance from the first position to the first pixel is short, and the distance from the first position to the second pixel is set to be high. An imaging apparatus that sets the amplification factor of the second amplification unit high when the length is short, and sets the amplification factor of the third amplification unit high when the distance from the first position to the third pixel is short . In the imaging device according to claim 3 or 4 ,
The imaging device, wherein the first color light is green light, the second color light is red light, and the third color light is blue light . The imaging apparatus according to claim 5,
The imaging device in which the plurality of first pixels, the plurality of second pixels, and the plurality of third pixels are arranged in a Bayer array .

Images