JP2005303687A - Smear removing circuit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a smear removing circuit calculating information electric charge amount by calculating smear electric charge amount, even with a solid-state imaging element in which the information electric charge is mixed at prescribed mixing pixel unit numbers in transfer direction. <P>SOLUTION: A subtractor 11 subtracts smear data T2 (n) from a first image data S1 (n) to output a second image data S2 (n). An adder 12 adds the second image data S2 (n) to cumulative data T1 (n) from a line memory 13, which is supplied to the line memory 13. A factor generator 16 generates a prescribed factor m×k that is determined by a mixing pixel unit number m, and an exposure time Ts at each light receiving bit of the solid-state imaging element, which is supplied to a multiplier 14. The multiplier 14 multiplies the cumulative data T1 (n) read out of the line memory 13 with a prescribed factor m×k to generate the smear data T2 (n) representing smear component. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a smear removing circuit that removes smear generated due to a vertical transfer operation of a solid-state image sensor from an image signal extracted from the solid-state image sensor.

  In an imaging apparatus using a solid-state imaging device, an exposure control unit is provided so as to keep the exposure state of the solid-state imaging device optimal. As this exposure control means, a mechanical aperture mechanism that controls the amount of light incident on the solid-state image sensor according to the brightness of the subject, or a so-called electronic shutter that controls the exposure time of the solid-state image sensor according to the brightness of the subject. Etc. are known.

  FIG. 5 is a block diagram illustrating a configuration of an imaging apparatus using a frame transfer type CCD solid-state imaging device.

  The solid-state imaging device 1 includes an imaging unit 1i, a storage unit 1s, a horizontal transfer unit 1h, and an output unit 1d. The imaging unit 1i and the storage unit 1s include a plurality of shift registers that are arranged in parallel to each other and are continuous in the vertical (column) direction. Each bit of the shift register of the imaging unit 1i forms a plurality of light receiving bits to form a light receiving unit, and information light generated corresponding to the subject image is stored in each light receiving bit. When the set exposure time has elapsed, frame transfer is performed in the vertical direction from the imaging unit 1i to the storage unit 1s. Since the accumulating unit 1s is covered with a light-shielding film and prevents generation of charges due to the incidence of light, the signal charges from the imaging unit 1i that have been frame-transferred can be held as they are. The horizontal transfer unit 1h is composed of a single shift register in which each output of the plurality of shift registers of the storage unit 1s is connected to each bit, and the information charge for one screen stored in the storage unit 1s is in units of one row. Are sequentially transferred in the horizontal (row) direction and output. The output unit 1d includes an electrically independent capacitor and an amplifier that extracts a change in the potential of the capacitor. The information charge output from the horizontal transfer unit 1h is received by the capacitor in units of 1 bit and converted into a voltage value. The image signal Y0 (t) is output.

  The analog signal processing circuit 2 takes in the image signal Y0 (t) output from the solid-state imaging device 1, performs processing such as sample hold, AGC (automatic gain control), and the like as an image signal Y1 (t) according to a predetermined format. Output. The A / D conversion circuit 3 converts the image signal Y1 (t) output from the analog signal processing circuit 2 into digital data in synchronization with the processing operation of the analog signal processing circuit 2 (output operation of the solid-state imaging device 1). Then, image data D1 (n) corresponding to each light receiving bit of the solid-state imaging device 1 is generated. The digital signal processing circuit 4 takes in the image data D1 (n), performs contour correction, integration processing in units of one screen, and in the case of color imaging, performs processing such as color balance control and filtering, Output as new image data D2 (n). The image data D2 (n) is converted into an analog value by a D / A conversion circuit and transferred to a display device, or is recorded on a recording medium as it is.

  In the solid-state imaging device 1 of the frame transfer method, when transferring the information charges accumulated in each light receiving bit of the imaging unit 1i to the storage bits of the storage unit 1s, the light receiving bits that always generate charges by receiving light. I let it pass. For this reason, unnecessary charges, that is, smear charges, are mixed into the information charges in the process of being transferred in the light receiving bit, and smear is generated on the reproduction screen. For example, as shown in FIG. 6, when a bright spot light is applied to a part of the imaging unit 1 i, when the information charge stored in each light receiving bit is transferred to the storage bit of the storage unit 1 s, the spot light is applied. A large amount of smear charge is mixed with the information charge transferred through the portion. As a result, smear is likely to occur at a position below the bright subject on the reproduction screen displaying the image signal obtained from the solid-state imaging device 1.

  FIG. 7 is a block diagram showing a configuration of a conventional smear removing circuit. The smear removing circuit performs arithmetic processing on the amount of information charges accumulated in each light receiving bit of the imaging unit of the solid-state imaging device as digital image data. In the imaging apparatus shown in FIG. 4 is added to the input side.

  The smear removing circuit 30 removes smear components generated due to the time difference of vertical transfer in the solid-state image sensor. That is, a smear removing circuit is provided to remove a smear component generated due to a difference in transfer path of information charges accumulated in each light receiving bit of the solid-state imaging device.

  The smear removing circuit 30 includes a subtractor 11, an adder 12, a line memory 13, a multiplier 14, and a coefficient generator 36. The subtractor 11 subtracts the smear data T2 (n) from the first image data S1 (n) that is continuously input in units of one row, and obtains second image data S2 (n) that does not include a smear component. Output. The adder 12 adds the second image data S2 (n) output from the subtractor 11 and the cumulative addition data T1 (n) read from the line memory 13, and supplies the added data to the line memory 13. . The line memory 13 is reset every time input of the first image data S1 (n) for one screen is completed, and stores the addition data input from the adder 12 for each row. As a result, the second image data S2 (n) of one screen is cumulatively added in each column in the adder 12, and the cumulative addition data T1 (n) is stored in the line memory 13.

  The multiplier 14 multiplies the cumulative addition data T1 (n) read from the line memory 13 by a predetermined coefficient k determined by the exposure time Ts to generate smear data T2 (n) representing a smear component. Then, the coefficient generator 36 generates a coefficient k corresponding to the exposure time for each light receiving bit of the solid-state image sensor in response to the exposure time Ts representing the exposure state of the solid-state image sensor, and supplies the coefficient k to the multiplier 14.

  According to such a smear removing circuit 30, the smear charge mixed from each light receiving bit is sequentially accumulated every time the information charge accumulated in each light receiving bit of the solid-state imaging device is transferred in the vertical direction one row at a time. The Then, since the smear component mixed in the process of transferring information charges is represented by the cumulative addition value, the smear component can be removed by subtracting this value from the first image data S1 (n).

  The coefficient generation circuit 36 receives an exposure time Ts for each frame, holds the same data for at least one vertical scanning period, and sets the same coefficient k during that period.

  Next, the principle of smear removal in the smear removal circuit 30 will be described.

  FIG. 8 is a diagram for explaining the principle of smear removal by the smear removal circuit 30 and shows the output charge amount Vi, j for each row of the light receiving bits of the solid-state imaging device. This figure shows a case where the amount of light received by each light receiving bit is uniform.

  In the case of a frame transfer type solid-state imaging device, the output charge amount Vi, j obtained from the light receiving bit of i row and j column is expressed by the following equation, assuming that the accumulated charge amount per unit time of the light receiving bit is Ei, j (t). Given by.

  Here, t1 and t2 are the exposure start time and the exposure end time of the information charge in each light receiving bit, and Δt is the information charge accumulated in each light receiving bit passes through one light receiving bit during frame transfer. It takes time. In Equation 1, the first term on the right side indicates the amount of information charge accumulated in one light receiving bit during a predetermined exposure time Ts (= t2-t1), and the second term is accumulated in one light receiving bit. It shows the amount of smear charge that is mixed in the process in which the transferred information charge is transferred from the imaging unit to the storage unit in a frame. That is, the information charge amount stored in each light receiving bit is represented by a value obtained by integrating the accumulated charge amount Ei, j (t) per unit time of the light receiving bit from time t1 to time t2. The amount of charge generated by the received light bit while the information charge passes through one received bit during frame transfer is represented by a value obtained by integrating the accumulated charge amount Ei, j (t) per unit time by Δt. The smear charge amount is represented by a value obtained by adding this integral value by the amount of the received light bit in the transfer path.

  Here, assuming that the time from time t1 to time t2 and further from time t2 + nΔt is short and the amount of incident light during that time does not change, the accumulated charge amount Ei, j (t) per unit time of the received light bit is constant. A value (this value is assumed to be Ei, j), and the calculation result of Equation 1 is Equation 2.

  Further, if the information charge amount accumulated in each light receiving bit from time t1 to time t2, that is, Ts · Ei, j is replaced with Si, j, Expression 2 is transformed into Expression 3.

  Therefore, the information charge amount Si, j that is truly accumulated in each light receiving bit is given by Equation 4.

  That is, the value obtained by subtracting the smear charge amount from the output charge amount Vi, j obtained from each light receiving bit of the solid-state imaging device is the information charge amount Si, j accumulated in each light receiving bit.

The smear removing circuit 30 executes the calculation of the above-described equation 4, the Σ operation of the second term on the right side is executed by the combination of the adder 12 and the line memory 13, and the coefficient of the second term on the right side is executed by the multiplier 14. k multiplication is performed. Then, the calculation result obtained from the multiplier 14 is subtracted from the first image data S1 (n) corresponding to the first term on the right side by the subtractor 11, whereby the calculation of Expression 4 is completed. Therefore, the smear component generated due to the vertical transfer of information charges in the solid-state image sensor is removed.

Japanese Patent No. 3157455

  In recent years, in digital cameras and other mobile terminals with camera functions, the number of pixels of a solid-state imaging device has been significantly improved compared to the number of pixels of a preview monitor. For this reason, when recording an image on a recording medium such as a memory, a high-resolution image corresponding to the number of pixels of the solid-state image sensor is captured, but at the time of preview, capturing with a small number of pixels corresponding to the number of pixels of the preview monitor is performed. Is enough. Also, flash photography is often not performed during preview in order to reduce power consumption. Under such circumstances, when the subject is dark, such as shooting at night or indoors, the amount of information charge accumulated in each pixel is small, so that the information charge is mixed between adjacent pixels.

  FIG. 9 is an example of a schematic cross-sectional view and a potential diagram in the transfer direction at the boundary between the imaging unit 1i and the storage unit 1s, and a solid-state imaging device that mixes information charges at the boundary between the imaging unit 1i and the storage unit 1s. It is. By repeating the timing T0 (T9) to T8, the three-phase frame transfer clocks φf1 to φf3 are applied to the transfer electrodes of the imaging unit 1i, and the three-phase vertical transfer clocks are applied to the transfer electrodes of the storage unit 1s. φv1 to φv3 are applied to control the potential in the channel region. Here, the vertical transfer clocks φv1 to φv3 are set to have a period three times that of the frame transfer clocks φf1 to φf3, and in the process of transferring information charges from the imaging unit to the storage unit, Information charges are mixed for every three adjacent mixed pixel units, and the number of pixels is thinned out.

FIG. 10 is another example of a schematic cross-sectional view and a potential diagram in the transfer direction at the boundary between the imaging unit 1i and the storage unit 1s, and the information charge is mixed at the boundary between the imaging unit 1i and the storage unit 1s. A solid-state imaging device. By repeating the timing T0 (T9) to T8, the three-phase frame transfer clocks φf1 to φf3 are applied to the transfer electrodes of the imaging unit 1i, and the three-phase vertical transfer clocks are applied to the transfer electrodes of the storage unit 1s. φv1 to φv3 are applied to control the potential in the channel region. Here again, the vertical transfer clocks φv1 to φv3 are set to have a period three times that of the frame transfer clocks φf1 to φf3, so that in the process of transferring information charges from the imaging unit to the storage unit, Information charges are mixed for every three adjacent mixed pixel units, and the number of pixels is thinned out. However, the information charge of one central pixel among the three pixels which are mixed pixel units is discharged to the substrate side at timing T4. By such potential control, in the case of a so-called mosaic type color filter arrangement as shown in FIG. 11, only information charges of the same color can be mixed.

Conventionally, the information charge amount Si, j is calculated by sequentially subtracting the smear charge amount from the output charge amount Vi, j for each bit obtained from each light receiving bit of the solid-state imaging device. When the information charges of adjacent pixels are mixed, there is a problem that the smear charge amount cannot be calculated and the information charge amount cannot be calculated.

An object of the present invention is to solve the above-described problems, and to calculate the amount of information charge by calculating the amount of smear charge even in a solid-state imaging device that mixes information charge by a predetermined number of mixed pixel units in the transfer direction. An object of the present invention is to provide a smear removing circuit capable of performing the above.

  The present invention transfers information charges accumulated in each light receiving bit in a vertical direction and also transfers information charges in the transfer direction from a solid-state imaging device having a light receiving portion in which a plurality of light receiving bits are arranged in a row direction and a column direction. Smear components mixed in the vertical transfer process of information charges are removed from the image signal obtained by sequentially transferring and outputting one row at a time in the horizontal direction after mixing for every predetermined number of mixed pixel units in adjacent pixels. In the smear removing circuit, the information charge amount accumulated in each light receiving bit of the solid-state imaging device is estimated based on the ratio between the exposure time of the information charge, the transfer period of the information charge in the vertical direction, and the number of mixed pixel units. It is obtained by accumulating the amount of smear charge for each light receiving bit by the amount of light receiving bits in the vertical transfer path of information charge, and transferring and outputting the information charge accumulated in each light receiving bit of the solid-state image sensor. Wherein the obtaining a second image data by subtracting individually the accumulated sum from the first image data continuous one line at a time representative of the output charge.

According to the present invention, smear components generated in the process of transferring in the vertical direction are smeared in a solid-state imaging device that mixes information charges accumulated in each light receiving bit in a predetermined number of mixed pixel units in pixels adjacent in the transfer direction. The smear component can be efficiently removed by subtracting the calculation result sequentially from the output charge amount. Therefore, the smear component can be removed even if the information charge of the solid-state image sensor is mixed in a predetermined number of mixed pixel units in adjacent pixels in the transfer direction according to the shooting state, and an optimal image can be obtained in all situations. Can be played.

  FIG. 1 is a block diagram showing a configuration of a smear removing circuit of the present invention. The smear removing circuit performs arithmetic processing on the amount of information charges accumulated in each light receiving bit of the imaging unit of the solid-state imaging device as digital image data.

  The smear removing circuit 10 removes smear components generated due to the time difference of vertical transfer in the solid-state image sensor. That is, a smear removing circuit is provided to remove a smear component generated due to a difference in transfer path of information charges accumulated in each light receiving bit of the solid-state imaging device.

  The smear removing circuit 10 includes a subtractor 11, an adder 12, a line memory 13, a multiplier 14, and a coefficient generator 16. A difference from the conventional smear removing circuit 30 is that the multiplier 14 is configured to apply a predetermined coefficient m determined by the exposure time Ts and the mixed pixel unit number m to the cumulative addition data T1 (n) read from the line memory 13. Multiply k to generate smear data T2 (n) representing a smear component. The coefficient generator 16 generates a predetermined coefficient m · k determined by the exposure time Ts and the mixed pixel unit number m for each light receiving bit of the solid-state imaging device, and supplies it to the multiplier 14.

  According to such a smear removing circuit 10, the amount of smear charge mixed from each light receiving bit is sequentially accumulated every time the information charge stored in each light receiving bit of the solid-state imaging device is transferred in the vertical direction row by row. Is added. Then, since the smear component mixed in the process of transferring information charges is represented by the cumulative addition value, the second smear component is removed by subtracting this value from the first image data S1 (n). Image data S2 (n) can be obtained.

  The coefficient generation circuit 16 receives the exposure time Ts and the mixed pixel unit number m for each frame, holds the same data for at least one vertical scanning period, and sets the same coefficient m · k during that period.

  Next, the principle of smear removal in the smear removal circuit 10 will be described.

FIG. 2 is a diagram for explaining the principle of smear removal by the smear removal circuit 10 in a solid-state imaging device that mixes information charges for every three mixed pixel units adjacent in the transfer direction as shown in FIG. The output charge amount Vi, j for each row obtained by mixing three pixels of the information charge amount Si, j of the light receiving bit of the solid-state imaging device is shown. This figure shows a case where the amount of light received by each light receiving bit is uniform.

Assuming that the time required for the information charge accumulated in each light-receiving bit to pass through one light-receiving bit during frame transfer and assuming that the amount of incident light does not change during exposure and frame transfer, Equation 5 Given by.

  Here, the third term and the fourth term on the right side are the charges accumulated when the information charges of the subsequent two pixels in the mixed pixel unit are transferred by passing through the light receiving bits in the same pixel unit. It is not a smear charge, but rather an information charge. Therefore, since the third term and the fourth term on the right side can be ignored, Equation 5 is transformed as Equation 6.

Therefore, if the first term on the right side of Equation 6 is replaced with the information charge amount Wi, j that is truly stored in the light receiving bit of the mixed pixel unit in which the information charges are mixed, Wi, j is given by Equation 7. Become.

  FIG. 3 shows smearing by the smear removing circuit 10 in the solid-state imaging device that discharges the information charges of one pixel at the center of the three pixels adjacent in the transfer direction and mixes the information charges every three mixed pixel units as shown in FIG. It is a figure explaining the principle of removal. An output charge amount Vi, j for each row in which the information charge amount Si, j of the light receiving bit of the solid-state imaging device is mixed for every three mixed pixel units at pixels adjacent in the transfer direction is shown. This figure shows a case where the amount of light received by each light receiving bit is uniform.

  Assuming that the time required for the information charge accumulated in each light-receiving bit to pass through one light-receiving bit during frame transfer and assuming that the amount of incident light does not change during exposure and frame transfer, Equation 8 Given by.

  Here, the third term and the fourth term on the right side are the charges accumulated when the information charge of the last one pixel among the mixed pixel units mixed with the information charge passes through the light receiving bit of the same mixed pixel unit by transfer. Therefore, it cannot be called a smear charge, but rather an information charge. Therefore, since the third term and the fourth term on the right side can be ignored, Equation 8 is transformed as Equation 9.

The first term on the right side of Equation 9 is replaced with the information charge amount Wi, j that is truly stored in the light receiving bit of the mixed pixel unit in which the information charge is mixed, and is generated in the center pixel of the mixed pixel unit in which the information charge is mixed Assuming that the information charge to be equal is the average value of the information charges generated at the pixels at both ends, Expression 9 can be transformed into Expression 10.

Eventually, the information charge amount Wi, j can be calculated by the same calculation as in Equation 7. In general, when the number of mixed pixel units is m, the information charge amount Wi, j that is truly accumulated in the light receiving bit of the mixed pixel unit in which the information charges are mixed is given by Equation 11.

The smear removing circuit 10 performs the calculation of the above-described equation 11, the combination of the adder 12 and the line memory 13 performs the Σ operation of the second term on the right side, and the multiplier 14 calculates the coefficient of the second term on the right side. m · k multiplication is performed. Then, by subtracting the calculation result extracted from the multiplier 14 from the first image data S1 (n) corresponding to the first term on the right side by the subtractor 11, the calculation of Expression 8 is completed. Therefore, even in a solid-state imaging device that mixes information charges accumulated in each light receiving bit in a predetermined number of mixed pixel units in pixels adjacent in the transfer direction, smear components generated due to vertical transfer of information charges according to the present invention Will be removed.

Note that the mixing of information charges at pixels adjacent in the transfer direction is performed during preview or the like, and when recording an image on a recording medium such as a memory, a high-resolution image corresponding to the number of pixels of the solid-state image sensor Not done for shooting. Therefore, a natural number having a mixed pixel unit number m of 1 or more in the present invention is input to the coefficient generation unit according to the shooting situation.

Further, in the above embodiment, the mixing of the information charges in the pixels adjacent in the transfer direction is performed at the boundary portion between the imaging unit 1i and the storage unit 1s. Alternatively, it may be performed at the boundary between the storage unit 1s and the horizontal transfer unit 1h.

  The method for calculating the smear charge amount of the solid-state imaging device as described above is based on the interline-type solid-state imaging in which a shift register for reading out information charges is arranged adjacent to each light receiving bit in addition to the frame transfer type solid-state imaging device. It can also be applied to elements.

Here, a is a coefficient representing the rate of charge leakage from each light receiving bit to the shift register unit, and is represented by a constant less than 1.

In the case of an interline solid-state image sensor, the information charge accumulated in each light-receiving bit is directly transferred into the light-shielded shift register, so that it is generated in each light-receiving bit like a frame transfer type solid-state image sensor. The generated charge does not become a smear charge as it is. However, since the information charge transferred from each light receiving bit is temporarily stored in the shift register arranged adjacent to the light receiving bit, it leaks from the light receiving bit storing the next screen information charge. A part of the generated information charge becomes a smear charge and is mixed into the information charge. The degree of mixing is proportional to the intensity of light incident on each light receiving bit, and the proportionality coefficient is represented as a. Since the value of the coefficient a varies depending on the structure of the solid-state imaging device and the wavelength of light incident on the light receiving bit, it is preferable to determine the optimum value while monitoring the output obtained by actual imaging.

As in the case of the frame transfer type solid-state imaging device, the smear component is removed by the smear removing circuit 20 in FIG. At this time, the coefficient generation circuit 26 outputs a coefficient a · m · k multiplied by a.

It is a block diagram which shows the structure of the smear removal circuit of this invention. It is a figure explaining the principle of a smear removal in the solid-state image sensor which mixes information charge for every 3 mixing pixel units in the transfer direction. It is a figure explaining the principle of a smear removal in the solid-state image sensor which mixes information charge for every 3 mixing pixel units in the transfer direction. It is a block diagram which shows the other structure of the smear removal circuit of this invention. It is a block diagram which shows the structure of the imaging device using the CCD solid-state image sensor of a frame transfer system. It is a figure showing the state of smear which generate | occur | produces in the solid-state image sensor of a frame transfer system. It is a block diagram which shows the structure of the conventional smear removal circuit. It is a figure explaining the principle of the smear removal by the conventional smear removal circuit. It is an example of a schematic cross-sectional view and a potential diagram of a solid-state imaging device that mixes information charges every three mixed pixel units in the transfer direction. It is an example of a schematic cross-sectional view and a potential diagram of a solid-state imaging device that mixes information charges every three mixed pixel units in the transfer direction. It is a figure explaining arrangement | positioning of a mosaic type color filter.

Explanation of symbols

1 Solid-state image sensor

1i Imaging unit 1s Accumulating unit 1h Horizontal transfer unit 1d Output unit 2 Analog signal processing circuit 3 A / D conversion circuit 4 Digital signal processing circuit 10 Smear removal circuit 11 Subtractor 12 Adder 13 Line memory 14 Multiplier 16 Coefficient generation unit

20 smear removing circuit 26 coefficient generating unit 30 smear removing circuit

36 Coefficient generator

Claims (4)

From a solid-state imaging device having a light receiving section in which a plurality of light receiving bits are arranged in a row direction and a column direction, information charges accumulated in each light receiving bit are transferred in a vertical direction, and information charges are transferred between adjacent pixels in the transfer direction. In a smear removing circuit that removes smear components mixed in a vertical transfer process of information charges from an image signal obtained by sequentially transferring and outputting one row at a time in the horizontal direction after mixing every predetermined number of mixed pixel units ,
For each light receiving bit estimated from the amount of information charge accumulated in each light receiving bit of the solid-state imaging device based on the ratio of the information charge exposure time to the information charge vertical transfer period and the number of mixed pixel units One row unit representing the amount of output charge obtained by accumulating the amount of smear charge by the amount of light receiving bits in the information charge vertical transfer path and transferring the information charge accumulated in each light receiving bit of the solid-state imaging device A smear removing circuit, wherein the second image data is obtained by subtracting the cumulative addition values individually from the first image data continuous in step (b). The smear removal circuit according to claim 1,
The smear removing circuit according to claim 1, wherein the number of mixed pixel units is switched according to a photographing situation.
The smear removal circuit according to any one of claims 1 and 2,
A subtractor that subtracts smear data from the first image data to generate second image data, and the second image data output from the subtractor is sequentially accumulated from the first row for each column. A line memory for storing cumulative addition data for one row while adding, and the cumulative addition data read from the line memory in the vertical direction of the information charge exposure time and information charge at each light receiving bit of the solid-state imaging device And a multiplier circuit for generating the smear data by multiplying a coefficient determined in accordance with a transfer cycle of the smear removal circuit.
The smear removal circuit according to claim 3.
A smear removal circuit comprising a smear coefficient generator for generating a predetermined coefficient based on an exposure time of information charges at each light receiving bit of the solid-state imaging device and the number of mixed pixel units.

Images