JP2016082454A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine the type of a detected defective pixel, and to correct the same correctly, in detection of a subsequent defective pixel.SOLUTION: When some pixel is detected as a defective pixel, a value depending on the detection conditions thereof is added to a determination value and stored. When a pixel regarded as a defective pixel in the previous defective pixel detection is regarded as a normal pixel in the next detection, the determination value is subtracted. The type of the defective pixels is determined depending on the sum total of the determination values of respective pixels, and correction is performed depending on the defective pixel. A subsequent defective pixel is corrected, a defective pixel generated for a certain period of time, and not generated for a certain period of time, is marked for reference of defective pixel correction performed in real time. This correction processing is performed for all pixels.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging apparatus using an imaging element.

  In an image sensor such as a CCD sensor or a CMOS sensor included in a digital camera, a defective pixel having an output signal level different from that of a normal pixel exists in some pixels. This defective pixel appears as a so-called scratch at the time of recording or displaying image data, which causes a deterioration in image quality performance.

  By correcting the output of the defective pixel using, for example, image information of peripheral non-defective pixels, it is possible to prevent deterioration in image quality performance to some extent. The position of the defective pixel is detected mainly during the manufacturing process of the digital camera, and the position information is stored in a memory such as a flash ROM provided in the camera. Then, based on the stored position information of the defective pixel, the defective pixel output is corrected for each photographing.

  By the way, it is known that some defective pixels in such an image sensor are generated after the digital camera is shipped as a product. In this way, it is necessary to deal with defective pixels generated after product shipment, for example, by the user bringing the product into a service center or by executing a program for detecting defective pixels by the user depending on the product. There is.

  Patent Documents 1 and 2 describe an imaging apparatus that records and corrects position information of defective pixels when a defective pixel is detected from the same pixel a plurality of times. According to this imaging apparatus, the output of late defective pixels can be corrected.

JP 2000-282166 A JP 2008-011567 A

  However, although the late defective pixels can be corrected in the above-described conventional example, defective pixels having characteristics that occur or do not occur for a certain period are always corrected. Therefore, correction processing is performed even when the defective pixel is not generated, and there is a problem that the quality of the image is deteriorated depending on the subject.

  The present invention has been made in order to solve the above-described problem. An image sensor including a plurality of pixels, a defective pixel detection unit that detects defective pixels of the image sensor, and information on defective pixels of the image sensor. A defective pixel information storage means for storing; and a detection frequency digitizing means for digitizing a detection frequency of the defective pixel detected by the defective pixel detection means to generate a determination value, and generated by the detection frequency digitizing means. The determined determination value is stored in the defective pixel information storage unit as information on the defective pixel.

  By generating a determination value obtained by quantifying the detection frequency of the defective pixel, it becomes possible to appropriately determine the defective pixel.

It is a block diagram of a present Example. It is a flowchart for a detection of a present Example. It is a flowchart for a detection of a present Example. It is a correction flowchart of the present embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of an embodiment of the present invention. In FIG. 1, an image processing apparatus 100 is a digital camera and includes each configuration described below.

  The taking lens 10 forms a subject image on an image sensor 14 that converts an optical image into an electrical signal. The mechanical shutter 12 has a diaphragm function and adjusts the exposure amount to the image sensor 14. The A / D converter 16 converts the analog signal output of the image sensor 14 into a digital signal.

  The timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50. In addition to the mechanical shutter 12, by controlling the reset timing of the image sensor 14 by the timing generation circuit 18, it is possible to control the accumulation time of the image sensor 14 as an electronic shutter when performing moving image shooting or the like.

  The image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, an electronic zoom function is realized by performing image cutting and scaling processing by the image processing circuit 20.

  In the image processing circuit 20, predetermined calculation processing is performed using the captured image data, and the system control circuit 50 controls the exposure control unit 40 and the distance measurement control unit 42 based on the obtained calculation result. Thus, TTL AF processing, AE processing, and EF processing are performed. Further, the image processing circuit 20 performs predetermined arithmetic processing using captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.

  The memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32. Data of the A / D converter 16 is written into the image display memory 24 or the memory 30 via the image processing circuit 20, the memory control circuit 22, or directly via the memory control circuit 22.

  The display image data written in the image display memory 24 is displayed on the image display unit 28 including an LCD or the like via the D / A converter 26. If the image data captured using the image display unit 28 is sequentially displayed, the electronic viewfinder function can be realized. The image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50. When the display is turned off, the power consumption of the image processing apparatus 100 can be greatly reduced. I can do it.

  The memory 30 stores captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and a predetermined time of moving images. Thereby, even in the case of continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, it is possible to write a large amount of images to the memory 30 at high speed. The memory 30 can also be used as a work area for the system control circuit 50.

  The compression / decompression circuit 32 reads image data stored in the memory 30, performs compression processing or decompression processing by adaptive discrete cosine transform (ADCT) or the like, and writes the processed data to the memory 30. The exposure control unit 40 controls the shutter 12 having a diaphragm function, and has a flash light control function in cooperation with the flash 48. The distance measurement control unit 42 controls the focusing of the photographing lens 10. The zoom control unit 44 controls zooming of the taking lens 10. The barrier control unit 46 controls the operation of the barrier 102 that is a protection unit. The flash 48 also has an AF auxiliary light projecting function and a flash light control function.

  The exposure control unit 40 and the distance measurement control unit 42 are controlled by the system control circuit 50 using the TTL method based on the calculation result obtained by calculating the captured image data by the image processing circuit 20. The system control circuit 50 controls the entire image processing apparatus 100. The memory 52 stores constants, variables, programs, and the like for operating the system control circuit 50.

  The display unit 54 displays an operation state, a message, and the like using characters, images, sounds, and the like according to the execution of the program in the system control circuit 50. The image processing apparatus 100 is installed near the operation unit in an easily visible position, and is configured by a combination of a liquid crystal display (LCD), an LED, a speaker, and the like. In addition, the display unit 54 is partially installed in the optical viewfinder 104.

  Among the display contents of the display unit 54, what is displayed on the LCD or the like is a single shot / continuous shooting display, a self-timer display, a compression rate display, a recording pixel number display, a recording number display, a remaining image number display, etc. is there. In addition, shutter speed display, aperture value display, exposure correction display, flash display, red-eye reduction display, macro shooting display, and the like may be displayed. Further, a buzzer setting display, a battery remaining amount display, an error display, an information display with a plurality of digits, an attachment / detachment state display of the recording media 200 and 210, a communication I / F operation display, and the like are possible. Further, among the display contents of the display unit 54, what is displayed in the optical viewfinder 104 includes in-focus display, camera shake warning display, flash charge display, shutter speed display, aperture value display, exposure correction display, and the like. The nonvolatile memory 56 can be electrically erased and recorded, and for example, an EEPROM or the like is used. The memory 56 stores information on defective pixels of the image sensor as will be described later.

  The operation means for inputting various operation instructions of the system control circuit 50 is configured by a single or a plurality of combinations such as a switch, a dial, a touch panel, pointing by visual line detection, and a voice recognition device.

  The mode dial switch 60 switches and sets each function mode such as power-off, automatic shooting mode, shooting mode, panoramic shooting mode, moving image shooting mode, playback mode, multi-screen playback / erase mode, PC connection mode, and TV reception mode. I can do it. The shutter switch 62 (SW1) is turned ON during the operation of the shutter button, and starts operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing. Instruct.

  The shutter switch 64 (SW2) is turned on when the operation of the shutter button is completed, and instructs the start of a series of processing operations including exposure processing, development processing, compression processing, and recording processing. The exposure process is a process of writing image data into the memory 30 via the A / D converter 16 and the memory control circuit 22 from the signal read from the image sensor 12. The development processing is image conversion processing by calculation in the image processing circuit 20 and the memory control circuit 22. The compression process is a process in which image data is read from the memory 30 and compressed by the compression / decompression circuit 32. The recording process is a process for writing image data to the recording medium 200 or 210.

  The display changeover switch 66 can change the display of the image display unit 28. With this function, when photographing is performed using the optical viewfinder 104, it is possible to save power by cutting off the current supply to the image display unit including a TFT LCD or the like. The operation unit 70 includes various buttons such as a menu button, a set button, a macro button, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, a touch panel, and the like. Also, menu movement + (plus) button, menu movement-(minus) button, playback image movement + (plus) button, playback image-(minus) button, shooting image selection button, exposure compensation button, date / time setting button, etc. There is also.

  In the zoom switch 72, the user gives an instruction to change the magnification of the captured image. The zoom switch 72 includes a tele switch that changes the imaging field angle to the telephoto side and a wide switch that changes the imaging angle of view to the wide angle side. By using the zoom switch 72, the zoom control unit 44 is instructed to change the imaging field angle of the photographing lens 10, and becomes a trigger for performing an optical zoom operation. In addition, it also serves as a trigger for electronic change of the imaging angle of view by image cutting by the image processing circuit 20 or pixel interpolation processing. The subject detection unit 74 includes an element that detects a subject. A case where a face is detected as a subject can also be considered.

  The power supply control unit 80 includes a connector 82 for connecting and communicating with a power supply 86, and includes a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like, Detects battery type and battery level. Further, the DC-DC converter is controlled based on the detection result and the instruction of the system control circuit 50, and a necessary voltage is supplied to each part including the recording medium for a necessary period. The power source 86 includes a connector 84 for connecting and communicating with the power source control unit 80, and includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like. .

  Interfaces 90 and 94 communicate with a recording medium such as a memory card or a hard disk, and connectors 92 and 96 connect with a recording medium such as a memory card or a hard disk. In this embodiment, it is assumed that there are two interfaces and connectors for attaching the recording medium. Of course, the interface and the connector for attaching the recording medium may have a single or a plurality of systems and any number of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. The interface and the connector may be configured using an SD card, a CF card, or the like that conforms to the standard.

  Further, the interfaces 90 and 94 and the connectors 92 and 96 can be configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like. That is, by connecting various types of communication cards such as LAN cards, modem cards, USB cards, IEEE 1394 cards, SCSI cards, PHS and other communication cards, image data and Management information attached to image data can be transferred to each other.

  The barrier 102 serving as a protection unit covers the imaging unit including the lens 10 of the image processing apparatus 100 to prevent the imaging unit from being soiled or damaged. The optical viewfinder 104 can perform shooting using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28. In the optical viewfinder 104, some functions of the display unit 54, for example, a focus display, a camera shake warning display, a flash charge display, a shutter speed display, an aperture value display, an exposure correction display, and the like are installed.

  The communication unit 110 has various communication functions such as USB, IEEE 1394, LAN, and wireless communication. A connector (antenna in the case of wireless communication) 112 connects the image processing apparatus 100 to another device via the communication unit 110. A recording medium 200 such as a memory card or a hard disk includes a recording unit 202 configured by a semiconductor memory, a magnetic disk, or the like, an interface 204 with the image processing apparatus 100, and a connector 206 for connecting to the image processing apparatus 100. A recording medium 210 such as a memory card or a hard disk includes a recording unit 212 composed of a semiconductor memory, a magnetic disk, or the like, an interface 214 with the image processing apparatus 100, and a connector 216 for connecting with the image processing apparatus 100.

  In the digital camera 100 as described above, shooting is performed when the user presses the shutter switch 64. Here, immediately after shooting a normal image, if the shooting condition acquisition means acquires shooting conditions that are likely to generate noise such as high sensitivity, high temperature, or slow shutter speed. Close the shutter and take a black image under the same conditions as the normal image. And the noise reduction process which subtracts a black image from a normal image is performed. Further, the photographed black image is stored in the memory 30, and defective pixels are detected based on the black image stored in the memory 30. 2 and 3 are flowcharts of the defective pixel detection process based on the black image in the present embodiment.

  First, the number X of pixels and the pixel position x of the image sensor 14 are acquired (S201). The determination value h and pixel value Sx of the pixel position obtained by the previous detection are acquired (S202). Here, the determination value h is generated by quantifying the detection frequency of defective pixels (detection frequency quantification means). Then, according to the determination value h, it can be determined whether the target pixel is a late defective pixel or a defective pixel having a characteristic that occurs or does not occur for a certain period.

  Next, the pixel value Sx at the pixel position x obtained in S202 is compared with the threshold value Tk for determining whether or not the pixel is a defective pixel, and it is determined whether or not the pixel at the pixel position x is a defective pixel. (S203). If it is determined that the pixel is a defective pixel in S203, the value is increased by adding a value corresponding to the condition at the time of detection to the determination value h obtained in the previous detection (S207). Since defective pixels have different easiness of detection depending on shooting conditions, when a defective pixel is determined under difficult detection conditions, a value corresponding to the condition is further added in addition to 1 that is normally added.

  FIG. 3 shows details of the determination value adding process corresponding to the detection condition of S207. It is determined whether or not the temperature Tm at the time of shooting is lower than S ° C. (S301). When the temperature Tm at the time of shooting is lower than S ° C., s is added to the determination value h (S305). Further, it is determined whether the temperature is lower than T ° C. (S302). When the temperature is lower than T ° C. (S <T), t (t <s) is added to the determination value h (S304).

  Returning to FIG. 2, if it is not determined as a defective pixel in S203, it is confirmed based on the determination value h whether the pixel has been detected as a defective pixel until the previous time (S204). If the pixel has been detected as a defective pixel until the previous time, it is decreased by subtracting u from the determination value h (S206). The reason for performing the subtraction process here is to clarify that this pixel is a defective pixel having a feature that may or may not occur for a certain period of time because it is unlikely that this pixel is a late defective pixel. .

  In S204, a pixel that has not been detected as a defective pixel until the previous time is determined to be a normal pixel (S205), and it is confirmed whether or not all pixels have been detected (S214). If all the pixels have not been detected, the pixel position x is advanced (S215), and the same processing is repeated. If all the pixels have been detected, the detection process ends.

  On the other hand, for the pixel for which the determination value h is subtracted in S206 or the determination value h is added in S207, it is determined whether or not the determination value h after subtraction and addition is larger than the defective pixel type determination threshold value H. (S208). The defective pixel type determination threshold value H is a threshold value for determining whether the defect is a late defect or a defect having a characteristic that occurs or does not occur for a certain period of time based on the determination value h. As a result of comparison with the defective pixel type determination threshold value H, if the determination value h is larger than the defective pixel type determination threshold value H (h> H), it is a late defective pixel (cosmic ray scar). Determine (S210). If the determination value h does not reach the defective pixel type determination threshold value H (h ≦ H), it is determined that the pixel is a defective pixel (variable flaw) having characteristics that occur or do not occur for a certain period (S209). ).

  Subsequently, it is confirmed whether or not there is an empty area in the memory 56 (defective pixel information storage means) that stores defective pixel data (S211). If there is no free space in the memory 56, the pixel data having a low determination value h is deleted (S212), and the determination value h and position information x of the defective pixel determined this time are stored (S213). If there is an empty area in the memory 56, the determination value h and position information x of the defective pixel determined this time are stored as they are (S213).

  Next, it is confirmed whether or not all pixels have been detected (S214). If all pixels have not been detected, the pixel position x is advanced (S215), and the same processing is repeated to detect all pixels. If it has been performed, the detection is terminated.

  Similar processing can also be performed in the moving image mode. However, since there is no opportunity to shoot a moving image in a light-shielded state, for example, when starting up or shutting down, the image sensor 14 is operated in the moving image mode to shoot a black image and detect defective pixels. Further, an example in which the pixel position is converted from the still image mode to the moving image mode based on the defective pixel detection data performed in the still image shooting mode of the present embodiment, and the defective pixel is corrected in the moving image mode can be considered. In this case, only the pixel of h> H needs to be corrected.

  Next, when the detection process of defective pixels in all the pixels according to the flowcharts of FIGS. 2 and 3 is completed, a correction process is subsequently performed. An embodiment in which defective pixels are corrected will be described with reference to FIG. The image processing circuit 20 includes a defective pixel correction circuit that interpolates pixels determined to be defective pixels based on pixel defect information from surrounding pixels, and a real-time defect that detects and corrects defective pixels in real time by analyzing a captured image. It is assumed that a pixel detection circuit exists. The real-time defective pixel detection circuit performs defective pixel determination by inputting an image. When reference pixel position information is input, detection of a pixel having reference information is detected with a threshold lower than that of a normal pixel.

  First, the pixel position x of the target defective pixel is acquired (S401). Then, the determination value h of the pixel saved at the time of detection is acquired (S402). It is determined whether or not the pixel is a late defective pixel (cosmic ray scar) based on the determination value h acquired in S402 (S403). If it is determined that the pixel is a late defective pixel, the surrounding pixel values are also included. Then, correction processing is performed (S406). If it is determined in S403 that the pixel is not a late defective pixel, it is determined whether or not it is a defective pixel (fluctuation flaw) having a feature that may or may not occur for a certain period of time based on the determination value h acquired in S402. (S404). If it is determined that the pixel is a defective pixel that may or may not occur for a certain period of time, a mark is attached as reference pixel position information for real-time defective pixel correction (S405).

  When the respective processes are performed in S405 and S406, it is confirmed whether or not all defective pixels have been corrected (S407). If all defective pixels have not been corrected, the pixel position x is advanced (S408), and the same processing is performed. If all defective pixels have been corrected, the correction process ends.

  It is possible to identify whether a defective pixel detected by the present embodiment is a late defect or a defect having a characteristic that may or may not occur for a certain period of time, and an appropriate correction for each defect. By performing the processing, there is a merit that the quality of the image can be maintained.

(Other examples)
Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

  Also, when a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly to a system or apparatus having a computer that can execute the program using wired / wireless communication, and the program is executed Are also included in the present invention.

  Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  The recording medium for supplying the program may be, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a nonvolatile semiconductor memory.

  As a program supply method, a computer program that forms the present invention is stored in a server on a computer network, and a connected client computer downloads and programs the computer program.

14 Image Sensor 20 Image Processing Circuit 50 System Control Circuit 56 Memory

Claims (9)

An image sensor comprising a plurality of pixels;
A defective pixel detecting means for detecting a defective pixel of the image sensor;
Defective pixel information storage means for storing information of defective pixels of the image sensor;
Detection frequency quantification means for quantifying the detection frequency of defective pixels detected by the defective pixel detection means and generating a determination value;
An image pickup apparatus, wherein the determination value generated by the detection frequency digitizing unit is stored in the defective pixel information storage unit as information on the defective pixel.   The imaging apparatus according to claim 1, wherein the detection frequency digitizing unit increases the determination value when a target pixel is detected as a defective pixel by the defective pixel detecting unit.   The imaging apparatus according to claim 1, wherein the detection frequency digitizing unit decreases the determination value when a pixel of interest is not detected as a defective pixel by the defective pixel detecting unit.   The imaging apparatus according to claim 1, wherein a type of defective pixel is determined according to the determination value generated by the detection frequency digitizing unit. A defective pixel correcting means for correcting the defective pixel;
5. The imaging apparatus according to claim 1, wherein whether or not the defective pixel is to be corrected is determined according to the determination value generated by the detection frequency digitizing unit. It further has photographing condition acquisition means for acquiring photographing conditions,
3. The imaging according to claim 2, wherein the detection frequency digitizing unit increases the determination value according to the imaging condition when a target pixel is detected as a defective pixel by the defective pixel detecting unit. apparatus.   The imaging apparatus according to claim 6, wherein the photographing condition includes a shutter speed.   The imaging apparatus according to claim 6, wherein the imaging condition includes imaging sensitivity.   The imaging apparatus according to claim 6, wherein the imaging condition includes a temperature.

Images