KR20240128380A - Fixed pattern noise diagonal detection method and system - Google Patents

Abstract

A method and system for detecting diagonal fixed pattern noise are disclosed. A system for detecting diagonal type fixed pattern noise according to one embodiment of the present invention may include an image acquisition unit for acquiring an inspection image output from an image sensor; a defect verification unit for finding white point and color point defects in the inspection image; a defect improvement unit for removing small defects and improving remaining defects; and a diagonal detection unit for analyzing the characteristics of the remaining defects to determine whether there is fixed pattern noise in the diagonal direction.

Description

{Fixed pattern noise diagonal detection method and system}

The present invention relates to an image sensor inspection technology, and more particularly, to a method and system for detecting diagonal type fixed pattern noise (FPN).

As the demand for electronic devices has increased in recent years, the supply of imaging equipment such as cameras and displays has increased. Image sensors are among the important components included in such imaging equipment.

In the case of image sensors, inspection is conducted using a hardware voltage distinction for flickering defects, output errors, etc. In the case of fixed pattern noise defects due to output signal non-uniformity, a detection method for linear defects such as vertical noise or horizontal noise is applied.

In the case of Korean Patent Publication No. 10-2011-0023648, "Image sensor and image detection method thereof," a method is used to amplify and inspect errors in a pixel array having a plurality of pixels that detect light from a subject, convert it into a voltage signal, and output it. This is a method of calculating error values between adjacent pixels, and refers to a general FPN inspection method. In this method, if there are no amplified error pixels themselves in the horizontal/vertical directions, the problem of the inspection itself continues to occur. Therefore, the standard for error pixels during FPN inspection is ambiguous, and there may be a problem of not being displayed in cases where the defect occurrence conditions are not satisfied.

In this case, the detection efficiency is low for fixed pattern noise in a diagonal direction rather than a straight line, so there is a limitation that additional problems may occur in the finished product due to voltage defects other than straight-line defects after shipment, and therefore a method for providing a reliable product is required.

Korean Patent Publication No. 10-2011-00023648 (Published on 2011.03.08.) - Image sensor and image detection method thereof

The present invention provides a method and system for detecting diagonal fixed pattern noise, which can detect fixed pattern noise in a diagonal direction in addition to horizontal or vertical directions for an image sensor, thereby securing quantitative evaluation criteria for FPN defects.

Other objects of the present invention will be readily understood through the following description.

According to one aspect of the present invention, a system for detecting fixed pattern noise of a diagonal type is provided, comprising: an image acquisition unit for acquiring an inspection image output from an image sensor; a defect verification unit for finding white point and color point defects in the inspection image; a defect improvement unit for removing small defects and improving remaining defects; and a diagonal detection unit for analyzing the characteristics of the remaining defects to determine whether there is fixed pattern noise in a diagonal direction.

The above inspection image may be an averaged image acquired in a dark environment or a low-light environment.

The above defect improvement unit can remove the small defect by applying Gaussian blur to the inspection image and then resizing it, and can maximize the brightness value by using the average value of the image spread through the morphological operation.

The above-mentioned diagonal detection unit sets a ROI spare block for an image with a maximized brightness value, counts the number of defective pixels in the ROI spare block, and sets it as an ROI block if the number is greater than a reference value, and can determine whether or not the fixed pattern noise is diagonal based on the position of the ROI block.

Meanwhile, according to another aspect of the present invention, a method for detecting a fixed pattern noise of an oblique type is provided, comprising: (a) a step of obtaining an inspection image output from an image sensor in an image acquisition unit; (b) a step of finding white point and color point defects in the inspection image in a defect verification unit; (c) a step of removing small defects and improving remaining defects in a defect improvement unit; and (d) a step of analyzing the characteristics of the remaining defect in a diagonal detection unit to determine whether or not there is fixed pattern noise in the diagonal direction.

The above step (c) may include a step of applying Gaussian blur to the inspection image; a step of performing resizing; a step of spreading the resized image through a morphological operation; and a step of maximizing the brightness value using an average value for the spread image.

The step (d) above may include a step of setting a ROI spare block for an image in which the brightness value is maximized; a step of counting the number of defective pixels in the ROI spare block and setting it as an ROI block if the number is greater than a reference value; and a step of determining whether or not there is a diagonal line of fixed pattern noise based on the position of the ROI block.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to an embodiment of the present invention, it is possible to detect fixed pattern noise in a diagonal direction in addition to horizontal or vertical directions for an image sensor, thereby providing an effect of securing a quantitative evaluation criterion for FPN defects.

Figure 1 is a block diagram of a configuration of a diagonal fixed pattern noise detection system according to one embodiment of the present invention.
Figure 2 is a flow chart of a diagonal fixed pattern noise detection method according to one embodiment of the present invention.
Figure 3 is an example screen for detecting diagonal fixed pattern noise.

The present invention can have various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only to distinguish one component from another.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood to not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

In addition, it is to be understood that the components of the embodiments described with reference to each drawing are not limited to the embodiments described herein, and may be implemented to be included in other embodiments within the scope in which the technical spirit of the present invention is maintained, and that multiple embodiments may be reimplemented as one integrated embodiment even if a separate description is omitted.

In addition, when describing with reference to the attached drawings, regardless of the drawing symbols, identical components are given identical or related reference symbols, and redundant descriptions thereof are omitted. When describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof is omitted.

Additionally, terms such as “part,” “unit,” “module,” and “device” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

FIG. 1 is a block diagram of a diagonal fixed pattern noise detection system according to one embodiment of the present invention, FIG. 2 is a flowchart of a diagonal fixed pattern noise detection method according to one embodiment of the present invention, and FIG. 3 is an example screen for detecting diagonal fixed pattern noise.

A system and method for detecting diagonal fixed pattern noise according to one embodiment of the present invention are characterized in that they can detect, without omission, fixed pattern noise in a diagonal direction in addition to horizontal and vertical directions during an inspection process for an image sensor. That is, when detecting fixed pattern noise, it is possible to detect noise in a diagonal direction as well as horizontal and vertical directions.

Referring to FIG. 1, the oblique fixed pattern noise detection system (100) according to the present embodiment may include an image acquisition unit (110), a defect verification unit (120), a defect improvement unit (130), and an oblique detection unit (140).

The image acquisition unit (110) acquires an inspection image output from the image sensor of the camera module that is the subject of inspection.

In order to select a reliable criterion for defects, the inspection images output from the image sensor are required to be images taken in the same or similar environment. The basic condition can be that the same result values are obtained in the environment and that there is no or minimal deviation in the image data.

For this purpose, in this embodiment, an averaged image can be used when acquiring an image for fixed pattern noise inspection.

Environmental conditions when acquiring inspection images may include a dark environment or a low-light environment.

A dark environment is one where light is completely blocked and O Lux. In this case, the analog gain (Gain_A) of the image sensor can be set to 16 times and the digital gain (Gain_D) to 2 times to increase the pixel brightness value. This increases the exposure in a dark environment and increases the brightness value of noise.

A low-light environment is an environment where very little light enters, typically less than 10 Lux. In this case, the analog gain of the image sensor can be set to 16x and the digital gain to 4x to obtain an average image.

Here, the setting multiples for analog gain and digital gain are exemplary, and of course, various multiples can be set as needed.

The defect verification unit (120) can perform image verification (defect verification) on the averaged inspection image acquired from the image acquisition unit (110).

Image verification is to check for white spot or color spot defects in a dark or low-light environment. If there is a white spot or color spot defect, you can re-shoot and re-verify whether there is a defect in the same location. This is a preprocessing task to inspect only the area with the same symptom, as noise increases when the sensor gain is increased.

[Mathematical formula 1]

Here, W _dot represents a white dot, C _dot represents a color dot, Image(Y) represents a grayscale image, and Image(RGB) represents a color image (e.g., 24 bits).

Image[i] is a one-dimensional representation of a pixel in the inspection image, Image[i-1] represents the previous pixel (i-1th pixel), and Image[i] represents the current pixel (ith pixel).

Dark is a brightness value representing a dark environment, and is an arbitrarily set value (e.g., 10). Average(Y) is the overall brightness value of the image, and Pixel is a pixel value.

If the environment is dark or the pixel value is greater than the overall brightness value of the image, the pixel can be set to a white dot (W _dot ).

For color images, if it is above a certain ratio, it is selected as a color dot (C _dot ).

Only when a pixel of the image is selected as a white dot (W _dot ) and a color dot (C _dot ) defect, it is selected as a color dot (C _dot ).

For the selected defect, it is possible to determine whether the matching point component is constant in order to distinguish whether it is a straight line type. The matching point component is used to determine whether the defect is a straight line, and represents the pixel array selected as a result of mathematical expression 1.

In the case of fixed pattern noise, there is a constant matrix, but it cannot be said to be an adjacent matrix. For the detection pixel according to this, if it satisfies the 5*5 internal pixel criterion, the count is increased, and if the count satisfies the data of Width/2, it is recognized as a straight line. Width represents the width of the straight line.

[Mathematical formula 2]

C_Image is a modified image that explicitly expresses defects in the image by changing the value of the pixel corresponding to C _dot to 0.

If the pixel data in C_Image is 0, it can be considered to correspond to the line position. In this case, if a color dot (C _dot ) exists within the 5*5 area around the line, the count is increased, and this count can be used to distinguish whether it is a straight-line fixed pattern noise (Line _FPN ) or a low-light image.

When the defect is identified, the defect improvement unit (130) can perform improvement work to remove small defects and easily detect any remaining residual defects.

First, you can perform blurring by applying Gaussian blur to remove the basic noise (small defects) of the image. You can perform resizing to minimize the difference between the surrounding pixels for the image from which the basic noise has been removed through blurring. Resizing is an operation for amplification. When there is a value corresponding to 0 in the surrounding area, the average value drops, so the size of the array unit decreases and it is output widely.

[Mathematical formula 3]

Gaussian means applying a blur, and Resize can be based on, for example, 1/4.

If Lowlight is output as a result of mathematical expression 2, the image can be changed by applying Gaussian blur and then resizing (1/4).

The defect improvement unit (130) can expand the size of the residual defect in order to clarify the location of the residual defect in the resized image. The pixel verification result of the resized image can maximize the brightness value by using the average value for the image that has been diffused (expanded) using morphology to perform expansion for the surrounding pixels (binarization). The average value can be amplified for the surrounding pixels through morphology.

[Mathematical Formula 4]

A peripheral morphology _dilation operation is performed on the resize result (Lowlight _Resize ) according to the result of mathematical expression 3. This is to check the diagonal result for the defect, and the purpose is to maximize the value during the operation after assuming the defective pixel as a diagonal pixel.

In mathematical expression 4, dividing by 2 after squaring the pixel value of the diagonal pixel is to distinguish cases where the pixel value is 150 or more, and the number of divisions can be adjusted depending on the reference value.

The diagonal detection unit (140) calculates the ROI location range of a predetermined size when the output result value reaches a predetermined reference value or more, and counts and compares the number of diagonal pixels (defective pixels) within the ROI to perform a final PASS/FAIL (Diagonal _FPN ) decision. A defective decision indicates that a fixed pattern noise in the diagonal direction has been detected.

[Mathematical Formula 5]

Previously, if the pixel value is 150 or more, the result value for the diagonal pixel can be output based on 4, so it is possible to determine whether the result value of the diagonal pixel exceeds 4. If it exceeds 4, the maximum value is set to 255, and the pass/fail of the ROI preliminary block can be determined based on the counting number for the diagonal pixels in the ROI preliminary block.

Referring to Fig. 2, a method for detecting diagonal lines in a fixed pattern image will be described.

In the image acquisition unit (110), multiple inspection images can be averaged and used according to the environment when an image is input (step S200). The environmental conditions may include a dark environment or a low-light environment.

The defect verification unit (120) checks white dots and color dots for the inspection image (step S205). If a certain amount of white dots or color dots occur, the inspection image can be re-acquired.

If the same location defect is confirmed to occur even when re-acquiring, the defect verification unit (120) can conduct an inspection to distinguish the straight line of the defect (step S210).

In the defect improvement unit (130), noise removal is performed on the inspection image using Gaussian blur (step S215).

And then, resizing is performed to average the resulting values of the image (step S220). Small defects can be removed through blurring and resizing.

In the defect improvement unit (130), morphology is performed based on the averaged image to expand pixels (step S225). Through this, the size of the remaining defect is expanded and its location can be clarified.

The defect improvement unit (130) maximizes the brightness value for data for the extended pixel (step S230).

The diagonal detection unit (140) sets a ROI spare block (step S235), and uses the output result to count the number of defects existing within the ROI spare block location, and if the number is greater than the reference value, can be set as an ROI block (step S240).

As illustrated in Fig. 3, it is possible to detect fixed pattern noise in a diagonal direction based on the location of the ROI block.

Additionally, in this embodiment, scratch band-type inspection for color spots and white spots is also possible.

The above-described diagonal fixed pattern noise detection method can also be implemented in the form of a recording medium including computer-executable instructions, such as an application or program module executed by a computer. The computer-readable medium can be any available medium that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium can include a computer storage medium. The computer storage medium includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data.

The above-described oblique fixed pattern noise detection method can be executed by an application that is basically installed in the terminal (which may include a program included in a platform or operating system that is basically installed in the terminal), and can also be executed by an application (i.e., a program) that the user directly installs in the master terminal through an application providing server, such as an application store server, an application, or a web server related to the corresponding service. In this sense, the above-described oblique fixed pattern noise detection method can be implemented as an application (i.e., a program) that is basically installed in the terminal or directly installed by the user, and recorded on a computer-readable recording medium such as the terminal.

Although the present invention has been described above with reference to embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

100: Diagonal fixed pattern noise detection system
110: Image acquisition section
120: Defect Verification Unit
130: Defect Improvement Section
140: Diagonal detection unit

Claims (7)

A system for detecting fixed pattern noise of diagonal type,
An image acquisition unit that acquires an inspection image output from an image sensor;
A defect verification unit that finds white dot and color dot defects within the above inspection image;
A defect improvement section that removes small defects and improves residual defects; and
A diagonal fixed pattern noise detection system including a diagonal detection unit that analyzes the characteristics of the above residual defects and determines whether there is diagonal fixed pattern noise.
In the first paragraph,
A diagonal fixed pattern noise detection system, characterized in that the above inspection image is an averaged image acquired in a dark environment or a low-light environment.
In the first paragraph,
The above defect improvement unit is a diagonal fixed pattern noise detection system characterized in that it removes the small defect by applying Gaussian blur to the inspection image and then resizing it, and maximizes the brightness value by using the average value for the image spread through the morphological operation.
In the third paragraph,
A diagonal fixed pattern noise detection system characterized in that the above diagonal detection unit sets an ROI spare block for an image with a maximized brightness value, counts the number of defective pixels in the ROI spare block, sets it as an ROI block if the number is greater than a reference value, and determines whether or not the fixed pattern noise is diagonal based on the position of the ROI block.
A method for detecting fixed pattern noise of diagonal type,
(a) A step of acquiring an inspection image output from an image sensor in an image acquisition unit;
(b) a step of finding white dot and color dot defects within the inspection image in the defect verification unit;
(c) a step of removing small defects from the defect improvement section and improving remaining defects; and
(d) A method for detecting diagonal fixed pattern noise, comprising a step of analyzing the characteristics of the residual defect in the diagonal detection unit to determine whether there is fixed pattern noise in the diagonal direction.
In paragraph 5,
The step (c) above is a method for detecting diagonal fixed pattern noise, characterized in that it includes: a step of applying Gaussian blur to the inspection image; a step of performing resizing; a step of spreading the resized image through a morphological operation; and a step of maximizing the brightness value using an average value for the spread image.
In Article 6,
The step (d) above is a method for detecting oblique fixed pattern noise, characterized in that it includes: a step of setting a ROI spare block for an image in which the brightness value is maximized; a step of counting the number of defective pixels in the ROI spare block and setting it as an ROI block if the number is greater than a reference value; and a step of determining whether or not the fixed pattern noise is oblique based on the position of the ROI block.