JP2008139074A - Method for detecting defect in image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately discriminate a defect at a pattern edge in a method for comparing a sample image with a target image and detecting the defect in the target image in units of pixel. <P>SOLUTION: In the method for comparing the sample image with the target image and detecting the defect in the target image in units of pixel, a feature vector comprising the feature quantity such as a density value is extracted pixel by pixel, a previously obtained positional relationship of the feature vector to an identification boundary or plane between a normal class and an abnormal class is discriminated pixel by pixel, and a defective pixel is determined if the feature vector exists in the abnormal class. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image defect detection method for detecting a defect by comparing a plurality of learning samples with a target by image processing, such as inspection of an object to be inspected, such as pattern inspection of a printed circuit board, inspection of different parts, and package print error inspection. About.

  FIG. 1 shows a conventional image defect detection method in which a non-defective product master and a target are compared by image processing to detect defects. The non-defective master image and the target image are aligned in advance. For the alignment, a residual correlation method, a normalized correlation method, or the like in image processing is used. The target image and the non-defective product master image are overlapped, a difference is taken for each pixel, and if the absolute value of the difference is larger than a certain threshold value, it is determined as a defective pixel. However, this method has the following three problems.

  (1) Both the non-defective master image and the target image have a significant change in density value at the edge of the pattern, and the absolute value of the difference is large. Since there is a tolerance in the dimension of the edge of the pattern, the position varies from target to target even within the resolution of one pixel. Such a variation in pattern position appears as a variation in the density value of pixels near the edge portion. For this reason, if there is only one threshold value in the entire image, defective pixels frequently occur at the edge of the pattern.

  (2) In order to solve the problem (1), there is a method of providing a threshold value for each pixel. However, manually setting a threshold value for each pixel is cumbersome and difficult to use.

(3) In order to solve the above problem (2), there is a method in which a non-defective master is replaced with a plurality of learning samples. This is because, by superimposing multiple learning sample images and calculating the average value μ _ij and standard deviation σ _ij for each pixel (i, j), μ _ij ± cσ _ij is set for each pixel using a certain constant c. It is set as a threshold value. This method can be automatically performed by image processing, and the density variation from pixel to pixel can be dealt with by the equation cσ _ij, so that defective pixels including the edge portion do not occur frequently. However, since the variation of the density value becomes large at the edge portion, the standard deviation σ _ij also becomes large and the occurrence of defective pixels can be suppressed, but conversely, the determination becomes complicated. Therefore, when there is a defect in the edge portion, there arises a problem that it is easy to overlook.
JP 2001-175865 A Japanese Patent Laid-Open No. 10-253544 JP 11-242746 A

  Therefore, an object of the present invention is to correctly discriminate a defect particularly in an edge portion of a pattern in a method for comparing a sample image with a target image and detecting a defect in the target image in pixel units.

  Under the above-described problems, the invention of claim 1 compares a sample image with a target image and detects a defect in the target image in units of pixels. The positional relationship of the feature vector with respect to the normal class / abnormal class identification boundary line or surface obtained in advance is determined for each pixel, and a defective pixel is determined when the feature vector is in the abnormal class.

  According to a second aspect of the present invention, in the first aspect of the present invention, a feature vector is obtained for each pixel of each image using a plurality of sample images, and an identification boundary line or surface between a normal class and an abnormal class is obtained for each pixel by learning. It has been determined in advance.

  According to a third aspect of the invention, in the first aspect of the invention, the positions of the target image and the plurality of sample images are aligned in advance.

  The invention of claim 4 is characterized in that, in the invention of claim 2, the discrimination boundary line or surface of the normal class and the abnormal class for each pixel is an equal probability ellipse or an equal probability ellipsoid.

  The invention of claim 5 is characterized in that, in the invention of claim 2, the plurality of sample images are images of samples randomly extracted from the manufacturing process of the inspection object.

  According to a sixth aspect of the present invention, in the second aspect of the present invention, the average, variance, and covariance between each feature amount are updated and stored each time the sample image increases, so that after learning ends. Using these calculated values, an identification boundary line or surface is calculated.

  The invention of claim 7 is the invention of claim 1, wherein both the sample image and the target image are color images, and the feature vectors are r (red) density values, g (green) density values, and b (blue) density values. .

  According to the first aspect of the present invention, a feature vector composed of a feature quantity such as a density value is extracted for each pixel, and the positional relationship of the feature vector with respect to the normal class / abnormal class identification boundary line or surface obtained in advance is determined for each pixel. Since the defective pixel is determined when the characteristic vector is in the abnormal class, the abnormal pixel in the edge portion of the image can be accurately found.

  According to a second aspect of the present invention, in the first aspect of the present invention, a feature vector is obtained for each pixel of each image using a plurality of sample images, and an identification boundary line or surface between a normal class and an abnormal class is obtained for each pixel by learning. Since it is obtained in advance, an appropriate identification boundary line or surface reflecting a plurality of sample images can be obtained.

  In the invention of claim 3, in the invention of claim 1, since the positions of the target image and the plurality of sample images are registered in advance, subsequent processing, that is, extraction of feature vectors and determination of defective pixels are facilitated. It can be carried out.

  Since the invention according to claim 4 uses the equiprobability ellipse or the equiprobability ellipsoid as the discrimination boundary line or surface between the normal class and the abnormal class for each pixel in the invention of claim 2, it is a defective pixel in the conventional method. Nevertheless, a pixel that has been determined to be a normal pixel can be correctly determined as a defective pixel.

  In the invention of claim 5, in the invention of claim 2, since the sample images randomly extracted from the manufacturing process of the inspection object are used as the plurality of sample images, the defective samples are continuously connected to the equal probability ellipse. Alternatively, the equal probability ellipsoid becomes large, and an identification boundary line or surface that includes a feature vector of a defective sample is not generated.

  According to a sixth aspect of the present invention, in the second aspect of the present invention, the average, variance, and covariance between each feature amount are updated and stored each time the sample image increases, so that after learning ends. Since the identification boundary line or surface is calculated using these calculated values, the storage capacity is greatly reduced, and the image defect detection method according to claim 2 can be executed even by a computer having a small storage capacity.

  According to the seventh aspect of the invention, the feature vector is set to the r (red) density value, the g (green) density value, and the b (blue) density value, so that the sample image of the color image and the target image of the color image are also claimed. The invention of 1 can be applied and the same effect can be expected.

FIG. 2 shows an image defect detection method of the present invention for detecting a defect in a target image in pixel units. In a pixel (i, j), the plurality of characteristic amounts x _1, x _2, ..., and x _m. The feature amount is, for example, x ₁ is a density value, x ₂ is an edge strength, x ₃ is an average density value of neighboring pixels, and the like. The number m of feature quantities is an integer greater than or equal to 2 and may be any number. In FIG. 2, for convenience, m = 3. When both the sample image and the target image are color images, r (red) density value, g (green) density value, and b (blue) density value can be used as the feature amount.

Assume that there are a normal class and an abnormal class in the x ₁ x ₂ ... x _m feature space. Wherein the feature amount of the pixel of the target image vector _{X = (x 1, x 2} , ..., x m) when the feature vector X is determined as a normal pixel if and only if they belong to a normal class, if and only if they belong to the defect classes It is determined as an abnormal pixel.

  Specifically, a feature vector consisting of feature values such as density values, edge strengths, and average density values of neighboring pixels is extracted for each pixel, and features for the normal class and abnormal class identification boundary lines or surfaces obtained in advance. The positional relationship of the vectors is determined for each pixel, and a defective pixel is determined when the feature vector is in the abnormal class.

  It should be noted that a feature vector is obtained for each pixel of each image using a plurality of sample images, and a normal class / abnormal class identification boundary line or surface is obtained in advance for each pixel by learning. At this time, the discrimination boundary line or surface between the normal class and the abnormal class for each pixel is an equal probability ellipse or an equal probability ellipsoid. The plurality of sample images are sample images randomly extracted from the manufacturing process of the inspection object (claim 5).

FIG. 3 shows the conventional technique shown in paragraph number 0005. In this method, the reason why it is difficult to correctly determine the defect at the edge of the pattern is that the standard deviation σ _ij of the sample image of the pixel (i, j) at the edge is large even if there is a defect on the edge. This is because even if the density value f _ij of the defective pixel is different from the average value μ _{ij for} each pixel of the sample image, it is determined to be normal.

On the other hand, in the method of the present invention shown in FIG. 4, a feature space is used for each pixel, and whether the feature vector X extracted for each pixel is inside or outside the identification boundary is a normal pixel or an abnormal pixel. Such a problem does not occur because the determination is made. Incidentally, the FIG. 4, m = 2, the density value of the pixel of the edge portion on the feature amount x _1, the feature quantity x ₂ is used the edge intensities of the pixels of the edge portion. Thus, a positive correlation or a negative correlation is usually found between the density value of a certain pixel in the edge portion and the edge intensity. Both positive and negative correlations are seen because the edge strength takes the absolute value of the difference value. Since the distribution of density values of a certain pixel in the sample image may be considered as a normal distribution centered on the average value μ _ij , the distribution of the edge intensity of the pixel may be considered as a normal distribution. Of course, the average value and the variance are different from those for the density value. For this reason, the equal probability line is an ellipse, and the identification boundary line is an equal probability ellipse. When there are three feature quantities, the identification boundary is a surface, which is an equal probability ellipsoid. When there are four or more feature quantities, the discrimination boundary becomes a hyperplane, which is an equiprobable hyperellipsoid. In this specification, for the sake of convenience, when there are three or more feature quantities, without distinguishing between these planes / hyperplanes or between ellipsoids / superellipsoids, “the identification boundary is a plane and an equiprobable ellipsoid” I will express that.

  In the conventional method of FIG. 3, there is only one feature quantity, so it was not possible to detect the pixel abnormality at the edge portion.However, since the technique of the present invention in FIG. By doing so, it is possible to detect an abnormality of a pixel in an edge portion (having a feature vector X belonging to an abnormality class).

  Then, although it seems that such a method of the present invention is merely the feature amount m of the conventional method of FIG. 3 increased from 1 to 2 or more, this is not the case. The conventional method of FIG. 3 is a statistical method in which the lower limit is μ−cσ and the upper limit is μ + cσ. This is shown in a flowchart in FIG. If the same statistical method is used and there are two feature values m, the same concept is used, as shown in FIG.

  In this way, with a simple combination of statistical methods, the feature vector X is determined to be a normal pixel even though it is an abnormal pixel (FIG. 7). (It is in the pixel determination area (halftone dot portion) and is determined to be a normal pixel. Actually, the pixel within the identification boundary line on the left in FIG. 7 is normal.)

  On the other hand, the method of the present invention uses not only a combination of statistical methods but also an ellipse (or an ellipsoid) as an identification boundary line (or surface) by learning. Therefore, the feature vector X outside the identification boundary line is used as an abnormal pixel. Can be determined correctly.

Embodiments of the present invention will be described below with reference to the drawings. First, n sample images aligned with each other are prepared (FIG. 8), and feature values such as density values and edge strength are extracted for each pixel. These feature amounts are assumed to be x ₁ , x ₂ ,..., X _m . When looking at a certain pixel of interest A, the feature quantity of each sample image is plotted in the m-th order x ₁ x ₂ ... X _m feature space (FIG. 9). Since the characteristic amount of each sample can be generally approximated by a normal distribution, a curve or curved surface having the same probability Φ is an ellipse or an ellipsoid.

  In FIG. 9, equal probability ellipsoids are drawn for Φ = 0.682, 0.955, and 0.997, respectively. These are equal probability ellipsoids (claim 4) each having a probability density of one, two, and three times the standard deviation of each feature quantity. The inside of this ellipsoid is statistically considered to be a normal class with reliability of 68.2%, 95.5%, and 99.7%, respectively.

Now, an equiprobability ellipse or ellipsoid with a reliability of 99.7% may be regarded as a discrimination boundary between a normal class and an abnormal class. _{This, c 1, c 2, ...} , c m an arbitrary constant, the standard deviation sigma ₁ of each feature quantity, sigma _2, ..., when a sigma _m, is equal probability ellipse or ellipsoid. Here, c ₁ = c ₂ = ... = c _m = 3. The line or surface of an equal probability ellipse or ellipsoid gradually changes the shape of the boundary line or boundary surface as the number of sample images increases. Specifically, the center position, the radius of each axis, and the inclination of the equal probability ellipse or ellipsoid change. However, as the number of sample images increases, the change in the shape of the boundary line or boundary surface converges.

  Note that the equiprobability ellipse or ellipsoid stores the average, variance, and covariance between each feature quantity in each pixel of each sample image without having to store all feature quantities. It is possible to calculate the center of an equal probability ellipse or ellipsoid, the radius of each axis, and the inclination by using these calculated values at the end of learning. For this reason, the storage capacity is greatly reduced. That is, when new learning samples are added, it is only necessary to recalculate each calculated value instead of recalculating the equal probability ellipse or ellipsoid.

  Subsequently, when a new target image is given, first, alignment with the learning sample image is performed (claim 3).

Then, feature amounts x ₁ , x ₂ ,..., X _m are obtained for all pixels. This is a feature vector X. When the feature vector X of the pixel A at the same position as the sample image of a certain target pixel is given, this is voted on the x ₁ x ₂ … x _m feature space of the learning sample as shown in Fig. 2, and the feature vector X is identified If it is within an equiprobable ellipse or ellipsoid that is a boundary, it is determined as a normal pixel, otherwise it is determined as an abnormal pixel (claim 1).

Note that all of the learning samples in the method of the present invention are not necessarily normal, and may be samples considered to be normal products. Such a sample may contain about 0.3% of bad samples, but the isoprobability ellipse or the equiprobability surface of the ellipsoid is originally Φ = 100% if c ₁ = c ₂ =… = c _m = 3 However, since Φ = 99.7%, even if a defective sample is included in the learning sample, the corresponding abnormal pixel is positioned outside the equal probability ellipse or ellipsoid. Of course, if the proportion of defective samples is large, the constant c _1, c _2, ..., may be set small c _m. By the way, if a bad sample continues continuously, an equal probability ellipse or an ellipsoid will become large and will contain the feature vector of a bad sample, and it is inconvenient. However, such a situation does not occur as long as the learning sample is randomly extracted from the manufacturing process of the test object that is the population (claim 5).

  Since the method of the present invention can accurately find abnormal pixels at the edge of an image, a plurality of learning samples and targets can be processed by image processing such as printed circuit board pattern inspection, heterogeneous component inspection, and package print error inspection. The present invention can be widely applied to image defect detection that compares and detects defects.

It is explanatory drawing of the defect detection method of the conventional image. It is explanatory drawing of the defect detection method of the image of this invention. It is explanatory drawing of the conventional method shown to the paragraph number 0005. It is explanatory drawing of the method of this invention. 10 is a flowchart of pixel unit determination (m = 1) according to a conventional method. It is a flowchart of the pixel unit determination (m = 2) by a conventional method. It is explanatory drawing which compared the normal pixel determination area | region in the pixel unit determination (m = 2) by the conventional method, and the identification boundary in the method (m = 2) of this invention. It is explanatory drawing of the n sample images used in the method of the present invention and aligned with each other. FIG. 5 is an explanatory diagram of an equal probability ellipse or ellipsoid obtained as a result of plotting the feature amount of each sample image in an m-th order x ₁ x ₂ ... X _m feature space when viewing a certain pixel of interest A.

Claims (7)

  In the method of comparing the sample image with the target image and detecting defects in the target image in units of pixels, feature vectors consisting of feature values such as density values are extracted for each pixel, and the normal class and the abnormal class obtained in advance A method for detecting a defect in an image, wherein a positional relationship of a feature vector with respect to an identification boundary line or a surface is determined for each pixel, and a defective pixel is determined when the feature vector is in an abnormal class.   The feature vector is obtained for each pixel of each image using a plurality of sample images, and a discrimination boundary line or surface between a normal class and an abnormal class is obtained in advance for each pixel by learning. Image defect detection method.   The image defect detection method according to claim 1, wherein positions of the target image and the plurality of sample images are registered in advance.   3. The image defect detection method according to claim 2, wherein the discrimination boundary line or surface between the normal class and the abnormal class for each pixel is an equal probability ellipse or an equal probability ellipsoid.   3. The image defect detection method according to claim 2, wherein the plurality of sample images are images of samples randomly extracted from the manufacturing process of the inspection object.   Each feature quantity average, variance, and covariance between each feature quantity are updated and stored as the number of sample images increases, and the calculated boundary value or surface is calculated using these calculated values after learning. 3. The image defect detection method according to claim 2, wherein:   2. The defect detection of an image according to claim 1, wherein both the sample image and the target image are color images, and the feature vectors are r (red) density value, g (green) density value, and b (blue) density value. Method.

Images