WO2020175666A1

WO2020175666A1 - Color filter inspection device, inspection device, color filter inspection method, and inspection method

Info

Publication number: WO2020175666A1
Application number: PCT/JP2020/008236
Authority: WO
Inventors: 戸塚　貴之; 岡沢　敦司; 泰孝尾崎; 俊晃上原; 貴司北口
Original assignee: 大日本印刷株式会社
Priority date: 2019-02-28
Filing date: 2020-02-28
Publication date: 2020-09-03
Also published as: TW202034421A; JP7415286B2; TWI822968B; JPWO2020175666A1

Abstract

A color filter inspection device (1) is provided with: a defect detection unit (12) that detects defect candidates on the basis of a captured image of a color filter; and a defect output unit (21) that determines whether each defect candidate is a defect by comparing at least one physical quantity of the detected defect candidate with a threshold of the physical quantity, and outputs defect candidates other than the defect candidates determined as not defects to a neural network for defect classification.

Description

\¥0 2020/175666 1 ?01/^2020/008236

Specification

Title of invention:

Color filter inspection device, inspection device, color filter inspection method and inspection method

Technical field

The present invention relates to a color filter inspection device, an inspection device, a color filter inspection method, and an inspection method.

Background technology

[0002] Conventionally, in the color filter manufacturing line, as one of the defects generated in the color filter substrate due to the fluctuation of the manufacturing conditions, a non-uniform region (unevenness) such as an opening area or the thickness of the colored layer is detected. A color filter unevenness inspection device is used. The color filter unevenness inspection device is a device that detects unevenness by performing image processing on a captured image of a color filter substrate. There is a correlation between the type of unevenness (attribute items determined by shape, area, shading, etc.) and the causal process. If the device detects unevenness, the operator determines the type of unevenness from the image of unevenness and The manufacturing conditions of the causative process were adjusted to suppress the occurrence.

[0003] However, it is very difficult for a worker to discriminate the type of the irregularity from the irregularity image, and it is very difficult to properly perform the discrimination even if the worker changes. High degree, automation is desired.

In addition, an image classification device that can be used for visual inspection of semiconductor substrates and the like is disclosed in "201 17-5 4 2 3 9 8". This technology extracts feature quantities from captured images and classifies defects, and classifies defects using neural networks, decision trees, and semi-differential analysis using teacher data.

[0004] However,

The technology disclosed in 2 0 1 7-5 4 2 3 9 8 detects defects based on the difference image between the captured image and the reference image. Color filter unevenness often appears as a very subtle difference in light and shade, and the method that uses the difference image is highly likely to cause false detection. \¥ 0 2020/175666 2 (:171? 2020/008236

Was not suitable. In addition, in the technique disclosed in “2 0 1 7— 5 4 2 3 9 8”, the feature quantity of the defect is input to the classifier to determine the type of the defect. However, when constructing the classifier, , It was necessary to decide which feature quantity to use, and it was difficult to use.

Disclosure of the invention

An object of the present invention is to provide a color filter inspection device, an inspection device, a color filter inspection method and an inspection method capable of simply and accurately detecting a defect due to unevenness of a color filter and classifying the defect. That is.

[0006] [Means for Solving the Problems]

The present invention solves the above problems by the following solution means. It should be noted that, in order to facilitate the understanding, the reference numerals corresponding to the embodiments of the present invention will be given and described, but the present invention is not limited thereto.

[0007] The present invention compares a defect detection unit that detects a defect candidate based on a captured image of a color filter, at least one physical quantity of the detected defect candidate, and a threshold of the physical quantity, A color filter inspection apparatus comprising: a defect output unit that determines whether the defect candidate is not a defect and outputs defect candidates other than the defect candidate that is determined not to be a defect to a neural network for defect classification. Is.

[0008] The present invention provides a defect detection unit that detects a defect candidate based on a captured image of a color filter, a defect output unit that outputs the detected defect candidate to a neural network for defect classification, A color filter inspection device comprising: a classification determination unit that determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the color filter.

[0009] The first analysis result may include generation position information indicating a generation position of a defect for each classification.

The color filter inspection device further includes a defect determination unit that determines whether or not the defect candidate for which the result of the defect classification has been determined is a defect based on the second analysis result of the captured image of the color filter. You may have it. \\0 2020/175666 3 (:171? 2020/008236

[001 1] The second analysis result may include generation history information indicating a defect generation history for each classification.

[0012] The neural network is configured to perform a convolution process from the captured image,

1 a convolutional layer that generates a feature map, a pooling layer that performs pooling processing to reduce the size or change of the first feature map to generate a second feature map, and an output layer that outputs the result of the defect classification, May be provided.

The color filter inspection device may further include an image cutout unit that cuts out a range including the defect candidate detected by the defect detection unit from the captured image.

[0014] The defect detection unit may obtain a range including a defect candidate from data obtained by first differentiating the captured image from a plurality of directions.

According to the present invention, by comparing a defect detection unit that detects a defect candidate based on a captured image of an object, at least one physical quantity of the detected defect candidate with a threshold value of the physical quantity. A defect output unit that determines whether the defect candidate is not a defect, and outputs defect candidates other than the defect candidate determined not to be a defect to a neural network for defect classification. is there.

The present invention includes: a defect detection unit that detects a defect candidate based on a captured image of an object; a defect output unit that outputs the detected defect candidate to a neural network for defect classification; An inspection apparatus comprising: a classification determination unit that determines the result of the defect classification output from the neural network based on a first analysis result of a captured image of an object.

According to the present invention, the defect detection unit detects a defect candidate based on a captured image of a color filter, and the defect output unit includes at least one physical quantity of the detected defect candidate, and By comparing with the physical quantity threshold value, it is determined whether the defect indicator is not a defect, and defect candidates other than the defect candidate determined not to be a defect are output to the neural network for defect classification. A method for inspecting a color filter, comprising:

According to the present invention, the defect detection unit is configured to detect defect candidates based on a captured image of a color filter. \\0 2020/175 666 4 (:171? 2020/008236

The defect output unit outputs the detected defect candidates to a neural network for defect classification, and the classification confirmation unit outputs the first analysis result of the captured image of the color filter. And a step of determining the result of the defect classification output from the neural network based on the result.

[0019] The present invention includes: a photographing unit photographing the color filter; a convolution process in which the neural network generates a first feature map from the photographed image by a convolution process; and a pooling process. A pooling process for reducing the size or change of one feature map to generate the second feature map may be further provided.

According to the present invention, the defect detecting unit detects a defect candidate based on a captured image of an object, and the defect output unit includes at least one physical quantity of the detected defect candidate and the physical quantity. By comparing with the threshold value of, to determine whether the defect candidate is not a defect, and output the defect candidates other than the defect candidate determined to be not a defect to the neural network for defect classification, It is an inspection method that includes

According to the present invention, the defect detection unit detects a defect candidate based on a captured image of an object, and the defect output unit uses the detected defect candidate for a dual network for defect classification. And a step of causing the classification determination unit to determine the result of the defect classification output from the neural network based on the first analysis result of the captured image of the object. is there.

According to the present invention, it is possible to provide a color filter inspecting device, an inspecting device, a color filter inspecting method and an inspecting method capable of easily and accurately detecting a defect of a color filter and classifying the defect. it can.

Brief description of the drawings

FIG. 1 is a diagram showing a first embodiment of a color filter inspection device 1 according to the present invention.

FIG. 2 is a flow chart showing a flow of operations of the color filter inspection device 1. \\0 2020/175666 5 (:171? 2020/008236

[Fig. 3] Fig. 3 is a diagram showing the configuration of the neural network of the defect classification unit 15 and the learning process.

[FIG. 4] A diagram for explaining average pooling.

[FIG. 5] A diagram showing an example in which a differential image is converted into a binary image.

[FIG. 6] FIG. 6 is a diagram showing an example in which a region formed of a set of adjacent pixels having a first pixel value is extracted from the binary image of FIG. 5.

FIG. 7 is a diagram showing a second embodiment of the color filter inspection device 1 according to the present invention.

[FIG. 8] A flowchart showing the flow of operations in the color filter inspection apparatus 1 according to the second embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0024] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. In the drawings referred to in the respective embodiments, the same portions or portions having similar functions are denoted by the same reference numerals or similar reference numerals, and repeated description thereof is omitted.

[0025] (First Embodiment)

FIG. 1 is a diagram showing an embodiment of a color filter inspection device 1 according to the present invention.

The color filter inspecting apparatus 1 is an apparatus for inspecting manufacturing defects in the manufacturing process of the color filter. Here, an example of examining the unevenness of the color filter, which is an example of the object, is exemplified. It may be for inspecting for defects.

The color filter inspection apparatus 1 according to the present embodiment includes an imaging unit 11, a defect detection unit 12, an image cutout unit 13, an input unit 14, a defect classification unit 15, and a learning model construction unit 1. 7 and. The defect detection unit 12, the image cutout unit 13, the input unit 14, the defect classification unit 15, and the learning model construction unit 17 are configured by incorporating a dedicated program into the computer. The program realizes the functions of each component. \\0 2020/175666 6 卩 (: 171? 2020 /008236

The image capturing unit 11 includes an illumination, a camera, a transport device, and the like (not shown), captures a color filter for an inspection, and captures a captured image. The photographed image photographed by the photographing unit 11 is sent to the defect detection unit 12 and the image cutout unit 13. The camera of the image capturing unit may be a line sensor camera or an area sensor camera.

The image capturing unit 11 may be configured to receive the reflected light from the color filter or to receive the transmitted light.

[0027] The defect detection unit 12 detects a defect candidate that may be uneven from an image by image processing. As the unevenness detection method performed by the defect detection unit 12 of the present embodiment, the method disclosed in Japanese Patent No. 43.63953 is used. That is, the defect detection unit 12 performs a first-order differentiation process on the captured image using a spatial filter to determine the area where unevenness is expected to exist, and the spatial change rate of the gradation value for this area. Areas where there is a possibility of unevenness are detected by using as an evaluation value indicating the degree of unevenness.

The image cutout unit 13 cuts out a predetermined range including the defect candidate detected by the defect detection unit 12 from the photographed image, and acquires a nonuniformity peripheral image. The image cutout unit 13 sends the cutout unevenness peripheral image and the captured image to the input unit 14

The input unit 14 inputs the unevenness peripheral image and the captured image to the input layer 1 of the defect classification unit 15.

5 Input to 1.

By detecting defect candidates by the defect detection unit 12 and cutting out the image by the image cutout unit 13, the processing by the defect classification unit 15 described later can be lightened.

The defect classifying unit 15 is composed of a neural network, analyzes the mura peripheral image by artificial intelligence, classifies the types of the mura, and outputs the categorized defects. The defect classification unit 15 has a learning model 16 and classifies irregularities (classification of defects) by comparing the peripheral image of irregularities with the learning model 16. Here, the unevenness of the color filter may vary depending on the cause of the unevenness. \\0 2020/175 666 7 卩 (: 171? 2020 /008236

State, for example, shape, size, area, etc. Therefore, by classifying the unevenness, it is possible to identify the cause of the unevenness (defect). Further, since the detection of the defect detection unit 12 may include erroneous detection, it is possible to classify this erroneous detection as well.

Further, the defect classifying unit 15 includes a learning model 16.

The learning model 16 is constructed by learning the learning data 18 (image group for each type of unevenness) in advance by the learning model constructing unit 17.

The learning model building unit 17 builds a learning model from the learning data 18. As described above, the learning model construction unit 17 constructs the learning model 19 by learning the learning data 18 (image group for each type of unevenness) in advance before executing the color filter inspection. Then, it shifts to the learning model 16 of the defect classification unit 15. The learning model 19 is composed of parameter sets used in the neural network.

Next, the operation of the color filter inspection apparatus 1 and the configuration and operation of the defect classification unit 15 will be described in more detail.

FIG. 2 is a flow chart showing the operation flow of the color filter inspection device 1.

In step (hereinafter referred to as “3”) 11, the image capturing unit 11 captures an image by capturing an image of the color filter.

In 3 1 2, the defect detection unit detects defect candidates. In the present embodiment, the defect detection unit 12 performs a first-order differentiation process using a spatial filter on the image acquired in step 11 to extract a region where unevenness is expected.

In 3 13, the image cutout unit 13 cuts out a predetermined range including a defect candidate from the photographed image, and acquires a nonuniformity peripheral image.

In 3 1 4, the input unit 14 inputs the non-uniformity peripheral image (cut out image) to the input layer of the defect classification unit 15.

[0034] In 315, the defect classifying unit 15 classifies defects from the uneven peripheral image (cut out image). \¥0 2020/175666 8 卩 (: 171? 2020 /008236

A method of detecting the defect candidate 3 1 2 will be described. A differential image is created by determining the differential direction on the captured image and obtaining the first-order differential in the differential direction for the grayscale value of each pixel that constitutes the image.

FIG. 5 is a diagram showing an example of converting a differential image into a binary image.

A predetermined threshold is set, and for the differential image, the first pixel value is given to pixels with pixel values above the threshold, and the second pixel is given to pixels with pixel values below the threshold. By giving the pixel value, the differential image is made into a binary image as shown in Fig. 5.

FIG. 6 is a diagram showing an example of extracting a region consisting of a set of adjacent pixels having the first pixel value in the binary image of FIG.

A region consisting of a set of adjacent pixels having the first pixel value is extracted from this binary image as shown in Fig. 6. The pixel group forming this area is set as a set of a plurality of one-dimensional pixel arrays along the differentiation direction, and the difference in gradation value of the pixels located at both ends of this one-dimensional pixel array is taken as the length of the one-dimensional pixel array. The representative value of the multiple evaluation values obtained for multiple one-dimensional pixel arrays is used as the evaluation value indicating the non-uniformity for the one-dimensional pixel array, and the value obtained by dividing the value by Use as an evaluation value. If the evaluation value is high, the unevenness is noticeable from the surroundings. The evaluation value is obtained for each differential direction, and if a certain predetermined condition is satisfied, it is evaluated that the mura exists in the evaluation target area.

Even if the size and shape of the uneven area cannot be predicted, the process of 3 1 2 can accurately evaluate the presence or absence of unevenness, so that the area with defect candidates can be accurately detected in the defect classification unit 1 5 Can be passed to the Neural Network.

The operation of the defect classifying unit 15 in 3 15 will be described together with the more specific configuration of the defect classifying unit 15.

FIG. 3 is a diagram showing the structure of the neural network of the defect classification unit 15 and the learning process.

The neural network of the defect classifying unit 15 of this embodiment has an input layer 1 51, a convolutional layer 1 52, a pooling layer 1 5 3, a fully connected layer 1 5 4 and an output layer 1 5 4. Layers 1 5 5 and.

The input layer 1 5 1 is a layer that receives an input of the uneven peripheral image from the input unit 14.

In the example of FIG. 3, it is shown that an image of 64×64 pixels is input, but this can be changed appropriately.

[0039] The convolutional layer 1 5 2 performs a convolution process using a coefficient matrix of an arbitrary size. Here, convolution processing is performed using the 3 x 3 matrix coefficient, and correction is performed using the bias value.

In the convolution processing, the first small image of 3 x 3 pixels is extracted from the input image of 6 4 x 6 4 pixels, the convolution calculation is performed using this small image and the 3 x 3 coefficient matrix, and the bias value and the bias coefficient are calculated. Multiply by and add ReLU (Rect if ied Near Near It) to generate the first feature map. Although FIG. 3 exemplifies an example in which 6 4 convolutional images are created using 6 4 types of matrix coefficients, this can be appropriately changed.

In the pooling layer 153, pooling processing is performed on the first feature map to obtain the second feature map. The pooling process reduces the size or change of the first feature map generated in the convolutional layer 152 to generate the second feature map. For example, a 2×2 pixel image is extracted from the first feature map, and the maximum brightness and average brightness of this image are calculated. Specifically, average pooling, maximum pooling,

L p Perform pooling, etc.

FIG. 4 is a diagram illustrating average pooling.

In the example of average pooling shown in Fig. 4, the size is reduced by dividing the pooling area into 2 x 2 pixels and averaging the luminance values.

[0041] The fully connected layer 154 in Fig. 3 combines the 64 second feature maps to create fully connected data.

[0042] The output layer 155 applies a parameter set (weighting parameter, bias parameter) to all the combined data, and outputs the classification results of nine types of defects using the activation function. The parameter set (weight parameter, bias parameter) is applicable to all 64 second feature maps. [0043] The parameter set (weighting parameter, bias parameter) is set using the back propagation method based on the learning data 18. The output error is calculated from the learning data 18, the parameter set (weighting parameter, bias parameter) is updated by the gradient descent method using the least squares method for the error function, and the parameter setting is performed by repeating the learning multiple times. Value (weight parameter, bias parameter).

[0044] In Fig. 3, the convolutional layer and the pooling layer are each formed as one layer to configure the network. However, a network including multiple convolutional layers and pooling layers is configured, and the convolutional process and the pooling process are predetermined. It may be repeated as many times as desired. Also, the coefficient matrix used for the convolutional layer is learned in advance and its value is fixed and fixed, but the convolutional layer and the parameter set (weight parameter, bias parameter) of all combined data may be learned at the same time. ..

[0045] In this embodiment, a deep lane earning technology called C onvolution Neural Network (CNN) can be used, and activation functions such as sigmoidal functions and R e Lu functions are combined in multiple layers. Connected networks may be formed.

The configuration example of the neural network described above is merely an example, and can be changed as appropriate.

As described above, the color filter inspecting apparatus 1 of the present embodiment can accurately classify defects by using the defect classifying unit 15 that uses the neural network. Therefore, it becomes possible to detect defects in the manufacturing process early and improve the manufacturing process efficiently.

[0047] For example, a specific mura caused by foreign matter in the step of applying a photosensitive material to a substrate can be recognized as a specific mura by a skilled worker, but even a particular mura may have a shape, an area, and a gradation. There are large variations. Therefore, it is difficult to determine a rule to discriminate a specific unevenness based on the shape, area, density, etc.However, by using the defect classifier 15 using a neural network, it is possible to accurately identify a specific irregularity. Can be determined. Foreign matter in the process of applying the photosensitive material to the substrate \\0 2020/175666 1 1 卩 (: 171? 2020 /008236

Since it is known that there is a correlation with unevenness, the worker who knows the occurrence of specific unevenness can expect early improvement by checking and removing foreign matter in the process of applying the photosensitive material to the substrate.

(Second Embodiment)

In addition to the configuration of the first embodiment, the color filter inspection apparatus 1 of the second embodiment further includes a defect classification unit 21 which is an example of a defect output unit, a classification confirmation unit 2 2 and a defect determination unit 2 3 It has and. The defect classification unit 21, the classification determination unit 22 and the defect determination unit 23 are configured by incorporating a dedicated program into the computer, and the program realizes the functions of each configuration.

The defect detection unit 12 detects a defect candidate that may be uneven, based on the captured image of the color filter taken by the image capturing unit 11. The defect detection unit 12 may directly detect the defect candidate from the photographed image, but it is preferable to detect the defect candidate from the image obtained by preprocessing the photographed image to improve the accuracy of the defect determination.

[0050] The defect classification unit 21 calculates at least one physical quantity of the defect candidates detected by the defect detection unit 12 and compares the calculated physical quantity with the threshold value of the physical quantity, so that the defect candidate is not a defect. Or not. The defect classification unit 21 outputs the defect candidates other than the defect candidates determined not to be defects to the dual network, that is, the defect classifier 15 for defect classification. —, The defect classification unit 21 does not output the defect candidates determined not to be defects to the defect classifier 15.

The classification confirming unit 22 confirms the result of defect classification output from the defect classifier 15 based on the first analysis result of the captured image of the color filter acquired in advance. Specifically, the classification determination unit 22 determines the defect classification by comparing the defect classification with the generation position information indicating the generation position of the defect for each classification which is an example of the first analysis result. .. \¥02020/175666 12 (:171?2020/008236

The defect determination unit 23 determines whether or not the defect candidate whose defect classification has been determined is a defect, based on the second analysis result of the captured image of the color filter acquired in advance. Specifically, the defect judgment unit 23 compares the defect indicator whose defect classification has been confirmed with the occurrence history information indicating the defect occurrence history for each classification which is an example of the second analysis result. , It is determined whether the defect candidate whose defect classification is confirmed is a defect.

[0053] Next, the operation of the color filter inspection apparatus 1 according to the second embodiment will be described in more detail.

FIG. 8 is a flow chart showing a flow of operations of the color filter inspection device 1 in the second embodiment.

[0054] First, in 321, the photographing unit 11 acquires a photographed image obtained by photographing the color filter.

After the captured image is obtained, in 3 2 2, the defect detection unit 12 performs preprocessing for improving the defect determination accuracy for the captured image. The pre-processing is, for example, a series of processing including shading processing, smoothing processing, radiation processing, and black-and-white inversion processing.

The shading process is a process for removing density unevenness from an image having density unevenness.

The smoothing process is a process of blurring a captured image in order to remove noise other than defect candidates included in the captured image. For smoothing processing, for example, Gaussian filter can be used.

The radiating process is a process required for a convolution calculation at the time of detecting a defect candidate, which will be described later, and is a process of copying the luminance value of the pixel located at the edge of the captured image to the outside thereof. Radiation processing is insufficient due to the lack of pixels around the target pixel when the pixel located at the edge of the captured image is used as the target pixel, that is, the center pixel, for example, 3 x 3 pixels are used for the registration. Is done to make up for.

The black-and-white reversal process is a process that black-and-white inverts a captured image when a black defect is the target of determination and does not invert a captured image when a white defect is the target of determination. \\0 2020/175 666 13 卩 (: 171? 2020 /008236

It Whether the judgment target is a black defect or a white defect depends on the kind of the defect to be judged. According to the black-and-white reversal process, a white image can be detected and determined as a defect candidate regardless of the type of defect, so that the defect determination can be simplified.

The above-mentioned shading processing, smoothing processing, radiation processing and black-and-white inversion processing that constitute the pre-processing may be appropriately replaced before and after.

After performing the pre-processing, in 323, the defect detection unit 12 performs a defect candidate detection process of detecting a defect candidate from the pre-processed captured image. The defect candidate detection process includes, for example, a difference filter process by a convolution operation, a binarization process, a horizontal direction closing process, a vertical direction closing process, a horizontal direction opening process, a vertical direction opening process, and a labeling process. It is composed of a series of processing together with processing.

The difference filtering process by the convolution calculation is a process of obtaining the magnitude of the change in the brightness value at each point, that is, each pixel of the captured image after preprocessing. In a place where there is a defect candidate such as unevenness or foreign matter, the brightness value changes greatly compared to the surroundings, so it is possible to detect the defect candidate by obtaining the magnitude of the change in the brightness value by differential filtering. it can. Specifically, in the differential filter processing, each pixel of the captured image after pre-processing is set as the target pixel in order, and the brightness value of each of the 3×3 pixels centered on the target pixel and the filter For example, a convolution operation is performed using the 3 x 3 coefficient matrix and. By the convolution calculation, the difference value of the brightness value between the pixel of interest and the adjacent pixel is calculated. The greater the difference in the brightness value, the larger the difference value, and it is possible to detect the place having the larger difference value as a defect candidate.

[0057] The binarization processing is based on the captured image after the difference filter processing by the convolution operation, and generates a binary image in which pixels having a difference value of the image above the threshold value are white and pixels below the threshold value are black. It is a process to do. Through the binarization process, areas with large changes in brightness value are detected as white images, that is, defect candidates, and areas with small changes in brightness value, that is, black images are displayed except for defect candidates. \¥0 2020/175666 14 卩 (: 171? 2020 /008236

[0058] The horizontal direction closing process is a process of expanding and contracting the white area of the binary image in the horizontal direction, that is, in the horizontal direction.

[0059] The vertical direction closing process is a process of expanding and contracting the white area of the binary image in the vertical direction, that is, in the vertical direction.

[0060] According to the horizontal direction closing process and the vertical direction closing process, holes existing in a white area can be crushed and white areas existing nearby can be stuck together. The accuracy of defect candidates can be improved.

[0061] The horizontal opening process is a process of contracting and expanding the white area of the binary image in the horizontal direction.

The vertical opening process is a process of contracting and expanding the white area of the binary image in the vertical direction.

[0063] According to the horizontal opening process and the vertical opening process, it is possible to remove small white lumps or to join two white areas that are close to each other. The accuracy of can be improved.

[0064] The labeling process is a process of assigning a number to each pixel in the white area of the binary image. In the labeling process, for example, the same number is assigned to a plurality of pixels belonging to the white region of a continuous block, and the numbers assigned to pixels are different between white regions of different blocks. Let By performing the labeling process, it becomes possible to classify defects by blob analysis in the defect classification process described later.

Note that the above-described difference filter processing, binarization processing, horizontal closing processing, vertical closing processing, horizontal direction opening processing, vertical direction opening processing, and labeling processing that constitute the defect candidate detection processing are The front and rear of these may be replaced appropriately.

[0065] After performing the defect candidate detection processing, in 324, the defect classification unit 21 determines the defect candidates of different numbers subjected to the labeling processing, that is, the defect areas individually for each white area. \¥0 2020/175 666 15 (:171? 2020/008236

Perform classification processing. The defect classification process calculates at least one physical quantity for each detected defect candidate and compares the calculated physical quantity with a threshold value of the physical quantity to determine whether the defect candidate is a defect. This is a process of outputting defect candidates other than the defect candidate determined to be not to the defect classifier 15. For example, defect classification processing includes horizontal width calculation processing, vertical width calculation processing, area calculation processing, luminance difference peak value calculation processing, luminance difference average value calculation processing, slope rate calculation processing, and area ratio calculation processing. It is configured by a series of processing including a calculation process, a streak rate calculation process, a circularity calculation process, a threshold value determination process, and an output process.

The width calculation process is a process of calculating the width of the defect candidate as one of the physical quantities of the defect candidate. For example, the width is calculated as the distance between the two points when a straight line passing through the center of gravity of the defect candidate intersects with both ends in the X direction of the defect candidate.

The vertical width calculation processing is processing for calculating the vertical width of the defect candidate as one of the physical quantities of the defect candidate. The vertical width is obtained as the distance in the direction of the defect between two points when a straight line passing through the center of gravity of the defect candidate intersects with both ends of the defect candidate in the direction of the defect.

The area calculation process is a process of calculating the area of the defect candidate as one of the physical quantities of the defect candidate.

The brightness difference peak value calculation process is a process of calculating the maximum difference value of the defect candidates by the difference filtering process as one of the physical quantities of the defect candidates.

[0070] The brightness difference average value calculation process is a process of calculating the average value of the brightness differences of the defect candidates as one of the physical quantities of the defect candidates.

[0071] The slope rate calculation process is a process of calculating the variation amount of the brightness difference of the defect candidate as one of the physical quantities of the defect candidate.

[0072] The area ratio calculation process is defined as one of the physical quantities of the defect candidates so as to include the whole of one defect candidate and to circumscribe the outermost end of the defect candidate in the X direction and the defect direction. This is a process of calculating the defect candidate in the rectangular area, that is, the area ratio of the white area and the black area. \\0 2020/175 666 16 卩 (: 171? 2020 /008236

The streak rate calculation process is a process of considering the defect candidate as one rectangular streak as one of the physical quantities of the defect candidate and calculating the ratio between the long side and the short side of the streak. More specifically, the streak rate calculation processing includes the long side and the short side of a rectangular area that is defined so as to include the entire one defect candidate and circumscribe the outermost end in the X direction and the direction of the defect of the defect candidate. Is a process of calculating the ratio of

The circularity calculation process is a process of calculating the circularity of the defect candidate as one of the physical quantities of the defect candidate. Circularity, an area of the defect candidate 3, in the case where the peripheral length of the defect candidate! _ And may be calculated by 4 3 / 1_ ^2.

The above-mentioned width calculation processing, height calculation processing, area calculation processing, brightness difference peak value calculation processing, brightness difference average value calculation processing, slope rate calculation processing, area ratio calculation processing, streak rate calculation processing and circularity calculation processing , These may be replaced before and after as appropriate

[0075] The threshold value determination process is carried out individually for each defect candidate of different lumps, with each of the physical amounts of the defect candidates calculated by the various calculation processes described above as a determination threshold value for each physical amount preset for each physical amount. This is a process of determining whether or not the defect candidate is not a defect by comparing. By the threshold determination process, defect candidates of different chunks are sorted into defect candidates determined not to be defects and defect candidates other than defect candidates determined not to be defects. Defect candidates other than the defect candidates that are determined not to be defects have not yet been determined to be defects at this point. Therefore, the defect candidates other than the defect candidates that are determined not to be defects may include not only the defect candidates that are finally determined to be defects but also the defect candidates that are suspicious of not being defects.

The output processing outputs the defect candidates other than the defect candidates determined not to be the defect by the threshold determination processing to the defect classifier 15 and the defect candidates determined not to be the defect by the threshold determination processing to the defect classifier 1 It is a process that does not output to 5. As a result, only defect candidates other than the defect candidates determined not to be defects are the targets of defect classification using the neural network, and the defect candidates determined not to be defects are the targets of defect classification using the neural network. Excluded from. \\0 2020/175 666 17 卩 (: 171? 2020 /008236

After performing the defect classification processing, at 3 25, the defect classifier 15 performs the defect classification processing. The defect classification process is a process for classifying the defect candidates other than the defect candidates that are determined not to be defects by the threshold value determination process using the neural network as in the first embodiment. ..

After performing the defect classification processing, at 3 26, the classification determination unit 22 performs the classification determination processing for determining the defect classification of the defect candidate output from the defect classifier 15. The classification determining unit 22 compares the defect classification with the occurrence position information indicating the occurrence position of the defect for each classification, and the position of the classified defect candidate is the occurrence position indicated in the occurrence position information of the corresponding classification. Defect classification is confirmed when On the other hand, the classification determination unit 22 does not determine the defect classification when the position of the classified defect candidate does not match the occurrence position indicated in the occurrence position information of the corresponding classification.

The classification confirming unit 22 may perform the classification confirming process by further using the threshold value of the reliability of the defect classification set individually for each classification in addition to the occurrence position information.

After performing the classification confirmation processing, at 327, the defect determination unit 23 performs a defect determination processing for determining whether or not the defect candidate whose defect classification has been determined is a defect. The defect determining unit 23 determines whether or not the defect candidate is a defect by comparing the defect candidate whose defect classification has been determined with the occurrence history information indicating the defect occurrence history for each classification. The defect determination unit 23 determines that the defect candidate is defective when the defect candidate whose defect classification has been confirmed matches the occurrence history indicated in the occurrence history information of the corresponding classification. The occurrence history may be, for example, the number of consecutive occurrences of defect candidates. On the other hand, the defect determination unit 23 determines that the defect candidate is not a defect when the defect candidate whose defect classification has been determined does not match the occurrence history indicated in the occurrence history information of the corresponding classification. In addition, the defect determination unit 23 determines that a defect candidate whose defect classification has not been determined is a defect that does not belong to any classification.

The defect determination unit 23 may perform the defect determination process by further using the defect intensity indicating the defect density, that is, the brightness difference, in addition to the occurrence history information. \¥0 2020/175666 18 卩 (: 171? 2020 /008236

According to the second embodiment, by combining the defect classification using the neural network and the defect judgment method using the judgment criterion other than the neural network, it is possible to use only the neural network. By comparison, it is possible to perform highly accurate defect determination.

[0083] (Variations)

The present invention is not limited to the embodiment described above, and various modifications and changes can be made, which are also within the scope of the present invention.

(1) In the first embodiment, an example in which a defect candidate is detected by the defect detection unit 12 and an image is cut out by the image cutout unit 13 to lighten the processing by the defect classification unit 15 described later I explained it. The invention is not limited to this, and the defect classification unit 15 may directly process the captured image of the color filter. In this case, the defect classification unit 15 outputs the defect classification from the entire image in the neural network.

(2) In the first embodiment, the defect detection unit 12 will be described with reference to an example in which the defect candidate is detected by using the method disclosed in Japanese Patent No. 4 3 6 3 9 5 3. did. The present invention is not limited to this, and a defect candidate may be detected using a conventionally known defect detection method.

The present invention can also be applied to defect inspection of objects other than color filters. For example, the present invention can be applied to a film, glass, silicon, metal or the like for inspecting a defect of an object having an appearance to be inspected by coating or self-luminous. The appearance of the inspection object is not limited to one that can be detected under visible light, and may be one that can be detected under infrared light or ultraviolet light. The present invention can also be applied to inspect defects in medical radiographic images.

The embodiments and modified embodiments may be used in combination as appropriate, but detailed description thereof will be omitted. Further, the present invention is not limited to the embodiments described above.

Explanation of symbols

[0088] 1 Color filter inspection device \¥02020/175666 19 卩<:171? 2020/008236

1 1 Shooting section

1 2 Defect detection section

1 3 Image cropping section

1 4 Input section

1 5 Defect classification section

1 6, 1 9 Learning model

1 7 Learning model construction department

1 8 learning data

1 5 1 Input layer

1 52 Imposition layer

1 53 Pooling layer

1 54 Fully coupled layer

1 55 Output layer

Claims

\¥0 2020/175 666 20 (:171? 2020/008236 Claims

[Claim 1] A defect detection unit that detects a defect candidate based on a captured image of a color filter,

By comparing at least one physical quantity of the detected defect candidate with the threshold value of the above-mentioned physical quantity, it is determined whether or not the defect candidate is a defect, and a defect other than the defect candidate determined not to be a defect is detected. A color filter inspection device comprising: a defect output unit that outputs candidates to a neural network for defect classification.

[Claim 2] A defect detection unit for detecting a defect candidate based on a captured image of a color filter,

A defect output unit that outputs the detected defect candidates to a neural network for defect classification,

A classification determination unit that determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the color filter,

A color filter inspection device including.

3. The color filter inspection device according to claim 2, wherein the first analysis result includes generation position information indicating a generation position of a defect for each classification.

4. A defect determination unit for determining whether or not the defect candidate for which the result of the defect classification has been determined is a defect based on the second analysis result of the captured image of the color filter. The color filter inspection device according to item 2 or 3.

5. The color filter inspection device according to claim 4, wherein the second analysis result includes generation history information indicating a generation history of defects for each classification.

[Claim 6] The neural network is

A convolutional layer that generates a first feature map from the captured image by convolutional processing;

Performs a pooling process to reduce the size or change of the first characteristic map \\0 2020/175 666 21 卩 (: 171? 2020 /008236

And a pooling layer to generate a second feature map,

An output layer for outputting the result of the defect classification,

The color filter inspection device according to any one of claims 1 to 5, further comprising:

7. The color filter inspection according to any one of claims 1 to 6, further comprising an image cutout unit that cuts out a range including the defect candidate detected by the defect detection unit from the captured image. apparatus.

8. The color filter inspection apparatus according to claim 7, wherein the defect detection unit obtains a range including a defect candidate from a data obtained by first-order differentiating the captured image from a plurality of directions.

[Claim 9] By comparing a defect detection unit that detects a defect candidate based on a captured image of an object, at least one physical quantity of the detected defect candidate, and a threshold of the physical quantity described above, An inspection apparatus comprising: a defect output unit that determines whether or not a defect candidate is a defect and outputs defect candidates other than the defect candidate determined to be not a defect to a neural network for defect classification.

[Claim 10] A defect detection unit that detects a defect candidate based on a captured image of an object; a defect output unit that outputs the detected defect candidate to a neural network for defect classification;

An inspection apparatus, comprising: a classification determination unit that determines a result of the defect classification output from the neural network based on a first analysis result of a captured image of the object.

[Claim 11] A step in which the defect detection unit detects a defect candidate based on a captured image of a color filter,

The defect output unit determines whether or not the defect candidate is a defect by comparing at least one physical quantity of the detected defect candidate with a threshold value of the physical quantity, and it is determined that the defect candidate is not a defect. Other than defect candidates \¥0 2020/175666 22 卩 (: 171? 2020 /008236

The process of outputting defect candidates to a neural network for defect classification,

A method for inspecting a color filter, comprising:

[Claim 12] a step in which the defect detection unit detects a defect candidate based on a captured image of the color filter,

The defect output unit outputs the detected defect candidates to a neural network for defect classification,

A step of determining the defect classification result output from the neural network based on the first analysis result of the captured image of the color filter,

A method for inspecting a color filter, comprising:

[Claim 13] A step of photographing the color filter by a photographing unit,

A convolution step in which the neural network generates a first feature map from the captured image by convolution processing;

The color filter inspection method according to claim 11 or 12, further comprising: a pooling step of performing a pooling process to reduce the size or change of the first feature map to generate a second feature map.

[Claim 14] A step in which the defect detection unit detects a defect candidate based on a captured image of an object,

The defect output unit determines whether or not the defect candidate is a defect by comparing at least one physical amount of the detected defect candidate with a threshold value of the physical amount, and it is determined that the defect candidate is not a defect. The process of outputting defect candidates other than the defect candidates to the neural network for defect classification,

Inspection method comprising.

[Claim 15] A process in which the defect detection unit detects a defect candidate based on a captured image of an object, \¥0 2020/175 666 23 (:171? 2020/008236

A defect output unit outputs the detected defect candidates to a neural network for defect classification;

A step in which the classification determining unit determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the object;

Inspection method comprising.