JP2015197396A - Image inspection method and image inspection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image inspection method and an image inspection device capable of easily specify a process in which the abnormal portion of an object to be inspection has occurred.SOLUTION: An image inspection device 100 determines, on the basis of the image of an object to be inspected that was captured after the end of each of a plurality of processes, the abnormality mode of an abnormal portion present in the object to be inspected after the end of each process, generates mapping data that includes the abnormality mode of the abnormal portion and the coordinates of the abnormal portion, standardizes the sizes of a plurality of mapping data on the basis of dimensional changes of the object to be inspected in the plurality of processes, classifies into groups the same abnormal portion included in two or more mapping data among a plurality of standardized mapping data, and specifies a process in which the abnormal portion has occurred.

Description

  The present invention relates to an image inspection method and an image inspection apparatus, and more particularly to an image inspection method and an image inspection apparatus for inspecting an inspection object in a production line including a plurality of steps for manufacturing the inspection object.

  2. Description of the Related Art Conventionally, there has been known a technique for inspecting a product and detecting a target object by capturing an image of the target object, acquiring an image, and executing processing on the acquired image. As such a technique, for example, there is a technique using an MTS (Mahalanobis-Taguchi System) method.

  Japanese Patent Laying-Open No. 2010-276481 (Patent Document 1) discloses an image processing apparatus. The image processing apparatus captures an inspection target product and acquires an image composed of a large number of pixels, and determines whether the inspection target product is a non-defective product or a defective product based on the image acquired by the imaging unit. Part. The calculation parameter of the storage unit includes a luminance value at each pixel of each image constituting the non-defective image group and a luminance difference value between the pixel adjacent to the pixel at each pixel of each image constituting the non-defective image group. Then, it is calculated from a correlation coefficient matrix of the luminance value of the pixel and surrounding pixels in each pixel of each image constituting the non-defective image group. The determination unit calculates a statistical distance between the image of the inspection target product and the non-defective image group data, and determines whether or not the calculated statistical distance is within a predetermined range.

  JP-A-11-306325 (Patent Document 2) discloses an object inspection apparatus. The target object inspection apparatus designates the position of the collation local region to which the region model is fitted by the collation region position designation unit with respect to the input image including the face image captured from the image input unit. The object inspection apparatus performs luminance normalization by the luminance normalization unit for each designated local region to be verified, and further performs image processing such as edge detection by the image processing unit. Then, the object inspection apparatus normalizes the local area to be verified by the determination element acquisition unit and applies the face area model to the image that has been subjected to image processing, so that the feature amount of each determination element acquisition area in the area model And the Mahalanobis distance is calculated for each local area to be checked based on the feature quantity extracted by the Mahalanobis distance determination unit, and the face is detected based on the calculation result.

  JP-A-8-327561 (Patent Document 3) discloses a glass plate defect inspection apparatus. The defect inspection apparatus continuously captures a bright-field image, a dark-field image, and a transmission image of a glass plate using a plurality of one-dimensional line cameras provided in the conveyance direction of the glass plate, and obtains three types of obtained images. Based on the image, foreign matter, bubbles, surface scratches, and surface irregularities present on the glass plate are determined. .

JP 2010-276481 A JP-A-11-306325 JP-A-8-327561

  However, with the conventional technology, it is possible to determine the quality of an inspection object, but when an abnormal part occurs in an inspection object manufactured through multiple processes, the abnormal part occurs in which process. It was difficult to identify what was done.

  Therefore, a main object of the present invention is to provide an image inspection method and an image inspection apparatus capable of easily specifying a process in which an abnormal part of an inspection object has occurred.

  An image inspection method according to the present invention is an image inspection method for inspecting an inspection object in a production line including a plurality of processes for manufacturing the inspection object, and the inspection object is imaged after completion of each of the plurality of processes. On the basis of the image, it is determined which abnormal mode the abnormal part existing in the inspection object after completion of each process belongs to, and the abnormal mode of the abnormal part and the abnormality in the inspection object Based on the first step of generating mapping data including the coordinates of the part and the dimensional change of the inspection object in the plurality of processes, so that the coordinates of the same abnormal part included in the plurality of mapping data match. The second step of normalizing the size of the mapping data and the same abnormal part included in two or more of the standardized mapping data are grouped. And up of, in which a third step of identifying the abnormal portion occurs process.

  According to the image inspection method according to the present invention, it is possible to easily identify in which process the abnormal portion of the inspection object has occurred.

It is a block diagram which shows the hardware constitutions of the image inspection apparatus according to one embodiment of this invention. It is a flowchart which shows the manufacturing process of the metal plate which is a test object of the image inspection apparatus demonstrated in FIG. It is a figure which shows the outline | summary of the test process provided with the image test | inspection apparatus demonstrated in FIG. FIG. 4 is a diagram illustrating a non-defective image captured by the CCD camera illustrated in FIG. 3. It is a figure which shows the abnormal product image imaged with the CCD camera shown in FIG. It is a figure which shows the original image before an image process by the image inspection apparatus shown in FIG. 3, and the image after a process. FIG. 4 is a block diagram illustrating a functional configuration of the image inspection apparatus illustrated in FIG. 3. It is a figure which shows typically an example of the information stored in the information storage part shown in FIG. It is a flowchart which shows operation | movement of the image inspection apparatus shown in FIG. It is a figure for demonstrating normalization of the mapping data shown in FIG.

<A. Overview>
The image inspection apparatus according to the present embodiment is an image inspection apparatus that inspects an inspection object in a production line including a plurality of processes for manufacturing the inspection object, and is an inspection object that is imaged after completion of each of the plurality of processes. Based on the image of the object, the abnormal part existing in the inspection object after completion of each step is determined to which of the abnormal modes belong to the abnormal mode, and the abnormal mode of the abnormal part and the inspection object A data generation unit that generates mapping data including the coordinates of the abnormal part, and a plurality of coordinates so that the coordinates of the same abnormal part included in the plurality of mapping data match based on the dimensional change of the inspection object in the plurality of processes. The normalization part that normalizes the size of mapping data and the same abnormal part included in two or more of the standardized mapping data are grouped Is obtained by a specifying unit configured to specify an abnormal portion occurs process.

Hereinafter, the image inspection apparatus as described above will be described in detail.
<B. Hardware configuration>
FIG. 1 is a block diagram showing a hardware configuration of the image inspection apparatus according to the present embodiment. Referring to FIG. 1, an image inspection apparatus 100 mainly includes a CPU (Central Processing Unit) 102, a keyboard 104 and a mouse 106 that receive an instruction input by a user of the image inspection apparatus 100, a memory 108, A display 110 for displaying various information, a memory interface (I / F) 115, and a communication interface (I / F) 116 are included. Each hardware is mutually connected by a data bus.

  The CPU 102 controls each unit of the image inspection apparatus 100 by reading and executing a program stored in the memory 108. More specifically, the CPU 102 implements each process (step) of the image inspection apparatus 100 described later by executing the program. The CPU 102 is, for example, a microprocessor. The hardware may be an FPGA (Field Programmable Gate Array) other than the CPU, an ASIC (Application Specific Integrated Circuit), a circuit having other arithmetic functions, or the like.

  The memory 108 is realized by a RAM (Random Access Memory), a ROM (Read-Only Memory), a hard disk, or the like. The memory 108 stores programs and data executed by the CPU 102.

  The processing in the image inspection apparatus 100 is realized by each hardware and a program executed by the CPU 102. Such a program may be stored in the memory 108 in advance. The program may be stored in the recording medium 114 and distributed. Alternatively, the program may be provided as a downloadable program by an information provider connected to the so-called Internet. Such a program is stored in the memory 108 after being downloaded through the memory interface 115 or the communication interface 116.

  Each hardware which comprises the image inspection apparatus 100 shown by FIG. 1 is common. Therefore, the operation of each hardware of the image inspection apparatus 100 is well-known and will not be described in detail.

<C. Inspection object>
Here, a case where image inspection apparatus 100 according to the present embodiment uses a metal plate as an inspection object will be described.

(C1. Metal plate manufacturing process)
Image inspection apparatus 100 according to the present embodiment is arranged on a production line including a plurality of processes for manufacturing an inspection object, and removes an inspection object based on an image of the inspection object captured in each of the plurality of processes. Inspect by destruction.

  First, a metal plate manufacturing process will be described as an example of such a manufacturing line. The metal plate as the inspection object passes through the manufacturing process while being conveyed, and the process proceeds to the inspection process. In the inspection process, the presence or absence of abnormality of the target object and the abnormal mode are inspected using the captured image of the target object obtained from the line camera, which is an imaging unit arranged so as to include a part of the conveyance path in the imaging field of view. Is done.

  FIG. 2 is a flowchart showing an example of a metal plate manufacturing process. Here, the metal plate is assumed to be a metal thin film of 1 mm or less. In the metal plate manufacturing process, first, the melting device melts a metal plate such as copper or gold (step ST1), and the casting device casts it into an ingot (step ST3). Next, the rolling device rolls the cast ingot with a plurality of rolls to thin the metal plate (step ST5). Next, the degreasing and cleaning device removes the oil and fat adhering to the metal surface (degreasing) and cleaning (step ST7). At this time, in the process of thinning the metal plate, there may be a defect included during melting of the metal, a stain attached to the surface, a scratch, or the like.

  Next, in the leveling step, the surface of the metal plate is flattened by the small diameter roller (step ST9). Next, the inspection device inspects the appearance of the metal plate to determine whether the product is good or defective (step ST11). And a metal plate is cut | disconnected by predetermined width (step ST13), and is shipped as a product (step ST15).

  In such a manufacturing process, the material is stretched in the conveying direction by rolling, but the stretching ratio can be estimated from the thickness of the metal.

  In the manufacturing process, the metal plate may be cut in the width direction, and this width is also monitored. In a manufacturing process, after a metal plate is once wound up in a roll shape, there is a process of re-winding around another roll in a separate process. At this time, the front and back of the product at the time of imaging are reversed.

  Since the product is conveyed at a high speed, an error is likely to occur in the coordinate position of the abnormal portion obtained from the number of rotations of the roll. It is also effective to fill in the position information with the laser mark, but it may cause malfunction of the product (the product breaks when tension is applied, the mark part cannot be used), the mark reading mechanism, the recording of position information, etc. The system becomes complicated.

  In addition, most of the foreign matter adhering in the degreasing and cleaning processes is redeposition of organic matter, so light of wavelengths in the ultraviolet or X-ray region is irradiated onto the metal surface, and the image of the light emitting part at that time is displayed. By acquiring, it can be distinguished from metal foreign matter (no foreign matter light emission) adhering during rolling and leveling.

  Next, an inspection process for inspecting the metal plate based on the image of the metal plate taken by the line camera will be described.

  FIG. 3 is a diagram showing an outline of the inspection process. In FIG. 3, the inspection process provided in the leveling process is illustrated. Referring to FIG. 3, the metal plate is transported to the inspection process when the surface is flattened in the leveling process. And a metal plate is taken up by the roll, after imaged with the line camera installed in the up-down direction in the state irradiated with light from the light. The line camera is, for example, a CCD (Charge Coupled Device) camera. The line camera transmits image data indicating the captured image of the surface of the metal plate to the image inspection apparatus 100. The image inspection apparatus 100 receives image data captured through the communication I / F 116 and stores it in a predetermined area of the memory 108.

  Similarly, for example, a CCD camera is used as the line camera, but it has a mechanism for setting the wavelength of the irradiated light to ultraviolet light or the wavelength of the X-ray region and capturing a fluorescent image obtained at that time.

  For reference, FIGS. 4A and 4B show an example of an image of a normal portion of the metal plate surface imaged by the line camera according to the present embodiment. With reference to FIGS. 4A and 4B, it can be seen that there is no characteristic portion in the image of the normal part image of the metal plate (hereinafter also referred to as “non-defective image”). That is, it can be seen that there is no characteristic difference between the non-defective image shown in FIG. 4A and the good image shown in FIG.

  On the other hand, FIGS. 5A to 5F are diagrams illustrating examples of images in which abnormal portions are observed on the surface of the metal plate imaged by the line camera. Referring to FIGS. 5A to 5F, an image having an abnormal portion of a metal plate (hereinafter also referred to as “abnormal product image”) is compared with the non-defective product images of FIGS. 4A and 4B. You can see that there is a characteristic part in the image. Here, the abnormal portion observed on the metal plate is classified into, for example, six types of abnormal modes M1 to M6 shown in FIGS.

  The abnormal part of mode M1 in FIG. 5A is due to precipitates formed during metal melting. Further, in the abnormal portion of mode M2 in FIG. 5B, a black streak structure is formed in the vicinity of the center of the metal plate, which indicates a scratch generated during rolling. The abnormal part of the mode M3 in FIG. 5C shows a black circular region formed near the center of the metal plate, which is due to a dent during manufacture.

  The abnormal part of mode M4 in FIG. 5D shows a relatively thick region extending in the lateral direction near the center and the upper part of the metal plate. This is due to deformation of the metal plate surface. The abnormal part of mode M5 in FIG. 5 (E) shows a streak line formed on the surface of the metal plate, which is due to the adhering of metal scrap generated when the metal (copper) plate is cut. The abnormal part of mode M6 in FIG. 5 (F) shows a pattern that spreads radially from the central part of the surface of the metal plate. This is due to organic substances adhering during cleaning. From FIGS. 5A to 5F, it can be seen that the shape and size of the abnormal part, which is a feature of the image, differ depending on the abnormal mode.

  In addition, since the abnormal part of mode M1 of FIG. 5 (A) is based on the precipitate formed at the time of metal melting, there is no correlation with apparatus parameters, such as the diameter of a rolling roll, the width | variety between rolls. Since the abnormal portion of mode M2 in FIG. 5B is a scratch generated during rolling, there is a correlation with the apparatus parameters of a specific rolling roll. Since the abnormal portion of mode M3 in FIG. 5C is a dent at the time of manufacture, the image positions on the front surface and the back surface are the same.

  Since the abnormal portion of mode M4 in FIG. 5D is due to minute deformation of the metal surface, correspondence with the roller interval and the roller diameter in the leveling process can be seen. In the abnormal part of the mode M5 in FIG. 5E, a streak-like line is included, and this is because the metal scrap generated at the time of cutting the copper is attached, so that it can be identified as generated in the cutting process. The abnormal part of mode M6 in FIG. 5F is an organic substance adhering during cleaning. In the case of an organic material, an abnormal shape similar to that of a normal image can be seen in a fluorescent image.

<D. Mahalanobis distance and calculation method>
First, the Mahalanobis distance calculation method according to the present embodiment will be briefly described. Specifically, this is a calculation method based on the MT (Mahalanobis-Taguchi) method.

For example, the acquired data is Y _ip , the number of data items is i = 1 to k, the number of data sets is p = 1 to n, the average value M _i of the data Y _ip for each data item, and the standard deviation σ _i Calculate

Next, the standardized value y _ip is calculated by the following equation (1) using the average value M _i and the standard deviation σ _i .

In the present embodiment, if the acquired data Y _ip is 100 images classified into the abnormal mode using nine types of feature values (1) to (9) described later for each abnormal mode, 9 × 100 total 900 pieces. As described above, the number of data items is nine, which is the feature amount of each image, and the number of data sets is the number of images classified for each mode (about 100 each).

Next, using the normalized value y _ip , a correlation matrix R of the normalized values is created by the following equations (2) and (3).

Next, an inverse matrix A _MT of the correlation matrix R is created by the following equation (4).

  Finally, the Mahalanobis distance D2 is calculated by the following equation (5).

<E. Feature Extraction>
Next, the following image processing is performed in order to extract the image feature amount necessary for image analysis.

  First, an image with a changed image resolution is generated during image processing. Specifically, an image to be inspected with the resolution changed in three stages of high resolution, medium resolution, and low resolution was used. As the image to be inspected, an image of an abnormal portion captured by reflected light of normal visible light (natural light) and an image captured by a CCD of fluorescence emitted when irradiated with light having a wavelength in the ultraviolet or X-ray region are used. That is, the number of images is six for one abnormal part.

For these six images, the following image processing was performed in order to emphasize the feature amount of the image.
(E1. Image smoothing and binarization by discriminant analysis)
Here, smoothing of an image based on a Gaussian distribution and binarization processing by image processing using a discriminant analysis method will be described. First, in order to remove image noise, the luminance value of the image is flattened based on the Gaussian distribution. Next, binarization processing to which the discriminant analysis method is applied is executed. The discriminant analysis method is a method of obtaining a threshold value that maximizes the separation metrics and automatically performing binarization. The degree of separation can be calculated by the ratio of between-class variance and within-class variance.

When binarization is performed with the threshold value t, the number of pixels on the side having a smaller luminance value than the threshold value t (black class) is ω ₁ , the average is m ₁ , the variance is σ ₁ , and the pixel having the larger luminance value (white class) When the number is the number of pixels ω2, the average is m ₂ , the variance is σ ₂ , the total number of pixels in the image is ω _t , the average is m _t , and the variance is σ _t , the intra-class variance σ ² _w is ).

Next, the interclass variance σ ² _{b and the} total variance σ _t are calculated by the following equations (7) and (8), respectively.

Therefore, the degree of separation, which is the ratio between the interclass variance σ ² _b and the intraclass variance σ ² _w , is calculated by the following equation (9).

Then, a threshold value t that maximizes the degree of separation is calculated, and binarization processing is automatically performed.
(E2. Binarization processing by average value, labeling processing and expansion processing)
Next, in order to extract the metal plate portion to be inspected from the image captured by the line camera, the binarization processing is performed by the above method in which the region (metal plate portion) to be extracted is determined in advance. Mask processing was performed using the converted image. That is, a portion other than the metal plate was deleted from the image captured by the line camera (the metal plate portion was extracted). In addition, about each image imaged with the line camera, after implementation of a binarization process, you may return to the state before a process and may extract a metal plate part.

  Next, after deleting portions other than metal, image noise is deleted and an effective inspection region is extracted. Here, the effective inspection area is obtained by excluding noise generated by a factor different from the factor determined as a defective product based on the binarized image. That is, an area other than the area to be actually inspected is excluded from the image. Here, it will be described that a shadow portion due to metal edge undulation that is not an abnormal portion is excluded as image noise by image processing.

  First, binarization processing is performed on an image using an average value of luminance of pixels as a threshold value. Next, a labeling process is performed on the image after the binarization process. Here, the labeling process is a process of assigning the same number to pixels in which a white area (or black area) is continuous in a binarized image, and a pixel assigned the same number is a single object. It is possible to grasp. Next, an area for acquiring the feature quantity of the image (hereinafter also referred to as “feature quantity acquisition area”) is specified. This is specified, for example, by analyzing the position on the image where the object of the white area extracted when the image is labeled.

  Next, expansion processing for reversing black and white is also performed on the peripheral portion in the specified feature amount acquisition region. The features of the region on the image are emphasized by unifying adjacent pixels that are different in white and black into white or black by expansion processing.

(E3. Feature amount extraction process)
FIG. 6A shows an original image before the above image processing is performed, and FIG. 6B shows an image after the above image processing is performed. In the present embodiment, the following image feature amount extraction is performed using the image after image processing.

The following feature amounts (1) to (9) for each image in the feature amount acquisition region specified by the image processing as described above are extracted.
(1) Number of abnormal portions in one image (2) Width of abnormal portion (3) Aspect ratio of abnormal portion (4) Pixel area of abnormal portion (5) Perimeter of abnormal portion (6) Circular shape of abnormal portion Including degrees (circularity is expressed in 4πS / L)
(7) Curvature of the outer shape of the abnormal portion (8) Luminance integrated value of the abnormal portion (9) Position of the center of gravity of the abnormal portion A known method can be used to calculate these feature amounts, so the description will be repeated here. For example, a method for calculating the luminance integral value will be briefly described.

  The X-axis (horizontal direction) and Y-axis (vertical direction) orthogonal to the partial image of the feature amount acquisition region are set, and the pixel luminance in the direction in which the Y-axis extends in a specific X coordinate is integrated, or a specific value A value obtained by integrating pixel luminances in the direction in which the X axis extends in the Y coordinate is acquired as a feature amount. Note that the plurality of integral values calculated for each of a plurality of coordinates may be acquired as feature amounts. That is, for example, an integrated value of pixel luminance in the direction in which the Y axis extends in the X coordinate may be acquired as one feature amount, and an integrated value of pixel luminance in the direction in which the X axis in the Y coordinate extends may be acquired as another feature amount.

  In the present embodiment, when calculating the Mahalanobis distance, at least one or more of the feature quantities (1) to (9) may be used. Moreover, you may use the feature-value different from said 9 types.

<F. Functional configuration>
Next, in the image inspection according to the present embodiment, a plurality of abnormal product images for each abnormal mode obtained by imaging a plurality of abnormal product objects that are known to be abnormal products in advance and the abnormal mode of the corresponding manufacturing process. The image group which consists of is acquired. As described above, a plurality of abnormal product images are prepared in advance for each of the above abnormal modes. In the present embodiment, this is also referred to as a reference image for each abnormal mode. For each abnormal mode, the image inspection apparatus 100 calculates and acquires a representative value of the Mahalanobis distance (hereinafter sometimes referred to as reference Mahalanobis) of a plurality of reference images corresponding to the abnormal mode. At the time of inspection, the abnormal mode to which the image to be inspected belongs is determined from the distance to the reference Mahalanobis in each abnormal mode with respect to the feature quantity of the image to be inspected. Here, an average value of Mahalanobis distances of a plurality of reference images is used as the reference Mahalanobis, but a mode value or a median value may be used as long as it is a representative value.

(F1. Device configuration)
FIG. 7 is a block diagram showing a functional configuration of image inspection apparatus 100 according to the present embodiment.

  Referring to FIG. 7, the image inspection apparatus 100 has, as a functional configuration for calculating the Mahalanobis distance for the reference image of each abnormal mode, an abnormal product feature amount extraction unit 32 and a reference space configuration unit 34 corresponding to the abnormal mode M1, An abnormal product feature amount extraction unit 42 and a reference space configuration unit 44 corresponding to the abnormal mode M2 and an abnormal product feature amount extraction unit 52 and a reference space configuration unit 54 corresponding to the abnormal mode M3 are provided. Here, for the sake of simplicity, illustration is omitted, but the abnormal product feature amount extraction unit and the reference space configuration unit corresponding to the abnormal modes M4 to M6 can be provided in the same manner as the abnormal modes M1 to M3. .

  In addition, the image inspection apparatus 100 includes a feature amount extraction unit 20 as a functional configuration related to calculation of the Mahalanobis distance of the inspection target image.

  In addition, the image inspection apparatus 100 includes an information storage unit 70 corresponding to a predetermined storage area of the memory 108 that stores information for determining an abnormal mode of an image to be inspected, and a reference Mahalanobis for each abnormal mode regarding the feature amount of the inspection target image. The distance calculation unit 90 for each mode for calculating the distance to the distance (hereinafter, sometimes referred to as the distance calculation unit 90), and the abnormal mode classification for determining and determining the abnormal mode to which the inspection target image belongs from the calculation result of the distance calculation unit 90 Part 95 is provided. The image inspection apparatus 100 inputs apparatus parameter data 112 indicating characteristics related to the manufacture of an apparatus applied to a metal plate manufacturing process, and inspection target images and process data 118 of each process, and stores them in the information storage unit 70. .

  Furthermore, the image inspection apparatus 100 includes an abnormal part normalization mapping data generation unit 120, an identical abnormality identification unit 122, an abnormality occurrence process identification unit 124, an abnormality cause determination unit 126, an abnormality occurrence distance calculation unit 128, and a factor analysis unit. 130.

  The above-described functions are basically realized by the CPU 102 of the image inspection apparatus 100 executing a program stored in the memory 108 and giving a command to the components of the image inspection apparatus 100. That is, the CPU 102 has a function as a control unit that controls the entire operation of the image inspection apparatus 100. Note that some or all of these functional configurations may be realized by hardware.

(F2. Information storage unit)
FIG. 8 is a diagram schematically illustrating an example of information stored in the information storage unit 70 according to the present embodiment. The area of the information storage unit 70 specifies an area E1 in which data referred to for inspection is stored in association with each other, an area E2 in which abnormal product images are classified and stored for each abnormal mode, and an abnormality cause. And an area E4 for storing inspection target images and process data for each process.

  In the area E1, the abnormal mode identifiers (M1 to M6) and the calculated reference Mahalanobis distances (D1g to D6g) are stored corresponding to the abnormal modes M1 to M6, respectively.

  In the area E2, an abnormal product image of the abnormal mode is stored for each abnormal mode. The abnormal product image includes a reference image prepared in advance and an inspection target image determined to belong to the abnormal mode at the time of inspection.

  In the area E3, abnormality cause identification data 701 is stored for each abnormality mode. The abnormality cause identification data 701 includes abnormal part related data 61 for each abnormal part on the metal plate. The abnormal part related data 61 includes a position parameter 111 indicating the position / cycle of occurrence of the abnormal part, device parameter data 112 indicating parameters relating to the manufacturing apparatus resulting from the occurrence of the abnormality, and a cause of occurrence indicating the cause of the abnormality. It includes data 113 in association with each other. The data in the area E3 is acquired and stored in advance through experiments or the like.

  The position parameter 111 indicates an abnormal portion occurrence position and an occurrence cycle in a metal plate per unit production amount, for example, per lot. At the time of inspection, position information indicating the position of the corresponding metal plate in the one lot is added to the inspection target image. The position information is calculated using the conveyance speed of the metal plate in the production line and the shooting speed (shutter speed) of the line camera. These speeds are generally fixed values. The position information is calculated from both speeds on the line camera side and added to the captured image, or calculated by the image inspection apparatus 100 and added to the image every time the inspection target image is input.

  As the apparatus parameter data 112, for example, the position of the abnormal part indicated by the corresponding position parameter 111 and the image of the abnormal part corresponding to the generation period and the position (coordinate information) of the product, and the manufacturing apparatus related to the generation of the abnormal part are included. Parameters (rolling roll circumference, rolling roll interval, leveling step roll diameter, leveling step roll interval, etc.), product width, shape of the abnormal part, normal mode image and fluorescence mode image One or more parameters resulting from the occurrence of the abnormality are included in the correspondence relationship.

  The occurrence cause data 113 indicates the cause of the occurrence of the abnormality (process type, location of occurrence of abnormality in the production line, etc.).

  In the area E4, the inspection target image and the process data 118 of each process are stored. Process data includes, for example, product processing temperature, product processing solution PH, product processing solution composition, product processing solution dissolved oxygen content, product processing solution viscosity, product processing tension, product Containing the hypochlorous acid concentration of the treatment solution.

(F3. Function of each part)
Referring to FIG. 7, abnormal product feature quantity extraction unit 32 is obtained by imaging with a line camera a plurality of abnormal product objects that are known to be abnormal (defective) products corresponding to mode M1 in advance. A plurality of feature amounts are acquired for each of a plurality of abnormal product images. Specifically, the abnormal product feature amount extraction unit 32 generates a binarized image by binarizing the target image based on the reference threshold, and is determined to be an abnormal product based on the binarized image. An effective inspection area excluding noise caused by a factor different from the above-described factor is extracted, and a feature amount is acquired based on the effective inspection area. The abnormal product feature amount extraction unit 32 acquires the nine types of feature amounts (1) to (9) in each pixel in the feature amount acquisition region specified from the effective inspection region.

  The reference space configuration unit 34 calculates the Mahalanobis distance for each abnormal product image based on the feature values (1) to (9) for each of the plurality of abnormal product images in mode M1. The reference space constituting unit 34 calculates a reference Mahalanobis distance D1g obtained by averaging the Mahalanobis distance for each abnormal product image.

  The reference Mahalanobis calculation method for the abnormal product image of the mode M1 is similarly performed in the abnormal product feature amount extraction unit 42 and the reference space configuration unit 44 for the abnormal product image of the mode M2, and the abnormal product image of the mode M2 is performed. A reference Mahalanobis distance D2g obtained by averaging the Mahalanobis distances for each is calculated. Similarly, a reference Mahalanobis distance D3g obtained by averaging the Mahalanobis distance for each abnormal product image in mode M3 is also calculated in the abnormal product feature amount extraction unit 52 and the reference space configuration unit 54 for the abnormal product image in mode M3. Further, the reference Mahalanobis distances D4g to D6g are calculated for each of the other abnormal modes M4 to M6.

  The information storage unit 70 stores the reference Mahalanobis distances D1g to D6g calculated for the modes M1 to M6 in association with each abnormal mode in the region E1.

  The feature amount extraction unit 20 acquires the above nine types of feature amounts (1) to (9) for the inspected image obtained by imaging the inspection object at the time of inspection. The feature quantity extraction unit 20 realizes the same function as that of the abnormal product feature quantity extraction units 32, 42, and 52 for each abnormal product image.

  The distance calculation unit 90 for each mode calculates the Mahalanobis distance Dо of the inspected image from the feature amount output from the feature amount extraction unit 20.

  The distance calculation unit 90 and the abnormal mode classification unit 95 determine the abnormal mode to which the inspection target image belongs from the Mahalanobis distance Dо of the inspection target image and the reference Mahalanobis distances D1g to D6g of the region E1. Specifically, among the reference Mahalanobis distances D1g to D6g, the difference between the Mahalanobis distance Do and the Mahalanobis distance Do of the image to be inspected is calculated, in other words, the difference between the Mahalanobis distance Do and the abnormal mode corresponding to the smallest difference Mahalanobis distance Do The abnormal mode to which the inspection target image belongs is determined.

  The CPU 102 stores the inspection target image in the area E2 in association with the determined abnormal mode.

  The abnormal part normalization mapping data generation part 120 generates mapping data including the abnormal mode of the abnormal part and the coordinates of the abnormal part in the inspection object, and based on the dimensional change of the inspection object in a plurality of processes, The sizes of a plurality of mapping data are normalized to the same size so that the coordinates of the same abnormal part included in the mapping data match.

  The same abnormality specifying unit 122 specifies and groups the same abnormal parts included in two or more mapping data among a plurality of standardized mapping data. The abnormality cause determination unit 126 identifies a process in which each abnormal part has occurred based on the plurality of standardized mapping data and the grouped abnormal parts. The abnormality cause determination unit 126 determines the cause of the abnormal part based on the abnormal mode of the abnormal part generated in the inspection target and the process in which the abnormal part has occurred, and outputs the determination result to the display 110 or the like. To do.

(F3. Method for identifying the cause process)
Next, a method for identifying the cause process when an abnormality occurs will be described. Process data in each process is stored in the information storage unit 70. In the information storage unit 70, a reference space is created using a process data group when a normal product is manufactured as a reference data group.

  The distance calculation unit 128 when an abnormality occurs calculates a Mahalanobis distance with respect to the created reference space using process data at the time of manufacturing an abnormal product. The factor analysis unit 130 obtains a factor that increases the Mahalanobis distance at the time of occurrence of abnormality by factor analysis, and thereby identifies the cause process of occurrence of the abnormality.

  When calculating the Mahalanobis distance of process data, the correlation coefficient between the process data becomes large, and thus a problem of multicollinearity often occurs. To solve this problem, a generalized inverse matrix is applied instead of performing a normal inverse matrix calculation.

  A method for calculating the Mahalanobis distance D using a generalized inverse matrix is shown below. First, the correlation coefficient matrix R is expressed by the following equation (10).

Here, the element r _ij of the correlation coefficient matrix R is the correlation coefficient of the i-th item and the j-th item for the unit data, and is represented by the following equation (11). However, r _ij = r _ji and r _ij = 1 when i = j.

  Next, a generalized inverse matrix A is calculated. Eigenvalue matrix Λ, eigenvector matrix W, and matrix V are obtained. Based on the following equation (12), an eigenvalue of the correlation coefficient matrix R is obtained.

Eigenvalues λ satisfying Equation (12) are obtained, and arranged in the order of λ ₁ , λ ₂ ,... Λ _k ≧ 0 to obtain a singular value matrix Λ represented by the following Equation (13).

Next, the eigenvector for λ ₁ that satisfies Equation (14), that is, the eigenvector represented by Equation (15) is obtained. Note that λ ₁ is not a singular value having a square root.

Similarly, eigenvectors are obtained for λ _i (i = 1 to k), and an eigenvector matrix W represented by the following equation (16) is obtained.

  Next, a matrix V expressed by the following equation (17) is obtained.

Further, a generalized inverse matrix A expressed by the following equation (18) is obtained. Note that V ^t is a transposed matrix of the matrix V.

Next, the Mahalanobis distance D (single) _{1 is obtained} for each sample of unit data based on the following equation (19).

Similarly, the Mahalanobis distance D (trust) _h of the signal data is obtained based on the formula (20), and the Mahalanobis distance D (not yet) _h of the road data is obtained based on the formula (21).

<G. Processing procedure>
FIG. 9 is a flowchart showing the operation of the image inspection apparatus 100. First, the Mahalanobis distance calculation processing procedure (Mahalanobis calculation step) executed by the image inspection apparatus will be described. As an outline, the image inspection apparatus 100 is captured by a line camera, acquires a feature amount of a metal plate image based on the acquired captured image, and calculates a Mahalanobis distance based on the acquired feature amount. Each step shown below is basically realized by the CPU 102 executing a program.

  In step S10, the CPU 102 acquires an image to be inspected captured by the line camera. Next, in step S12, the CPU 102 extracts the feature amount of the acquired image. That is, the CPU 102 smoothes the captured image using a Gaussian filter. Subsequently, the CPU 102 extracts an abnormal portion on the surface of the metal plate by using a binarization process by a discriminant analysis method for the smoothed image. Subsequently, the CPU 102 performs binarization processing on the image from which the metal plate portion is extracted, using the average value of the luminance of the pixels as a threshold value, and characterizes a plurality of abnormal portions by labeling processing. Subsequently, the CPU 102 specifies a region from which the feature amount is acquired, performs an expansion process on the region, and emphasizes the feature amount of the image.

  Next, the CPU 102 normalizes the acquired feature amount according to a predetermined procedure (step S14). Next, the CPU 102 reads out and acquires an abnormal product image for each abnormal mode from the region E2. Subsequently, the abnormal product feature amount extraction unit and the reference space configuration unit corresponding to each abnormal mode perform the abnormalities by calculating the Mahalanobis distance described above for a plurality of abnormal product images corresponding to the abnormal mode read from the region E2. The mode reference Mahalanobis distance is calculated (step S16). Thereby, the reference Mahalanobis distances D1g to D6g are calculated and stored in the area E1.

  Subsequently, in step S18, the distance calculation unit 90 extracts the feature amount of the inspection target image, and calculates the Mahalanobis distance Dо for each abnormal mode of the inspection target image. The abnormal mode classification unit 95 compares the Mahalanobis distance Dо of the inspection target image with a predetermined threshold value related to the Mahalanobis distance, and when the value is smaller than the predetermined threshold value (YES in step S20), the inspection target image is It is determined that the image is a non-defective image.

  On the other hand, if it is determined that the threshold value is greater than or equal to the predetermined threshold value (NO in step S20), the abnormal mode classification unit 95 determines the Mahalanobis distance Dо of the image to be inspected and the reference Mahalanobis distance D1g of the abnormal modes M1 to M6 read from the region E1. Each of the D6g is compared, and the abnormal mode classification unit 95 specifies (determines) the abnormal mode of the reference Mahalanobis distance that is the closest from the comparison result (step S22).

  The CPU 102 additionally stores the inspection target image for which the abnormal mode is determined as an abnormal product image corresponding to the abnormal mode in the region E2. The added image is included in the reference space data to reconstruct the abnormal mode reference space, and the next image to be inspected is determined. This increases the inspection accuracy as the number of images to be inspected increases.

  In step S24, the process data 118 is acquired and stored in the information storage unit 70. Next, in step S26, the shape change in each process is predicted from the thickness and width of the metal plate.

  Next, in step S28, the shape of the mapping data is normalized with respect to the images classified for each abnormal mode. That is, mapping data standardized to the size defined in the upstream process from the thickness and width of the base material is created.

  Here, mapping data is defined. The metal plate handled in this embodiment has a thickness of about 20 mm in a casting process that is manufactured as a plate material after melting. This plate material is processed into a metal thin plate and metal foil having a thickness of 500 μm to 10 μm while repeating rolling and annealing. At that time, the plate thickness is reduced by rolling, the length of the plate is increased, and finally the length is several hundreds to several thousand meters.

  In the image inspection apparatus 100, the image of the abnormal portion on the surface is stored in association with coordinate information on the metal thin plate or metal thin film. This data is collected for each process, and the position (ie, coordinates) of the abnormal part of the sample surface in that process, the abnormal mode to which the abnormal part belongs, and the image information are stored as one database. Call it.

  With respect to the standardized mapping data, the same abnormality is grouped with respect to the mapping data between processes based on the positional relationship information of the relative distance and relative angle for each abnormality mode. Here, the meaning of the relative position and the relative angle will be described. In the mapping data before normalization, the absolute position of the occurrence point of the abnormal mode differs between processes. Further, in the relative positional relationship between the abnormal points, the angles of other abnormal points based on a certain point differ depending on the rolling direction and the cutting position. Therefore, it is necessary to standardize the sample based on the rolling rate of the sample, the rolling direction, and the shape before cutting between each process. By carrying out standardized mapping in this way, it becomes possible to compare the relative positions and angles of the abnormal parts, and thereby, it is possible to identify abnormalities on the mapping.

  FIGS. 10A and 10B are diagrams showing a method for specifying an abnormality occurrence step in the present embodiment. According to this method, it is possible to specify at which stage of the process the same abnormality specified by grouping has occurred. FIG. 10A shows mapping data for each process recorded by the image inspection apparatus. When casting, the plate thickness is large, and instead the plate length is short. While this is repeated rolling and annealing, the plate thickness is small, and it is processed into a long state in the rolling direction. In the process, scratches on the roll and defects inherent in the metal become apparent and are recorded in the image mapping data as an abnormal part.

  Since the mapping data of each image is stored as one data set for each process, the information cannot be compared. Further, part of the process by cutting and re-rolling of the cut part are added to the process. Since each mapping data is a data set for each process, the occurrence status of abnormalities in the process can be compared between different lots, but the occurrence status of abnormalities in the same lot between processes cannot be compared.

  Therefore, normalization of mapping data is performed as shown in FIG. That is, the position information of the mapping data DA is converted into the position information of the mapping data DB from the thickness information and the cutting information of the plate material. By doing so, it is possible to compare the occurrence positions of abnormalities between processes with the same standardized in-plane information.

  For example, an abnormal portion present in the metal plate after the casting process is present in the metal plate even after the rolling process is completed. Since the length of the metal plate is different before and after the rolling process, in the mapping data DA, the position of the abnormal part after the casting process is different from the position of the abnormal part after the rolling process. However, in the standardized mapping data DB, the position of the abnormal part after the end of the casting process is the same as the position of the abnormal part after the end of the rolling process.

  In addition, in the in-plane information standardized from the mode classification based on the image of the abnormal part described above, the mapping data DB specifying the coordinates of the abnormal part (after normalization) and the abnormal mode can be obtained. By using mapping data including an abnormal mode after normalization, it is possible to compare the presence / absence of a specific abnormality between different processes in the same lot. Therefore, the mapping data of a plurality of processes is compared in step S30 of FIG. 9, the same abnormality is specified in step S32, and the abnormality occurrence process is specified in step S34.

  For example, in the mapping data DB of FIG. 10B, the position and the abnormal mode of the abnormal part existing in the metal plate after the rolling process is the same as the position and the abnormal mode of the abnormal part existing in the metal plate after the end of the casting process. In some cases, it can be determined that the abnormal portion (x) has occurred in the casting process.

  Moreover, in mapping data DB, when the abnormal part which exists in a metal plate after completion | finish of a rolling process does not exist in a metal plate after completion | finish of a casting process, it will discriminate | determine that the abnormal part ((triangle | delta)) newly generate | occur | produced in the rolling process. Can do.

  Similarly, since the abnormality indicated by ☆ seen in the mapping data DB at the time of annealing is not seen at the time of rolling, it can be specified that this abnormality has occurred at the time of annealing. Further, since the linear abnormality observed after the cutting (1/4) process is not observed during the annealing, it can be specified that it has occurred in the cutting (1/4) process. Moreover, since the abnormality shown by ★ after the cutting (1/4) part re-rolling process is not seen in the cutting (1/4) part taking-out process, it occurs in the cutting (1/4) part re-rolling process. Can be identified.

  In addition, process data in each process is stored in the information storage unit 70, and a reference space is created in the information storage unit 70 using a process data group when a normal product is manufactured as a reference data group. In step S36, a Mahalanobis distance is calculated for the created reference space using process data at the time of product manufacture. A factor that increases the Mahalanobis distance when an abnormality occurs is obtained by factor analysis, and in step S38, the cause process of the abnormality is specified.

  When calculating the Mahalanobis distance of process data, there is often a problem of multicollinearity that occurs because the correlation coefficient between process data increases. To solve this problem, a generalized inverse matrix was applied instead of performing a normal inverse matrix calculation.

<Modification of Embodiment>
In the above description, the metal plate is described as the inspection object, but the present invention is not limited to this as long as it can be acquired as an image. For example, it is good also considering the polarizing film used for a liquid crystal panel as a test subject. The inventors applied the above-described inspection method to a polarizing film inspection process for a liquid crystal panel and a panel inspection process after being attached to a liquid crystal panel.

  In the manufacturing process of a polarizing film, there is a problem of discoloration due to scratches on the film surface, adhesion of dust, and the like. In particular, when the liquid crystal panel is attached to the liquid crystal panel by applying static electricity or the like to give orientation to the liquid crystal, a fine scratch or disorder of alignment occurs due to the contact object for applying static electricity.

  Again, it was confirmed that the Mahalanobis distance calculated for each abnormal mode was different. Inspection that has been conducted for inspection of specific abnormalities that have occurred so far in specific parts by introducing the inspection method described above into the polarizing film inspection process for liquid crystal panels and the panel inspection process after being attached to the liquid crystal panel. The process can detect anomalies with high speed and accuracy even with unknown anomalies.

<Effect of Embodiment>
According to the present embodiment, the abnormal mode of the inspected image can be specified by the Mahalanobis distance calculated from the image feature amount. Further, the increase in the number of images to be inspected, the classification into modes, and the construction of a new reference space can improve the determination accuracy for each mode as the number of images to be inspected increases.

  Further, it is possible to identify an image group that is presumed that the abnormality has occurred due to the same cause from the Mahalanobis distance of the image to be inspected. The abnormality generation process can be specified from the relative relationship between the position parameters of the image groups. In that case, a size change by rolling and cutting may occur between processes. The size of the base material is standardized from the change in the thickness of the base material that changes due to rolling and the change in the width of the base material that changes due to cutting. The process in which the specific abnormality has occurred can be determined from the relative positional relationship for each abnormality mode in the standardized size.

  Further, in the causal process in which the abnormality has occurred, the process data acquired at the normal time is calculated as the reference space data. The Mahalanobis distance at the time of product manufacture with respect to the reference spatial data is monitored, and parameters that have been a factor affecting the change in the Mahalanobis distance when an abnormality occurs in the image inspection are specified by factor analysis.

  From the above implementation results, it is possible to identify the generation process and generation cause for each abnormal mode. Moreover, even if it is difficult to mark products such as glass and film, it is possible to specify the process and cause of abnormality without marking.

  In addition, by characterizing the abnormal mode by using multivariate pattern recognition instead of the number and size of abnormal parts, even when the size changes due to rolling, the classification for each abnormal mode using a CCD image is possible. It becomes possible.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  20 feature quantity extraction unit, 32, 42, 52 abnormal product feature quantity extraction unit, 34, 44, 54 reference space configuration unit, 70 information storage unit, distance calculation unit for each 90 modes, 95 abnormal mode classification unit, 100 image inspection Device, 102 CPU, 104 Keyboard, 106 Mouse, 108 Memory, 110 Display, 111 Position parameter data, 112 Device parameter data, 113 Source data, 115 Memory interface, 116 Communication interface, 118 Image to be inspected and process data, 120 Abnormal Normalization mapping data generation unit, 122 Same abnormality identification unit, 124 Abnormality generation process identification unit, 126 Abnormal cause determination unit, 128 Distance calculation unit when abnormality occurs, 130 Factor analysis unit, D1g to D6g Reference Mahalanobis distance, E1 E4 Region, M1-M6 Abnormal mode.

Claims (6)

An image inspection method for inspecting the inspection object in a production line including a plurality of steps for producing the inspection object,
Based on the image of the inspection object imaged after the completion of each of the plurality of processes, the abnormal part existing in the inspection object after the completion of each process is in any of the plurality of abnormal modes. A first step of determining whether it belongs and generating mapping data including an abnormal mode of an abnormal part and coordinates of the abnormal part in the inspection object;
A second step of normalizing the sizes of the plurality of mapping data so that the coordinates of the same abnormal portion included in the plurality of mapping data coincide with each other based on the dimensional change of the inspection object in the plurality of steps; ,
A third step of grouping the same abnormal parts included in two or more mapping data out of the plurality of standardized mapping data and specifying a process in which the abnormal part has occurred. In advance, images of a plurality of abnormal portions belonging to the plurality of abnormal modes are captured in advance, and a plurality of reference Mahalanobis distances corresponding to the plurality of groups of abnormal portions are calculated from feature portions of the images of the plurality of abnormal portions, respectively. And
In the first step, a first Mahalanobis distance is calculated from a feature portion of the image of the inspection object, the calculated first Mahalanobis distance is compared with the plurality of reference Mahalanobis distances, and based on the comparison result The image inspection method according to claim 1, wherein it is determined which of the plurality of abnormal modes the abnormal part belongs to. In advance, a reference space is created with a process data group when a test object that does not include an abnormal part is manufactured as a reference data group,
Further, a fourth Mahalanobis distance is calculated using process data when an inspection object including an abnormal part is manufactured, and a cause for the occurrence of the abnormal part is specified based on the calculation result and the reference space. The image inspection method according to claim 1, further comprising a step.   The image inspection method according to claim 3, wherein in the fourth step, the second Mahalanobis distance is calculated using a generalized inverse matrix. The inspection object is a film,
The image inspection method according to any one of claims 1 to 4, wherein at least one of the plurality of steps is a rolling step. An image inspection apparatus for inspecting the inspection object in a production line including a plurality of steps for manufacturing the inspection object,
Based on the image of the inspection object imaged after the completion of each of the plurality of processes, the abnormal part existing in the inspection object after the completion of each process is in any of the plurality of abnormal modes. A data generation unit that determines whether it belongs, and generates mapping data including the abnormal mode of the abnormal part and the coordinates of the abnormal part in the inspection object;
Based on the dimensional change of the inspection object in the plurality of steps, a normalization unit that normalizes the size of the plurality of mapping data so that the coordinates of the same abnormal part included in the plurality of mapping data match,
An image inspection apparatus comprising: a specific unit that groups the same abnormal parts included in two or more mapping data of the plurality of standardized mapping data and specifies a process in which the abnormal part has occurred.