KR102597081B1 - Method, apparatus and system for ensemble non-constructing inspection of object based on artificial intelligence - Google Patents

Abstract

Provided herein are artificial intelligence-based object ensemble non-destructive testing methods, devices, and systems. The electronic device that performs ensemble non-destructive testing on the artificial intelligence-based object includes: an input unit that receives image data of the object; an inspection unit that inspects the input image data for defects; and an output unit that outputs a defect inspection result, wherein the inspection unit includes at least one processing unit that implements an ensemble from image data of the object and performs a defect inspection, and outputs a defect inspection result of the at least one processing unit. It includes at least one integration unit that performs integration processing.

Description

Artificial intelligence-based object ensemble non-destructive inspection method, device, and system {METHOD, APPARATUS AND SYSTEM FOR ENSEMBLE NON-CONSTRUCTING INSPECTION OF OBJECT BASED ON ARTIFICIAL INTELLIGENCE}

The present invention relates to non-destructive inspection of an object, and more specifically, to a method, device, and system for inspecting and providing images of the object to determine whether defects are detected through an artificial intelligence-based ensemble non-destructive inspection system.

Product defects can cause deterioration of supply chain services and loss of automation equipment. Therefore, it is very important to properly detect product defects.

In relation to this, non-destructive testing, that is, non-destructive testing that does not destroy the object by utilizing radiation, especially Detected whether I was defective or not. At this time, representative examples of single technologies for detecting defects include pattern recognition using SVM (Support Vector Machines), bump detection through pattern matching, and segmentation-based bump detection. . However, this conventional inspection method has limitations in defect detection performance, so there is a problem in that it cannot detect all defects from the X-ray image, that is, there are defects that are not detected. To solve these problems, a method of applying a single technology step by step was proposed, but there were still problems with limitations in inspection performance based on defect detection.

Republic of Korea Patent Publication No. 10-2249836 (2021.05.03)

The problem to be solved by the present invention is to provide a method of minimizing non-detection of defects from images of objects obtained through radiation irradiation.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

An electronic device that performs ensemble non-destructive testing on an artificial intelligence-based object according to one aspect of the present invention to solve the above-described problem includes: an input unit that receives image data of the object; an inspection unit that inspects the input image data for defects; and an output unit that outputs a defect inspection result, wherein the inspection unit includes at least one processing unit that implements an ensemble from image data of the object and performs a defect inspection, and outputs a defect inspection result of the at least one processing unit. It includes at least one integration unit that performs integrated processing.

An object ensemble non-destructive inspection method performed by an artificial intelligence-based electronic device according to one aspect of the present invention includes receiving image data of the object; preprocessing the received image data of the object in at least one preprocessing module; inspecting defective portions from the preprocessed image data of the object using a defect inspection method predefined in a plurality of inspection modules; Integrating defective inspection result data for the object inspected in each inspection module; and outputting the integrated defect inspection result data.

An artificial intelligence-based object ensemble non-destructive inspection system according to one aspect of the present invention includes an image acquisition device that acquires image data for an object by irradiating radiation; and an electronic device, wherein the electronic device includes: an input unit that receives and inputs image data of an object from the image acquisition device; an inspection unit that inspects the input image data for defects; and an output unit that outputs a defect inspection result, wherein the inspection unit includes at least one processing unit that implements an ensemble from image data of the object and performs a defect inspection, and outputs a defect inspection result of the at least one processing unit. It includes at least one integration unit that performs integration processing.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, the following effects can be achieved.

According to the present invention, it is possible to provide a method of minimizing non-detection of defects from images of objects obtained through radiation irradiation.

Additionally, according to the present invention, the reliability of non-destructive testing can be increased by minimizing non-detection of defects during non-destructive testing of an object.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a block diagram illustrating an artificial intelligence-based object non-destructive inspection system according to an embodiment of the present invention.
Figure 2 is a flowchart illustrating an artificial intelligence-based non-destructive inspection method for an object according to an embodiment of the present invention.
Figure 3 is a diagram illustrating an artificial intelligence-based non-destructive inspection method for an object according to an embodiment of the present invention.
Figure 4 is a block diagram of an electronic device according to an embodiment of the present invention.
5 to 7 are diagrams to explain the ensemble inspection system according to the present invention.
Figure 8 is a block diagram of an electronic device according to another embodiment of the present invention.
Figure 9 is a block diagram of an electronic device according to another embodiment of the present invention.
Figure 10 is a diagram for explaining a configuration method of an inspection module according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Spatially relative terms such as “below”, “beneath”, “lower”, “above”, “upper”, etc. are used as a single term as shown in the drawing. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms that include different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is flipped over, a component described as “below” or “beneath” another component will be placed “above” the other component. You can. Accordingly, the illustrative term “down” may include both downward and upward directions. Components can also be oriented in other directions, so spatially relative terms can be interpreted according to orientation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

In this specification, 'image or image data' refers to still image or video data obtained through a tube or detector using radiation. In one embodiment, the image may be an X-ray image of an object through an X-ray tube or X-ray detector. At this time, the X-ray image is, for example, a 2D (Dimensional) image, a CT (Computed Tomography) image reconstructed from a continuous 2D image aggregation, and reconstructed CT volume data. Can include slice images.

In this specification, 'defect' refers to a part that is not a part that is defined or can be defined as normal for the object during non-destructive testing of the object that is the subject of inspection according to the present invention, and is a defect or It can also be expressed by various names, such as error. Depending on the embodiment, the present invention is not limited to such expressions, and may also include meanings that are the same as or similar to defects in the conventional sense.

1 is a block diagram illustrating an artificial intelligence-based object non-destructive inspection system according to an embodiment of the present invention.

Referring to FIG. 1, a system that performs artificial intelligence-based non-destructive inspection of an object according to an embodiment of the present invention may be configured to include an electronic device 100 and an image acquisition device 150. At this time, the configuration of the electronic device 100 and the image acquisition device 150 shown in FIG. 1 is an example, and is not limited thereto. In relation to performing the operation according to the present invention, one or more components may be used. It may be composed by addition, or vice versa.

The electronic device 100 may be configured to include a memory and a processor. In this case, the memory may correspond to or include the database 120, and the processor may include the control unit 110 and the AI. It may include at least one of the engines 130. At this time, the AI engine 130 includes, but is not limited to, a deep learning network.

The electronic device 100 may be connected to the image acquisition device 150 through a network to receive an X-ray image of an object.

The image acquisition device 150 may be configured to include a detector 160, an X-ray tube 170, and a lighting source (not shown), and the detector 160 may be at least one of a 2D detector and a 3D detector. You can. In the above, the detector 160 and the In addition, the image acquisition device 150 may additionally include a device capable of photographing the motion of a moving object and a CT detector (not shown). The light source includes, but is not limited to, a terahertz transmissive light source.

As a component of the electronic device 100, the control unit 110 can control operations performed in the electronic device 100, and the database 120 includes the image of the object received from the image acquisition device 150, the Data that is received and processed by the electronic device 100, such as a learning dataset for detecting defects in an object image and a learning model generated based on the learning dataset, may be stored.

The control unit 110 has a hardware unit with computing power to perform algorithms and related applications of various machine-learning models, including deep-learning for ensemble. It can be included. For example, the control unit 110 may include at least one of a central processing unit, a microprocessor, and a graphics processor. Additionally, the control unit 110 may further include a memory (not shown) that stores a machine learning model algorithm or application.

For defect inspection, the electronic device 100 can learn image data about an object and input it to the AI engine 130 to obtain (high-definition or improved) object image data. At this time, the image acquired through the AI engine 130 must not only be image data of higher quality or improved overall than the object image data input from the image acquisition device 150 in a general sense, but also be used for non-destructive inspection of the object based on artificial intelligence. It may represent completely or partially improved or new image data from the point of view of defect inspection.

According to one embodiment, the electronic device 100 compares the inspection result of the image data for the object before learning with the inspection result for the image data for the object obtained through the AI engine 130 to model a new learning model. can also be additionally defined. Based on the added learning model, the electronic device 100 determines whether to use the AI engine 130 based on the image of the input object, that is, the object acquired through the AI engine 130 rather than the image data of the input object. It is also possible to decide whether to use image data to inspect for defects and perform actions accordingly.

Therefore, in relation to the present invention, the It can be.

Figure 2 is a flowchart illustrating an artificial intelligence-based non-destructive inspection method for an object according to an embodiment of the present invention. Figure 3 is a diagram illustrating an artificial intelligence-based non-destructive inspection method for an object according to an embodiment of the present invention. Figure 4 is a block diagram of the electronic device 100 according to an embodiment of the present invention.

The operations of FIG. 2 may be performed through the electronic device 100 of FIG. 1 . Here, FIG. 2 is explained with particular reference to the configuration of the electronic device 100 shown in FIG. 4.

Hereinafter, the operations of FIG. 2 will be described with reference to FIGS. 3 and 4.

Referring to FIG. 4 , the electronic device 100 may be configured to include an input unit 210, an ensemble inspection system unit (inspection unit), and an output unit 240. At this time, the ensemble inspection system unit may be configured to include at least one processing part 220 and at least one integrator 230 that implement the ensemble. The ensemble can be created by combining a rule base and deep learning, each of which is a rule base learning model that inputs specific features and detects defects based on the features, and deep learning learned based on image data. Based on the model. Here, the processing unit 220 may be configured to include at least one preprocessing module (image preprocessing module) and a plurality of inspection modules. The integration unit 230 may receive user parameters and refer to them for integration of data processed by the processing unit 220. Various embodiments of the configuration of the ensemble inspection system unit will be described later. In the above, user parameters may represent parameters used in the rule base and deep learning.

In operation 11, the input unit 210 receives image data of the object as shown in (a) of FIG. 3 obtained through radiation irradiation from the image acquisition device 150.

At this time, the image data of the input object shown in (a) of FIG. 3 is raw data received from the image acquisition device 150 or, as described above, is processed through the AI engine 130 as described later. The image data may be at least partially processed to be suitable for processing in operation 12.

In one embodiment, the electronic device 100 may determine in advance whether to immediately perform a preprocessing operation on the received image data of the object, that is, raw data, in operation 12. If there is an error in the raw data, the subsequent operations described later will not only make it difficult to obtain correct results, for example, ground truth results such as (d) in Figure 3, but even if similar results are obtained, Even if you get it, you can't trust it. Therefore, if there is a problem such as an error in the raw data, as described above, either obtain image data in which such problem is resolved through the AI engine 130, etc., or re-request the image data of the object to the image acquisition device 150. It is desirable to obtain new raw data, and the decision can be made based on standards preset in the electronic device 100. Meanwhile, the above-described process can also be performed on re-acquired new raw data.

The ensemble inspection system unit inspects the image data of the input object for the presence of defects through the following operations. At this time, in the performed inspection, N (where N is a positive integer) inspection techniques can be applied, as shown in (b) of FIG. 3. The N may be preset in the electronic device 100 or arbitrarily adjusted. Meanwhile, as shown in (b) of FIG. 3, the portion where the presence of a defective portion is determined within the entire image data of the object by each inspection technology may be the same or different. In other words, referring to (b) of FIG. 3, the part where the presence of a defective part is determined by inspection technology 1 from the image data of the input object and the part where the presence of a defective part is determined by inspection technology N may be different. Maybe, maybe not. However, in this case, the N inspection technologies must be configured to determine whether a defective part exists at least once for all parts to be inspected in the image data of the object, and the result of determining whether a defective part exists by the N inspection technologies is The integration unit 230 performs integrated processing as shown in (c) of FIG. 3. At this time, according to one embodiment, the presence or absence of a defective part may be determined by voting of each inspection module.

At least one preprocessing module in the processing unit 220 preprocesses the received image data of the object in operation 12.

In operation 13, the plurality of inspection modules in the processing unit 220 inspect defective portions from the preprocessed image data of the object using a predefined defect inspection method.

In operation 14, the integrating unit 230 receives defective inspection result data for the object from each inspection module in the processing unit 220 and integrates and processes it.

In operation 15, the output unit 240 outputs the integrated defect inspection result data, for example, as shown in (c) of FIG. 3.

At this time, the output unit 240 may control the input image data of the object shown in (a) of FIG. 3 to be output together with defect detection test result data as shown in (c) of FIG. 3 . In one embodiment, the output unit 240 may be controlled to output only defective portion detection result data determined from image data of the object. At this time, the output-controlled defective part detection result data may be provided in any one or a combination of various formats, such as text, audio, graph, and images. In the above, when the image data of the object is 2D image data and the defective part detection result data is implemented as text, for example, the location of the defective part may be displayed through X-Y coordinates, or only the presence or absence of the defective part may be displayed.

Meanwhile, when providing defective part inspection result data in operation 15, the output unit 240 may provide detailed information about the defective part along with an image as shown in (c) of FIG. 3. For example, the detailed information includes the location of the detected defective part, the indication of the defective part (using color, flickering, etc. to distinguish it from the non-defective part), and the type of the defective part (what type or type of defect it is, etc.) ), the number of defective parts, etc. may be included. This may be determined, for example, by the configuration, performance, etc. of the ensemble inspection system.

The preprocessing and inspection technologies used in the processing of the input data described above may have differences in performance depending on the features and processing algorithms used. In relation to this, as described above, in the present invention, processing data is integrated to prevent undetected errors that may occur due to differences in performance, and the accuracy of defective part detection is dramatically improved, resulting in errors resulting from non-detection of defective parts. Possible problems can be minimized.

Meanwhile, in relation to the present invention, the ensemble inspection system uses artificial intelligence, especially deep learning technology, for the ensemble, where the deep learning technology includes 2D detection technologies such as XavisDet, Faster RCNN, Mask RCNN, YOLO, XavisSeg, Defect detection can be performed using 2D segmentation technology such as Mask RCNN. However, the present invention is not limited to the deep learning technology described above.

Next, with reference to FIGS. 5 to 7, a method of configuring the ensemble inspection system, particularly the processing unit 220, will be described.

5 to 7 are diagrams to explain the ensemble inspection system according to the present invention.

Referring to FIG. 5, the processing unit 220 may be configured to include a plurality of processing modules (process #1 to process #M, where M is a positive integer).

Each processing module may consist of one preprocessing module and one inspection module. At this time, each preprocessing module constituting each processing module may perform different types of preprocessing operations.

Meanwhile, in the above, each inspection module constituting each processing module may or may not adopt a different inspection method depending on or unrelated to the corresponding preprocessing module. For a separate description related to the inspection module, refer to FIG. 10, which will be described later, and herein, only the pre-processing portion within the processing unit 220 is described in FIGS. 5 to 7.

Unlike shown in FIG. 5, referring to FIG. 6, the processing unit 220 includes one pre-processing module and a plurality of inspection modules. That is, image data input from the input unit 210 may pass through the same preprocessing module and be input to each of the plurality of inspection modules at the same time or at different times.

Meanwhile, referring to FIG. 7, the processing unit 220 includes at least two preprocessing modules and a plurality of inspection modules. At this time, one preprocessing module among the at least two preprocessing modules may be matched with at least two or more inspection modules among the plurality of inspection modules, and the same preprocessed image data may be input to each inspection module. That is, from the perspective of the inspection module, some of the plurality of preprocessing modules may be shared.

Meanwhile, although not shown, the processing unit 220 may be configured opposite to that of Figure 6 or Figure 7. For example, unlike the configuration of the processing unit 220 shown in FIG. 6, the processing unit 220 may be implemented with a plurality of preprocessing modules and one inspection module. Additionally, unlike the configuration of the processing unit 220 shown in FIG. 7, a plurality of preprocessing modules in the processing unit 220 may be matched with one inspection module and the inspection module may be partially shared. This configuration may generally be less efficient than Figure 6 or Figure 7, but this may not necessarily be the case depending on the performance and processing capacity of the pre-processing module and/or inspection module.

The configuration of the processing unit 220 of FIGS. 5 to 7 described above may be determined according to image data input to the ensemble inspection system unit through the input unit 210, for example, or may be preset regardless.

Figure 8 is a block diagram of the electronic device 100 according to another embodiment of the present invention. Figure 9 is a block diagram of the electronic device 100 according to another embodiment of the present invention.

In Figure 4, the ensemble inspection system unit can be seen as basically assuming one processing unit and one integration unit.

On the other hand, referring to FIG. 8, the ensemble inspection system unit may be configured to include n processing units 220-1, ..., 220-n, where n is a positive integer and is at least 2 or more) and one integration unit. . Each of the processing units may or may not have the same configuration as at least one of FIGS. 5 to 7.

Referring to FIG. 9, the ensemble inspection system unit includes n processing units 220-1,..., 220-n, m first integration units 230-1,..., 230-m, and at least one second integration unit. It may be configured to include a unit 235. At this time, n is a positive integer and is at least 2 or more as shown in FIG. 8. Meanwhile, m is also a positive integer, which is at least 2 or more, but may or may not be equal to n. In other words, m may be at least 2 or greater or smaller than n. The results integrated and processed in the m first integration units may be reconstructed in the second integration unit 235. According to one embodiment, the second integration unit 235 may not be necessary.

Figure 10 is a diagram for explaining a configuration method of an inspection module according to an embodiment of the present invention.

As described above, with the image data pre-processed in the pre-processing module as input, the inspection module inspects defects in the image data through at least one of three types, as shown in FIG. 10.

In the first type of inspection module 1010, every inspection module is comprised of a separate algorithm. Here, individual algorithms include rule-based inspection technology and deep learning inspection technology.

In the second type of inspection module 1020, all inspection modules are configured with the same algorithm. However, at this time, individual inspection modules may have different user parameters. In one embodiment, at least one inspection module among all inspection modules may operate with different user parameters from other inspection modules. In another embodiment, all inspection modules may operate with different user parameters.

The third type of inspection module 1030 may be a combination type in which some of the inspection modules are composed of individual algorithms and the remaining portions are composed of the same algorithm. At this time, user parameters of the remaining part of the inspection modules, that is, inspection modules with the same algorithm, may be configured with reference to the contents of the user parameters of the second type described above.

The combination of the second type of user parameter and the third type of algorithm described above may be preset in the electronic device 100 or may be flexibly changed depending on image data.

The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. The software module may be RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

Above, embodiments of the present invention have been described with reference to the attached drawings, but those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

100: electronic device
150: image acquisition device
210: input unit
220: processing unit
230: Integration Department
240: output unit

Claims (10)

In an electronic device that performs ensemble non-destructive testing on an artificial intelligence-based object,
an input unit that receives image data of the object;
an inspection unit that inspects the input image data for defects; and
Includes an output unit that outputs defect inspection results,
The inspection unit includes at least one processing unit that performs a defect inspection by implementing an ensemble generated from image data of the object through a combination of a rule base and deep learning, and at least one processor that integrates and processes the defect inspection result of the at least one processing unit. Contains one integrated part,
The at least one processing unit,
a plurality of preprocessing modules that individually preprocess image data of the object;
One preprocessing module that preprocesses image data of the object; and
Among the plurality of preprocessing modules, defects are inspected using a predefined defect inspection method from image data of the preprocessed object in an individually matched preprocessing module, and preprocessed from image data of the object preprocessed in one preprocessing module. A plurality of inspection modules that inspect defective parts using a defined defect inspection method; Includes,
Among the plurality of inspection modules, some inspection modules are composed of individual algorithms, and other inspection modules are composed of the same algorithm,
Provide detailed information about the defect along with the image data,
The detailed information includes the location of the defective part, the display of the defective part using color and flickering, the type of the defective part, and the number of defective parts.
Electronic devices. delete delete delete According to paragraph 1,
Each of the plurality of inspection modules,
Composed of individual algorithms,
The individual algorithm includes rule-based inspection technology and deep learning inspection technology,
Electronic devices. According to paragraph 1,
Each of the plurality of inspection modules,
All are composed of the same algorithm but operate with different user parameters,
Electronic devices. delete delete In an artificial intelligence-based object ensemble non-destructive testing method performed on an electronic device,
Receiving image data of the object;
preprocessing the received image data of the object in at least one preprocessing module;
Implementing an ensemble generated from a combination of a rule base and deep learning from the preprocessed image data of the object using a defect inspection method predefined in a plurality of inspection modules to inspect a defective portion;
Integrating defective inspection result data for the object inspected by the plurality of inspection modules; and
Comprising the step of outputting the integrated defect inspection result data,
The preprocessing module is,
a plurality of preprocessing modules that individually preprocess image data of the object;
One preprocessing module that preprocesses image data of the object; Includes,
The plurality of inspection modules are,
Among the plurality of preprocessing modules, defects are inspected using a predefined defect inspection method from image data of the preprocessed object in an individually matched preprocessing module, and preprocessed from image data of the object preprocessed in one preprocessing module. Defects are inspected using a defined defect inspection method.
Among the plurality of inspection modules, some inspection modules are composed of individual algorithms, and other inspection modules are composed of the same algorithm,
Provide detailed information on the defect part along with the image data,
The detailed information includes the location of the defective part, the display of the defective part using color and flickering, the type of the defective part, and the number of defective parts.
Object ensemble non-destructive testing method. In the artificial intelligence-based object ensemble non-destructive inspection system,
An image acquisition device that acquires image data about an object by irradiating radiation; and
Including an electronic device, wherein the electronic device includes:
an input unit that receives image data of an object from the image acquisition device; an inspection unit that inspects the input image data for defects; and an output unit that outputs a defect inspection result, wherein the inspection unit includes at least one processing unit that performs a defect inspection by implementing an ensemble generated from image data of the object through a combination of a rule base and deep learning, and It includes at least one integration unit that integrates and processes the defect inspection results of one processing unit,
The at least one processing unit,
a plurality of preprocessing modules that individually preprocess image data of the object;
One preprocessing module that preprocesses image data of the object;
Among the plurality of preprocessing modules, defects are inspected using a predefined defect inspection method from image data of the preprocessed object in an individually matched preprocessing module, and preprocessed from image data of the object preprocessed in one preprocessing module. A plurality of inspection modules that inspect defective parts using a defined defect inspection method; Includes,
Among the plurality of inspection modules, some inspection modules are composed of individual algorithms, and other inspection modules are composed of the same algorithm,
Provide detailed information about the defect along with the image data,
The detailed information includes the location of the defective part, the display of the defective part using color and flickering, the type of the defective part, and the number of defective parts.
Object ensemble non-destructive testing system.

Images