[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2020119169A1 - 计算机可读存储介质、输入数据检查方法以及计算装置 - Google Patents

计算机可读存储介质、输入数据检查方法以及计算装置 Download PDF

Info

Publication number
WO2020119169A1
WO2020119169A1 PCT/CN2019/102013 CN2019102013W WO2020119169A1 WO 2020119169 A1 WO2020119169 A1 WO 2020119169A1 CN 2019102013 W CN2019102013 W CN 2019102013W WO 2020119169 A1 WO2020119169 A1 WO 2020119169A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
target
input data
computer
classification model
Prior art date
Application number
PCT/CN2019/102013
Other languages
English (en)
French (fr)
Inventor
宋基永
Original Assignee
数优(苏州)人工智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201811528293.5A external-priority patent/CN111325224A/zh
Application filed by 数优(苏州)人工智能科技有限公司 filed Critical 数优(苏州)人工智能科技有限公司
Publication of WO2020119169A1 publication Critical patent/WO2020119169A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition

Definitions

  • the present disclosure relates to data processing using a computing device, and more particularly, to data processing of artificial intelligence embodied by using a computing device.
  • neural network that models human biological nerve cells.
  • Neural networks use algorithms that mimic the learning capabilities held by humans.
  • the neural network performs the mapping between the input mode and the output mode by learning. Also, based on the learning results, the neural network has the generalization ability to generate relatively correct output for input patterns that are not used for learning.
  • the present disclosure is for providing a method of processing data using a computing device.
  • An embodiment of the present disclosure for solving the problem discloses a computer-readable storage medium that stores a computer program.
  • the work includes: a work for deriving the results of the first check of the input data; The task of classifying the input data by the classification model learned by the learning data with cluster information; and the work of outputting the final inspection result based on at least a part of the classification result of the input data of the classification model.
  • the classification model is learned using a learning data set that contains multiple subsets of learning data. It is possible to classify similar data into the same cluster in the learning data contained in the learning data set, and classify non-similar The data is classified into different clusters to learn the classification model.
  • the work of deriving the first inspection result of the input data may include processing the input data using a pre-learned inspection model containing more than one network function, thereby, the input data Perform anomaly detection or perform anomaly detection on the input data based on the comparison between the input data and the reference data.
  • the classification model may be learned using a learning data set containing a subset of learning data, the learning data set containing learning data labeled with different cluster information.
  • the classification model may be learned using a learning data set that includes a subset of learning data, the learning data set including target data, target similar data, and target non-similar data.
  • the target data and the target similar data may be data marked with first cluster information, and the target non-similar data may be data marked with second cluster information.
  • the target similar data may be data containing types of abnormalities similar to the abnormalities contained in the target data, and the target is not similar
  • the data may be data that does not contain abnormalities.
  • the target similar data may be an image cropped in a manner that includes at least a part of the abnormality included in the target data, and the target is not Similar data can be images that do not contain anomalies.
  • the target data may be an image containing the abnormality at the center of the image.
  • the target non-similar data may be an image in which at least a part of the part other than the abnormality of the target data is repeated.
  • the target data and the target similar data may be classified into the same cluster, and the target non-similar data may be classified into a cluster different from the cluster to which the target data and the target similar data belong. Way to learn the classification model.
  • the target data includes an image related to an abnormal condition that can occur separated from the object of the object
  • the target similar data may include an image having at least a part of the target data
  • the target data and the target similar data may be data containing false positives.
  • outputting the final inspection result based on at least a part of the classification result of the input data of the classification model may include when the classification model classifies the input data as belonging to a specific cluster , Generate the inspection result matching the specific cluster as the second inspection result, and output the final inspection result based on the second inspection result, and, in the case where the classification of the input data fails, convert The output of the first inspection result as a final inspection result.
  • the second inspection result may include information about at least one of whether there is an abnormality, the location of the abnormality, the type of the abnormality, and the type of misdetection.
  • classifying the input data by using the classification model learned by the learning data marked with cluster information may include: using the pre-learned classification model to process the input data, so that all The work of mapping the features of the input data to the solution space of the pre-learned classification model; and based on the position of the input data on the solution space and based on whether the input data belongs to the solution space One of the more than one clusters to classify the work of the input data.
  • the solution space may be composed of more than one dimension, and may contain more than one cluster, and each cluster is based on the characteristics of each target data and the position of the features based on similar data of the target on the solution space. constitute.
  • An embodiment of the present disclosure for solving the problem discloses an input data checking method executed in a computing device including more than one processor.
  • the method includes the steps of deriving the first inspection result of the input data; the step of classifying the input data by using a classification model learned by the learning data marked with cluster information; and the step of classifying the classification model The step of outputting the final inspection result based on at least a part of the classification results of the input data.
  • the computing device includes: more than one processor; and a memory for storing instructions that can be executed in the processor, the processor performs the following tasks: exporting the first inspection result of the input data; by using Classifying the input data by a classification model learned by the learning data marked with cluster information; and outputting a final inspection result based on at least a part of the classification results of the input data of the classification model.
  • the present disclosure may provide a method of processing data using a computing device.
  • FIG. 1 is a block diagram illustrating a computing device that performs an input data checking method according to an embodiment of the present disclosure.
  • FIG. 2a is a diagram illustrating a network function according to an embodiment of the present disclosure.
  • FIG. 2b is a schematic diagram illustrating a convolutional neural network according to an embodiment of the present disclosure.
  • 3a, 3b, 3c, 3d, 3e, and 3f are diagrams showing learning data according to an embodiment of the present disclosure.
  • FIG. 4 is a diagram illustrating a method of training a classification model according to an embodiment of the present disclosure.
  • FIG. 5 is a diagram showing the solution space of the classification model according to an embodiment of the present disclosure.
  • FIG. 6 is a flowchart of a method for checking input data inspection according to an embodiment of the present disclosure.
  • FIG. 7 is a block configuration diagram showing a unit for embodying a method of input data inspection according to an embodiment of the present disclosure.
  • FIG. 8 is a block configuration diagram showing a module for embodying an input data checking method according to an embodiment of the present disclosure.
  • FIG. 9 is a block diagram showing logic for embodying an input data checking method according to an embodiment of the present disclosure.
  • FIG. 10 is a block configuration diagram showing a circuit for embodying an input data checking method according to an embodiment of the present disclosure.
  • FIG. 11 is a brief and general diagram showing an exemplary computing environment that can be embodied by an embodiment of the present disclosure.
  • component refers to computer-related entities, hardware, firmware, software, software, and a combination of hardware or execution of software.
  • the component is a procedure executed on a processor, a processor, an object, an execution thread, a program, and/or a computer, but it is not limited thereto.
  • both the application executed on the computing device and the computing device can be components. More than one component may reside in the processor and/or thread of execution.
  • One component can be logicalized in one computer.
  • a component can be distributed between more than two computers. And, such components can be executed from various computer-readable media having various data structures stored internally.
  • a component is based on a signal with more than one data packet (for example, in a logic system, a decentralized system, through data and/or signals from a component interacting with other components, transmitted through a network such as other systems and the Internet Data), to communicate via local and/or remote processing.
  • a signal with more than one data packet for example, in a logic system, a decentralized system, through data and/or signals from a component interacting with other components, transmitted through a network such as other systems and the Internet Data
  • the terms “include” and/or “include” mean the presence of corresponding features and/or structural elements. However, the terms “include” and/or “include” do not exclude the presence or addition of more than one other feature, structural element, and/or combination. In addition, when there is no special definition or the context does not clearly indicate the singular form, the singular number usually means “one or more” in the scope of the specification and the invention.
  • network functions and artificial neural networks and neural networks can be exchanged with each other.
  • FIG. 1 is a block diagram illustrating a computing device that performs an input data checking method according to an embodiment of the present disclosure.
  • the structure of the computing device 100 shown in FIG. 1 is an illustration briefly shown. In an embodiment of the present disclosure, the computing device 100 may include other structures for performing the embodiments of the present disclosure.
  • the computing device 100 may include a processor 110, a memory 120, a communication module 130, and a camera module 140.
  • the processor 110 may be composed of more than one core, and may include a central processing unit (CPU) of a computing device, a general graphics processing unit (GPGPU, general purpose graphics processing unit), and a tensor processing unit (TPU, tensor processing unit) ) And other data analysis, deep learning processors.
  • the processor 110 reads the computer program stored in the memory 120 to execute the input data checking method according to an embodiment of the present disclosure.
  • the processor 110 may perform calculations for neural network learning.
  • the processor 110 may perform input data processing for deep learning (DN, deep learning), feature extraction from input data, error calculation, and weighting using backpropagation neural network Calculation of neural network learning such as value update.
  • At least one of the CPU, GPGPU, and TPU of the processor 110 may be in the learning of network functions.
  • the CPU and GPGPU work together to learn network functions and use the network functions to classify data.
  • the processors of a plurality of computing devices are used together to learn network functions and classify data using the network functions.
  • the computer program executed by the computing device according to an embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.
  • the computing device 100 utilizes at least one of CPU, GPGPU, and TPU to decentralize and process network functions. Moreover, in an embodiment of the present disclosure, the computing device 100 decentralizes network functions together with other computing devices for processing.
  • the image processed by the network function may be an image stored in the storage medium of the computing device 100, an image captured by the camera module 140 of the computing device 100, and/or a communication module 130, from the image Images transmitted by other computing devices such as databases.
  • the image processed by the network function may be an image stored in a computer-readable storage medium (for example, may include flash memory, etc., but the present disclosure is not limited thereto).
  • the computing device 100 receives an image file stored in a computer-readable storage medium through an input-output interface (not shown).
  • the computing device 100 receives image data and uses pre-learned network functions to check the input data.
  • the input data check of the present disclosure may include anomaly detection for determining whether there is an abnormality in the input data.
  • the input data inspection of the present disclosure may include misdetection judgment related to abnormality detection and abnormality detection.
  • the memory 120 may store a computer program for executing the input data checking method according to an embodiment of the present disclosure.
  • the stored computer program is read and driven by the processor 110.
  • the communication module 130 transmits and receives an abnormality determination method for performing image data according to an embodiment of the present disclosure to other computing devices, servers, and the like.
  • the communication module 130 transmits and receives data for image data and other embodiments of the present disclosure to other computing devices, servers, and the like.
  • the communication module 130 receives learning image data in a learning image database or the like.
  • the communication module 130 may enable communication between a plurality of computing devices, thereby performing learning dispersion of network functions in each of the plurality of computing devices.
  • the communication module 130 enables communication between a plurality of computing devices, so that data classification using network functions can be distributed.
  • the camera module 140 captures an inspection object to generate image data in order to execute the image data abnormality determination method according to an embodiment of the present disclosure.
  • the computing device 100 may include more than one camera module 140.
  • FIG. 2a is a diagram illustrating a network function 200 according to an embodiment of the present disclosure.
  • a neural network may be composed of a collection of interconnected computing units generally referred to as "nodes”. Such “nodes” may also be called “neurons”.
  • the neuron includes at least one or more nodes.
  • the nodes (or neurons) that make up the neural network are connected by more than one "connection unit”.
  • connection unit In the neural network, more than one node connected by the connection unit forms the relationship between the input node and the output node.
  • the concept of input node and output node is relativity.
  • any node that has an output node relationship has a relationship with an input node in the relationship with other nodes, and vice versa.
  • the relationship between the input node and the output node is generated with the connection unit as the center.
  • At one input node more than one output node is connected through the connection unit, and vice versa.
  • the output node determines the value based on the data input to the input node.
  • the nodes connecting the input node and the output node with each other may have a weight.
  • the weighting value can be changed, and the neural network can be changed by the user or algorithm in order to perform the required function. For example, in the case where one output node is connected to more than one input node through each connection unit, the output node has the value input to the input node connected to the output node and the value corresponding to each input node
  • the weighted value set by the connection unit determines the output node value based on the weighted value.
  • a neural network more than one node is connected to each other through more than one connection unit, thereby forming an input node and an output node relationship in the neural network.
  • the characteristics of the neural network are set according to the number of nodes and connection units and the connection relationship between the nodes and connection units, and the weighted value restored to the connection unit. For example, there are the same number of nodes and connection units, and when there are two neural networks with different weighting values between the connection units, the two neural networks are different.
  • the neural network may include more than one node.
  • a part of the nodes constituting the neural network form a layer based on the distance from the initial input node.
  • the set of nodes with distance n may constitute n layers.
  • the distance from the initial input node is determined by the minimum number of connection units that need to be traversed in order to reach the corresponding node from the initial input node.
  • the definition of this layer is used to illustrate that within the neural network, the difference in layers can be defined by a method different from that described above.
  • the layer of a node can be defined by the distance of the final node.
  • the input node is a node in the neural network.
  • the input node In the relationship with other nodes, more than one node that directly inputs data without going through the connection unit.
  • the nodes of other input nodes connected through the connection unit are not blocked.
  • the final output node is one of the nodes in the neural network, and in the relationship with other nodes, there is more than one node that does not have an output node.
  • the hidden node is a node that does not constitute the neural network for the first input node and the last output node.
  • the number of nodes in the input layer is the same as the number of nodes in the output layer.
  • the number of nodes in the input layer is greater than the number of nodes in the output layer, and in the input layer, the number of nodes decreases along hidden nodes.
  • the number of nodes in the input layer is greater than the number of nodes in the output layer. In the input layer, the number of nodes increases along hidden nodes.
  • the neural network of another embodiment of the present disclosure is a neural network of a combined form of neural networks.
  • a deep neural network is a neural network that includes multiple hidden layers in addition to the input layer and the output layer. If the deep neural network is used, the latent structures of the data can be grasped. That is, the underlying structure of photos, texts, videos, voices, and music can be grasped (for example, which objects exist in the photos, what are the contents and feelings of the text, and what are the contents and feelings of the speech, etc.).
  • Deep neural networks may include convolutional neural networks (CNN, convolutional neural networks), recurrent neural networks (RNN, recurrent neural networks), autoencoders (autoencoders), generative adversarial networks (GAN, Generative Adversarial Networks), restricted Boltzmann machine (RBM, restricted boltzmann machine), deep belief network (DBN, deep Belief network), Q network, U network, Siam network, etc.
  • CNN convolutional neural networks
  • RNN recurrent neural networks
  • autoencoders autoencoders
  • GAN Generative Adversarial Networks
  • restricted Boltzmann machine RBM, restricted boltzmann machine
  • DBN deep Belief network
  • Q network U network
  • Siam network etc.
  • the description of the deep neural network is an example, and the present disclosure is not limited thereto.
  • the network function 200 may further include an auto encoder.
  • the automatic encoder may be one type of artificial neural network for outputting output data similar to the input data.
  • the autoencoder may include at least one hidden layer, and a singular hidden layer may be disposed between the input and output layers.
  • the number of nodes in each layer is reduced to the middle layer called the bottleneck layer (coding) in the number of nodes in the input layer, and then in the bottleneck layer, it is expanded symmetrically with the reduction to the output layer (symmetrical to the input layer) .
  • the automatic encoder can perform non-linear dimensionality reduction.
  • the number of input layers and output layers corresponds to the number of sensors remaining after the preprocessor of the input data.
  • the number of nodes of the hidden layer in the encoder decreases in a direction away from the input layer.
  • the number of nodes in the bottleneck layer (the layer with the fewest nodes between the encoder and the decoder) is too small, a sufficient amount of information cannot be transferred, and therefore, it can be maintained above a certain number (for example, half of the input layer) Etc.).
  • Neural networks can be learned in at least one of supervised learning, unsupervised learning and semi-supervised learning.
  • the learning of the neural network is used to minimize the output error.
  • the learning process of the neural network repeatedly input the learning data to the neural network, and calculate the output and target errors of the neural network related to the learning data, along the direction used to reduce the errors, remove the errors of the neural network from the output layer of the neural network Backward propagation in the direction of the input layer to update the weighted value of each node of the neural network.
  • learning data marked with correct answers in each learning data ie, marked learning data
  • unsupervised learning each learning data is not marked with correct answers.
  • the learning data in the case of supervised learning related to data classification is data marked with categories in the learning data.
  • the labeled learning data is input to the neural network, and the output (category) of the neural network and the label of the learning data are compared to calculate errors.
  • the input learning data in the case of unsupervised learning related to data classification is compared with the neural network output, thereby calculating errors. Calculated errors propagate backward in the neural network (that is, in the direction of the output layer to the input layer). With the reverse propagation, the connection weight of each node of each layer of the neural network will be updated. The updated connection weight value of each node may change according to the learning rate.
  • the calculation of the neural network related to the input data and the back propagation of errors can constitute a learning loop (epoch).
  • the learning rate can be applied differently according to the number of repetitions of the learning cycle of the neural network. For example, at the beginning of the learning of the neural network, a high learning rate is applied.
  • the neural network quickly ensures a specified level of performance to improve efficiency, and at the later stage of learning, a low learning rate is used to improve accuracy.
  • the learning data is a partial collection of actual data (ie, data processed by the learned neural network), therefore, there is a learning cycle that reduces errors related to the learning data or adds errors to the actual data .
  • overfitting over-learns, resulting in an increase in errors related to actual data.
  • one type of overfitting is that after seeing a yellow cat, the neural network that learned the cat cannot recognize cats other than the yellow cat. Overfitting increases the error of the machine running the algorithm.
  • a variety of optimization methods can be used. In order to prevent overfitting, increase the learning data, or regularization (regularization), omitting the dropout of a part of the network nodes during the learning process, etc.
  • the convolutional neural network (CNN, convolutional neural network) shown in FIG. 2b is a kind of deep neural network, including a neural network with a convolutional layer.
  • Convolutional neural networks are a type of multilayer receptors designed with minimal preprocess.
  • the convolutional neural network can be composed of one or more convolutional layers and an artificial neural network layer combined with this, and the weighted value and pooling layer are additionally used. Because of this structure, the convolutional neural network can fully use the input data of the two-dimensional structure.
  • Convolutional neural networks are used to identify targets in images.
  • the convolutional neural network can divide the image data into rows and columns with dimensions to be processed.
  • image data encoded in red-green-blue (RGB, red-green-blue)
  • RGB red-green-blue
  • the image data can be divided into three two-dimensional rows and columns (three-dimensional data array).
  • the convolution filter is moved to multiply the convolution filter and the row and column components at various positions of the image to perform the convolution process (input and output of the convolution array).
  • the convolution filter may be in the form of n*n rows and columns, and generally, it may be composed of a filter of a fixed form that is smaller than the number of overall pixels of the image. That is, when an m*m image is input to a convolution layer (for example, a convolution layer with a size of n*n of a convolution filter), the rows and convolutions of n*n pixels including each pixel of the image are represented The filter components are multiplied (ie, the components of the rows and columns are multiplied).
  • the components matching the convolution filter can be extracted from the image.
  • the 3*3 convolution filter used to extract the upper and lower straight line components is composed of [[0, 1, 0], [0, 1, 0], [0, 1, 0]].
  • the convolution filter is applied to the input image, and the upper and lower straight line components matching the convolution filter are extracted from the image and output.
  • the convolution layer may use a convolution filter in each row and column related to each channel (that is, in the case of R, G, and B coating images, R, G, and B colors).
  • the convolutional layer uses a convolution filter on the input image to extract features in the input image that match the convolution filter.
  • the filter value of the convolution filter (that is, the value of each component of the row and column) is updated by inverse propagation during the learning process of the convolutional neural network.
  • the output of the convolutional layer is connected to the sub-sampling layer, thereby simplifying the output of the convolutional layer to reduce memory usage and calculation values. For example, when the output of the convolutional layer is input to the collection layer having the 2*2 maximum collection filter, each pixel of the image outputs the maximum value included in each patch to compress the image for each 2*2.
  • the aggregate output minimum value of the patch, or the average value of the output patch, any aggregation method may be included in the present disclosure.
  • the convolutional neural network may include more than one convolutional layer and sub-sampling layer.
  • the convolutional neural network repeatedly performs a convolution process and a sub-sampling process (for example, the maximum aggregation, etc.) to extract features from the image. Through repeated convolution and sub-sampling processes, the neural network can extract the global features of the image.
  • the output of the convolutional layer or sub-sampling layer can be input to a fully connected layer.
  • a fully connected layer is a layer connected to all neurons of a layer adjacent to all neurons of one layer.
  • a fully connected layer is a structure in a neural network where all nodes of each layer are connected to all nodes of other layers.
  • the neural network may include a deconvolutional neural network (DCNN).
  • the deconvolution neural network performs similar work to calculate the convolutional neural network in the reverse direction, and outputs the features extracted from the convolutional neural network as features related to the original data.
  • the processor 110 may derive the first inspection result of the input data.
  • the first inspection result may include abnormal information related to the input data, and the abnormal information may include arbitrary information related to the abnormality existing in the input data.
  • the first inspection result may include the presence or absence of an abnormality, the location of the abnormality in the input data, and the type of abnormality.
  • the first inspection result may include information about whether the input data is abnormal, which pixel of the input data is abnormal, and what type of abnormality is included in the input data ( For example, in the case of images related to semiconductors, poor soldering, defective wiring, defective components, etc.).
  • the processor 110 may derive the first check result related to whether there is an abnormality in the input data.
  • the processor 110 processes the input data using a pre-learned inspection model containing more than one network function, thereby performing an abnormality inspection related to the input data to derive the first inspection result.
  • the inspection model is a semi-supervised learning model that judges whether there is an unlearned new pattern (ie, abnormality) in the input data by learning only from normal data, thereby determining whether there is an abnormality in the input data.
  • the inspection model is learned from the marked normal data and abnormal data, so that it is possible to determine whether the input data contains abnormal supervised learning models. Checking the model may include any machine learning model that determines whether there is an abnormality in the input data.
  • the processor 110 performs abnormality detection related to the input data based on the comparison of the input data and the reference data to derive the first inspection result.
  • the processor 110 determines whether there is an abnormality in the input data based on the comparison of the reference data containing the abnormality or the reference data not containing the abnormality and the input data.
  • the processor 110 is based on an arbitrary machine vision (machine vision) solution to judge the abnormality in the input data.
  • the processor 110 classifies the input data using the classification model learned using the learning data marked with cluster data.
  • the classification model uses learning data sets that contain multiple subsets of learning data to learn.
  • the learning data contained in the learning data set classifies similar data into the same cluster and classifies dissimilar data into different clusters.
  • the learning data subset may contain multiple learning data.
  • the learning data subset may contain multiple learning data, and each learning data is marked with cluster information.
  • the learning data is a subset of data used in one iteration in the learning of the network function.
  • the learning data set is the entire data used for looping in the learning of the network function.
  • the learning data subset may contain dissimilar learning data, and the dissimilar learning data contained in the learning data subset may be classified into different clusters through a classification model.
  • the learning data subset includes a plurality of learning data, each of the learning data is labeled with cluster information, and the learning data subset may further include learning data labeled with different cluster information.
  • the learning data subset may include target data, target similar data, and target non-similar data, and may be labeled with cluster information such as target data and target similar data, and target non-similar data may be marked with different data from target data and target similar data.
  • Cluster information The target data and target similar data contained in the learning data subset may be marked with first cluster information, and the target non-similar data may be marked with second cluster information.
  • the target similar data is data containing abnormalities
  • the target non-similar data is data containing no abnormalities.
  • the target non-similar data is data in which at least part of the part other than the abnormality is repeated in the target data.
  • the target data is an image
  • the target non-similar data is an image of at least a part of the part other than the abnormal part in the image containing the abnormality.
  • the target similar data is data including an abnormality similar to the abnormality included in the target data.
  • the target similar data is cropped in such a manner as to include at least a part of the abnormality included in the target data.
  • the abnormality of the similar type included in the target data may include the abnormality at the same position on the object of the inspection object of the abnormality contained in the target data, and the same type of defect as the object of the inspection object of the abnormality included in the target data (For example, when the object to be inspected is a circuit board and the abnormality included in the target data is poor soldering, the abnormality included in the target similar data is also poor soldering).
  • the target similar data may include at least a part of the image of the target data, and the pixel part corresponding to the abnormality in the image of the target data is the data that must be included.
  • Target non-similar data are images that do not contain abnormalities.
  • the target data is an image that contains abnormality in the center of the image.
  • the target non-similar data is an image in which at least a part of the part other than the abnormality of the target data is repeated.
  • the abnormal part of the image containing the abnormality and the image not containing the abnormality are different, and other parts may be similar, so that in the case of classification by the usual method, it can be classified as a similar image.
  • the abnormal part is 5*5 pixels, and the abnormal part is usually classified at 0.25% of the whole, 99.75% of the two images are similar, Therefore, it is classified as similar by the usual classification method.
  • target data, target similar data, and target dissimilar data respectively labeled with cluster information according to an embodiment of the present disclosure to learn a classification model even if a part of the image is different, it is classified as Non-similar data.
  • the processor 110 classifies the target data and target similar data into the same cluster, and classifies the target non-similar data into clusters that are different from the cluster to which the target data and target similar data belong.
  • the abnormalities that the target data can include, in the case of the target data bit image, at least one of the abnormalities related to the inspection target object included in the image and the abnormalities separated from the inspection target object.
  • the abnormalities that the target data may include may be abnormalities that can occur in semiconductor products (eg, defects, etc.) and abnormalities that are not related to the object of the inspection object (eg, images Acquire at least one of the problem of the device, the lens foreign object, etc.). That is, from the viewpoint of abnormality detection, the target data and the target similar data may include false positives. More specifically, when there is no abnormality in the inspection object and there is a situation that can occur that is separate from the inspection object, the target data and the target similar data may contain such a false alarm. Therefore, the classification model classifies abnormal types and false positives into clusters.
  • the result of the first inspection may contain a false positive.
  • the input data is classified for the second time using the classification model, and the processor 110 uses the classification model to classify the result of the input data.
  • the second time In the classification it can be judged whether it is an abnormality related to the actual inspection object or an abnormality not related to the inspection object (ie, a false alarm).
  • FIGS. 3a to 3 e are diagrams showing learning data according to an embodiment of the present disclosure.
  • the images shown in FIGS. 3 a to 3 e are only examples of learning data, and the present disclosure may include arbitrary learning data and arbitrary images.
  • FIG. 3a is a diagram showing a target base image 300 that can be used as learning data.
  • the target base image 300 itself can be used as the target image for learning the classification model, and the target base image 300 is cropped for the target image 310.
  • the target base image 300 may include a defect 301 abnormality of the inspection object.
  • the plurality of target similar images 311, 313 may be images cropped in such a manner as to contain the defective 301 portion of the target image 310.
  • the plurality of crop lines 310, 311, and 313 in FIG. 3a are crops of the target image 310 and crops 311 and 313 of the target similar image.
  • FIG. 3b shows a target image 310 (target image) and target similar images 311, 313 (target similar image) cropped by the cropping line of FIG. 3a.
  • the target image 310 and the target similar images 311, 313 both contain the abnormality 301, and may include other parts of the inspection object.
  • FIG. 3c is an illustration showing the target non-similar base image 320.
  • the target non-similar base image 320 shown in FIG. 3c itself can be used as a target dissimilar image, a part of which is cropped and used as a target non-similar image.
  • the target non-similar image 320 may have resolutions such as the target image 310 and the target similar image 311. The resolutions of the target images, target similar images, and target non-similar images are also examples, and the resolutions of these may be the same or different.
  • the target non-similar image 320 may be an image that does not contain abnormalities.
  • the target non-similar image 320 is an image that does not contain abnormality in a portion 321 corresponding to the target image.
  • FIG. 3d is an illustration showing a subset of learning data used to learn the classification model of an embodiment of the present disclosure.
  • the learning data subset may include a target image 310 with an anomaly 301, a target similar image 311, and a target non-similar image 320 without an anomaly 321.
  • the target image 310 and the target similar image 311 are marked with the same cluster information in a manner of being classified into the same cluster.
  • the target non-similar image 320 is marked with cluster information that is different from the cluster information marked on the target image 310 and the target similar image 311.
  • the target image 310 and the target similar image 311 may be marked with first cluster information, and the target non-similar image 320 may be labeled with second cluster information.
  • the classification model may classify the images with abnormalities and the images without abnormalities into different clusters.
  • the abnormal part is 5*5 pixels, and the abnormal part is usually classified at 0.25% of the whole, 99.75% of the two images are similar, Therefore, it is classified as similar by the usual classification method.
  • target data, target similar data, and target dissimilar data respectively labeled with cluster information according to an embodiment of the present disclosure to learn a classification model even if a part of the image is different, it is classified as Non-similar data.
  • FIG. 3e is an illustrative diagram showing other learning data subsets for learning the classification model of an embodiment of the present disclosure.
  • the target image 330 and the target similar image 333 of FIG. 3e may be images containing false positives. More specifically, the abnormality 331 included in the target image 330 and the target similar image 333 is separated from the inspection target object and an abnormality condition (for example, a lens foreign object, etc.) can occur.
  • the target image 330 may be an image related to a normal product or an abnormal image that occurs according to an abnormal condition separated from the normal product. According to the conventional abnormality inspection method, when an abnormal image containing such a false alarm is included, it is determined to be abnormal, and an erroneous result is output.
  • the classification model is used to generate clusters related to false alarms using images containing false alarms, whereby it is possible to detect whether the inspection object included in the input image to be inspected actually contains abnormalities
  • the object to be inspected does not contain anomalies, but for other reasons, the input data is judged to be anomaly detection.
  • the target non-similar image 340 is an image that does not contain abnormalities.
  • the classification model may classify images with false alarms and images without false alarms into different clusters.
  • FIG. 3f is an illustrative diagram showing another learning data subset for learning the classification model of an embodiment of the present disclosure.
  • the target image 330 and the target similar image 333 of FIG. 3f may be images containing false positives. More specifically, the abnormality 331 included in the target image 330 and the target similar image 333 is an abnormal condition (for example, a lens foreign object, etc.) that can occur when separated from the inspection target object.
  • the target image 330 may be an image related to a normal product or an abnormal image that occurs according to an abnormal condition separated from the normal product. According to the conventional abnormality inspection method, when an abnormal image containing such a false alarm is included, it is determined to be abnormal, and an erroneous result is output.
  • the classification model is used to generate clusters related to false alarms using images containing false alarms, whereby it is possible to detect whether the inspection object included in the input image to be inspected actually contains abnormalities
  • the object to be inspected does not contain anomalies, but for other reasons, the input data is judged to be anomaly detection.
  • the target non-similar image 310 is an image that includes an abnormality 301 (that is, an abnormality related to the inspection target object), and does not include a false alarm.
  • the classification model can classify images with false alarms and images with abnormalities in the actual inspection object (ie, images classified as abnormal in the abnormality inspection ) Is classified into different clusters.
  • the target image 330 including a false alarm is added according to the type of false alarm.
  • the new misdetection is used as a target image including a false alarm to additionally learn the classification model.
  • the classification model of the present disclosure can cluster well-known anomaly types and well-known misdetection types, and can determine which type the input data belongs to.
  • FIG. 4 is a diagram illustrating a method of training a classification model according to an embodiment of the present disclosure.
  • the classification model of the present disclosure clusters similar data on the solution space 400. More specifically, the classification model includes target data 401 and target similar data 402 in one cluster 410, and target non-similar data 403 is included in a different cluster from target data 401 and target similar data 402.
  • each cluster has a prescribed distance margin 420.
  • the classification model receives a subset of learning data containing target data 401, target similar data 402, and target non-similar data 403 to match each data with the solution space, and updates the included classification in a clustered manner based on the cluster information marked on the solution space
  • the weighted value of more than one network function of the model increases the distance in the solution space between the target data 401 and the target similar data 402 and the target non-similar data 403 in such a manner that the distance in the solution space of the target data 401 and the target similar data 402 becomes shorter.
  • the classification model uses a triplet-based cost function to learn.
  • the difference between the second distance between the input data is at least the distance margin 420, and the method of learning the classification model includes reducing the first distance to the distance margin Steps below the scale.
  • the distance margin 420 is always a positive number.
  • the weighted value of more than one network function included in the classification model may be updated, and the weighted value update is performed every iteration or one epoch.
  • the classification model is learned by a model based on magnet loss considering not only cluster classification of dissimilar data, but also the semantic relationship between data in a cluster or other clusters.
  • the initial distance of the center point difference of each cluster is executed during the learning process.
  • the position on the solution space of each data is adjusted based on the similarity to the cluster to which each data belongs and the data within and outside the cluster.
  • FIG. 5 is a diagram showing the solution space of the classification model according to an embodiment of the present disclosure.
  • FIG. 5 is a diagram showing the solution space 400 of the classification model.
  • the solution space 400 shown in FIG. 5 is merely an example, and the classification model may include any number of clusters and any number of data of each cluster.
  • the data 431, 433, 441, and 443 in the cluster shown in FIG. 5 are similar data.
  • the solution space is composed of a space of more than one dimension, and includes more than one cluster, and each cluster is constructed based on the features based on the respective target data and the positions on the solution space based on the features based on the target-like data.
  • the first cluster 430 and the second cluster 440 may be clusters related to dissimilar data.
  • the third cluster 450 may be a cluster related to data that is not similar to the first and second clusters.
  • the distance 445, 435 between clusters is a scale that shows the difference in data belonging to each cluster.
  • the twelfth distance 445 between the first cluster 430 and the second cluster 440 is a measure of the difference between the data belonging to the first cluster 430 and the data belonging to the second cluster 440.
  • the thirteenth distance 435 between the first cluster 430 and the second cluster 440 is a measure of the difference between the data belonging to the first cluster 430 and the data belonging to the third cluster 450.
  • the data belonging to the first cluster 430 is more dissimilar to the data belonging to the second cluster than the data belonging to the third cluster 450. That is, when the distance between clusters is long, the data belonging to each cluster is more dissimilar, and when the distance between clusters is short, the data belonging to each cluster is more similar.
  • the distances 435, 445 between several districts are larger than the radius of the cluster by a predetermined ratio.
  • the processor 110 uses the classification model to calculate the input data, thereby classifying the input data based on the position where the feature of the input data matches the solution space of the classification model.
  • the processor 110 processes the input data using the pre-learned classification model, thereby matching the characteristics of the input data with the solution space of the pre-learned classification model.
  • the processor 110 classifies the input data based on the position on the solution space of the input data, based on which of the one or more clusters on the solution space the input data belongs to.
  • the processor 110 determines the final inspection result based on at least a part of the classification result related to the input data of the classification model.
  • the processor 110 In the case where the split model classifies the input data as belonging to a specific cluster, the processor 110 generates the inspection result matching the specific cluster as the second inspection result, and determines the final inspection result with the second inspection result. In addition, when the classification of the classification model and the input data fails, the processor 110 determines the first inspection result as the final inspection result, where the second inspection result may include the presence or absence of the abnormality, the location of the abnormality, and the abnormality Information about at least one of the type, misdetection, and misdetection type.
  • the pre-learned classification model retains clusters in the solution space according to the type of anomaly. Therefore, the processor 110 uses the classification model to generate the second inspection result related to the input data.
  • the processor 110 determines whether there is an abnormality in the input data based on which cluster on the solution space of the classification model the feature of the input data belongs to, and if there is an abnormality, it can be determined whether the abnormality is an abnormality on the inspection object Or an abnormality that has nothing to do with the object of inspection (ie, false detection).
  • the processor 110 may generate The data exists as a second inspection result that is an abnormality of substrate soldering failure, and the final inspection result is determined based on the first inspection result and the second inspection result.
  • the processor 110 may generate a second corresponding to the input data that is a false detection and there is no abnormal normal data
  • the results of the second inspection are based on the results of the first inspection and the second inspection to determine the final inspection results.
  • the processor 110 determines that the substrate welding defects are related to the input data The final result of anomaly detection.
  • the processor 110 may inform the operator of the content.
  • the processor 110 marks the corresponding input data in order to inform the operator of the difference between the first inspection result and the second inspection result.
  • the processor 110 ensembles the first inspection result and the second inspection result to generate a final inspection result.
  • the ensemble method may use a voting method, a method based on the weighted value of each neural network, and may include any other ensemble method.
  • the processor 110 determines that the classification using the input data classification model failed, and generates the first inspection result as the final inspection result. For example, in the case where the input data includes a new abnormality related to the inspection target object with a defective circuit wire or an abnormality related to the inspection target object with a broken lens, the processor 110 (that is, the learned classification model has an unlearned new pattern, etc. ), the input data cannot be classified using the classification model, therefore, the final inspection result related to the corresponding input data is generated based on the first inspection result. In this case, the processor 110 may inform the operator of the content. In order to inform the operator of the content, the processor 110 needs to mark the corresponding input data.
  • the processor 110 may generate the corresponding input data as new learning data.
  • the processor 110 uses the corresponding input data as the target image, and can generate the corresponding The other part of the anomaly of the input data is used as a new learning data subset with similar data as the target.
  • the processor 110 uses the newly generated subset of the learning data to generate clusters on the solution space of the classification model related to the new abnormal pattern and adds the learning classification model.
  • the abnormality of the input data is determined by the first and second inspections, and the input data includes abnormalities related to abnormal conditions that occur through factors other than the object of the inspection object (ie, false positives) ), the inspection results can be derived, not for false detections.
  • the classification model is learned again by using it, whereby the predetermined inspection performance corresponding to the newly generated misdetection and abnormality can be maintained.
  • the first inspection and the second inspection are separated, so that adding a new type of abnormality or a new type of misdetection to the abnormality detection solution does not affect the classification performance related to the previous data. That is, in the conventional solution, in the case of repeated false detections, the solution itself is modified to solve the problem of corresponding false detections, and thus other erroneous operations may occur. In the abnormality detection of an embodiment of the present disclosure, it may be broken Open the possibility of additional error work.
  • FIG. 6 is a flowchart of a method for checking input data inspection according to an embodiment of the present disclosure.
  • the computing device 100 may derive the first inspection result of the input data (step 610).
  • the input data may include image data
  • the computing device 100 may determine the first inspection result related to the captured image data or the image data transmitted from an external device.
  • the result of the first inspection may contain abnormal information related to the input data.
  • the computing device 100 classifies the input data by using a classification model learned using the learning data labeled with cluster information (step S630).
  • the classification model uses a learning data set that contains multiple subsets of learning data to learn. In the learning data contained in the learning data set, similar data is classified into the same cluster, and non-similar data is classified into other clusters.
  • the computing device 100 outputs a final inspection result based on at least a part of the classification result of the input data of the classification model (step 650).
  • the classification model classifies the input data as belonging to a specific cluster
  • the computing device 100 generates the inspection result matching the specific cluster as the second inspection result, and determines the final result based on at least a part of the second inspection result test result.
  • the computing device 100 determines the first inspection result as the final inspection result.
  • FIG. 7 is a block configuration diagram showing a unit for embodying a method of input data inspection according to an embodiment of the present disclosure.
  • a method for checking input data may be embodied by the following unit.
  • a method for inspecting input data may be embodied by the following units: a unit 710 that derives the first inspection result of the input data; a classification model learned by using learning data marked with cluster information A unit 730 to classify the input data; and a unit 750 to output a final inspection result based on at least a part of the classification result of the input data of the classification model.
  • the unit 710 for deriving the first inspection result of the input data may include using a pre-learned inspection model containing more than one network function to The input data is processed to thereby perform abnormality detection on the input data, or a unit that performs abnormality check related to the input data based on the comparison of the input data and reference data.
  • a unit 750 for outputting the final inspection result based on at least a part of the classification result of the input data of the classification model may be included in the When the classification model classifies the input data as belonging to a specific cluster, the inspection result matching the specific cluster is generated as the second inspection result, and the final inspection result is output based on the second inspection result, and, When the classification of the input data fails, the first inspection result is output as a unit of the final inspection result.
  • the unit 730 for classifying the input data by using a classification model learned using the learning data marked with cluster information may include: The learned classification model processes the input data, thereby mapping the features of the input data to units of the solution space of the pre-learned classification model; and using the input data on the solution space Based on the location, the units of the input data are classified based on whether the input data belongs to one of more than one cluster on the solution space.
  • FIG. 8 is a block configuration diagram showing a module for embodying an input data checking method according to an embodiment of the present disclosure.
  • a method for checking input data may be embodied by the following modules.
  • a method for inspecting input data may be embodied by the following modules: a module 810 for deriving the first inspection result of the input data; a classification model learned by using learning data marked with cluster information A module 830 to classify the input data; and a module 850 to output a final inspection result based on at least a part of the classification result of the input data of the classification model.
  • FIG. 9 is a block diagram showing logic for embodying an input data checking method according to an embodiment of the present disclosure.
  • a method for checking input data may be embodied by the following logic.
  • the method for checking the input data may be embodied by the following logic: Logic 910 that derives the first check result of the input data; by using the classification model learned by the learning data marked with cluster information Logic 930 to classify the input data; and logic 950 to output a final inspection result based on at least a portion of the classification result of the input data of the classification model.
  • FIG. 10 is a block configuration diagram showing a circuit for embodying an input data checking method according to an embodiment of the present disclosure.
  • a method for checking input data may be embodied by the following unit.
  • the method for checking the input data may be embodied by the following circuit: a circuit 1010 that derives the first check result of the input data; a classification model learned by using learning data marked with cluster information A circuit 1030 to classify the input data; and a circuit 1050 to output a final inspection result based on at least a part of the classification result of the input data of the classification model.
  • FIG. 11 is a brief and general diagram showing an exemplary computing environment that can be embodied by an embodiment of the present disclosure.
  • program modules include routines, programs, components, data structures, etc. that perform specific tasks or embody specific abstract data types.
  • the method of the present disclosure is known not only in single-processor or multi-processor computer systems, minicomputers, and mainframe computers, but also in personal computers, handheld computing devices, and microprocessor-based Implement other computer systems including devices such as devices, programmable home appliances, and others (these are connected to more than one related device to work).
  • the embodiments described in the present disclosure may be implemented in a decentralized computing environment executed by a remote processing device connected through a communication network through any task.
  • program modules can be located in both local and remote storage devices.
  • Computers generally include a variety of computer-readable media. Through the computer, accessible media can become computer-readable media, such computer-readable media include volatile and non-volatile media, transitory (transitory) and non-transitory (non-transitory) media, mobile And non-removable media.
  • the computer-readable media may include computer-readable storage media and computer-readable transmission media.
  • Computer-readable storage media includes volatile and non-volatile media media, transitory and non-transitory media embodied by any method or technology that stores information such as computer-readable instructions, data structures, program modules, or other data, Removable and non-removable media.
  • Computer storage media may include RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital video disk (DVD, digital) or other optical disk storage devices, magnetic tape, magnetic tape, magnetic disk storage devices or other magnetic storage devices or Any other medium that accesses and stores required information through a computer, but is not limited to this.
  • Computer-readable transmission media generally include a medium that embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism and transmits all information. Modulated data signal terminology sets or changes more than one signal in the characteristics of the signal in such a manner as to encode information within the signal.
  • computer-readable transmission media include wired media such as a wired network or direct-wired connection, audio, RF, infrared, and other wireless media such as other wireless media. Any combination of such media is also included within the scope of computer-readable transmission media.
  • An exemplary environment 1100 embodying multiple aspects of the present disclosure including a computer 1102 is presented, which includes a processing device 1104, a system memory 1106, and a system bus 1108.
  • the system bus 1108 connects system components including the system memory 1106 (not limited thereto) to the processing device 1104.
  • the processing device 1104 may be any processor among various commonly used processors. Dual-processor and other multi-processor architectures can also be utilized as the processing device 1104.
  • the system bus 1108 may be one of several types of bus structures that are connected to each other using an arbitrary local bus in a memory bus, a peripheral device bus, and various common bus architectures.
  • the system memory 1106 includes a read-only memory 2110 and a random access memory 2112.
  • the basic input/output system (BIOS) is stored in non-volatile memory 2110 such as RROM, EPROM, EEPROM, etc., and includes a basic history of transferring information between structural elements in the computer 1102 when the input/output system is started.
  • the random access memory 2112 may include a high-speed random access memory such as a static random access memory for caching data.
  • the computer 1102 includes an internal hard disk drive 2114 (HDD) (for example, EIDE, SATA, and the internal hard disk drive 2114 is also used as an external type in a suitable chassis (not shown)), and a magnetic floppy disk drive 2116 (FDD) (for example, Read from the removable floppy disk 2118 or use it to record thereon) and the optical disk drive 1120 (for example, read the CD-RON disk 1122 or read from other high-capacity optical media such as DVD or use it to record thereon).
  • the hard disk drive 2114, the magnetic disk drive 2116, and the optical disk drive 1120 may be connected to the system bus 1108 through a hard disk drive interface 1124, a magnetic disk drive interface 1126, and an optical disk drive interface 1128, respectively.
  • the interface 1124 for external drive embodiment includes at least one or two of Universal Serial Bus (USB, Universal Serial Bus) and IEEE 1394 interface technologies.
  • drives and related computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and others.
  • the drive and the medium store arbitrary data in an appropriate digital form.
  • the description of the computer-readable medium refers to a hard disk drive, a removable magnetic disk, and a portable optical medium such as a CD or DVD, as long as it is a person of ordinary skill in the technical field to which this disclosure belongs, through a zip drive, a magnetic tape,
  • Other types of media read by computers, such as flash memory cards, cassette tapes, and others, are also used in the exemplary operating environment, and any such media may contain computer-executable instructions for performing the present disclosure.
  • Multiple program modules including the operating system 2130, one or more application programs 2132, other program modules 2134, and program data 2136 may be stored in the drive and the random access memory 2112. All or part of the operating system, applications, modules, and/or data is also cached in the random access memory 2112.
  • the present disclosure may be embodied as an operating system or a combination of operating systems available in various commercial aspects.
  • the user inputs commands and information to the computer 1102 through more than one wired or wireless input device, such as a pointing device such as a keyboard 2138 and a mouse 1140.
  • a pointing device such as a keyboard 2138 and a mouse 1140.
  • Other input devices are microphone, IR remote control, joystick, gamepad, stylus, touch screen, etc.
  • These and other input devices are connected to the processing device 1104 through an input device interface 1142 connected to the system bus 1108, and connected through other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc. .
  • the display 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146 or the like. Attached to the display 1144, the computer usually includes speakers, printers, other peripheral output devices (not shown).
  • the computer 1102 operates in a networked environment using the logical connection of more than one remote computer, such as a remote computer 1148 via wired and/or wireless communication.
  • the remote computer 1148 may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network nodes.
  • the computer 1102 includes many of the described structural elements. For simplicity, only the storage device 1150 is shown.
  • the illustrated logical connection includes a short-range communication network 1152 (LAN) and/or a larger network, for example, a wired or wireless connection of a long-range communication network 1154 (WAN).
  • LAN short-range communication network
  • WAN long-range communication network
  • Such short-distance communication and long-distance communication network environments are generally used in offices and companies to make enterprise-wide computer networks such as intranets simple, which are connected to computer networks all over the world, for example, networks.
  • the computer 1102 When used in a short-range communication network environment, the computer 1102 is connected to the local network 1152 through a wired and/or wireless communication network interface or adapter 1156.
  • the adapter 1156 can be wired or wireless for short-range communication 1152.
  • the short-range communication 1152 includes a wireless access point provided for communication with the wireless adapter 1156.
  • the computer 1102 may include a modem 1158, connected to a communication computing device on the long-distance communication 1154, or set other units of communication through the long-distance communication 1154 through a network or the like.
  • the modem 1158 which may be an internal or external type and a wired or wireless device, is connected to the system bus 1108 through a serial interface 1142.
  • the described program module or a part thereof is stored in the remote memory/storage device 1150.
  • the illustrated network connection is an exemplary embodiment, and other units for setting a communication connection between computers may also be used.
  • the computer 1102 performs any wireless device or individual configured and operated by wireless communication, for example, a printer, a scanner, a desktop computer and/or a portable computer, a personal data assistant (PDA, portable data), a communication satellite, and a wireless detection mark Related to any device or place and telephone communication work.
  • PDA personal data assistant
  • Wireless fidelity (Wi-Fi, Wireless, Fidelity) can be connected through the network even if it is not wired.
  • Wireless fidelity is such a device, for example, the wireless technology of a telephone in which a computer can send and receive data anywhere in the conversation circle of a base station, that is, a computer.
  • the wireless fidelity network is reliable and uses IEEE 802.11 (a, b, g, other) wireless technologies to provide high-speed wireless connections.
  • IEEE 802.11 a, b, g, other
  • wireless fidelity is used in order to connect the computer to the network and the wired network (using IEEE 802.3 or Ethernet).
  • the wireless fidelity network operates in the unlicensed 2.4 and 5 GHz radio frequency bands, for example, at a data rate of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include dual bands.
  • the various embodiments disclosed herein are embodied by methods, devices, or articles using standard transformation and/or functional technologies.
  • article of manufacture includes a computer program, carrier, or media accessed from any computer-readable device.
  • computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory devices (eg, EEPROMs, cards, strips, key drives, etc.), But it is not limited to this.
  • the various storage media disclosed herein include more than one device and/or other machine-readable media for storing information.
  • the specific order or hierarchical structure of the steps in the disclosed process is an example of illustrative proximity. Based on the limited order of design, the specific order or hierarchical structure of the steps of processes within the scope of the present disclosure may be rearranged again.
  • the additional method invention claims that the elements of the multiple steps are provided in the order of the samples, and is not limited to the specific order or hierarchical structure disclosed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)

Abstract

根据本公开的一实施例，本公开揭示了计算机可读存储介质，存储有计算机程序。所述计算机程序在一个以上的处理器中执行的情况下，执行用于对输入数据进行检查的以下的工作，所述工作包括：导出对输入数据的第一次检查结果的工作；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的工作；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的工作。

Description

计算机可读存储介质、输入数据检查方法以及计算装置 技术领域
本公开涉及利用计算装置的数据处理，更具体地，涉及通过利用计算装置来体现的人工智能的数据处理。
背景技术
为了解决复杂或未知的问题，进行了将人类所知的识别方法适用于设备的研究。作为这种研究中的一个，具有将人类的生物学神经细胞建模的神经网络(neural network)。神经网络利用模仿人类所持有的学习能力的算法。神经网络通过学习执行输入模式和输出模式之间的思想(mapping)。并且，神经网络以学习的结果为基础，具有对不用于学习的输入模式，可生成比较正确输出的一般化能力。
发明内容
本公开用于提供利用计算装置来处理数据的方法。
用于解决所述问题的本公开一实施例公开了计算机可读存储介质，存储有计算机程序。所述计算机程序在一个以上的处理器中执行的情况下，执行用于对输入数据进行检查的以下的工作，所述工作包括：导出对输入数据的第一次检查结果的工作；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的工作；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的工作。
代替性地，所述分类模型利用包含多个学习数据子集的学习数据集来学习，能够以在包含在所述学习数据集中的学习数据中将类似的数据分类为相同集群，且将非类似的数据分类为不同集群的方式学习所述分类模型。
代替性地，导出对所述输入数据的第一次检查结果的工作可包括利用包含一个以上的网络函数的预先学习的检查模型来对所述输入数据进行处理，由此，对所述输入数据执行异常检测或者以所述输入数据与参考数据的比较为基础来对所述输入数据执行异常检测的工作。
代替性地，所述分类模型可利用包含学习数据子集的学习数据集来学习，所述学习数据集包含标记有不同集群信息的学习数据。
代替性地，所述分类模型可利用包含学习数据子集的学习数据集来学习，所述学习数据集包含目标数据、目标类似数据及目标非类似数据。
代替性地，所述目标数据和所述目标类似数据可以为标记有第一集群信息的数据，而且，目标非类似数据可以为标记有第二集群信息的数据。
代替性地，在所述目标数据为包含异常的数据的情况下，所述目标类似数据可以为包含与所述目标数据所包含的异常类似的类型的异常的数据，而且，所述目标非类似数据可以为不包含异常的数据。
代替性地，在所述目标数据为包含异常的图像的情况下，所述目标类似数据可以为以包含所述目标数据所包含的异常的至少一部分的方式裁剪的图像，而且，所述目标非类似数据可以为不包含异常的图像。
代替性地，所述目标数据可以为在图像的中心部包含所述异常的图像。
代替性地，所述目标非类似数据可以为除所述目标数据的异常之外的部分的至少一部分重复的图像。
代替性地，能够以将所述目标数据和所述目标类似数据分类为相同集群，且将所述目标非类似数据分类为与所述目标数据及所述目标类似数据所属的集群不相同的集群的方式学习所述分类模型。
代替性地，所述目标数据包含与可与对象客体分离来发生的异常状况有关的图像，所述目标类似数据可包含具有所述目标数据的至少一部分的图像。
代替性地，所述目标数据及所述目标类似数据可以为包含误报的数据。
代替性地，以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的工作可包括在所述分类模型将所述输入数据分类为属于特定集群的情况下，将与所述特定集群匹配的检查结果生成为第二次检查结果，并以第二次检查结果为基础来输出最终检查结果，而且，在对所述输入数据的分类失败的情况下，将所述第一次检查结果作为最终检查结果来输出的工作。
代替性地，所述第二次检查结果可包含与是否存在异常、异常的位置、异常的类型及误检类型中的至少一种有关的信息。
代替性地，通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的工作可包括：利用所述预先学习的分类模型来对所述输入数据进行 处理，从而将所述输入数据的特征映射到所述预先学习的分类模型的解空间的工作；以及以所述解空间上的所述输入数据的位置为基础，并基于所述输入数据是否属于所述解空间上的一个以上的集群中的一个集群来分类所述输入数据的工作。
代替性地，所述解空间可由一维以上的空间构成，可包含一个以上的集群，而且，各个集群以基于各个目标数据的特征及以基于目标类似数据的特征在解空间上的位置为基础构成。
用于解决所述问题的本公开一实施例公开在包括一个以上的处理器的计算装置中执行的输入数据检查方法。所述方法包括：导出对输入数据的第一次检查结果的步骤；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的步骤；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的步骤。
用于解决所述问题的本公开的本公开另一实施例公开计算装置。所述计算装置包括：一个以上的处理器；以及存储器，用于存储能够在所述处理器中执行的指令，所述处理器执行如下工作：导出对输入数据的第一次检查结果；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果。
本公开可提供利用计算装置来处理数据的方法。
附图说明
图1为示出根据本公开一实施例执行输入数据检查方法的计算装置的框结构图。
图2a为示出本公开一实施例的网络函数的简图。
图2b为示出本公开一实施例的卷积神经网络(convolutional neural network)的简图。
图3a、3b、3c、3d、3e及3f为示出本公开的一实施例的学习数据的简图。
图4为示出训练本公开一实施例的分类模型的方法的简图。
图5为示出本公开一实施例的分类模型的解空间的简图。
图6为用于检查本公开一实施例的输入数据检查的方法的流程图。
图7为示出用于体现本公开一实施例的输入数据检查的方法的单元的框结构图。
图8为示出用于体现本公开一实施例的输入数据检查方法的模块的框结构图。
图9为示出用于体现本公开一实施例的输入数据检查方法的逻辑的框结构图。
图10为示出用于体现本公开一实施例的输入数据检查方法的电路的框结构图。
图11为示出本公开一实施例可体现的例示性计算环境的简要且一般的简图。
具体实施方式
参照附图，说明多种实施例。在本说明书中，多种说明为了提供本发明的理解而揭示。但是，这种实施例即使没有这种具体说明也可以执行。
在本说明书中使用的术语“组件”、“模块”、“系统”等为计算机相关实体、硬件、固件、软件、软件及硬件的组合或软件的执行。例如，组件为在处理器上执行的处理过程(procedure)、处理器、客体、执行线程、程序和/或计算机，但并不局限于此。例如，在计算装置执行的应用及计算装置均可以为组件。一个以上的组件可常驻在处理器和/或执行线程内。一组件可在一个计算机内逻辑化。一组件可分配在2个以上的计算机之间。并且，这种组件可从具有在内部存储的多种数据结构的多种计算机可读介质执行。例如，组件根据具有一个以上的数据包的信号(例如，逻辑系统、分散系统中，通过从与其他组件相互作用的一个组件的数据和/或信号，通过如其他系统和互联网的网路传送的数据)，通过本地和/或远程处理进行通信。
同时，术语“或”并非排他性“或”，而是包括的“或”。即，在并未特殊定义或在文脉上并未明确表示的情况下，“X利用A或B”意味着自然包括性置换中的一个。即，在X利用A，或者X利用B，或X利用A及B的情况下，“X利用A或B”的情况适用。并且，在本说明书中使用的“和/或”术语包括列举的相关项目的所有可能的组合。
并且，“包括”和/或“包括”术语意味着对应特征和/或结构要素的存在。只是，“包括”和/或“包括”术语并不排除一个以上的其他特征、结构要素和/或这些组合的存在或追加。并且，在并未特殊定义或文脉上并未明确表示指示单数形态的情况下，本说明书和发明要求范围中，单数通常意味着“一个或一个以上”。
本发明所属技术领域的普通技术人员知道追加在此揭示的实施例相关说明的多种例示性逻辑块、结构、模块、电路、单元、逻辑及算法可通过电子硬件、计算机软件或两者的组合体现。为了明确例示硬件及软件的相互交换性，多种例示性组 件、块、结构、单元、逻辑、模块、电路及部件在这些的功能性侧面如上相同。这种功能性是否作为硬件或软件体现取决于在系统中呈现的特定应用(appl ication)及设计限制。本公开所属技术领域的普通技术人员为了各个特定应用而通过多种方法体现说明的功能性。只是，这种体现的确定并不超出本解释内容的领域。
对于揭示的实施例的说明以使本公开所属技术领域的普通技术人员利用或实施本公开而提供。对于这种实施例的多种变形对本公开所属技术领域的普通技术人员来说是显而易见的。在此定义的一般原理在不超出本公开的范围的情况下可适用于其他实施例。因此，本公开并不局限于在此说明的实施例。本公开需要在与揭示的原理及新特征一贯的最广的范围中解释。
在本公开中，网络函数和人工神经网及神经网络(neural network)可以相互交换。
图1为示出根据本公开一实施例执行输入数据检查方法的计算装置的框结构图。
图1所示的计算装置100的结构为简要示出的例示，本公开一实施例中，计算装置100可包括用于执行本公开的实施例的其他结构。
计算装置100可包括处理器110、存储器120、通信模块130、摄像头模块140。
处理器110可由一个以上的芯构成，可包括计算装置的中央处理装置(CPU，central processing unit)、通用图形处理单元(GPGPU，general purpose graphics processing unit)、张量处理单元(TPU，tensor processing unit)等的数据分析、深度学习的处理器。处理器110读取存储于存储器120的计算机程序来执行本公开一实施例的输入数据检查方法。根据本公开的一实施例，处理器110可执行用于神经网学习的计算。处理器110可执行用于在深度学习(DN，deep learning)中，用于学习的输入数据的处理、输入数据中的特征(feature)提取、误差计算、利用逆传播(backpropagation)神经网的加权值更新等的神经网的学习的计算。处理器110的CPU、GPGPU及TPU中的至少一个可处于网络函数的学习。例如，CPU和GPGPU一同处理网络函数的学习、利用网络函数的数据分类。并且，在本公开的一实施例中，一同使用多个计算装置的处理器来处于网络函数的学习、利用网络函数的数据分类。并且，本公开的一实施例的计算装置执行的计算机程序可以为CPU、GPGPU或TPU可执行程序。
在本公开的一实施例中，计算装置100利用在CPU、GPGPU及TPU中的至少一 个来分散网络函数并处理。并且，在本公开的一实施例中，计算装置100与其他计算装置一同分散网络函数来处理。
在本公开的一实施例中，利用网络函数来处理的图像可以为存储于计算装置100的存储介质的图像、通过计算装置100的摄像头模块140拍摄的图像和/或通过通信模块130，从图像数据库等其他计算装置传送的图像。并且，在本公开的一实施例中，利用网络函数来处理的图像可以为存储于计算机可读存储介质(例如，可包括闪存等，但本公开并不局限于此)的图像。计算装置100通过输入输出界面(未图示)接收存储于计算机可读存储介质的图像文件。
在本公开一实施例中，计算装置100接收图像数据，利用预先学习的网络函数来检查输入数据。本公开的输入数据检查可包含用于判断在输入数据是否存在异常的异常检测。本公开的输入数据检查可包括与异常检测及异常检测有关的误检判断。
存储器120可存储用于执行本公开一实施例的输入数据检查方法的计算机程序，存储的计算机程序通过处理器110读取并驱动。
通信模块130向其他计算装置、服务器等收发用于执行本公开一实施例的图像数据的异常判断方法。通信模块130向其他计算装置、服务器等收发用于图像数据等本公开的实施例的数据。例如，通信模块130在学习图像数据库等中接收学习图像数据。并且，通信模块130可以使多个计算装置之间通信，从而在多个计算装置各自中执行网络函数的学习分散。通信模块130使多个计算装置之间通信，从而可分散处理使用网络函数的数分类。
摄像头模块140为了执行本公开一实施例的图像数据的异常判断方法而拍摄检查对象物来生成图像数据。本公开一实施例中，计算装置100可包括一个以上的摄像头模块140。
图2a为示出本公开一实施例的网络函数200的简图。
在本说明书中，计算模型、神经网、网络函数、神经网络(neural network)可以为相同含义。神经网可由一般称为“节点”的相互连接的计算单元的集合构成。这种“节点”也可称为“神经元(neuron)”。神经元包括至少一个以上的节点。构成神经网的节点(或神经元)通过一个以上的“连接单元”相连接。
在神经网内，通过连接单元连接的一个以上的节点形成输入节点及输出节点的关系。输入节点及输出节点的概念为相对性，对一个节点，存在输出节点关系的任 意的节点在与其他节点的关系中，存在与输入节点关系，反之亦然。如上所述，输入节点与输出节点关系以连接单元为中心生成。在一个输入节点，通过连接单元与一个以上的输出节点相连接，反之亦然。
通过一个连接单元连接的输入节点及输出节点关系中，输出节点以向输入节点输入的数据为基础确定值。其中，相互连接输入节点和输出节点的节点可具有加权值(weight)。加权值可以改变，神经网为了执行所需要的功能，可通过用户或算法改变。例如，在通过各个的连接单元，一个输出节点与一个以上的输入节点相连接的情况下，输出节点以向与所述输出节点相连接的输入节点输入的值及在与各个输入节点相对应的连接单元设定的加权值为基础来确定输出节点值。
如上所述，神经网中，一个以上的节点通过一个以上的连接单元相互连接，从而在神经网内形成输入节点及输出节点关系。在神经网内，根据节点和连接单元的数量及节点与连接单元之间的连接关系、向连接单元复原的加权值设定神经网的特性。例如，存在相同数量的节点及连接单元，在存在连接单元之间的加权值不相同的两个神经网的情况下，两个神经网不相同。
神经网可包括一个以上的节点。构成神经网的节点中的一部分以从最初输入节点的距离为基础来构成一个层(layer)。例如，从最初输入节点开始，距离为n的节点的集合可构成n层。从最初输入节点的距离通过为了从最初输入节点到达对应节点而需要经过的连接单元的最少数量确定。但是，这种层的定义用于说明，在神经网内，层的差异可通过与所述不同的方法定义。例如，节点的层可通过最终节点的距离定义。
最初输入节点为在神经网内的节点中，在与其他节点的关系中，不经过连接单元，直接输入数据的一个以上的节点。或者，在神经网络内，以连接单元为基准的节点之间的关系中，不遮挡通过连接单元连接的其他输入节点的节点。与此类似，最终输出节点为在神经网内的节点中，与其他节点的关系中，不具有输出节点的一个以上的节点。并且，隐藏节点为并非为最初输入节点及最后输出节点的构成神经网的节点。本公开一实施例的神经网中，输入层的节点数量与输出层的节点数量相同，在输入层中，沿着隐藏节点，节点的数量减少并再次增加的神经网。并且，本公开另一实施例的神经网中，输入层的节点的数量大于输出层的节点的数量，在输入层中，沿着隐藏节点，节点的数量减少。并且，本公开另一实施例的神经网中，输入层的节点的数量大于输出层的节点数量，输入层中，沿着隐藏节点，节点的数 量增加。本公开的另一实施例的神经网为神经网的组合形态的神经网。
深度神经网络(DNN，deep neural network)为除输入层和输出层之外，包括多个隐藏层的神经网。若利用深度神经网络，则可把握数据的潜在结构(latent structures)。即，可把握照片、文字、视频、语音、音乐的潜在结构(例如，照片中存在哪一物体，文字的内容和感情为何，语音的内容和感情为何等)。深度神经网络可包括卷积神经网络(CNN，convolutional neural network)、递归神经网络(RNN，recurrent neural network)、自动编码器(auto encoder)、生成性对抗网络(GAN，Generative Adversarial Networks)、受限玻尔兹曼机(RBM，restricted boltzmann machine)、深度信念网络(DBN，deep belief network)、Q网络、U网络、暹罗网络等。所述深度神经网络的记载为例示，本公开并不局限于此。
在本公开的一实施例中，网络函数200还可包括自动编码器。自动编码器可以为用于输出与输入数据类似的输出数据的人工神经网的一种。自动编码器可包括至少一个隐藏层，单数的隐藏层可配置于输入输出层之间。各个层的节点数量在输入层的节点的数量中，被缩小为称为瓶颈层(编码)的中间层，之后，在瓶颈层中，与缩小为输出层(与输入层对称)的对称地扩大。在此情况下，图2的例示中示出维度减少层和维度复原层对称，本公开并不局限于此，维度减少层和维度复原层的节点可以对称，也可以不对称。自动编码器可执行非线性维度减少。输入层及输出层的数量与在输入数据的预处理器之后剩下的传感器的数量相对应。自动编码器结构中，在编码器中的隐藏层的节点的数量沿着远离输入层的方向减少。在瓶颈层(具有位于编码器和解码器之间的最少节点的层)的节点的数量过少的情况下，无法传递充分量的信息，因此，可维持特定数量以上(例如，输入层的一半以上等)。
神经网络可通过监督学习(supervised learning)、无监督学习(unsupervised learning)及半监督学习(semi supervised learning)中的至少一种方式学习。神经网络的学习用于使输出的误差最小化。神经网络的学习过程中，反复向神经网络输入学习数据，并计算与学习数据有关的神经网络的输出和目标的错误，沿着用于减少错误的方向，将神经网络的错误从神经网络的输出层向输入层方向逆传播来更新神经网络的各个节点的加权值。在监督学习的情况下，使用在各个学习数据标记有正确答案的学习数据(即，标记有的学习数据)，在无监督学习的情况下，在各个学习数据并未标记有正确答案。即，例如，在与数据分类有关的监督学习的情 况下的学习数据为在学习数据标记有类别的数据。标记有的学习数据向神经网络输入，比较神经网络的输出(类别)和学习数据的标记来计算错误(error)。作为另一例，与数据分类有关的无监督学习的情况下的输入的学习数据与神经网络输出比较，由此计算错误。计算的错误在神经网络中向逆方向(即，在输出层向输入层方向)逆传播，随着逆传播，神经网络的各个层的各个节点的连接加权值会更新。更新的各个节点的连接加权值可根据学习率(learning rate)变化。与输入数据有关的神经网络的计算和错误的逆传播可构成学习循环(epoch)。学习率可根据神经网络的学习循环的反复次数不同地适用。例如，在神经网络的学习初期，适用高的学习率，神经网络迅速确保规定水平的性能来提高效率，在学习后期，使用低的学习率来提高准确度。
在神经网络的学习中，通常，学习数据为实际数据(即，利用学习的神经网络来处理的数据)的部分集合，因此，存在与学习数据有关的错误减少或者对实际数据增加错误的学习循环。如上所述，过度拟合(overfitting)过度进行学习，从而导致与实际数据有关的错误增加的现象。例如，过度拟合的一种为看到黄色猫之后，学习到猫的神经网络无法识别除黄色猫之外的猫。过度拟合增加机器运行算法的错误。为了防止这种过度拟合算法，可以使用多种最优化方法。为了防止过度拟合而增加学习数据，或者正规化(regularization)、学习过程中省略网络的节点一部分的流失(dropout)等的方法。
图2b所示的卷积神经网络(CNN，convolutional neural network)为深度神经网络的一种，包含具有卷积层的神经网。卷积神经网络为以使用最小限度预处理(preprocess)的方式设计的多层受体(multilayer perceptrons)的一种。卷积神经网络可由一个或一个以上的卷积层和与此相结合的人工神经网层构成，追加使用加权值和汇集层(pooling layer)。因这种结构，卷积神经网络可充分使用二维结构的输入数据。卷积神经网络用于在图像中识别目标。卷积神经网络可将图像数据分为具有次元的行列来处理。例如，在以红-绿-蓝(RGB，red-green-blue)编码的图像数据的情况下，按红绿蓝颜色分别分为二维(例如，二维图像的情况)行列。即，图像数据的各个像素的颜色可分为行列的成分，行列的大小与图像的大小相同。因此，图像数据可分为3个二维行列(三维数据阵列)。
卷积神经网络中，移动卷积滤镜来将卷积滤镜和图像的各个位置的行列成分相乘来执行卷积过程(卷积阵列的输入输出)。卷积滤镜可呈n*n形态的行列，通常， 可由小于图像的整体像素的数量的固定形态的滤镜构成。即，在向卷积层(例如，卷积滤镜的尺寸为n*n的卷积层)输入m*m图像的情况下，表示包含图像的各个像素的n*n像素的行列与卷积滤镜成分相乘(即，行列的各个成分相乘)。通过与卷积滤镜的相乘，在图像中可提取与卷积滤镜相匹配的成分。例如，图像中，用于提取上下直线成分的3*3卷积滤镜如[[0，1，0]、[0，1，0]、[0，1，0]]构成，若这种卷积滤镜适用于输入图像，则图像中提取与卷积滤镜匹配的上下直线成分并输出。卷积层可在于各个频道有关的各个行列(即，R、G、B涂敷图像的情况下，R、G、B颜色)使用卷积滤镜。卷积层在输入图像使用卷积滤镜来在输入图像中提取与卷积滤镜匹配的特征。卷积滤镜的滤镜值(即，行列的各个成分的值)在卷积神经网络的学习过程中通过逆传播更新。卷积层的输出与子采样层相连接，从而使卷积层的输出简化来减少存储器的使用量和计算值。例如，在向具有2*2最大汇集滤镜的汇集层输入卷积层的输出的情况下，图像的各个像素中，每个2*2输出包含在各个补丁中的最大值来压缩图像。所述汇集在补丁输出最小值，或者输出补丁的平均值，任意汇集方法可包含在本公开。
卷积神经网络可包括一个以上的卷积层、子采样层。卷积神经网络反复执行卷积过程和子采样过程(例如，所述最大汇集等)来在图像中提取特征。通过反复的卷积过程和子采样过程，神经网络可提取图像的全球特征。
卷积层或子采样层的输出可向完全连接层(fully connected layer)输入。完全连接层为与一个层的所有神经元相邻的层的所有神经元相连接的层。完全连接层为在神经网络中，各个层的所有节点与其他层的所有节点相连接的结构。
在本公开的一实施例中，为了执行图像数据的分割(segmentation)，神经网络可包括反卷积神经网络(DCNN，deconvolutional neural network)。反卷积神经网络执行将卷积神经网络向逆方向计算类似的工作，将在卷积神经网络中提取的特征作为与原本数据有关的特征输出。
在本公开的一实施例中，处理器110可导出对输入数据的第一次检查结果。第一次检查结果可包含与输入数据有关的异常信息，异常信息可包含存在于输入数据的异常有关的任意信息。例如，第一次检查结果可包括异常的存在与否、输入数据中异常的位置、异常的类型。在输入数据为图像的情况下，第一次检查结果可包括在输入数据是否存在异常、输入数据的哪个像素存在异常的区域有关的信息、包含在输入数据中的异常为哪一类型的异常(例如，在与半导体有关的图像的情况下， 焊接不良、导线不良、元件不良等)等有关的信息。
处理器110可导出与在输入数据是否存在异常有关的第一次检查结果。处理器110利用包含一个以上的网络函数的预先学习的检查模型来处理输入数据，由此，执行与输入数据有关的异常检查来导出第一次检查结果。检查模型为仅通过正常数据学习，判断在输入数据是否存在未学习的新模式(即，异常)，由此，可判断在输入数据是否存在异常的半监督学习模型。并且，检查模型为通过标记有的正常数据及异常数据学习，从而，可判断在输入数据是否包含异常的监督学习模型。检查模型可包括判断在输入数据是否存在异常的任意的机械学习模型。
并且，处理器110以输入数据与参照数据的比较为基础来执行与输入数据有关的异常检测来导出第一次检查结果。在输入数据为图像的情况下，处理器110以包含异常的参考数据或不包含异常的参考数据与输入数据的比较为基础来判断在输入数据是否存在异常。处理器110以任意机器视觉(machine vision)解决方案为基础来在输入数据判断异常。
处理器110将输入数据利用使用标记有集群数据的学习数据来学习的分类模型分类。分类模型利用包含多个学习数据子集的学习数据集来学习，包含在学习数据集中的学习数据中，将类似的数据分类为相同集群，将非类似的数据分类为不同集群。学习数据子集可包含多个学习数据。学习数据子集可包含多个学习数据，各个学习数据标记有集群信息。在本公开中，学习数据为子集在网络函数的学习中，1反复(iteration)中使用的数据。在本公开中，学习数据集为在网络函数的学习中，用于循环的数据整体。学习数据子集可包含非类似的学习数据，包含在学习数据子集的非类似的学习数据通过分类模型分类为不同集群。
即，学习数据子集包含多个学习数据，学习数据各自标记有集群信息，学习数据子集还可包含标记有不同集群信息的学习数据。具体地，学习数据子集可包含目标数据、目标类似数据、目标非类似数据，可标记有如目标数据和目标类似数据的集群信息，目标非类似数据标记有与目标数据及目标类似数据不相同的集群信息。包含在学习数据子集中的目标数据和目标类似数据可标记有第一集群信息，目标非类似数据可标记有第二集群信息。
例如，在目标数据为包含异常的数据的情况下，目标类似数据为包含异常的数据，目标非类似数据为不包含异常的数据。目标非类似数据为在目标数据中，除异常之外的部分的至少一部分重复的数据。例如，在目标数据为图像的情况下，目标 非类似数据为在包含异常的图像中，除异常部分之外的部分的至少一部分的图像。并且，例如，目标类似数据为包含与目标数据所包含的异常类似类型的异常的数据。例如，在目标数据为包含异常的图像的情况下，目标类似数据以包含目标数据所包含的异常的至少一部分的方式裁剪(cropping)的图像。例如，目标数据所包含的异常类似类型的异常可包含与目标数据所包含的异常的检查对象客体上的位置相同位置的异常，与目标数据所包含的异常的检查对象客体的不良类型相同的类型(例如，在检查对象客体为电路板，目标数据所包含的异常为焊接不良的情况下，目标类似数据所包含的异常也是焊接不良)的异常。
目标类似数据可包含目标数据的图像的至少一部分，与在目标数据的图像中的异常相对应的像素部分为必须包含的数据。
目标非类似数据为不包含异常的图像。其中，目标数据为在图像的中心部包含异常的图像。目标非类似数据为除目标数据的异常之外的部分的至少一部分重复的图像。
在数据为图像的情况下，包含异常的图像和不包含异常的图像的异常部分不相同，此外的部分可以类似，从而，在通过通常方法分类的情况下，可分类为类似的图像。例如，在半导体工序中获取的100*100像素的图像中，在异常部分为5*5像素的情况下，异常部分以整体的0.25％通常被分类的情况下，两个图像的99.75％类似，因此，通过通常的分类方法，被分类为类似。但是，在利用本公开一实施例的分别标记有集群信息的目标数据、目标类似数据及目标非类似数据来学习分类模型的情况下，即使在图像上的一部分不相同的情况下，被分类为非类似的数据。
处理器110在分类模型中，将目标数据和目标类似数据分类为相同集群，将目标非类似数据分类为与目标数据及目标类似数据所属的集群不相同的集群。
本公开中，目标数据可包含的异常中可包含在目标数据位图像的情况下，包含在图像中的检查对象客体有关的异常及从与检查对象客体分离的异常中的至少一个。例如，在目标数据为从半导体工序中获取的图像的情况下，目标数据可包含的异常可以为在半导体产品可发生的异常(例如，不良等)及与检查对象客体无关的异常(例如，图像获取装置的问题、镜头异物等)中的至少一个。即，在异常检测的观点，目标数据及目标类似数据可包含误报(false positive)。更具体地，在检查对象客体不存在异常，在存在与检查对象客体分离而可发生的状况的情况下，目标数据及目标类似数据可包含这种误报。因此，分类模型将异常类型和误报分类 为集群。
在输入数据包含与对象客体无关的异常的情况下，第一次检查结果可包含误报。在此情况下，输入数据利用分类模型来第二次分类，处理器110利用分类模型来分类输入数据的结果，通过利用包含误报的学习数据来学习的分类模型，在输入数据的第二次分类中，可判断是否为与实际检查对象客体有关的异常或与检查对象客体无关的异常(即，误报)。
以下，参照附图，详细说明本公开的学习数据。
图3a、3b、3c、3d、3e及3f为示出本公开的一实施例的学习数据的简图。图3a至图3e所示的图像仅是学习数据的例示，本公开可包含任意的学习数据、任意的图像。
图3a为示出可作为学习数据的目标基础图像300的简图。目标基础图像300自身可作为用于分类模型的学习的目标图像，裁剪目标基础图像300来用于目标图像310。例如，目标基础图像300可包括检查对象客体的不良301异常。多个目标类似图像311、313可以为以包含目标图像310的不良301部分的方式裁剪的图像。图3a的多个裁剪线310、311、313为目标图像310的裁剪、目标类似图像的裁剪311、313。
图3b为示出通过图3a的裁剪线裁剪的目标图像310(target image)、目标类似图像311、313(target similar image)。参照图3b，目标图像310和目标类似图像311、313均包含异常301，可包含检查对象客体的其他部分。
图3c为示出目标非类似基础图像320的例示图。图3c所示的目标非类似基础图像320自身可用为目标非类似图像(target dissimilar image)使用，一部分被裁剪并用为目标非类似图像。为了学习本公开的分类模型而使用裁剪的目标非类似图像的情况下，目标非类似图像320可具有如目标图像310、目标类似图像311的分辨率。所述目标图像、目标类似图像及目标非类似图像的分辨率也是例示，这些的分辨率可以相同或不相同。目标非类似图像320可以为不包含异常的图像。目标非类似图像320为在与目标图像相对应的一部分321不包含异常的图像。
图3d为示出用于学习本公开一实施例的分类模型的学习数据子集的例示图。
学习数据子集可包含具有异常301的目标图像310、目标类似图像311及不具有异常321的目标非类似图像320。目标图像310和目标类似图像311以被分类为相同集群的方式标记有相同集群信息。目标非类似图像320标记有与在目标图像310 及目标类似图像311标记有的集群信息不相同的集群信息。例如，在目标图像310和目标类似图像311可标记有第一集群信息，在目标非类似图像320可标记有第二集群信息。
在利用图3d所示学习数据子集学习分类模型的情况下，分类模型可将存在异常的图像和不存在异常的图像分类为不同集群。
例如，在半导体工序中获取的100*100像素的图像中，在异常部分为5*5像素的情况下，异常部分以整体的0.25％通常被分类的情况下，两个图像的99.75％类似，因此，通过通常的分类方法，被分类为类似。但是，在利用本公开一实施例的分别标记有集群信息的目标数据、目标类似数据及目标非类似数据来学习分类模型的情况下，即使在图像上的一部分不相同的情况下，被分类为非类似的数据。
图3e为示出用于学习本公开一实施例的分类模型的其他学习数据子集的例示图。
图3e的目标图像330、目标类似图像333可以为包含误报的图像。更具体地，包含在目标图像330及目标类似图像333中的异常331与检查对象客体分离而可发生的异常状况(例如，镜头异物等)。目标图像330可以为与正常产品有关的图像或根据与正常产品分离的异常状况发生的异常的图像。根据以往的异常检查方式，在包含这种误报的异常图像的情况下，被判断为异常，从而输出错误的结果。但是，在本公开的一实施例中，利用包含误报的图像，分类模型生成与误报有关的集群，由此，可检测包含在作为检查对象的输入图像的检查对象客体实际上是否包含异常，检查对象客体不包含异常，但是，因其他理由，输入数据被判断为异常的检测。图3e的例示中，目标非类似图像340为不包含异常的图像。
在利用图3e所示的学习数据子集来学习分类模型的情况下，分类模型可将存在误报的图像和不存在误报的图像分类为不同集群。
图3f为示出用于学习本公开一实施例的分类模型的另一学习数据子集的例示图。
图3f的目标图像330、目标类似图像333可以为包含误报的图像。更具体地，包含在目标图像330及目标类似图像333中的异常331为与检查对象客体分离而可发生的异常状况(例如，镜头异物等)。目标图像330可以为与正常产品有关的图像或根据与正常产品分离的异常状况发生的异常的图像。根据以往的异常检查方式，在包含这种误报的异常图像的情况下，被判断为异常，从而输出错误的结果。 但是，在本公开的一实施例中，利用包含误报的图像，分类模型生成与误报有关的集群，由此，可检测包含在作为检查对象的输入图像的检查对象客体实际上是否包含异常，检查对象客体不包含异常，但是，因其他理由，输入数据被判断为异常的检测。图3e的例示中，目标非类似图像310为包含异常301(即，与检查对象客体有关的异常)的图像，而并非包含误报。在利用图3f所示的学习数据子集来学习分类模型的情况下，分类模型可将存在误报的图像和在实际检查对象客体存在异常的图像(即，在异常检查中分类为异常的图像)分类为不同集群。
包含误报的目标图像330按误报的类型追加。换句话说，在产业现场，在执行异常检测解决方案的过程中，在发生新的误检的情况下，将新的误检作为包含误报的目标图像来追加学习分类模型。本公开的分类模型可将众所周知的异常类型和众所周知的误检类型集群化，可判断输入数据属于哪一类型。
图4为示出训练本公开一实施例的分类模型的方法的简图。
本公开的分类模型在在解空间400上集群类似的数据。更具体地，分类模型使目标数据401和目标类似数据402包含在一个集群410，目标非类似数据403包含在与目标数据401和目标类似数据402不同的集群。
在学习的分类模型的解空间上，各个集群具有规定距离边距420。
分类模型接收包含目标数据401、目标类似数据402、目标非类似数据403的学习数据子集来将各个数据与解空间匹配，以根据在解空间上标记有的集群信息来集群的方式更新包含分类模型的一个以上的网络函数的加权值。即，分类模型以在目标数据401和目标类似数据402的解空间上的距离变短的方式使目标数据401及目标类似数据402和目标非类似数据403之间的解空间上的距离增加。例如，分类模型使用基于三重(triplet)费用函数来学习。基于三重费用函数将输入数据从其他分类的第三输入数据分离，作为相同分类的输入数据之间的第一距离(即，集群410的大小)与作为相同分类的输入数据中的一个与第三输入数据之间的第二距离(即，401或402与403之间的距离)之间的差值为至少距离边距420，学习分类模型的方法包括将第一距离减少至距离边距的规定比例以下的步骤。其中，距离边距420一直为正数。为了到达距离边距420，包含在分类模型中的一个以上的网络函数的加权值可以更新，加权值更新在每个反复(iteration)或1时代执行。与这种距离边距有关，详细内容作为本说明书中的参照合并的Schroff等的“FaceNet：A Unified Embedding for Face Recognition and Clustering”及 韩国公开专利10-2018-0068292中公开。
并且，分类模型通过不仅考虑非类似数据的集群分类，而且考虑一个集群或其他集群之间的各个数据之间的语义(semantic)的关系的基于磁铁损失(magnet loss)的模型进行学习。在分类模型的解空间上，在学习过程中执行各个集群的中心点差异的初期距离。分类模型在解空间上匹配数据之后，以与各个数据所属的集群及集群内部及集群外部的数据的类似性为基础来调节各个数据的解空间上的位置。基于磁铁损失的分类模型的学习有关的详细内容整体作为本说明书中的参照合并的O.Rippel等的“METRIC LEARNING WITH ADAPTIVE DENSITY DISCRIMINATION”。
图5为示出本公开一实施例的分类模型的解空间的简图。
图5为示出分类模型的解空间400的简图。图5所示的解空间400仅是例示，分类模型可包含任意数量的集群及各个集群的任意的数量的数据。图5所示的集群中的数据431、433、441、443为类似的数据。
在本公开中，解空间由一维以上的空间构成，且包括一个以上的集群，各个集群以基于各个目标数据的特征及以基于目标类似数据的特征在解空间上的位置为基础构成。
在解空间，第一集群430和第二集群440可以为与非类似的数据有关的集群。并且，第三集群450可以为与第一及第二集群非类似的数据有关的集群。集群之间的距离445、435为呈现出属于各个集群的数据差异的尺度。
第一集群430与第二集群440之间的第十二距离445为属于第一集群430的数据与属于第二集群440的数据差异的尺度。并且，第一集群430与第二集群440之间的第十三距离435为属于第一集群430的数据与属于第三集群450的数据差异的尺度。图5所示的例示中，与属于第三集群450的数据相比，属于第一集群430的数据与属于第二集群的数据更非类似。即，在集群之间的距离远的情况下，属于各个集群的数据更非类似，在集群之间的距离短的情况下，属于各个集群的数据更类似。几区之间的距离435、445比集群的半径大预先确定的比例以上。处理器110利用分类模型来计算输入数据，由此，以在分类模型的解空间匹配输入数据的特征的位置为基础分类输入数据。
处理器110利用预先学习的分类模型处理输入数据，由此，将输入数据的特征与预先学习的分类模型的解空间进行匹配。处理器110以输入数据的解空间上的位置为基础，以所述输入数据属于在解空间上的一个以上的集群中的哪个集群来分类 输入数据。
处理器110以与分类模型的输入数据有关的分类结果的至少一部分为基础来确定最终检查结果。
在分裂模型将输入数据分类为属于特定集群的情况下，处理器110将与特定集群匹配的检查结果生成为第二次检查结果，以第二次检查结果确定最终检查结果。并且，在分类模型与输入数据的分类失败的情况下，处理器110将第一次检查结果确定为最终检查结果，其中，第二次检查结果可包含异常的存在与否、异常的位置、异常的类型、误检与否及误检类型中的至少一个有关的信息。
预先学习的分类模型在解空间按异常的类型保留集群，因此，处理器110利用分类模型来生成与输入数据有关的第二次检查结果。处理器110以输入数据的特征属于在分类模型的解空间上的哪一集群为基础来判断在输入数据是否存在异常，在存在异常的情况下，可判断异常是否为存在检查对象客体上的异常或与检查对象客体无关的异常(即，误检)。例如，在输入数据的特征在解空间上的位置属于与基板焊接不良有关的集群的情况下(即，利用分类模型，输入数据被分类为基板焊接不良的情况)，处理器110可生成在输入数据存在作为基板焊接不良的异常的第二次检查结果，以第一次检查结果及第二次检查结果为基础来确定最终检查结果。并且，例如，在输入数据的特征在解空间上的位置属于与对象客体拍摄镜头异物有关的集群的情况下，处理器110可生成对应输入数据为误检，不存在异常的正常数据的第二次检查结果，以第一次检查及第二次检查结果为基础来确定最终检查结果。
在所述例示的输入数据，在第一次检查中判断为基板焊接不良的情况下，第一次及第二次检查结果相同，因此，处理器110将基板焊接不良确定为与输入数据有关的异常检测的最终结果。在所述例示中，在输入数据在第一次检查中被分类为其他异常的情况下(即，第一次和第二次分类结果不同的情况)，处理器110可向操作者告知所述内容。处理器110为了向操作者告知第一次检查结果及第二次检查结果的不相同而在对应输入数据进行标记。并且，处理器110在第一次分类结果及第二次分类结果不相同的情况下，将第一次检查结果和第二次检查结果合奏来生成最终检查结果。例如，合奏方法可使用表决方法、基于各个神经网的加权值的方法，可包括任意其他合奏方法。
并且，在输入数据的特征在解空间上的位置不属于任何集群的情况下，处理器110确定利用输入数据的分类模型的分类失败，将第一次检查结果生成为最终检查 结果。例如，在输入数据包含电路导线不良的检查对象客体有关的新异常或镜头破碎的检查对象客体有关的异常的情况下，处理器110(即，学习的分类模型存在未学习的新模式的情况等)，利用分类模型无法分类输入数据，因此，与对应输入数据有关的最终检查结果以第一次检查结果为基础生成。在此情况下，处理器110可向操作者告知所述内容。处理器110为了向操作者告知所述内容，需要标记对应输入数据。并且，利用分类模型无法分类的对应输入数据包含异常的情况下，处理器110可将对应输入数据生成为新的学习数据。换句话说，在具有利用分类模型无法分类的异常的数据(即，第一次检查为异常的情况等)为图像的情况下，处理器110将对应输入数据作为目标图像，可生成将包含对应输入数据的异常的其他部分作为目标类似数据的新学习数据子集。处理器110利用新生成的学习数据子集来生成在新异常模式有关的分类模型的解空间上的集群而追加学习分类模型。
即，在本公开的一实施例中，通过第一次及第二次检查判断输入数据的异常，输入数据包含通过除检查对象客体之外的因素发生的异常状况有关的异常(即，误报)的情况下，可导出检查结果，而并非为误检。并且，在发生新误检类型或异常类型的情况下，利用其来再次学习分类模型，由此，可维持与新发生的误检和异常相对应的规定检查性能。
并且，分离第一次检查和第二次检查，由此，向异常检测解决方案追加新类型的异常或新类型的误检并不对与以往数据有关的分类性能产生影响。即，在以往的解决方案，在发生反复误检的情况下，修改解决方案自身来解决对应误检的问题，由此，可发生其他错误工作，本公开一实施例的异常检测中，可断开追加错误工作的可能性。
图6为用于检查本公开一实施例的输入数据检查的方法的流程图。
计算装置100可导出对输入数据的第一次检查结果(步骤610)。在本公开中，输入数据可包含图像数据，计算装置100可确定与拍摄的图像数据或从外部设备传送的图像数据有关的第一次检查结果。第一次检查结果可包含与输入数据有关的异常信息。
计算装置100通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类(步骤S630)。分类模型使用包含多个学习数据子集的学习数据集来学习，在包含在学习数据集的学习数据中，将类似的数据分类为相同集群，将非类似的数据分类为其他集群来。
计算装置100以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果(步骤650)。在分类模型将输入数据分类为属于特定集群的情况下，计算装置100将与特定集群匹配的检查结果生成为第二次检查结果，以在第二次检查结果中的至少一部分为基础来确定最终检查结果。并且，在分类模型于输入数据有关的分类失败的情况下，计算装置100将第一次检查结果确定为最终检查结果。
图7为示出用于体现本公开一实施例的输入数据检查的方法的单元的框结构图。
根据本公开的实施例，用于对输入数据进行检查的方法可通过如下单元体现。
根据本公开的一实施例，用于对输入数据进行检查的方法可通过如下单元体现：导出对输入数据的第一次检查结果的单元710；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的单元730；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的单元750。
本公开的用于对输入数据进行检查的方法的另外实施例中，用于导出对输入数据的第一次检查结果的单元710可包括利用包含一个以上的网络函数的预先学习的检查模型来对所述输入数据进行处理，由此对所述输入数据执行异常检测，或者以所述输入数据和参考数据的比较为基础来执行与所述输入数据有关的异常检查的单元。
本公开的用于对输入数据进行检查的方法的另一实施例中，对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的单元750可包括在所述分类模型将所述输入数据分类为属于特定集群的情况下，将与所述特定集群匹配的检查结果生成为第二次检查结果，以第二次检查结果为基础来输出最终检查结果，而且，在对所述输入数据的分类失败的情况下，将所述第一次检查结果输出为最终检查结果的单元。
本公开的用于对输入数据进行检查的放大的另一实施例中，通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的单元730可包括：利用所述预先学习的分类模型对所述输入数据进行处理，由此，将所述输入数据的特征映射到所述预先学习的分类模型的解空间的单元；以及以所述解空间上的所述输入数据的位置为基础，以所述输入数据是否属于在所述解空间上的一个以上的集 群中的一个集群来分类所述输入数据的单元。
图8为示出用于体现本公开一实施例的输入数据检查方法的模块的框结构图。
根据本公开的一实施例，用于对输入数据进行检查的方法可通过如下模块体现。
根据本公开的一实施例，用于对输入数据进行检查的方法可通过如下模块体现：导出对输入数据的第一次检查结果的模块810；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的模块830；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的模块850。
图9为示出用于体现本公开一实施例的输入数据检查方法的逻辑的框结构图。
根据本公开的一实施例，用于对输入数据进行检查的方法可通过如下逻辑体现。
根据本公开的实施例，用于对输入数据进行检查的方法可通过如下逻辑体现：导出对输入数据的第一次检查结果的逻辑910；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的逻辑930；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的逻辑950。
图10为示出用于体现本公开一实施例的输入数据检查方法的电路的框结构图。
根据本公开的一实施例，用于对输入数据进行检查的方法可通过如下单元体现。
根据本公开的一实施例，用于对输入数据进行检查的方法可通过如下电路体现：导出对输入数据的第一次检查结果的电路1010；通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的电路1030；以及以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的电路1050。
本公开所属技术领域的普通技术人员知道追加在此揭示的实施例相关地说明的多种例示性逻辑块、结构、模块、电路、单元、逻辑及算法步骤通过电子硬件、计算机软件或两个的组合体现。为了明确例示硬件及软件的相互交换性，多种例示性组件、块、结构、单元、逻辑、模块、电路及部件在这些的功能性侧面如上相同。这种功能性是否作为硬件或软件体现取决于在系统中呈现的特定应用(application)及设计限制。本公开所属技术领域的普通技术人员为了各个特定 应用而通过多种方法体现说明的功能性。只是，这种体现的确定并不超出本解释内容的领域。
图11为示出本公开一实施例可体现的例示性计算环境的简要且一般的简图。
本揭示与一般在一个以上的计算机上执行的计算机可执行指令相关来说明，只要本公开所属技术领域的普通技术人员，可通过与模块相结合和/或硬件和软件的组合体现。
通常，程序模块包括执行特定任务或体现特定抽象数据类型的例程、程序、组件、数据结构、其他等等。并且，只要是本公开所属技术领域的普通技术人员，知道本公开的方法不仅在单一处理器或多处理器计算机系统、小型计算机、大型计算机，而且在包括个人电脑、手持计算装置、基于微处理器或可编程家电、其他等等(这些与一个以上的相关装置相连接来进行工作)在内的其他计算机系统结构实施。
本公开说明的实施例可在通过任何任务通过通信网络连接的远程处理装置执行的分散计算环境中实施。在分散计算环境中，程序模块均可位于本地及远程存储装置。
计算机通常包括多种计算机可读介质。通过计算机，可访问介质均可变为计算机可读介质，这种计算机可读介质包括易失性及非易失性介质、暂时性(transitory)及非暂时性(non-transitory)介质、移动式及非移动式介质。作为不限制的例，计算机可读介质可包括计算机可读存储介质及计算机可读传送介质。计算机可读存储介质包括存储通过如计算机可读指令、数据结构、程序模块或其他数据的信息的任意方法或技术体现的易失性及非易失性介质介质、暂时性及非暂时性介质、移动式及非移动式介质。计算机存储介质可包括RAM、ROM、EEPROM、闪存或其他存储技术、CD-ROM、数字视频磁盘(DVD，digital video disk)或其他光盘存储装置、磁带、磁带、磁盘存储装置或其他磁存储装置或通过计算机访问并存储所需要的信息的任意其他介质，但并不局限于此。
计算机可读传送介质通常包括在如载波(carrier wave)或其他传送机制的调制数据信号(modulated data signal)体现计算机可读指令、数据结构、程序模块或其他数据等并传递所有信息的介质。调制数据信号术语以在信号内编码信息的方式在所述信号的特性中，设定或变更一个以上的信号。作为不限制的例，计算机可读传送介质包括如有线网络或直接配线连接(direct-wired connection)的有 线介质、音响、RF、红外线、如其他无线介质的无线介质。所述介质中的任意组合也包括在计算机可读传送介质的范围内。
呈现出体现包括计算机1102的本公开的多个侧面的例示性环境1100，计算机1102包括处理装置1104、系统存储器1106及系统总线1108。系统总线1108使包括系统存储器1106(并不局限于此)在内的系统组件与处理装置1104相连接。处理装置1104可以为多种常用处理器中的任意处理器。双处理器及其他多处理器架构也可作为处理装置1104利用。
系统总线1108可以为在存储器总线、周边装置总线及多种常用总线架构中使用任意的本地总线追加相互连接的几个类型的总线结构中的一个。系统存储器1106包括只读存储器2110及随机存取存储器2112。基本输入输出系统(BIOS)存储于RROM、EPROM、EEPROM等的非易失性存储器2110，包括所述输入输出系统当启动时，传送计算机1102内的结构要素之间的信息的基本历程。随机存取存储器2112可包括用于高速缓存数据的静止随机存取存储器等的高速随机存取存储器。
计算机1102包括内置型硬盘驱动器2114(HDD)(例如，EIDE、SATA，内置型硬盘驱动器2114也在适当底盘(未图示)内被用为外置型)、磁性软盘驱动器2116(FDD)(例如，从可移动软盘2118读取或者用于在其进行记录)及光盘驱动器1120(例如，读取CD-RON盘1122或者从DVD等的其他高容量光介质读取或者用于在其进行记录)。硬盘驱动器2114、磁盘驱动器2116及光盘驱动器1120可分别通过硬盘驱动器接口1124、磁盘驱动器接口1126及光盘驱动器接口1128与系统总线1108相连接。用于外置型驱动器体现的接口1124包括在通用串行总线(USB，Universal Serial Bus)及IEEE 1394接口技术中的至少一个或两个。
这些驱动器及与此相关的计算机可读介质提供数据、数据结构、计算机可执行指令、其他等的非易失性存储。在计算机1102的情况下，驱动器及介质通过适当数字形式存储任意数据。对于所述计算机可读介质的说明提及硬盘驱动器、移动式磁盘及CD或DVD等的移动式光介质，只要是本公开所属技术领域的普通技术人员，通过拉链驱动器(zip drive)、磁带、闪存卡、盒式磁带、其他等的计算机读取的其他类型的介质也在例示性运营环境中使用，任意这种介质可包含用于执行本公开的的计算机可执行指令。
包括操作系统2130、一个以上的应用程序2132、其他程序模块2134及程序数据2136在内的多个程序模块可存储于驱动器及随机存取存储器2112。操作系统、 应用、模块和/或数据的全部或其部分也高速缓存在随机存取存储器2112。本公开可体现为在多种商业方面可利用的操作系统或操作系统的组合。
用户通过一个以上的有线、无线输入装置，例如，键盘2138及鼠标1140等的指装置向计算机1102输入指令及信息。其他输入装置(未图示)为麦克风、IR遥控器、操纵杆、游戏手柄、手写笔、触摸屏、其他等等。这些及其他输入装置通过与系统总线1108相连接的输入装置接口1142与处理装置1104相连接，通过并列端口、IEEE 1394串联端口、游戏端口、USB端口、IR接口、其他等等的其他接口相连接。
显示器1144或其他类型的显示装置也通过视频适配器1146等的接口与系统总线1108相连接。向显示器1144附加，计算机通常包括扬声器、打印机、其他等其他周边输出装置(未图示)。
计算机1102使用通过有线和/或无线通信的远程计算机1148等的一个以上的远程计算机的逻辑连接在网络化的环境下工作。远程计算机1148可以为工作站、计算设备计算机、路由器、个人计算机、便携式计算机、基于微处理器的娱乐设备、同行设备或其他通常的网络节点，通常，对计算机1102，包括记述的结构要素中的多个或全部，为了简化，仅示出存储装置1150。图示的逻辑连接包括近距离通信网1152(LAN)和/或更大的网络，例如，远距离通信网1154(WAN)的有线、无线连接。这种近距离通信及远距离通信网络环境一般在事务所及公司使用，使内部网等的企业级计算机网络(enterprise-wide computer network)简单，这些与全世界计算机网络，例如，网络连接。
当在近距离通信网络环境下使用时，计算机1102通过有线和/或无线通信网络接口或适配器1156与本地网络1152相连接。适配器1156可以近距离通信1152的有线或无线通信简单，所述近距离通信1152为了与无线适配器1156进行通信而包括设置于其的无线接入点，当在远距离通信网络环境时使用时，计算机1102可包括调制解调器1158，与远距离通信1154上的通信计算设备相连接，或者通过网络等，通过远距离通信1154设定通信的其他单元。可以为内置型或外置型及有线或无线装置的调制解调器1158通过串口接口1142与系统总线1108相连接。在网络化的环境下，对计算机1102，说明的程序模块或其的一部分存储于远程存储器/存储装置1150。图示的网络连接为例示性实施例，也可使用在计算机之间设定通信连接的其他单元。
计算机1102进行通过无线通信配置并工作的任意的无线装置或个体，例如，打印机、扫描仪、台式计算机和/或便携式计算机、个人数据助力(PDA，portable data assistant)、通信卫星、可无线检测标记有关的任意装置或场所及电话进行通信的工作。这至少包括无线保真及蓝牙无线技术。因此，如以往的网络，通信可以为预先确定的结构或至少2个装置件的自组织通信(ad hoc communication)。
无线保真(Wi-Fi，Wireless Fidelity)即使没有有线，也可通过网络连接。无线保真为这种装置，例如，计算机在室内及室外，即，基站的通话圈内的任何位置可收发数据的电话的无线技术。无线保真网络具有可靠性并为了提供高速无线连接而使用IEEE 802.11(a、b、g、其他)无线技术。为了使计算机与网络及有线网络(使用IEEE 802.3或以太网络)相连接而使用无线保真。无线保真网络在未经许可的2.4及5GHz无线电频段，例如，以11Mbps(802.11a)或54Mbps(802.11b)数据率工作，或者在包括双频段的产品中工作。
本公开所属技术领域的普通技术人员知道信息及信号通过多种多样的不同技术及工法来表现。例如，可通过在所述说明中可参照的数据、指示、指令、信息、信号、比特、标志及芯片、电压、电流电磁波、磁场或粒子、光学场或粒子、或这些的任意结合体现。
本公开所属技术领域的普通技术人员知道与在此说明的实施例相关地说明的多种例示性逻辑块、模块、处理器、单元、电路及算法步骤通过电子硬盘、(为了方便，在此称之为“软件”)多种形态的程序或设计代码或这些的结合体现。为了明确说明硬件及软件的这种相互互换性，多种例示性组件、块、模块、电路及步骤与这些的功能相关地如上说明。这种功能是否作为硬件或软件体现取决于对特定应用及整体系统的设计限制。本公开所属技术领域的普通技术人员对各个特定应用体现通过多种方式说明的功能，这种体现确定并不超出本公开的范围。
在此揭示的多种实施例通过方法、装置或使用标准变成和/或功能技术的制品(article)体现。术语“制品”包括从任意计算机可读装置访问的计算机程序、载体或介质(media)。例如，计算机可读介质包括磁存储装置(例如，硬盘、软盘、磁条等)、光盘(例如，CD、DVD等)、智能卡及闪存装置(例如，EEPROM、卡、条、键驱动器等)，但并不局限于此。并且，在此揭示的多种存储介质包括用于存储信息的一个以上的装置和/或其他机械可读介质。
揭示的过程中的步骤的特定顺序或阶层结构为例示性接近的一例。基于设计有 限顺序，本公开范围内的过程的步骤的特定顺序或阶层结构可以再次排列。附加的方法发明要求保护范围按样品顺序提供多种步骤的元素，并不局限于揭示的特定顺序或阶层结构。
提供对于揭示的实施例的说明，以使本公开所属技术领域的普通技术人员简单利用或实施本公开。对于这种实施例的多种变形对本公开所属技术领域的普通技术人员来说是显而易见的，在此定义的一般原理在不超出本公开的范围的情况下适用于其他实施例。因此，本公开并不局限于在此说明的实施例，需要在与在此揭示的原理及新特征一贯的最广的范围中解释。

Claims (19)

  1. 一种计算机可读存储介质，存储有计算机程序，其特征在于，所述计算机程序在计算装置的一个以上的处理器中执行的情况下，执行用于对输入数据进行检查的以下的工作，所述工作包括：
    导出对输入数据的第一次检查结果的工作；
    通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的工作；以及
    以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的工作。
  2. 根据权利要求1所述的计算机可读存储介质，其特征在于，所述分类模型利用包含多个学习数据子集的学习数据集来学习，以在包含在所述学习数据集中的学习数据中将类似的数据分类为相同集群，且将非类似的数据分类为不同集群的方式学习所述分类模型。
  3. 根据权利要求1所述的计算机可读存储介质，其特征在于，导出对所述输入数据的第一次检查结果的工作，包括利用包含一个以上的网络函数的预先学习的检查模型来对所述输入数据进行处理，由此，对所述输入数据执行异常检测或者以所述输入数据与参考数据的比较为基础来对所述输入数据执行异常检测的工作。
  4. 根据权利要求1所述的计算机可读存储介质，其特征在于，所述分类模型利用包含学习数据子集的学习数据集来学习，所述学习数据集包含标记有不同集群信息的学习数据。
  5. 根据权利要求1所述的计算机可读存储介质，其特征在于，所述分类模型利用包含学习数据子集的学习数据集来学习，所述学习数据集包含目标数据、目标类似数据及目标非类似数据。
  6. 根据权利要求5所述的计算机可读存储介质，其特征在于，所述目标数据和所述目标类似数据为标记有第一集群信息的数据，而且，目标非类似数据为标记有第二集群信息的数据。
  7. 根据权利要求5所述的计算机可读存储介质，其特征在于，在所述目标数据为包含异常的数据的情况下，所述目标类似数据为包含与所述目标数据所包含的异常类似的类型的异常的数据，而且，所述目标非类似数据为不包含异常的数据。
  8. 根据权利要求5所述的计算机可读存储介质，其特征在于，在所述目标数据为包含异常的图像的情况下，所述目标类似数据为以包含所述目标数据所包含的异常的至少一部分的方式裁剪的图像，而且，所述目标非类似数据为不包含异常的图像。
  9. 根据权利要求8所述的计算机可读存储介质，其特征在于，所述目标数据为在图像的中心部包含所述异常的图像。
  10. 根据权利要求8所述的计算机可读存储介质，其特征在于，所述目标非类似数据为除所述目标数据的异常之外的部分的至少一部分重复的图像。
  11. 根据权利要求5所述的计算机可读存储介质，其特征在于，以将所述目标数据和所述目标类似数据分类为相同集群，且将所述目标非类似数据分类为与所述目标数据及所述目标类似数据所属的集群不相同的集群的方式学习所述分类模型。
  12. 根据权利要求5所述的计算机可读存储介质，其特征在于，所述目标数据包含与能够与对象客体分离来发生的异常状况有关的图像，所述目标类似数据包含具有所述目标数据的至少一部分的图像。
  13. 根据权利要求5所述的计算机可读存储介质，其特征在于，所述目标数据及所述目标类似数据为包含误报的数据。
  14. 根据权利要求1所述的计算机可读存储介质，其特征在于，以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的工作包括在所述分类模型将所述输入数据分类为属于特定集群的情况下，将与所述特定集群匹配的检查结果生成为第二次检查结果，并以第二次检查结果为基础来输出最终检查结果，而且，在对所述输入数据的分类失败的情况下，将所述第一次检查结果作为最终检查结果来输出的工作。
  15. 根据权利要求14所述的计算机可读存储介质，其特征在于，所述第二次检查结果包含与是否存在异常、异常的位置、异常的类型及误检类型中的至少一种有关的信息。
  16. 根据权利要求1所述的计算机可读存储介质，其特征在于，通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的工作包括：
    利用所述预先学习的分类模型来对所述输入数据进行处理，从而将所述输入数据的特征映射到所述预先学习的分类模型的解空间的工作；以及
    以所述解空间上的所述输入数据的位置为基础，并基于所述输入数据是否属于 所述解空间上的一个以上的集群中的一个集群来分类所述输入数据的工作。
  17. 根据权利要求16所述的计算机可读存储介质，其特征在于，所述解空间由一维以上的空间构成，包含一个以上的集群，而且，各个集群以基于各个目标数据的特征及以基于目标类似数据的特征在解空间上的位置为基础构成。
  18. 一种输入数据检查方法，在包括一个以上的处理器的计算装置中执行，其特征在于，包括：
    导出对输入数据的第一次检查结果的步骤；
    通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类的步骤；以及
    以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果的步骤。
  19. 一种计算装置，其特征在于，包括：
    一个以上的处理器；以及
    存储器，用于存储能够在所述处理器中执行的指令，
    所述处理器执行如下工作：
    导出对输入数据的第一次检查结果；
    通过利用标记有集群信息的学习数据学习的分类模型来对所述输入数据进行分类；以及
    以对所述分类模型的所述输入数据的分类结果的至少一部分为基础来输出最终检查结果。
PCT/CN2019/102013 2018-12-13 2019-08-22 计算机可读存储介质、输入数据检查方法以及计算装置 WO2020119169A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201811528293.5A CN111325224A (zh) 2018-12-13 2018-12-13 计算机可读存储介质、输入数据检查方法以及计算装置
CN201811528293.5 2018-12-13
KR1020190002307A KR102246085B1 (ko) 2018-12-13 2019-01-08 어노말리 디텍션
KR10-2019-0002307 2019-01-08

Publications (1)

Publication Number Publication Date
WO2020119169A1 true WO2020119169A1 (zh) 2020-06-18

Family

ID=71075937

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/102013 WO2020119169A1 (zh) 2018-12-13 2019-08-22 计算机可读存储介质、输入数据检查方法以及计算装置

Country Status (1)

Country Link
WO (1) WO2020119169A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110295778A1 (en) * 2010-05-27 2011-12-01 Sony Corporation Information processing apparatus, information processing method, and program
CN106294762A (zh) * 2016-08-11 2017-01-04 齐鲁工业大学 一种基于学习的实体识别方法
CN107884475A (zh) * 2017-10-18 2018-04-06 常州大学 一种基于深度学习神经网络的城市燃气管道故障诊断方法
CN108027911A (zh) * 2015-07-29 2018-05-11 伊利诺斯工具制品有限公司 促进焊接即服务软件的系统及方法
CN108985214A (zh) * 2018-07-09 2018-12-11 上海斐讯数据通信技术有限公司 图像数据的标注方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110295778A1 (en) * 2010-05-27 2011-12-01 Sony Corporation Information processing apparatus, information processing method, and program
CN108027911A (zh) * 2015-07-29 2018-05-11 伊利诺斯工具制品有限公司 促进焊接即服务软件的系统及方法
CN106294762A (zh) * 2016-08-11 2017-01-04 齐鲁工业大学 一种基于学习的实体识别方法
CN107884475A (zh) * 2017-10-18 2018-04-06 常州大学 一种基于深度学习神经网络的城市燃气管道故障诊断方法
CN108985214A (zh) * 2018-07-09 2018-12-11 上海斐讯数据通信技术有限公司 图像数据的标注方法和装置

Similar Documents

Publication Publication Date Title
JP7112501B2 (ja) ディープラーニングに基づく画像比較装置と方法及びコンピューター可読媒体に保存されたコンピュータープログラム
KR102246085B1 (ko) 어노말리 디텍션
CN107430705B (zh) 用于重新训练分类器的样本选择
US10275719B2 (en) Hyper-parameter selection for deep convolutional networks
KR20200091808A (ko) 어노말리 디텍션
US11443514B2 (en) Recognizing minutes-long activities in videos
US11769060B2 (en) Method for determining optimal anomaly detection model for processing input data
KR20200000541A (ko) 인공 신경망의 학습 방법
KR102078133B1 (ko) 데이터의 어노말리 감지 방법
US20220269718A1 (en) Method And Apparatus For Tracking Object
KR102402194B1 (ko) 딥러닝 기반 엔드-투-엔드 o-ring 결함 검출 방법
KR20190041961A (ko) 딥러닝 기반 이미지 비교 장치, 방법 및 컴퓨터 판독가능매체에 저장된 컴퓨터 프로그램
US11625574B2 (en) Method for generating abnormal data
KR20200010984A (ko) 어노말리 디텍션
KR20220112164A (ko) 목표 물체 검출 방법 및 장치
KR20210098420A (ko) 비정상 데이터 생성 방법
KR20220105102A (ko) 레이더 영상 처리 방법 및 장치
WO2020119169A1 (zh) 计算机可读存储介质、输入数据检查方法以及计算装置
KR20200019919A (ko) 데이터의 어노말리 감지 방법
KR102070730B1 (ko) 이미지 세그먼테이션 방법
KR102260169B1 (ko) 데이터 생성 방법
KR102283885B1 (ko) 이미지 정합 방법
KR102718267B1 (ko) 웨이퍼맵의 패턴에 기반한 불량 장비 검출 방법, 장치 및 프로그램
KR102588998B1 (ko) 영상에서 이상행동 검출을 위한 딥러닝 모델을 이용하여 태스크를 수행하는 방법 및 장치
KR20230154601A (ko) 표의 픽셀 정보를 획득하는 방법 및 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19894961

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19894961

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 03.08.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19894961

Country of ref document: EP

Kind code of ref document: A1