CN114638809A - Multi-scale micro-defect detection method based on PA-MLFPN workpiece surface - Google Patents

Abstract

The application discloses a multi-scale micro-defect detection method based on a PA-MLFPN workpiece surface, which comprises the following steps: and (3) constructing a PA-MLFPN feature extraction model. A VGG16 main feature extraction network model is used as a first model, PA-MLFPN is used as a middle model, and a prediction layer is used as a later model to form a detection model. And training a detection model by taking the surface image of the multi-scale micro-defect target workpiece as input and the defect positioning frame and classification as output. PA-MLFPN uses the hole convolution of different scaled rates to replace the traditional convolution in the TUM coder part at first, and uses the average pooling operation to carry out down-sampling; adding a bottom-up characteristic enhancement path on the basis of the original TUM to interpolate shallow characteristics into a deep layer; introducing an ECA module to carry out weight distribution on a channel at the second stage of the SFAM; the invention can detect the workpiece surface defect target with variable and smaller dimensions.

Description

Multi-scale micro-defect detection method on workpiece surface based on PA-MLFPN

technical field

The invention relates to a multi-scale small defect detection method based on a PA-MLFPN workpiece surface, and belongs to the technical field of computer vision.

Background technique

The task of workpiece surface defect detection is an indispensable link in workpiece production. However, due to the influence of workpiece production equipment and production technology, the surface of the produced workpiece often has various types of defects such as scratches, bumps, and abrasions. These defects will not only affect the appearance, but also bury many hidden dangers in the subsequent use of the workpiece, such as accelerating the aging of the workpiece and affecting the service life, and even affecting the normal use of the workpiece. Therefore, it is particularly important to use suitable defect detection technology to improve the production quality of workpieces. Traditional surface defect detection tasks are usually done manually, but manual detection is often affected by factors such as personal subjective experience, work environment, and mental state, which lacks normativeness. At the same time, false detection and missed detection are easy to occur when the manual method is used to detect high-speed movement or small defects.

With the development of deep learning, more and more target detection algorithms based on convolutional neural networks have been proposed. However, due to the structure of the convolutional network and the characteristics of the surface defects of the workpiece, we believe that there are still problems in the task of detecting the surface defects of the workpiece based on the convolutional neural network: (1) The randomness of the surface defects of the workpiece leads to the change of the defect scale. For example, two different types of defect targets may have similar sizes, but the complexity of their performance may be very different, or in the same defect type, their scale changes will also be very different. However, in the traditional convolutional network model, the feature layer used to detect objects in a specific size range is mainly composed of a single-level feature layer or two adjacent layers for final detection, which will result in poor detection performance. (2) In the actual workpiece surface defect detection task, the defects often have many small defect targets on the basis of changing shapes. In the convolutional neural network, the shallow feature layer has higher resolution and contains more location and detail information that is conducive to small target detection. Deep features have stronger semantic information, but have low resolution and poor perception of small objects.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a multi-scale micro-defect detection method based on the PA-MLFPN workpiece surface, so as to solve the problems in the prior art. The Multi-Level Feature Pyramid Network (MLFPN) is selected to solve the multi-scale defect target problem by extracting multi-level and multi-scale features. On the basis of MLFPN, an improved method is proposed for the task of workpiece surface defect detection, and a Path-Augmentation Multi-Level Feature Pyramid Network (PA-MLFPN) is proposed to further utilize shallow features and embed them. The workpiece surface defect detection is carried out in SSD to improve the detection accuracy of multi-scale small defect targets.

In order to achieve the above object, the present invention provides the following technical solutions: a multi-scale micro-defect detection method based on the PA-MLFPN workpiece surface, characterized in that the following steps A to D are performed to obtain a workpiece surface defect detection model, and then based on the VGG16 backbone feature extraction The sample images of tiny defects of the target workpiece detected by the network are applied, and the surface defect detection model of the workpiece is applied to obtain the preset defect classifications corresponding to the multi-scale tiny defects in the image of the target workpiece;

Step A: Based on the MLFPN model, take the small defect sample image as the input and the multi-layer feature pyramid as the output to construct the PA-MLFPN feature extraction model

Step B: Based on the Prediction layer, the multi-layer feature pyramid image of the micro-defect image is used as the input, and the preset defect classification corresponding to the multi-scale micro-defects in the image is used as the output to construct a classification model;

Step C: The PA-MLFPN feature extraction model is first, the classification model is connected sequentially, and the input end of the classification model is connected to the output end of the PA-MLFPN feature extraction model to form an image of multi-scale tiny defects on the surface of the workpiece at a single defect position. Input, take the preset defect classification corresponding to the multi-scale tiny defects in the image as the output detection model;

Step D: Based on the preset number of multi-scale micro-defect sample images on the workpiece surface each containing a single defect position, and the preset defect classification corresponding to each multi-scale micro-defect sample image on the workpiece surface, the micro-defect sample image is used as input, The preset defect classification corresponding to the multi-scale tiny defects in the image is used as the output, and the detection model is trained to obtain the workpiece surface defect detection model.

Further, the preset defects included in the aforementioned multi-scale micro-defects on the workpiece surface include scratch defects, convex powder defects, and scratch defects.

Further, the aforementioned method for detecting multi-scale micro-defects on workpiece surfaces based on PA-MLFPN further includes obtaining images of multi-scale micro-defects on the workpiece surface each including a single defect position, and performing the multi-scale micro-defect image data set on the workpiece surface according to a preset ratio. The data is divided into training set, test set and verification set, specifically: on the workpiece production line, use a sampling device to take pictures and sample each workpiece at a fixed position; Defects are marked, and the images marked on the surface by Labelimg are made into VOC format data sets; data enhancement is performed on the images in the data set through imgaug data enhancement; the enhanced data set is (training set + validation set): test set The ratio is 9:1, and the ratio of training set:validation set is 9:1 for random division.

Further, in the aforementioned step A, the PA-MLFPN model includes a FFMv1 feature fusion module, a basic feature layer, a multi-level FFMv2 feature fusion module, a multi-level improved TUM refinement U-shaped module, and an improved SFAM scale-based Feature aggregation module, the FFMv1 feature fusion module is used as the input end of the PA-MLFPN model, the output end of the FFMv1 feature fusion module is connected with the input end of the basic feature layer, and the output end of the basic feature layer is connected with each FFMv2 The input end of the feature fusion module and the input end of the first-level improved TUM refinement U-shaped module are respectively connected, and the multi-level FFMv2 feature fusion module and the multi-level improved TUM refinement U-shaped module are alternately stacked and connected. The output end of the improved TUM refinement U-shaped module of the advanced level is connected to the input end of the improved SFAM scale-based feature aggregation module, and the output end of the improved SFAM scale-based feature aggregation module is used as the output end of the PA-MLFPN model ; The described step of training the PA-MLFPN model includes the following steps 101 to 104:

Step 101: Based on the FFMv1 feature fusion module, take conv4_3 and conv5_3 as input, and take the multi-scale micro-defect basic feature layer on the surface of multiple workpieces as the output, then enter step 102;

Step 102: take the multi-scale micro-defect basic feature layer on the workpiece surface as an input, input to the first-level TUM model and carry out further feature extraction, take the six-scale micro-defect basic feature maps of different levels of output as output, then enter step 103;

Step 103: the maximum feature map in the six-scale basic feature map obtained by the basic feature layer and the first-level TUM model is an input, input to the first-level FFMv2 feature fusion module, and obtain the first-level fusion feature map; The feature map and basic feature layer map of the first-level fusion are input to the second-level FFMv2 model, and the second-level fusion feature map is obtained, and the multi-level improved TUM refinement U-shaped model and the FFMv2 feature fusion module are stacked alternately. The former-level fusion feature map and the basic feature layer map are used as the input, and the latter-level fusion feature map is used as the output to obtain the shallow-level feature pyramid feature map, the middle-level feature pyramid feature map and the deep-level feature pyramid feature map, and then enter step 104;

Step 104: carry out feature aggregation based on the SFAM module, take the pyramid feature map with the same size in the shallow feature pyramid feature map, the middle feature pyramid feature map, and the deep feature pyramid feature map as the input, and take the multi-layer feature pyramid map as the output, then The ECA module is used to assign the weights on the channels to the obtained multi-layer feature pyramid.

Further, the aforementioned TUM refinement U-shaped model adopts 3×3 hole convolution with dilated rate set to (2, 2, 3, 3, 1) and stride 1 in the encoder part for feature learning, and The average pooling operation with a step size of 2 is used for downsampling, and an information transmission path from the shallow layer to the deep layer is added on the basis of the original U-shaped structure to interpolate the detailed position information of the shallow layer feature pyramid feature to the deep layer, namely Ti After convolution with a kernel size of 3×3 and a stride of 2, the feature map size is reduced to half of the original size, namely Conv(T _i ), and then added element by element with C _i+1 , and the obtained result is then A convolution with a kernel size of 3×3 and a stride of 1 obtains T _i+1 , and the calculation formula is:

Further, the aforementioned SFAM uses the ECA module to assign the weights on the channel to the obtained multi-layer feature pyramid, and the specific steps are:

Step Ⅴ-1: Perform global average pooling on the feature layers of the multi-layer feature pyramid graph initially aggregated by the SFAM module to obtain the weights of C channels, and then enter step Ⅴ-2;

Step Ⅴ-2: The weight coefficients of C channels are obtained by implementing local cross-channel mutual information through one-dimensional convolution with a kernel size of k, where the kernel size k is adaptively determined by a function of the number of channels C, and the calculation formula is:

Among them, |t| _odd means to take the nearest odd number, and b and γ take 1 and 2 respectively. Finally, the weight coefficients of the C channels are passed through the Sigmoid function to obtain C values between 0 and 1, which correspond to the weights of the original channels respectively. C, and multiply it with the multi-level feature pyramid map for weighting and then output to obtain the multi-level feature pyramid feature map.

Further, in the aforementioned step A, it also includes performing fine-tuning training on the obtained PA-MLFPN model, specifically:

During training, Focal loss is used to calculate the classification loss, and Smooth L1 is used to calculate the regression loss. The final loss function used is the combination of Focal loss and Smooth L1:

L=L _fl +L _SL1

其中y′是预测输出，y是真实样本的标签，α是正负样本权重、γ是易分类样本和难分类样本权重；所述回归损失Smooth L1计算公式如下:

其中x为预测框与真实框的差值。The classification loss Focal loss calculation formula is:

Where y' is the predicted output, y is the label of the real sample, α is the weight of positive and negative samples, γ is the weight of easy-to-classify samples and hard-to-classify samples; the regression loss Smooth L1 is calculated as follows:

where x is the difference between the predicted box and the ground-truth box.

Further, the aforementioned defect classification results that meet the preset conditions are obtained by verifying the actual samples, specifically: setting the evaluation index precision rate P and recall rate R for model evaluation, and the calculation formula is:

Among them, TP means the number of correctly judged defective areas, TN means the number of correctly judged background areas, FP means the number of correct background areas misjudged as defective areas, FN means the number of faults misjudged as background;

The average precision AP is set to evaluate the detection performance of the model for various types of defects on the test set. The area enclosed by the PR curve and the abscissa and ordinate axes is the AP of this type of defects. The calculation formula is as follows:

The detection results of multi-category defects are set to use the average precision value mAP, and the calculation formula of mAP of the test results is as follows:

Further, the aforementioned multi-scale micro-defect detection method based on the PA-MLFPN workpiece surface is used to verify the preset defect classification corresponding to the micro-defect image, and the number of loop iteration steps is set to 100. First, the batch size is set to 32, and the learning rate is set to 32. The initialization is 5e-4. When the number of iteration steps reaches 50, the batch size is reset to 16 and the learning rate is 1e-4.

Further, for the aforementioned verification of actual samples to obtain defect classification results that meet the preset conditions, the early stopping method is used during training, and the verification loss is calculated in each iteration. When the verification loss value reaches the local optimum, iteration is continued for 6 times. , and stop training if the model no longer converges.

Compared with the prior art, the invention has the following technical effects based on the multi-scale micro-defect detection method on the workpiece surface based on PA-MLFPN. The traditional target detection algorithm cannot Well done detection. A workpiece surface defect detection algorithm based on Path Enhanced Multilevel Feature Pyramid Network (PA-MLFPN) is proposed. On the basis of extracting multi-level and multi-scale defect features, the representation ability of small defect targets is further enhanced, the detection accuracy of different scale targets is improved, and the problem of missed detection of small defect targets is effectively solved.

Description of drawings

Fig. 1 is the flow chart of the present invention;

Figure 2 is the network model diagram after PA-MLFPN is embedded in the detection model;

Figure 3 is a schematic diagram of the use of hole convolution in the downsampling part;

4 is a schematic diagram of a feature combination mode of a feature enhancement path;

Fig. 5 is the improved TUM structure diagram;

Figure 6 is an improved SFAM structure diagram;

FIG. 7 is a display diagram of the detection results of workpiece surface defects.

Detailed ways

In order to better understand the technical content of the present invention, specific embodiments are given and described below in conjunction with the accompanying drawings.

As shown in FIG. 1 , the process steps of the present invention are as follows: The first step is to collect images of surface defects of the workpiece.

The second step is to perform defect annotation and data enhancement to construct a workpiece surface defect data set. Specifically, on the workpiece production line, use a sampling device to take pictures and sample each workpiece at a fixed position to construct workpiece surface defect data. set. The surface defects of the collected workpiece surface images are marked by labelimg. The data set of workpiece surface defects is divided according to the preset ratio, and divided into training set, test set and verification set, and the images marked on the surface by Labelimg are made into a data set in VOC format; the data set is enhanced by imgaug data. Image data enhancement; the enhanced data set is randomly divided with the ratio of (training set + validation set):test set 9:1, and the ratio of training set:validation set is 9:1.

The third step is the construction of PA-MLFPN; based on the MLFPN model, the PA-MLFPN feature extraction model is constructed with the micro-defect image as the input and the multi-layer feature pyramid image of the micro-defect image as the output;

The fourth step, based on the Prediction layer, takes the multi-layer feature pyramid image of the micro-defect image as the input, and uses the preset defect classification corresponding to the multi-scale micro-defects in the image as the output to construct a classification model;

The fifth step, based on the small defect sample image of the detected target workpiece obtained by the VGG16 backbone feature extraction network, the PA-MLFPN feature extraction model is first, the classification model is connected sequentially, and the input end of the classification model is connected to the PA-MLFPN feature extraction model. The output end constitutes a detection model that takes the image of the multi-scale micro-defects on the workpiece surface at a single defect position as the input, and uses the preset defect classification corresponding to the multi-scale micro-defects in the image as the output;

In the sixth step, based on the preset number of multi-scale micro-defect sample images on the workpiece surface each containing a single defect position, and the preset defect classification corresponding to each multi-scale micro-defect sample image on the workpiece surface, the micro-defect sample image is used as the input. , take the preset defect classification corresponding to the multi-scale tiny defects in the image as the output, and train the detection model. Obtain the workpiece surface defect detection model.

As shown in Figure 2, based on the VGG16 network model, the defect image is used as the input, and the conv4_3 and conv5_3 feature layers in VGG16 are used as the output to construct the backbone feature extraction network. The network model is formed in series in the order of the PA-MLFPN first and the Prediction layer prediction layer; the PA-MLFPN model includes the FFMv1 feature fusion module, the basic feature layer, the multi-level FFMv2 feature fusion module, and the multi-level improved TUM detail. The U-shaped module and the improved SFAM scale-based feature aggregation module, through FFMv1, the initial feature fusion of conv4_3 and conv5_3 is used to obtain the base feature layer (Basefeature) for the next series of further feature fusions.

The FFMv1 feature fusion module is used as the input end of the PA-MLFPN model, the output end of the FFMv1 feature fusion module is connected to the input end of the basic feature layer, and the output end of the basic feature layer is connected with each FFMv2 feature fusion module. The input end and the input end of the first-level improved TUM refinement U-shaped module are respectively connected, the multi-level FFMv2 feature fusion module and the multi-level improved TUM refinement U-shaped module are alternately stacked and connected. The output end of the TUM refinement U-shaped module is connected to the input end of the improved SFAM scale-based feature aggregation module, and the output end of the improved SFAM scale-based feature aggregation module is used as the output end of the PA-MLFPN model; The steps of training the PA-MLFPN model include the following steps 101 to 104:

Step 101: Based on the FFMv1 feature fusion module, take conv4_3 and conv5_3 as input, and take the multi-scale micro-defect basic feature layer on the surface of multiple workpieces as the output, then enter step 102;

Step 102: take the multi-scale micro-defect basic feature layer on the workpiece surface as an input, input to the first-level TUM model and carry out further feature extraction, take the six-scale micro-defect basic feature maps of different levels of output as output, then enter step 103;

Step 103: specifically: the maximum feature map in the six-scale basic feature map obtained by the basic feature layer and the first-level TUM model is an input, input to the first-level FFMv2 feature fusion module, and obtain the first-level fusion feature map; Then, the first-level fusion feature map and basic feature layer map are used as input, and input to the second-level FFMv2 model to obtain the second-level fusion feature map, and the multi-level improved TUM refinement U-shaped model and FFMv2 feature fusion are alternately stacked. The module method takes the former-level fusion feature map and the basic feature layer map as input, and the latter-level fusion feature map as the output, and obtains the shallow-level feature pyramid feature map, the middle-level feature pyramid feature map and the deep-level feature pyramid feature map, and then goes to step 104 ;

Step 104: carry out feature aggregation based on the SFAM module, take the pyramid feature map with the same size in the shallow feature pyramid feature map, the middle feature pyramid feature map, and the deep feature pyramid feature map as the input, and use the multi-layer feature pyramid map as the output, then The ECA module is used to assign the weights on the channels to the obtained multi-layer feature pyramid.

Then, the multi-layer feature pyramid image of the micro-defect image is used as the input, and the defect classification corresponding to the multi-scale micro-defects in the image is used as the output, and the prediction layer is input to the prediction layer to obtain the defect classification corresponding to the micro-defects on the workpiece surface.

As shown in Figure 3, the downsampling part uses a schematic diagram of hole convolution, and the TUM refinement U-shaped model uses a dilated rate in the encoder part, which is set to (2, 2, 3, 3, 1) respectively. The step size is 3 × 1 3-hole convolution is used for feature learning, and an average pooling operation with stride 2 is used for downsampling.

As shown in Figure 4, the feature combination method of the feature enhancement path is specifically, on the basis of the original U-shaped structure, an information transmission path from the shallow layer to the deep layer is added to interpolate the detailed position information of the shallow feature pyramid feature to the deep layer, that is, Ti undergoes convolution with a convolution kernel size of 3 × 3 and a stride of 2, and the feature map size is reduced to half of the original size, namely Conv(Ti), and then added element by element with Ci+1, and the obtained result is then passed through a The size of the convolution kernel is 3×3, and the convolution with a stride of 1 obtains Ti+1. The calculation formula is:

As shown in Figure 5, the improved TUM structure, the TUM refined U-shaped model uses a 3×3 hole volume with dilated rate set to (2, 2, 3, 3, 1) with a step size of 1 in the encoder part. The product is used for feature learning, and the average pooling operation with stride 2 is used for downsampling. By setting different dilated rates to obtain receptive fields of different sizes, multi-scale information is obtained, and at the same time, information loss in the process of continuous convolution of downsampling is avoided to retain more shallow defect target information. Further, a 1×1 convolution with a stride of 1 and a channel number of 128 is used to adjust the number of channels and output.

As shown in Figure 6, the improved SFAM structure diagram; the specific process of using the ECA module in the second stage of the improved SFAM is as follows: the ECA module is introduced to perform a one-dimensional convolution of size k to replace the two fully connected layers, where k passes through The function of the number of channels C is adaptively determined. This not only avoids the dimensionality reduction operation, but also conducts local cross-channel information interaction through one-dimensional convolution of adaptive size. Specifically, for the feature layer initially aggregated in the first stage of SFAM, the global average pooling is first performed to obtain the weight of each channel. If there are C channels, a total of C channel weights are obtained.

Then, the weights are obtained by implementing local cross-channel mutual information through a one-dimensional convolution of size k, in which the kernel size k needs to be determined adaptively:

其中，|t|_odd表示取最近的奇数，且b与γ在实验中分别取1和2。最后将上一步的输出通过Sigmoid函数得到C个0到1之间的值，分别对应原始通道的权重，并将其与输入特征相乘来进行加权然后输出。

Among them, |t| _odd means to take the nearest odd number, and b and γ are taken as 1 and 2 respectively in the experiment. Finally, the output of the previous step is passed through the Sigmoid function to obtain C values between 0 and 1, which correspond to the weights of the original channels, and are multiplied by the input features for weighting and then output.

The fine-tuning training of the constructed model on the workpiece surface defect dataset is as follows: during training, Focal loss is used to calculate the classification loss, and Smooth L1 is used to calculate the regression loss. The loss function finally adopted is the combination of Focalloss and Smooth L1: L=L _fl +L _SL1 , the calculation formula of the classification loss Focal loss is:

其中y是预测输出，y是真实样本的标签，α是正负样本权重、γ是易分类样本和难分类样本权重。回归损失Smooth L1计算公式如下:

where y is the predicted output, y is the label of the real sample, α is the weight of positive and negative samples, and γ is the weight of easy-to-classify samples and hard-to-classify samples. The regression loss Smooth L1 calculation formula is as follows:

其中x为预测框与真实框的差值。

where x is the difference between the predicted box and the ground-truth box.

In order to evaluate the performance of the algorithm proposed in this paper, the relevant evaluation indicators precision (Precision, P) and recall (Recall, R) are used for model evaluation.

Among them, TP, TN and FP respectively represent the number of correctly judged defects, the number of correctly judged background regions and the number of misjudged backgrounds as defects:

平均精度(Average Precision,AP)用来评价模型在测试集上对各类型缺陷的检测性能，如下式所示，PR曲线与横纵坐标轴围成的面积为该类缺陷的AP:

Average Precision (AP) is used to evaluate the detection performance of the model for various types of defects on the test set. As shown in the following formula, the area enclosed by the PR curve and the horizontal and vertical axes is the AP of this type of defect:

The detection result of the multi-category defect adopts the mean average precision (mean Average Precision, mAP) to evaluate, and the test result mAP of the present invention is as shown in the formula:

The transfer learning method is used to train the network, and the weight files are obtained by pre-training on the VOC dataset, and then fine-tuned on the workpiece surface defect dataset.

The number of loop iteration steps is set to 100. First, the batch size is set to 32, and the learning rate is initialized to 5e-4. When the number of iteration steps reaches 50, the batch size is reset to 16 and the learning rate is 1e-4. And the early stopping method is used during training to avoid overfitting caused by continued training. The validation loss is calculated in each iteration. When the validation loss value reaches the local optimum, iterates for 6 times. If the model no longer converges just stop training.

Fig. 7 shows the detection results of workpiece surface defects obtained by inputting a small defect image, specifically scratch defects, convex powder defects, and scratch defects.

Aspects of the invention are described herein with reference to the accompanying drawings, in which a number of illustrative embodiments are shown. Embodiments of the present invention are not limited to those shown in the accompanying drawings. It should be understood that the present invention can be implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail, since the concepts and embodiments disclosed in the present invention are not limited to any embodiment. . Additionally, some aspects of the present disclosure may be used alone or in any suitable combination with other aspects of the present disclosure.

Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art to which the present invention pertains can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be determined according to the claims.

Compared with the prior art, the invention has the following technical effects based on the multi-scale micro-defect detection method on the workpiece surface based on PA-MLFPN. The traditional target detection algorithm cannot Well done detection. A workpiece surface defect detection algorithm based on Path Enhanced Multilevel Feature Pyramid Network (PA-MLFPN) is proposed. On the basis of extracting multi-level and multi-scale defect features, the representation ability of small defect targets is further enhanced, the detection accuracy of different scale targets is improved, and the problem of missed detection of small defect targets is effectively solved.

Claims (10)

1. Based on the PA-MLFPN workpiece surface multi-scale micro-defect detection method, it is characterized in that the following steps A to D are performed to obtain a workpiece surface defect detection model, and then based on the VGG16 backbone feature extraction network obtained detection target workpiece micro-defect samples image, applying the workpiece surface defect detection model to obtain the preset defect classification corresponding to the multi-scale tiny defects in the target workpiece image; Step A: Based on the MLFPN model, take the small defect sample image as the input and the multi-layer feature pyramid as the output to construct the PA-MLFPN feature extraction model Step B: Based on the Prediction layer, the multi-layer feature pyramid image of the micro-defect image is used as the input, and the preset defect classification corresponding to the multi-scale micro-defects in the image is used as the output to construct a classification model; Step C: The PA-MLFPN feature extraction model is first, the classification model is connected sequentially, and the input end of the classification model is connected to the output end of the PA-MLFPN feature extraction model to form an image of multi-scale tiny defects on the surface of the workpiece at a single defect position. Input, take the preset defect classification corresponding to the multi-scale tiny defects in the image as the output detection model; Step D: Based on the preset number of multi-scale micro-defect sample images on the workpiece surface each containing a single defect position, and the preset defect classification corresponding to each multi-scale micro-defect sample image on the workpiece surface, the micro-defect sample image is used as input, The preset defect classification corresponding to the multi-scale tiny defects in the image is used as the output, and the detection model is trained to obtain the workpiece surface defect detection model. 2 . The method for detecting multi-scale micro-defects on the workpiece surface based on PA-MLFPN according to claim 1 , wherein the preset defects contained in the multi-scale micro-defects on the workpiece surface include scratch defects, convex powder defects, and scratch defects. 3 . 3. The multi-scale micro-defect detection method based on the PA-MLFPN workpiece surface according to claim 1, characterized in that, further comprising obtaining a multi-scale micro-defect image on the workpiece surface each comprising a single defect position, and the multi-scale micro-defect image on the workpiece surface is obtained. The defect image data set is divided into a training set, a test set and a verification set according to a preset ratio. Specifically, on the workpiece production line, a sampling device is used to take pictures and samples of each workpiece at a fixed position; The obtained workpiece surface image is marked with surface defects by labelimg, and the surface-marked image by Labelimg is made into a VOC format data set; the data in the data set is enhanced by imgaug data enhancement; the enhanced data set is ( training set + validation set): the ratio of test set is 9:1, and the ratio of training set:validation set is 9:1 for random division. 4. The multi-scale tiny defect detection method based on the PA-MLFPN workpiece surface according to claim 1, is characterized in that, in described step A, described PA-MLFPN model comprises FFMv1 feature fusion module, basic feature layer, multi-level FFMv2 feature fusion module, multi-level improved TUM refinement U-shaped module, and improved SFAM scale-based feature aggregation module, the FFMv1 feature fusion module is used as the input of the PA-MLFPN model, and the FFMv1 feature fusion module The output end is connected with the input end of the basic feature layer, and the output end of the basic feature layer is connected with the input end of each FFMv2 feature fusion module and the input end of the first-level improved TUM refinement U-shaped module, respectively. The multi-level FFMv2 feature fusion module and the multi-level improved TUM refinement U-shaped module are alternately stacked and connected, and the output of the last stage of the improved TUM refinement U-shaped module is connected to the input of the improved SFAM scale-based feature aggregation module. , the output of the improved SFAM scale-based feature aggregation module is used as the output of the PA-MLFPN model; the step of training the PA-MLFPN model includes the following steps 101 to 104: Step 101: Based on the FFMv1 feature fusion module, take conv4_3 and conv5_3 as input, and take the multi-scale micro-defect basic feature layer on the surface of multiple workpieces as the output, then enter step 102; Step 102: take the multi-scale micro-defect basic feature layer on the workpiece surface as an input, input to the first-level TUM model and carry out further feature extraction, take the six-scale micro-defect basic feature maps of different levels of output as output, then enter step 103; Step 103: the maximum feature map in the six-scale basic feature map obtained by the basic feature layer and the first-level TUM model is an input, input to the first-level FFMv2 feature fusion module, and obtain the first-level fusion feature map; The feature map and basic feature layer map of the first-level fusion are input to the second-level FFMv2 model, and the second-level fusion feature map is obtained, and the multi-level improved TUM refinement U-shaped model and the FFMv2 feature fusion module are stacked alternately. The former-level fusion feature map and the basic feature layer map are used as the input, and the latter-level fusion feature map is used as the output to obtain the shallow-level feature pyramid feature map, the middle-level feature pyramid feature map and the deep-level feature pyramid feature map, and then enter step 104; Step 104: carry out feature aggregation based on the SFAM module, take the pyramid feature map with the same size in the shallow feature pyramid feature map, the middle feature pyramid feature map, and the deep feature pyramid feature map as the input, and take the multi-layer feature pyramid map as the output, then The ECA module is used to assign the weights on the channels to the obtained multi-layer feature pyramid. 5. based on the PA-MLFPN workpiece surface multi-scale micro-defect detection method according to claim 3, it is characterized in that, described TUM refinement U-shaped model adopts dilated rate to be respectively set to (2,2,3 in encoder part) , 3, 1) with a stride of 1 for 3×3 atrous convolutions for feature learning, and an average pooling operation with a stride of 2 for downsampling, and on the basis of the original U-shaped structure, a shallow The layer-to-deep information transmission path interpolates the detailed position information of the shallow feature pyramid feature to the deep layer, that is, Ti undergoes convolution with a convolution kernel size of 3 × 3 and a stride of 2, and the feature map size is reduced to half of the original size. Conv(T _i ), and then add C _i+1 element by element, and the result obtained is obtained through a convolution with a convolution kernel size of 3×3 and a stride of 1 to obtain T _i+1 , and the calculation formula is:

. 6. the multi-scale micro-defect detection method based on the PA-MLFPN workpiece surface according to claim 3, is characterized in that, the concrete steps of using ECA module to carry out the distribution of weights on the channel to the multi-layer feature pyramid diagram obtained based on SFAM, for: Step Ⅴ-1: Perform global average pooling on the feature layers of the multi-layer feature pyramid graph initially aggregated by the SFAM module to obtain the weights of C channels, and then enter step Ⅴ-2; Step Ⅴ-2: The weight coefficients of C channels are obtained by implementing local cross-channel mutual information through a one-dimensional convolution with a kernel size of k, where the kernel size k is adaptively determined by the function of the number of channels C, and the calculation formula is:

Among them, |t| _odd means to take the nearest odd number, and b and γ take 1 and 2 respectively. Finally, the weight coefficients of the C channels are passed through the Sigmoid function to obtain C values between 0 and 1, which correspond to the weights of the original channels respectively. C, and multiply it with the multi-level feature pyramid map for weighting and then output to obtain the multi-level feature pyramid feature map. 7. according to claim 1 based on the PA-MLFPN workpiece surface multi-scale tiny defect detection method, it is characterized in that, in step A, also comprise to the PA-MLFPN model that obtains, carry out fine-tuning training, be specially: During training, Focal loss is used to calculate the classification loss, and Smooth L1 is used to calculate the regression loss. The final loss function used is the combination of Focal loss and Smooth L1: L=L _fl +L _SL1 The classification loss Focal loss calculation formula is:

Where y' is the predicted output, y is the label of the real sample, α is the weight of positive and negative samples, γ is the weight of easy-to-classify samples and hard-to-classify samples; the regression loss Smooth L1 is calculated as follows:

where x is the difference between the predicted box and the ground-truth box. 8. The multi-scale tiny defect detection method based on PA-MLFPN workpiece surface according to claim 1, is characterized in that, obtains the defect classification result satisfying preset condition for actual sample verification, is specifically: The evaluation index precision rate P and recall rate R are set for model evaluation, and the calculation formula is:

Among them, TP means the number of correctly judged defective areas, TN means the number of correctly judged background areas, FP means the number of correct background areas misjudged as defective areas, FN means the number of faults misjudged as background; The average precision AP is set to evaluate the detection performance of the model for various types of defects on the test set. The area enclosed by the PR curve and the abscissa and ordinate axes is the AP of this type of defects. The calculation formula is as follows:

The detection results of multi-category defects are set to use the average precision value mAP, and the calculation formula of mAP of the test results is as follows:

. 9. The multi-scale micro-defect detection method based on the PA-MLFPN workpiece surface according to claim 1, characterized in that, in the verification of the preset defect classification corresponding to the micro-defect image, the set cycle iteration step number is set to 100. First, the batch size is set to 32, and the learning rate is initialized to 5e-4. When the number of iteration steps reaches 50, the batch size is reset to 16 and the learning rate is 1e-4. 10. The multi-scale micro-defect detection method based on the PA-MLFPN workpiece surface according to claim 1, characterized in that, for actual sample verification, a defect classification result that satisfies a preset condition is obtained, an early stop method is used during training, and each The validation loss is calculated for each iteration. When the validation loss value reaches the local optimum, the iteration is continued for 6 times. If the model no longer converges, the training is stopped.

Images