[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN118429728B - A method for detecting fertilizer particles based on small sample learning - Google Patents

A method for detecting fertilizer particles based on small sample learning Download PDF

Info

Publication number
CN118429728B
CN118429728B CN202410881242.XA CN202410881242A CN118429728B CN 118429728 B CN118429728 B CN 118429728B CN 202410881242 A CN202410881242 A CN 202410881242A CN 118429728 B CN118429728 B CN 118429728B
Authority
CN
China
Prior art keywords
image
fertilizer particle
fertilizer
dataset
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410881242.XA
Other languages
Chinese (zh)
Other versions
CN118429728A (en
Inventor
周林立
曾涵
汤才国
熊建巧
谢中成
卫学友
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei Institutes of Physical Science of CAS
Original Assignee
Hefei Institutes of Physical Science of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei Institutes of Physical Science of CAS filed Critical Hefei Institutes of Physical Science of CAS
Priority to CN202410881242.XA priority Critical patent/CN118429728B/en
Publication of CN118429728A publication Critical patent/CN118429728A/en
Application granted granted Critical
Publication of CN118429728B publication Critical patent/CN118429728B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/09Supervised learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/30Noise filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于小样本学习的化肥颗粒检测方法,涉及人工智能技术领域,将包含化肥颗粒的化肥颗粒图像与参考标准图像集送入到训练完成的孪生网络模型中,输出化肥颗粒图像分别与合格的参考标准图像和不合格的参考标准图像之间的相似度得分;对得到的所有相似度得分计算加权相似度;将得到的加权相似度分别与预设的合格阈值和不合格阈值进行差值比较,若,则判断待检测的化肥颗粒图像对应的化肥颗粒合格,若,则判断待检测的化肥颗粒图像对应的化肥颗粒不合格;该化肥颗粒检测方法实现了化肥颗粒监测的无人化与智能化,不仅减少了人力成本,而且提高了安全性。

The present invention discloses a fertilizer particle detection method based on small sample learning, which relates to the field of artificial intelligence technology. A fertilizer particle image containing fertilizer particles and a reference standard image set are sent to a trained twin network model, and similarity scores between the fertilizer particle image and a qualified reference standard image and an unqualified reference standard image are output; weighted similarities are calculated for all similarity scores obtained; and the obtained weighted similarities are compared with preset qualified thresholds and unqualified thresholds respectively. If the weighted similarities are equal, the weighted similarities are compared with preset qualified thresholds and unqualified thresholds respectively. , then the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be qualified. , then the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be unqualified; the fertilizer particle detection method realizes the unmanned and intelligent monitoring of fertilizer particles, which not only reduces the labor cost but also improves the safety.

Description

一种基于小样本学习的化肥颗粒检测方法A method for detecting fertilizer particles based on small sample learning

技术领域Technical Field

本发明涉及人工智能技术领域,尤其涉及一种基于小样本学习的化肥颗粒检测方法。The present invention relates to the field of artificial intelligence technology, and in particular to a fertilizer particle detection method based on small sample learning.

背景技术Background Art

随着科学技术的发展,工厂生产方式已从机械化转向自动化和智能化。化肥在作物生长中扮演着重要角色,因此化肥的造粒粒度成为影响最终质量的关键因素。氨酸法造粒工艺是一种常见的化肥生产工艺,通过在转鼓造粒机中进行化学反应,将硫酸、氨、磷酸一铵、硫酸与尿素、氯化钾、蒸馏水等原料转化为化肥小颗粒。然而,传统的氨酸法造粒工艺需要技术工人近距离观察造粒机内的物料状态和化肥,以判断加入的水蒸气量是否合适,并调节加入的水蒸气。这种依赖于肉眼观察的方式耗时耗力,而且化肥造粒过程中散发出的化学气体,如氨气,可能导致工人氨中毒。With the development of science and technology, factory production methods have shifted from mechanization to automation and intelligence. Fertilizer plays an important role in crop growth, so the granulation size of fertilizer has become a key factor affecting the final quality. The amino acid granulation process is a common fertilizer production process. Through chemical reactions in a drum granulator, sulfuric acid, ammonia, monoammonium phosphate, sulfuric acid and raw materials such as urea, potassium chloride, and distilled water are converted into small fertilizer particles. However, the traditional amino acid granulation process requires technical workers to observe the material state and fertilizer in the granulator at close range to determine whether the amount of water vapor added is appropriate and adjust the added water vapor. This method that relies on visual observation is time-consuming and labor-intensive, and chemical gases emitted during the fertilizer granulation process, such as ammonia, may cause workers to suffer from ammonia poisoning.

发明内容Summary of the invention

基于背景技术存在的技术问题,本发明提出了一种基于小样本学习的化肥颗粒检测方法,不仅减少了人力成本,而且提高了工作环境的安全性。Based on the technical problems existing in the background technology, the present invention proposes a fertilizer particle detection method based on small sample learning, which not only reduces the labor cost but also improves the safety of the working environment.

本发明提出的一种基于小样本学习的化肥颗粒检测方法,用于判断化肥颗粒是否合格,其特征在于,将包含化肥颗粒的化肥颗粒图像与参考标准图像集送入到训练完成的孪生网络模型中,所述参考标准图像集包括合格的参考标准图像和不合格的参考标准图像,输出化肥颗粒图像分别与合格的参考标准图像和不合格的参考标准图像之间的相似度得分;The present invention proposes a fertilizer particle detection method based on small sample learning, which is used to judge whether the fertilizer particles are qualified. The method is characterized in that a fertilizer particle image containing fertilizer particles and a reference standard image set are sent to a trained twin network model, wherein the reference standard image set includes qualified reference standard images and unqualified reference standard images, and similarity scores between the fertilizer particle image and the qualified reference standard image and the unqualified reference standard image are output;

基于合格的参考标准图像以及不合格的参考标准图像对应的权重,对得到的所有相似度得分计算加权相似度;Calculate weighted similarity for all similarity scores obtained based on weights corresponding to qualified reference standard images and unqualified reference standard images;

将得到的加权相似度分别与预设的合格阈值和不合格阈值进行差值比较,若,则判断待检测的化肥颗粒图像对应的化肥颗粒合格,若,则判断待检测的化肥颗粒图像对应的化肥颗粒不合格,其中表示加权相似度与合格阈值之间的差值,表示加权相似度与不合格阈值之间的差值;The obtained weighted similarity is compared with the preset qualified threshold and unqualified threshold respectively. , then the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be qualified. , then the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be unqualified, where represents the difference between the weighted similarity and the qualified threshold, represents the difference between the weighted similarity and the disqualified threshold;

所述孪生网络模型的主体结构为事先通过大型数据集预训练后的ResNet50结构,将ResNet50结构的最后一层全连接层移除后冻结其余所有层,然后在移除全连接层的ResNet50结构的输出端依次连接池化层和比较层。The main structure of the twin network model is a ResNet50 structure that has been pre-trained with a large data set. After removing the last fully connected layer of the ResNet50 structure, all remaining layers are frozen, and then the pooling layer and the comparison layer are sequentially connected to the output end of the ResNet50 structure with the fully connected layer removed.

所述孪生网络模型的训练过程如下:The training process of the twin network model is as follows:

S1、获取多个化肥颗粒图像并预处理,以预处理后的化肥颗粒图像作为化肥颗粒图像数据集;S1, acquiring a plurality of fertilizer particle images and preprocessing them, and using the preprocessed fertilizer particle images as a fertilizer particle image dataset;

S2、基于化肥颗粒图像数据集构建图像对数据集,所述图像对数据集包括多个正样本图像对和多个负样本图像对,所述正样本图像对为从化肥颗粒图像数据集中选取的两张具有相同属性的图像组合,所述负样本图像对为从化肥颗粒图像数据集中选取的两张具有不同属性的图像组合;S2. construct an image pair dataset based on the fertilizer particle image dataset, wherein the image pair dataset includes a plurality of positive sample image pairs and a plurality of negative sample image pairs, wherein the positive sample image pair is a combination of two images with the same attributes selected from the fertilizer particle image dataset, and the negative sample image pair is a combination of two images with different attributes selected from the fertilizer particle image dataset;

S3、将图像对数据集中正样本图像对或负样本图像对输入到孪生网络模型中,基于ResNet50结构进行特征提取,分别得到对应的两个图像特征;S3, input the positive sample image pair or the negative sample image pair in the image pair dataset into the twin network model, perform feature extraction based on the ResNet50 structure, and obtain the corresponding two image features respectively;

S4、两个图像特征经过池化层的池化处理后进入比较层计算相似度,分别生成相似度得分;S4, after the two image features are pooled by the pooling layer, they enter the comparison layer to calculate the similarity and generate similarity scores respectively;

S5、根据两个图像特征分别对应的标签以及相似度得分,应用分级动态相似性损失函数计算孪生网络模型的损失,并通过优化算法更新孪生网络模型的模型参数。S5. According to the labels and similarity scores corresponding to the two image features, the hierarchical dynamic similarity loss function is applied to calculate the loss of the twin network model, and the model parameters of the twin network model are updated through the optimization algorithm.

进一步地,其中分级动态相似性损失函数具体如下:Furthermore, the hierarchical dynamic similarity loss function The details are as follows:

其中,是批次中图像对数据集中图像对的数量,图像对数据集中图像对为正样本图像对和负样本图像对混合后的总称,表示图像对数据集中第个图像对的标签,为图像对数据集中图像对的两个图像之间的相似度得分,分别是相似图像对和不相似图像对的损失函数,分别表示相似样本对的阈值以及不相似样本对得阈值,是一个超参数,用于平衡正则项的影响。in, is the number of image pairs in the batch image pair dataset. The image pairs in the image pair dataset are the general term for the mixture of positive sample image pairs and negative sample image pairs. Represents the image pair in the dataset The labels of the image pairs, is the similarity score between the two images of an image pair in the image pair dataset, and are the loss functions for similar image pairs and dissimilar image pairs, respectively. and Respectively represent the thresholds of similar sample pairs and dissimilar sample pairs, is a hyperparameter used to balance the regularization term impact.

进一步地,孪生网络模型在训练完成后,通过化肥颗粒图像验证数据集进行验证,具体验证过程如下:Furthermore, after the training of the twin network model is completed, it is verified through the fertilizer particle image verification dataset. The specific verification process is as follows:

获取多个化肥颗粒图像并预处理得到化肥颗粒图像验证数据集,基于化肥颗粒图像验证数据集构建图像对验证数据集;Acquire multiple fertilizer particle images and preprocess them to obtain a fertilizer particle image verification dataset, and construct an image pair verification dataset based on the fertilizer particle image verification dataset;

将图像对验证数据集中正样本验证图像对中的一组图像组合或负样本验证图像对中的一组图像组合输入到训练完成的孪生网络模型中,输出化肥颗粒的预测分类结果;Input a group of image combinations in the positive sample verification image pair or a group of image combinations in the negative sample verification image pair in the image pair verification data set into the trained twin network model, and output the predicted classification results of the fertilizer particles;

若化肥颗粒的预测分类结果始终偏离化肥颗粒的真实分类结果,则将冻结后的ResNet50结构从最顶层进行逐层解冻,通过调小学习率对依次对解冻后的孪生网络模型进行训练以及验证,直至孪生网络模型的性能不再提升为止。If the predicted classification results of the fertilizer particles always deviate from the actual classification results of the fertilizer particles, the frozen ResNet50 structure will be unfrozen layer by layer from the top layer, and the unfrozen twin network models will be trained and verified in sequence by reducing the learning rate until the performance of the twin network model no longer improves.

进一步地,在步骤S1中,获取多个化肥颗粒图像并预处理具体为:Further, in step S1, a plurality of fertilizer particle images are obtained and preprocessed as follows:

S11、采用加权平均法对所获取的化肥颗粒图像的同一个像素位置的三个通道RGB赋予不同的权重值,具体为;S11, using a weighted average method to assign different weight values to the three channels RGB of the same pixel position of the obtained fertilizer particle image, specifically:

其中,表示计算出的该像素位置的灰度值,表示该像素位置的红色分量值;表示该像素位置的绿色分量值;表示该像素位置的蓝色分量值;in, Represents the calculated grayscale value of the pixel position, Indicates the red component value of the pixel position; Indicates the green component value of the pixel position; Indicates the blue component value of the pixel position;

S12、采用锐化滤波对灰度化处理后的化肥颗粒图像进行滤波处理,得到预处理后的化肥颗粒图像。S12, using a sharpening filter to filter the grayscale processed fertilizer particle image to obtain a preprocessed fertilizer particle image.

进一步地,在步骤S12中,所述锐化滤波操作使用一个预定义的滤波核,所述滤波核配置为强化图像边缘,其中所述预定义的滤波核为一个3x3的滤波矩阵,滤波矩阵中心元素为正值,周围元素为负值,所述预定义的滤波核具体为以下滤波矩阵:Further, in step S12, the sharpening filter operation uses a predefined filter kernel, and the filter kernel is configured to enhance the edge of the image, wherein the predefined filter kernel is a 3x3 filter matrix, the center element of the filter matrix is a positive value, and the surrounding elements are negative values, and the predefined filter kernel is specifically the following filter matrix:

将通过滤波处理后的化肥颗粒图像储存,并作为预处理后的化肥颗粒图像以构建化肥颗粒图像数据集。The fertilizer particle images after filtering are stored and used as preprocessed fertilizer particle images to construct a fertilizer particle image dataset.

本发明提供的一种基于小样本学习的化肥颗粒检测方法的优点在于:本发明结构中提供的一种基于小样本学习的化肥颗粒检测方法,通过工业摄像头定时拍摄生产过程中的化肥颗粒图像,工控机读取工业摄像头的RGB图像,然后采用加权平均法将RGB图像灰度化;接着锐化滤波进行图像特征突出;然后根据现有化肥颗粒图像数据集构建图像对数据集,克服了由于不合格样本稀缺导致的数据不平衡问题;再进行孪生网络模型的训练得到权重;最后使用训练好的孪生网络模型进行化肥颗粒进行识别监测,对监测结果进行反馈与上传;通过这种非接触式监测方法实现了化肥颗粒监测的无人化与智能化,不仅减少了人力成本,而且避免了工人长期吸入混有氨酸的空气会导致工人氨中毒的潜在危害,提高了安全性。The advantages of the fertilizer particle detection method based on small sample learning provided by the present invention are: the fertilizer particle detection method based on small sample learning provided in the structure of the present invention uses an industrial camera to regularly shoot images of fertilizer particles in the production process, an industrial computer reads the RGB image of the industrial camera, and then uses the weighted average method to grayscale the RGB image; then sharpening filtering is performed to highlight the image features; then an image pair data set is constructed based on the existing fertilizer particle image data set, thereby overcoming the data imbalance problem caused by the scarcity of unqualified samples; then the twin network model is trained to obtain weights; finally, the trained twin network model is used to identify and monitor fertilizer particles, and the monitoring results are fed back and uploaded; this non-contact monitoring method realizes unmanned and intelligent fertilizer particle monitoring, which not only reduces labor costs, but also avoids the potential harm of ammonia poisoning caused by workers inhaling air mixed with ammonia for a long time, thereby improving safety.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1为化肥颗粒检测方法的流程图;FIG1 is a flow chart of a method for detecting fertilizer particles;

图2为检测系统的结构流程示意图;FIG2 is a schematic diagram of the structural flow of the detection system;

图3为孪生网络模型的结构与训练策略示意图;Figure 3 is a schematic diagram of the structure and training strategy of the twin network model;

图4为化肥颗粒图像预处理的示意图;FIG4 is a schematic diagram of fertilizer particle image preprocessing;

图5为孪生网络模型的训练流程图;Figure 5 is a training flow chart of the twin network model;

图6为检测系统对化学颗粒合格性的判断流程图。FIG. 6 is a flow chart showing the determination process of the chemical particle eligibility by the detection system.

具体实施方式DETAILED DESCRIPTION

下面,通过具体实施例对本发明的技术方案进行详细说明,在下面的描述中阐述了很多具体细节以便于充分理解本发明。但是本发明能够以很多不同于在此描述的其他方式来实施,本领域技术人员可以在不违背本发明内涵的情况下做类似改进,因此本发明不受下面公开的具体实施的限制。Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

如图1至6所示,本发明提出的一种基于小样本学习的化肥颗粒检测方法,用于判断化肥颗粒是否合格,具体为:As shown in FIGS. 1 to 6 , the present invention proposes a fertilizer particle detection method based on small sample learning, which is used to determine whether the fertilizer particles are qualified, specifically:

将包含化肥颗粒的化肥颗粒图像与参考标准图像集送入到训练完成的孪生网络模型中,所述参考标准图像集包括合格的参考标准图像和不合格的参考标准图像,输出化肥颗粒图像分别与合格的参考标准图像和不合格的参考标准图像之间的相似度得分;Sending a fertilizer particle image containing fertilizer particles and a reference standard image set into a trained twin network model, wherein the reference standard image set includes qualified reference standard images and unqualified reference standard images, and outputting similarity scores between the fertilizer particle image and the qualified reference standard images and the unqualified reference standard images respectively;

基于合格的参考标准图像以及不合格的参考标准图像对应的权重,对得到的所有相似度得分计算加权相似度;Calculate weighted similarity for all similarity scores obtained based on weights corresponding to qualified reference standard images and unqualified reference standard images;

将得到的加权相似度分别与预设的合格阈值和不合格阈值进行差值比较,若,则判断待检测的化肥颗粒图像对应的化肥颗粒合格,可以将该化肥颗粒图像加入到合格的参考标准图像,以增加合格的参考标准图像中的图像数目,若,则判断待检测的化肥颗粒图像对应的化肥颗粒不合格,可以将该化肥颗粒图像加入到不合格的参考标准图像,以增加不合格的参考标准图像中的图像数目,其中表示加权相似度与合格阈值之间的差值,表示加权相似度与不合格阈值之间的差值;The obtained weighted similarity is compared with the preset qualified threshold and unqualified threshold respectively. , then the fertilizer particle corresponding to the fertilizer particle image to be detected is judged to be qualified, and the fertilizer particle image can be added to the qualified reference standard image to increase the number of images in the qualified reference standard image. , then the fertilizer particle corresponding to the fertilizer particle image to be detected is judged to be unqualified, and the fertilizer particle image can be added to the unqualified reference standard image to increase the number of images in the unqualified reference standard image, where represents the difference between the weighted similarity and the qualified threshold, represents the difference between the weighted similarity and the disqualified threshold;

可以定期评估整个检测系统的性能,根据性能评估结果,调整检测系统,以提高化肥颗粒是否合格的检测准确性。The performance of the entire detection system can be evaluated regularly, and based on the performance evaluation results, the detection system can be adjusted to improve the accuracy of detecting whether the fertilizer particles are qualified.

孪生网络模型的训练过程如下:The training process of the twin network model is as follows:

S1、获取多个化肥颗粒图像并预处理,以预处理后的化肥颗粒图像作为化肥颗粒图像数据集,具体包括步骤S11至S12;S1, acquiring a plurality of fertilizer particle images and preprocessing them, and using the preprocessed fertilizer particle images as a fertilizer particle image dataset, specifically including steps S11 to S12;

S11、采用加权平均法将所获取的化肥颗粒图像灰度化;S11, graying the obtained fertilizer particle image by using a weighted average method;

对于电子图像,组成图像的最小单元为像素,每个像素由R\G\B(红\绿\蓝)三个分量表示,且各通道的取值范围为0~255。在图像处理时,往往把RGB图像的红、绿、蓝三个颜色通道转换为一个颜色通道,以此来减少图像处理的计算量和图像存储的数据量,极大提高图像处理的效率。本实施例采用加权平均法进行灰度化,对对所获取的化肥颗粒图像的同一个像素位置的三个通道RGB赋予不同的权重值,其转换公式为:For electronic images, the smallest unit that makes up the image is a pixel. Each pixel is represented by three components, R\G\B (red\green\blue), and the value range of each channel is 0 to 255. When processing images, the red, green, and blue color channels of RGB images are often converted into one color channel to reduce the amount of image processing calculations and the amount of image storage data, greatly improving the efficiency of image processing. This embodiment uses the weighted average method for grayscale conversion, and assigns different weight values to the three channels RGB at the same pixel position of the obtained fertilizer particle image. The conversion formula is:

其中,表示计算出的该像素位置的灰度值,表示该像素位置的红色分量值;表示该像素位置的绿色分量值;表示该像素位置的蓝色分量值;in, Represents the calculated grayscale value of the pixel position, Indicates the red component value of the pixel position; Indicates the green component value of the pixel position; Indicates the blue component value of the pixel position;

采用如上公式来对图像进行灰度化,相比只用单通道灰度化的图像,使用此公式的更加层次分明;具体为:提高后续处理效率:灰度图像只包含亮度信息,数据量比彩色图像小,处理起来更加高效。提高图像质量:加权平均法能更有效地保留原图像的视觉信息和细节,特别是在颜色对比和边缘清晰度方面。兼容性和实现简单:虽然加权平均法涉及不同的权重,但其实现仍然简单直观。这使得它易于在各种编程和图像处理环境中实现和应用,无需复杂的算法支持。The above formula is used to grayscale the image. Compared with the image grayscaled with only a single channel, the image grayscaled with this formula has more distinct layers. Specifically: Improve the efficiency of subsequent processing: Grayscale images only contain brightness information, and the amount of data is smaller than that of color images, so they are more efficient to process. Improve image quality: The weighted average method can more effectively preserve the visual information and details of the original image, especially in terms of color contrast and edge clarity. Compatibility and simple implementation: Although the weighted average method involves different weights, its implementation is still simple and intuitive. This makes it easy to implement and apply in various programming and image processing environments without the need for complex algorithm support.

S12、采用锐化滤波对灰度化处理后的化肥颗粒图像进行滤波处理,得到预处理后的化肥颗粒图像;S12, using a sharpening filter to filter the grayscale processed fertilizer particle image to obtain a preprocessed fertilizer particle image;

图像在生成和传输过程中常常因受到各种噪声的干扰和影响而产生图像噪点,这对后续图像的处理将产生不利影响。因此,为了抑制噪点,改善图像质量,需对图像进行去噪预处理。本实施例采用锐化滤波的方式来对灰度化处理后的化肥颗粒图像进行降噪处理,从而突出特征。应用一个线性滤波操作于灰度化处理后的化肥颗粒图像,所述线性滤波操作使用一个预定义的滤波核,该滤波核配置为强化图像边缘,其中所述预定义的滤波核为一个3x3滤波矩阵,该滤波矩阵的中心元素为正值,周围元素为负值,以增强所述当前处理灰度化处理后的化肥颗粒图像的对比度并锐化图像边缘。其中,预定义的滤波核具体为以下滤波矩阵:During the generation and transmission of images, image noise is often generated due to the interference and influence of various noises, which will have an adverse effect on the subsequent image processing. Therefore, in order to suppress noise and improve image quality, the image needs to be pre-processed for denoising. This embodiment uses a sharpening filter to reduce the noise of the fertilizer particle image after grayscale processing, thereby highlighting the features. A linear filtering operation is applied to the fertilizer particle image after grayscale processing, and the linear filtering operation uses a predefined filter kernel, which is configured to enhance the image edge, wherein the predefined filter kernel is a 3x3 filter matrix, the central element of the filter matrix is a positive value, and the surrounding elements are negative values, so as to enhance the contrast of the fertilizer particle image after grayscale processing and sharpen the image edge. Among them, the predefined filter kernel is specifically the following filter matrix:

该滤波矩阵通过增加图像中心像素值相对于其周围像素值的权重,从而实现图像的锐化。将通过滤波处理后的化肥颗粒图像储存,并作为预处理后的化肥颗粒图像以构建化肥颗粒图像数据集,其过程如图4所示。The filter matrix increases the weight of the central pixel value of the image relative to the surrounding pixel values, thereby achieving image sharpening. The fertilizer particle image after filtering is stored and used as the preprocessed fertilizer particle image to construct the fertilizer particle image dataset, and the process is shown in Figure 4.

S2、基于化肥颗粒图像数据集构建图像对数据集,所述图像对数据集包括多个正样本图像对和多个负样本图像对,所述正样本图像对为从化肥颗粒图像数据集中选取的两张具有相同属性的图像组合,所述负样本图像对为从化肥颗粒图像数据集中选取的两张具有不同属性的图像组合;S2. construct an image pair dataset based on the fertilizer particle image dataset, wherein the image pair dataset includes a plurality of positive sample image pairs and a plurality of negative sample image pairs, wherein the positive sample image pair is a combination of two images with the same attributes selected from the fertilizer particle image dataset, and the negative sample image pair is a combination of two images with different attributes selected from the fertilizer particle image dataset;

将化肥颗粒图像数据集分为两类,一类代表化肥颗粒合格情况,记为合格样本,另一类代表不合格情况,记为不合格样本。从合格样本中选取两幅图像形成正样本图像对,同理在不合格样本中选取两幅图像同样构成正样本图像对,以表示具有相似属性或类别的图像组合。另外,分别从合格样本和不合格样本中各选取一幅图像形成负样本图像对,以表示具有不同属性或类别的图像组合。The fertilizer particle image dataset is divided into two categories, one category represents the qualified situation of fertilizer particles, recorded as qualified samples, and the other category represents the unqualified situation, recorded as unqualified samples. Two images are selected from the qualified samples to form a positive sample image pair. Similarly, two images are selected from the unqualified samples to form a positive sample image pair to represent a combination of images with similar attributes or categories. In addition, one image is selected from each of the qualified samples and the unqualified samples to form a negative sample image pair to represent a combination of images with different attributes or categories.

将这些图像对以及对应的标签储存,形成一个专门用于训练孪生网络模型的图像对数据集。此举旨在为孪生网络模型提供充分的训练数据,使其能够准确识别和分类化肥颗粒的合格与不合格情况。这种数据集的构建方法可以克服小样本带来的问题是数据不平衡,即在不合格样本数量不足时能够弥补数据不平衡的问题,通过孪生网络模型关注图像对之间的相似度,从而克服了由于不合格样本稀缺导致的数据不平衡问题。有助于提高模型的准确性和鲁棒性,从而更好地满足工业生产中对化肥质量控制的需求。These image pairs and corresponding labels are stored to form an image pair dataset specifically used to train the twin network model. This is intended to provide sufficient training data for the twin network model so that it can accurately identify and classify qualified and unqualified fertilizer particles. This dataset construction method can overcome the problem of data imbalance caused by small samples, that is, it can make up for the problem of data imbalance when the number of unqualified samples is insufficient. The twin network model focuses on the similarity between image pairs, thereby overcoming the data imbalance problem caused by the scarcity of unqualified samples. It helps to improve the accuracy and robustness of the model, so as to better meet the needs of fertilizer quality control in industrial production.

S3、将图像对数据集中正样本图像对或负样本图像对输入到孪生网络模型中,基于ResNet50结构进行特征提取,分别得到对应的两个图像特征;S3, input the positive sample image pair or the negative sample image pair in the image pair dataset into the twin network model, perform feature extraction based on the ResNet50 structure, and obtain the corresponding two image features respectively;

孪生网络模型的主体结构为事先通过大型数据集预训练后的ResNet50结构,将ResNet50结构的最后一层全连接层移除后冻结其余所有层,然后在移除全连接层的ResNet50结构输出端依次连接池化层和比较层,用于计算相似度,开始时冻结ResNet50预训练网络的卷积层,通过冻结这些层,能够直接利用ResNet50在大型数据集(如ImageNet)上学到的丰富特征表示。这些特征往往具有很好的泛化能力。然后根据需要对ResNet50逐步解冻。根据孪生网络模型在图像对验证数据集上的表现,评估是否需要进一步微调孪生网络模型。The main structure of the twin network model is the ResNet50 structure that has been pre-trained with a large dataset. After removing the last fully connected layer of the ResNet50 structure, all other layers are frozen. Then, the pooling layer and the comparison layer are sequentially connected to the output of the ResNet50 structure with the fully connected layer removed to calculate the similarity. At the beginning, the convolutional layers of the ResNet50 pre-trained network are frozen. By freezing these layers, the rich feature representations learned by ResNet50 on large datasets (such as ImageNet) can be directly used. These features often have good generalization capabilities. Then, ResNet50 is gradually unfrozen as needed. Based on the performance of the twin network model on the image pair verification dataset, it is evaluated whether the twin network model needs to be further fine-tuned.

ResNet 网络(Residual net)是残差网络的通用概念;ResNet50 是指包含了50个卷积层(包括卷积层、池化层、全连接层等)的 ResNet 网络,ResNet50 是基于 ImageNet数据集上的训练所提出的一个具体网络结构。ResNet network (Residual net) is a general concept of residual network; ResNet50 refers to a ResNet network containing 50 convolutional layers (including convolutional layers, pooling layers, fully connected layers, etc.). ResNet50 is a specific network structure proposed based on training on the ImageNet dataset.

孪生网络模型在训练完成后,通过化肥颗粒图像验证数据集进行验证,验证过程如下:获取多个化肥颗粒图像并预处理得到化肥颗粒图像验证数据集,基于化肥颗粒图像验证数据集构建图像对验证数据集;将图像对验证数据集中正样本验证图像对中的一组图像组合或负样本验证图像对中的一组图像组合输入到训练完成的孪生网络模型中,输出化肥颗粒的预测分类结果;若化肥颗粒的预测分类结果始终偏离化肥颗粒的真实分类结果,则将冻结后的ResNet50结构从最顶层进行逐层解冻,通过调小学习率对依次对解冻后的孪生网络模型进行训练以及验证,直至孪生网络模型的性能不再提升为止After the training of the twin network model is completed, it is verified using the fertilizer particle image verification dataset. The verification process is as follows: multiple fertilizer particle images are obtained and preprocessed to obtain the fertilizer particle image verification dataset, and an image pair verification dataset is constructed based on the fertilizer particle image verification dataset; a group of image combinations in the positive sample verification image pairs or a group of image combinations in the negative sample verification image pairs in the image pair verification dataset are input into the trained twin network model, and the predicted classification results of the fertilizer particles are output; if the predicted classification results of the fertilizer particles always deviate from the true classification results of the fertilizer particles, the frozen ResNet50 structure is thawed layer by layer from the top layer, and the thawed twin network models are trained and verified in turn by reducing the learning rate until the performance of the twin network model no longer improves

具体为:最初孪生网络模型是在ResNet50结构完全冻结的基础上进行训练,然后通过图像对验证数据集验证训练完成后的孪生网络模型,如果若化肥颗粒的预测分类结果始终偏离化肥颗粒的真实分类结果,则将冻结后的ResNet50结构从最顶层进行逐层解冻,每次解冻ResNet50结构的一部分结构,然后将通过用于训练的图像对数据集训练部分解冻后的孪生网络模型,然后再使用图像对验证数据集验证训练完成后的孪生网络模型,如果若化肥颗粒的预测分类结果始终偏离化肥颗粒的真实分类结果,则将部分后的ResNet50结构继续解冻一部分结构,再次进入孪生网络模型的训练和验证环节,直至孪生网络模型的性能不再显著提升为止。Specifically: initially, the twin network model is trained on the basis of a completely frozen ResNet50 structure, and then the trained twin network model is verified by an image pair verification data set. If the predicted classification results of the fertilizer particles always deviate from the actual classification results of the fertilizer particles, the frozen ResNet50 structure will be unfrozen layer by layer from the top layer, and a part of the ResNet50 structure will be unfrozen each time. Then, the partially unfrozen twin network model will be trained with the image pair data set used for training, and then the image pair verification data set will be used to verify the trained twin network model. If the predicted classification results of the fertilizer particles always deviate from the actual classification results of the fertilizer particles, a part of the ResNet50 structure will continue to be unfrozen, and the training and verification phase of the twin network model will be entered again until the performance of the twin network model is no longer significantly improved.

每次解冻的层与之前解冻的层一起作为ResNet50结构的已解冻层,其余未解冻的层扔处于冻结状态,ResNet50结构的解冻顺序为从顶层开始,一般来说,越接近顶部的层越是针对具体任务的特征,而底层则更多关注通用特征。当解冻层进行训练时,使用较小的学习率是很重要的,以避免破坏这些层已有的有价值的特征。重复这个过程,每次解冻更多的层,直到模型性能不再显著提升为止。Each unfrozen layer is used together with the previously unfrozen layer as the unfrozen layer of the ResNet50 structure, and the remaining unfrozen layers are still frozen. The unfreezing order of the ResNet50 structure starts from the top layer. Generally speaking, the layers closer to the top are more task-specific, while the bottom layers focus more on general features. When unfreezing layers for training, it is important to use a smaller learning rate to avoid destroying the valuable features that already exist in these layers. Repeat this process, unfreezing more layers each time, until the model performance is no longer significantly improved.

S4、两个图像特征经过池化层的池化处理后进入比较层计算相似度,分别生成相似度得分;S4, after the two image features are pooled by the pooling layer, they enter the comparison layer to calculate the similarity and generate similarity scores respectively;

S5、根据两个图像特征分别对应的标签以及相似度得分,应用分级动态相似性损失函数计算孪生网络模型的损失,并通过优化算法更新孪生网络模型的模型参数。S5. According to the labels and similarity scores corresponding to the two image features, the hierarchical dynamic similarity loss function is applied to calculate the loss of the twin network model, and the model parameters of the twin network model are updated through the optimization algorithm.

孪生网络模型对所输入的图像对数据集的处理具体为:首先,从之前的图像对数据集中提取训练数据,该数据是由图像对和标签构成;接着,图像对里的图像分别经过ResNet50结构部分,该部分网络权重是共享的,用于捕获深层次的图像特征;再接着,利用所提取的图像特征表示,通过一个比较层计算两个图像特征之间的相似度,生成一个相似度得分。The twin network model processes the input image pair dataset as follows: first, training data is extracted from the previous image pair dataset, which consists of image pairs and labels; then, the images in the image pairs are respectively passed through the ResNet50 structure part, where the weights of this part of the network are shared and used to capture deep image features; then, the extracted image feature representation is used to calculate the similarity between the two image features through a comparison layer to generate a similarity score.

然后,根据图像对的标签和相似度得分,应用分级动态相似性损失函数计算孪生网络的损失值,并通过优化算法更新网络参数,以减小损失值;分级动态相似性损失函数,对于孪生网络中的每个图像对及其标签=1表示图像对相似,=0表示图像对不相似),该损失函数的公式为:Then, according to the labels and similarity scores of the image pairs, the hierarchical dynamic similarity loss function is applied to calculate the loss value of the twin network, and the network parameters are updated through the optimization algorithm to reduce the loss value; the hierarchical dynamic similarity loss function, for each image pair in the twin network and its tags =1 means the image pair is similar, = 0 means the image pair is not similar), the loss function The formula is:

其中,是批次中图像对数据集中图像对的数量,图像对数据集中图像对为正样本图像对和负样本图像对混合后的总称,表示图像对数据集中第个图像对的标签,为图像对数据集中图像对的两个图像之间的相似度得分,分别是相似图像对和不相似图像对的损失函数,分别表示相似样本对的阈值以及不相似样本对得阈值,是一个超参数,用于平衡正则项的影响。in, is the number of image pairs in the batch image pair dataset. The image pairs in the image pair dataset are the general term for the mixture of positive sample image pairs and negative sample image pairs. Represents the image pair in the dataset The labels of the image pairs, is the similarity score between the two images of an image pair in the image pair dataset, and are the loss functions for similar image pairs and dissimilar image pairs, respectively. and Respectively represent the thresholds of similar sample pairs and dissimilar sample pairs, is a hyperparameter used to balance the regularization term impact.

阈值:这个阈值通常用于区分相似样本对中的相似度水平,低于该阈值的样本对被认为是"高度相似"。阈值:这个阈值通常用于区分不相似样本对中的差异程度,超过该阈值的样本对被认为是"显著不相似"。Threshold : This threshold is usually used to distinguish the similarity level in similar sample pairs, and sample pairs below this threshold are considered "highly similar". Threshold : This threshold is usually used to distinguish the degree of difference in dissimilar sample pairs. Sample pairs exceeding this threshold are considered to be "significantly dissimilar".

其中,考虑了相似度的不同级别,例如通过阈值来定义不同的相似度或距离区间。其中,对于相似样本对,希望样本间的相似度得分越小越好,特别是当相似度得分小于阈值时,公式为:in, and Different levels of similarity are considered, for example by thresholding and To define different similarity or distance intervals. For similar sample pairs, we hope that the similarity score between samples is The smaller the better, especially when the similarity score is less than the threshold When , the formula is:

对于不相似样本对,希望样本间的相似度得分越大越好,特别是当相似度得分大于阈值时,公式为:For dissimilar sample pairs, we hope that the similarity score between the samples The larger the better, especially when the similarity score is greater than the threshold When , the formula is:

是一个正则项,基于孪生网络模型对于当前图像对的分类信心(模型预测的概率)动态调整每个图像对对的损失贡献,旨在增加对于那些孪生网络模型难以正确分类的样本对(即孪生网络模型预测的置信度低)的惩罚。 It is a regularization term that dynamically adjusts the loss contribution of each image pair based on the classification confidence of the twin network model for the current image pair (the probability predicted by the model), aiming to increase the penalty for those sample pairs that are difficult for the twin network model to classify correctly (i.e., the confidence predicted by the twin network model is low).

正则项可以定义为:Regularization term It can be defined as:

是第个图像对、标签为的概率,是一个正的调节参数,用于控制对低置信度预测的惩罚程度。 It is image pairs, labeled The probability of is a positive tuning parameter that controls the degree of penalty for low confidence predictions.

这个损失函数的核心思想是在孪生网络模型中引入对相似度分级的概念,并且根据孪生网络模型在训练过程中对不同级别相似度的图像对的处理能力动态调整学习焦点。This loss function The core idea of this method is to introduce the concept of similarity grading into the twin network model, and dynamically adjust the learning focus according to the processing ability of the twin network model for image pairs with different levels of similarity during training.

分级处理:对于相似图像对:可以根据的关系定义,例如使用分段函数或基于之差的平滑函数,以区分不同程度的相似性。对于不相似图像对:类似地处理,可以根据的关系定义,区分不同程度的差异性。Hierarchical processing: For similar image pairs: Can be based on and , such as using piecewise functions or based on and A smooth function of the difference between and to distinguish different degrees of similarity. For dissimilar image pairs: Similarly, we can process and The relationship definition distinguishes different degrees of difference.

将待测化肥颗粒送入训练完成的孪生网络模型中,以输出化肥颗粒的分类结果,并对化肥颗粒的分类结果进行判断,如果发现化肥颗粒情况不合格,LED警报器将亮起红色灯光,提示作业人员需要调整原料投入比例。在正常情况下,LED警报器为绿色,所有的检测指标都会显示在工控机界面上。同时,工控机会通过HTTPS传输协议将工业摄像头拍摄的待测化肥颗粒的图像和最终的检测结果(合格还是不合格)发送至云平台,以便远程人员能够实时查看当前的化肥生产状况。The fertilizer particles to be tested are sent to the trained twin network model to output the classification results of the fertilizer particles and judge the classification results of the fertilizer particles. If the fertilizer particles are found to be unqualified, the LED alarm will light up red to prompt the operator to adjust the raw material input ratio. Under normal circumstances, the LED alarm is green and all test indicators will be displayed on the industrial computer interface. At the same time, the industrial computer will send the image of the fertilizer particles to be tested taken by the industrial camera and the final test results (qualified or unqualified) to the cloud platform through the HTTPS transmission protocol, so that remote personnel can view the current fertilizer production status in real time.

本实施例在进行化肥合格性判断时,检测系统包括监测模块、警报模块和云平台模块;In this embodiment, when determining the eligibility of fertilizers, the detection system includes a monitoring module, an alarm module and a cloud platform module;

监测模块,工控机通过以太网传输方式与工业摄像头进行通讯,进而控制工业摄像头工作,读取工业摄像头所获取的化肥颗粒图像信息,化肥颗粒图像经过预处理后被送入孪生网络模型进行识别监测。化肥颗粒是否合格的判断流程如图6所示,这个流程不仅包含了孪生网络模型对化肥颗粒大小进行自动检测的步骤,还整合了一个动态学习机制,以不断优化和更新参考标准图像库,以下是详细的流程:(1)相似度计算:将待检测的化肥颗粒图像与预先设置的样本图像集中的所有参考标准图像进行相似度计算得到相似度评分,所述预先设置的样本图像集包括多张合格的参考标准图像与多张不合格的参考标准图像;(2)综合评分:基于上一步得到的所有相似度评分,采用一种综合评分机制来评估待检测的化肥颗粒图像的状态,基于合格的参考标准图像和不合格的参考标准图像对应的权重,对得到的所有相似度评分计算加权相似度,其中合格的参考标准图像与不合格的参考标准图像的权重可以不同;(3)判断与分类:将得到的加权相似度分别与预设的合格阈值和不合格阈值进行比较,若加权相似度更接近于合格阈值,则判断待检测的化肥颗粒图像对应的化肥颗粒合格,将合格的待检测的化肥颗粒图像加入到合格的参考标准图像中,若加权相似度更接近于不合格阈值,则判断待检测的化肥颗粒图像对应的化肥颗粒不合格,将不合格的待检测的化肥颗粒图像加入到不合格的参考标准图像中;(4)人工复检:对于系统判断结果和实际情况不一致的案例,即人工复检发现的误判,将这些图像视作新的情况,将误判的合格的待检测的化肥颗粒图像加入到合格的参考标准图像中,将误判的不合格的待检测的化肥颗粒图像加入到不合格的参考标准图像,更新参考标准图像库后,可用于未来的检测中,以提高判断的准确性;(5)动态优化:定期评估整个检测系统的性能,特别是人工复检后发现的误判率。根据性能评估结果,调整阈值、决策逻辑、更新的参考标准图像库信息,甚至是重新训练孪生网络模型,确保检测逻辑能够适应生产条件和标准的变化。这个流程通过引入动态学习和反馈机制,不仅使检测系统能够根据新的数据自我优化,还增加了系统应对未知或新出现情况的能力。这种方法能够显著提高长期的检测准确率和系统的可靠性。In the monitoring module, the industrial computer communicates with the industrial camera through Ethernet transmission, thereby controlling the operation of the industrial camera and reading the fertilizer particle image information obtained by the industrial camera. After preprocessing, the fertilizer particle image is sent to the twin network model for identification and monitoring. The process of judging whether the fertilizer particles are qualified is shown in Figure 6. This process not only includes the steps of automatically detecting the size of fertilizer particles by the twin network model, but also integrates a dynamic learning mechanism to continuously optimize and update the reference standard image library. The following is the detailed process: (1) Similarity calculation: The fertilizer particle image to be detected is similar to all the reference standard images in the pre-set sample image set to obtain a similarity score, and the pre-set sample image set includes multiple qualified reference standard images and multiple unqualified reference standard images; (2) Comprehensive scoring: Based on all the similarity scores obtained in the previous step, a comprehensive scoring mechanism is used to evaluate the status of the fertilizer particle image to be detected. Based on the weights corresponding to the qualified reference standard images and the unqualified reference standard images, the weighted similarity of all the similarity scores obtained is calculated, where the weights of the qualified reference standard images and the unqualified reference standard images can be different; (3) Judgment and classification: The obtained weighted similarity is compared with the pre-set The qualified threshold and unqualified threshold are set for comparison. If the weighted similarity is closer to the qualified threshold, the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be qualified, and the qualified fertilizer particle image to be detected is added to the qualified reference standard image. If the weighted similarity is closer to the unqualified threshold, the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be unqualified, and the unqualified fertilizer particle image to be detected is added to the unqualified reference standard image. (4) Manual re-inspection: For cases where the system judgment results are inconsistent with the actual situation, that is, misjudgments found by manual re-inspection, these images are regarded as new situations, and the misjudged qualified fertilizer particle images to be detected are added to the qualified reference standard images, and the misjudged unqualified fertilizer particle images to be detected are added to the unqualified reference standard images. After the reference standard image library is updated, it can be used in future detections to improve the accuracy of judgment. (5) Dynamic optimization: Regularly evaluate the performance of the entire detection system, especially the misjudgment rate found after manual re-inspection. Based on the performance evaluation results, adjust the threshold, decision logic, update the reference standard image library information, and even retrain the twin network model to ensure that the detection logic can adapt to changes in production conditions and standards. This process introduces dynamic learning and feedback mechanisms, which not only enables the detection system to self-optimize based on new data, but also increases the system's ability to cope with unknown or emerging situations. This approach can significantly improve long-term detection accuracy and system reliability.

警报模块,工控机通过USB接口连接一个LED警报器灯,根据检测结果的不同显示不同的LED警报器灯状态,当出现化肥颗粒不合格的情况会及时通知操作人员异常情况,以便调整生产原料比例;正常状态下LED警报器为绿灯,所有检测指标等均显示在工控机界面上;Alarm module: The industrial computer is connected to an LED alarm light through a USB interface. Different LED alarm light states are displayed according to different test results. When unqualified fertilizer particles appear, the operator will be notified of the abnormal situation in time so as to adjust the production raw material ratio. Under normal conditions, the LED alarm is green, and all test indicators are displayed on the industrial computer interface.

云平台模块。工控机将工业摄像头的图像和最终的检测结果通过HTTPS传输协议发送至WEB云平台,使远程人员也能查看到当前的化肥生产状况。Cloud platform module. The industrial computer sends the images and final detection results of the industrial camera to the WEB cloud platform through the HTTPS transmission protocol, so that remote personnel can also view the current fertilizer production status.

以上所述,仅为本发明较佳的具体实施方式,但本发明的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范围内,根据本发明的技术方案及其发明构思加以等同替换或改变,都应涵盖在本发明的保护范围之内。The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims (4)

1.一种基于小样本学习的化肥颗粒检测方法,用于判断化肥颗粒是否合格,其特征在于,将包含化肥颗粒的化肥颗粒图像与参考标准图像集送入到训练完成的孪生网络模型中,所述参考标准图像集包括合格的参考标准图像和不合格的参考标准图像,输出化肥颗粒图像分别与合格的参考标准图像以及不合格的参考标准图像之间的相似度得分;1. A fertilizer particle detection method based on small sample learning, used to determine whether the fertilizer particles are qualified, characterized in that a fertilizer particle image containing fertilizer particles and a reference standard image set are sent to a trained twin network model, the reference standard image set includes qualified reference standard images and unqualified reference standard images, and similarity scores between the fertilizer particle image and the qualified reference standard image and the unqualified reference standard image are output; 基于合格的参考标准图像以及不合格的参考标准图像对应的权重,对得到的所有相似度得分计算加权相似度;Calculate weighted similarity for all similarity scores obtained based on weights corresponding to qualified reference standard images and unqualified reference standard images; 将得到的加权相似度分别与预设的合格阈值和不合格阈值进行差值比较,若,则判断待检测的化肥颗粒图像对应的化肥颗粒合格,若,则判断待检测的化肥颗粒图像对应的化肥颗粒不合格,其中表示加权相似度与合格阈值之间的差值,表示加权相似度与不合格阈值之间的差值;The obtained weighted similarity is compared with the preset qualified threshold and unqualified threshold respectively. , then the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be qualified. , then the fertilizer particles corresponding to the fertilizer particle image to be detected are judged to be unqualified, where represents the difference between the weighted similarity and the qualified threshold, represents the difference between the weighted similarity and the disqualified threshold; 所述孪生网络模型的主体结构为事先通过大型数据集预训练后的ResNet50结构,将ResNet50结构的最后一层全连接层移除后冻结其余所有层,然后在移除全连接层的ResNet50结构的输出端依次连接池化层和比较层;The main structure of the twin network model is a ResNet50 structure that has been pre-trained with a large data set. After removing the last fully connected layer of the ResNet50 structure, all other layers are frozen. Then, the pooling layer and the comparison layer are sequentially connected to the output end of the ResNet50 structure from which the fully connected layer is removed. 所述孪生网络模型的训练过程如下:The training process of the twin network model is as follows: S1、获取多个化肥颗粒图像并预处理,以预处理后的化肥颗粒图像作为化肥颗粒图像数据集;S1, acquiring a plurality of fertilizer particle images and preprocessing them, and using the preprocessed fertilizer particle images as a fertilizer particle image dataset; S2、基于化肥颗粒图像数据集构建图像对数据集,所述图像对数据集包括多个正样本图像对和多个负样本图像对,所述正样本图像对为从化肥颗粒图像数据集中选取的两张具有相同属性的图像组合,所述负样本图像对为从化肥颗粒图像数据集中选取的两张具有不同属性的图像组合;S2. construct an image pair dataset based on the fertilizer particle image dataset, wherein the image pair dataset includes a plurality of positive sample image pairs and a plurality of negative sample image pairs, wherein the positive sample image pair is a combination of two images with the same attributes selected from the fertilizer particle image dataset, and the negative sample image pair is a combination of two images with different attributes selected from the fertilizer particle image dataset; S3、将图像对数据集中正样本图像对或负样本图像对输入到孪生网络模型中,基于ResNet50结构进行特征提取,分别得到对应的两个图像特征;S3, input the positive sample image pair or the negative sample image pair in the image pair dataset into the twin network model, perform feature extraction based on the ResNet50 structure, and obtain the corresponding two image features respectively; S4、两个图像特征经过池化层的池化处理后进入比较层计算相似度,分别生成相似度得分;S4, after the two image features are pooled by the pooling layer, they enter the comparison layer to calculate the similarity and generate similarity scores respectively; S5、根据两个图像特征分别对应的标签以及相似度得分,应用分级动态相似性损失函数计算孪生网络模型的损失,并通过优化算法更新孪生网络模型的模型参数;S5. According to the labels and similarity scores corresponding to the two image features, a hierarchical dynamic similarity loss function is applied to calculate the loss of the twin network model, and the model parameters of the twin network model are updated through an optimization algorithm; 其中分级动态相似性损失函数具体如下:The hierarchical dynamic similarity loss function The details are as follows: 其中,是批次中图像对数据集中图像对的数量,图像对数据集中图像对为正样本图像对和负样本图像对混合后的总称,表示图像对数据集中第个图像对的标签,为图像对数据集中图像对的两个图像之间的相似度得分,分别是相似图像对和不相似图像对的损失函数,分别表示相似样本对的阈值以及不相似样本对得阈值,是一个超参数,用于平衡正则项的影响,表示乘积。in, is the number of image pairs in the batch image pair dataset. The image pairs in the image pair dataset are the general term for the mixture of positive sample image pairs and negative sample image pairs. Represents the image pair in the dataset The labels of the image pairs, is the similarity score between the two images of an image pair in the image pair dataset, and are the loss functions for similar image pairs and dissimilar image pairs, respectively. and Respectively represent the thresholds of similar sample pairs and dissimilar sample pairs, is a hyperparameter used to balance the regularization term The impact of Represents the product. 2.根据权利要求1所述的基于小样本学习的化肥颗粒检测方法,其特征在于,孪生网络模型在训练完成后,通过化肥颗粒图像验证数据集进行验证,具体验证过程如下:2. The fertilizer particle detection method based on small sample learning according to claim 1 is characterized in that after the training of the twin network model is completed, it is verified by a fertilizer particle image verification data set, and the specific verification process is as follows: 获取多个化肥颗粒图像并预处理得到化肥颗粒图像验证数据集,基于化肥颗粒图像验证数据集构建图像对验证数据集;Acquire multiple fertilizer particle images and preprocess them to obtain a fertilizer particle image verification dataset, and construct an image pair verification dataset based on the fertilizer particle image verification dataset; 将图像对验证数据集中正样本验证图像对中的一组图像组合或负样本验证图像对中的一组图像组合输入到训练完成的孪生网络模型中,输出化肥颗粒的预测分类结果;Input a group of image combinations in the positive sample verification image pair or a group of image combinations in the negative sample verification image pair in the image pair verification data set into the trained twin network model, and output the predicted classification results of the fertilizer particles; 若化肥颗粒的预测分类结果始终偏离化肥颗粒的真实分类结果,则将冻结后的ResNet50结构从最顶层进行逐层解冻,通过调小学习率对依次对解冻后的孪生网络模型进行训练以及验证,直至孪生网络模型的性能不再提升为止。If the predicted classification results of the fertilizer particles always deviate from the actual classification results of the fertilizer particles, the frozen ResNet50 structure will be unfrozen layer by layer from the top layer, and the unfrozen twin network models will be trained and verified in sequence by reducing the learning rate until the performance of the twin network model no longer improves. 3.根据权利要求1所述的基于小样本学习的化肥颗粒检测方法,其特征在于,在步骤S1中,获取多个化肥颗粒图像并预处理具体为:3. The fertilizer particle detection method based on small sample learning according to claim 1 is characterized in that, in step S1, a plurality of fertilizer particle images are obtained and preprocessed as follows: S11、采用加权平均法对所获取的化肥颗粒图像的同一个像素位置的三个通道RGB赋予不同的权重值,具体为;S11, using a weighted average method to assign different weight values to the three channels RGB of the same pixel position of the obtained fertilizer particle image, specifically: 其中,表示计算出的该像素位置的灰度值,表示该像素位置的红色分量值;表示该像素位置的绿色分量值;表示该像素位置的蓝色分量值;in, Represents the calculated grayscale value of the pixel position, Indicates the red component value of the pixel position; Indicates the green component value of the pixel position; Indicates the blue component value of the pixel position; S12、采用锐化滤波对灰度化处理后的化肥颗粒图像进行滤波处理,得到预处理后的化肥颗粒图像。S12, using a sharpening filter to filter the grayscale processed fertilizer particle image to obtain a preprocessed fertilizer particle image. 4.根据权利要求3所述的基于小样本学习的化肥颗粒检测方法,其特征在于,在步骤S12中,所述锐化滤波操作使用一个预定义的滤波核,所述滤波核配置为强化图像边缘,其中所述预定义的滤波核为一个3x3的滤波矩阵,滤波矩阵中心元素为正值,周围元素为负值,所述预定义的滤波核具体为以下滤波矩阵:4. The fertilizer particle detection method based on small sample learning according to claim 3 is characterized in that, in step S12, the sharpening filter operation uses a predefined filter kernel, and the filter kernel is configured to strengthen the edge of the image, wherein the predefined filter kernel is a 3x3 filter matrix, the central element of the filter matrix is a positive value, and the surrounding elements are negative values, and the predefined filter kernel is specifically the following filter matrix: 将通过滤波处理后的化肥颗粒图像储存,并作为预处理后的化肥颗粒图像以构建化肥颗粒图像数据集。The fertilizer particle images after filtering are stored and used as preprocessed fertilizer particle images to construct a fertilizer particle image dataset.
CN202410881242.XA 2024-07-03 2024-07-03 A method for detecting fertilizer particles based on small sample learning Active CN118429728B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410881242.XA CN118429728B (en) 2024-07-03 2024-07-03 A method for detecting fertilizer particles based on small sample learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410881242.XA CN118429728B (en) 2024-07-03 2024-07-03 A method for detecting fertilizer particles based on small sample learning

Publications (2)

Publication Number Publication Date
CN118429728A CN118429728A (en) 2024-08-02
CN118429728B true CN118429728B (en) 2024-09-27

Family

ID=92316464

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410881242.XA Active CN118429728B (en) 2024-07-03 2024-07-03 A method for detecting fertilizer particles based on small sample learning

Country Status (1)

Country Link
CN (1) CN118429728B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022134728A1 (en) * 2020-12-25 2022-06-30 苏州浪潮智能科技有限公司 Image retrieval method and system, and device and medium
CN116067848A (en) * 2023-02-15 2023-05-05 中国科学院合肥物质科学研究院 Deep learning-based fertilizer granulation granularity detection method and device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113177916B (en) * 2021-04-21 2023-04-07 清华大学深圳国际研究生院 Slight hypertension fundus identification model based on few-sample learning method
CN113223068B (en) * 2021-05-31 2024-02-02 西安电子科技大学 Multi-mode image registration method and system based on depth global features
CN115830701A (en) * 2022-10-12 2023-03-21 中用科技有限公司 Human violation behavior prediction method based on small sample learning

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022134728A1 (en) * 2020-12-25 2022-06-30 苏州浪潮智能科技有限公司 Image retrieval method and system, and device and medium
CN116067848A (en) * 2023-02-15 2023-05-05 中国科学院合肥物质科学研究院 Deep learning-based fertilizer granulation granularity detection method and device

Also Published As

Publication number Publication date
CN118429728A (en) 2024-08-02

Similar Documents

Publication Publication Date Title
JP7074460B2 (en) Image inspection equipment and methods
CN112036755B (en) Supervision method and system for quality detection of building engineering
WO2019214309A1 (en) Model test method and device
CN110057820B (en) Method, system and storage medium for on-line detection of chlorine-hydrogen ratio of hydrogen chloride synthesis furnace
CN115331172A (en) Workshop dangerous behavior recognition alarm method and system based on monitoring video
CN113239991B (en) Flame image oxygen concentration prediction method based on regression generation countermeasure network
CN113259379A (en) Abnormal alarm identification method, device, server and storage medium based on incremental learning
CN113723229A (en) Signal lamp detection method and device and computer readable storage medium
CN117372819B (en) Target detection increment learning method, device and medium for limited model space
US20230206601A1 (en) Device and method for classifying images and accessing the robustness of the classification
CN117253192A (en) Intelligent system and method for silkworm breeding
CN116485796A (en) Pest detection method, pest detection device, electronic equipment and storage medium
CN111310837A (en) Vehicle refitting recognition method, device, system, medium and equipment
CN118429728B (en) A method for detecting fertilizer particles based on small sample learning
CN111488939A (en) Model training method, classification method, device and equipment
CN119478729A (en) A wheat leaf pest detection method based on YOLOv10
CN117349734B (en) Water meter equipment identification method and device, electronic equipment and storage medium
CN117974005A (en) A warehouse management method and system based on RFID technology
CN117772641A (en) Material enhancement feature recognition picking method of color selector
CN116563730A (en) A method for poppy detection in UAV remote sensing images based on re-learning
CN116977256A (en) Training method, device, equipment and storage medium for defect detection model
Kumar et al. Advanced Predictive Models for Decisive Stage Detection in Apple Canker Disease
CN113486354A (en) Firmware safety evaluation method, system, medium and electronic equipment
CN116563770B (en) Method, device, equipment and medium for detecting vehicle color
CN119418132B (en) A method and system for identifying and processing information of multiple types of materials

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant