AU2020102091A4 - Intelligent steel slag detection method and system based on convolutional neural network - Google Patents
Intelligent steel slag detection method and system based on convolutional neural network Download PDFInfo
- Publication number
- AU2020102091A4 AU2020102091A4 AU2020102091A AU2020102091A AU2020102091A4 AU 2020102091 A4 AU2020102091 A4 AU 2020102091A4 AU 2020102091 A AU2020102091 A AU 2020102091A AU 2020102091 A AU2020102091 A AU 2020102091A AU 2020102091 A4 AU2020102091 A4 AU 2020102091A4
- Authority
- AU
- Australia
- Prior art keywords
- steel slag
- image
- steel
- neural network
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent steel slag detection method and system based on
a convolutional neural network. The intelligent steel slag detection method includes
5 the following steps: steel slag image recognition: by taking a color steel slag image in
a video frame image as an object, recognizing the color steel slag image in the video
frame image by adopting an image recognition method based on an improved AlexNet
convolutional neural network; steel flow target detection: detecting steel flow
information in the color steel slag image, detecting a steel flow from a complex
10 background by using a target detection method based on a YOLOv3 convolutional
neural network model, thereby accurately detecting a slag inclusion condition of the
steel flow; and color steel slag image segmentation: preprocessing the color steel slag
image by using a K-means clustering algorithm based on a Lab color space, and
completely separating steel slag from molten steel by adopting an improved Otsu
15 image segmentation algorithm. Visual detection is performed on the steel slag by
using a visual user interface system. The intelligent steel slag detection method is
simple, practicable, low in cost and capable of distinguishing the steel slag from the
molten steel, avoiding false detection and improving the real-time recognition
precision of the steel slag image and the purity of the molten steel.
1/8
Digital steel flow Steel fowanalog
video transmission data tran mission
SDrive the imag
Infrared camera | mag acquisition i acquisition card to 1 Remeive steel
cad capture avideo flow video data
Imag processing
technology computer 4
Steel slag image Steel slag image Steel slag image
recognition module detection module segmentation module
Display a
g Proportion of steel
slag in real time
Alann
Fig. 1
Description
1/8
Digital steel flow Steel fowanalog video transmission data tran mission SDrive the imag Infrared camera | mag acquisition i acquisition card to 1 Remeive steel cad capture avideo flow video data
Imag processing technology computer 4
Steel slag image Steel slag image Steel slag image recognition module detection module segmentation module
Display a g Proportion of steel slag in real time
Alann
Fig. 1
The present invention belongs to the technical field of steel slag detection and
particularly relates to an intelligent steel slag detection method and system based on a
convolutional neural network.
In order to produce steel with high quality and high added value, it is necessary to
strictly control steel slag to enter the next process, for example, hot metal
desulfurization slag removal and converter steelmaking are indispensable process
steps in a steelmaking process. The hot metal desulfurization slag removal is a hot
metal pretreatment process. A desulfurizing agent needs to be added in a
desulfurization process, a layer of residues will float on the surface of hot metal after
desulfurization, and slag needs to be removed by using a mucking loader before the
hot metal enters a converter. Hot metal desulfurization decides the sulfur content of
the hot metal after a final treatment, while slag removal is an important means for
removing high-sulfur slag from the desulfurized hot metal and is a main factor for
deciding the total amount of sulfur entering the converter. In order to meet the demand
of the converter on high-quality hot metal, shorten the smelting time and effectively
remove the steel slag, it is very necessary to detect the steel slag in the hot metal
before converter steelmaking. Therefore, the detection of the steel slag in the hot
metal in the process steps, namely the hot metal desulfurization slag removal and the
converter steelmaking, is also significant.
At present, there are many methods, such as an eye observation method, a weighing detection method, an electromagnetic detection method, an ultrasonic detection method and an infrared detection method, for detecting steel slag at home and abroad.
However, these methods have more or less deficiencies in application. The eye
observation method is greatly subjective so as to require an operating worker to have
abundant operational experience. For the weighing detection method, whether a
nozzle is closed is mainly decided by taking an average value of repeatedly poured
molten steel quality as a reference value which will directly affect a detection result, if
the reference value is set to be too high, slag contained in a steel flow will exceed
standard, and if the reference value is set to be too low, resources will be wasted. For
the electromagnetic detection method, a special coil is required to be mounted near a
tap hole, the tap hole is high in temperature to result in short service life of the coil,
and therefore, the reconstruction and maintenance expenses are very high. In the
ultrasonic detection method, the temperature of a working environment of a probe
used reaches up to about 1500°C, and therefore, there are relatively high requirements
on the reconstruction expense and device maintenance expense caused by harsh
environments. For the infrared detection method in which the steel flow is required
not to be shielded during detection, in order to avoid a shielding phenomenon, a long
nozzle needs to be removed, which may result in secondary oxidation of the molten
steel. Due to the harsh environment of a steelmaking site, there are various external
vibration interference signals, and thus, it is easy to affect the accuracy rate of
detection.
In addition, at present, for application of visual inspection in steel slag detection, for
example, the infrared detection method is generally implemented by using a
segmentation method in an image processing stage, the steel slag and the molten steel
need to be detected together, and a final purpose is not achieved, and therefore, a steel
slag detecting effect is not ideal.
The technical problem to be solved by the present invention is to provide an
intelligent steel slag detection method and system based on a convolutional neural
network, which are simple, practicable, low in cost and capable of distinguishing steel
slag from molten steel, avoiding false detection and improving the real-time
recognition precision of a steel slag image and the purity of the molten steel.
In order to solve the above-mentioned technical problem, the present invention
provides technical solutions described as follows.
An intelligent steel slag detection method based on a convolutional neural network, is
characterized by including the steps sequentially performed as follows:
steel slag image recognition: acquiring a steel slag video frame image including steel
flow and steel slag brightness information by using an infrared detector and a camera;
and by taking a color steel slag image in the video frame as an object, recognizing the
color steel slag image in the video frame image by adopting an image recognition
method based on an improved AlexNet convolutional neural network;
steel flow target detection: detecting steel flow information in the color steel slag
image, detecting a steel flow from a complex background by using a target detection
method based on a YOLOv3 convolutional neural network model, thereby reducing
influences of the complex background to a detection target; and
color steel slag image segmentation: with specific to problems that there is a greater
color difference between a color steel slag image target and a background region and
steel slag light is segmented into steel slag by mistake, preprocessing the color steel
slag image by using a K-means clustering algorithm based on a Lab color space, and
completely separating the steel slag from molten steel by adopting an improved Otsu
image segmentation algorithm.
Further, in the step of steel slag image recognition, the image recognition method
based on the improved AlexNet convolutional neural network specifically includes the
following steps:
selecting sample images to perform sample data classification, wherein a large part of
the sample images are used as a training set, a remaining part of the sample images
are used as a test set, the sample images are classified into positive samples and
negative samples, the positive samples are named as "have" representing high steel
slag content, and the negative samples are named as "no" representing low steel slag
content;
establishing a tensorflow deep learning framework and a keras deep learning
framework, and creating an improved AlexNet convolutional neural network model
by virtue of the two frameworks;
preprocessing the images, and storing images in a data set by h5py so as to achieve
flexible and efficient I/O data, high-capacity data and complex data; and importing os,
numpy and matplotlib libraries in Python;
defining a function to acquire a path list and a label list of the data set, and defining a
type corresponding to a label as steel slag;
converting all images in the training set and the test set into a numpy array;
storing the data of the training set and data of the test set into an h5 file;
importing the data set, testing the data set, and performing visualization on a part of
the data set;
training the AlexNet convolutional neural network model, and improving the AlexNet
convolutional neural network model; and
acquiring the optimal recognition precision by virtue of the trained AlexNet
convolutional neural network model.
Further, in the step of steel flow target detection, the target detection method based on
the YOLOv3 convolutional neural network includes the following steps: firstly,
normalizing the size of an input steel slag image, and training the data set of the steel
slag by virtue of a designed network to obtain a convolutional neural network model;
and then, acquiring the confidence of a bounding box of a current steel slag target by
virtue of the trained model, and classifying objects in the bounding box; and finally,
filtering the bounding box by using a non-maximum suppression algorithm to obtain
an optimal result.
Further, in the step of color steel slag image segmentation, firstly, an RGB color space
is converted into a uniform Lab color space; then, the image is preprocessed by using
the K-means clustering algorithm so that the steel slag is primarily separated from a
background; and finally, a proportion of pixel points of the background to the overall
image is used as a weight to be introduced to a target equation, threshold
segmentation is performed on the preprocessed steel slag image by adopting the
improved Otsu image segmentation algorithm, and thus, the steel slag is relatively
accurately separated from the steel flow.
Further, threshold segmentation on the preprocessed steel slag image by adopting the
improved Otsu image segmentation algorithm includes the following specific steps:
converting the preprocessed color steel slag image into a grey scale image, and
calculating a grey scale histogram of the image;
setting a threshold t;
classifying the image into a background and a target, calculating grey scale mean
values of the target and the background, and acquiring an average grey scale mean
value according to the histogram;
traversing a grey scale according to an improved optimal threshold selection formula, and determining a maximum function value t as an optimal threshold Th; and performing image segmentation according to the optimal threshold, and outputting an segmented image.
Further, visual detection on the steel slag by using a visual user interface system
includes the following steps:
newly establishing a main window;
converting a ui file into a py file;
calling a function file;
performing layout by using a layout manager, and performing layout on a play button,
a pause button, a classification button, a detection button and a segmentation button;
and
during video playing, clicking the "classification" button to recognize the steel slag
image, thus displaying recognition precision on a user interface; clicking the
"detection" button to make a detection box appear on the user interface, thus
automatically labeling a detected molten steel part and the confidence thereof; and
clicking the "segmentation" button to display the segmented steel slag on the user
interface.
The present invention further provides an intelligent steel slag detection system based
on a convolutional neural network, including a color steel slag image recognition
module, a color steel slag image detection module, a color steel slag image
segmentation module and a visual user interface system;
the color steel slag image recognition module configured to learn, process and analyze
an image acquired by an infrared detector and a camera by virtue of an improved
AlexNet convolutional neural network as a steel slag image recognition model
framework, wherein the improved AlexNet convolutional neural network is provided with five convolutional layers, five pooling layers and two fully connected layers in total, a RELU function is used as an activation function of the CNN (Convolutional
Neural Network), a part of neurons are randomly ignored by using Dropout during
draining, and overlapped maximum pooling is used in the CNN;
the color steel slag image detection module configured to be based on a YOLOv3
convolutional neural network model and detect a color steel slag image recognized by
the color steel slag image recognition module by using a target detection method
based on the YOLOv3 convolutional neural network model so as to detect a steel flow
from a complex background;
the color steel slag image segmentation module configured to preprocess the color
steel slag image processed by the color steel slag image detection module by using a
K-means clustering algorithm based on a Lab color space and segment the color steel
slag image by using an improved Otsu image segmentation algorithm so as to
completely separate steel slag from molten steel; and
the visual user interface system configured to perform visual operation by virtue of
buttons; and the visual user interface system including a menu system for newly
establishing a main window, converting a file and calling the converted file and
further including a layout manager for performing layout on a play button, a pause
button, a classification button, a detection button and a segmentation button, wherein
during video playing, the "classification" button is clicked to recognize the steel slag
image so that recognition precision can be displayed on a user interface; the
"detection" button is clicked so that a detection box may appear on the user interface
to automatically label a detected molten steel part and the confidence thereof; and the
"segmentation" button is clicked so that the segmented steel slag may be displayed on
the user interface.
The principle of the present invention is described as follows: according to the present invention, the color steel slag video image is acquired by using the infrared detector and the camera, and there is a significant color difference between the molten steel and the steel slag, and therefore, the steel flow and the steel slag in the acquired video frame image are different in brightness. Firstly, the steel slag image taken as the object to be processed by a computer so as to be recognized, then, the steel flow is detected, and finally, the steel slag in the steel flow is segmented, thereby accurately detecting the slag inclusion condition of the steel flow. With specific to the steel slag image recognition, the image acquisition capability of the current camera and the computing capability of the computer are sufficiently utilized to recognize the steel slag image by adopting the improved AlexNet convolutional neural network, and the improved AlexNet convolutional neural network improves the recognition precision of the steel slag image. With specific to the steel flow target detection, the steel flow is detected from the complex background by using the target detection method based on the YOLOv3 convolutional neural network model, which replaces a field worker to complete steel slag detection work, so that steel slag detection no longer depends on observation of human eyes, and furthermore, not only is the vision of operating personnel protected, but also a detection result is not too subjective. The steel slag image segmentation aims at solving the problems that there is the greater color difference between the color steel slag image target and the background region and the steel slag light is segmented into the steel slag by mistake.
Compared with the prior art, the processing method in which the target detection
method based on deep learning is creatively applied to the field of steel slag detection,
the convolutional neural network model is established, and the target is primarily
recognized and selected has the advantages that slag surfaces (the steel slag floats on
the surface of the molten steel or is included in the steel flow) may be recognized and
detected; then, a target region is intercepted by virtue of detected position information so that influences of background interference to the detection target are eliminated; and finally, the steel slag is segmented from the molten steel by using the improved threshold segmentation method so that a segmentation effect is improved.
The advantages and positive effects of the present invention are mainly reflected in
several aspects as follows.
1) Steel flow recognition function: complex display feature extraction is avoided by
using the recognition method based on the improved AlexNet convolutional neural
network, the feature analysis on the image is incorporated into the neural network,
and features of the image are effectively distinguished by adjusting the weight and
bias of the neural network.
2) Steel flow detection function: the labeled images are detected by using a YOLOv3
detection algorithm in which softmax regression is replaced with logistic
regression, the target is detected by virtue of multi-scale features, three prior
boxes are set, the prior boxes with nine sizes are copolymerized in total, and each
feature diagram is allocated with three of the prior boxes. YOLOv3 draws on the
thinking of a residual network, a Darknet-53 network is used, a long jump
connection is additionally provided, and a deeper network is trained by virtue of
the long jump connection, so that the detection precision is higher, and the
algorithm is higher in speed.
3) Steel slag segmentation function: since there is the greater color difference
between the color steel slag image target and the background region and the steel
slag light is segmented into the steel slag by mistake, it is unrealizable that an
ideal segmentation effect is achieved by using a segmentation method based on color spatial clustering alone. Therefore, the color steel slag image is required to be preprocessed by using the K-means clustering algorithm in the Lab color space, and threshold segmentation is required to be performed on the preprocessed image by using the improved Otsu algorithm, and thus, the steel slag image is effectively segmented.
Fig. 1 is a flow diagram of an intelligent steel slag detection method based on a
convolutional neural network according to the present invention;
Fig. 2 is an overall structural diagram of an intelligent steel slag detection system
based on a convolutional neural network according to an embodiment of the present
invention;
Fig. 3 is a diagram showing visualization on a part of a data set according to an
embodiment of the present invention;
Fig. 4 is a structural diagram of an AlexNet convolutional neural network model
according to an embodiment of the present invention;
Fig. 5 is a structural diagram of a YOLOv3 convolutional neural network model
according to an embodiment of the present invention;
Fig. 6 is a diagram showing a detection effect of a YOLOv3 detection algorithm
according to an embodiment of the present invention;
Fig. 7 is a diagram showing a region where a target is located according to an
embodiment of the present invention;
Fig. 8 is a diagram showing a preprocessing effect of a K-means algorithm according
to an embodiment of the present invention;
Fig. 9 is a flow diagram of the K-means algorithm according to the embodiment of the
present invention;
Fig. 10 is a flow diagram of an improved Otsu algorithm according to an embodiment of the present invention; and
Fig. 11 is a diagram showing a segmentation effect of the improved Otsu algorithm
according to the embodiment of the present invention.
In order to facilitate those of ordinary skill in the art to understand and implement the
present invention, the present invention will be further described in detail below in
conjunction with the accompanying drawings and embodiments. It should be
understood that the embodiments described herein are merely intended to describe and
explain the present invention, rather than to limit the present invention.
Fig. 1 to Fig. 2 are overall structural diagrams of an intelligent steel slag detection
system based on a convolutional neural network according to an embodiment of the
present invention. In the embodiment, a steel slag video image is acquired by using an
infrared detector and a camera (such an infrared camera in Fig. 2), and a notebook
computer is used as a terminal control platform and display hardware so as to have the
advantages of flexibility and convenience in use and simplicity in maintenance.
The basic configuration of a notebook terminal adopted in the embodiment is
described as follows:
(1) a processor: Intel Core i7-8550M, main frequency: 2.60 GHz;
(2) an internal memory: 8 GB, 799 MHz;
(3) a video card: NVIDA GeForce MX130, display: 15.6 inches;
(4) an environment of a notebook operating system: Windows 7; and
(5) a storage hard disk: 500 GB, rotating speed: 5,400 r/min.
The above-mentioned basic configuration does not limit the protective scope of the present invention, and using the infrared detector and the camera to acquire the steel slag video image and adopting the notebook computer as the terminal control platform and the display hardware all fall within the protective scope of the present invention.
An intelligent steel slag detection system based on a convolutional neural network,
provided by an embodiment of the present invention, is characterized by including a
color steel slag image recognition module, a color steel slag image detection module
and a color steel slag image segmentation module.
For the color steel slag image recognition module, deep learning has great
breakthrough in image recognition, and research scholars gradually deepen a network
model so as to improve the capability that the convolutional neural network extracts
high-level features. However, with the deepening of the network, it is possible that
there are more or less problems such as information loss and consumption during
information transmission, and meanwhile, a deep network is difficult to train due to
the presence of gradient explosion or gradient diffusion problem.
Therefore, in the embodiment, an AlexNet convolutional neural network is utilized as
a steel slag image recognition model framework to learn, process and analyze features
of an image and is improved on the basis of an AlexNet convolutional neural network
model to recognize the steel slag image, so that the recognition precision is improved.
The improved AlexNet network is provided with five convolutional layers, five
pooling layers and two fully connected layers in total. A RELU function is
successfully used as an activation function of a CNN, so that the phenomenon of
gradient diffusion of sigmoid when the network is relatively deep is overcome. A part
of neurons are randomly ignored by using Dropout during training, so that overfitting
is avoided. Overlapped maximum pooling is used in the CNN, so that the richness of the features is improved, and information loss is reduced.
For the color steel slag image detection module, a YOLOv3 network is not only
greatly improved on recognition speed as comparison with other algorithms, but also
excellent in effects in terms of recognition performance. Therefore, in the
embodiment, steel flow target detection is performed on the color steel slag image by
using the method.
In the embodiment, the convolutional neural network is adopted as a model
framework of the color steel slag image detection module, and a steel slag target is
detected by using aYOLOv3 algorithm. Firstly, the size of an input steel slag image is
normalized, and a data set of the steel slag is trained by virtue of a designed network
to obtain a convolutional neural network model; and then, the confidence of a
bounding box of a current steel slag target is acquired by virtue of the trained model,
and objects in the bounding box are classified; and finally, the bounding box is filtered
by using a non-maximum suppression algorithm to obtain an optimal result.
For the color steel slag image segmentation module, it is impossible to completely
separate the steel slag from molten steel by using a single method when a color steel
slag image is segmented, and a threshold obtained by applying an Otsu algorithm is
relatively small when there is a little difference in variances of steel slag light and a
target part in the image, and thus, a segmentation threshold is excessively low to
result in false segmentation.
In the embodiment, the color steel slag image is preprocessed by using a K-means
clustering algorithm based on a Lab color space, and an improved image segmentation
algorithm of the Otsu method is provided. Firstly, an RGB color space is converted into a uniform Lab color space; then, steel slag light and steel slag in the color steel slag image are separated from a background by using the K-means clustering algorithm; and finally, an optimal threshold selection formula in a traditional Otsu algorithm is analyzed and is modified according to a proportion of the size of the background in the image on the basis of a target variance weighting method, so that the problem that a part of the steel slag light is classified as the target by mistake due to a slightly low threshold is solved, the execution speed of the algorithm is increased, classification errors are reduced, and the segmentation effect is improved.
The K-means clustering algorithm is described as follows:
a sample set e R i=,2,,n is set to be composed of n pixel points,
and an pixel point Ai represented by each sample is composed of b data
characterizing features of the pixel point. K-means clustering aims at classifying the n
pixel points into k types and forming k clustering centers, and a data set formed by the C={ ck |k =1, 2,.- k} o- clustering centers isC k 1'2, wherein k is the cluster center of C
An Euclidean distance is defined as: .
D(xi,,o,)= (i -a,|,[i c') (1),
wherein the n pixel points are classified into a type Ck, and thus, the sum of the
Euclidean distance from all the pixel points classified into the type Ck to the
clustering center to which the pixel points belong is defined as: N(c,)=Z D(x,,o-,) X Ck (2),
subtypes of the clustering center are counted once to obtain the sum of the Euclidean
distance from all the pixel points to the clustering center to which the pixel points
belong, which is defined as: k K J(xi)_Z N(ck)= D(x,,ok k-I k-I xpeo
_-Z~awD(x,,ok)
1, ifypc,
wherein 0 , if gCk
The improved Otsu algorithm is described as follows:
since there is a little difference in variances of the steel slag light and the target part in
the image, and thus, a traditional Otsu algorithm is improved in the present invention,
and threshold segmentation is performed on the image subjected to K-means
segmentation by using the improved Otsu algorithm.
The basic principle of the traditional Otsu algorithm is that: an image is set as I, the
range of a grey scale value of the image is set as [L-1], N represents the total
number of pixels, 'i represents the number of pixel points of which the grey scales
are Ii(E [0,L- 1]), and Arepresents a probability that all the pixel points with the
grey scales i occur, then,
IN (4),
a proportion of a target region is defined as:
=o0 (5), a proportion of a background region is defined as: L-1 C,(t)= p, (6),
a target mean value is defined as:
POgte = i=°
a background mean value is defined as:
L-1
CO t (8),
an interclass variance calculation formula is defined as:
7lt) ,W W (t) (t+ CW 07 (t)
an improved threshold selection criterion formula is defined as:
0(t)=a(t)pu2(t)+c a (t)pI(t) 21 (10),
wherein 0 and 02 respectively represent weight coefficients of two variances
ool(tOpe2(tand o(tu()
A visual user interface system is configured to perform visual operation by virtue of
buttons. The visual user interface system includes a menu system for newly
establishing a main window, converting a file and calling a converted file and further
includes a layout manager for performing layout on a play button, a pause button, a
classification button, a detection button and a segmentation button. During video
playing, the "classification" button is clicked to recognize the steel slag image so that
recognition precision may be displayed on a user interface; the "detection" button is
clicked so that a detection box may appear on the user interface to automatically label
a detected molten steel part and the confidence thereof; and the "segmentation" button
is clicked so that segmented steel slag may be displayed on the user interface.
An intelligent steel slag detection method based on a convolutional neural network,
provided by an embodiment of the present invention, as shown in Fig. 1, includes the
steps sequentially performed as follows.
Step 1, a steel slag video image is acquired by using an infrared detector and a camera,
a video frame image of sampled steel slag is acquired, and the images are subjected to
data enhancement and preprocessed.
Step 2, the image is recognized by using an image recognition method based on an
improved AlexNet convolutional neural network.
Step 3, a steel flow is detected by using a YOLOv3 detection algorithm.
Step 4, a region where a target is located is intercepted by virtue of position
information detected by using the YOLOv3 detection algorithm.
Step 5, the image is preprocessed by using a K-means clustering algorithm.
Step 6, the image is segmented by using an improved Otsu algorithm.
Further, the step 1 includes the following steps:
step 1.1, the steel slag video image is acquired by using the infrared detector and the
camera;
step 1.2, an image acquisition card is driven to capture a video, and steel slag video
frame data are received;
step 1.3, a video frame is processed and is converted into a sample image, and a
tapping video is converted into frame-by-frame images by using an opencv library
and the images are saved with an extension name in a ".jpg"format so as to be used as
an experiment material; and
step 1.4, the images are subjected to data enhancement; in the embodiment, a great
deal of data are acquired by using a data enhancement method, i.e., more data are
derived by utilizing existing data in a way such as flip, translation or rotation, so that
the neural network has a better generalization effect, and the robustness of a model is
improved. A flip factor is 2.3, a translation factor is 0.4, and a rotation factor is 2.5.
Step 2, the image recognition method based on the AlexNet convolutional neural
network specifically includes the following steps.
Step 2.1, experimental data is classified as follows:
2000 images (Fig. 3 is a diagram showing visualization on a part of data of the 2000
images) are acquired, wherein 1400 images are used as a training set, and 600 images
are used as a test set. The images are classified into positive samples and negative
samples. The positive samples are named as "have" representing high steel slag
content. The negative samples are named as "no" representing low steel slag content.
A flow zone with the most obvious brightness in each image in Fig. 3 represents a slag
tapping process, the brightness of a steel flow represented by "have" is relatively high,
the brightness of steel slag included in the steel flow is relatively low, and therefore,
the content of the steel slag is relatively high. The brightness of a steel flow
represented by "no" in Fig. 3 is relatively high, the brightness of steel slag included in
the steel flow is almost consistent with the brightness of the steel flow, and therefore,
the content of the steel slag is relatively low.
Step 2.2, a tensorflow deep learning framework and a keras deep learning framework
are established, and a model is created by virtue of the two frameworks.
Step 2.3, the images are preprocessed, and images in a data set are stored by h5py so
as to achieve flexible and efficient 1/O data, high-capacity data and complex data.
Step 2.3.1, os, numpy and matplotlib libraries in Python are imported.
Step 2.3.2, a function is defined to acquire a path list and a label list of the data set, and a type corresponding to a label is defined as steel slag.
Step 2.3.3, all images in the training set and the test set are converted into a numpy
array.
Step 2.3.4, data of the training set and data of the test set are stored into an h5 file.
Step 2.3.5, the data set is imported, the data set is tested, and visualization is
performed on parts of the data set.
Step 2.4, an AlexNet convolutional neural network model is trained, and the AlexNet
convolutional neural network model is improved.
Step 2.5, the optimal recognition precision is acquired by virtue of the trained AlexNet
convolutional neural network model.
Fig. 4 is a structural diagram of the AlexNet convolutional neural network model
provided by the present invention. The size of a first layer of convolution kernel in the
AlexNet convolutional neural network model is changed from Ix11 to 3x3, the size
of a second layer of convolution kernel is changed from 7x7 to 3 x3, the step length is
changed from 4 to 1, a RELU activation function layer and a Dropout layer are added
behind each convolutional layer, and the value of the Dropout layer is 0.3. Since a
fully connected layer may result in the rapid increment of parameters, another fully
connected layer is additionally provided behind the fully connected layer, and the
number of channels is changed to be 2 (the number of types is 2).
Refer to model parameters in Table 1 and training precision and test precision contrast in Table 2.
Table 1: Model Parameters
Parameter Numerical value
Learning rate 0.001
Epoch 1
Batch size 16
Dropout 0.4
Table 2: Training Precision and Test Precision Contrast
Sample Precision
Training set 0.9629
Test set 0.9833
Step 3, the steel flow is detected by using the YOLOv3 detection algorithm.
The step 3 specifically includes the following steps.
Step 3.1, the data set of the images is labeled.
During target detection, a data set of original images is required to be labeled to tell a
target required by a machine, then, machine learning is performed, the data set of the
images is labeled by using LabellImg software, each labeled image forms an xml file,
and then, the xml file is converted into a txt file to prepare for subsequent detection.
Step 3.2, a data set of to-be-detected images is read, and the size of an input steel slag
image is normalized.
Step 3.3, positioning information is predicted.
Step 3.3.1, an Anchor Box is initialized by adopting a K-means clustering method,
wherein such priori knowledge is beneficial to predicting a bounding box.
Step 3.3.2, a score of a possibility that a target of each bounding box exists is
predicted by using a YOLOv3 network model by virtue of logistic regression.
If the currently predicted bounding box may be better overlapped with a ground truth
object, the confidence of the bounding box is 1. If the currently predicted bounding
box is not the best, but the bounding box and the ground truth object are overlapped to
exceed a certain threshold, the neural network may ignore the prediction. The closer
the confidence is to 1, the better the detection effect.
Step 3.3.3, single-label classification is improved into multi-label classification by the
YOLOv3 network model, a Softmax classifier for the single-label classification is
changed into a Logistic classifier for the multi-label classification in terms of network
structures to perform type information prediction. In the embodiment, type
information of the embodiment is 1, and a type is named as "gangzha".
Step 3.4, multi-scale feature detection is performed.
The YOLOv3 network model adopts an up-sampling and feature fusion method in
which three scales are fused, and independent detection is performed on each of
multi-scale fusion feature diagrams so that the steel flow is detected.
Step 3.5, the bounding box is filtered by using a non-maximum suppression algorithm to obtain an optimal result.
Fig. 5 is a structural diagram of the YOLOv3 network model. A YOLOv3 network
structure is composed of convolutional layers, Res layers, a Darknet-53 structure and
YOLO layers.
(1) For the convolutional layers, an image of which a pixel is 640 x 480 and the
number of channels is 3 is input by a YOLOv3 network. BN operation may be
performed on input data by each convolutional layer. 32 convolution kernels are
adopted for the convolution of each convolutional layer, and each convolution
kernel has the size of 33 and the step length of 1.
(2) For the Res layers, five Res layers with different scales and depths are selected in
total, and residual calculation operation among different layers of output are only
performed on the Res layers.
(3) For the Darknet-53 structure, YOLOv3 adopts a Darknet-53 network structure
and includes 53 convolutional layers and the remaining Res layers, these
convolutional layers are obtained by integrating convolutional layers with better
performances from mainstream network structures and are formed by combining
continuous 3x3 and lxIconvolutional layers. The Darknet-53 structure draws on
the thinking of a residual network, a long jump connection is arranged among
some layers, and a deeper network may be trained by virtue of the long jump
connection.
(4) For the YOLO layers, the 75th layer to the 105th layer are feature fusion layers of
the YOLOv3 network and are divided into three scales (13 x 13, 26 x
26 and 52 x 52), feature diagrams with the different scales are stacked under each scale, and then, local feature fusion among the feature diagrams with the different scales is realized in a way of the convolution kernel. A feature diagram with the depth of 18, namely the tensor 3 x (4 + 1 + 1) = 18, including three bounding boxes, four bounding box coordinate parameters, one target prediction confidence and one type, is finally output.
Refer to network model parameters in Table 3.
Table 3: Network Model Parameters
Parameter Numerical value
Learning rate 0.001028
Epoch 273
Batch size 16
Fig. 6 is a diagram showing a detection effect of a YOLOv3 detection algorithm
according to an embodiment of the present invention. Fig. 6(a) and Fig. 6(b) are
diagrams showing hot metal desulfurization slag removal. Fig. 6(c) is a diagram
showing converter slag tapping. By using the YOLOv3 detection algorithm, the steel
slag may be detected from respective background, and the confidences of three
images (a)-(c) arranged from front to back in Fig. 6 are respectively 1, 0.83 and 0.95.
The range of the confidence is [0-1], and the closer the confidence is to 1, the better
the detection effect. The confidences of the images (a)-(c) are all greater than 0.8, and
it may be seen from the images (a)-(c) that the images may be accurately recognized
as "gangzha" instead of other types. It is proven that the steel flow may be relatively
accurately detected by using the YOLOv3 detection algorithm in combination with
the confidences and the type information.
Step 4, the region where the target is located is intercepted by virtue of the position
information detected by using the YOLOv3 detection algorithm. By using such a
method, influences of a complex background to target detection are reduced.
The step 4 specifically includes the following steps.
Step 4.1, os, numpy and cv2 libraries in Python are imported.
Step 4.2, a color steel slag image is read, and a data set of each image is traversed.
Step 4.3, position coordinates of two pixel points at an upper left position and a right
lower position are set by virtue of target position coordinates obtained by using the
YOLOv3 detection algorithm.
Step 4.4, the image is clipped according to the position coordinates set according to
step 4.3 to obtain a target region.
Fig. 7 is a diagram showing the region where the target is located, Fig. 7(a) is an
original diagram showing hot metal desulfurization slag removal according to an
embodiment of the prevent invention, wherein molten steel and steel slag in a ladle
are used as the target. A darkness region is a background, and a brightness region and
a darkness region in the brightness region are respectively the molten steel and the
steel slag. Fig. 7(b) is a diagram showing the region where the target is located.
Position information of a detected object is contained in the YOLOv3 detection
algorithm, and then, a ladle body as well as the molten steel and the steel slag in the
ladle are intercepted according to the position information, so that influences of the
complex background to target detection of the steel slag are reduced. Characters
"desulfurization slag removal" in Fig. 7(a) are interferences carried in a video, and
therefore, the molten steel and the steel slag in the ladle are required to be intercepted
to eliminate influences of such interferences to a detection target. The operation
described herein aims at removing the characters in the image and highlighting a steel
slag region and a molten steel region, in this way, the target is easier to segment in the
next step.
Step 5, the color steel slag image is preprocessed by using the K-means clustering
algorithm based on the Lab color space. Refer to a flow diagram showing
preprocessing of the K-means clustering algorithm in Fig. 8, the step 5 includes the
following specific steps.
Step 5.1, a to-be-segmented color steel slag image is read.
Step 5.2, an RGB color space is converted into a Lab color space, and three feature
components L, a and b of each pixel point sample are extracted.
Step 5.3, initialization is performed, and four objects are randomly selected as initial
clustering centers.
Step 5.4, pixel points are classified according to the clustering centers.
Step 5.5, the clustering center of each type is corrected according to a result, a mean
value of the types is calculated, and the centers of the types are updated according to
the mean value of the types to obtain new centers.
Step 5.6, when four new centers are acquired, the pixel points in the data set are required to be reclassified into the type of the new center closest to the pixel points to perform loop iteration until a criterion function is converged, the clustering centers are not changed any more and a squared error function value is minimum, and then, the loop iteration is stopped.
Step 5.7, a preprocessed image is output.
With specific to a segmentation technology, a grey scale image is generally adopted in
other fields, however, a steel slag grey scale image may be lack of many important
feature information to result in a nonideal image segmentation effect. A color image
may provide more feature information such as color, brightness and saturation than
the grey scale image, and therefore, a grey scale image segmentation method is not
suitable for steel slag image detection and segmentation. The research on the color
image segmentation has great significance, a proper color space is required to be
selected if the color steel slag image is segmented, then, a color feature of the steel
slag image is extracted, and thus, the target may be more easily extracted. In order to
solve the problem that there is a great color difference between a target region and a
background region of the color steel slag image, the color steel slag image is
preprocessed by using the K-means clustering algorithm based on the Lab color space.
By using the algorithm, the steel slag light and the steel slag are classified into the
same type, the steel slag and the molten steel light are segmented from the
background, and thus, first segmentation is realized.
Refer to Fig. 9 which is a diagram showing a preprocessing effect of the K-means
clustering algorithm according to an embodiment of the present invention. The image
is preprocessed by using the K-means clustering algorithm, and separation of the steel
slag and the molten steel light from the background is basically realized in the preprocessed images (a)-(c).
Step 6, the steel slag in the steel flow is segmented by using the improved Otsu
algorithm. Refer to a flow diagram of the improved Otsu algorithm in Fig. 10. The
step 6 includes the following specific steps.
Step 6.1, the preprocessed color steel slag image is converted into a grey scale image,
and a grey scale histogram of the image is calculated.
Step 6.2, a threshold t is set.
Step 6.3, the image is classified into a background and a target, grey scale mean
values of the target and the background are calculated, and an average grey scale
mean value is acquired according to the histogram.
Step 6.4, a grey scale is traversed according to an improved optimal threshold
selection formula, and a maximum function value t is determined as an optimal
threshold Th.
Step 6.5, image segmentation is performed according to the optimal threshold, and a
segmented image is output. Fig. 11(a)-(c) are diagrams showing an effect that the
image is segmented by using the improved Otsu algorithm after being processed in the
step 5, wherein in Fig. 11(a) and Fig. 11(b), a brightness region in the ladle body is
steel slag, brightness regions on an inner wall of and above the ladle body are steel
slag light, and a black region is a background. In Fig. 11(c), a brightness region in a
steel flow zone is steel slag, and a black region is a background.
Since the ladle body is inclined downwards during slag tapping, the steel slag is
gathered below the ladle body, the steel flow is exposed to emit light, and the light
may be segmented into the steel slag by mistake during segmentation.
the steel slag cannot be completely separated from the molten steel by using a single
method when the color steel slag image is segmented, and a threshold obtained by
applying an Otsu algorithm is relatively small when there is a little difference in
variances of the steel slag light and a target part in the image, and thus, a
segmentation threshold is excessively low to result in false segmentation. An optimal
threshold selection formula in a traditional Otsu algorithm is analyzed and is modified
according to a proportion of the size of the background in the image on the basis of a
target variance weighting method, so that the problem that a part of the steel slag light
is classified as the target by mistake due to a slightly low threshold is solved, the
execution speed of the algorithm is increased, classification errors are reduced, and
the segmentation effect is improved. In Fig. 9(a) and Fig. 9(b), the brightness region
in the ladle body is steel slag, the brightness regions on the inner wall of and above
the ladle body are steel slag light, and a black region is the background. In Fig. 9(c), a
brightness region in a steel flow zone is steel slag, a darkness region is steel flow light,
and a black region is a background. It may be seen from Fig. 9(a) to Fig. 9(c) that the
steel slag may not be completely segmented by using the K-means clustering
algorithm, and a part of steel slag light is segmented into the target by mistake. In Fig.
11(a) and Fig. 11(b), the brightness regions on the inner wall of and above the ladle
body become small after being processed in the step 6. In Fig. 11(c), the darkness
region in the steel flow zone becomes small after being processed in the step 6. Seen
from Fig. 11(a) to Fig. 11(c), by using the improved Otsu algorithm, influences of the
steel slag light to the steel slag in the target region are reduced, and the steel slag may
be relatively accurately segmented.
In the embodiment, two types of images are adopted with one being images (provided
with a ladle) in a hot metal desulfurization slag removal process and the other one
being images (in a form of steel flow) in a converter slag tapping process. In Fig. 6,
Fig. 9 and Fig. 11 (a) and (b) are images in the hot metal desulfurization slag removal
process, and images (c) are images in the converter slag tapping process. Images
selected in Fig. 7 are images in the hot metal desulfurization slag removal process.
Step 7, a visualization operation is performed as follows.
Step 7.1, a main window is newly established.
Step 7.2, a ui file is converted into a py file.
Step 7.3, a function file is called.
Step 7.4, layout is performed by using a layout manager, and layout is performed on a
play button, a pause button, a classification button, a detection button and a
segmentation button.
During video playing, the "classification" button is clicked to recognize the color steel
slag image so that recognition precision may be displayed on a user interface; the
"detection" button is clicked so that a detection box may appear on the user interface
to automatically label a detected molten steel part and the confidence thereof; and the
"segmentation" button is clicked so that the segmented steel slag may be displayed on
the user interface.
It should be understood that parts which are not described in detail herein fall within
the prior art.
It should be understood that the above-mentioned preferred embodiments are
described in detail, but cannot be regarded as a limitation on the patent protection
scope of the present invention. Those of ordinary skill in the art may further perform
replacements or deformations on the preferred embodiments without departing from
the protection scope of claims of the present invention, these replacements or
deformations fall within the protection scope of the present invention, and the
requested protection scope of the present invention should be based on the appended
claims.
Claims (7)
1. An intelligent steel slag detection method based on a convolutional neural network,
characterized by comprising the steps sequentially performed as follows:
steel slag image recognition: acquiring a steel slag video frame image comprising
steel flow and steel slag brightness information by using an infrared detector and a
camera; and by taking a color steel slag image in the video frame image as an object,
recognizing the color steel slag image in the video frame image by adopting an image
recognition method based on an improved AlexNet convolutional neural network;
steel flow target detection: detecting steel flow information in the color steel slag
image, detecting a steel flow from a complex background by using a target detection
method based on a YOLOv3 convolutional neural network model, thereby accurately
detecting a slag inclusion condition of the steel flow; and
color steel slag image segmentation: with specific to problems that there is a
greater color difference between a color steel slag image target and a background
region, and steel slag light is segmented into steel slag by mistake, preprocessing the
color steel slag image by using a K-means clustering algorithm based on a Lab color
space, and completely separating the steel slag from molten steel by adopting an
improved Otsu image segmentation algorithm.
2. The intelligent steel slag detection method based on the convolutional neural
network according to claim 1, characterized in that in the step of steel slag image
recognition, the image recognition method based on the improved AlexNet
convolutional neural network specifically comprises the following steps:
selecting sample images to perform sample data classification, wherein a large
part of the sample images are used as a training set, a remaining part of the sample
images are used as a test set, the sample images are classified into positive samples and negative samples, the positive samples are named as "have" representing high steel slag content, and the negative samples are named as "no" representing low steel slag content; establishing a tensorflow deep learning framework and a keras deep learning framework, and creating an improved AlexNet convolutional neural network model by virtue of the two frameworks; preprocessing the images, and storing images in a data set by h5py so as to achieve flexible and efficient I/O data, high-capacity data and complex data; and importing os, numpy and matplotlib libraries in Python; defining a function to acquire a path list and a label list of the data set, and defining a type corresponding to a label as steel slag; converting all images in the training set and the test set into a numpy array; storing data of the training set and data of the test set into an h5 file; importing the data set, testing the data set, and performing visualization on a part of the data set; training the AlexNet convolutional neural network model, and improving the
AlexNet convolutional neural network model; and
acquiring optimal recognition precision by virtue of the trained AlexNet
convolutional neural network model.
3. The intelligent steel slag detection method based on the convolutional neural
network according to claim 1, characterized in that in the step of steel flow target
detection, the target detection method based on the YOLOv3 convolutional neural
network model comprises the following steps: firstly, normalizing the size of an input
steel slag image, and training the data set of the steel slag by virtue of a designed
network to obtain a convolutional neural network model; and then, acquiring the
confidence of a bounding box of a current steel slag target by virtue of the trained model, and classifying objects in the bounding box; and finally, filtering the bounding box by using a non-maximum suppression algorithm to obtain an optimal result.
4. The intelligent steel slag detection method based on the convolutional neural
network according to claim 1, characterized in that in the step of color steel slag
image segmentation, firstly, an RGB color space is converted into a uniform Lab color
space; then, the image is preprocessed by using the K-means clustering algorithm so
that the steel slag is primarily separated from a background; and finally, a proportion
of pixel points of the background to the overall image is used as a weight to be
introduced to a target equation, threshold segmentation is performed on the
preprocessed steel slag image by adopting the improved Otsu image segmentation
algorithm, and thus, the steel slag is relatively accurately separated from the steel
flow.
5. The intelligent steel slag detection method based on the convolutional neural
network according to claim 1, characterized in that threshold segmentation on the
preprocessed steel slag image by adopting the improved Otsu image segmentation
algorithm comprises the following specific steps:
converting the preprocessed color steel slag image into a grey scale image, and
calculating a grey scale histogram of the image;
setting a threshold t;
classifying the image into a background and a target, calculating grey scale mean
values of the target and the background, and acquiring an average grey scale mean
value according to the histogram;
traversing a grey scale according to an improved optimal threshold selection
formula, and determining a maximum function value t as an optimal threshold Th; and
performing image segmentation according to the optimal threshold, and outputting an segmented image.
6. The intelligent steel slag detection method based on the convolutional neural
network according to claim 1, characterized in that visual detection on the steel slag
by using a visual user interface system comprises the following steps:
newly establishing a main window;
converting a ui file into a py file;
calling a function file;
performing layout by using a layout manager, and performing layout on a play
button, a pause button, a classification button, a detection button and a segmentation
button; and
during video playing, clicking the "classification" button to recognize the steel
slag image, so as to display recognition precision on a user interface; clicking the
"detection" button, to make a detection box appear on the user interface to
automatically label a detected molten steel part and the confidence thereof; and
clicking the "segmentation" button, to display segmented steel slag on the user
interface.
7. An intelligent steel slag detection system based on a convolutional neural network,
characterized by comprising a color steel slag image recognition module, a color steel
slag image detection module, a color steel slag image segmentation module and a
visual user interface system;
the color steel slag image recognition module configured to learn, process and
analyze an image acquired by an infrared detector and a camera by virtue of an
improved AlexNet convolutional neural network as a steel slag image recognition
model framework, wherein the improved AlexNet convolutional neural network is
provided with five convolutional layers, five pooling layers and two fully connected layers in total, a RELU function is used as an activation function of a CNN
(Convolutional Neural Network), a part of neurons are randomly ignored by using
Dropout during training, and overlapped maximum pooling is used in the CNN;
the color steel slag image detection module configured to be based on a YOLOv3
convolutional neural network model and detect a color steel slag image recognized by
the color steel slag image recognition module by using a target detection method
based on the YOLOv3 convolutional neural network model so as to detect a steel flow
from a complex background;
the color steel slag image segmentation module configured to preprocess the
color steel slag image preprocessed by the color steel slag image detection module by
using a K-means clustering algorithm based on a Lab color space and segment the
color steel slag image by using an improved Otsu image segmentation algorithm so as
to completely separate steel slag from molten steel; and
the visual user interface system configured to perform visual operation by virtue
of buttons; and the visual user interface system comprising a menu system for newly
establishing a main window, converting a file and calling a converted file and further
comprising a layout manager for performing layout on a play button, a pause button, a
classification button, a detection button and a segmentation button, wherein during
video playing, the "classification" button is clicked to recognize the color steel slag
image so that recognition precision can be displayed on a user interface; the
"detection" button is clicked so that a detection box can appear on the user interface to
automatically label a detected molten steel part and the confidence thereof; and the
"segmentation" button is clicked so that segmented steel slag can be displayed on the
user interface.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910990000.3 | 2019-10-17 | ||
| CN201910990000.3A CN110796046B (en) | 2019-10-17 | 2019-10-17 | Intelligent steel slag detection method and system based on convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| AU2020102091A4 true AU2020102091A4 (en) | 2020-10-08 |
Family
ID=69439619
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| AU2020102091A Ceased AU2020102091A4 (en) | 2019-10-17 | 2020-09-01 | Intelligent steel slag detection method and system based on convolutional neural network |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110796046B (en) |
| AU (1) | AU2020102091A4 (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348791A (en) * | 2020-11-04 | 2021-02-09 | 中冶赛迪重庆信息技术有限公司 | Intelligent scrap steel detecting and judging method, system, medium and terminal based on machine vision |
| CN112750117A (en) * | 2021-01-15 | 2021-05-04 | 重庆邮电大学 | Blood cell image detection and counting method based on convolutional neural network |
| CN112766170A (en) * | 2021-01-21 | 2021-05-07 | 广西财经学院 | Self-adaptive segmentation detection method and device based on cluster unmanned aerial vehicle image |
| CN112949500A (en) * | 2021-03-04 | 2021-06-11 | 北京联合大学 | Improved YOLOv3 lane line detection method based on spatial feature coding |
| CN113191235A (en) * | 2021-04-22 | 2021-07-30 | 上海东普信息科技有限公司 | Sundry detection method, device, equipment and storage medium |
| CN113326734A (en) * | 2021-04-28 | 2021-08-31 | 南京大学 | Rotary target detection method based on YOLOv5 |
| CN113420716A (en) * | 2021-07-16 | 2021-09-21 | 南威软件股份有限公司 | Improved Yolov3 algorithm-based violation behavior recognition and early warning method |
| CN113569943A (en) * | 2021-07-26 | 2021-10-29 | 中铁工程装备集团有限公司 | Deep neural network-based slag piece bulk early warning method, system and device |
| CN113569777A (en) * | 2021-08-02 | 2021-10-29 | 中国石油天然气集团有限公司 | Video detection method and device for equipment rotation working state |
| CN113591591A (en) * | 2021-07-05 | 2021-11-02 | 北京瑞博众成科技有限公司 | Artificial intelligence field behavior recognition system |
| CN113688777A (en) * | 2021-09-07 | 2021-11-23 | 西北工业大学 | Airport pavement airplane real-time detection method based on embedded CPU |
| CN114037650A (en) * | 2021-05-17 | 2022-02-11 | 西北工业大学 | Ground target visible light damage image processing method for change detection and target detection |
| CN114359199A (en) * | 2021-12-28 | 2022-04-15 | 集美大学 | Fish counting method, device, equipment and medium based on deep learning |
| CN114492766A (en) * | 2022-02-25 | 2022-05-13 | 南京工程学院 | Convolutional neural network model design space McaNetX and optimization method thereof |
| CN114782400A (en) * | 2022-05-17 | 2022-07-22 | 东风本田发动机有限公司 | Method, device, equipment, medium and program product for detecting slag point of metal material |
| CN114774605A (en) * | 2022-03-15 | 2022-07-22 | 中南大学 | Intelligent forecasting device for iron notch blocking time of blast furnace |
| CN114798764A (en) * | 2022-03-01 | 2022-07-29 | 南京航空航天大学 | Method for measuring yield strength of cold-rolled strip steel on line based on multi-magnetic fusion |
| CN115029704A (en) * | 2022-08-09 | 2022-09-09 | 江苏贯森新材料科技有限公司 | Intelligent control method and device for stainless steel pickling process |
| CN115231525A (en) * | 2022-07-29 | 2022-10-25 | 福建德尔科技股份有限公司 | Intelligent separation and purification system for electronic-grade chlorine trifluoride |
| CN115691026A (en) * | 2022-12-29 | 2023-02-03 | 湖北省林业科学研究院 | Intelligent early warning monitoring management method for forest fire prevention |
| CN115841489A (en) * | 2023-02-21 | 2023-03-24 | 华至云链科技(苏州)有限公司 | Intelligent point inspection method and platform |
| CN115879834A (en) * | 2023-03-03 | 2023-03-31 | 北京科技大学 | Method for evaluating slag rolling behavior of continuous casting tundish |
| CN116127345A (en) * | 2022-12-23 | 2023-05-16 | 北京科技大学 | Converter steelmaking process mode design method based on deep clustering generation countermeasure network |
| CN116310713A (en) * | 2023-02-16 | 2023-06-23 | 嘉洋智慧安全科技(北京)股份有限公司 | Infrared image recognition method and device, electronic equipment and storage medium |
| CN116385466A (en) * | 2023-05-05 | 2023-07-04 | 北京信息科技大学 | Method and system for dividing targets in image based on boundary box weak annotation |
| CN116503420A (en) * | 2023-04-26 | 2023-07-28 | 佛山科学技术学院 | Image segmentation method based on federal learning and related equipment |
| CN116665138A (en) * | 2023-08-01 | 2023-08-29 | 临朐弘泰汽车配件有限公司 | Visual detection method and system for stamping processing of automobile parts |
| CN116681992A (en) * | 2023-07-29 | 2023-09-01 | 河南省新乡生态环境监测中心 | Ammonia nitrogen detection method based on neural network |
| CN116682067A (en) * | 2023-07-28 | 2023-09-01 | 中国华能集团清洁能源技术研究院有限公司 | Thermal runaway risk early warning method and device for battery pack of energy storage station and electronic equipment |
| CN116935229A (en) * | 2023-09-12 | 2023-10-24 | 山东博昂信息科技有限公司 | Method and system for identifying hook-in state of ladle hook |
| CN116958713A (en) * | 2023-09-20 | 2023-10-27 | 中航西安飞机工业集团股份有限公司 | Quick recognition and statistics method and system for surface fastener of aviation part |
| CN117351049A (en) * | 2023-12-04 | 2024-01-05 | 四川金信石信息技术有限公司 | Thermal imaging and visible light fusion measuring point registration guiding method, device and medium |
| CN117710962A (en) * | 2023-10-19 | 2024-03-15 | 天津立中车轮有限公司 | Material identification and counting method, device, equipment and storage medium |
| CN117746272A (en) * | 2024-02-21 | 2024-03-22 | 西安迈远科技有限公司 | Unmanned aerial vehicle-based water resource data acquisition and processing method and system |
| CN117788871A (en) * | 2023-12-26 | 2024-03-29 | 海南言发高科技有限公司 | Vehicle-mounted weighing management method and platform based on artificial intelligence |
| CN118038172A (en) * | 2024-03-11 | 2024-05-14 | 广东石油化工学院 | Temperature controller quality detection method and system based on feature enhancement and depth network |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111398291B (en) * | 2020-03-31 | 2023-04-18 | 南通远景电工器材有限公司 | Flat enameled electromagnetic wire surface flaw detection method based on deep learning |
| CN111523429A (en) * | 2020-04-16 | 2020-08-11 | 中冶赛迪重庆信息技术有限公司 | Deep learning-based steel pile identification method |
| CN115060012B (en) * | 2020-05-19 | 2023-10-31 | 山东大学 | Loop heat pipe solar energy temperature descaling method based on machine learning |
| CN111898467B (en) * | 2020-07-08 | 2023-02-28 | 浙江大华技术股份有限公司 | Attribute identification method and device, storage medium and electronic device |
| CN111881984A (en) * | 2020-07-30 | 2020-11-03 | 北京环境特性研究所 | Target detection method and device based on deep learning |
| CN111985406B (en) * | 2020-08-21 | 2023-05-23 | 武汉科技大学 | Zinc slag image identification and classification method |
| CN112011750B (en) * | 2020-08-21 | 2023-02-17 | 武汉科技大学 | Slag dragging method based on machine vision and robot system |
| CN112150421B (en) * | 2020-09-14 | 2023-07-25 | 武汉钢铁有限公司 | Ladle slag removing path determining method and device and slag removing machine |
| CN112364918B (en) * | 2020-11-10 | 2024-04-02 | 深圳力维智联技术有限公司 | Abnormality recognition method, terminal, and computer-readable storage medium |
| CN112381803B (en) * | 2020-11-17 | 2023-11-21 | 广东鉴面智能科技有限公司 | Molten iron overflow detection method and tapping machine system based on machine vision |
| CN112749735B (en) * | 2020-12-30 | 2023-04-07 | 中冶赛迪信息技术(重庆)有限公司 | Converter tapping steel flow identification method, system, medium and terminal based on deep learning |
| CN112950586A (en) * | 2021-03-02 | 2021-06-11 | 攀钢集团攀枝花钢铁研究院有限公司 | LF furnace steel slag infrared identification method and system |
| CN113042696B (en) * | 2021-03-12 | 2022-11-11 | 广东韶钢松山股份有限公司 | Crystallizer covering slag replacing device and crystallizer slag fishing method |
| CN113239960B (en) * | 2021-04-09 | 2024-05-28 | 中用科技有限公司 | Intelligent road protection early warning method and system integrating AI vision algorithm |
| CN113333700B (en) * | 2021-06-30 | 2023-01-06 | 广东韶钢松山股份有限公司 | Casting and pouring method for automatically inspecting liquid level of crystallizer |
| CN115558740A (en) * | 2021-07-02 | 2023-01-03 | 北京博谦工程技术有限公司 | Explosion-proof early warning judgment method in intelligent diagnosis system |
| CN114511453A (en) * | 2021-12-08 | 2022-05-17 | 上海威派格智慧水务股份有限公司 | Artificial intelligence water environment detecting system based on image recognition |
| CN114581859B (en) * | 2022-05-07 | 2022-09-13 | 北京科技大学 | Converter slag discharging monitoring method and system |
| CN115049678A (en) * | 2022-08-17 | 2022-09-13 | 南昌工程学院 | Transmission line corona discharge image segmentation method based on night ultraviolet imaging technology |
| CN115100492B (en) * | 2022-08-26 | 2023-04-07 | 摩尔线程智能科技(北京)有限责任公司 | Yolov3 network training and PCB surface defect detection method and device |
| CN116339128B (en) * | 2023-05-30 | 2023-07-28 | 北京国电富通科技发展有限责任公司 | Online monitoring method and system for slag falling rainfall of slag conveying system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102921915A (en) * | 2012-10-23 | 2013-02-13 | 杭州谱诚泰迪实业有限公司 | Slag carry-over detection method and device based on image recognition of vortex on surface of molten steel |
| CN105956618B (en) * | 2016-04-27 | 2021-12-03 | 云南昆钢集团电子信息工程有限公司 | Converter steelmaking blowing state identification system and method based on image dynamic and static characteristics |
| CN105911095A (en) * | 2016-05-04 | 2016-08-31 | 东北电力大学 | Visual recognition method of continuous casting billet surface longitudinal cracks |
| CA2948499C (en) * | 2016-11-16 | 2020-04-21 | The Governing Council Of The University Of Toronto | System and method for classifying and segmenting microscopy images with deep multiple instance learning |
| CN110231811B (en) * | 2019-06-19 | 2020-07-28 | 华北理工大学 | Continuous casting slag intelligent prediction system based on hierarchical real-time memory algorithm |
-
2019
- 2019-10-17 CN CN201910990000.3A patent/CN110796046B/en active Active
-
2020
- 2020-09-01 AU AU2020102091A patent/AU2020102091A4/en not_active Ceased
Cited By (61)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348791A (en) * | 2020-11-04 | 2021-02-09 | 中冶赛迪重庆信息技术有限公司 | Intelligent scrap steel detecting and judging method, system, medium and terminal based on machine vision |
| CN112348791B (en) * | 2020-11-04 | 2023-03-14 | 中冶赛迪信息技术(重庆)有限公司 | Intelligent scrap steel detecting and judging method, system, medium and terminal based on machine vision |
| CN112750117A (en) * | 2021-01-15 | 2021-05-04 | 重庆邮电大学 | Blood cell image detection and counting method based on convolutional neural network |
| CN112750117B (en) * | 2021-01-15 | 2024-01-26 | 河南中抗医学检验有限公司 | Blood cell image detection and counting method based on convolutional neural network |
| CN112766170A (en) * | 2021-01-21 | 2021-05-07 | 广西财经学院 | Self-adaptive segmentation detection method and device based on cluster unmanned aerial vehicle image |
| CN112766170B (en) * | 2021-01-21 | 2024-04-16 | 广西财经学院 | Self-adaptive segmentation detection method and device based on cluster unmanned aerial vehicle image |
| CN112949500A (en) * | 2021-03-04 | 2021-06-11 | 北京联合大学 | Improved YOLOv3 lane line detection method based on spatial feature coding |
| CN113191235B (en) * | 2021-04-22 | 2024-05-17 | 上海东普信息科技有限公司 | Sundry detection method, sundry detection device, sundry detection equipment and storage medium |
| CN113191235A (en) * | 2021-04-22 | 2021-07-30 | 上海东普信息科技有限公司 | Sundry detection method, device, equipment and storage medium |
| CN113326734A (en) * | 2021-04-28 | 2021-08-31 | 南京大学 | Rotary target detection method based on YOLOv5 |
| CN113326734B (en) * | 2021-04-28 | 2023-11-24 | 南京大学 | Rotational target detection method based on YOLOv5 |
| CN114037650A (en) * | 2021-05-17 | 2022-02-11 | 西北工业大学 | Ground target visible light damage image processing method for change detection and target detection |
| CN114037650B (en) * | 2021-05-17 | 2024-03-19 | 西北工业大学 | Ground target visible light damage image processing method for change detection and target detection |
| CN113591591A (en) * | 2021-07-05 | 2021-11-02 | 北京瑞博众成科技有限公司 | Artificial intelligence field behavior recognition system |
| CN113420716A (en) * | 2021-07-16 | 2021-09-21 | 南威软件股份有限公司 | Improved Yolov3 algorithm-based violation behavior recognition and early warning method |
| CN113420716B (en) * | 2021-07-16 | 2023-07-28 | 南威软件股份有限公司 | Illegal behavior identification and early warning method based on improved Yolov3 algorithm |
| CN113569943A (en) * | 2021-07-26 | 2021-10-29 | 中铁工程装备集团有限公司 | Deep neural network-based slag piece bulk early warning method, system and device |
| CN113569943B (en) * | 2021-07-26 | 2023-10-24 | 中铁工程装备集团有限公司 | Slag piece large-block early warning method, system and device based on deep neural network |
| CN113569777A (en) * | 2021-08-02 | 2021-10-29 | 中国石油天然气集团有限公司 | Video detection method and device for equipment rotation working state |
| CN113569777B (en) * | 2021-08-02 | 2024-08-09 | 中国石油天然气集团有限公司 | Video detection method and device for equipment rotation working state |
| CN113688777A (en) * | 2021-09-07 | 2021-11-23 | 西北工业大学 | Airport pavement airplane real-time detection method based on embedded CPU |
| CN113688777B (en) * | 2021-09-07 | 2024-09-06 | 西北工业大学 | Airport pavement plane real-time detection method based on embedded CPU |
| CN114359199A (en) * | 2021-12-28 | 2022-04-15 | 集美大学 | Fish counting method, device, equipment and medium based on deep learning |
| CN114492766A (en) * | 2022-02-25 | 2022-05-13 | 南京工程学院 | Convolutional neural network model design space McaNetX and optimization method thereof |
| CN114492766B (en) * | 2022-02-25 | 2024-10-29 | 南京工程学院 | Convolutional neural network model design space McaNetX and optimization method thereof |
| CN114798764A (en) * | 2022-03-01 | 2022-07-29 | 南京航空航天大学 | Method for measuring yield strength of cold-rolled strip steel on line based on multi-magnetic fusion |
| CN114774605B (en) * | 2022-03-15 | 2023-06-13 | 中南大学 | Intelligent forecasting device for blast furnace iron notch blocking time |
| CN114774605A (en) * | 2022-03-15 | 2022-07-22 | 中南大学 | Intelligent forecasting device for iron notch blocking time of blast furnace |
| CN114782400B (en) * | 2022-05-17 | 2023-06-20 | 东风本田发动机有限公司 | Method, device, equipment, medium and program product for detecting slag point of metal material |
| CN114782400A (en) * | 2022-05-17 | 2022-07-22 | 东风本田发动机有限公司 | Method, device, equipment, medium and program product for detecting slag point of metal material |
| CN115231525B (en) * | 2022-07-29 | 2022-12-23 | 福建德尔科技股份有限公司 | Intelligent separation and purification system for electronic-grade chlorine trifluoride |
| CN115231525A (en) * | 2022-07-29 | 2022-10-25 | 福建德尔科技股份有限公司 | Intelligent separation and purification system for electronic-grade chlorine trifluoride |
| CN115029704B (en) * | 2022-08-09 | 2022-12-09 | 江苏贯森新材料科技有限公司 | Intelligent control method and device for stainless steel pickling process |
| CN115029704A (en) * | 2022-08-09 | 2022-09-09 | 江苏贯森新材料科技有限公司 | Intelligent control method and device for stainless steel pickling process |
| CN116127345A (en) * | 2022-12-23 | 2023-05-16 | 北京科技大学 | Converter steelmaking process mode design method based on deep clustering generation countermeasure network |
| CN116127345B (en) * | 2022-12-23 | 2023-11-14 | 北京科技大学 | Converter steelmaking process mode design method based on deep clustering generation countermeasure network |
| CN115691026B (en) * | 2022-12-29 | 2023-05-05 | 湖北省林业科学研究院 | Intelligent early warning monitoring management method for forest fire prevention |
| CN115691026A (en) * | 2022-12-29 | 2023-02-03 | 湖北省林业科学研究院 | Intelligent early warning monitoring management method for forest fire prevention |
| CN116310713A (en) * | 2023-02-16 | 2023-06-23 | 嘉洋智慧安全科技(北京)股份有限公司 | Infrared image recognition method and device, electronic equipment and storage medium |
| CN116310713B (en) * | 2023-02-16 | 2024-03-19 | 嘉洋智慧安全科技(北京)股份有限公司 | Infrared image recognition method and device, electronic equipment and storage medium |
| CN115841489A (en) * | 2023-02-21 | 2023-03-24 | 华至云链科技(苏州)有限公司 | Intelligent point inspection method and platform |
| CN115879834A (en) * | 2023-03-03 | 2023-03-31 | 北京科技大学 | Method for evaluating slag rolling behavior of continuous casting tundish |
| CN116503420B (en) * | 2023-04-26 | 2024-05-14 | 佛山科学技术学院 | Image segmentation method based on federal learning and related equipment |
| CN116503420A (en) * | 2023-04-26 | 2023-07-28 | 佛山科学技术学院 | Image segmentation method based on federal learning and related equipment |
| CN116385466A (en) * | 2023-05-05 | 2023-07-04 | 北京信息科技大学 | Method and system for dividing targets in image based on boundary box weak annotation |
| CN116682067A (en) * | 2023-07-28 | 2023-09-01 | 中国华能集团清洁能源技术研究院有限公司 | Thermal runaway risk early warning method and device for battery pack of energy storage station and electronic equipment |
| CN116682067B (en) * | 2023-07-28 | 2023-10-20 | 中国华能集团清洁能源技术研究院有限公司 | Thermal runaway risk early warning method and device for battery pack of energy storage station and electronic equipment |
| CN116681992B (en) * | 2023-07-29 | 2023-10-20 | 河南省新乡生态环境监测中心 | Ammonia nitrogen detection method based on neural network |
| CN116681992A (en) * | 2023-07-29 | 2023-09-01 | 河南省新乡生态环境监测中心 | Ammonia nitrogen detection method based on neural network |
| CN116665138A (en) * | 2023-08-01 | 2023-08-29 | 临朐弘泰汽车配件有限公司 | Visual detection method and system for stamping processing of automobile parts |
| CN116665138B (en) * | 2023-08-01 | 2023-11-07 | 临朐弘泰汽车配件有限公司 | Visual detection method and system for stamping processing of automobile parts |
| CN116935229A (en) * | 2023-09-12 | 2023-10-24 | 山东博昂信息科技有限公司 | Method and system for identifying hook-in state of ladle hook |
| CN116958713A (en) * | 2023-09-20 | 2023-10-27 | 中航西安飞机工业集团股份有限公司 | Quick recognition and statistics method and system for surface fastener of aviation part |
| CN116958713B (en) * | 2023-09-20 | 2023-12-15 | 中航西安飞机工业集团股份有限公司 | Quick recognition and statistics method and system for surface fastener of aviation part |
| CN117710962A (en) * | 2023-10-19 | 2024-03-15 | 天津立中车轮有限公司 | Material identification and counting method, device, equipment and storage medium |
| CN117351049B (en) * | 2023-12-04 | 2024-02-13 | 四川金信石信息技术有限公司 | Thermal imaging and visible light fusion measuring point registration guiding method, device and medium |
| CN117351049A (en) * | 2023-12-04 | 2024-01-05 | 四川金信石信息技术有限公司 | Thermal imaging and visible light fusion measuring point registration guiding method, device and medium |
| CN117788871A (en) * | 2023-12-26 | 2024-03-29 | 海南言发高科技有限公司 | Vehicle-mounted weighing management method and platform based on artificial intelligence |
| CN117788871B (en) * | 2023-12-26 | 2024-08-09 | 海南言发高科技有限公司 | Vehicle-mounted weighing management method and platform based on artificial intelligence |
| CN117746272A (en) * | 2024-02-21 | 2024-03-22 | 西安迈远科技有限公司 | Unmanned aerial vehicle-based water resource data acquisition and processing method and system |
| CN118038172A (en) * | 2024-03-11 | 2024-05-14 | 广东石油化工学院 | Temperature controller quality detection method and system based on feature enhancement and depth network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110796046A (en) | 2020-02-14 |
| CN110796046B (en) | 2023-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2020102091A4 (en) | Intelligent steel slag detection method and system based on convolutional neural network | |
| CN113269073B (en) | Ship multi-target tracking method based on YOLO V5 algorithm | |
| CA2228062C (en) | Robustness of classification measurement | |
| CN112818952B (en) | Coal rock boundary recognition method and device and electronic equipment | |
| CN108921099A (en) | Moving ship object detection method in a kind of navigation channel based on deep learning | |
| Zhao et al. | Research on detection method for the leakage of underwater pipeline by YOLOv3 | |
| CN115294332B (en) | Image processing method, device, equipment and storage medium | |
| Inoue et al. | Crack detection as a weakly-supervised problem: Towards achieving less annotation-intensive crack detectors | |
| CN112241693A (en) | Illegal welding fire image identification method based on YOLOv3 | |
| Yu et al. | SignHRNet: Street-level traffic signs recognition with an attentive semi-anchoring guided high-resolution network | |
| Chen et al. | EEE-Net: Efficient edge enhanced network for surface defect detection of glass | |
| CN112561885B (en) | YOLOv 4-tiny-based gate valve opening detection method | |
| CN117437647B (en) | Oracle character detection method based on deep learning and computer vision | |
| CN116152576B (en) | Image processing method, device, equipment and storage medium | |
| CN114596242A (en) | Defect detection method, defect detection device, electronic equipment and computer readable storage medium | |
| CN112365440A (en) | Yoov 3-based mastopathy image mitosis detection device and method | |
| CN112364687A (en) | Improved Faster R-CNN gas station electrostatic sign identification method and system | |
| CN111414895A (en) | Face recognition method and device and storage equipment | |
| Rettenberger et al. | Uncertainty-aware particle segmentation for electron microscopy at varied length scales | |
| CN116524410A (en) | Deep learning fusion scene target detection method based on Gaussian mixture model | |
| CN118038026B (en) | Substation secondary wiring credible quality inspection method based on rotation target detection | |
| Chen | Bank Card Number Identification Program Based on Template Matching | |
| Smit et al. | Generation of metrics by semantic segmentation of high speed lightning footage using machine learning | |
| Li et al. | A Candy Defect Detection Method Based on StyleGAN2 and Improved YOLOv7 for Imbalanced Data | |
| Sharma et al. | Hand Gesture Recognition using OpenCV |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FGI | Letters patent sealed or granted (innovation patent) | ||
| MK22 | Patent ceased section 143a(d), or expired - non payment of renewal fee or expiry |