CN107516315B - Tunneling machine slag tapping monitoring method based on machine vision - Google Patents
Tunneling machine slag tapping monitoring method based on machine vision Download PDFInfo
- Publication number
- CN107516315B CN107516315B CN201710743406.2A CN201710743406A CN107516315B CN 107516315 B CN107516315 B CN 107516315B CN 201710743406 A CN201710743406 A CN 201710743406A CN 107516315 B CN107516315 B CN 107516315B
- Authority
- CN
- China
- Prior art keywords
- slag
- image
- monitoring
- belt
- load
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000002893 slag Substances 0.000 title claims abstract description 126
- 238000012544 monitoring process Methods 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000010079 rubber tapping Methods 0.000 title claims abstract description 37
- 230000005641 tunneling Effects 0.000 title claims abstract description 12
- 239000011435 rock Substances 0.000 claims abstract description 37
- 239000004575 stone Substances 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims description 23
- 238000001514 detection method Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 10
- 230000005484 gravity Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 4
- 230000004069 differentiation Effects 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000009826 distribution Methods 0.000 claims description 3
- 238000005096 rolling process Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims 1
- 238000007599 discharging Methods 0.000 abstract description 5
- 230000008859 change Effects 0.000 abstract description 4
- 230000018109 developmental process Effects 0.000 description 10
- 238000011161 development Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 238000001914 filtration Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 230000002708 enhancing effect Effects 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000003064 k means clustering Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000010438 granite Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000007620 mathematical function Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20152—Watershed segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30132—Masonry; Concrete
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a machine vision-based heading machine slag tapping monitoring method, which comprises the following steps: a surrounding rock slag discharging and crushing degree monitoring step, a conveying belt load monitoring step and a surrounding rock classification monitoring step; the slag discharging and crushing degree monitoring step is used for counting the maximum size, the minimum size and the average size of the stone slag and providing the surrounding rock crushing capacity under the current tunneling condition; in the step of monitoring the load of the conveyer belt, the load condition of the current belt conveyor is evaluated by detecting the depth of slag in the whole belt conveyor; and in the step of classifying and monitoring the surrounding rocks, judging the current tunneling surrounding rock condition through the slag discharge surface characteristics. The invention improves the efficiency and the accuracy of slag monitoring by using a machine vision method, avoids the problems of inaccurate monitoring and the like caused by severe environment change, and can also reduce the labor cost.
Description
Technical Field
The invention relates to the technical field of machine vision application, in particular to a slag tapping monitoring method of a heading machine based on machine vision.
Background
The 21 st century is an information age, and continuous innovation in production modes is needed for realizing modern production and information management. The application technology of the development machine follows the era pace, and strengthens automatic management so as to adapt to the requirements of informatization construction. However, the current detection system of the heading machine is based on various sensors, realizes data receiving and analyzing through wireless equipment or is a heading machine state monitoring system based on a virtual instrument; under severe environment, the sensor is influenced by environmental factors, so that the problems of inaccurate data acquisition or equipment damage and the like are caused, and the condition of the development machine is not favorably and accurately and stably monitored in real time; moreover, much manpower and material resources are required for equipment and maintenance.
In recent years, industrial manufacturing 2025 indicates that machine vision detection is a necessary trend of the development of the industry of China towards intellectualization, and has important significance on the development of the industry towards intellectualization and automation. The machine vision technology level is still continuously improved, and a machine vision system based on an embedded type becomes a future development direction; the embedded system can carry out real-time visual image acquisition and visual image processing control and has the characteristics of compact structure, low cost and low power consumption. The basic characteristics of the machine vision system are high speed, large information amount, high precision and non-contact, and the flexibility and the automation degree of production can be greatly improved. In some environments unsuitable for manual operation and occasions where manual vision is difficult to meet requirements or large-batch repetitive operation is available, the production efficiency can be greatly improved by replacing the manual vision with machine vision.
The integration of machine vision technology with other sensing technologies is also a trend, and multi-sensor technology can improve the reliability of the system in the aspects of detection, tracking and target identification. For the slag tapping detection of the heading machine, the detection environment is severe, the quality of the obtained image is poor, if the machine vision technology is combined with other sensor technologies, the defects of the image are made up by using the superiority of the sensor, and the slag tapping detection can be better completed.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a machine vision-based heading machine slag monitoring method, which improves the efficiency and accuracy of slag monitoring by using the machine vision method, avoids the problems of inaccurate monitoring and the like caused by severe environment change, and can also reduce the labor cost.
The purpose of the invention is realized by the following technical scheme:
a heading machine slag tapping monitoring method based on machine vision comprises the following steps: a surrounding rock slag discharging and crushing degree monitoring step, a conveying belt load monitoring step and a surrounding rock classification monitoring step; the slag discharging and crushing degree monitoring step is used for counting the maximum size, the minimum size and the average size of the stone slag and providing the surrounding rock crushing capacity under the current tunneling condition; in the step of monitoring the load of the conveyer belt, the load condition of the current belt conveyor is evaluated by detecting the depth of slag in the whole belt conveyor; and in the step of classifying and monitoring the surrounding rocks, judging the current tunneling surrounding rock condition through the slag discharge surface characteristics.
Specifically, the operation steps are as follows:
s1, carrying out preprocessing of enhancing and denoising on the slag image;
s2, monitoring the slag discharge and crushing degree of surrounding rock: on the basis of the preprocessed slag image, firstly carrying out binarization processing on the image, then carrying out reverse color processing, finally finding out a communication region with the largest area by marking a white communication region in the image, removing boundary white blocks, drawing a contour, wherein the middle white block is a slag block, calculating the maximum value, the minimum value, the average value and the standard deviation of the areas of all the middle white blocks, and evaluating the crushing degree of slag by counting the size of the slag block;
s3, a conveyor belt load monitoring step: after the crushing degree analysis of the slag is finished, based on the slag image preprocessed in the step S1, the position of a roller on a conveyor belt is identified and the roller is connected to obtain a belt side line, the tangent line of the slag and the conveyor belt junction is identified by utilizing a straight line detection algorithm, the top width L2 and the slag surface width L1 of the belt are further calculated, and finally the slag load degree is calculated:
h is the belt depth, and the load can be calculated according to the following formula:
wherein d represents the load percentage and is used for representing the load condition of the slag tapping belt;
s4, surrounding rock classification monitoring:
classifying the surrounding rocks by adopting a clustering algorithm aiming at the training samples, drawing an image gray value histogram aiming at the image preprocessed in the step S1, extracting the average gray value of the image gray value histogram and the peak value of the histogram as input variables of clustering, and acquiring the centroid point of each type as a classification criterion of test input after training;
when a new image is input, extracting the characteristics of the image, including the average gray value of the image gray value histogram and the peak value of the histogram, and then calculating the class of the image according to the following formula:
wherein Pi represents the centroid point of each class; p represents the feature point of the current input image; dis is the distance between the current feature point and the centroid-like point, and is used as the classification standard.
Preferably, the step S1 includes the following steps: firstly, carrying out gray level transformation on a slag tapping video image collected by a camera to obtain a slag tapping gradient image containing noise; then, segmenting the image by using a small probability strategy and a two-dimensional maximum inter-class variance method to obtain each region of the slag image: an interference noise region, a texture region and a smooth region; and finally, processing each region by adopting fractional calculus masks of different orders to obtain a self-adaptive de-noising and enhanced slag image.
Specifically, each region is processed by adopting fractional calculus masks of different orders to obtain a self-adaptive de-noising and enhanced slag image: according to the definition of the fractional order G-L, the differential operation is performed when the v order of the fractional order is a positive number, and the integral operation is performed when the v order of the fractional order is a negative number:
when v > 0, G-L defines the fractional differentiation of the v order as:
wherein,
the integral operator of fractional order under the definition of G-L is shown, the superscript G-L shows the definition of G-L, the superscript v shows the differentiation of v order, subscripts a and t show the upper bound and the lower bound of the integral expression, and a is the initial value of time t;
when v < 0, the fractional order integral under the definition of G-L is:
and carrying out convolution operation on the fractional order mask with the changing order and the image collected by the camera to obtain an enhanced slag image.
Preferably, in step S2, in consideration of the light imbalance of the slag tapping video image, the segmentation threshold binarization is applied to different parts of the slag tapping image according to the distribution characteristics of the image, so as to mark the white connected region in the slag tapping video image to the maximum extent.
Specifically, in step S2, the contour of the stone block is extracted by using an operator findContours provided by OpenCV, and the contour of the image after binarization thresholding can be selected; determining a gravity center point of a good contour by extracting an approximate contour of the stone, taking the gravity center point as a seed point of the stone region, carrying out appropriate corrosion and expansion operations according to the actual condition of the image, then obtaining a mark image which needs to be used next, and separating out the needed region by using a watershed algorithm; finally, the contoursArea function provided by OpenCV is used to find the area contained by the contour, sort the areas, find the maximum and minimum areas of the stone in each frame of the input image, and calculate the average area and the standard deviation of the areas.
Preferably, the value of the color component of the roller is increased from the whole image, and the image is thresholded using the single-channel image of the color component of the roller enhanced in step S1 as the processing target, the contour of the roller region is acquired, and the center of gravity of the roller contour is determined as the two end points of the straight line, and the two end points are connected to form the belt edge line.
Preferably, the training sample required to be provided in step S4 is { x }(1),…,x(m)}, each of which is
Without corresponding classification labels, in order to aggregate known samples into K classes and obtain the centroid point of the class as the classification criterion of the test input, the specific process of slag tapping image clustering can be expressed as follows:
step 1) randomly selecting K cluster centroids (cluster centroids) as
Step 2) the following process is repeated until convergence:
for sample i, the class to which it belongs is judged by calculating the distance:
c(i):＝argminj||x(i)-μj||2
for class j, the centroid point is calculated again from all samples belonging to this class:
k is the given number of classes required, c(i)Representing the class in which the sample i is less distant in comparison to the distances of the centroid points of the K classes, the centroid μjRepresentative is the center point of the sample belonging to the same class.
Compared with the prior art, the invention has the following advantages and beneficial effects:
the invention applies the machine vision detection technology to the slag tapping monitoring aspect of the development machine, and is used for monitoring indexes such as conveying belt load, slag block crushing degree, surrounding rock classification and identification in real time; the efficiency and the accuracy of slag monitoring are improved by using a machine vision method, the problems of inaccurate monitoring and the like caused by severe environment change are avoided, and meanwhile, the labor cost can be reduced.
Drawings
FIG. 1 is a schematic diagram of slag tapping monitoring image shooting of the heading machine in the embodiment.
Fig. 2 is a general flow chart of a slag tapping monitoring system of the heading machine in the embodiment.
FIG. 3 is a flowchart of a fractional order denoising and enhancing method for a slag tapping image in the embodiment.
FIG. 4 is a flow chart of slag crushing degree monitoring in the example.
FIG. 5 is a schematic view of the slag crushing degree monitoring process in the example.
FIG. 6 is a flowchart of load monitoring in an embodiment.
FIG. 7 is a schematic diagram of machine vision-based identification of belt edge information in an embodiment.
FIG. 8 is a sectional analysis view of the tapping belt conveyor in the example.
FIG. 9 is a flow chart of the classification of surrounding rocks in the example.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.
Examples
The visual techniques used in processing video images mainly involve the following categories of techniques: 1) enhancement of the image: the fractional order image enhancement algorithm enhances the edge information of the detection target, and is convenient to extract; 2) extracting the contour of the image: extracting the outline of a detected target, and determining the position and area information of the target; 3) k-means clustering algorithm: extracting image features and classifying the images; 4) straight line detection of the image: the borderline straight line of the conveying belt and the slag discharging area is detected, the detection area is obtained, the interference caused by the environment is reduced, the processing speed is improved, and the like.
An OpenCV image processing API function library and VS2010 integrated development environment: OpenCV is an open-source image processing function library, is developed by C and C + + languages, and comprises an image processing library and a machine vision algorithm library, and can be used in a cross-platform manner, functions provided by OpenCV can facilitate programmers to call the function library to realize image algorithms, and provide multiple interfaces, image processing interfaces and matrix operation interfaces for realizing the image algorithms, and meanwhile, the functions also cover many high-level mathematical functions such as fourier transform, integral operation, differential operation and the like, and the functions of camera modeling can be calibrated.
Visual Studio Integrated Development Environment (IDE) under Windows platform is one of the first choice for software development. The Visual Studio 2010 provides a new template, a design tool and a test and debugging tool for a client, optimization upgrading is also carried out on Web development, and an API function provided by the VS2010 enables a user to conveniently use Windows system resources.
The embodiment provides a tunneling machine slag-out monitoring method based on machine vision, as shown in fig. 1, image data of a slag-out conveyor belt is collected through a camera, and the method is used for identifying the slag-out area, the conveyor belt load and surrounding rock classification of a processing area.
Fig. 2 is a general flow chart of the method, which is divided into three aspects of monitoring the slag discharge and crushing degree of the surrounding rock, monitoring the load of a conveying belt and monitoring the classification of the surrounding rock. And the slag crushing degree monitoring module is used for counting the maximum size, the minimum size and the average size of the stone slag and providing the capacity of crushing the surrounding rock under the current tunneling condition. The conveyer belt load monitoring module evaluates the load condition of the current belt conveyor by detecting the depth of slag in the whole belt conveyor. And the surrounding rock classification monitoring module judges the current tunneling surrounding rock condition through the slag discharge surface characteristics. The method comprises the following specific steps:
s1, because the belt conveyor runs at a high speed (the speed is about 2 m/S), the obtained slag image has certain fuzziness, and therefore the slag image needs to be subjected to preprocessing of enhancing and denoising.
Fig. 3 is a flow chart of performing fractional order image enhancement on an acquired video surveillance image.
Firstly, carrying out gray level transformation on a slag tapping video image collected by a camera to obtain a slag tapping gradient image containing noise; then, segmenting the image by using a small probability strategy and a two-dimensional maximum inter-class variance method to obtain each region of the slag image: and finally, processing each area by adopting fractional calculus masks of different orders to obtain a self-adaptive de-noising and enhanced slag image.
The slag image is inevitably affected by noise in the process of collection and transmission, so that the image information is uncertain, and the subsequent image processing process is difficult. Although the common methods such as non-local mean filtering, kalman filtering, wavelet image denoising, median filtering, low-pass filtering, wiener filtering and the like have a certain denoising effect, the image denoising algorithms directly or indirectly adopt integer order integration in the construction of a denoising model, so that the texture information of an image is lost while the noise is removed. The fractional order integral is utilized to carry out denoising processing on the image without estimating the noise variance of the image in advance, and filtering processing is directly carried out, so that compared with other denoising algorithms, the fractional order integral algorithm has higher efficiency in the aspect of image denoising and has better effect in the aspect of retaining image texture detail information.
In order to effectively extract the characteristics of the slag image, the method utilizes a small probability strategy and a two-dimensional maximum inter-class variance method to segment the image, and adopts fractional calculus masks of different orders to process each region to obtain the self-adaptive denoised and enhanced slag image. According to the definition of the fractional order G-L, the differential operation is performed when the v order of the fractional order is a positive number, and the integral operation is performed when the v order of the fractional order is a negative number:
when v > 0, Griinwald-Letnikov (G-L) defines the fractional differential of order v as:
wherein,
the integral operator of fractional order under the definition of G-L is shown, the superscript G-L shows the definition of G-L, the superscript v shows the differentiation of v order, subscripts a and t show the upper and lower bounds of the integral expression, and a is the initial value of time t.
When v < 0, the fractional order integral under the definition of G-L is:
the enhanced slag image can be obtained by performing convolution operation on the fractional order mask with the change order and the image collected by the camera, the enhanced image has reduced noise, clearer texture and more obvious edge.
TABLE 1G-L Definitions of masks
| v(v-1)/2 | 0 | v(v-1)/2 | 0 | v(v-1)/2 |
| 0 | -v | -v | -v | 0 |
| v(v-1)/2 | -v | 8 | -v | v(v-1)/2 |
| 0 | -v | -v | -v | 0 |
| v(v-1)/2 | 0 | v(v-1)/2 | 0 | v(v-1)/2 |
As shown in table 1, a mask of 5 x 5 giving G-L defined fractional steps achieves fractional step image enhancement.
S2, and FIG. 4 is a flow chart of slag crushing degree monitoring. Based on the preprocessed slag image, as shown in fig. 5, firstly, binarization processing is performed on the image, then, reverse color processing is performed, finally, a white connected region in the image is marked, a connected region with the largest area is found, a boundary white block is removed (the white connected region with the largest area is the boundary white block), a contour is drawn, the middle white block is a slag block, the maximum value, the minimum value, the average value and the standard deviation of the areas of all the middle white blocks are calculated, and the crushing degree of slag is evaluated by counting the size of the slag block.
In consideration of the light imbalance of the slag tapping video image, according to the distribution characteristics of the image, the present embodiment performs segmentation threshold binarization on different parts of the slag tapping image, so as to mark a white connected region in the slag tapping video image to the maximum extent.
The method utilizes an operator findContours () provided by OpenCV to extract the outline of the stone block, and can select the outline of the image after binarization thresholding; the method comprises the steps of determining a center of gravity point of a good contour by extracting an approximate contour of a stone block, using the center of gravity point as a seed point of the stone block region, carrying out appropriate corrosion and expansion operations according to the actual situation of an image, then obtaining a mark image which needs to be used next, and separating out the needed region by utilizing a watershed algorithm. Finally, areas contained by the contour are obtained by using a contoursArea () function provided by OpenCV, the areas are ranked, the maximum and minimum areas of the stone in each frame of the input image are found, and the average area and the standard deviation of the areas are calculated.
S3 and fig. 6 are flowcharts of load level monitoring. After the crushing degree analysis of slag discharge is completed, based on a slag discharge video image, the tangent line of the outer edge of the conveying belt is obtained by identifying the position and the connection of a red rolling shaft on the conveying belt, the tangent line of the boundary of the slag discharge and the conveying belt is identified by Hough transform (linear detection algorithm), the upper top width L2 of the belt and the slag discharge surface width L1 are further calculated, and finally the load degree of the slag discharge is calculated.
As shown in fig. 7, the value of the red component of the roller is increased from the whole image, the single-channel image of the red component enhanced in step S1 is used as the processing target, the image is thresholded to obtain the contour of the roller region, and the center of gravity of the roller contour is determined as the two end points of the straight line, and the two end points are connected to form the belt edge line. The image profile structure shown in FIG. 8 was obtained, L2 being the belt top width, L1 being the slag tapping surface width, and H being the belt depth (H varies little with load and is considered to be constant by dynamic estimation). From the profile, the load can be calculated according to the following formula:
wherein d represents the percentage load, which is characteristic of the load condition of the tapping belt. The tapping load percentage can be dynamically estimated according to equation (3).
S4 and FIG. 9 are flow charts of surrounding rock classification. According to the method, the surrounding rocks are classified by adopting a K-means clustering algorithm, an image gray value histogram is drawn for the image preprocessed in the step S1, the average gray value of the image gray value histogram and the peak value of the histogram are extracted as input variables of clustering, and the centroid point of each type is obtained after training.
The training sample to be provided is { x }(1),…,x(m)}, each of which is
There is no corresponding class flag. In order to aggregate known samples into K classes and obtain the centroid point of the class as the classification criterion of the test input, the specific process of slag image clustering can be expressed as follows:
step 1) randomly selecting K cluster centroids (cluster centroids) as
Step 2) the following process is repeated until convergence:
for sample i, the class to which it belongs is judged by calculating the distance:
c(i):＝argminj||x(i)-μj||2
for class j, the centroid point is calculated again from all samples belonging to this class:
k is the given number of classes required, c(i)The class is represented where sample i is less distant in comparison to the distance of the centroid point of the K classes. Centroid mujRepresentative is the center point of the sample belonging to the same class.
Therefore, when a new image is input, the features of the image (including the average gray value of the image gray value histogram and the peak value of the histogram) are extracted, and then the class to which the image belongs is calculated according to the following formula:
wherein Pi represents the centroid point of each class; p represents the feature point of the current input image; dis is the distance between the current feature point and the centroid-like point, and is used as the classification standard.
Based on the clustering method given in the steps, classification and identification are carried out on the slag discharged by the heading machine, the classification and identification mainly comprise types of granite, soil, igneous rock, sedimentary rock, metamorphic rock and the like, and the classification condition of the surrounding rock is estimated based on the gray value analysis of the monitoring picture.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (8)
1. A heading machine slag tapping monitoring method based on machine vision is characterized by comprising the following steps:
carrying out enhancement and denoising pretreatment on the slag image;
monitoring the slag discharge crushing degree of the surrounding rock, counting the maximum size, the minimum size and the average size of the stone slag, and providing the surrounding rock crushing capability under the current tunneling condition;
a step of monitoring the load of the conveyer belt, which is to evaluate the load condition of the current belt conveyor by detecting the depth of slag in the whole belt conveyor;
and a surrounding rock classification monitoring step, namely judging the current tunneling surrounding rock condition through the slag surface characteristics;
wherein, in the conveyor belt load monitoring step: based on the preprocessed slag image, acquiring a belt side line by identifying the position of a rolling shaft on a conveying belt and connecting the rolling shaft with the conveying belt, and identifying a tangent line of a boundary of slag and the conveying belt by utilizing a linear detection algorithm so as to obtain the top width of the belt and the surface width of the slag, and finally acquiring the load degree of the slag;
in the step of monitoring the surrounding rock classification:
classifying the surrounding rocks by adopting a clustering algorithm aiming at the training samples, drawing an image gray value histogram aiming at the preprocessed image, extracting an average gray value of the image gray value histogram and a peak value of the histogram as input variables of clustering, and acquiring a centroid point of each class as a classification criterion of test input after training;
when a new image is input, extracting the characteristics of the image, including the average gray value of the image gray value histogram and the peak value of the histogram, and then calculating the class of the image according to the following formula:
wherein Pi represents the centroid point of each class; p represents the feature point of the current input image; pi.x represents the centroid point abscissa for each class; pi.y represents the centroid point ordinate of each class; p.x, P.y, and dis are the distances between the current feature point and the centroid-like point, which are used as the classification criteria.
2. The machine vision based heading machine slag tapping monitoring method of claim 1,
and (3) monitoring the slag discharge crushing degree of the surrounding rock: on the basis of the preprocessed slag image, firstly carrying out binarization processing on the image, then carrying out reverse color processing, finally finding out a communication region with the largest area by marking a white communication region in the image, removing boundary white blocks, drawing a contour, wherein the middle white block is a slag block, calculating the maximum value, the minimum value, the average value and the standard deviation of the areas of all the middle white blocks, and evaluating the crushing degree of slag by counting the size of the slag block;
in the step of monitoring the load of the conveying belt, after the analysis of the crushing degree of the slag discharge is completed, the calculation formula of the slag discharge load degree is as follows:
h is the belt depth, and the load can be calculated according to the following formula:
wherein d represents the load percentage for representing the load condition of the slag tapping belt, α represents the angle formed by the upper surface of the belt and the side edge of the belt, L1 represents the width of the slag tapping surface, and L2 represents the width of the upper top of the belt.
3. The machine vision-based heading machine slag tapping monitoring method according to claim 1, wherein the preprocessing step is specifically: firstly, carrying out gray level transformation on a slag tapping video image collected by a camera to obtain a slag tapping gradient image containing noise; then, segmenting the image by using a small probability strategy and a two-dimensional maximum inter-class variance method to obtain each region of the slag image: an interference noise region, a texture region and a smooth region; and finally, processing each region by adopting fractional calculus masks of different orders to obtain a self-adaptive de-noising and enhanced slag image.
4. The machine vision-based heading machine slag monitoring method of claim 3, wherein fractional calculus masks of different orders are used to process each region to obtain a self-adaptive denoised and enhanced slag image: according to the definition of the fractional order G-L, the differential operation is performed when the v order of the fractional order is a positive number, and the integral operation is performed when the v order of the fractional order is a negative number:
when v > 0, G-L defines the fractional differentiation of the v order as:
wherein,
the fractional order differential operator under the definition of G-L is represented, the superscript G-L represents the definition of G-L, the superscript v represents solving the differential of order v, subscripts a and t represent the upper bound and the lower bound of a differential expression, a is the initial value of time t, h represents a number with the limit tending to 0, j is a variable, and the value of the variable is from 0 to (t-a)/h; f (t) represents a one-dimensional function of the slag image relative to t, f (t-jh) represents a one-dimensional function of the slag image relative to t-jh, h-vRepresents the negative v power of h;
when v < 0, the fractional order integral under the definition of G-L is:
wherein,
representing an integral operator under the definition of G-L;
and carrying out convolution operation on the fractional order mask with the changing order and the image collected by the camera to obtain an enhanced slag image.
5. The machine vision-based heading machine slag monitoring method according to claim 2, wherein in the surrounding rock slag crushing degree monitoring step, in consideration of light imbalance of the slag video image, segmented threshold binarization is adopted for different parts of the slag image according to distribution characteristics of the image, so as to mark a white connected region in the slag video image to the maximum extent.
6. The tunneling machine slag monitoring method based on the machine vision is characterized in that in the surrounding rock slag crushing degree monitoring step, an operator findContours provided by OpenCV is used for extracting the contour of a slag block, and the contour of an image subjected to binarization thresholding can be selected; determining a gravity center point of a good contour by extracting the approximate contour of the slag block, taking the gravity center point as a seed point of the slag block, corroding and expanding the image to obtain a mark image which needs to be used next, and separating out a needed area by utilizing a watershed algorithm; finally, the areas included by the contour are obtained by using a contoursArea function provided by OpenCV, the areas are sorted, the maximum and minimum areas of the slag blocks in each frame of input image are found, and the average area and the standard deviation of the areas are calculated.
7. The machine vision-based slag tapping monitoring method for the heading machine according to claim 2, wherein in the belt load monitoring step, the color component value of the roller is increased in consideration of the entire image, the image is thresholded using the single-channel image of the color component of the roller enhanced in the preprocessing step as a processing object, the profile of the roller region is obtained, and the center of gravity point of the roller profile is obtained as two end points of a straight line to connect the two end points to form a belt edge line.
8. The machine vision-based heading machine slag tapping monitoring method according to claim 1, wherein the training sample to be provided in the surrounding rock classification monitoring step is { x }(1)，...，x(m)}, each of which is
Without corresponding classification marks, in order to aggregate known samples into K classes and obtain the centroid points of the classes as the classification criteria of the test input, the specific process of slag tapping image clustering is as follows:
step 1) randomly selecting K clustering centroid points as
Step 2) the following process is repeated until convergence:
for sample i, the class to which it belongs is judged by calculating the distance:
c(i)：＝arg minj||x(i)-μj||2
for class j, the centroid point is calculated again from all samples belonging to this class:
k is the given number of classes required, c(i)Representing the class in which the sample i is less distant in comparison to the distances of the centroid points of the K classes, the centroid μjRepresenting the center points of the samples belonging to the same class, function 1 c(i)J represents when c(i)The output is 1 when j is equal, otherwise 0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710743406.2A CN107516315B (en) | 2017-08-25 | 2017-08-25 | Tunneling machine slag tapping monitoring method based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710743406.2A CN107516315B (en) | 2017-08-25 | 2017-08-25 | Tunneling machine slag tapping monitoring method based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107516315A CN107516315A (en) | 2017-12-26 |
| CN107516315B true CN107516315B (en) | 2020-04-28 |
Family
ID=60724315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710743406.2A Active CN107516315B (en) | 2017-08-25 | 2017-08-25 | Tunneling machine slag tapping monitoring method based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107516315B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734703B (en) * | 2018-04-26 | 2020-12-01 | 广东道氏技术股份有限公司 | Polished tile printing pattern detection method, system and device based on machine vision |
| CN109002777B (en) * | 2018-06-29 | 2021-03-30 | 电子科技大学 | Infrared small target detection method for complex scene |
| CN109305534B (en) * | 2018-10-25 | 2020-03-13 | 安徽理工大学 | Self-adaptive control method of belt conveyor of coal wharf based on computer vision |
| CN110188624A (en) * | 2019-05-10 | 2019-08-30 | 国网福建省电力有限公司龙岩供电公司 | A kind of substation's wind drift recognition methods and system based on deep learning |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104502990A (en) * | 2015-01-06 | 2015-04-08 | 中国地质大学（武汉） | Geological survey method of tunnel face based on digital image |
| CN206016814U (en) * | 2016-09-18 | 2017-03-15 | 中铁工程装备集团有限公司 | A kind of record and the device for analyzing TBM slag pieces |
-
2017
- 2017-08-25 CN CN201710743406.2A patent/CN107516315B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104502990A (en) * | 2015-01-06 | 2015-04-08 | 中国地质大学（武汉） | Geological survey method of tunnel face based on digital image |
| CN206016814U (en) * | 2016-09-18 | 2017-03-15 | 中铁工程装备集团有限公司 | A kind of record and the device for analyzing TBM slag pieces |
Non-Patent Citations (4)
| Title |
|---|
| Ｄetermination of geological strength index of jointed rock mass based on image processing;Kunui Hong et al.;《Journal of Rock Mechanics and Geotechnical Engineering》;20170710;第702-708页 * |
| 双护盾TBM施工隧洞围岩分类方法及应用;袭著晖 et al.;《土工基础》;20140630;第28卷(第3期);第117页 * |
| 基于分数阶微分的图像增强;杨柱中 等;《计算机辅助设计与图形学学报》;20080331;第20卷(第3期);第343-348页 * |
| 基于小概率策略的Otsu图像分割方法;范朝东 等;《电子与信息学报》;20130930;第35卷(第9期);第2081-2087页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107516315A (en) | 2017-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN110334706B (en) | Image target identification method and device | |
| Jahedsaravani et al. | An image segmentation algorithm for measurement of flotation froth bubble size distributions | |
| CN109101924B (en) | Machine learning-based road traffic sign identification method | |
| CN113781402A (en) | Method and device for detecting chip surface scratch defects and computer equipment | |
| CN107516315B (en) | Tunneling machine slag tapping monitoring method based on machine vision | |
| CN108711149B (en) | Mineral rock granularity detection method based on image processing | |
| CN110827235B (en) | Steel plate surface defect detection method | |
| CN114972356B (en) | Plastic product surface defect detection and identification method and system | |
| CN114332026B (en) | Visual detection method and device for scratch defects on surface of nameplate | |
| CN113393426B (en) | Steel rolling plate surface defect detection method | |
| CN109781737B (en) | Detection method and detection system for surface defects of hose | |
| CN102346854A (en) | Method and device for carrying out detection on foreground objects | |
| CN115115612A (en) | Surface defect detection method and system for mechanical parts | |
| CN114926410A (en) | Method for detecting appearance defects of brake disc | |
| CN115311507B (en) | Building board classification method based on data processing | |
| CN112233116A (en) | Concave-convex mark visual detection method based on neighborhood decision and gray level co-occurrence matrix description | |
| CN111476804A (en) | Method, device and equipment for efficiently segmenting carrier roller image and storage medium | |
| CN116630813A (en) | Highway road surface construction quality intelligent detection system | |
| CN116309577B (en) | Intelligent detection method and system for high-strength conveyor belt materials | |
| CN110175556B (en) | Remote sensing image cloud detection method based on Sobel operator | |
| Zhao et al. | Recognition of flooding and sinking conditions in flotation process using soft measurement of froth surface level and QTA | |
| CN115100196A (en) | Method for evaluating derusting effect of stamping part based on image segmentation | |
| CN115294379B (en) | Flotation method foam identification method based on optical information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |