Disclosure of Invention
The present invention provides a method for identifying ores rapidly, which uses a processor to perform automatic optical identification of ores, and thus has the advantages of high production efficiency, low cost and high separation rate.
The technical scheme adopted by the invention for solving the technical problems is as follows: a rapid ore identification method is constructed, and comprises the following steps:
s1, obtaining an ore training image and training the ore training image based on deep learning to obtain a plurality of modeling coefficients;
s2, constructing a processing model based on the modeling coefficients;
s3, testing the ore image to be tested based on the processing model to obtain a mask image;
and S4, obtaining an ore image based on the ore image to be detected and the mask image to identify the ore.
In the ore rapid identification method according to the present invention, the step S1 further includes:
s11, acquiring an ore training image and extracting characteristic properties and ore space coordinates of the ore training image;
s12, obtaining a property saliency map of the ore training image based on the feature property structure and obtaining a position saliency map of the ore training image based on the ore spatial coordinates;
s13, calculating the plurality of modeling coefficients based on the property saliency map and the position saliency map.
In the ore rapid identification method according to the present invention, the step S12 further includes:
s121, constructing a plurality of characteristic property graphs of the ore training image based on a plurality of characteristic properties by adopting a Gaussian pyramid and a central peripheral difference algorithm;
s122, obtaining a plurality of property saliency maps of the ore training image based on the plurality of characteristic property maps by adopting cross-scale combination and normalization operators; and
and S123, obtaining a position saliency map of the ore training image based on the ore space coordinate by adopting two-dimensional Gaussian distribution.
In the ore rapid identification method according to the present invention, the step S13 further includes:
s131, calculating the modeling coefficients based on the property saliency maps and the position saliency map by adopting a scale invariant feature transformation algorithm.
In the method for quickly identifying ores, the characteristic properties comprise: color, brightness, transparency, and reflectivity.
In the ore rapid identification method according to the present invention, the step S2 further includes:
s21, constructing the processing model based on the modeling coefficients by using a convolutional neural network, wherein the convolutional neural network uses a ReLU activation function.
In the ore rapid identification method according to the present invention, the step S3 further includes:
s31, scanning by using a CCD area-array camera to obtain an image of the ore to be detected;
s32, testing the ore image to be tested by adopting the processing model to generate a test saliency map;
s33, optimizing the test saliency map to generate the mask image;
wherein the optimization process comprises a threshold segmentation, a morphology process, and a median filtering process.
In the method for quickly identifying ore in the present invention, the step S4 further includes
S41, performing corner point detection on the ore image to be detected to obtain a corner point feature point set;
s42, multiplying the corner point feature point set and the mask image to obtain an ore image;
s43, carrying out segmentation processing on the ore image to obtain an ore image area and a background area;
and S44, identifying the ore image area as the ore.
In the ore rapid identification method according to the present invention, the step S4 further includes:
s45, converting the position of the ore image area into the motion position of the ore;
and S46, driving a spraying device to spray the ore based on the movement position.
Another technical solution adopted by the present invention to solve the technical problem is to construct a computer-readable storage medium, on which a computer program is stored, wherein the program is executed by a processor to implement the ore rapid identification method.
By implementing the ore rapid identification method and the computer readable storage medium, the ore can be identified rapidly, accurately and automatically with high precision by identifying the ore based on the processing model obtained by deep learning training. Furthermore, more ore characteristic properties can be obtained at the same time by adopting the CCD area-array camera, the spatial resolution is improved, and the processing capacity is large. And furthermore, a processing model is constructed based on the convolutional neural network adopting the ReLU activation function, so that the ore image to be detected which is displaced and deformed can be more effectively identified, and the identification accuracy is further improved.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention relates to a method for quickly identifying ores. Aspects of the invention include obtaining an ore training image and training the ore training image based on deep learning to obtain a plurality of modeling coefficients; constructing a process model based on the plurality of modeling coefficients; and obtaining an ore image based on the ore image to be detected and the mask image to identify the ore. By implementing the ore rapid identification method and the computer readable storage medium, the ore can be identified rapidly, accurately and automatically with high precision by identifying the ore based on the processing model obtained by deep learning training.
Fig. 1 is a flowchart of a first embodiment of the ore rapid identification method of the present invention. As shown in fig. 1, the ore rapid identification method of the present invention includes the following steps. In step S1, an ore training image is acquired and trained based on deep learning to obtain a plurality of modeling coefficients. In a preferred embodiment of the present invention, industrial cameras, such as area cameras and line cameras, may be used for photographing. The captured ore training images are then transferred directly to a processor, such as a computer, via a gigabit, camera link, USB3.0, or other interface. The computer then uses these ore training images, which are trained based on deep learning, to construct the required modeling coefficients. For example, a large number of samples can be taken from ore training images belonging to different mines, a selectable relation between characteristic parameters such as color, brightness, transparency and reflectivity of the stone to be tested and whether the stone to be tested is an ore or not is established in a self-organizing manner, and a required modeling coefficient is constructed through learning and training.
In the step S2, a process model is constructed based on the plurality of modeling coefficients. For example, in a preferred embodiment of the present invention, a convolutional neural network may be employed to construct the process model based on the plurality of modeling coefficients. In the present invention, since the identification of the ore is done by treating the image identification problem as an energy function minimization problem, the energy function determines the network structure. The basic idea of image recognition segmentation based on the neural network is to obtain a linear decision function by training a multilayer perceptron, and then classify pixels by using the decision function to achieve the purpose of segmentation. Neural networks are typically trained using training samples. Image recognition can be viewed as a Constraint Satisfaction Problem (CSP) and solved using constraint satisfaction neural networks. The recognition of the mode signal by the neuro cognitive machine is much stronger than that by the cognitive machine, and the change of transformation, conversion, distortion and size of the signal can be processed. The neuro-cognitive machine decomposes a visual pattern into many sub-patterns (features) and then enters into a feature plane connected in a hierarchical manner for processing, which attempts to model the visual system so that it can perform recognition even when the object is displaced or slightly deformed. Generally, neurocognitive machines comprise two types of neurons, the S-element responsible for feature extraction and the anti-deforming C-element. The S-element involves two important parameters, namely a receptive field and a threshold parameter, the former determines the number of input connections, and the latter controls the degree of response to the characteristic sub-pattern. Thus, in a preferred embodiment of the present invention, a convolutional neural network LeNet-5 may be employed.
In step S3, the ore image under test is tested based on the process model to obtain a mask image. In a preferred embodiment of the present invention, industrial cameras, such as area cameras and line cameras, may be used for photographing. The captured image is then transferred directly to a processor, such as a computer, via a gigabit network, camera link, USB3.0, or the like interface. These images can then be used as the image of the ore to be measured. After the ore images to be tested are tested by the processing model, the mask images can be directly generated. Of course, in other preferred embodiments of the present invention, the generated image may be optimized, and then the optimized image is used as the mask image.
In step S4, an ore image is obtained based on the ore image to be measured and the mask image to identify an ore. In a preferred embodiment of the present invention, the ore image to be tested may be preprocessed, or the ore image to be tested and the mask image may be directly multiplied to directly obtain the ore image. The ore image can be directly segmented into an ore image area and a background area. The ore image region is then identified as ore. In a further preferred embodiment of the invention, it is also possible to convert the position of the ore image area into a movement position of the ore, and then drive the ejection means to eject the ore based on the movement position.
It is understood by those skilled in the art that the steps S1-S2 and the test identification steps S3-S4 of establishing the process model do not necessarily need to be performed in a sequential order in the present invention. For example, steps S1-S2 may be performed first to obtain a suitable process model, which is then reused to perform steps S3-S4 to complete ore identification.
By implementing the rapid ore identification method, the ore is identified through the processing model obtained based on deep learning training, and the ore can be automatically identified rapidly and accurately with high precision.
Fig. 2 is a flow chart of a second embodiment of the ore rapid identification method of the present invention. As shown in fig. 2, in step S1, an ore training image is acquired and the characteristic properties and ore spatial coordinates of the ore training image are extracted. In a preferred embodiment of the present invention, industrial cameras, such as area cameras and line cameras, may be used for photographing. The captured ore training images are then transferred directly to a processor, such as a computer, via a gigabit, camera link, USB3.0, or other interface. Then, the characteristic properties and ore spatial coordinates of the ore training images can be extracted separately. For example, one, two, three or more characteristic properties may be extracted. The characteristic property may include one or more of characteristic parameters of color, brightness, transparency, and reflectivity, among others. In a preferred embodiment of the invention, multiple feature properties and ore spatial coordinates may be simultaneously extracted in multiple threads.
In step S2, a property saliency map of the ore training image is obtained based on the feature properties and a location saliency map of the ore training image is obtained based on the ore spatial coordinates. In a preferred embodiment of the invention, a plurality of feature property maps of the ore training image can be constructed simultaneously and multithreadingly based on a plurality of feature properties by respectively adopting a Gaussian pyramid algorithm and a central peripheral difference algorithm. Then, a plurality of the property saliency maps of the ore training image are obtained based on a plurality of the feature property maps using a cross-scale combination and normalization operator. At the same time, or thereafter, a two-dimensional gaussian distribution may be employed to obtain a location saliency map of the ore training image based on the ore spatial coordinates.
In step S3, the plurality of modeling coefficients are calculated based on the property saliency map and the position saliency map. In a preferred embodiment of the present invention, a scale invariant feature transformation algorithm may be employed to calculate the plurality of modeling coefficients based on a plurality of the property saliency maps and the location saliency map.
In step S4, the process model is constructed based on the plurality of modeling coefficients using a convolutional neural network. In this embodiment, the convolutional neural network employs a ReLU activation function. Fig. 3 is a model structure diagram of a convolutional neural network used in the method for quickly identifying ores shown in fig. 2, and as shown in fig. 3, the convolutional neural network is, from front to back, conv1, pool1, conv2, pool2, inner product layer 1, ReLU activation function, and inner product layer 2, respectively. Grouping 64 ore training images with the size of 256 × 256 of input data (64 × 3 × 256 elements in total); the conv1 layer reads the input data and performs convolution operation, the size of the filter (i.e. convolution kernel) in the conv1 layer is 5 × 5, the step size is 1, and 20 characteristic graphs (total 64 × 20 × 252 elements) with the size of 252 × 252 are output; the conv1 is maximally pooled to pool1 layers, the width and height of the feature map are pooled to half of the size of the previous layer, the number of the feature maps is unchanged, and 20 feature maps with the size of 126 by 126 are output (64 by 20 by 126 elements in total); similarly, conv2 outputs 50 feature maps of size 122 × 122 (total of 64 × 50 × 122 elements); pool2 outputs 50 signatures of size 61 x 61 (total of 64 x 50 x 61 elements). The inner laminated layer 1 outputs 500 characteristic graphs (64 elements by 500 elements in total); then, through ReLU, the number of elements is not changed; the output characteristic diagram of the inner lamination layer 2 is N (64 × N, N is an integer greater than or equal to 2), which is intended to represent that the network model performs N-classification, and finally, the calculation result of the SoftMaxWithLoss function is used as an output result.
In the present embodiment, the activation function used is ReLU, but in other embodiments of the present invention, sigmoid may also be used as the activation function. In the present invention, the advantage of using the ReLU activation function is not only to effectively avoid the local optimization problem, but also to map the input data to the final output layer, so that the data samples in the output layer become linearly separable.
In step S5, a CCD area-array camera is used to scan and obtain an image of the ore to be measured. In a preferred embodiment of the present invention, industrial cameras, such as area cameras and line cameras, may be used for photographing. The captured image is then transferred directly to a processor, such as a computer, via a gigabit network, camera link, USB3.0, or the like interface. These images can then be used as the image of the ore to be measured.
In step S6, the ore image to be tested is tested using the processing model obtained in step S4 to generate a test saliency map. In a preferred embodiment of the present invention, a process model may be employed for both result analysis and visual optimization. One skilled in the art may employ any known processing method to test the ore image under test using the processing model to generate a test saliency map.
In step S7, the test saliency map is optimized to generate the mask image. In a preferred embodiment of the present invention, threshold segmentation, morphological processing, and median filtering processing may be employed for the optimization process. In other preferred embodiments of the present invention, other suitable processing methods may be used for the correlation optimization.
In step S8, corner point detection is performed on the ore image to be detected obtained in step S5 to obtain a set of corner point feature points. Of course, in other preferred embodiments of the present invention, the ore image to be detected may not be processed, and other types of preprocessing may also be performed.
In step S9, the set of corner point feature points obtained from step S8 is multiplied by the mask image obtained from step S7 to obtain an ore image.
In step S10, a segmentation process may be performed on the ore image to obtain an ore image region and a background region. In the present invention, the image segmentation may be performed using any image segmentation algorithm known in the art, such as a watershed segmentation algorithm, a pyramid segmentation algorithm, and a mean-shift segmentation algorithm, among others.
In step S11, the ore image region may be identified as ore. In a preferred embodiment of the invention, it is also possible to convert the position of the ore image area into a movement position of the ore, and then drive the ejection means to eject the ore based on the movement position. In a preferred embodiment of the present invention, the step of comparing the area identified as ore with the actual situation to calculate the accuracy of the ore rapid identification method is also possible.
Fig. 4A-4E are schematic diagrams illustrating the effect of ore of the first mine site identified using the rapid ore identification method shown in fig. 2. Fig. 5A-5D are schematic diagrams illustrating the effect of ore of the second mine identified using the rapid ore identification method shown in fig. 2. As shown in fig. 4A to 5D, although the image is displaced and deformed, the recognition can be performed accurately.
As will be appreciated by those skilled in the art, as mentioned above, the execution sequence of the steps S1-S11 may be random, simultaneous, or reverse, or performed discontinuously, but at intervals, except as otherwise defined herein.
By implementing the rapid ore identification method, the ore is identified through the processing model obtained based on deep learning training, and the ore can be automatically identified rapidly and accurately with high precision. Furthermore, more ore characteristic properties can be obtained at the same time by adopting the CCD area-array camera, the spatial resolution is improved, the ore with smaller size fraction can be sorted, the processing capacity is large, and 40t/h can be realized. And furthermore, a processing model is constructed based on the convolutional neural network adopting the ReLU activation function, so that the ore image to be detected which is displaced and deformed can be more effectively identified, and the identification accuracy is further improved.
Another embodiment of the present invention provides a machine-readable storage and/or storage medium having stored thereon a machine code and/or a computer program having at least one code section for execution by a machine and/or a computer to cause the machine and/or computer to perform the steps of the method for rapid ore identification described herein.
In the invention, the convolutional neural network is integrally designed, a proper activation function and pooling scheme are defined, and a multi-GPU parallel design idea is adopted, so that the rapid ore identification technology is realized. In addition, the CCD area-array camera is adopted to scan and obtain the ore image, more ore surface characteristic information can be obtained at the same time, and the spatial resolution is improved to 0.05/mm. In addition, a classification model of ores and non-ores is constructed through a convolutional neural network, and a millisecond-level image processing engine based on an FPGA is designed, so that the vein-containing ores can be rapidly and accurately sorted under the condition of a free-fall feeding mode, and the accuracy is high.
Accordingly, the present invention can be realized in hardware, software, or a combination of hardware and software. The present invention can be realized in a centralized fashion in at least one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods of the present invention is suited. A typical combination of hardware and software could be a general purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein.
The present invention may also be implemented by a computer program product, comprising all the features enabling the implementation of the methods of the invention, when loaded in a computer system. The computer program in this document refers to: any expression, in any programming language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to other languages, codes or symbols; b) reproduced in a different format.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from its scope. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.