ARM-based fruit category and color sorting method and system
Technical Field
The invention relates to the technical field of image processing and identification, in particular to a method and a system for sorting categories and colors of fruits based on an ARM.
Background
For a long time, the fruit post-production treatment still stays in the manual sorting stage, and the classification and color of the fruits are manually classified by a large amount of manpower. Manual sorting not only consumes a lot of manpower and time, but also is subjectively affected (human visual deviation, subjective understanding, etc.), and sorting errors can occur.
With the development of machine learning technology, machine learning also starts to be used in the field of fruit sorting, but machine learning has the following disadvantages: 1. the target feature needs to be preset and different features will have a large contrast to the recognition result. Developers need to have a priori knowledge of the relevant features; 2. the recognition rate of machine learning is not high, and the recognition rate is lower when the image has the conditions of chromatic aberration, shielding and the like.
Therefore, there is a need to provide an ARM-based fruit sorting method to improve the above situation.
Disclosure of Invention
The invention aims to solve the defects in the prior art and provides a method and a system for sorting the categories and colors of fruits based on ARM.
According to the disclosed embodiments, the first aspect of the invention discloses an ARM-based fruit classification and color sorting method, which comprises the following steps:
s1, carrying out image acquisition on various fruits through a camera module in the embedded equipment with the ARM architecture;
s2, classifying all pictures for the first time according to the types of the fruits, and then classifying the fruits of the same type for the second time according to the colors of the fruits;
s3, building a deep learning framework on a PC, training the class pictures and the color pictures respectively through the PC, and respectively obtaining a type recognition model and a color classification model which are stored in an SD card storage module in the embedded equipment;
s4, fruit sorting is carried out by using the embedded equipment of the ARM framework, image acquisition is carried out through the camera module, the position of the fruit is located and the type of the fruit is identified by using the type identification model aiming at the acquired picture, and then color classification is carried out on each fruit by using the color classification model to obtain a detection result.
And S5, automatically controlling the manipulator to sort the fruits according to the detection result.
Further, the sorting method further comprises the following steps:
and S6, displaying the detection result through an LCD display module in the embedded equipment or sending the detection result to a PC terminal.
Further, the step S4 includes:
and S41, the type recognition model simultaneously outputs the coordinate position of the fruit and the type of the fruit through a full convolution network. The coordinate position is composed of an upper left corner horizontal coordinate, an upper left corner vertical coordinate, a lower right corner horizontal coordinate and a lower right corner vertical coordinate according to the position of a pixel point in the picture;
and S42, carrying out color classification on the detected fruit through the convolution network by using the color classification model.
Further, the color of the fruit includes the following information: bright, dark and/or with or without spots.
Furthermore, the type recognition model sequentially recombines and samples image pixel points through a convolutional layer, a pooling layer and a full-link layer, and then calculates to obtain the probability value of the coordinates and the category of the target.
Furthermore, after the type recognition model is calculated by a plurality of convolution layers, a plurality of groups of coordinates and categories are obtained simultaneously, and then predicted coordinates and categories are output through threshold setting of probability values.
Further, the color classification model sequentially carries out recombination and sampling on image pixel points through a convolution layer, a pooling layer and a full-connection layer, and then the color of the fruit is obtained through calculation.
Further, the number of target fruits in each picture is not limited.
According to a second aspect of the disclosed embodiments, the present invention discloses an ARM based fruit sorting system for category and color, the system comprising: a fruit conveying device used for conveying fruits, a manipulator used for sorting the fruits, and an embedded device based on ARM architecture and used for carrying out type identification and color classification on the fruits,
the embedded equipment comprises a camera module, an SD card storage module and an LCD display module, wherein the camera module is arranged right above the fruit conveying device and is used for shooting fruit images on the fruit conveying device; the embedded equipment is connected with an external PC, a deep learning frame is built on the PC, and the category pictures and the color pictures are respectively trained through the PC to respectively obtain a type recognition model file and a color classification model file; the SD card storage module is used for storing a type identification model file and a color classification model file; the LCD display module is used for displaying the detection result of the fruit;
and the manipulator sorts the fruit transmission device according to the fruit detection result of the embedded equipment based on the ARM architecture.
Compared with the prior art, the invention has the following advantages and effects:
the invention provides a deep learning method, which is characterized in that a target image is collected through a camera, then the types of a plurality of targets (such as apples, bananas and oranges) in the image are detected and identified by the deep learning method, and then fruits of the same type are classified and ordered according to the color and luster. Deep learning not only saves manpower and time, but also the sorting accuracy is much higher than that of machine learning.
Drawings
FIG. 1 is a flow chart of model training for the method of the present invention;
FIG. 2 is a sorting flow diagram of the method of the present invention;
FIG. 3 is a diagram of a category model I for fruit type identification in the present invention;
FIG. 4 is a diagram of a category model II for fruit type identification in the present invention;
FIG. 5 is a schematic diagram of a color classification model for color classification of fruits according to the present invention;
FIG. 6 is a diagram of an embodiment of the ARM-based fruit type and color sorting method disclosed in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Examples
This embodiment describes the specific implementation and operation of the present invention in detail through the attached fig. 1 to fig. 6.
First, an embedded device of the ARM architecture is developed. The embedded device needs to include modules such as a camera module, an SD card storage module, an LCD display module and the like.
Secondly, image acquisition is carried out on various fruits by using the embedded equipment with the ARM architecture. Every kind of fruit all needs to contain many pictures, and the quantity does not set up the upper limit, and the picture size needs to be unanimous. All pictures are classified once according to the type of fruit (such as apple, banana, orange, etc., the number of classes is not limited). Then, the fruits of the same type are classified secondarily according to the color (such as light, dim, spot, etc., with no limitation on the number of categories) of the fruits.
And then, a deep learning frame is set up on the PC, and the class picture and the color picture are respectively trained through the PC to respectively obtain a type recognition model file and a color classification model file.
Then, the type recognition model and the color classification model obtained by training are stored in an SD card storage module of the embedded device, and the deep learning framework and the recognition classification code are transplanted to the embedded device.
Fruit sorting is performed using an embedded device of the ARM architecture. And carrying out image acquisition, using a full convolution network to locate the position of the fruit and identify the type of the fruit according to the acquired image, and then using the convolution network to classify the color of each fruit.
And finally, displaying the detection result through an LCD display module in the embedded equipment or sending the detection result to a PC (personal computer) end, and automatically controlling the manipulator to sort the fruits according to the detection result.
As shown in fig. 1, fig. 1 is a model training preparation stage of the sorting method based on ARM fruit category and color of the present invention:
1. the embedded equipment of ARM framework is placed right above the fruit conveying device, and the camera module on the embedded equipment is right opposite to the fruit conveying device. In the process of uniform running of the transmission belt, the embedded equipment collects images at the speed of 1 frame per second. And storing the acquired image on an SD card storage module of the embedded equipment. The acquired images are stored in a jpg or bmp form, and in order to ensure the accuracy of identification in subsequent deep learning, the training data set ensures that the number of the images meets the requirements as much as possible, so that each fruit and each color and luster store a plurality of images, and the larger the number is, the better the number is.
2. A PC provided with an Gpu video card is prepared, and a deep learning open source framework is built on the PC.
3. And (4) storing the picture collected in the step (1) on a PC. Recording the pixel point positions (horizontal coordinates at the upper left corner, vertical coordinates at the upper left corner, horizontal coordinates at the lower right corner and vertical coordinates at the lower right corner) of the objects (the objects in the image refer to fruits, and the number of the objects in the image is not limited) in each image, and marking the type (such as apples, bananas, oranges, and the like) and the color (such as light, dim, speckles, and the like) of the fruits.
4. And respectively training the fruit type and the fruit color by using a convolution network through a PC (personal computer) to obtain a type identification model and a color classification model. Both models are saved to ARM SD. FIGS. 3 and 4 illustrate type recognition models used in the present invention, both of which can separately implement the function of type classification. FIG. 5 is a color classification model used in the present invention. In fig. 3, the image pixel points are recombined and sampled by the convolution layer, the pooling layer and the full-link layer, and then the probability value of the coordinates and the category of the target is calculated. Fig. 4 simultaneously obtains a plurality of sets of coordinates and categories by calculation of a plurality of convolutional layers, and then outputs predicted coordinates and categories by threshold setting of probability values. Fig. 5 mainly recombines and samples image pixels through a convolution layer, a pooling layer and a full-link layer, and then calculates to obtain the color of the fruit.
5. And (3) building a deep learning framework on the embedded equipment of the ARM architecture, and compiling codes for target detection and image classification.
As shown in fig. 2, fig. 2 illustrates the sorting stage of the sorting method based on the ARM type and color of the fruit disclosed by the invention:
1. the embedded equipment of the ARM framework is placed right above the fruit transmission device, and the camera module on the embedded equipment of the ARM framework is right opposite to the fruit transmission device. In the process of uniform running of the conveying belt, the camera collects images at the speed of 1 frame per second.
2. And detecting and identifying fruits appearing in the image by using the convolutional network and the type identification model, and then classifying the colors of the identified fruits by using the convolutional network and the color classification model.
3. And sending an instruction to the mechanical arm to carry out sorting operation according to the identification result.
4. And displaying the current picture and the recognition result on an LCD display module of the embedded device or transmitting the current picture and the recognition result to a PC.
As shown in fig. 6, fig. 6 discloses an ARM-based fruit type and color sorting system according to the present invention, which implements fruit sorting based on the above-disclosed ARM-based fruit type and color sorting method, and specifically includes: the fruit sorting machine comprises a fruit conveying device for conveying fruits, a mechanical ARM for sorting the fruits and an embedded device based on an ARM (advanced RISC machine) architecture for carrying out type identification and color classification on the fruits.
The embedded equipment based on the ARM architecture comprises a camera module, an SD card storage module and an LCD display module, wherein the camera module is arranged right above the fruit conveying device and used for shooting fruit images on the fruit conveying device; the embedded equipment is connected with an external PC, a deep learning frame is built on the PC, and the category pictures and the color pictures are respectively trained through the PC to respectively obtain a type recognition model file and a color classification model file; the SD card storage module is used for storing a type identification model file and a color classification model file; the LCD display module is used for displaying the detection result of the fruit.
The manipulator is controlled by embedded equipment based on an ARM framework, and the fruit conveying device is sorted according to the fruit detection result of the embedded equipment.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.