CN106203498B - Method and system for garbage detection in urban scenes - Google Patents

Abstract

The invention relates to a method and a system for detecting urban scene garbage. The urban scene garbage detection method comprises the following steps: the method comprises the steps of selecting a visual object classification VOC data set as a basic data set for garbage detection, obtaining a city image, marking out a garbage area, fusing and expanding the city image with the VOC data set and enriching the VOC data set, then building a deep learning platform based on a deep learning technology, selecting a pre-training model on the deep learning platform, carrying out garbage detection on the newly obtained city image through the deep learning platform and the pre-training model after carrying out prior parameter setting on the pre-training model, automatically giving a detection result, not needing a specially-assigned person to take a vehicle for photographing registration and manual garbage area detection, monitoring and detecting all-weather urban unordered garbage discarding conditions, and is low in cost and short in time consumption, so that the method is greatly convenient for detecting and cleaning the unordered garbage in the city, and guarantees urban sanitation and image.

Description

Method and system for garbage detection in urban scenes

technical field

The invention relates to a detection method and system, in particular to a method and system for detecting garbage in urban scenes.

Background technique

The disorderly discarded garbage in the city seriously affects the appearance of the city, pollutes the living environment, and has a great impact on the city and residents.

In order to clean up the disorderly discarded garbage in the city and maintain the city's sanitation and image, it is necessary to detect and locate the garbage in the urban scene, and then clean up according to the location. The current garbage detection method in urban scenes is mainly to send special personnel to conduct inspections and take photos for registration. During the inspection process, it is necessary to manually locate the garbage discarded in an orderly manner, operate a hand-held camera to take pictures, and organize and file after the inspection to record the distribution of garbage. corresponding responsible person. This method requires a special person to take photos and register by means of transportation, which is greatly affected by traffic, weather, personnel vacations and working hours, etc. It cannot monitor and detect the disorderly discarded garbage in cities around the clock, and manual photography, sorting and return There are problems such as high cost and long time consumption, which are not conducive to the detection and cleaning of disorderly discarded garbage in the city, and cannot guarantee the sanitation and image of the city.

Contents of the invention

In view of this, it is necessary to provide a garbage detection method and system in an urban scenario for the problems that the garbage detection and monitoring in the urban scenario cannot be carried out around the clock, and the cost is high and time-consuming.

A method for detecting garbage in an urban scene provided by the present invention comprises the following steps:

S10: Select the VOC data set as the basic data set for garbage detection, collect urban images and select urban images containing disorderly discarded garbage as candidate sets, and use the format defined by the VOC data set to select urban images containing disorderly discarded garbage in the candidate set Mark the area, and fuse the marked city image with the existing data in the VOC dataset;

S20: On the basis of the fused VOC data set, build a deep learning platform for garbage detection. On the built deep learning platform, obtain the pre-training model for garbage detection provided by the deep learning platform and perform adaptive pre-training on the pre-training model. Test parameter settings;

S30: Use the pre-trained model to detect garbage on the newly acquired urban image, detect whether there is garbage and the area where garbage exists in the newly acquired urban image, and give the detection result.

In one of the implementations, the step S10 is specifically: collecting urban images, selecting urban images containing disorderly discarded garbage as a candidate set, and using a rectangular selection box to select the urban images in the candidate set according to the format defined by the VOC data set. The area containing disorderly discarded garbage is marked. After the marking is completed, the marked city image is randomly divided into training set, verification set and test set, and the newly acquired training set, verification set and test set are combined with the VOC data. Concentrate the existing training set, verification set and test set for fusion.

In one of the implementation manners, the step S20 is specifically:

Choose the Caffe deep learning framework to implement the deep learning platform, and use the ZF model in Model Zoo as the pre-training model for the garbage detection task.

In one of the implementations, the step S20 is specifically: using a grid search method to verify the detection accuracy of different prior parameters on the city image on the fused VOC data set.

In one of the implementation manners, the step S30 specifically includes:

Preprocessing the newly acquired urban images, specifically including cropping, scaling, and/or performing mean value extraction on the newly acquired urban images;

Input the preprocessed urban image into the deep learning neural network, and obtain the classification and position regression of the candidate areas in the urban image to obtain the detection result.

An urban scene garbage detection system provided by the present invention includes:

The data fusion module selects the VOC data set as the basic data set for garbage detection, collects urban images and selects urban images containing disorderly discarded garbage as candidate sets, and uses the format defined by the VOC data set to select urban images containing disorderly garbage in the candidate set. Mark the area where the garbage is discarded, and fuse the marked city image with the existing data in the VOC dataset;

The deep learning platform building module builds a deep learning platform for garbage detection on the basis of the fused VOC data set. On the built deep learning platform, obtains the pre-training model for garbage detection provided by the deep learning platform and evaluates the pre-trained model. Perform adaptive prior parameter setting;

The urban image garbage detection module uses a pre-trained model to detect garbage in newly acquired urban images, detects whether there is garbage and the area where garbage exists in the newly acquired urban images, and gives the detection results.

In one of the implementations, the data fusion module collects urban images, selects urban images containing disorderly discarded garbage as a candidate set, and uses a rectangular selection box to select urban images in the candidate set containing no After the labeling is completed, the marked city images are randomly divided into training set, verification set and test set, and the newly acquired training set, verification set and test set are respectively combined with the VOC data set. Some training set, verification set and test set are fused.

In one of the implementations, the deep learning platform building module selects the Caffe deep learning framework to implement the deep learning platform, and uses the ZF model in the Model Zoo as a pre-training model for garbage detection tasks.

In one of the implementations, the deep learning platform building module uses a grid search method to verify the detection accuracy of different prior parameters for urban images on the fused VOC data set.

In one of the implementations, the urban image garbage detection module performs preprocessing on the newly acquired urban image, specifically including cropping, scaling, and/or performing mean value extraction processing on the newly acquired urban image; The urban image is input into the deep learning neural network, and the classification and position regression of the candidate areas in the urban image are obtained to obtain the detection result. The garbage detection method and system in the urban scene of the present invention selects the visual object classification VOC data set as the basic data set of garbage detection, obtains the urban image and marks the garbage area, and then fuses and expands the VOC data set with the VOC data set to enrich the VOC data set, and then based on the depth Learning technology builds a deep learning platform, selects a pre-training model on the deep learning platform, and performs garbage detection on newly acquired urban images through the deep learning platform and the pre-training model after setting the prior parameters of the pre-training model, and automatically gives the detection As a result, there is no need for a special person to take a vehicle to take pictures and register and detect artificial garbage areas. It can monitor and detect the status of disorderly discarded garbage in cities around the clock, with low cost and short time-consuming, which greatly facilitates disorderly disposal in cities. Garbage detection and cleaning to ensure urban sanitation and image.

Description of drawings

Fig. 1 is the flowchart of the urban scene rubbish detection method in an embodiment;

Fig. 2 is a structural diagram of an urban scene garbage detection system in an embodiment.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Fig. 1 is the flow chart of urban scene rubbish detection method in an embodiment, as shown in Fig. 1, this method comprises the steps:

S10: Select the VOC (visual object classes, visual object classification) data set as the basic data set for garbage detection, collect urban images and select urban images containing disorderly discarded garbage as candidate sets, and process them according to the format defined by the VOC data set. The urban images in the candidate set contain the areas where garbage is discarded in an orderly manner, and then the marked urban images are fused with the existing data in the VOC dataset.

VOC (visual object classes, visual object classification) data set is an authoritative scene detection data set, including a large number of training verification pictures and test pictures, with many types of labeled objects, including pedestrians, bicycles, buses, cars, motorcycles Common objects in urban scenes, such as common objects in urban scenes, can detect and recognize objects in urban images through the marked objects in the VOC dataset. Therefore, in this embodiment, the VOC (visual object classes, visual object classification) data set is selected as the basic data set for garbage detection, and the data and marked objects in the VOC data set are used to detect disorderly discarded garbage in urban scenes.

Due to the limited data in the VOC data set, there may be a problem of insufficient training data. In this embodiment, city images are obtained to expand the VOC data set. Specifically, urban images are collected, and urban images include urban images captured by street view vehicles, urban images crawled from the Internet, and the like. Then select the urban images containing disorderly discarded garbage as the candidate set, and use a rectangular selection box to mark the areas containing disorderly discarded garbage in the candidate set of urban images according to the format defined by the VOC data set (during the labeling process, as much as possible in the The object of interest is fully marked under the condition that less is marked into the background). After labeling, the labeled city images are randomly divided into training set, validation set and test set, and the newly acquired training set, validation set and test set are compared with the existing training set and validation set in the VOC dataset. merged with the test set.

Furthermore, the VOC2007 dataset is selected as the basic dataset for garbage detection. The VOC2007 data set includes 5011 training and verification pictures and 4952 test pictures, with a total of 20 types of labeled objects, which are suitable as a basic data set.

S20: On the basis of the fused VOC data set, build a deep learning platform for garbage detection. On the built deep learning platform, obtain the pre-training model for garbage detection provided by the deep learning platform and perform adaptive pre-training on the pre-training model. Check parameter settings.

After merging the acquired city image with the VOC data set, on the basis of the fused data, the method of the present invention detects disorderly discarded garbage in the urban scene based on a deep learning algorithm, builds a deep learning platform for garbage detection, and integrates the deep learning Apply it to garbage detection in urban scenes to expand the application of deep learning. After the deep learning platform for garbage detection is built, the deep learning platform has multiple pre-training models, and the pre-training model suitable for garbage detection is selected from the various pre-training models. In order to make the obtained pre-training model well adapted to the garbage detection in the current area, it is also necessary to set the adaptive prior parameters of the pre-training model.

In order to improve the detection accuracy, in a specific way, in this step, the Caffe deep learning framework is selected to realize the deep learning platform. The Caffe deep learning framework has complete documentation, an active community and a rich model library, which is suitable for platform construction. At the same time, the hardware configuration of the platform uses GPU (with stronger floating-point computing capability) as the computing core. Further, Nividia Geforce GTX 980 is used as the GPU, and Intel Core i7 and 16G memory are used as the main hardware configuration.

After choosing the Caffe deep learning framework for the realization of the deep learning platform, the complete community ecology of the Caffe deep learning framework provides a wealth of well-pretrained models. In this embodiment, the ZF model in the Model Zoo is used as the garbage detection task. pre-trained model.

Since different application scenarios require different adjustments to the prior parameters of the pre-training model, for garbage detection application scenarios, in this embodiment, the grid search method is used to verify the different prior parameters on the fused VOC data set. Detection accuracy in urban imagery. After repeated verification, this embodiment finally selects 0.001 as the initial learning rate, 0.0005 as the weight attenuation, and 0.9 as the impulse, and selects 128 candidate areas from each city image as a mini-batch (subset) for loss Backpropagation to update the pre-trained model weights.

S30: Use the pre-trained model to detect garbage on the newly acquired urban image, detect whether there is garbage and the area where garbage exists in the newly acquired urban image, and give the detection result.

After building a deep learning platform and choosing to obtain a pre-trained model, the obtained pre-trained model is used to detect disorderly discarded garbage in urban scenes. Specifically, newly acquire urban images of urban scenes, and then use the pre-trained model to detect garbage on the newly acquired urban images. Through the detection of the pre-training model, it is judged whether there is garbage and the area where the garbage exists in the urban image, and the detection result is given, so as to realize the detection of disorderly discarded garbage in the urban scene through deep learning and pre-training model.

In a further way, in order to make the city image meet the requirements of the pre-training model, the newly acquired city image is preprocessed, and the newly acquired city image is cropped or scaled, so that the newly acquired city image meets the requirement of the pre-training model.

At the same time, since the pixel range of the urban image is fixed, it is necessary to perform mean value extraction on the newly acquired urban image. Specifically, count the pixel values of the red, green, and blue bands in the training set of the fused VOC dataset to obtain the mean value of the pixels in the three bands, and subtract the corresponding mean values from the three bands of the newly acquired urban images. After preprocessing, the preprocessed city image is input into the neural network of deep learning, and the classification and position regression of the candidate areas in the city image are obtained to obtain the detection result. According to different application scenarios, select different confidence levels to screen the candidate results, find out the areas most likely to be disorderly discarded garbage, and realize the detection of disorderly discarded garbage in urban images.

The urban scene garbage detection method selects the visual object classification VOC data set as the basic data set of garbage detection, obtains urban images to mark the garbage area, and then fuses with the VOC data set to expand and enrich the VOC data set, and then builds it based on deep learning technology. Deep learning platform, select the pre-training model on the deep learning platform, after setting the prior parameters of the pre-training model, use the deep learning platform and the pre-training model to detect garbage on the newly acquired urban images, and automatically give the detection results. It is necessary to take a special person to take a vehicle for photo registration and detection of artificial garbage areas. It can monitor and detect the status of disorderly discarded garbage in cities around the clock, with low cost and short time-consuming, which greatly facilitates the detection of disorderly discarded garbage in cities. and clean up to ensure urban sanitation and image.

Simultaneously, the present invention also provides a kind of city scene rubbish detection system, as shown in Figure 2, this city scene rubbish detection system comprises:

The data fusion module 100 selects the VOC (visual object classes, visual object classification) data set as the basic data set for garbage detection, collects urban images and selects urban images containing disorderly discarded garbage as candidate sets, defined according to the VOC data set The format of the candidate set is used to mark the areas containing disorderly discarded garbage in the urban images in the candidate set, and fuse the marked urban images with the existing data in the VOC dataset.

VOC (visual object classes, visual object classification) data set is an authoritative scene detection data set, including a large number of training verification pictures and test pictures, with many types of labeled objects, including pedestrians, bicycles, buses, cars, motorcycles Common objects in urban scenes, such as common objects in urban scenes, can detect and recognize objects in urban images through the marked objects in the VOC dataset. Therefore, in this embodiment, the data fusion module 100 selects the VOC (visual objectclasses, visual object classification) data set as the basic data set of garbage detection, and uses the data and marked objects in the VOC data set to carry out disorderly discarding of garbage in urban scenes. detection.

Due to the limited data in the VOC dataset, there may be a problem of insufficient training data. In this embodiment, the data fusion module 100 acquires city images to expand the VOC dataset. Specifically, the data fusion module 100 collects urban images, which include urban images captured by street view vehicles, urban images crawled from the Internet, etc.; Defined format, using a rectangular selection box to mark the area of the urban image in the candidate set that contains disorderly discarded garbage (during the labeling process, mark the object of interest as much as possible under the condition that the background is marked as little as possible); after the labeling , divide the labeled city images randomly into training set, verification set and test set, and respectively combine the training set, verification set and test set of the newly acquired data with the existing training set, verification set and test set in the VOC dataset. The test set is fused.

Furthermore, the data fusion module 100 selects the VOC2007 data set as the basic data set for garbage detection. The VOC2007 data set includes 5011 training and verification pictures and 4952 test pictures, with a total of 20 types of labeled objects, which are suitable as a basic data set.

The deep learning platform building module 200, on the basis of the fused VOC data set, builds a deep learning platform for garbage detection. The model is set with adaptive prior parameters.

After merging the acquired city image with the VOC data set, on the basis of the fused data, the system of the present invention detects disorderly discarded garbage in the urban scene based on a deep learning algorithm, and the deep learning platform building module 200 builds the depth of garbage detection The learning platform applies deep learning to garbage detection in urban scenes and expands the application of deep learning. After building the deep learning platform for garbage detection, the deep learning platform has multiple pre-training models, and the deep learning platform building module 200 selects from the multiple pre-training models to obtain a pre-training model suitable for garbage detection. In order to make the obtained pre-training model well adapted to the garbage detection in the current area, it is also necessary to set the adaptive prior parameters of the pre-training model.

In order to improve the detection accuracy, the deep learning platform building module 200 selects the Caffe deep learning framework to realize the deep learning platform. The Caffe deep learning framework has complete documentation, an active community and a rich model library, which is suitable for platform construction. At the same time, the hardware configuration of the deep learning platform uses GPU (with stronger floating-point computing capability) as the computing core. Further, Nividia Geforce GTX 980 is used as the GPU, and Intel Core i7 and 16G memory are used as the main hardware configuration.

After choosing the Caffe deep learning framework for the implementation of the deep learning platform, the complete community ecology of the Caffe deep learning framework provides a wealth of well-pretrained models. In this embodiment, the deep learning platform building module 200 uses ZF in the Model Zoo. The model serves as a pre-trained model for garbage detection tasks.

Since different application scenarios require different adjustments to the prior parameters of the pre-training model, for the garbage detection application scenario, in this embodiment, the deep learning platform construction module 200 uses a grid search method on the fused VOC data set Verify the detection accuracy of different prior parameters on urban images. After repeated verification, this embodiment finally selects 0.001 as the initial learning rate, 0.0005 as the weight attenuation, and 0.9 as the impulse, and selects 128 candidate areas from each city image as a mini-batch (subset) for loss Backpropagation to update the pre-trained model weights.

The urban image garbage detection module 300 uses a pre-trained model to perform garbage detection on newly acquired urban images, detects whether there is garbage and the area where garbage exists in the newly acquired urban images, and provides the detection results.

After building the deep learning platform and choosing to acquire the pre-trained model, the urban image garbage detection module 300 uses the acquired pre-trained model to detect disorderly discarded garbage in the urban scene. Specifically, newly acquire urban images of urban scenes, and then use the pre-trained model to detect garbage on the newly acquired urban images. Through the detection of the pre-training model, it is judged whether there is garbage and the area where the garbage exists in the urban image, and the detection result is given, so as to realize the detection of disorderly discarded garbage in the urban scene through deep learning and pre-training model.

In a further way, in order to make the urban image meet the requirements of the pre-training model, the urban image garbage detection module 300 preprocesses the newly acquired urban image, crops or scales the newly acquired urban image, so that the newly acquired urban image meets the pre-trained model requirements. Requirements for training the model.

At the same time, since the pixel range of the urban image is fixed, the garbage detection module 300 of the urban image also needs to perform mean value extraction preprocessing on the newly acquired urban image. Specifically, the urban image garbage detection module 300 counts the pixel values of the red, green, and blue bands in the training set of the fused VOC dataset to obtain the average value of the pixels in the three bands, and subtracts the corresponding mean. After preprocessing, the preprocessed city image is input into the neural network of deep learning, and the classification and position regression of the candidate areas in the city image are obtained to obtain the detection result. According to different application scenarios, select different confidence levels to screen the candidate results, find out the areas most likely to be disorderly discarded garbage, and realize the detection of disorderly discarded garbage in urban images.

The urban scene garbage detection system selects the visual object classification VOC data set as the basic data set of garbage detection, obtains urban images to mark the garbage area, and then fuses with the VOC data set to expand and enrich the VOC data set, and then builds it based on deep learning technology Deep learning platform, select the pre-training model on the deep learning platform, after setting the prior parameters of the pre-training model, use the deep learning platform and the pre-training model to detect garbage on the newly acquired urban images, and automatically give the detection results. It is necessary to take a special person to take a vehicle for photo registration and detection of artificial garbage areas. It can monitor and detect the status of disorderly discarded garbage in cities around the clock, with low cost and short time-consuming, which greatly facilitates the detection of disorderly discarded garbage in cities. and clean up to ensure urban sanitation and image.

The garbage detection method and system in the urban scene of the present invention selects the visual object classification VOC data set as the basic data set of garbage detection, obtains the urban image and marks the garbage area, and then fuses and expands the VOC data set with the VOC data set to enrich the VOC data set, and then based on the depth Learning technology builds a deep learning platform, selects a pre-training model on the deep learning platform, and performs garbage detection on newly acquired urban images through the deep learning platform and the pre-training model after setting the prior parameters of the pre-training model, and automatically gives the detection As a result, there is no need for a special person to take a vehicle to take pictures and register and detect artificial garbage areas. It can monitor and detect the status of disorderly discarded garbage in cities around the clock, with low cost and short time-consuming, which greatly facilitates disorderly disposal in cities. Garbage detection and cleaning to ensure urban sanitation and image.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention. Inside.

Claims (4)

1. A method for detecting garbage in an urban scene, comprising the steps of: S10: Select the VOC data set as the basic data set for garbage detection, collect urban images and select urban images containing disorderly discarded garbage as candidate sets, and use the format defined by the VOC data set to select urban images containing disorderly discarded garbage in the candidate set Mark the area, and fuse the marked city image with the existing data in the VOC dataset; S20: On the basis of the fused VOC data set, build a deep learning platform for garbage detection. On the built deep learning platform, obtain the pre-training model for garbage detection provided by the deep learning platform and perform adaptive pre-training on the pre-training model. Test parameter settings; S30: Use the pre-training model to detect garbage on the newly acquired urban image, detect whether there is garbage and the area where garbage exists in the newly acquired urban image, and give the detection result; The step S10 is specifically: collecting urban images, selecting urban images containing disorderly discarded garbage as a candidate set, and using a rectangular selection box to select areas containing disorderly discarded garbage in the candidate set of urban images according to the format defined by the VOC data set. Labeling. After the labeling is completed, the labeled city images are randomly divided into training set, verification set and test set, and the newly acquired training set, verification set and test set are respectively combined with the existing training set and verification set in the VOC dataset. fusion set and test set; The step S20 is specifically: using a grid search method to verify the detection accuracy of different prior parameters for urban images on the fused VOC data set; selecting 0.001 as the initial learning rate, 0.0005 as the weight attenuation, and 0.9 as the impulse , and select N candidate areas from each urban image as a subset, and perform backpropagation of the loss to update the weights of the pre-trained model; The step S30 specifically includes: Preprocessing the newly acquired urban images, specifically including cropping, scaling, and/or performing mean value extraction on the newly acquired urban images; Input the preprocessed urban image into the deep learning neural network, and obtain the classification and position regression of the candidate areas in the urban image to obtain the detection result. 2. The urban scene garbage detection method according to claim 1, wherein said step S20 is specifically: Choose the Caffe deep learning framework to implement the deep learning platform, and use the ZF model in Model Zoo as the pre-training model for the garbage detection task. 3. An urban scene garbage detection system, characterized in that it comprises: The data fusion module selects the VOC data set as the basic data set for garbage detection, collects urban images and selects urban images containing disorderly discarded garbage as candidate sets, and uses the format defined by the VOC data set to select urban images containing disorderly garbage in the candidate set. Mark the area where the garbage is discarded, and fuse the marked city image with the existing data in the VOC dataset; The deep learning platform building module builds a deep learning platform for garbage detection on the basis of the fused VOC data set. On the built deep learning platform, obtains the pre-training model for garbage detection provided by the deep learning platform and evaluates the pre-trained model. Perform adaptive prior parameter setting; The urban image garbage detection module uses the pre-training model to detect garbage in the newly acquired urban images, detects whether there is garbage and the area where garbage exists in the newly acquired urban images, and gives the detection results; The deep learning platform building module uses a grid search method to verify the detection accuracy of different prior parameters on the city image on the fused VOC data set; select 0.001 as the initial learning rate, 0.0005 as the weight attenuation, and 0.9 as the impulse , and select N candidate areas from each urban image as a subset, and perform backpropagation of the loss to update the weights of the pre-trained model; The data fusion module collects urban images, selects urban images containing disorderly discarded garbage as a candidate set, and uses a rectangular selection box to mark areas containing disorderly discarded garbage in the urban images in the candidate set according to the format defined by the VOC data set. After the labeling is completed, the labeled city images are randomly divided into training set, validation set and test set, and the newly acquired training set, validation set and test set are combined with the existing training set, validation set and test set in the VOC dataset. The test set is fused; The urban image garbage detection module preprocesses the newly acquired urban image, specifically including cutting, zooming, and/or performing mean value extraction processing on the newly acquired urban image; inputting the preprocessed urban image into the neural network of deep learning , to regress the classification and location of candidate regions in urban imagery to derive detection results. 4. the urban scene rubbish detection system according to claim 3, it is characterized in that, described deep learning platform builds module and selects Caffe deep learning framework to carry out the realization of deep learning platform, uses the ZF model in ModelZoo as the prediction of rubbish detection task Train the model.