CN114463722A - Pavement type identification method and device, electronic equipment and storage medium - Google Patents
Pavement type identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114463722A CN114463722A CN202210114382.5A CN202210114382A CN114463722A CN 114463722 A CN114463722 A CN 114463722A CN 202210114382 A CN202210114382 A CN 202210114382A CN 114463722 A CN114463722 A CN 114463722A
- Authority
- CN
- China
- Prior art keywords
- road surface
- convolutional neural
- neural network
- network model
- surface image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000003860 storage Methods 0.000 title claims abstract description 26
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 165
- 238000012549 training Methods 0.000 claims abstract description 87
- 238000012795 verification Methods 0.000 claims abstract description 37
- 238000011176 pooling Methods 0.000 claims abstract description 27
- 238000012360 testing method Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 11
- 238000002372 labelling Methods 0.000 claims description 10
- 210000002569 neuron Anatomy 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 9
- 239000010426 asphalt Substances 0.000 claims description 8
- 239000004568 cement Substances 0.000 claims description 8
- 230000006872 improvement Effects 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 7
- 244000025254 Cannabis sativa Species 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 4
- 239000002689 soil Substances 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000003190 augmentative effect Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 57
- 238000010586 diagram Methods 0.000 description 11
- 230000004913 activation Effects 0.000 description 7
- 238000013507 mapping Methods 0.000 description 4
- 238000010200 validation analysis Methods 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000008707 rearrangement Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000005728 strengthening Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- QBPFLULOKWLNNW-UHFFFAOYSA-N chrysazin Chemical compound O=C1C2=CC=CC(O)=C2C(=O)C2=C1C=CC=C2O QBPFLULOKWLNNW-UHFFFAOYSA-N 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a road surface type identification method and device, electronic equipment and a storage medium. The method comprises the following steps: collecting a road surface image of a set type, and preprocessing the road surface image to obtain a sample set containing the road surface image; improving a pooling layer and a full-link layer in the convolutional neural network model, and adding a set network layer to obtain an improved convolutional neural network model; inputting a training set with a set proportion in a sample set into an improved convolutional neural network model to obtain a trained convolutional neural network model; inputting a verification set with a set proportion in a sample set into a trained convolutional neural network model, and verifying the trained convolutional neural network model; and inputting the test set with the sample set in a set proportion into the trained convolutional neural network model to obtain a pavement type recognition result. By the technical scheme of the embodiment of the invention, how to efficiently and accurately identify the type of the road surface on which the vehicle runs can be realized.
Description
Technical Field
The present invention relates to a road surface type identification technology, and in particular, to a road surface type identification method and apparatus, an electronic device, and a storage medium.
Background
With the continuous progress of science and technology, people have higher requirements on safety and comfort in the driving process of vehicles, and as the vehicles are vehicles driving on roads, the condition of the road surface directly influences the operation of the vehicles, so that the identification of the road surface type in the driving process of the vehicles is of great importance for improving the safety and comfort in the driving process of the vehicles.
In the prior art, a direct identification method based on a vision sensor and an indirect identification method based on a vehicle response parameter are generally adopted. However, if a direct identification method based on a vision sensor is adopted, when a complex road surface type is encountered, the identification precision is not high; if an indirect identification method based on vehicle response parameters is adopted, the method is easily influenced by accidental errors, and the identification precision is reduced. Therefore, how to efficiently and accurately identify the type of the road surface on which the vehicle runs is an urgent problem to be solved at present.
Disclosure of Invention
Embodiments of the present invention provide a method and an apparatus for identifying a road surface type, an electronic device, and a storage medium, which can efficiently and accurately identify a road surface type on which a vehicle is traveling.
In a first aspect, an embodiment of the present invention provides a road surface type identification method, including:
acquiring a road surface image of a set type through a vehicle-mounted front camera, and preprocessing the road surface image to obtain a sample set containing the road surface image;
improving a pooling layer and a full-link layer in the convolutional neural network model, and adding a set network layer to obtain an improved convolutional neural network model;
inputting the training set with the set proportion in the sample set into an improved convolutional neural network model to obtain a trained convolutional neural network model;
inputting the verification set with the set proportion in the sample set into a trained convolutional neural network model, and verifying the trained convolutional neural network model;
and inputting the test set with the set proportion in the sample set into a trained convolutional neural network model to obtain a pavement type recognition result.
In a second aspect, an embodiment of the present invention further provides a road surface type identification device, where the device includes:
the system comprises a sample set acquisition module, a data acquisition module and a data processing module, wherein the sample set acquisition module is used for acquiring a road surface image of a set type through a vehicle-mounted front camera and preprocessing the road surface image to obtain a sample set containing the road surface image;
the model improvement module is used for improving the pooling layer and the full-link layer in the convolutional neural network model and adding a set network layer to obtain an improved convolutional neural network model;
the model training module is used for inputting the training set with the set proportion in the sample set into an improved convolutional neural network model to obtain a trained convolutional neural network model;
the model verification module is used for inputting a verification set with a set proportion in the sample set into a trained convolutional neural network model and verifying the trained convolutional neural network model;
and the type identification module is used for inputting the test set with the set proportion in the sample set into the trained convolutional neural network model to obtain a pavement type identification result.
In a third aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes: one or more processors; a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the road surface type identification method according to any one of the embodiments of the present invention.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the method for identifying a road surface type according to any one of the embodiments of the present invention.
The technical scheme of the embodiment of the invention includes that a sample set containing a road surface image is obtained by preprocessing the collected road surface image with a set type, a pooling layer and a full connection layer in a convolutional neural network model are improved, a set network layer is added to obtain an improved convolutional neural network model, and then a training set with a set proportion in the sample set is input into the improved convolutional neural network model for training to obtain a trained convolutional neural network model; further, inputting a verification set with a set proportion in the sample set into the trained convolutional neural network model, and verifying the trained convolutional neural network model; and finally, inputting the test set with the sample set in a set proportion into the trained convolutional neural network model to obtain a road type recognition result, solving the problems of low recognition precision and low efficiency of the road type in the prior art, and realizing the efficient and accurate recognition of the road type of the vehicle running.
Drawings
Fig. 1a is a flowchart of a road surface type identification method according to an embodiment of the present invention;
FIG. 1b is a schematic diagram of an accuracy curve provided by an embodiment of the present invention;
fig. 2a is a flowchart of a road surface type identification method according to an embodiment of the present invention;
FIG. 2b is a schematic diagram of a preferred accuracy curve acquisition method flow according to an embodiment of the present invention;
FIG. 2c is a schematic diagram of a preferred method for identifying a road surface type according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a road surface type identification device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
At present, reducing traffic accidents and improving the driving safety of vehicles are the primary tasks of intelligent driving research, and in order to enable active safety control systems such as an anti-lock braking system and a driving anti-skid system to change the tire adhesion according to different road surface types and better exert the performance, the identification work of the road surface types becomes especially important.
In the prior art, the identification of the road surface type is mainly based on the acquisition and processing of vehicle-mounted visual images, and the acquired road surface images are subjected to deep feature extraction and are classified according to features. However, the traditional feature extraction method requires a large amount of work and cannot take good advantage of the high-speed parallel computing of the display core (GPU) today.
Therefore, the embodiment of the invention provides a road type identification method, which can be used for quickly and accurately identifying the road type.
Fig. 1a is a flowchart of a road surface type identification method according to an embodiment of the present invention, where the present embodiment is applicable to a case of identifying a road surface type on which a vehicle travels, and the method may be executed by a road surface type identification device, and the device may be implemented in a hardware and/or software manner, and may be generally integrated in an electronic device.
As shown in fig. 1a, a method for identifying a road surface type provided by an embodiment of the present invention includes the following specific steps:
s110: the method comprises the steps of collecting road surface images of a set type through a vehicle-mounted front camera, and preprocessing the road surface images to obtain a sample set containing the road surface images.
The vehicle-mounted front camera can be a Gucee LI-USB3.0-AR03ZWDRB vehicle-mounted front camera, the resolution of a static image of the camera can be changed from 3648 multiplied by 2736 to 384 multiplied by 480, and a dynamic image can be changed from 1280 multiplied by 720 to 320 multiplied by 240; the video frame rate may be between about 24 frames/second and about 60 frames/second.
The road surface image of the set type may refer to a road surface image of a type selected in advance by a technician in the embodiment of the present invention, and usually, the road surface image acquired by the vehicle-mounted front camera is a video file. Optionally, the set type of road surface image includes: an ice film type, a compacted snow type, a loose snow type, a dry asphalt type, a wet asphalt type, a dry cement type, a wet cement type, a dirt type, and a grass type.
The preprocessing of the pavement image can refer to the operations of screening, cutting, data strengthening or labeling and the like of the pavement image in the video format acquired by the vehicle-mounted front camera so as to obtain the pavement image meeting the requirements of the convolutional neural network model.
S120: and improving the pooling layer and the full-connection layer in the convolutional neural network model, and adding a set network layer to obtain the improved convolutional neural network model.
The set network layer may refer to a network layer added to the convolutional neural network model according to an actual application requirement and used for improving the convolutional neural network model, and may be, for example, a Batch Normalization (BN) layer or a Dropout layer.
In an optional embodiment, the modifying the pooling layer and the full-link layer in the convolutional neural network model, and adding the set network layer, to obtain the modified convolutional neural network model includes: setting the pooling layer in the convolutional neural network model as a maximum pooling layer, setting the number of neurons of the full-link layer of the last layer in the convolutional neural network model as a designated number, and adding a batch normalization layer and a Dropout layer to obtain an improved convolutional neural network model; the convolutional neural network model is a DenseNet network model. The pooling layer in the convolutional neural network model may be generally divided into an average pooling layer and a maximum pooling layer, and the maximum pooling layer is preferably used as the pooling layer in the convolutional neural network model in the embodiment of the present invention. The specified number may refer to a value for limiting the number of neurons in the full-link layer of the last layer according to the classification number of the road surface types of the actual road surface image, since the convolutional neural network may extract features layer by layer, and the feature extraction capability is enhanced with the increase of the network depth, the output layer must be the full-link layer, and it is ensured that the number of neurons is consistent with the type of the expected output, therefore, the number of neurons in the full-link layer of the last layer must be limited to ensure that the number of neurons in the full-link layer of the last layer is consistent with the type of the expected output, for example, in the embodiment of the present invention, 9 road surface types of road surface images are collected, and therefore, the number of neurons in the full-link layer of the last layer may be set to 9. The DenseNet model is selected because it can reduce overfitting and ensure accuracy of road surface type recognition with a small number of road surface images
In another alternative embodiment, the activation function in the convolutional neural network model may be a modified linear unit (ReLU) with a unilateral suppression characteristic, and since all the outputs of the inputs less than or equal to zero are zero after passing through the ReLU activation function, the outputs of the positive inputs after passing through the ReLU activation function are consistent with the inputs. Specifically, taking the implementation of the ReLU activation function in the tensrflow as an example, an interface of a function of tf.nn.relu may be used, and a layer.relu () statement may be directly used like the BN layer to insert the ReLU activation function as a separate layer into the custom container. The activation function is not generally considered the primary computational layer, and therefore not included in the total number of layers, since some special network structure is removed.
Therefore, by adopting a migration learning method, the DenseNet121 on the last layer in the traditional DenseNet model is replaced by a full connection layer with 1024 nodes, a BN layer and a Dropout layer are added, and a ReLU activation function is selected, so that the total parameters of the improved convolutional neural network model can be ensured to be 8100425, wherein the trainable parameters are 1060873, less samples and calculation resources can be used while the accuracy is ensured, and the over-fitting problem caused by the insufficient number of samples is avoided.
S130: and inputting the training set with the set proportion in the sample set into an improved convolutional neural network model to obtain the trained convolutional neural network model.
The set proportion may refer to a proportion value for adjusting the division ratio of the training set, the verification set and the test set in the sample set according to the actual situation, exemplarily, the sample set may be loaded through a load _ csv () function, and the corresponding road surface image may be loaded according to the division parameters of the sample set, specifically, if the division parameters of the sample set are train, 60% of the sample set is randomly taken as the training set; if the partition parameter of the sample set is val, 40% of the sample set is randomly taken as a verification set and a test set.
In an optional embodiment, the inputting the training set with the set proportion in the sample set into the improved convolutional neural network model to obtain the trained convolutional neural network model includes: inputting the training set with the set proportion in the sample set into an improved convolutional neural network model, and training the improved convolutional neural network model to obtain a trained convolutional neural network model; in the training process, setting a training termination point according to the fluctuation state of the accuracy curve, and adjusting the convolutional neural network model according to the accuracy curve; and storing the trained convolutional neural network model by utilizing a high-level application programming interface library. In the embodiment of the present invention, Early Stopping is adopted, that is, by observing the fluctuation of an accuracy curve, an optimal training end point is expected to be found, for example, first, a verification accuracy curve of a convolutional neural network model is recorded in advance, and the fluctuation change of the verification accuracy curve is monitored, and when it is found that the verification accuracy curve does not decrease for continuous training cycles of a pat (tolerance cycle), training of the convolutional neural network model is considered to have reached the appropriate training end point, so as to terminate training in advance, and the pat is usually set to a value less than or equal to 5; specifically, for a training task, as long as the sequence of the pictures in the training set is not disturbed every time of training, a quite similar training accuracy curve can be obtained, so that a set number of epochs (generations) can be directly trained, then the most appropriate training end point can be selected according to the fluctuation of the verification accuracy, usually, the number of epochs can be set to be 60 to 100, as shown in fig. 1b, a schematic diagram of an accuracy curve provided by an embodiment of the present invention is shown, wherein the abscissa represents the number of epochs for training, the ordinate represents the accuracy, the dotted line represents the training accuracy curve, and the solid line represents the verification accuracy curve. It can be seen from the figure that the verification accuracy curve starts to decrease when the Epoch is 3, and therefore, the Epoch 3 may be used as the training termination point, and then the Epoch parameter is reset, so as to make the training Epoch parameter of the convolutional neural network model be the optimal value. In the embodiment of the present invention, a Keras high-level interface may be selected, a tf.save _ model.save (net, path) statement is used to store the convolutional neural network model to a path desired to be stored in a SavedModel format, and further, as the training is performed, when the maximum value of the verification accuracy is updated, an Epoch parameter is recorded and loaded through a network word.model.load _ model function, so that the optimal Epoch parameter can be obtained without training the convolutional neural network model for the second time by using the model storing and loading capability of the Keras high-level interface, thereby shortening the training time and reducing the workload of the worker.
S140: and inputting the verification set with the set proportion in the sample set into a trained convolutional neural network model, and verifying the trained convolutional neural network model.
Specifically, a verification set with a set proportion in a sample set is input into a trained convolutional neural network model, so that a road surface type identification result corresponding to each picture in the sample set can be obtained, the identification result of the road surface type is compared with a real value, so that the verification accuracy of the convolutional neural network model can be obtained, and the verification of the convolutional neural network model is realized, wherein the real value can refer to each road surface image with a corresponding accurate road surface type in the sample set, and can mark the road surface image with the accurate road surface type in advance for a worker.
S150: and inputting the test set with the set proportion in the sample set into a trained convolutional neural network model to obtain a pavement type recognition result.
Specifically, a test set with a sample set in a set proportion is input into a trained convolutional neural network model, and a road surface image with a road surface type can be obtained.
The technical scheme of the embodiment of the invention includes that a sample set containing a road surface image is obtained by preprocessing the collected road surface image with a set type, a pooling layer and a full connection layer in a convolutional neural network model are improved, a set network layer is added to obtain an improved convolutional neural network model, and then a training set with a set proportion in the sample set is input into the improved convolutional neural network model for training to obtain a trained convolutional neural network model; further, inputting a verification set with a set proportion in the sample set into the trained convolutional neural network model, and verifying the trained convolutional neural network model; and finally, inputting the test set with the sample set in a set proportion into the trained convolutional neural network model to obtain a road surface type identification result, solving the problems of low accuracy and low efficiency of identifying the road surface type in the prior art, and realizing the efficient and accurate identification of the road surface type of the vehicle running.
Fig. 2a is a flowchart of a road surface type identification method according to an embodiment of the present invention. In this embodiment, optionally, the preprocessing the road surface image to obtain a sample set including the road surface image includes: carrying out format processing on the road surface image to obtain a processed road surface image; expanding the processed road surface image through an improved countermeasure generating network to obtain an expanded road surface image; performing data enhancement on the expanded road surface image to obtain a data-enhanced road surface image; labeling the data enhanced pavement image to obtain a sample set containing the pavement image.
Optionally, before the expanding the processed road surface image through the improved countermeasure generating network, the method further includes: and adding a batch normalization layer between convolution layers of the generated network in the original countermeasure generating network to obtain the improved countermeasure generating network.
As shown in fig. 2a, a method for identifying a road surface type according to an embodiment of the present invention includes the following specific steps:
s210: the method comprises the steps of collecting road surface images of a set type through a vehicle-mounted front camera, and carrying out format processing on the road surface images to obtain processed road surface images.
The format processing can be that the collected road surface image is extracted according to a specified frame frequency and is uniformly cut into a fixed size, for example, a python program can be adopted to process a collected road surface image video file, one image is extracted every 30 frames and is uniformly cut into a size of 640 multiplied by 360, so that the problems of opposite lanes except the lane where the vehicle is located and interference factors such as pedestrians, trees or buildings and the like which are shot by a vehicle-mounted front camera are solved, the materials of available pictures are increased, and the workload of manually removing the interference factors in the road surface image is reduced.
S220: and adding a batch normalization layer between convolution layers of the generated network in the original countermeasure generating network to obtain the improved countermeasure generating network.
Wherein, the original confrontation generation network may refer to the generation confrontation network without improvement; the generation of the countermeasure network comprises two sub-networks of a generation network and a discrimination network, and in the embodiment of the invention, the generation network can be formed by stacking 5 transposed convolutional layer modules, so that the size of the feature diagram is enlarged and the number of channels is reduced. The improved countermeasure generation network can be obtained by adding a batch normalization layer between each convolution layer of the generation network, so that the training speed of the improved countermeasure generation network is increased, and the stability of the improved countermeasure generation network is improved.
S230: and expanding the processed road surface image through an improved countermeasure generating network to obtain an expanded road surface image.
Since the road surface images of the ice film type, the soil type, the grass type and other types are difficult to acquire, and the available road surface images are relatively few, in order to alleviate the problem caused by the small number of the road surface images, an improved countermeasure generation network needs to be adopted to generate additional images for the road surface type with the small number of the road surface images, so as to meet the requirement of the convolutional neural network model.
In an alternative embodiment, the processing the road surface image by the improved countermeasure generating network to obtain an expanded road surface image, including: acquiring a random sampling hidden vector through the prior distribution of the processed road surface image, and inputting the random sampling hidden vector into a generation network of an improved countermeasure generation network to obtain a generated picture; and inputting the generated pictures into a discrimination network of an improved countermeasure generation network to obtain the probability of the two classification tasks, and storing the generated pictures meeting the set probability into a file corresponding to the road surface type to obtain an expanded road surface image. The two classification tasks can refer to comparing the generated pictures with corresponding truth values to judge whether the generated pictures can correspond to corresponding road surface types or not; the set probability may be a value used to evaluate the availability of the generated picture, and for example, the set probability may be a value between 0 and 1, if the probability of the two classification tasks is close to the set probability, it may be determined that the generated picture may correspond to the corresponding road surface type, and if the probability of the two classification tasks is greater than the set probability, it may be determined that the generated picture may not correspond to the corresponding road surface type, and the generated picture may be discarded. Specifically, firstly, a random sampling hidden vector z with the length of 100 is obtained through the prior distribution of a processed road surface image, is adjusted into a 4-dimensional tensor of [ b,1, 100] through Reshape (rearrangement) operation, and is input into a generation network of an improved countermeasure generation network to amplify the dimensions of the height and the width of a picture, so as to obtain a color generation picture with the height and the width of 64 and the number of channels of 3, then, a generation picture tensor with the size of [ b,64,64,3] is subjected to 5 continuous convolution layers to extract a characteristic vector, a tensor with the size of [ b,2, 1024] is output, the characteristic size is converted into [ b,1024] through a pooling layer GlobalatagePooling 2D, finally, the probability of a two-classification task is obtained through a full-connection layer, and the generation picture which meets the set probability is stored into a file of a corresponding road surface type, so as to obtain an expanded road surface image.
S240: and performing data enhancement on the expanded road surface image to obtain a data-enhanced road surface image.
The data enhancement may refer to performing convolution or filtering operation, such as random horizontal flipping, random vertical flipping, or random cropping and scaling, on the expanded road surface image to increase diversity of the road surface image and reduce overfitting.
S250: labeling the data enhanced pavement image to obtain a sample set containing the pavement image.
The labeling can refer to that a coding label corresponding to a road surface type is attached to a data-enhanced road surface image according to a pre-created coding table, since the road surface type is generally defined as a character string type, such as a soil type or a grass type, and the like, but in order to put the road surface type into a convolutional neural network for training, the character strings must be digitally coded and then converted into a coding format matched with a convolutional neural network model when appropriate, for a sample set with n types of things, each type is randomly coded into a number of l epsilon [0, n-1], one road surface type plus the coding thereof is called a set of coding, and a list containing all the coding is called a coding table. In the embodiment of the invention, a One-Hot coding mode can be adopted for coding the road surface type, wherein the One-Hot is originally an idea of mapping n classes by adopting an n-bit register, each class has exclusive register bits, and only One bit is effective generally, so the One-Hot coding is also called a One-bit effective coding which is used as a binary form of a classification variable and can map a label to a binary integer, and the register bits with the indexes out are limited to 0 or 1. Illustratively, the grass type code is 0, the dry asphalt type code is 1, the dry cement type code is 2, the ice film type code is 3, the dirt type code is 4, the wet asphalt type code is 5, the wet cement type code is 6, the loose snow type code is 7, and the compacted snow type code is 8. It is noted that the encoding table cannot be easily changed after creation. A sample set may refer to a storage path containing a data-enhanced road surface image and a corresponding encoded file, such as a csv formatted file.
Specifically, in the embodiment of the present invention, the coding table may be created by, first, classifying and packaging all the data-enhanced road surface images into folders, placing the folders under one large folder, traversing all the subdirectories under the root directory using the read _ file () function in the io library in the tensoflow frame, and reading the images as the 3-channel pixel matrix of RGBA using the decode () function in the image library in the tensoflow frame. And aiming at each subdirectory, the road surface type is used as a key of the name2label of the coding table, and the number of the existing key value pairs of the coding table is used as the corresponding coding of the road surface type. After the code table is determined, all the storage paths and codes of the data-enhanced road surface image need to be acquired and stored in two lists, namely images and labels. The images list is responsible for storing a storage path of the data enhanced road surface image, the labels list is responsible for storing a code of the data enhanced road surface image, the lengths of the images and the code are consistent, and elements at corresponding positions are associated with each other, so that the storage path containing the data enhanced road surface image and a sample set of the code are obtained. In an alternative embodiment, the shuffle () function of the random library may also be used to shuffle the order of the data stored in the list.
S260: and improving the pooling layer and the full-connection layer in the convolutional neural network model, and adding a set network layer to obtain the improved convolutional neural network model.
Specifically, firstly, the convolutional neural network DenseNet is trained by using an ImageNet data set to obtain a DenseNet network with fixed parameters, then the last layer of full-connection network is replaced by a full-connection layer with a specified number of neurons, so that the number of neurons in the output full-connection layer is 9, and finally, the solidified convolutional neural network and a new network layer are defined as a new container SELFNET through an encapsulation container provided by Keras.
S270: inputting the training set with the set proportion in the sample set into an improved convolutional neural network model, and training the improved convolutional neural network model to obtain a trained convolutional neural network model; in the training process, a training termination point is set according to the fluctuation state of the accuracy curve, and the convolutional neural network model is adjusted according to the accuracy curve.
In the embodiment of the invention, the parameter net, train can be made to be False to fix the parameter of the DenseNet, so that most of the parameters in the DenseNet network do not participate in the training, most of the parameters reacting to the training of the DenseNet network are greatly reduced, and the training time of the DenseNet network is shortened on the premise of ensuring the accuracy. Conversely, if the parameter net is set, the total number of parameters in the DenseNet network will be trained.
Wherein, the accuracy curve can be drawn according to each parameter in the training process. Fig. 2b is a schematic diagram illustrating a preferred accuracy curve obtaining method according to an embodiment of the present invention. The method comprises the following specific steps: 1. creating an encoding table: specifically, a function load _ pictures (root, mode) is defined, wherein the root is a folder of data-enhanced road surface images, the mode decides to insert the road surface images into a corresponding set of one of a training set, a test set and a verification set, and a default value is set as "train". The os.listdir () function traverses all subfolders under the images folder in order after calling the function, and takes the subfolder name as the key of the encoding table, through the sentence len (name 2label)Keys ()) reads the key value logarithm currently loaded in the code table, and takes the key value logarithm as the mapping number of the road surface image type, thereby completing the creation of the code table. 2. Create sample and label table: specifically, a function load _ csv (root, filename, name2label) is defined, where root represents a root directory of the sample set, filename is a file name of csv, labels is a road surface image type encoding table, and the function may return a storage path list and a tag list of the road surface image. In the embodiment of the invention, whether the csv file exists or not can be judged by using a statement exist (os.path.join (root, filename)); if the csv file does not exist, an image can be newly created]An empty list, then sequentially extracting the road surface images from the folder, and adding the stored paths to images [ [ solution ] ]]In the list, creating the csv file in a mode of disturbing all storage paths of the road images in the list by using a shuffle (images) function; and if the csv file exists, directly reading the storage path information of the road surface image through a csv reader (f) statement. It should be noted that if a plurality of csv files exist in the folder, there are problems that reading errors occur or the number of tags does not match the output of the convolutional neural network, and therefore, if the name of the csv file that is desired to be saved is modified, the existing csv file must be deleted. 3. Dividing the data set: specifically, an image list and a label list in the csv file are loaded by using a load _ csv () function, and a corresponding road surface image and a label thereof are loaded according to a dividing parameter of the road surface image in the sample set. If the dividing parameter of the pavement images in the sample set is train, 60% of the pavement images and the labels are respectively taken as training sets, and correspondingly, if the dividing parameter of the pavement images in the sample set is val, 40% of the pavement images and the labels in the sample set are taken as verification sets and test sets. 4. Create a Dataset object: firstly, a road surface image storage path list, a label list and an encoding table of a training set are returned through a function load _ pictures, and then operations such as input disordering, strengthening, batching and the like are completed through a statement shuffle (4000). Wherein, the map (executable) function maps the sequence of numbers with the function of function, and all objects in the sequence of numbers can call the function of function and return the sequence of numbers mapped by functionA preprocess (r,1) function is selected in the embodiment of the invention. 5. Preprocessing data: since the content tensor of the road surface image is not stored in the images list, and only the storage path character string is stored, the reading and tensor conversion of the road surface image need to be completed in the image processing function. In particular, this can be achieved by defining a preprocess (r,1) function, and it is noted that the input parameters of the preprocess (r,1) function need to be consistent with the format used when creating the Dataset object. Wherein, the map function is located in the tuple object of the (db ═ db. shuffle (1000). map (process). batch (32)) incoming function (r,1) before the batch () function, r represents the path list of all pictures, 1 represents the label number list of all pictures, and the input parameter of the function process (r, l) is (x)iYi) where xiAnd yiRespectively representing the storage path and the label of the ith road surface image. On the basis of the above embodiment, in view of the small scale of the collected road surface image data set, in the embodiment of the present invention, the road surface image is subjected to appropriate enhancement transformation to obtain more diversified available samples, and the value range is [0,255 ]]Inner RGB pixel mapping to [0,1]And finally, realizing standardized operation through a standardized function, and mapping the pixels to an expected interval, thereby playing a vital role in enhancing a future convolutional neural network model. Finally, the data is converted to a tensor type return, at which point the images and labels tensors are called cyclically for the variable db. Notably, standardized inputs make model training more accurate, and if it is desired to finally output a line graph for visualization, it is necessary to map the data from the tensor model to [0,1]The numerical range of (2). 6. Initializing a model: specifically, a net [ business ] statement may be used to call a densneet 121 network with initialized parameters, and a Sequential container provided by keras may be used to package the net and the newly added full connection layer, BN layer, and dropout layer into a customized network newnet. Illustratively, since the degree of keras facing the object is very high, the fully-connected layer can be added by directly using layer. 7. Training network parameters: in particular, the sentence netzer, loss, metrics) statements determine the loss function and optimizer type used by the convolutional neural network, wherein an optimizer parameter represents the optimizer type, and in the embodiment of the present invention, an Adam optimizer can be selected; the loss parameter represents the type of the loss function, and a cross entropy loss function can be selected in the embodiment of the invention; the metrics parameters are used to specify the measurement index, and the accuracy is selected. 8. Printing accuracy curve: firstly, a training set and a verification set can be sent into a convolutional neural network to be trained by using a sentence history ═ network.fit (train _ db, epoch, validation _ data, validation _ freq), wherein the train _ db is a created Datasets object and can be directly referred to by the sentence tf.data.dataset or can be input numpy sequence data; epochs can be used to specify the number of cycles for which training is performed in total; validity _ data can specify the number of cycles for which verification is to be performed once. Given variable record receives data obtained by training of fit () function, obtains a Dictionary object through statement record, records the record items such as error, measurement index and the like generated by each training, and can directly check the training data in the console window of the editor, the optional parameter callbacks of the fit () function is a callback function adopted by the neural network, and Early Stopping function can be realized through the parameter. For EarlyStopping, the traditional EarlyStopping is a class in keras and comprises three parameters, and a monitor (attention index) can be selected as the accuracy or the error; the minimum rising rate min _ delta stops training by judging whether the accuracy rate rises in a limited period or not; endurance defines the maximum period over which training ceases immediately when the accuracy does not rise. In the embodiment of the invention, the latest neural network callback function ModelCheckPoint can be adopted, the parameter when the convolutional neural network model reaches the optimal verification accuracy can be stored, the parameter also comprises three indexes, wherein the monitor is consistent with the concerned index included by EarlyStopping, and the accuracy is selected; save _ best _ only indicates whether only the result of the optimal training is saved; validation represents imported validation set data. After Early Stopping optimization, the convolutional neural network training can obtain the optimal parameters in a limited time. In addition, the convolution neural network model is carried outThe accuracy curve can be drawn while the model training is carried out, the dictionary variable record stores each index after each training is finished, and each index is obtained through a function record [ 'val _ accuracy']And calling the verification accuracy of each training, acquiring the epoch number of the training in total through a function len (), determining the coordinate axis scale, and saving the training curve by using a function savefig () in a pyppolt library. Thus, the drawing of the accuracy curve can be completed.
In an alternative embodiment, before performing the convolutional neural network model training, it is necessary to initialize each parameter of the convolutional neural network, and adjust the learning rate, the loss function, and the like. Specifically, the initial learning rate and the adjustment of the learning rate are determined by an optimizer, wherein the optimizer may refer to a specific gradient descent strategy to update parameters of the convolutional neural network, for example, an Adam optimizer may be adopted to more comprehensively consider a mean value of a gradient and an un-centralized variance of the gradient, and then obtain the learning rate of the next cycle, and the update rule is: gt＝▽θJ(θt-1) Wherein g istRepresenting the gradient of each parameter at time t, J (θ)t-1) Representing the value of a random objective function of a parameter at time t-1, the gradient momentum mtComprises the following steps: m ist＝β1mt-1+(1-β1)gtWherein m istTypically initialized to 0, beta1The weighted mean value of the gradient square is as follows:
wherein v istAn exponential moving average representing the square of the gradient, typically initialized to 0, β2And is typically set to 0.999 for an exponential decay rate. Due to m0Initialized to 0, possibly mtResults in approaching 0 at the initial stage of training, for mtThe deviation is corrected to
Likewise, for vtAlso, a correction is made:
for the Adam optimizer, the initial learning rate is typically set to 0.001, and the update parameters are:
furthermore, in the embodiment of the present invention, the error function adopts a cross-entropy loss function, which is usually used together with the Softmax function, because the encoding tag in the One-hot encoding format can simplify the cross-entropy loss function and the partial derivative of the Softmax function, which are mathematically expressed as:
where p and q are two different mathematical differences. The cross entropy can be decomposed into the entropy of p, H (p), and the sum of KL divergence of p and q: h (p | | q) ═ H (p) + DKL(p | | q). Wherein KL is defined as:
when One-hot coding is used, h (p) is 0, in which case:
therefore, the cross entropy loss function can be greatly simplified by using One-hot coding, so that the training of the convolutional neural network is faster.
S280: and inputting the verification set with the set proportion in the sample set into a trained convolutional neural network model, and verifying the trained convolutional neural network model.
S290: and inputting the test set with the set proportion in the sample set into a trained convolutional neural network model to obtain a pavement type recognition result.
Specifically, as shown in fig. 2c, which is a schematic diagram of a flow of a preferred road surface type identification method provided by an embodiment of the present invention, firstly, a video camera (i.e., a vehicle-mounted front camera) is used to collect road surface images in front of a vehicle, then, video images (i.e., the collected road surface images) are extracted according to a suitable frame rate, then, image preprocessing and labeling are performed on the extracted road surface images to obtain a sample set including the road surface images, and finally, a training set in the sample set is input to a convolutional neural network to obtain a trained convolutional neural network model; inputting a verification set in the sample set into the trained convolutional neural network model, and verifying the trained convolutional neural network model; and inputting the test set in the sample set into the trained convolutional neural network model to obtain a road image classification result (namely a road type identification result).
According to the technical scheme of the embodiment of the invention, a processed road surface image obtained by format processing is input into an improved countermeasure generation network for expansion to obtain an expanded road surface image, then the expanded road surface image is subjected to data enhancement and labeling to obtain a sample set containing the road surface image, a pooling layer and a full connection layer in a convolutional neural network model are improved, a set network layer is added to obtain an improved convolutional neural network model, and then a training set with a set proportion in the sample set is input into the improved convolutional neural network model for training to obtain a trained convolutional neural network model; further, inputting a verification set with a set proportion in the sample set into the trained convolutional neural network model, and verifying the trained convolutional neural network model; and finally, inputting the test set with the sample set in a set proportion into the trained convolutional neural network model to obtain a road surface type identification result, solving the problems of low accuracy and low efficiency of identifying the road surface type in the prior art, and realizing the efficient and accurate identification of the road surface type of the vehicle running.
Fig. 3 is a schematic structural diagram of a road surface type identification device according to an embodiment of the present invention, which can execute the road surface type identification method according to the above embodiments. The device can be implemented in software and/or hardware, and as shown in fig. 3, the road surface type identification device specifically includes: a sample set acquisition module 310, a model improvement module 320, a model training module 330, a model verification module 340, and a type identification module 350.
The system comprises a sample set acquisition module 310, a road surface image preprocessing module and a road surface image processing module, wherein the sample set acquisition module 310 is used for acquiring a road surface image of a set type through a vehicle-mounted front camera and preprocessing the road surface image to obtain a sample set containing the road surface image;
the model improvement module 320 is used for improving the pooling layer and the full-link layer in the convolutional neural network model and adding a set network layer to obtain an improved convolutional neural network model;
the model training module 330 is configured to input the training set with the set proportion in the sample set into an improved convolutional neural network model to obtain a trained convolutional neural network model;
the model verification module 340 is configured to input a verification set with a set proportion in the sample set into a trained convolutional neural network model, and verify the trained convolutional neural network model;
and the type identification module 350 is configured to input the test set with the set proportion in the sample set into the trained convolutional neural network model to obtain a road surface type identification result.
The technical scheme of the embodiment of the invention includes that a sample set containing a road surface image is obtained by preprocessing the collected road surface image with a set type, a pooling layer and a full connection layer in a convolutional neural network model are improved, a set network layer is added to obtain an improved convolutional neural network model, and then a training set with a set proportion in the sample set is input into the improved convolutional neural network model for training to obtain a trained convolutional neural network model; further, inputting a verification set with a set proportion in the sample set into the trained convolutional neural network model, and verifying the trained convolutional neural network model; and finally, inputting the test set with the sample set in a set proportion into the trained convolutional neural network model to obtain a road type recognition result, solving the problems of low recognition precision and low efficiency of the road type in the prior art, and realizing the efficient and accurate recognition of the road type of the vehicle running.
Optionally, the model improvement module 320 may be specifically configured to set a pooling layer in the convolutional neural network model as a maximum pooling layer, set the number of neurons in a full connectivity layer of a last layer in the convolutional neural network model as a specified number, and add a batch normalization layer and a Dropout layer to obtain an improved convolutional neural network model; wherein, the convolutional neural network model is a DenseNet model.
Optionally, the sample set obtaining module 310 may specifically include a format processing unit, an image expansion unit, a data enhancement unit, and a labeling unit;
the format processing unit is used for carrying out format processing on the road surface image to obtain a processed road surface image;
the image expansion unit is used for expanding the processed road surface image through an improved countermeasure generation network to obtain an expanded road surface image;
the data enhancement unit is used for carrying out data enhancement on the expanded road surface image to obtain a data enhanced road surface image;
and the labeling unit is used for labeling the data enhanced road surface image to obtain a sample set containing the road surface image.
Optionally, the device for identifying a road surface type further includes a countermeasure generation network improving module, configured to add a batch normalization layer between convolution layers of a generation network in an original countermeasure generation network before the processed road surface image is expanded by the improved countermeasure generation network, so as to obtain the improved countermeasure generation network.
Optionally, the image expansion unit may be specifically configured to obtain a random sampling hidden vector through prior distribution of the processed road surface image, and input the random sampling hidden vector into a generation network of an improved countermeasure generation network to obtain a generated picture; and inputting the generated pictures into a discrimination network of an improved countermeasure generation network to obtain the probability of the two classification tasks, and storing the generated pictures meeting the set probability into a file corresponding to the road surface type to obtain an expanded road surface image.
Optionally, the model training module 330 may be specifically configured to input the training set with the set proportion in the sample set into an improved convolutional neural network model, and train the improved convolutional neural network model to obtain a trained convolutional neural network model; in the training process, setting a training termination point according to the fluctuation state of the accuracy curve, and adjusting the convolutional neural network model according to the accuracy curve; and storing the trained convolutional neural network model by utilizing a high-level application programming interface library.
Optionally, the set type of road surface image includes: an ice film type, a compacted snow type, a loose snow type, a dry asphalt type, a wet asphalt type, a dry cement type, a wet cement type, a soil type, and a grass type.
The road surface type identification device provided by the embodiment of the invention can execute the road surface type identification method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 4, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the processors 410 in the electronic device may be one or more, and one processor 410 is taken as an example in fig. 4; the processor 410, the memory 420, the input device 430 and the output device 440 in the electronic apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 4.
The memory 420 may be used as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the road surface type identification method in the embodiment of the present invention (for example, the sample set acquisition module 310, the model improvement module 320, the model training module 330, the model verification module 340, and the type identification module 350 in the road surface type identification device). The processor 410 executes various functional applications and data processing of the electronic device by executing software programs, instructions and modules stored in the memory 420, that is, implements the above-described road surface type recognition method.
The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 420 may further include memory located remotely from processor 410, which may be connected to an electronic device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 430 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus. The output device 440 may include a display device such as a display screen.
Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method of road surface type identification, the method comprising:
acquiring a road surface image of a set type through a vehicle-mounted front camera, and preprocessing the road surface image to obtain a sample set containing the road surface image;
improving a pooling layer and a full-link layer in the convolutional neural network model, and adding a set network layer to obtain an improved convolutional neural network model;
inputting the training set with the set proportion in the sample set into an improved convolutional neural network model to obtain a trained convolutional neural network model;
inputting the verification set with the set proportion in the sample set into a trained convolutional neural network model, and verifying the trained convolutional neural network model;
and inputting the test set with the set proportion in the sample set into a trained convolutional neural network model to obtain a pavement type recognition result.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the operations of the method described above, and may also execute the relevant operations in the road surface type identification method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the road surface type identification device, each included unit and each included module are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be implemented; in addition, the specific names of the functional units are only for the convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A road surface type identification method, characterized by comprising:
acquiring a road surface image of a set type through a vehicle-mounted front camera, and preprocessing the road surface image to obtain a sample set containing the road surface image;
improving a pooling layer and a full-link layer in the convolutional neural network model, and adding a set network layer to obtain an improved convolutional neural network model;
inputting the training set with the set proportion in the sample set into an improved convolutional neural network model to obtain a trained convolutional neural network model;
inputting the verification set with the set proportion in the sample set into a trained convolutional neural network model, and verifying the trained convolutional neural network model;
and inputting the test set with the set proportion in the sample set into a trained convolutional neural network model to obtain a road surface type recognition result.
2. The method of claim 1, wherein the improving the pooling layer and the full-link layer in the convolutional neural network model and adding the set network layer to obtain an improved convolutional neural network model comprises:
setting the pooling layer in the convolutional neural network model as a maximum pooling layer, setting the number of neurons of the full-link layer of the last layer in the convolutional neural network model as a designated number, and adding a batch normalization layer and a Dropout layer to obtain an improved convolutional neural network model; wherein, the convolutional neural network model is a DenseNet model.
3. The method of claim 1, wherein the preprocessing the road surface image to obtain a sample set containing the road surface image comprises:
carrying out format processing on the road surface image to obtain a processed road surface image;
expanding the processed road surface image through an improved countermeasure generating network to obtain an expanded road surface image;
performing data enhancement on the expanded road surface image to obtain a data-enhanced road surface image;
labeling the data enhanced pavement image to obtain a sample set containing the pavement image.
4. The method of claim 3, further comprising, prior to augmenting the processed road surface image with the improved countermeasure generation network:
and adding a batch normalization layer between convolution layers of the generated network in the original countermeasure generating network to obtain the improved countermeasure generating network.
5. The method of claim 3, wherein the expanding the processed road surface image through the improved countermeasure generation network to obtain an expanded road surface image comprises:
acquiring a random sampling hidden vector through the prior distribution of the processed road surface image, and inputting the random sampling hidden vector into a generation network of an improved countermeasure generation network to obtain a generated picture;
and inputting the generated pictures into a discrimination network of an improved countermeasure generation network to obtain the probability of the two classification tasks, and storing the generated pictures meeting the set probability into a file corresponding to the road surface type to obtain an expanded road surface image.
6. The method of claim 1, wherein inputting the training set with the set proportion in the sample set into an improved convolutional neural network model, and obtaining the trained convolutional neural network model comprises:
inputting the training set with the set proportion in the sample set into an improved convolutional neural network model, and training the improved convolutional neural network model to obtain a trained convolutional neural network model; in the training process, setting a training termination point according to the fluctuation state of the accuracy curve, and adjusting the convolutional neural network model according to the accuracy curve;
and storing the trained convolutional neural network model by utilizing a high-level application programming interface library.
7. The method according to claim 1, wherein setting the type of road surface image comprises: an ice film type, a compacted snow type, a loose snow type, a dry asphalt type, a wet asphalt type, a dry cement type, a wet cement type, a soil type, and a grass type.
8. A road surface type identification device, characterized in that the device comprises:
the system comprises a sample set acquisition module, a data acquisition module and a data processing module, wherein the sample set acquisition module is used for acquiring a road surface image of a set type through a vehicle-mounted front camera and preprocessing the road surface image to obtain a sample set containing the road surface image;
the model improvement module is used for improving a pooling layer and a full connection layer in the convolutional neural network model and adding a set network layer to obtain an improved convolutional neural network model;
the model training module is used for inputting the training set with the set proportion in the sample set into an improved convolutional neural network model to obtain a trained convolutional neural network model;
the model verification module is used for inputting a verification set with a set proportion in the sample set into a trained convolutional neural network model and verifying the trained convolutional neural network model;
and the type identification module is used for inputting the test set with the set proportion in the sample set into the trained convolutional neural network model to obtain a pavement type identification result.
9. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the road surface type identification method of any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the road surface type recognition method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210114382.5A CN114463722A (en) | 2022-01-30 | 2022-01-30 | Pavement type identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210114382.5A CN114463722A (en) | 2022-01-30 | 2022-01-30 | Pavement type identification method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114463722A true CN114463722A (en) | 2022-05-10 |
Family
ID=81412383
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210114382.5A Pending CN114463722A (en) | 2022-01-30 | 2022-01-30 | Pavement type identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463722A (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111311538A (en) * | 2019-12-28 | 2020-06-19 | 北京工业大学 | Multi-scale lightweight road pavement detection method based on convolutional neural network |
-
2022
- 2022-01-30 CN CN202210114382.5A patent/CN114463722A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111311538A (en) * | 2019-12-28 | 2020-06-19 | 北京工业大学 | Multi-scale lightweight road pavement detection method based on convolutional neural network |
Non-Patent Citations (3)
| Title |
|---|
| 严春满 等: "《 卷积神经网络模型发展及应用》", 计算机科学与探索, 29 September 2020 (2020-09-29), pages 27 - 41 * |
| 王志红 等: "《基于语义分割模型的路面类型识别技术研究》", 公路交通科技, 15 January 2021 (2021-01-15), pages 128 - 133 * |
| 陆峰 等: "《基于深度学习的目标检测技术综述》", 计算机系统应用, 15 March 2021 (2021-03-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112734775B (en) | Image labeling, image semantic segmentation and model training methods and devices | |
| CN111242180B (en) | Image identification method and system based on lightweight convolutional neural network | |
| CN111275107A (en) | Multi-label scene image classification method and device based on transfer learning | |
| CN112101364B (en) | Semantic segmentation method based on parameter importance increment learning | |
| CN112507912A (en) | Method and device for identifying illegal picture | |
| CN113724286A (en) | Method and device for detecting saliency target and computer-readable storage medium | |
| CN114495060B (en) | Road traffic marking recognition method and device | |
| CN111832435A (en) | Beauty prediction method and device based on migration and weak supervision and storage medium | |
| CN118736226A (en) | Method and system for dividing small sample dam cracks based on general dividing large model | |
| CN115292538A (en) | Map line element extraction method based on deep learning | |
| KR20210111677A (en) | Method for clipping neural networks, method for calculating convolution of neural networks and apparatus for performing the methods | |
| CN114463722A (en) | Pavement type identification method and device, electronic equipment and storage medium | |
| US20240135679A1 (en) | Method for classifying images and electronic device | |
| CN113095185B (en) | Facial expression recognition method, device, equipment and storage medium | |
| CN113947154B (en) | Target detection method, target detection system, electronic equipment and storage medium | |
| CN112906824B (en) | Vehicle clustering method, system, device and storage medium | |
| CN116189130A (en) | Lane line segmentation method and device based on image annotation model | |
| CN111046934B (en) | SWIFT message soft clause recognition method and device | |
| CN114202765A (en) | Image text recognition method and storage medium | |
| CN116246128B (en) | Training method and device of detection model crossing data sets and electronic equipment | |
| CN116958176B (en) | Image segmentation method, device, computer equipment and medium | |
| CN113627537B (en) | Image recognition method, device, storage medium and equipment | |
| CN115761681A (en) | Neural network-based road surface static object detection method, system, equipment and medium | |
| CN116524244A (en) | Machine vision image processing method and system | |
| CN114663831A (en) | Vehicle-mounted image target detection method, system and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |