CN116152750A - Vehicle feature recognition method based on monitoring image - Google Patents
Vehicle feature recognition method based on monitoring image Download PDFInfo
- Publication number
- CN116152750A CN116152750A CN202211495622.7A CN202211495622A CN116152750A CN 116152750 A CN116152750 A CN 116152750A CN 202211495622 A CN202211495622 A CN 202211495622A CN 116152750 A CN116152750 A CN 116152750A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- image
- training
- priority
- label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000012544 monitoring process Methods 0.000 title claims abstract description 51
- 238000012549 training Methods 0.000 claims abstract description 99
- 238000001514 detection method Methods 0.000 claims abstract description 16
- 238000010276 construction Methods 0.000 claims abstract description 9
- 238000012360 testing method Methods 0.000 claims description 18
- 230000001502 supplementing effect Effects 0.000 claims description 9
- 238000002790 cross-validation Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 6
- 238000006116 polymerization reaction Methods 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 230000002776 aggregation Effects 0.000 claims 1
- 238000004220 aggregation Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 description 13
- 238000012795 verification Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 235000019580 granularity Nutrition 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013145 classification model Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013137 model compression technique Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The disclosure relates to a vehicle feature recognition method based on a monitoring image, and belongs to the technical field of vehicle type recognition. The vehicle characteristic recognition method based on the monitoring image comprises the steps of training a vehicle head and/or vehicle tail detector, wherein the step of training the vehicle head and/or vehicle tail detector comprises the following steps of: constructing a semantic hierarchy of a multi-level label for a training image, the multi-level label being used to characterize a vehicle model in the training image; detecting an image of a vehicle head and/or a vehicle tail using an object detection recognition technique; the vehicle training image is trained according to the multi-level label construction semantic softmax method. The method can identify the characteristics of the vehicle based on the monitoring image, further identify the specific model of the vehicle, effectively improve the information granularity of the training image, and can obtain more accurate identification results when identifying the vehicle with serious homogeneity.
Description
Technical Field
The disclosure relates to the technical field of vehicle type recognition, in particular to a vehicle characteristic recognition method based on a monitoring image.
Background
The intelligent transportation system is a comprehensive transportation system which effectively and comprehensively applies advanced scientific technologies (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operation research, artificial intelligence and the like) to transportation, service control and vehicle manufacturing, and strengthens the connection among vehicles, roads and users, thereby forming a comprehensive transportation system which ensures safety, improves efficiency, improves environment and saves energy.
Vehicle identification is one of important technologies in an intelligent traffic system, and is an essential ring for information acquisition and safety management in the intelligent traffic system. At present, the vehicle identification technology mainly processes a picture containing a vehicle acquired by a monitoring camera through a computer, and identifies the specific type of the vehicle.
However, since the model for vehicle type recognition in the computer is generally trained based on the close-up photos of the whole vehicle, the close-up photos are generally trained at specific angles (for example, the looking-down angle of the right front or the left front of the vehicle when the vehicle is normally parked, or the front angle of the vehicle, etc.), and the accuracy of the model for vehicle type recognition trained according to the close-up photos of the whole vehicle is difficult to be suitable for the requirement of vehicle type recognition under the monitoring angle.
Disclosure of Invention
An object of the present disclosure is to provide a vehicle feature recognition method based on a monitoring image, which can be applied to the need of high accuracy of vehicle type recognition under a monitoring view.
In order to achieve the above object, according to a first aspect of the present disclosure, there is provided a vehicle feature recognition method based on a monitoring image, including:
training vehicle head and/or tail detectors;
the step of training the vehicle head and/or tail detector comprises:
constructing a semantic hierarchy of a multi-level label for a training image, the multi-level label being used to characterize a vehicle model in the training image; detecting an image of a vehicle head and/or a vehicle tail using an object detection recognition technique; the vehicle training image is trained according to the multi-level label construction semantic softmax method.
As an optional technical scheme, the multi-level label at least comprises a first priority label, a second priority label and a third priority label with sequentially reduced priorities;
wherein each training image includes at least one label.
As an optional technical solution, in the multi-level tag, the first priority tag is a vehicle year, the second priority tag is a vehicle train, and the third priority tag is a vehicle brand;
if the training image only has the first priority label, supplementing the training image with the second priority label and the third priority label;
if the training image only has the second priority label, or the training image has the first priority label and the second priority label at the same time, supplementing the training image with the third priority label.
As an optional technical solution, the detecting the image of the vehicle head and/or the vehicle tail by using the object detection and identification technology specifically includes the following steps:
acquiring RGB image frames;
acquiring a target image containing a vehicle in an image frame;
inputting the target image as an original image into a vehicle head detector and/or a vehicle tail detector to detect the position of the vehicle head and/or the vehicle tail in the target image;
intercepting an image of a vehicle head and/or a vehicle tail from the target image;
inputting the intercepted images of the head and/or the tail of the vehicle into a neural network multi-level semantic softmax classifier to judge the classification result.
As an optional solution, the acquiring an RGB image frame includes: and extracting from the monitoring video.
As an optional solution, the detecting the image of the vehicle head and/or the vehicle tail by using the object detection and identification technology further includes:
dividing the data set with the vehicle head and/or the vehicle tail into a training set, a cross-validation set and a test set;
the training set is used for training a YOLOV5 neural network model;
the cross validation set is used for debugging the YOLOV5 neural network model to enable the model to be converged to a certain calculation loss so as to obtain a trained final model;
the test set is used to test the final model.
As an optional solution, the step of training the vehicle model sample according to the semantic softmax method of multi-level label construction includes: a corresponding softmax hierarchy is built for each level of labels, and each softmax activation function will learn the associated classification experience from the corresponding level.
As an alternative, a balanced polymerization loss L is obtained tot For semantic softmax training;
the polymerization loss L tot The method meets the following conditions:
in which W is k Calculating a weight coefficient for each layer, the weight coefficient comprising N of the types of labels from layer 0 to layer j j Determining that the relative occurrence coefficient is O k K is the total layer number, L k For loss of each layer.
As an optional technical solution, the vehicle feature recognition method based on the monitoring image adopts a lightweight network as a backbone network.
As an optional solution, the lightweight network includes any one of MobileNet, shuffleNet, squeezeNet or Xception.
Through the technical scheme, the vehicle characteristics can be identified based on the monitoring images, so that the specific model of the vehicle can be identified, and for example, the brand, the train and the year of the vehicle can be identified; moreover, the recognition accuracy and the recognition speed of the recognition method under the monitoring image are far superior to those of the existing recognition model. Wherein the step of training the vehicle head and/or tail detector comprises: constructing a semantic hierarchy of a multi-level label for the training image, wherein the multi-level label is used for representing the model of the vehicle in the training image; detecting an image of a vehicle head and/or a vehicle tail using an object detection recognition technique; the vehicle training image is trained according to the multi-level label construction semantic softmax method.
Therefore, the information granularity of the training image can be effectively improved; specifically, the multi-level label of the training image is introduced into the method for constructing the semantic softmax, so that the training image can be identified and classified under a plurality of classification models, different levels of the models can be smoothed, and further, when a specific image (such as a monitoring image) is identified, the method disclosed by the invention has robustness on a plurality of label levels, and also can obtain more accurate identification results when a vehicle with serious homogeneity is identified.
According to a second aspect of the present disclosure, there is provided a monitoring image-based vehicle feature recognition apparatus, the apparatus comprising: a processor and a communication interface; the communication interface is coupled to a processor for running a computer program or instructions to implement the surveillance image based vehicle feature recognition method as described in any one of the possible implementations of the first aspect and the first aspect.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium having instructions stored therein which, when run on a terminal, cause the terminal to perform a method of monitoring image based vehicle feature identification as described in any one of the possible implementations of the first aspect and the first aspect.
According to a fourth aspect of the present disclosure, the presently disclosed embodiments provide a computer program product containing instructions that, when run on a monitoring image based vehicle feature recognition device, cause the monitoring image based vehicle feature recognition device to perform the monitoring image based vehicle feature recognition method as described in any one of the possible implementations of the first aspect and the first aspect.
Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
FIG. 1 is a schematic step diagram of a method for monitoring image-based vehicle feature identification provided in one embodiment of the present disclosure;
FIG. 2 is a schematic step diagram of a method for monitoring image-based vehicle feature identification provided in another embodiment of the present disclosure;
FIG. 3 is a schematic illustration of steps provided by one embodiment of the present disclosure for detecting images of a vehicle head and/or tail using object detection recognition techniques;
FIG. 4 is a schematic diagram of steps for constructing a first priority label, a second priority label, and a third priority label in one embodiment of the present disclosure;
fig. 5 is a schematic diagram of detection of a vehicle head provided by an embodiment of the present disclosure.
Detailed Description
Specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the disclosure, are not intended to limit the disclosure.
As shown in fig. 1 to 5, according to an aspect of the present disclosure, there is provided a vehicle feature recognition method based on a monitoring image, including:
training vehicle head and/or tail detectors;
the step of training the vehicle head and/or tail detector comprises:
constructing a semantic hierarchy of a multi-level label for a training image, the multi-level label being used to characterize a vehicle model in the training image; detecting an image of a vehicle head and/or a vehicle tail using an object detection recognition technique; the vehicle training image is trained according to the multi-level label construction semantic softmax method.
Through the technical scheme, the vehicle characteristics can be identified based on the monitoring images, so that the specific model of the vehicle can be identified, and for example, the brand, the train and the year of the vehicle can be identified; moreover, the recognition accuracy and the recognition speed of the recognition method under the monitoring image are far superior to those of the existing recognition model. Wherein the step of training the vehicle head and/or tail detector comprises: constructing a semantic hierarchy of a multi-level label for the training image, wherein the multi-level label is used for representing the model of the vehicle in the training image; detecting an image of a vehicle head and/or a vehicle tail using an object detection recognition technique; the vehicle training image is trained according to the multi-level label construction semantic softmax method.
Therefore, the information granularity of the training image can be effectively improved; specifically, the multi-level label of the training image is introduced into the method for constructing the semantic softmax, so that the training image can be identified and classified under a plurality of classification models, different levels of the models can be smoothed, and further, when a specific image (such as a monitoring image) is identified, the method disclosed by the invention has robustness on a plurality of label levels, and also can obtain more accurate identification results when a vehicle with serious homogeneity is identified.
According to the actual data testing effect, the vehicle characteristic recognition method based on the monitoring image can improve the recognition accuracy of a certain characteristic (such as a vehicle brand) of a vehicle in a monitoring scene, and the recognition accuracy can be improved by at least 21%. The improvement of the identification accuracy is enough to meet the requirements of intelligent transportation in daily life.
Furthermore, the softmax function is a commonly used multi-classifier in machine learning, and the softmax function is a general form of logistic function, which is a way to transform classification problems into probability problems. The specific principles and working processes thereof have been described with respect to various types of neural networks and deep learning applications, and the disclosure will not be repeated. The term "vehicle" in this disclosure should be construed broadly to include all types of motor vehicles, i.e., vehicles of this disclosure including, but not limited to, passenger vehicles (which should include electrically-driven vehicles, gas-powered vehicles, fuel-powered vehicles, hybrid vehicles, buses, and specialty vehicles, etc.) and trucks (which should include flatcars, open cars, tank cars, vans, and insulated vehicles, etc.).
It is to be understood that the identification method of the present disclosure can be applied to a variety of scenarios including, but not limited to, vehicle identification in a monitoring image, vehicle identification in a normal image, and vehicle identification in an incomplete image (the incomplete image including at least an image of a vehicle head or a vehicle tail). As shown in fig. 5, the present disclosure is illustrated by taking a monitoring image as an example, but it is not meant that the scope of protection of the present disclosure is limited only to vehicle identification in the monitoring image.
The number of the multi-level tags is not particularly limited in the present disclosure, for example, in one embodiment of the present disclosure, as shown in fig. 4 and 5, the multi-level tags of the present disclosure may include a first priority tag, a second priority tag, and a third priority tag, which sequentially decrease in priority; wherein each training image includes at least one label. For example, a training image may include only first priority tags; the method can also comprise a first priority label and a second priority label, or a first priority label and a third priority label, or a second priority label and a third priority label; a first priority label, a second priority label, and a third priority label may also be included. In this way, the specific model or information of the vehicle in the training image can be more accurately represented by the labels of the multiple levels, so that the recognition model obtained by training according to the multi-level label construction semantic softmax method is more accurate, more accurate indexes can be obtained for a sample with a certain label missing in a training verification evaluation stage, model convergence of the method is more accurately assisted, and recognition accuracy and verifiability of the recognition method are improved.
In another embodiment of the present disclosure, the multi-level tag of the present disclosure may also include a first priority tag and a second priority tag having sequentially decreasing priorities, or the multi-level tag of the present disclosure may include a first priority tag, a second priority tag, a third priority tag, and a fourth priority tag having sequentially decreasing priorities. It can be understood that the smaller the number of the multi-level labels is, the more favorable the quick establishment of the identification model is, and the faster the identification rate of the image to be identified is; the more the number of the multi-level labels is, the more the accuracy and precision of the identification model are facilitated, and the higher the identification accuracy of the image to be identified is.
The specific characterization information of the multi-level tag is not specifically limited in the present disclosure, for example, in an exemplary embodiment of the present disclosure, as shown in fig. 4 and 5, in the multi-level tag of the present disclosure, the first priority tag may be a year of a vehicle, the second priority tag may be a train of the vehicle, and the third priority tag may be a brand of the vehicle; if the training image only has the first priority label, supplementing the training image with the second priority label and the third priority label; if the training image only has the second priority label, or the training image has the first priority label and the second priority label at the same time, supplementing the training image with the third priority label.
For example, in one example, as shown in fig. 4, the training image has only the first priority label (vehicle year): golf-2021, the training image may be automatically supplemented with a second priority label (vehicle train): golf, and third priority label (vehicle brand): and is popular. For example, in another example, the training image has only the second priority label (vehicle train): feast (canne), the training image may be automatically supplemented with a third priority label (vehicle brand): porsche (Porsche).
Therefore, the vehicle model in the training image can be thinned as far as possible by means of the multi-level label, so that the recognition accuracy of the recognition model of the disclosure can be improved, the recognition granularity of the recognition model of the disclosure can be enlarged, the prediction deviation of the recognition model of the disclosure under different semantic levels can be balanced, and a more robust result can be obtained under the classification conditions of different granularities.
In another embodiment of the present disclosure, the multi-level tag of the present disclosure may include a first priority tag, a second priority tag, a third priority tag, and a fourth priority tag, where the first priority tag may be a tail tag, the second priority tag may be a year of a vehicle, the third priority tag may be a train of the vehicle, and the fourth priority tag may be a brand of the vehicle. If the training image only has the first priority label, supplementing the training image with the second priority label, the third priority label and the fourth priority label; if the training image only has the second priority label, or the training image has the first priority label and the second priority label at the same time, supplementing the training image with the third priority label and the fourth priority label; if the training image only has the third priority label, or the training image has the second priority label and the third priority label at the same time, supplementing the training image with the fourth priority label.
For example, in another example, the training image has only a first priority label (vehicle trailer): golf-2021-200 TSI, the training image may be automatically supplemented with a second priority label (vehicle year): golf-2021, third priority label (vehicle train): golf, and fourth priority label (vehicle brand): and is popular.
In this way, the accuracy and precision of the recognition model can be further improved, and the higher the recognition accuracy of the image to be recognized in the present disclosure is.
In an optional technical solution of the present disclosure, the detecting an image of a vehicle head and/or a vehicle tail by using an object detection and recognition technology of the present disclosure specifically includes the following steps:
acquiring RGB image frames;
acquiring a target image containing a vehicle in an image frame;
inputting the target image as an original image into a vehicle head detector and/or a vehicle tail detector to detect the position of the vehicle head and/or the vehicle tail in the target image;
intercepting an image of a vehicle head and/or a vehicle tail from the target image;
inputting the intercepted images of the head and/or the tail of the vehicle into a neural network multi-level semantic softmax classifier to judge the classification result.
Therefore, the vehicle head detector and/or the vehicle tail detector can be used for detecting partial images of the vehicles contained in the images to be detected (namely RGB image frames), the characteristics of the vehicles in the images to be detected can be effectively identified, and then a judgment and classification result is output, so that the intelligent transportation system can conveniently process the next step.
In an optional technical solution of the present disclosure, the acquiring an RGB image frame of the present disclosure includes: and extracting from the monitoring video.
It is understood that the RGB images obtained by the present disclosure may be from monitoring videos recorded by any device or apparatus, including but not limited to monitoring videos collected by cameras such as infrared waterproof gun type cameras, infrared spherical cameras, constant speed dome cameras, high speed dome cameras, wide dynamic cameras, video terminals, video cards, DVR (Digital Video Recorder) video recorders, and the like. The surveillance video according to the present disclosure may be in various formats, and the present disclosure is not limited thereto, for example, the surveillance video of the present disclosure may include: AVI (Audio Video Interleave) format, WMV (Windows Media Video) format, MPEG (Moving Pictures Expert Group) format, quickTime (i.e., file nominal ". Mov" ending) format, realVideo (i.e., file nominal ". Rm" or ". Ram" ending) format, flash (i.e., file nominal ". Swf" or ". Flv" ending), mpeg-4 (i.e., file nominal ". Mp4" ending) format.
In an optional aspect of the disclosure, the detecting the image of the vehicle head and/or the vehicle tail using the object detection and recognition technology of the disclosure further includes:
dividing the data set with the vehicle head and/or the vehicle tail into a training set, a cross-validation set and a test set;
the training set is used for training a YOLOV5 neural network model;
the cross validation set is used for debugging the YOLOV5 neural network model to enable the model to be converged to a certain calculation loss so as to obtain a trained final model;
the test set is used to test the final model.
In this way, the training set may have the effect of training the vehicle head and/or vehicle tail detector model of the present disclosure, determining internal parameters of the vehicle head and/or vehicle tail detector model of the present disclosure, such that the vehicle head and/or vehicle tail detector of the present disclosure may be suitable for vehicle feature recognition in RGB images of any source; also, the cross-validation set of the present disclosure may determine the grid structure of the vehicle head and/or vehicle tail detector model of the present disclosure and adjust the superparameters of the vehicle head and/or vehicle tail detector model of the present disclosure (superparameters refer to relevant parameters set according to modeling experience, e.g., the superparameters of the present disclosure may be iteration times, level times, and parameters of neurons per layer, learning rates, etc.); in addition, the test set of the present disclosure may verify the generalization ability of the vehicle head and/or tail detector models of the present disclosure, truly conforming to the need for verifying the resulting vehicle head and/or tail detector models of the present disclosure in a manner sufficient to meet the needs of vehicle feature recognition in various surveillance images.
In addition, it should be noted that the data volume ratio of the training set, the cross test set, and the validation set of the present disclosure may be selected according to actual needs. For example, in one embodiment of the present disclosure, when the data amount is small, for example, the data amount is around ten thousand levels (that is, when the total amount of the training image, the test image, and the verification image included in the training set, the cross test set, and the verification set is Zhang Zuoyou), the data amount ratio of the training set, the cross test set, and the verification set may be set as: 7:2:1. In this way, the vehicle head and/or tail detector models of the present disclosure are facilitated to be quickly built and the accuracy (where there are fewer identified features) is high. In another embodiment of the present disclosure, when the data amount is large, for example, when the data amount is on the order of tens of millions (that is, when the total amount of the training image, the test image, and the verification image included in the training set, the cross test set, and the verification set is on the order of tens of millions Zuo You), the data amount ratio of the training set, the cross test set, and the verification set may be set as: 97:2:1. In this way, it can be advantageous to promote the accuracy of the vehicle head and/or vehicle tail detector models of the present disclosure, and the accuracy (even in cases where there are more features to identify) can be higher.
In an optional technical solution of the present disclosure, the step of training the vehicle model sample according to the multi-level label construction semantic softmax method of the present disclosure includes: a corresponding softmax hierarchy is built for each level of labels, and each softmax activation function will learn the associated classification experience from the corresponding level.
As an alternative, a balanced polymerization loss L is obtained tot For semantic softmax training;
the polymerization loss L tot The method meets the following conditions:
in which W is k Calculating a weight coefficient for each layer, the weight coefficient comprising N of the types of labels from layer 0 to layer j j Determining that the relative occurrence coefficient is O k K is the total layer number, L k For loss of each layer.
Relative to a method employing naive total loss (i.e., computation
The value of (i), i.e. loss L for each layer k Sum), the naive approach to summing losses ignores the fact that: that is, under a lower hierarchy, the number of times the softmax layer is activated at the lower hierarchy may be high compared to the softmax layer at the higher hierarchy, thereby overstressing the category of the lower hierarchy. The classification result based on the semantic hierarchy softmax of the present disclosure can obtain prediction results under multiple semantic hierarchies for one input image to be identified (for example, a monitoring image), and more importantly, in the total loss calculation mode of the present disclosure, during training, each training image can obtain an independent classification on each semantic layer L. Therefore, as the semantic hierarchy softmax loss balances the deviation of classification results under different hierarchies, the influence of sample data with lower data labeling quality on model convergence can be effectively improved, and better classification results are provided for cooler vehicle types. On the other hand, the vehicle appearance is seriously homogenized, and the vehicle model recognition model trained by using the method disclosed by the invention can be aimed at different brandsThe similar vehicle models are observed, and more accurate brand prediction results are achieved. In addition, the model is capable of reasonably predicting the results of the new model (e.g., brand and family of the new model) from different levels (e.g., brand, family, year) for the new model lacking training sample data.
In an optional technical solution of the present disclosure, the vehicle feature recognition method based on the monitoring image of the present disclosure uses a lightweight network as a backbone network. The adoption of the lightweight network (shallow network) can effectively reduce calculation time consumption on the basis of ensuring the real-time identification effect in the monitored scene, and is beneficial to improving the running efficiency of the vehicle head and/or vehicle tail detector model.
As an optional solution, the lightweight network includes any one of MobileNet, shuffleNet, squeezeNet or Xception.
For example, in an exemplary embodiment of the present disclosure, when the lightweight network of the present disclosure employs a ShuffleNet, the main Module of the ShuffleNet is Fire Module, which mainly from the perspective of network architecture optimization, can reduce network parameters from at least the following 3-point strategy, improving network performance: first, 1 x 1 convolution can be used instead of partial 3 x 3 convolution, and parameters can be reduced to 1/9 of the original while reducing the number of input channels. Second, a 1×1 convolution can be utilized to reduce the number of input channels. Thirdly, after the number of channels is reduced, convolution kernels with multiple sizes are used for calculation, so that more information is reserved, and the classification accuracy is improved.
In another embodiment of the present disclosure, the lightweight network of the present disclosure may employ SqueezeNet, squeezeNet as a carefully designed lightweight network that not only performs similar to AlexNet, but also has model parameters of only 1/50 of AlexNet. The size of the model can be further compressed using common model compression techniques such as SVD, pruning, quantization, and the like. For example, when compressed using Deep compression techniques, the model size can be compressed to 0.5MB with little loss in performance. Based on the characteristic of light weight of the SquezeNet, the SquezeNet can be widely applied to any mobile terminal, and the deployment and application of the detection technology of the vehicle head and/or the vehicle tail in the mobile terminal are promoted.
In another embodiment of the present disclosure, the lightweight network of the present disclosure may employ a manner in which MobileNet, mobileNet may utilize more efficient depth separable convolution (Depthwise Separable Convolution), which may further accelerate the deployment and application of the detection techniques of the vehicle head and/or vehicle tail of the present disclosure to mobile terminals.
According to a second aspect of the present disclosure, an embodiment of the present disclosure provides a monitoring image-based vehicle feature recognition apparatus, the apparatus including: a processor and a communication interface; the communication interface is coupled to a processor for running a computer program or instructions to implement the surveillance image based vehicle feature recognition method as described in any one of the possible implementations of the first aspect and the first aspect.
According to a third aspect of the present disclosure, embodiments of the present disclosure provide a computer-readable storage medium having instructions stored therein that, when run on a terminal, cause the terminal to perform a surveillance image-based vehicle feature recognition method as described in any one of the possible implementations of the first aspect and the first aspect.
According to a fourth aspect of the present disclosure, embodiments of the present disclosure provide a computer program product containing instructions that, when run on a monitoring image based vehicle feature recognition device, cause the monitoring image based vehicle feature recognition device to perform the monitoring image based vehicle feature recognition method as described in any one of the possible implementations of the first aspect and the first aspect.
The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure, and all the simple modifications belong to the protection scope of the present disclosure.
In addition, the specific features described in the foregoing embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, the present disclosure does not further describe various possible combinations.
Moreover, any combination between the various embodiments of the present disclosure is possible as long as it does not depart from the spirit of the present disclosure, which should also be construed as the disclosure of the present disclosure.
Claims (10)
1. A method for identifying characteristics of a vehicle based on a monitoring image, comprising:
training vehicle head and/or tail detectors;
the step of training the vehicle head and/or tail detector comprises:
constructing a semantic hierarchy of a multi-level label for a training image, the multi-level label being used to characterize a vehicle model in the training image; detecting an image of a vehicle head and/or a vehicle tail using an object detection recognition technique; constructing a semantic softmax method according to the multi-level label to train a vehicle training image;
the resulting vehicle head and/or vehicle tail detector is used to identify the surveillance image.
2. The monitoring image-based vehicle feature recognition method according to claim 1, wherein the multi-level tag includes a first priority tag, a second priority tag, and a third priority tag, which are sequentially reduced in priority;
wherein each training image includes at least one label.
3. The monitoring image-based vehicle feature recognition method according to claim 2, wherein in the multi-level tag, the first priority tag is a vehicle year, the second priority tag is a vehicle train, and the third priority tag is a vehicle brand;
if the training image only has the first priority label, supplementing the training image with the second priority label and the third priority label;
if the training image only has the second priority label, or the training image has the first priority label and the second priority label at the same time, supplementing the training image with the third priority label.
4. The method for recognizing vehicle characteristics based on monitoring image according to claim 1, wherein the detecting the image of the head and/or the tail of the vehicle using the object detection recognition technique comprises the steps of:
acquiring RGB image frames;
acquiring a target image containing a vehicle in an image frame;
inputting the target image as an original image into a vehicle head detector and/or a vehicle tail detector to detect the position of the vehicle head and/or the vehicle tail in the target image;
intercepting an image of a vehicle head and/or a vehicle tail from the target image;
inputting the intercepted images of the head and/or the tail of the vehicle into a neural network multi-level semantic softmax classifier to judge the classification result.
5. The monitoring image-based vehicle feature recognition method of claim 4, wherein the acquiring RGB image frames comprises: and extracting from the monitoring video.
6. The monitored image-based vehicle feature recognition method of claim 4, wherein said detecting an image of a vehicle head and/or a vehicle tail using an object detection recognition technique further comprises:
dividing the data set with the vehicle head and/or the vehicle tail into a training set, a cross-validation set and a test set;
the training set is used for training a YOLOV5 neural network model;
the cross validation set is used for debugging the YOLOV5 neural network model to enable the model to be converged to a certain calculation loss so as to obtain a trained final model;
the test set is used to test the final model.
7. The method for monitoring image based vehicle feature recognition of claim 1, wherein the step of training the vehicle model sample according to the multi-level tag construction semantic softmax method comprises: a corresponding softmax hierarchy is built for each level of labels, and each softmax activation function will learn the associated classification experience from the corresponding level.
8. The method for monitoring image-based vehicle feature recognition according to claim 7, wherein a balanced aggregation loss L is obtained tot For semantic softmax training;
the polymerization loss L tot The method meets the following conditions:
in which W is k Calculating a weight coefficient for each layer, the weight coefficient comprising N of the types of labels from layer 0 to layer j j Determining that the relative occurrence coefficient is O k K is the total layer number, L k For loss of each layer.
9. The monitoring image-based vehicle feature recognition method according to any one of claims 1 to 8, characterized in that the monitoring image-based vehicle feature recognition method employs a lightweight network as a backbone network.
10. The monitored image based vehicle feature recognition method according to claim 9, wherein said lightweight network comprises any one of MobileNet, shuffleNet, squeezeNet or Xception.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211495622.7A CN116152750A (en) | 2022-11-27 | 2022-11-27 | Vehicle feature recognition method based on monitoring image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211495622.7A CN116152750A (en) | 2022-11-27 | 2022-11-27 | Vehicle feature recognition method based on monitoring image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116152750A true CN116152750A (en) | 2023-05-23 |
Family
ID=86355274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211495622.7A Pending CN116152750A (en) | 2022-11-27 | 2022-11-27 | Vehicle feature recognition method based on monitoring image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116152750A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117456416A (en) * | 2023-11-03 | 2024-01-26 | 北京饼干科技有限公司 | Method and system for intelligently generating material labels |
-
2022
- 2022-11-27 CN CN202211495622.7A patent/CN116152750A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117456416A (en) * | 2023-11-03 | 2024-01-26 | 北京饼干科技有限公司 | Method and system for intelligently generating material labels |
| CN117456416B (en) * | 2023-11-03 | 2024-06-07 | 北京饼干科技有限公司 | Method and system for intelligently generating material labels |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368687B (en) | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation | |
| CN113378890B (en) | Lightweight pedestrian vehicle detection method based on improved YOLO v4 | |
| CN111079640B (en) | Vehicle type identification method and system based on automatic amplification sample | |
| US9864923B2 (en) | Assisted surveillance of vehicles-of-interest | |
| CN112776650B (en) | Multi-element fusion perception intelligent charging system and method | |
| WO2010141369A1 (en) | Semantic scene segmentation using random multinomial logit (rml) | |
| CN109948643A (en) | A kind of type of vehicle classification method based on deep layer network integration model | |
| CN111178235A (en) | Target quantity determination method, device, equipment and storage medium | |
| CN111428735A (en) | Truck brand classification method based on migration learning deep network fusion model | |
| CN114639067A (en) | Multi-scale full-scene monitoring target detection method based on attention mechanism | |
| CN112613434B (en) | Road target detection method, device and storage medium | |
| CN113762144A (en) | Deep learning-based black smoke vehicle detection method | |
| CN116152750A (en) | Vehicle feature recognition method based on monitoring image | |
| CN110503049B (en) | Satellite video vehicle number estimation method based on generation countermeasure network | |
| CN117651976A (en) | Defect detection method and device | |
| CN114596548A (en) | Target detection method, target detection device, computer equipment and computer-readable storage medium | |
| CN114359196A (en) | Fog detection method and system | |
| CN113609320A (en) | Truck re-identification retrieval method based on multi-branch feature fusion | |
| WO2022228325A1 (en) | Behavior detection method, electronic device, and computer readable storage medium | |
| CN116977995A (en) | Vehicle-mounted front license plate recognition method and system | |
| CN113780462B (en) | Vehicle detection network establishment method based on unmanned aerial vehicle aerial image and application thereof | |
| CN115171080A (en) | Compressed video driver behavior identification method for vehicle-mounted equipment | |
| CN114495160A (en) | Pedestrian detection method and system based on improved RFBNet algorithm | |
| CN115393743A (en) | Vehicle detection method based on double-branch encoding and decoding network, unmanned aerial vehicle and medium | |
| CN111881843A (en) | Taxi passenger carrying number counting method based on face detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |