CN114387571A - Hierarchical matching-based method and device for unsupervised vehicle re-identification - Google Patents

Abstract

The invention discloses an unsupervised vehicle re-identification method and a device based on hierarchical matching, wherein the method comprises the following steps: acquiring a picture of a vehicle to be detected from a video source; extracting overall characteristics, vehicle local component information, vehicle body color information and vehicle type information of a vehicle from a picture of the vehicle to be detected; carrying out first processing on the overall characteristics of the vehicle to obtain the overall similarity of the vehicle; performing second processing on the vehicle local component information to obtain a vehicle local characteristic distance value; performing third processing on the vehicle body color information and the vehicle type information to obtain a color characteristic vector and a vehicle type characteristic vector; carrying out feature fusion on the overall vehicle similarity, the local vehicle feature distance value, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity; and determining a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set according to the vehicle similarity. The invention improves the accuracy of vehicle weight recognition and can be widely applied to the technical field of vehicle recognition.

Description

Hierarchical matching-based method and device for unsupervised vehicle re-identification

technical field

The invention relates to the technical field of vehicle identification, in particular to an unsupervised vehicle re-identification method and device based on hierarchical matching.

Background technique

Vehicle re-identification problem refers to the retrieval problem of judging whether the vehicles in the images captured by cameras installed in different areas belong to the same vehicle in a specific traffic monitoring scenario. The vehicle re-identification algorithm can be applied to many difficult problems such as vehicle identification, tracking and positioning, and further improves the security and reliability of the monitoring system.

Vehicle re-identification is still an extremely challenging problem due to constraints such as complex traffic scenes, limited camera viewpoints, and illumination changes. At present, license plate recognition technology is the simplest and most direct method to judge the identity of a vehicle, but there are still obvious drawbacks only relying on this method. For example, traffic monitoring has a relatively fixed viewing angle due to the limitation of the camera position. In complex scenes such as congestion queuing, blocking by large vehicles, or drivers deliberately blocking fake license plates, the difficulty of license plate detection increases or is no longer applicable. In order to make up for the shortcomings of license plate recognition, more and more researchers have begun to use the visual information of vehicles other than the number plate for vehicle retrieval tasks. In the past research in the field of vision, vehicle retrieval mainly involves the retrieval and identification of content information such as vehicle color or model, but this often fails to accurately retrieve a specific vehicle, but obtains the same content as the target vehicle (such as color, model )Vehicles. In recent years, the research on vehicle re-identification, a sub-field of vehicle retrieval, has gradually become the focus of research. The shortcomings of the existing vehicle re-identification technology are mainly reflected in two aspects: First, under different camera perspectives, the characteristics of the same vehicle are quite different, resulting in large intra-class differences in the data of the vehicle re-identification task. The second is that the differences between different vehicles with similar global features (such as color, model, etc.) under multi-view are further reduced, resulting in smaller inter-class differences in the vehicle re-identification task.

SUMMARY OF THE INVENTION

In view of this, embodiments of the present invention provide a method and device for unsupervised vehicle re-identification based on hierarchical matching with high accuracy.

A first aspect of the present invention provides an unsupervised vehicle re-identification method based on hierarchical matching, including:

Obtain the image of the vehicle to be detected from the video source;

Extracting overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the image of the vehicle to be detected;

Perform a first process on the overall characteristics of the vehicle to obtain the overall similarity of the vehicle;

performing a second process on the vehicle local component information to obtain a vehicle local feature distance value;

performing a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

Performing feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity;

According to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected is determined from all vehicles in the candidate vehicle set.

Optionally, the obtaining a picture of the vehicle to be detected from a video source includes:

Acquiring image frames from the video source according to preset time intervals;

Perform target detection on the extracted image frame to find the vehicle target in the image frame;

A vehicle set is obtained by intercepting an image frame with a vehicle target, and the vehicle set is divided into a search picture set and a candidate picture set.

Optionally, the step of acquiring the picture of the vehicle to be detected from the video source further includes:

Calculate the intersection ratio between the detection frame of any current image frame and the detection frame of the corresponding previous image frame;

When the intersection ratio is greater than a preset threshold, remove the detection result corresponding to the current image frame;

Wherein, the calculation formula of the intersection ratio is:

Among them, IoU represents the intersection ratio; R ₁ is the detection frame of the previous image frame; R ₂ is the detection frame of the current image frame; area(R ₁ )∩area(R ₂ ) finds the area intersection between the two detection frames ;area(R ₁ )∪area(R ₂ ) finds the area union of the two detection boxes.

Optionally, the extraction of overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected includes:

The vehicle is divided into overall vehicle features and vehicle local component information through the ResNet50 feature extractor;

Through global average pooling, the image of the vehicle to be detected is processed to obtain the overall characteristics of the vehicle;

Output the overall characteristics of the vehicle through the CNN network;

Compare the features of each view angle of the vehicle to obtain the view angle visibility score under each view angle of the vehicle;

After calculating the weight of the visual angle visibility score, the local feature distance value is obtained by combining with the Euclidean distance calculation method, and the local component information of the vehicle is determined;

The color feature vector and the model feature vector of all vehicles are obtained by feature extraction through the Resnet50 network, and the vehicle body color information and the model information are determined.

Optionally, the extraction of overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected further includes:

Divide the vehicles in the picture of the vehicle to be detected into four directions: front, top, back and side;

Calculate the area values of the four directions of the vehicle as the view angle visibility score, and use the view angle visibility score as the confidence value of the local features corresponding to the vehicle, and the confidence value is the confidence value of the four local features of the vehicle, using It is used to characterize the weight of the feature distance of each component between the vehicles in the subsequent calculation of the local feature distance.

Optionally, the calculation formula for calculating the weight of the visual angle visibility score is:

代表所述视角可视性分数的权重；

代表待检测车辆的可视性分数；

代表图像库中对应的车辆的可视性分数；m,n代表待检测车辆与图像库中对应的车辆；i代表可视性分数和权重对应的区域；N代表待搜索的图像库中车辆数；in,

a weight representing the visibility score of said view;

represents the visibility score of the vehicle to be detected;

represents the visibility score of the corresponding vehicle in the image database; m, n represent the vehicle to be detected and the corresponding vehicle in the image database; i represents the area corresponding to the visibility score and weight; N represents the number of vehicles in the image database to be searched ;

The calculation formula of the local feature distance value is:

Among them, D ^m,n represents the local feature distance value; D represents the Euclidean distance of the vehicle in all directions; f _i ^m represents the feature information of the vehicle to be detected; f _i ⁿ represents the feature information of the corresponding vehicle in the image library.

Optionally, performing feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity, including:

Performing L2 norm normalization processing on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain normalized features;

According to the normalized features, calculate the initial re-identification distance value between the vehicle to be detected and the corresponding vehicle in the image library;

The initial re-recognition distance value of the overall similarity of the vehicle, the initial re-recognition distance value of the vehicle local feature distance value, the initial re-recognition distance value of the color feature vector and the initial re-recognition distance of the vehicle type feature vector The value is weighted and summed to obtain the total weight identification distance value;

The vehicle similarity is determined according to the total weight identification distance value.

Another aspect of the embodiments of the present invention also provides an apparatus for unsupervised vehicle re-identification based on hierarchical matching, including:

The first module is used to obtain the picture of the vehicle to be detected from the video source;

The second module is used for extracting overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected;

The third module is used to perform the first processing on the overall characteristics of the vehicle to obtain the overall similarity of the vehicle;

a fourth module, configured to perform second processing on the vehicle local component information to obtain a vehicle local characteristic distance value;

a fifth module, configured to perform third processing on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

The sixth module is used to perform feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity;

The seventh module is configured to determine, according to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set.

Another aspect of the embodiments of the present invention further provides an electronic device, including a processor and a memory;

the memory is used to store programs;

The processor executes the program to implement the method as described above.

Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the aforementioned method.

The embodiment of the present invention also discloses a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The computer instructions can be read from the computer-readable storage medium by a processor of the computer device, and the processor executes the computer instructions to cause the computer device to perform the foregoing method.

The embodiment of the present invention obtains a picture of a vehicle to be detected from a video source; extracts overall vehicle features, vehicle local component information, body color information and vehicle type information from the picture of the vehicle to be detected; and performs first processing on the overall vehicle characteristics , obtain the overall similarity of the vehicle; perform the second processing on the local component information of the vehicle to obtain the local characteristic distance value of the vehicle; perform the third processing on the body color information and the vehicle type information to obtain the color feature vector and the vehicle type feature vector. ; Perform feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity; In the vehicle, a target vehicle corresponding to the vehicle to be detected is determined. The invention improves the accuracy of vehicle re-identification.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 is a flowchart of an overall step provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

In view of the problems existing in the prior art, an embodiment of the present invention provides an unsupervised vehicle re-identification method based on hierarchical matching. As shown in FIG. 1 , the method of the present invention includes:

Obtain the image of the vehicle to be detected from the video source;

Extracting overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the image of the vehicle to be detected;

Perform a first process on the overall characteristics of the vehicle to obtain the overall similarity of the vehicle;

performing a second process on the vehicle local component information to obtain a vehicle local feature distance value;

performing a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

Performing feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity;

According to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected is determined from all vehicles in the candidate vehicle set.

Optionally, the obtaining a picture of the vehicle to be detected from a video source includes:

Acquiring image frames from the video source according to preset time intervals;

Perform target detection on the extracted image frame to find the vehicle target in the image frame;

A vehicle set is obtained by intercepting an image frame with a vehicle target, and the vehicle set is divided into a search picture set and a candidate picture set.

Optionally, the step of acquiring the picture of the vehicle to be detected from the video source further includes:

Calculate the intersection ratio between the detection frame of any current image frame and the detection frame of the corresponding previous image frame;

When the intersection ratio is greater than a preset threshold, remove the detection result corresponding to the current image frame;

Wherein, the calculation formula of the intersection ratio is:

Among them, IoU represents the intersection ratio; R ₁ is the detection frame of the previous image frame; R ₂ is the detection frame of the current image frame; area(R ₁ )∩area(R ₂ ) finds the area intersection between the two detection frames ; area(R ₁ )∪area(R ₂ ) finds the area union of the two detection boxes.

Optionally, the extraction of overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected includes:

The vehicle is divided into overall vehicle features and vehicle local component information through the ResNet50 feature extractor;

Through global average pooling, the image of the vehicle to be detected is processed to obtain the overall characteristics of the vehicle;

Output the overall characteristics of the vehicle through the CNN network;

Compare the features of each view angle of the vehicle to obtain the view angle visibility score under each view angle of the vehicle;

After calculating the weight of the visual angle visibility score, the local feature distance value is obtained by combining with the Euclidean distance calculation method, and the local component information of the vehicle is determined;

The color feature vector and the model feature vector of all vehicles are obtained by feature extraction through the Resnet50 network, and the vehicle body color information and the model information are determined.

Optionally, the extraction of overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected further includes:

Divide the vehicles in the picture of the vehicle to be detected into four directions: front, top, back and side;

Calculate the area values of the four directions of the vehicle as the view angle visibility score, and use the view angle visibility score as the confidence value of the local features corresponding to the vehicle, and the confidence value is the confidence value of the four local features of the vehicle, using It is used to characterize the weight of the feature distance of each component between the vehicles in the subsequent calculation of the local feature distance.

Optionally, the calculation formula for calculating the weight of the visual angle visibility score is:

代表所述视角可视性分数的权重；

代表待检测车辆的可视性分数；

代表图像库中对应的车辆的可视性分数；m,n代表待检测车辆与图像库中对应的车辆；i代表可视性分数和权重对应的区域；N代表待搜索的图像库中车辆数；in,

a weight representing the visibility score of said view;

represents the visibility score of the vehicle to be detected;

represents the visibility score of the corresponding vehicle in the image database; m, n represent the vehicle to be detected and the corresponding vehicle in the image database; i represents the area corresponding to the visibility score and weight; N represents the number of vehicles in the image database to be searched ;

The calculation formula of the local feature distance value is:

Among them, D ^m,n represents the local feature distance value; D represents the Euclidean distance of the vehicle in all directions; f _i ^m represents the feature information of the vehicle to be detected; f _i ⁿ represents the feature information of the corresponding vehicle in the image library.

Optionally, performing feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity, including:

Performing L2 norm normalization processing on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain normalized features;

According to the normalized features, calculate the initial re-identification distance value between the vehicle to be detected and the corresponding vehicle in the image library;

The initial re-recognition distance value of the overall similarity of the vehicle, the initial re-recognition distance value of the vehicle local feature distance value, the initial re-recognition distance value of the color feature vector and the initial re-recognition distance of the vehicle type feature vector The value is weighted and summed to obtain the total weight identification distance value;

The vehicle similarity is determined according to the total weight identification distance value.

Another aspect of the embodiments of the present invention also provides an apparatus for unsupervised vehicle re-identification based on hierarchical matching, including:

The first module is used to obtain the picture of the vehicle to be detected from the video source;

The second module is used for extracting overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected;

The third module is used to perform the first processing on the overall characteristics of the vehicle to obtain the overall similarity of the vehicle;

a fourth module, configured to perform second processing on the vehicle local component information to obtain a vehicle local characteristic distance value;

a fifth module, configured to perform third processing on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

The sixth module is used to perform feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity;

The seventh module is configured to determine, according to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set.

Another aspect of the embodiments of the present invention further provides an electronic device, including a processor and a memory;

the memory is used to store programs;

The processor executes the program to implement the method as described above.

Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the aforementioned method.

The embodiment of the present invention also discloses a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The computer instructions can be read from the computer-readable storage medium by a processor of the computer device, and the processor executes the computer instructions to cause the computer device to perform the foregoing method.

The specific implementation principle of the present invention will be described in detail below in conjunction with the accompanying drawings:

In view of the problems existing in the prior art, the present invention provides a vehicle re-identification method based on multi-model fusion, including a complete process of vehicle detection, extraction and comparison from video. First, the vehicle is detected and extracted from the video by the YOLO detection algorithm and saved. When the vehicle license plate is clear and can be recognized, the license plate will be used first for comparison, and the remaining vehicle images with low license plate definition will go through the subsequent re-identification process. In the re-identification part, the vehicle is divided into four parts: the overall characteristics of the vehicle, the local components of the vehicle, the color of the body, and the model of the vehicle. After inputting the overall features of the vehicle into the network, the CNN is used to output the overall similarity; the local component features of the vehicle are used to compare the vehicle features under different perspectives by introducing the perspective visibility score. The perspective visibility score divides the vehicle into four categories: front, back, side, and top. In each direction, count the area of the vehicle in these four directions and use it as a weight to weight the Euclidean distance value of each component to obtain the local characteristic distance value of the vehicle. The body color and model are classified through the ResNet50 network to obtain the color feature vector of the vehicle and the model feature vector S. Finally, the final vehicle similarity is obtained by fusing the above four vehicle features. The purpose is to expand the inter-class difference of data and reduce the intra-class difference through the fusion of global features and vehicle parsed features, so as to achieve the effect of improving the accuracy of vehicle re-identification. Finally, the similarity between the target vehicle and all the vehicles in the candidate vehicle set is sorted, and the top 5 (or 10) vehicles with the highest similarity are screened out using the camera space and vehicle appearance time information as the output.

The invention realizes a vehicle re-identification method based on multi-feature fusion. By detecting and extracting video vehicles, classifying vehicle colors and vehicle types, comparing vehicle component features from different viewing angles, and comparing vehicle appearance positions, and then using weighted feature values, Sort the entire vehicle similarity value, and select the vehicle with the highest similarity as the output vehicle.

Vehicle detection and license plate recognition section:

The vehicle detection part firstly takes the frame of the given video at a certain interval, and determines whether there is a vehicle (including three types of car, public bus and truck) in the scene by performing target detection on the frame image. The fixed frame interval, video frame rate and the speed of moving objects are all related. The target detection uses the YOLO v4 algorithm, which converts the target detection into a regression problem. In the model, classification and border regression are performed at the same time. The input image is passed through the backbone network (Backbone) CSPDarknet-53, neck (Neck) Pyramid pooling ( SPP) and Path Aggregation Network (PAN) and Head (Head) YOLO v3 to extract effective features, and then map to a tensor to get the target detection result. The method takes video frame images as input, and then detects whether there is a vehicle target in the image.

For the problem of repeated detection of stationary vehicles or low-speed vehicles that may exist in the input video, calculate the Intersection over Union (IoU) between the current detection frame and the previous detection frame, and eliminate the detection results with IoU greater than 0.5, thereby improving the repeated detection. question.

Among them, R ₁ is the detection frame of the previous video frame, R ₂ is the detection frame of the current video frame; area(R ₁ )∩area(R ₂ ) finds the area intersection of the two detection frames, area(R ₁ )∪area (R ₂ ) Find the area union of the two detection boxes.

After the vehicle detection is completed, the vehicle detection part in the frame image is intercepted and saved as a vehicle set, which can be divided into a query set (query set) and a candidate image set (gallery set) according to user needs, which are used for the vehicle re-identification part. .

The vehicle set obtained by vehicle detection is not only directly used in the vehicle re-identification part, but also used for license plate recognition as auxiliary reference information for vehicle re-identification. For the recognition part of the license plate, the license plate recognition network (License Plate Recognition Net, LPRNet) is used to input the vehicle picture, and the license plate part is recognized by the CNN model built by the SqueezeNet Fire module and the Inception module. Then, the license plate recognition result is saved as a csv result file, which is used for the license plate information comparison in the re-recognition process.

In the vehicle re-identification comparison part: first, the vehicle is divided into three main parts for prediction. In the vehicle component feature comparison part, the view classification network is used to divide the vehicle into four directions: front, top, back, and side, and the area values of the four directions of the vehicle are counted as the view visibility score (vis_scores), and the score is regarded as The confidence value of the vehicle corresponding to the local feature.

First, the vehicle is divided into two parts: global features and vehicle component (local) features by the ResNet50 feature extractor. The global average pooling is used for the feature map to obtain the global features of the vehicle, and the global features are output through the CNN network, which can bring higher detection accuracy when matching vehicles with the same viewing angle.

For the feature comparison of various perspectives of the vehicle, after obtaining the perspective visibility scores of the various perspectives of the vehicle, the vehicle feature information of different perspectives can be decoupled into corresponding local features to realize the feature comparison under each perspective. Since the visibility score is the area area at this time, it needs to be weighted again:

为权重，

和

为对应两辆车的可视性分数，i为可视性分数和权重对应的区域，m,n为对应待搜索车辆及图像库对应比对车辆。之后计算车辆各个方向特征的欧氏距离D并将可视性分数权重与其相乘得到最终的局部特征距离值：

is the weight,

and

is the visibility score corresponding to the two vehicles, i is the area corresponding to the visibility score and weight, m, n are the corresponding vehicle to be searched and the image library corresponding to the comparison vehicle. Then calculate the Euclidean distance D of the features in each direction of the vehicle and multiply the visibility score weight with it to get the final local feature distance value:

为上述计算得到的权重。f _i ^m and f _i ⁿ are the corresponding features of the two vehicles for comparison,

is the weight calculated above.

The above two parts seek the similarity of the overall features and local features of the vehicle, and the following will further improve the re-identification accuracy by classifying the vehicle type and color features. The model and color are extracted through the Resnet50 network to obtain the color feature vector and model feature vector S of all vehicles.

At this point, four vehicle feature values have been obtained in this embodiment, namely the vehicle global feature vector, the vehicle local feature vector, the vehicle type feature vector, and the vehicle body color feature vector. Distance calculation, in which the calculation methods of local distance and global distance are shown in the previous formulas (1) and (2), the color feature distance and the model feature distance are obtained by calculating the Euclidean distance between the features, and finally weighted by linear combination to obtain the overall Vehicle re-identification distance value D.

Among them, F′ _i represents the normalized feature, and F _i is the feature vector output by the network.

Among them, F ^m , F ⁿ represent the eigenvectors of different vehicles, and N represents the dimension of the eigenvectors.

D=mD _global +βD _local +γD _color +λD _type #(5)

To sum up, the present invention firstly selects the vehicle in the photo or video stream by using the vehicle identification technology, and then predicts the selected vehicle photo in three parts. In the feature of the vehicle components, the perspective classification network is used to score the four perspectives of the vehicle as the confidence value of the corresponding local features of the vehicle; then the vehicle is divided into two parts, the global feature and the local feature, by the feature extractor. The global features of the vehicle are obtained from the feature map; finally, the vehicle type and color features are used for identification. Finally, the feature distances of these four features are calculated, and the linear combination is weighted to obtain the final total distance value, and then the highest score is output as the result of vehicle re-identification to realize the complete re-identification process from video to vehicle comparison. Practicality and feasibility, and the final detection accuracy is high.

In some alternative implementations, the functions/operations noted in the block diagrams may occur out of the order noted in the operational diagrams. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/operations involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of the various operations are altered and in which sub-operations described as part of larger operations are performed independently.

Furthermore, while the invention is described in the context of functional modules, it is to be understood that, unless stated to the contrary, one or more of the described functions and/or features may be integrated in a single physical device and/or or software modules, or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to understand the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the modules will be within the routine skill of the engineer. Accordingly, those skilled in the art, using ordinary skill, can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the invention, which is to be determined by the appended claims along with their full scope of equivalents.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus.

More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, The scope of the invention is defined by the claims and their equivalents.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments, those skilled in the art can also make various equivalent deformations or replacements under the premise of not violating the spirit of the present invention, These equivalent modifications or substitutions are all included within the scope defined by the claims of the present application.

Claims (10)

1. An unsupervised vehicle re-identification method based on hierarchical matching, characterized in that it comprises: Obtain the image of the vehicle to be detected from the video source; Extracting overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the image of the vehicle to be detected; Perform a first process on the overall characteristics of the vehicle to obtain the overall similarity of the vehicle; performing a second process on the vehicle local component information to obtain a vehicle local feature distance value; performing a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector; Performing feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity; According to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected is determined from all vehicles in the candidate vehicle set. 2. The unsupervised vehicle re-identification method based on hierarchical matching according to claim 1, wherein the obtaining a picture of a vehicle to be detected from a video source comprises: Acquiring image frames from the video source according to preset time intervals; Perform target detection on the extracted image frame to find the vehicle target in the image frame; A vehicle set is obtained by intercepting an image frame with a vehicle target, and the vehicle set is divided into a search picture set and a candidate picture set. 3. The unsupervised vehicle re-identification method based on hierarchical matching according to claim 2, wherein the step of obtaining a picture of the vehicle to be detected from the video source further comprises: Calculate the intersection ratio between the detection frame of any current image frame and the detection frame of the corresponding previous image frame; When the intersection ratio is greater than a preset threshold, remove the detection result corresponding to the current image frame; Wherein, the calculation formula of the intersection ratio is:

Among them, IoU represents the intersection ratio; R ₁ is the detection frame of the previous image frame; R ₂ is the detection frame of the current image frame; area(R ₁ )∩area(R ₂ ) finds the area intersection between the two detection frames ; area(R ₁ )∪area(R ₂ ) finds the area union of the two detection boxes. 4 . The method for unsupervised vehicle re-identification based on hierarchical matching according to claim 1 , wherein the extraction of overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected is performed. 5 . ,include: The vehicle is divided into overall vehicle features and vehicle local component information through the ResNet50 feature extractor; Through global average pooling, the image of the vehicle to be detected is processed to obtain the overall characteristics of the vehicle; Output the overall characteristics of the vehicle through the CNN network; Compare the features of each view angle of the vehicle to obtain the view angle visibility score under each view angle of the vehicle; After calculating the weight of the visual angle visibility score, the local feature distance value is obtained by combining with the Euclidean distance calculation method, and the local component information of the vehicle is determined; The color feature vector and the model feature vector of all vehicles are obtained by feature extraction through the Resnet50 network, and the vehicle body color information and the model information are determined. 5 . The method for unsupervised vehicle re-identification based on hierarchical matching according to claim 4 , wherein the extraction of overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected is performed. 6 . ,Also includes: Divide the vehicles in the picture of the vehicle to be detected into four directions: front, top, back and side; Calculate the area value of the four directions of the vehicle as the view angle visibility score, and use the view angle visibility score as the confidence value of the local features corresponding to the vehicle, and the confidence value is the confidence value of the four local features of the vehicle, using It is used to characterize the weight of the feature distance of each component between the vehicles in the subsequent calculation of the local feature distance. 6. The unsupervised vehicle re-identification method based on hierarchical matching according to claim 4, wherein, The calculation formula for calculating the weight of the viewing angle visibility score is:

in,

a weight representing the visibility score of said view;

represents the visibility score of the vehicle to be detected;

represents the visibility score of the corresponding vehicle in the image database; m, n represent the vehicle to be detected and the corresponding vehicle in the image database; i represents the area corresponding to the visibility score and weight; N represents the number of vehicles in the image database to be searched ; The calculation formula of the local feature distance value is:

Among them, D ^m,n represents the local feature distance value; D represents the Euclidean distance of the vehicle in all directions; f _i ^m represents the feature information of the vehicle to be detected; f _i ⁿ represents the feature information of the corresponding vehicle in the image library. 7 . The method for unsupervised vehicle re-identification based on hierarchical matching according to claim 1 , wherein the overall similarity of the vehicle, the distance value of the local feature of the vehicle, the color feature vector and the The vehicle model feature vector is used for feature fusion to obtain vehicle similarity, including: L2 norm normalization processing is performed on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain normalized features; According to the normalized features, calculate the initial re-identification distance value between the vehicle to be detected and the corresponding vehicle in the image library; The initial re-recognition distance value of the overall similarity of the vehicle, the initial re-recognition distance value of the vehicle local feature distance value, the initial re-recognition distance value of the color feature vector and the initial re-recognition distance of the vehicle type feature vector The value is weighted and summed to obtain the total weight identification distance value; The vehicle similarity is determined according to the total weight identification distance value. 8. An unsupervised vehicle re-identification device based on hierarchical matching, characterized in that it comprises: The first module is used to obtain the picture of the vehicle to be detected from the video source; The second module is used for extracting overall vehicle features, vehicle local component information, vehicle body color information and vehicle type information from the picture of the vehicle to be detected; The third module is used to perform the first processing on the overall characteristics of the vehicle to obtain the overall similarity of the vehicle; a fourth module, configured to perform second processing on the vehicle local component information to obtain a vehicle local characteristic distance value; a fifth module, configured to perform third processing on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector; A sixth module, configured to perform feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle type feature vector to obtain the vehicle similarity; The seventh module is configured to determine, according to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set. 9. An electronic device, comprising a processor and a memory; the memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7 . 10. A computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images