CN117576380A - Target autonomous detection tracking method and system - Google Patents

Abstract

The invention relates to the technical field of target tracking, and discloses a target autonomous detection tracking method and a target autonomous detection tracking system, wherein the method comprises the following steps: s1, target detection: circularly detecting the image until a target is detected; s2, initializing a tracker template: initializing a tracker template; s3, target tracking: tracking the target. The invention solves the problems of failure in tracking the target or tracking drift in the prior art.

Description

A method and system for autonomous target detection and tracking

Technical field

The invention relates to the technical field of target tracking, specifically a method and system for autonomous target detection and tracking.

Background technique

Target tracking technology has always been a very important and challenging task in the field of computer vision research, which integrates computer science, statistical learning, pattern recognition, machine learning, image processing, and many other disciplines. With the development of computer vision and deep learning artificial intelligence, target tracking technology is mainly used to track the status of targets and accurately locate targets. It has applications in security monitoring, intelligent transportation, human-computer interaction, aerospace, military reconnaissance and many other aspects. Wide range of applications.

Target tracking is a basic problem in computer vision. The purpose of tracking is to determine the continuous position of the target we are interested in in the video sequence, that is, to obtain the parameters of the moving target, such as position, size, speed, acceleration, and motion trajectory. , so as to carry out further processing and analysis to achieve behavioral analysis and understanding of moving targets to complete more advanced tasks.

Target tracking can be divided into single target tracking and multiple target tracking according to the number of tracking targets. Single target tracking refers to determining the target to be tracked first (for example, finding the target you want to track in the first frame), and then determining the target's position in subsequent frames. This is a time + space connection problem, so there will be Challenges include target disappearance, target appearance changes, background interference, target movement, etc. Target tracking algorithms can be divided into generative algorithms and discriminative algorithms according to the modeling method. According to the length of the tracking sequence, they can be divided into short-term tracking algorithms and long-term tracking algorithms.

The generative classic tracking algorithm mainly extracts target features, establishes a target model, performs template matching in subsequent frames, and gradually iterates to achieve target positioning. The discriminative tracking algorithm is the mainstream tracking algorithm at this stage. It handles the tracking problem as a classification problem. Most of the discriminative algorithm frameworks use a correlation filtering framework, which extracts and models target features and background features respectively, and treats them as positive and negative respectively. Samples are used to train the corresponding classifier, and then the candidate samples are input into the classifier in subsequent frames to obtain the sample with the maximum probability value, which is the tracking target.

However, due to the complexity and change of actual scenes, factors such as occlusion and disappearance of targets in videos, morphological changes, reappearance, rapid movement, motion blur, scale changes and lighting will affect the accuracy of tracking, making it still difficult to solve in practical applications. With many challenges, designing fast and robust tracking algorithms remains very difficult and remains one of the most active research areas in computer vision.

In existing target tracking solutions, tracking failure is prone to occur when the target is blocked, deformed, blurred, or affected by lighting. Generative model algorithms for single target tracking algorithms such as optical flow method, Meanshift algorithm, particle filter, Camshift tracking algorithm, etc. do not consider background information. When the target is blocked, deformed, blurred, or affected by lighting, tracking failure is likely to occur, and generative tracking The algorithm is inefficient. Compared with generative tracking algorithms, discriminative tracking algorithms based on correlation filtering such as CSK algorithm, KCF algorithm, DSST algorithm, etc. can better obtain the specific location of the target in the current image, and have high recognition accuracy and tracking speed. It is fast and therefore widely studied. With the development of deep learning, CNN convolutional neural network is used to extract image features, and the neural network is directly used to build an end-to-end twin network target tracking model.

Although target tracking has gradually matured, there are still many challenges when facing actual scenes, especially when the target is partially or completely occluded, and when tracking moving targets, the target appears out of the camera, resulting in the loss of the tracking target, which easily makes filter template learning Characteristics of obstructing obstacles may lead to contamination of the filter template, failure to track the target, or tracking drift. In existing technical solutions, in order to ensure long-term or stable tracking of targets, a re-detection scheme is used to constrain the tracking of new tracker targets to achieve target tracking.

Disadvantages of existing technology:

1. Existing correlation filter trackers, when the target is blocked, changes in posture, runs blurred, or the target exits the country, the model drifts and the target is lost, making it impossible to complete subsequent tracking and retrieval of the target;

2. A separate target tracker needs to be set manually when initializing the target in the initial frame, and cannot automatically detect the target to complete the initial tracking.

Contents of the invention

In order to overcome the shortcomings of the existing technology, the present invention provides a target autonomous detection and tracking method and system to solve the problems of target tracking failure or tracking drift in the existing technology.

The technical solution adopted by the present invention to solve the above problems is:

A target autonomous detection and tracking method includes the following steps:

S1, target detection: cyclically detect the image until the target is detected;

S2, tracker template initialization: initialize the tracker template;

S3, target tracking: track the target.

As a preferred technical solution, in step S1, the positioning frame information of the area where the target is located is obtained, and the center point (x, y) and width and height (w, h) of the positioning frame position of the target in the image are output; where x represents The X-axis coordinate of the center point, y represents the Y-axis coordinate of the center point, w represents the width of the positioning box, and h represents the height of the positioning box.

As a preferred technical solution, step S2 includes the following steps:

S21, use the (x, y, w, h) parameters of the detected positioning frame and set filling parameters to determine the size of the target search area, and create an initialization cosine window and Gaussian ideal response label according to the size of the target search area;

S22, extract the grayscale features of the target area, and multiply the cosine window and the extracted feature map to solve the edge effect;

S23, the result of multiplying the obtained grayscale feature and the cosine window is converted into the frequency domain through Fourier transform, and multiplied by the Gaussian ideal response label after Fourier transform, thereby completing the initialization of the tracker template.

As a preferred technical solution, step S3 includes the following steps:

S31, obtain the frequency domain characteristics of the target search area;

S32, obtain the positioning point coordinates of the target in the current frame image and its response value;

S33. After obtaining the target position of the current frame, extract the grayscale features of the target search area with the target position as the center, perform a weighted average on the tracker template and then update the template as the tracking filter model of the next frame.

As a preferred technical solution, in step S31, the image data of the current frame is read in real time, and the target center point predicted by tracking and prediction of the previous frame image is taken as the center, an image of the size of the target search area in the initialization stage is intercepted, and the target area is extracted. The grayscale features are multiplied by the cosine window and then Fourier transformed to obtain the frequency domain features of the target search area.

As a preferred technical solution, in step S32, a matching calculation is performed with the tracking filter model, and the calculation result is subjected to inverse Fourier transformation to obtain a response map of the search area. The coordinates of the maximum peak in the response map are the predicted positions of the target. The center coordinates of the target are converted to the original image to obtain the positioning point coordinates of the target in the current frame image and its response value.

As a preferred technical solution, step S3 includes the following steps:

S34, use the tracking quality evaluation module to evaluate the tracking quality of each tracking prediction result, and determine whether to start the target detection module for global re-detection based on the evaluation results: if it is determined that re-detection is needed, start the target detection module to perform the full image range. For target detection, select the target whose detection target output confidence is higher than the set threshold as the retrieved tracking target, and output the target's positioning box information to reinitialize the tracker model; otherwise, continue tracking with the tracker.

As a preferred technical solution, in step S34, the tracking quality assessment module assessment strategy is to use the response value and the average peak correlation energy of the tracking prediction output to make a comprehensive assessment and judgment.

As a preferred technical solution, in step S34, the average peak correlation energy is calculated as:

in, Represents the average peak correlation energy energy,/> Represents the maximum value in the response graph of the tracking prediction result,/> Represents the minimum value in the response graph of the tracking prediction result,/> Represents the response value at the (w, h) position in the response graph,/> Represents the averaging function.

A target autonomous detection and tracking system, used to implement the above-mentioned target autonomous detection and tracking method, including the following modules connected in sequence:

Target detection module: used to perform cyclic detection on images until the target is detected;

Tracker template initialization module: used to initialize the tracker template;

Target tracking module: used to track targets.

Compared with the prior art, the present invention has the following beneficial effects:

(1) The present invention can automatically detect the target through the detection module, complete the initialization of the tracker template, and automatically start tracking the target;

(2) The re-detection mechanism of the present invention can perform global detection to retrieve the target when the target drifts or is lost, ensuring the continuity and robustness of tracking;

(3) The present invention combines the tracking speed of mutual filtering and the accuracy of the deep learning detection algorithm, and can perform continuous tracking with a real-time tracking algorithm and is compatible with better speed and accuracy.

Description of the drawings

Figure 1 is a flow chart of a target autonomous detection and tracking method according to the present invention;

Figure 2 is a schematic diagram of the network structure of Yolov3;

Figure 3 is the flow chart of the KCF single target tracking algorithm.

Detailed ways

The present invention will be further described in detail below with reference to the examples and drawings, but the implementation of the present invention is not limited thereto.

Example 1

As shown in Figures 1 to 3, a target autonomous detection and tracking method is aimed at the KCF-based correlation filter tracking algorithm, which cannot automatically determine the target for tracking, and the target moves quickly, is occluded, and exceeds the field of view during the tracking process. As a result, the target is lost and the target cannot be effectively tracked continuously. Start to acquire the video sequence, and use the target detection module to perform cyclic detection in the full image range until the detection algorithm detects the target.

Obtain the rectangular frame information of the target area and output the center point (x, y) and width and height (w, h) of the target positioning frame in the image. To initialize the tracker template, first use the (x, y, w, h) parameters of the detected positioning frame and set the padding parameters to determine the size of the target search area, and create an initialization cosine based on the size of the target search area. window and Gaussian ideal response label; then extract the Gray grayscale features of the target area, and multiply the cosine window and the extracted feature map to solve the edge effect; finally, the obtained two-dimensional data (the result of multiplying the grayscale features and the cosine window) After Fourier transform, it is converted into the frequency domain and multiplied with the Gaussian ideal response label after Fourier transform to complete the initialization of the tracker template. When the program is running, it determines whether the video ends. If it ends, it will directly end the entire program. If it does not end, it will enter the target tracking.

In the tracking phase, the image data of the current frame is read in real time, and the target center point predicted by the previous frame image tracking is taken as the center. The image of the size of the target search area in the initialization phase is intercepted, and the gray grayscale features of the target area are extracted, and the cosine window is used After multiplication, Fourier transform is performed to obtain the frequency domain characteristics of the target search area, and then the matching calculation is performed with the tracking filter model. The calculation result is subjected to the inverse Fourier transform to obtain the response map of the search area. The maximum peak value in the response map is The coordinates are the center coordinates of the predicted position of the target. By converting the coordinates to the original image, the coordinates of the target's positioning point in the current frame image and its response value are obtained. After obtaining the target position of the current frame, extract the grayscale features of the target search area with the target position as the center, perform a weighted average on the tracker template and then update the template as the tracking filter model for the next frame.

At the same time, in the tracking stage, the tracking quality assessment module is used to evaluate the tracking quality of each tracking prediction result. Based on the evaluation results, it is judged whether to start the target detection module for global re-detection. If it is determined that re-detection is needed, the target detection module is started. For target detection in the full image range, the target whose detection target output confidence is higher than the threshold is selected as the retrieved tracking target, and the positioning rectangular box information of the target is output, and the tracker model is reinitialized; otherwise, the tracker continues to track. The evaluation strategy of the tracking quality evaluation module is to use the response value of the tracking prediction output and the average peak-to-correlation energy (APCE) to conduct a comprehensive evaluation and judgment. APCE is the average peak correlation energy. That is, when the target is blocked by interference, the output response graph will show a state of multi-peak oscillation and the average peak energy will be significantly reduced. The calculation method of APCE is as follows:

in and/> Represents the maximum and minimum values in the response graph of the tracking prediction results,/> Represents the response value at the (w, h) position in the response graph.

The specific evaluation strategy is: during the real-time tracking process, continuously record the response value and APCE value of the tracking target from the current frame to the previous 6 frames. When the absolute value of the target response value predicted by tracking is less than a certain threshold, the detection module is started for re-detection; when the APCE value of the current frame predicted by tracking is less than a certain threshold, the detection module is started for re-detection; when the recorded APCE If the average value of the latest three frames is less than a certain ratio threshold of the average value of the previous three frames, the detection module will be started for re-detection; when the response value drops for 6 consecutive times in the recorded history and the current frame response value is less than a certain ratio of the previous six frames in history. If the threshold is exceeded and the recorded APCE is in a downward trend for 6 consecutive times, the detection module will be started for re-detection.

Yolov3 network mainly consists of three parts: backbone network (backbone), enhanced feature extraction layer (Neck), and network prediction head (Head). Backbone network: It is a Darknet-53 feature extraction network stacked with large residual blocks. Instead of using a pooling layer, the network uses convolution with a stride of 2 instead of downsampling. The Darknet-53 network can obtain 32x, 16x, and 8x downsampled feature maps to obtain different receptive fields. Strengthen the feature extraction layer: extract feature maps of different scales from the backbone feature network, and perform multi-scale feature fusion through bottom-up paths and lateral connections, so that both the high semantic information of the top layer and the high resolution of the bottom layer can be used rate information. Network prediction head (Head): Utilizes multi-scale output of feature layers of different scales to adapt to the detection of small-scale, medium-scale, and large-scale targets.

The network structure of Yolov3 is shown in Figure 2.

The input of the Yolov3 target detection model is an image of 416×416 size. Finally, after passing through the backbone feature extraction network, three feature maps of different sizes y1, y2, and y3 are obtained. The outputs are 13×13×C dimension and 26×26× respectively. C dimension, 52×52×C dimension, where C is (3×(4+1+L)), 3 means that the anchor point box of each grid predicts 3 candidate frame parameters, and the parameters predicted by each candidate frame ( tx, ty, tw, th, tcof, L) are respectively the positioning center (tx, ty), width and height (tw, th), confidence level tcof, and category information L of the target. After detecting multiple candidate bounding boxes through the Yolov3 target detection algorithm, the non-maximum suppression algorithm is finally used to remove redundant prediction boxes, and only the predicted positioning box with the highest confidence is retained as the target detection output box.

Yolov3's backbone network Darknet-53 is composed of 5 large ResX components, where X are 1, 2, 8, 8, and 4 respectively representing the number of repetitions of the residual block component. ResX is composed of a DBL module and It consists of a module and a residual edge. If the input image size is 416×416, three feature maps of different sizes will be obtained after passing through the backbone feature extraction network: 13×13 dimensional feature map, 26×26 dimensional feature map, and 52×52 dimensional feature map. In the enhanced feature extraction part of the network, the (13×13) feature map is first passed through a DBL*5 module (a stacked combination of 5 DBL modules), and then upsampled and spliced with the (26×26) feature map, and then passed through A DBL*5 module is upsampled and spliced with the (52×52) feature map, and finally a three-layer feature output layer is obtained.

The prediction head Head in the prediction output part is composed of DBL+Conv_1×1 convolution and is used to output prediction parameters. Finally, the dimension of the prediction head obtained by 32 times down sampling is 13×13 The output dimension of the sampled features is 26×26×C, which is responsible for the prediction of medium-scale targets; finally, 8x downsampling combines 32x and 16x downsampling features, and the output dimension of the prediction head is 52x52xC. The feeling in the original image is smaller, and it is mainly responsible for the prediction of small-scale targets.

Key points combine target detection and tracker to realize automatic detection and tracking of tracking targets. When the tracking quality is reduced and the target is blocked beyond the field of view, resulting in the target being lost, the target detection algorithm is automatically started based on the tracking quality assessment. Re-detect and retrieve the target to continue tracking, ensuring the persistence and robustness of target tracking. The tracking quality evaluation module comprehensively considers the response value and APCE value to determine whether to start target detection, and effectively integrates the detection accuracy of target detection based on deep learning with the speed of related filter template tracking to achieve sustained and stable tracking.

The target detection and tracker are combined to realize the automatic detection and tracking of the tracking target. When the tracking quality is reduced or the target is blocked beyond the field of view, causing the target to be lost, re-detection will be performed based on the tracking quality evaluation, and the response of the tracking target will be comprehensively considered. The tracking quality evaluation is carried out using the strategy of APCE value and APCE value, which is compatible with the detection accuracy and the tracking speed of correlation filtering.

For the Yolov3 detection algorithm in the target detection module, it can be replaced by target detection algorithms such as SSD and FastRCNN, which can also realize the target detection function.

For the KCF target tracking algorithm in the tracker template, related filter tracking algorithms such as MOSSE algorithm and CSK algorithm can be used to achieve the target tracking function.

The process of KCF single target tracking algorithm based on correlation filtering is shown in Figure 3.

KCF first extracts the feature Gray of the image block in the target area of the first frame, uses cos_windows for filtering, and then converts it to the frequency domain through fast Fourier transform to train the initial filter. In the detection stage, the filter model is used to calculate the relevant response, and the maximum response peak is selected as the target's motion position; the image features of the image area centered on the corresponding peak are extracted, the filter model of the desired output is learned and updated, and the above steps are executed in a loop to update the filter to track the position of the target on the next frame of image.

As described above, the present invention can be better implemented.

All features disclosed in all embodiments in this specification, or all steps in methods or processes implicitly disclosed, except for mutually exclusive features and/or steps, may be combined and/or extended or replaced in any manner.

The above are only preferred embodiments of the present invention and do not limit the present invention in any form. According to the technical essence of the present invention and within the spirit and principles of the present invention, any simple modifications to the above embodiments may be made. Modifications, equivalent substitutions and improvements, etc., all still fall within the protection scope of the technical solution of the present invention.

Claims (10)

1. The autonomous target detecting and tracking method is characterized by comprising the following steps:

s1, target detection: circularly detecting the image until a target is detected;

s2, initializing a tracker template: initializing a tracker template;

s3, target tracking: tracking the target.

2. The autonomous detection and tracking method of a target according to claim 1, wherein in step S1, positioning frame information of an area where the target is located is obtained, and a center point (x, y) and a width and height (w, h) of the positioning frame position of the target in the image are output; wherein X represents the X-axis coordinate of the center point, Y represents the Y-axis coordinate of the center point, w represents the width of the positioning frame, and h represents the height of the positioning frame.

3. The method of autonomous target detection and tracking according to claim 1, wherein step S2 comprises the steps of:

s21, determining the size of a target search area by using the (x, y, w, h) parameters of the detected positioning frame and the set filling parameters, and creating an initialized cosine window and a Gaussian ideal response label according to the size of the target search area;

s22, extracting gray scale features of the region where the target is located, and multiplying a cosine window by the extracted feature map to solve the edge effect;

s23, the obtained result of multiplying the gray features by the cosine window is converted into a frequency domain through Fourier transformation and multiplied by the Gaussian ideal response label after Fourier transformation, and therefore initialization of the tracker template is completed.

4. The method of autonomous target detection and tracking according to claim 1, wherein step S3 comprises the steps of:

s31, obtaining frequency domain characteristics of a target search area;

s32, obtaining the positioning point coordinates and the response values of the targets in the current frame image;

s33, after the target position of the current frame is obtained, gray features of a target search area are extracted by taking the target position as a center, and a template is updated after weighted average is carried out on the tracker template to be used as a tracking filtering model of the next frame.

5. The method of autonomous target detection and tracking according to claim 4, wherein in step S31, image data of a current frame is read in real time, an image of a target search area size in an initialization stage is intercepted with a target center point of tracking prediction of a previous frame image as a center, gray scale characteristics of the target area are extracted, and fourier transform is performed after multiplication with a cosine window to obtain frequency domain characteristics of the target search area.

6. The autonomous detection and tracking method according to claim 4, wherein in step S32, a matching calculation is performed with the tracking filter model, a response chart of the search area is obtained after the calculation result is subjected to inverse fourier transform, the coordinate of the maximum peak in the response chart is the center coordinate of the predicted position of the target, and the coordinate is converted to the original chart, so as to obtain the positioning point coordinate of the target in the current frame image and the response value thereof.

7. The method according to any one of claims 1 to 6, wherein step S3 includes the steps of:

s34, carrying out tracking quality evaluation on the result of each tracking prediction by using a tracking quality evaluation module, and judging whether to start a target detection module to carry out global re-detection according to the evaluated result: if the detection is judged to be needed to be re-detected, starting a target detection module to detect the target in the full-frame image range, selecting a target with the detection target output confidence higher than a set threshold as a retrieved tracking target, outputting positioning frame information of the target, and re-initializing a tracker model; otherwise, continuing the tracker to track.

8. The method according to claim 7, wherein in step S34, the tracking quality evaluation module evaluates a policy as follows: and carrying out comprehensive evaluation judgment by utilizing the response value of the tracking prediction output and the average peak correlation energy.

9. The method for autonomous target detection and tracking according to claim 8, wherein in step S34, the average peak correlation energy is calculated by:

wherein,represents the average peak correlation energy, +.>Representing the maximum value in the response graph of the tracking prediction result, +.>Representing the minimum value in the trace prediction response map, < >>Representing the response value at the (w, h) position in the response map, < >>Representing the averaging function.

10. An autonomous target detection and tracking system, characterized by being used for realizing the autonomous target detection and tracking method according to any one of claims 1 to 9, comprising the following modules connected in sequence:

the target detection module: the image detection device is used for circularly detecting the image until a target is detected;

tracker template initialization module: the tracker template is used for initializing;

a target tracking module: for tracking the target.