CN105469429B - Method for tracking target and device - Google Patents

Abstract

The invention provides a target tracking method and device. The method comprises: obtaining the displacement ranges of the target nodes in consecutive n frames of images in the control video when the inter-frame intervals are respectively 1, 2, ..., n-1; according to the positions and The displacement range of each target node when the inter-frame interval is 1, 2, ..., n-1, correlate each target node in n frames of images, and obtain the current target tracking trajectory; in the control video continuous n frames of images, Use the same marker to display the corresponding target nodes on the same target tracking track in the current target tracking track; where n is a preset frame number greater than 3. The target tracking method and device provided by the embodiments of the present invention realize the association of aircraft targets in the control video, and display the target nodes on a trajectory with the same mark, so that the images in the control video are clear and easy to identify.

Description

Target tracking method and device

technical field

The present invention relates to tracking technology, in particular to a target tracking method and device.

Background technique

With the development of aviation technology, the number and speed of aviation equipment have increased significantly. Air traffic control is an important technical means to ensure flight safety. The current air traffic control system mainly relies on the air traffic controllers on the ground to coordinate and guide the flight routes and flight modes of different aircraft in the airspace or airport, so as to prevent accidents on the ground or in the air, and ensure unimpeded air traffic and safe operation of aircraft. Sequential flight for maximum efficiency.

When conducting air traffic control, the controller mainly relies on the image information provided by the radar control aircraft. The radar control image is complex and chaotic, and the overlapping phenomenon of each marking frame is serious. The controller can only eliminate interference by continuously operating the marking frame. According to experience, the work is heavy, and it is easy to cause visual fatigue, which makes it impossible to respond quickly to emergencies. Moreover, there are many aircraft targets in the radar control video, which also causes great interference to the controller's operation and is prone to major problems. Security risks.

Contents of the invention

The invention provides a target tracking method and device, which are used to solve the problems of chaotic and complex images and large interference in existing control videos.

On the one hand, an embodiment of the present invention provides a target tracking method, including:

Obtain the displacement range of the target node in the consecutive n frames of images in the control video when the inter-frame intervals are 1, 2, ..., n-1;

According to the position of the frame where each of the target nodes is located in the control video and the displacement range of each of the target nodes when the inter-frame intervals are 1, 2, ..., n-1, for the n frames of images Each of the target nodes is associated to obtain the current target tracking trajectory;

In the continuous n-frame images of the control video, the same mark is used to display the corresponding target nodes on the same target tracking track in the current target tracking track;

Where n is a preset frame number greater than 3.

Another aspect of the embodiment of the present invention provides a target tracking device, including:

The displacement range acquisition module is used to obtain the displacement range of the target nodes in the continuous n-frame images in the control video when the inter-frame intervals are 1, 2, ..., n-1;

The tracking trajectory acquisition module is used for according to the position of the frame where each target node is located in the control video and the displacement range of each target node when the inter-frame interval is 1, 2, ..., n-1, Associating each of the target nodes in the n frames of images to obtain the current target tracking trajectory;

A marking module, configured to use the same mark to display the corresponding target nodes on the same target tracking track in the current target tracking track in the continuous n-frame images of the control video;

Where n is a preset frame number greater than 3.

The target tracking method and device provided by the embodiments of the present invention obtain the displacement range of each target node in the control video, and perform hyperedge growth on the target nodes that meet the displacement range, obtain the target tracking trajectory, and realize the association of aircraft targets in the control video , by displaying the target nodes on a trajectory with the same mark, the image in the control video is clear and clear, providing visual aid information for the controller, reducing the interference of many targets and logo frames in the video to the controller, and reducing the The controller's labor intensity provides convenience for the controller's identification and operation.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a flow chart of Embodiment 1 of the object tracking method of the present invention;

FIG. 2 is a flow chart of Embodiment 2 of the target tracking method of the present invention;

FIG. 3 is a flow chart of Embodiment 1 of the displacement range acquisition method in the target tracking method of the present invention;

FIG. 4 is a flow chart of Embodiment 1 of the displacement range acquisition method in the target tracking method of the present invention;

FIG. 5 is a flow chart of Embodiment 2 of the displacement range acquisition method in the target tracking method of the present invention;

FIG. 6 is a schematic diagram of Embodiment 1 of the object tracking device of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The embodiment of the present invention provides a target tracking method, aiming at the complex and chaotic images in the control video provided by the existing radar control aircraft, distinguishing and associating the aircraft targets in the control video, and realizing the trajectory tracking of the aircraft targets in the control video, so that The image in the control video is clear and clear, which is easy for the controller to identify and operate. For multiple aircraft targets in the control video, as the aircraft targets move in continuous multi-frame images, the target movement trajectory is formed, but the multiple trajectories that are not marked and distinguished are displayed in confusion and complexity. Therefore, the imaging of each aircraft target in each frame of image can be regarded as a target node, and by judging the distance relationship between multiple target nodes in each frame, the target nodes of the same target in different frames can be associated to achieve the goal Trajectory tracking, and distinguish the trajectories of different targets. The following uses specific embodiments to describe the target tracking method in detail.

FIG. 1 is a flow chart of Embodiment 1 of the object tracking method of the present invention. The execution subject of this embodiment is an object tracking device, which can be implemented by hardware and/or software. As shown in Figure 1, the method of this embodiment may include:

Step 101, obtaining the displacement ranges of the target nodes in consecutive n frames of images in the regulated video when the inter-frame intervals are 1, 2, ..., n-1;

Step 102, according to the position of the frame where each target node is located in the control video and the displacement range of each target node when the inter-frame interval is respectively 1, 2, ..., n-1, carry out each target node in the n frame image Associated to get the current target tracking trajectory;

Step 103, using the same marker to display the corresponding target node on the same node tracking track in the current target tracking track in the consecutive n frames of the control video;

Where n is a preset frame number greater than 3.

Specifically, select n consecutive frames of images in the control video, the frame interval between two adjacent frames is 1, and the intervals between frames of all target nodes in the continuous n frames of images are 1, 2, ..., n The displacement range when -1, the displacement range of the target node indicates the possible position range of the target node in the previous frame image in the subsequent frame image, the larger the inter-frame interval, the larger the corresponding displacement range, when obtaining the displacement range , for example, can be estimated according to the coordinate position of each target node in the frame. Where n is a preset frame number greater than 3.

In step 102, after obtaining the displacement ranges of all target nodes in the continuous n frames of images, according to the displacement ranges, the target nodes corresponding to the target nodes in the previous frame are searched in the next frame, and the continuous n frames of images are The corresponding target nodes in are associated as the moving track of the target node in consecutive n frames of images, and the hyperedge growth is completed to obtain the target tracking track. A target tracking track represents the moving track of an aircraft target in consecutive n frames of images.

After the target trajectory tracking is completed, the corresponding target nodes on the same target tracking track in the current target tracking track are displayed with the same mark. Specifically, different colors can be used to distinguish different targets, and different marker sizes or shapes can also be used to distinguish different targets. By repeating the above method for many times, the target tracking is completed for all the images in the control video, and the target tracking trajectory of the entire control video is obtained.

The target tracking method provided by the embodiment of the present invention obtains the displacement range of each target node in the control video, and performs hyperedge growth on the target nodes conforming to the displacement range, obtains the target tracking trajectory, and realizes the association of the aircraft target in the control video. The target nodes on a trajectory are displayed with the same mark, which makes the image in the control video clear and clear, provides visual aid information for the controller, reduces the interference of many targets and logo boxes in the video to the controller, and reduces the risk of the controller Labor intensity, providing convenience for the controller to identify and operate.

The technical solution of the embodiment of the target tracking method shown in FIG. 1 will be described in detail below using several specific embodiments.

On the basis of the embodiment shown in Figure 1, when obtaining the current tracking trajectory of continuous n-frame images in the control video, it can also be combined with the current tracking trajectory data of the continuous n-frame images that have been obtained in the control video, To simplify the calculation process and improve the accuracy of tracking results.

As shown in FIG. 2 , FIG. 2 is a flow chart of Embodiment 2 of the object tracking method of the present invention. The method includes:

Step 201, determine that there is an initial target tracking track;

Step 202, obtaining the displacement ranges of the target nodes in consecutive n frames of images in the controlled video when the inter-frame intervals are 1, 2, ..., n-1;

Step 203, according to the position of the frame where each target node is located in the control video and the displacement range of each target node when the inter-frame interval is respectively 1, 2, ..., n-1, carry out a process for each target node in n frames of images Associated to get the current target tracking trajectory;

Step 204, performing hyperedge fusion on the current target tracking trajectory and the initial target tracking trajectory to obtain the fusion trajectory, and using the fusion trajectory as a new initial target tracking trajectory;

Step 205 , in the n consecutive frames of the surveillance video, use the same marker to display the corresponding target nodes on the same target tracking track in the new initial target tracking track.

In this embodiment, on the basis of the embodiment shown in FIG. 1 , it is first determined that there is an initial target tracking track; The trajectory and the initial target tracking trajectory are hyperedge-fused to obtain the fusion trajectory, and the fusion trajectory is used as a new initial target tracking trajectory for the next calculation of the trajectory of the new continuous n-frame images in the control video. Specifically, the initial target tracking trajectory is the target tracking trajectory obtained when this embodiment was executed last time.

Optionally, step 204 specifically includes:

For any current target tracking track in the current target tracking track and any initial target tracking track in the initial target tracking track, if it is determined that any current target tracking track and any initial target tracking track have the same target in the same frame , then perform fusion processing on any current target tracking trajectory and any initial target tracking trajectory to obtain the fusion trajectory.

The fusion method is described in detail below:

For any current target tracking track in the current target tracking track and any initial target tracking track in the initial target tracking track, if it is determined that any current target tracking track and any initial target tracking track are in the same frame target node If they are the same, fusion processing is performed on any current target tracking trajectory and any initial target tracking trajectory to obtain a fusion trajectory.

Specifically, first determine whether there is an initial target tracking track, if not, then perform steps 101 to 103 to obtain the current target tracking track, and can choose to execute steps 101 to 103 multiple times to obtain the target tracking track of the control video; The trajectory is used as the initial target tracking trajectory when steps 101 to 103 are executed next time, and then the target tracking trajectory of the control video is generated according to the method proposed in this embodiment.

If it is determined that there is an initial target tracking track, it means that the target tracking track has been obtained according to the continuous n frames of images in the control video before step 201, and it can be assumed that the continuous n frames of images corresponding to the initial target tracking track are recorded as the first to n frames . Steps 202 to 203 are executed to obtain the current target tracking track of the current n consecutive frames of images, and it can be assumed that the n consecutive frames of images corresponding to the current target tracking track are the cth to n+c frames. For the current target tracking trajectory and the initial target tracking trajectory, since there are n-c frames of overlapping frames between two consecutive n-frame images, one trajectory can be taken from each of the current target tracking trajectory and the initial target tracking trajectory, and the two trajectories can be determined at After the target nodes in the same frame are the same, the two trajectories are fused and expanded to form a longer trajectory. Exemplarily, c may be any positive integer smaller than n.

In this embodiment, for the continuous images in the control video, firstly carry out target tracking for the continuous n frames of images to obtain the initial target tracking trajectory; then use the initial tracking trajectory to track the new continuous n Frame images are used for target tracking to obtain the current target tracking trajectory; among them, two consecutive n-frame images have certain overlapping frames, generally n-1 frames, and then the initial target tracking trajectory is fused with the current target tracking trajectory to obtain fusion trajectory, as the new initial tracking trajectory. When a new image is updated in the real-time updated control video, take consecutive n frames of images that have overlapping frames with the consecutive n frames of images in the last target tracking, and perform target tracking according to the latest initial tracking trajectory.

Steps 202 , 203 , and 205 in this embodiment are implemented in the same way as steps 101 , 102 , and 103 in the embodiment shown in FIG. 1 , and will not be repeated in the present invention.

On the basis of the above-mentioned embodiments, the displacement of the target nodes in the consecutive n frames of images in the acquisition control video in the embodiments shown in Fig. 1 and Fig. 2 when the inter-frame intervals are 1, 2, ..., n-1 The specific implementation of the scope will be described in detail.

A feasible implementation manner is shown in FIG. 3 , which is a flow chart of Embodiment 1 of the displacement range acquisition method in the target tracking method of the present invention. The method includes:

Step 301, obtaining the coordinate positions of each target node in consecutive n frames of images in the controlled video through template matching;

Step 302 , according to the coordinate position of each target node, obtain the displacement range of each target node in consecutive n frames of images when the inter-frame intervals are 1, 2, . . . , n-1.

In the control video, first use the template matching method to match each target node frame by frame, and record the coordinate position of each target node in the frame, and calculate each target node according to the coordinate position of each target node in consecutive n frames of images The displacement range when the interframe interval is 1, 2, ..., n-1 respectively.

Further, on the basis of this implementation, FIG. 4 is a flow chart of Embodiment 1 of a method for acquiring a displacement range in the object tracking method of the present invention. Step 302, as shown in Figure 4, specifically includes:

Step 31. According to the coordinate positions of all target nodes in the i-th frame image and the i+j-th frame image in the consecutive n-frame images, for each target node in the i-th frame, determine the distance between the i+j-th frame and the i+j-th frame The nearest target node of each target node in the i frame and the corresponding closest distance, the second closest target node and the corresponding second closest distance; wherein, the value range of i is a positive integer from 1 to n-j, and the value of j A positive integer ranging from 1 to n-1;

Step 32, calculate the ratio of the next closest distance to the closest distance d corresponding to each target node in the i-th frame, and obtain the maximum ratio;

Step 33, judging whether the maximum ratio is greater than the preset threshold, if so, go to step 34, if not, go to step 35;

Step 34, using the displacement range [d-2j, d+2j] as the displacement range of each target node in consecutive n frames of images when the inter-frame interval is j;

Step 35, increment i by 1, and repeatedly execute steps 31 to 35;

Step 36, after step 34 is determined, repeat step 31 to step 35, until after traversing all the values of j, it is obtained that the inter-frame intervals of the target nodes in n consecutive frames of images in the regulated video are 1, 2, ... , the displacement range at n-1.

Next, take the i-th frame image and the i+j-th frame image in the consecutive n frames of images in the control video, and take the calculation of the displacement range of the target node in the i-th frame image when the inter-frame interval is j as an example, and describe this implementation in detail example.

First, for each target node in the i-th frame image, determine the target node closest to each target node in the i-th frame in the i+j-th frame and the corresponding closest distance, the next closest target node and the corresponding second close range. Specifically, for a target node (x ₀ , y ₀ ) in the i-th frame image, determine the target node (x ₁ , y ₁ ) closest to the target node (x ₀ , y ₀ ) in the i+j-th frame ), and the next closest target node (x ₂ , y ₂ ), and get the corresponding closest distance d and second closest distance dd. Among them, for any two target nodes (x, y) and (xx, yy), the formula for calculating the distance is

Second, calculate the ratio of the next closest distance dd to the closest distance d corresponding to each target node in the i-th frame image, and compare to obtain the maximum ratio k. When there are few targets, since the time interval between frames is small, it can be directly considered that the target node in the next frame image that is closest to the target node in the previous frame image is the target node of the same target in different frames. As the number of targets increases, the distance between targets decreases, and the speed difference between multiple targets is large. There may be multiple target nodes in a small area of a frame. At this time, it cannot be simply considered that the nearest target node is the same as the previous The corresponding target nodes in the frame are the same target. The ratio of the next closest distance dd to the shortest distance d represents the degree of interference of other targets when the target moves to the next frame. When the ratio is large, it can be considered that within a certain range of the nearest target, there are no other interference targets. When the ratio is small, it can be considered that there is at least a second-closest interference target within a certain range of the nearest target. Then, compare whether the maximum ratio k is greater than a preset threshold k ₀ .

Secondly, when the maximum ratio k is greater than the preset threshold k ₀ , the displacement range [d-2j,d+2j] can be used as the displacement range of each target node in consecutive n frames of images when the inter-frame interval is j, the The displacement range represents the area covered by the position of the target node as the center of the circle and the displacement range as the radius, indicating the possible position of the target node in the previous frame in the subsequent frame image after j frames are separated. When the maximum ratio k is less than the preset threshold k ₀ , it indicates that among the selected two frames of images, the distribution of target nodes in the next frame is relatively concentrated and the interference is relatively large. At this time, increase the value of i by 1 and reselect Two images, for example, the first frame image and the 1+j frame image in the consecutive n frames of images were originally selected, and now the second frame image and the 2+j frame image in the consecutive n frames of images are selected, and the frame size is recalculated The displacement range when the interval is j. If there is no maximum ratio between the image in the i-th frame and the image in the i+j-th frame in the n consecutive frames of images that is greater than the preset threshold k ₀ , reselect the images in the n consecutive frames. Optionally, the preset threshold k ₀ may be 12/j.

Finally, after obtaining the displacement range of each target node in consecutive n frames of images when the inter-frame interval is j, calculate the displacement range of each target node in consecutive n frames of images when the inter-frame interval is j+1 , until all values of j have been traversed. Taking n as 5 as an example, it is necessary to calculate the displacement range of each target node in 5 consecutive frames of images when the inter-frame intervals are 1, 2, 3, and 4.

Another feasible implementation is based on the embodiment shown in FIG. 2 , as shown in FIG. 5 . FIG. 5 is a flowchart of Embodiment 2 of the displacement range acquisition method in the target tracking method of the present invention. The method includes:

Step 501, according to the coordinate positions of all target nodes in the initial target tracking track, calculate the target displacements of the target nodes in the initial target tracking track when the inter-frame intervals are 1, 2, ..., n-1;

Step 502, according to the average value of the target displacements of all target nodes in the initial target tracking trajectory when the inter-frame intervals are 1, 2, ..., n-1, obtain the inter-frame distance of each target node in consecutive n frames of images The range of displacement when the intervals are 1, 2, ..., n-1.

Specifically, after it is determined that there is an initial target tracking track, since the initial target tracking track is an already determined target node with an associated relationship, and these target nodes have accurate positional relationships in consecutive n frames of images, the initial target tracking track can be used Track the coordinate positions of all target nodes in the trajectory, distribute and calculate the target displacements of the target nodes on each trajectory when the inter-frame intervals are 1, 2, ..., n-1, and calculate the target displacements obtained by all trajectories by frame The average value A _j is calculated at different intervals, and [A _j -2j, A _j +2j] is used as the displacement range of each target node in the continuous n frames of images when the inter-frame interval is j.

On the basis of any of the above-mentioned embodiments, according to the position of the frame where each target node is located in the control video in the method and the displacement range of each target when the interval between frames is 1, 2, ..., n-1, The specific implementation of associating each target node in n frames of images to obtain the current target tracking trajectory will be described in detail.

Specifically, the association can adopt the method of hyperedge growth, and the above steps specifically include:

In the continuous n-frame images of the controlled video, if any three target nodes are determined to satisfy that no two are in the same frame and the coordinate distance satisfies the displacement range, then perform hyperedge connection on any three target nodes to obtain all hyperedges with a length of 3 The set P composed of;

In the set P, perform the hyperedge growth process to obtain the set Q composed of all hyperedges with a length of 4;

Perform the hyperedge growth process multiple times until all hyperedges with a length of n are obtained, and use all hyperedges as the current target tracking trajectory, where n is greater than 3;

Among them, the process of hyperedge growth is, in the set of hyperedges with a length of x, determine x+1 hyperedges that satisfy the number of common target nodes between each two hyperedges is x-1, and x The +1 hyperedges include x+1 different target nodes in total, then hyperedge connection is performed on the x+1 hyperedges to obtain a hyperedge with a length of x+1; x is a positive integer from 3 to n-1.

When associating all target nodes of consecutive n-frame images of the regulated video, the method of hyperedge growth is adopted, first hyperedge connection is performed to obtain a hyperedge with a length of 3, and then the length is grown from the hyperedge set with a length of 3 A hyperedge of length 4, and then a hyperedge of length 5 is grown from the set of hyperedges of length 4, and finally a hyperedge of length n is obtained. The trajectory obtained by hyperedge growing has a higher accuracy.

Furthermore, on the basis of hyperedge growth, the association can also adopt the method of hyperedge merging. After obtaining the set Q composed of all hyperedges with a length of 4, the above steps also include:

In the set Q, carry out the hyperedge growth process to obtain the set composed of all hyperedges with a length of 5 and the set Q ^* composed of hyperedges with a length of 4 that cannot be grown into a hyperedge with a length of 5; The edge grows into a hyperedge of length 4 and length 3 to form a set P ^* ;

A hyperedge of length 3 is randomly selected in the set P ^* , and a hyperedge of length 4 is randomly selected in the set Q ^* . If the hyperedge of length 3 and the hyperedge of length 4 contain two common target node, the hyperedge with a length of 3 and any hyperedge with a length of 4 are merged into a hyperedge with a length of 5;

Each time the hyperedge growth process is performed on the set X composed of hyperedges of length y-1, the set Y composed of hyperedges of length y and the hyperedges of length y-1 that cannot be grown into hyperedges of length y are obtained. After the set X ^* composed of edges, execute the hyperedge merging process to obtain a hyperedge with a length of y;

Among them, the process of merging hyperedges is: in the set X ^* , any hyperedge with a length of y-1 is selected, and in the set T ^* composed of hyperedges with a length of y-2 that cannot grow into a hyperedge with a length of y-1 Take any hyperedge with a length of y-2, if any hyperedge with a length of y-1 and any hyperedge with a length of y-2 contain y-3 common target nodes, then the arbitrary length A hyperedge of y-1 and an arbitrary hyperedge of length y-2 are merged into a hyperedge of length y; the value of y is a positive integer ranging from 5 to n-1.

Optionally, in the process of merging hyperedges, it can also be composed of hyperedges of length y-1 randomly selected in the set X and hyperedges of length y-2 that cannot be grown into hyperedges of length y-1 Any hyperedge of length y-2 in the set T ^* of .

Specifically, for n consecutive frames of images, the target nodes in each frame of images can be searched for the possible corresponding target nodes in the next j frames according to the displacement range. In order to obtain a longer and more accurate trajectory, the length of 3, and then gradually increase the length of the hyperedge, and finally obtain a hyperedge whose length meets the requirements. The following takes n as 6 as an example to describe in detail.

First, for all target nodes in 6 consecutive frames of images, determine three targets whose distances between two targets that are not on the same frame meet the corresponding displacement range, associate the three targets, and combine all targets that meet the above conditions After the three targets are associated, a set P consisting of hyperedges with a length of 3 is obtained. Exemplarily, in the specific calculation, when the three targets come from the first frame, the third frame and the fourth frame respectively, the displacement ranges that these three targets should satisfy are that the targets from the first frame and the third frame meet The displacement range when the interframe interval is 2, the targets from the first frame and the fourth frame meet the displacement range when the interframe interval is 3, and the objects from the third and fourth frames meet the displacement when the interframe interval is 1 scope.

Secondly, if there are 4 hyperedges with a length of 3 in the set P, these 4 hyperedges contain 4 different target nodes, and the number of common target nodes between each two hyperedges is 2, then it is considered These 4 hyperedges can be fused together to form a hyperedge of length 4, which consists of these 4 different target nodes. All hyperedges of length 4 form set Q, and hyperedges of length 3 that cannot grow to length 4 in set P form set P ^* .

Again, in the set Q, the process of hyperedge growth is carried out again, and the set Q ^* composed of all hyperedges with a length of 5 and the hyperedges with a length of 4 that cannot be grown into a hyperedge with a length of 5 is obtained; the set Q ^* is fused again with the hyperedges in the set P ^* , when the arbitrary length 3 hyperedge and the arbitrary length 4 hyperedge contain two common target nodes, then the arbitrary length 3 hyperedge An edge is merged with an arbitrary hyperedge of length 4 into a hyperedge of length 5.

Again, in the set of all hyperedges with a length of 5, perform hyperedge growth to obtain a hyperedge with a length of 6 and obtain a set of hyperedges with a length of 5 that cannot be grown to a length of 6, and will not be able to grow Perform hyperedge merging for a hyperedge of length 5 with length 6 and a hyperedge set of length 4 that cannot be grown into a hyperedge of length 5 to obtain a hyperedge of length 6.

Finally, all hyperedges with lengths of 3, 4, 5, and 6 are used as the current target tracking trajectory.

Further, on the basis of any of the above-mentioned embodiments, the present invention also includes:

Delete the shorter hyperedge among the hyperedges with containment relationship among all hyperedges.

Exemplarily, for hyperedges with lengths of 3, 4, 5, and 6, there may be two hyperedges with containment relations, and the hyperedge with a longer length excludes all In addition to the target node, there are other target nodes. At this time, the hyperedge with a shorter length is deleted in the current target tracking track.

Further, on the basis of any of the above-mentioned embodiments, the present invention also includes:

Obtain the moving speed of the target in n consecutive frames of images.

Specifically, the moving speed of the target can be obtained according to the fusion trajectory or the current target tracking trajectory.

Next, take n equal to 5 as an example, that is, take the continuous 5 frames of images in the control video as an example, and describe the steps involved in the above-mentioned embodiments of the present invention in detail:

Step 1: Use the template matching detection method to detect the first frame to the fifth frame image of five consecutive frames of images, and obtain the coordinate positions of all target nodes in each frame.

Step 2. For 5 consecutive frames of images, take the i-th frame image and the i+j-th frame image, and record the coordinate positions of the target nodes detected in the two images, assuming that a total of n _i target nodes are detected in the i-th frame image , the coordinates of the target node are A total of ni+j target nodes are detected in the i+j frame image, and the coordinates of the target nodes are For each target node detected in the i-th frame, among all the detected target nodes in the i+j-th frame image, determine the target node closest to the coordinates of the target node and the next closest target node; assuming The position coordinates of a certain target node in the i-th frame are t ∈ {1, 2, ... n _i }, the coordinate positions of all target nodes detected in the i+jth frame image , the distance from the target node The shortest distance of can be calculated by the following formula: The corresponding index number is next closest to The ratio between the two is of all target nodes in the i-th frame image In , take out the corresponding target node when the ratio of the next closest distance to the closest distance is the largest.

Step 3, determine the largest ratio Whether it is true, if true, go to step 4; if not, go to step 5.

Step 4, take the shortest distance d corresponding to the target node with the largest ratio of the next closest distance to the shortest distance, and use S _j = [d-2j, d+2j] as the target displacement range corresponding to the target node when the inter-frame interval is j; set Add 1 to the value of j, and re-execute steps 2 to 3 until the value of j is traversed from 1 to 4 once.

Step five, increase the value of i by 1, and re-execute steps two to three.

Before step six, by performing steps two and three multiple times, the displacement ranges of the target nodes can be obtained when the inter-frame intervals are 1, 2, 3, and 4, respectively.

Step 6, a total of 5 consecutive frames of images are detected with detection results, expressed as The five consecutive frames of images are named frame 0, frame 1, frame 2, frame 3 and frame 4, which represent all detection results in frame 0 and all detection results in frame 1. test results As the target node v _i for trajectory tracking, the value range of i is {1, 2, ... n _t }; for any 3 nodes satisfy t ₁ ≠t ₂ ≠t ₃ , and and Then establish a hyperedge e={v ₁ , v ₂ , v ₃ } whose length is 3, where As an example, it represents the displacement range when the interval between frames is |t ₁ -t ₂ |, and repeats this process to determine the qualified hyperedges in all detection results, expressed by the set E _tri ={e _i }, and e _i represents For any hyperedge in the hyperedge set with length 3, i is a positive integer, and the value range of i does not exceed the size of the set E _tri .

Step seven, perform hyperedge growth on the set E _tri . If in the hyperedge set E _tri ={e _i }, there are 4 hyperedges e ₁ , e ₂ , e ₃ , and e ₄ satisfying Card(e _i ∩e _j )=2, And Card(e ₁ ∪e ₂ ∪e ₃ ∪e ₄ )=4; then hyperedges e ₁ , e ₂ , e ₃ , e ₄ can be merged into a new hyperedge with length 4, and these four hyperedges called growable hyperedges. This step divides the set E _tri into two disjoint parts, one part can be merged to grow into a hyperedge with length 4, and the other part cannot. The merged hyperedge set of length 4 is denoted as E _quad , and the remaining hyperedge set of length 3 that cannot be hyperedge grown is denoted as Further, perform hyperedge growth on the hyperedge set E _quad with a length of 4 to obtain a hyperedge set E _quin with a length of 5 and the remaining hyperedge sets with a length of 4 that cannot be hyperedge grown Further, perform hyperedge merge, if Each hyperedge e _i in the hyperedge set Find the corresponding subset in Subset The combination of all elements in means that if Then compare e _i with merge into hyperedges of length 5 Together with the hyperedge set of length 5 obtained by hyperedge growth, it is expressed by E _quin . If there is a hyperedge set Any one of the hyperedges does not satisfy Card(e _i ∩e _j )=2, at this time is an empty set, so we have There may also be case, the two cases of e _i in cannot be merged into a hyperedge with a length of 5, and for this type e _i is recorded as The set of hyperedge length 3 after hyperedge merging The remaining hyperedge set of length 3 in is denoted as Among them, Card means to find the number of elements in the finite set.

Step 8, verify three hyperedge sets of different lengths In _Equin , whether there is a situation where a hyperedge with a shorter length is covered by a hyperedge with a longer length, and if so, delete the hyperedge with a shorter length. for collections If there are two hyperedges e _i , e _j ∈ E _all , i≠j, satisfy Then remove e _i and keep e _j . Three sets of hyperedges with different lengths are the current target tracking trajectory of five consecutive frames of images, which are used as the initial target tracking trajectory.

Step 9: For the 6th frame image that is updated in real time, take the 2nd to 6th frame images, and track the trajectory. When tracking the trajectory, you can still use the methods shown in steps 2 to 8, or you can use the following The way:

According to the initial target tracking trajectory obtained from the first frame to the fifth frame image, each trajectory is represented by the position coordinates of the target node: Among them, n _i represents the number of target nodes in trajectory e _i . Calculate the target displacement when the frame interval is 1, 2, 3, 4 for each trajectory e _i where t=|pq|, p≠q, p, q∈{1, 2, . . . n _i }. Calculated mean from all trajectories As the shortest distance d ^t when the inter-frame interval of the second to sixth frame images is t, S ^t = [d ^t -2t, d ^t +2t] as the target corresponding to the target node when the inter-frame interval is t displacement range. According to the new target displacement range, perform steps 6 to 8 again to obtain the current target tracking trajectory.

In step ten, the current target tracking trajectory and the initial target tracking trajectory are hyper-edge fused. The initial target tracking trajectory is represented by the symbol T ¹ , and the tracking trajectory obtained from the second frame to the sixth frame, that is, the current target tracking trajectory is represented by T ² , and the two are fused to obtain a longer trajectory result, that is, from the first frame to the first frame The tracking result of 6 frames is denoted by symbol T ⁶ . The specific fusion method is as follows:

for any track If a track can be found Satisfy where kj represents the trajectory The number that appears in the sixth frame, including 1 or 0. but and Merging into a new trajectory All trajectories in the initial target tracking trajectory T ¹ are fused to obtain a fused trajectory T ⁶ . Using the fusion trajectory as the new initial target tracking trajectory, repeat steps 9 and 10 to obtain all tracking trajectories of the control video updated in real time.

Optionally, the current speed of the target may be estimated according to the track each time the current target tracking track is obtained.

Another aspect of the embodiments of the present invention also provides a target tracking device. Fig. 6 is a schematic diagram of Embodiment 1 of the target tracking device of the present invention. As shown in Fig. 6, the target tracking device includes:

The displacement range acquisition module 601 is used to obtain the displacement range of the target nodes in the continuous n frames of images in the control video when the inter-frame intervals are 1, 2, ..., n-1;

Tracking track acquisition module 602, for according to the position of the frame where each target node is located in the control video and the displacement range of each target node when the inter-frame interval is 1, 2, ..., n-1, for n frames of image Each target node is associated to obtain the current target tracking trajectory;

The marking module 603 is used to display the corresponding target nodes on the same target tracking track in the current target tracking track using the same mark in the continuous n-frame images of the control video;

Where n is a preset frame number greater than 3.

Optionally, on the basis of the above embodiments, the target tracking device further includes: an initial target tracking track storage module, configured to store the initial target tracking track;

The fusion module is used to carry out hyperedge fusion between the current target tracking track and the initial target tracking track to obtain the fusion track, and use the fusion track as a new initial target tracking track to update the initial target tracking track in the initial target tracking track storage module;

The marking module 603 is configured to use the same mark to display the corresponding target nodes on the same target tracking track in the new initial target tracking track in the consecutive n frames of images of the control video.

Optionally, the displacement range acquisition module 601 is used to obtain the coordinate position of each target node in consecutive n frames of images in the controlled video through template matching; obtain the coordinate position of each target node in consecutive n frames of images according to the coordinate position of each target node The displacement ranges when the intervals are 1, 2, ..., n-1.

Optionally, the displacement range acquisition module 601 is specifically configured to perform the following steps:

Step 31. According to the coordinate positions of all target nodes in the i-th frame image and the i+j-th frame image in the consecutive n-frame images, for each target node in the i-th frame, determine the distance between the i+j-th frame and the i+j-th frame The nearest target node of each target node in the i frame and the corresponding closest distance, the second closest target node and the corresponding second closest distance; wherein, the value range of i is a positive integer from 1 to n-j, and the value of j A positive integer ranging from 1 to n-1;

Step 32, calculate the ratio of the next closest distance to the closest distance d corresponding to each target node in the i-th frame, and obtain the maximum ratio;

Step 33, judging whether the maximum ratio is greater than the preset threshold, if so, go to step 34, if not, go to step 35;

Step 34, using the displacement range [d-2j, d+2j] as the displacement range of each target node in consecutive n frames of images when the inter-frame interval is j;

Step 35, increment i by 1, and repeatedly execute steps 31 to 35;

Step 36, after step 34 is determined, repeat step 31 to step 35, until after traversing all the values of j, it is obtained that the inter-frame intervals of the target nodes in n consecutive frames of images in the regulated video are 1, 2, ... , the displacement range at n-1.

Optionally, the displacement range acquisition module 601 can be used to calculate the inter-frame intervals of the target nodes in the initial target tracking track as 1, 2, ..., n-1 according to the coordinate positions of all target nodes in the initial target tracking track target displacement when

According to the average value of the target displacements of all target nodes in the initial target tracking trajectory when the inter-frame intervals are 1, 2, ..., n-1, the inter-frame intervals of each target node in consecutive n frames of images are respectively 1, 2, ..., the displacement range at n-1.

Optionally, the fusion module is specifically used for any current target tracking trajectory in the current target tracking trajectory and any initial target tracking trajectory in the initial target tracking trajectory, if it is determined that any current target tracking trajectory and any initial target tracking trajectory If the target nodes in the same frame of the target tracking track are the same, any current target tracking track and any initial target tracking track are fused to obtain a fusion track.

Further, the target tracking device further includes: a speed estimation module, configured to acquire the moving speed of the target of each fusion track in consecutive n frames of images according to the fusion track.

Optionally, on the basis of any of the above-mentioned device embodiments, the tracking trajectory acquisition module 602 is specifically used for:

In the continuous n-frame images of the controlled video, if any three target nodes are determined to satisfy that no two are in the same frame and the coordinate distance satisfies the displacement range, then perform hyperedge connection on any three target nodes to obtain all hyperedges with a length of 3 The set P composed of;

In the set P, perform the hyperedge growth process to obtain the set Q composed of all hyperedges with a length of 4;

Perform the hyperedge growth process multiple times until all hyperedges with a length of n are obtained, and all hyperedges are used as the current target tracking trajectory, and n is greater than 3;

Among them, the process of hyperedge growth is, in the set of hyperedges with a length of x, determine x+1 hyperedges satisfying that the number of common target nodes between each two hyperedges is x-1, and x +1 hyperedges include x+1 different target nodes in total, then connect x+1 hyperedges to obtain a hyperedge with length x+1;

x is a positive integer from 3 to n-1.

Optionally, the track acquisition module 602 is also used for:

In the set Q, carry out the hyperedge growth process to obtain the set composed of all hyperedges with a length of 5 and the set Q ^* composed of hyperedges with a length of 4 that cannot be grown into a hyperedge with a length of 5; The edge grows into a hyperedge of length 4 and length 3 to form a set P ^* ;

A hyperedge of length 3 is randomly selected in the set P ^* , and a hyperedge of length 4 is randomly selected in the set Q ^* . If the hyperedge of length 3 and the hyperedge of length 4 contain two common target node, the hyperedge with a length of 3 and any hyperedge with a length of 4 are merged into a hyperedge with a length of 5;

Each time the hyperedge growth process is performed on the set X composed of hyperedges of length y-1, the set Y composed of hyperedges of length y and the hyperedges of length y-1 that cannot be grown into hyperedges of length y are obtained. After the set X ^* composed of edges, execute the hyperedge merging process to obtain a hyperedge with a length of y;

Among them, the process of merging hyperedges is: in the set X ^* , any hyperedge with a length of y-1 is selected, and in the set T ^* composed of hyperedges with a length of y-2 that cannot grow into a hyperedge with a length of y-1 Take any hyperedge with a length of y-2, if any hyperedge with a length of y-1 and any hyperedge with a length of y-2 contain y-3 common target nodes, then the arbitrary length A hyperedge of y-1 and an arbitrary hyperedge of length y-2 are merged into a hyperedge of length y; the value of y is a positive integer ranging from 5 to n-1.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (9)

1. A target tracking method, characterized in that, comprising: Obtain the displacement range of the target node in the consecutive n frames of images in the control video when the inter-frame intervals are 1, 2, ..., n-1; According to the position of the frame where each of the target nodes is located in the control video and the displacement range of each of the target nodes when the inter-frame intervals are 1, 2, ..., n-1, for the n frames of images Each of the target nodes is associated to obtain the current target tracking trajectory; In the continuous n-frame images of the control video, the same mark is used to display the corresponding target nodes on the same target tracking track in the current target tracking track; Where n is a preset number of frames greater than 3; According to the position of the frame where each of the target nodes is located in the control video and the displacement range of each of the target nodes when the inter-frame intervals are 1, 2, ..., n-1, for the n frames Each of the target nodes in the image is associated to obtain the current target tracking track, including: In the continuous n frames of images of the control video, it is determined that any two of the three target nodes that are not in the same frame and whose coordinate distance satisfies the displacement range, then perform hyperedge connection on the any three target nodes to obtain all A set P of hyperedges of length 3; In the set P, perform a hyperedge growing process to obtain a set Q composed of all hyperedges with a length of 4; Perform the hyperedge growth process multiple times until all hyperedges with a length of n are obtained, and all hyperedges are used as the current target tracking trajectory, and the n is greater than 3; Wherein, the hyperedge growth process is, in the set of hyperedges with a length of x, determine x+1 hyperedges satisfying that the number of common target nodes between each two hyperedges is x-1, And the x+1 hyperedges include x+1 different target nodes in total, then performing hyperedge connection on the x+1 hyperedges to obtain a hyperedge with a length of x+1; x is a positive integer from 3 to n-1. 2. method according to claim 1, is characterized in that, before the displacement range when inter-frame interval is respectively 1, 2, ..., n-1 time, Also includes: Determining that there is an initial target tracking trajectory; After obtaining the current target tracking trajectory, it also includes: Perform hyperedge fusion on the current target tracking trajectory and the initial target tracking trajectory to obtain a fusion trajectory, and use the fusion trajectory as a new initial target tracking trajectory; In the continuous n-frame images of the control video, the same mark is used to display the corresponding target nodes on the same target tracking track in the current target tracking track, including: In the consecutive n frames of images of the control video, the corresponding target nodes on the same target tracking track in the new initial target tracking track are displayed with the same marker. 3. The method according to claim 1, wherein the displacement ranges of the target nodes in the consecutive n frames of images in the acquired control video are respectively 1, 2, ..., n-1 when the inter-frame intervals include : Obtaining the coordinate positions of each of the target nodes in consecutive n frames of images in the control video by template matching; The displacement ranges of each of the target nodes in the continuous n frames of images when the inter-frame intervals are 1, 2, . . . , n−1 are acquired according to the coordinate positions of each of the target nodes. 4. method according to claim 3, is characterized in that, described according to the coordinate position of each described target node, obtains each described target node in described continuous n frame image at inter-frame interval respectively 1,2, ..., the displacement range at n-1, including: Step 31, according to the coordinate positions of all target nodes in the i-th frame image and the i+j-th frame image in the continuous n-frame images, for each target node in the i-th frame, determine in the i+j-th frame The target node closest to each target node in the i-th frame and the corresponding closest distance, the second closest target node and the corresponding second closest distance; wherein, the value range of i is a positive integer from 1 to n-j, The value range of j is a positive integer from 1 to n-1; Step 32, calculate the ratio of the next closest distance to the closest distance d corresponding to each target node in the i-th frame, and obtain the maximum ratio; Step 33, judging whether the maximum ratio is greater than a preset threshold, if yes, execute step 34, if not, execute step 35; Step 34, using the displacement range [d-2j, d+2j] as the displacement range of each target node in the continuous n frames of images when the inter-frame interval is j; Step 35, incrementing the i by 1, and repeatedly executing steps 31 to 35; Step 36, after confirming that step 34 is executed, repeat step 31 to step 35, until after traversing all the values of j, obtain the inter-frame intervals of the target nodes in consecutive n frames of images in the controlled video, respectively The displacement range when it is 1, 2, ..., n-1. 5. The method according to claim 2, wherein the displacement ranges of the target nodes in the consecutive n frames of images in the acquisition control video are respectively 1, 2, ..., n-1 when the inter-frame intervals include : According to the coordinate positions of all target nodes in the initial target tracking track, calculate the target displacements of the target nodes in the initial target tracking track when the inter-frame intervals are 1, 2, ..., n-1; According to the average value of the target displacements of all target nodes in the initial target tracking trajectory when the inter-frame intervals are 1, 2, ..., n-1, obtain the frame of each target node in the continuous n frame images The displacement range when the intervals are 1, 2, ..., n-1 respectively; According to the average value of the target displacements of all target nodes in the initial target tracking trajectory when the inter-frame intervals are 1, 2, ..., n-1, each target node in the continuous n frames of images is obtained The displacement range when the inter-frame interval is 1, 2, ..., n-1, including: According to the average value A _j of the target displacement of all target nodes in the initial target tracking trajectory when the inter-frame intervals are 1, 2, ..., n-1, [A _j -2j, A _j +2j] is used as The displacement range of each target node in consecutive n frames of images when the inter-frame interval is j. 6. The method according to claim 2, wherein said performing hyperedge fusion on said current target tracking track and said initial target tracking track to obtain a fusion track, comprising: For any current target tracking track in the current target tracking track and any initial target tracking track in the initial target tracking track, if it is determined that any current target tracking track and any initial target tracking track If the trajectories have the same target node in the same frame, fusion processing is performed on any current target tracking trajectory and any initial target tracking trajectory to obtain a fusion trajectory. 7. method according to claim 6, is characterized in that, after described obtaining fusion trajectory, also comprises: Acquiring the moving speed of the target of each of the fusion trajectories in the consecutive n frames of images according to the fusion trajectories. 8. The method according to claim 1, characterized in that, after the set Q formed by all hyperedges with a length of 4 is obtained, the method further comprises: In the set Q, carry out the hyperedge growing process, obtain the set Q ^* that all lengths are composed of hyperedges with a length of 5 and the hyperedges with a length of 4 that cannot be grown into a length of 5; In P, hyperedges that cannot be grown into hyperedges with a length of 4 and a length of 3 form a set P ^* ; A hyperedge with a length of 3 is randomly selected in the set P ^* , and a hyperedge with a length of 4 is randomly selected in the set Q ^* . If the hyperedge with a length of 3 and the hyperedge with a length of 4 If the hyperedge contains two common target nodes, the hyperedge with a length of 3 and the hyperedge with a length of 4 are merged into a hyperedge with a length of 5; Each time the hyperedge growth process is performed on the set X composed of hyperedges of length y-1, the set Y composed of hyperedges of length y and the hyperedges of length y-1 that cannot be grown into hyperedges of length y are obtained. After the set X ^* composed of edges, execute the hyperedge merging process to obtain a hyperedge with a length of y; Wherein, the hyperedge merging process is as follows: in the set X ^* , any hyperedge with a length of y-1 is randomly selected, and if no hyperedge can be grown into a set T composed of hyperedges with a length of y-1 and a length of y-2 ^* Arbitrarily select a hyperedge with length y-2, if any hyperedge with length y-1 and any hyperedge with length y-2 contain y-3 common target nodes, then any A hyperedge of length y-1 and an arbitrary hyperedge of length y-2 are merged into a hyperedge of length y; the value of y is a positive integer ranging from 5 to n-1. 9. A target tracking device, characterized in that it comprises: The displacement range acquisition module is used to obtain the displacement range of the target nodes in the continuous n-frame images in the control video when the inter-frame intervals are 1, 2, ..., n-1; The tracking trajectory acquisition module is used for according to the position of the frame where each target node is located in the control video and the displacement range of each target node when the inter-frame interval is 1, 2, ..., n-1, Associating each of the target nodes in the n frames of images to obtain the current target tracking trajectory; A marking module, configured to use the same mark to display the corresponding target nodes on the same target tracking track in the current target tracking track in the continuous n-frame images of the control video; Where n is a preset number of frames greater than 3; The tracking trajectory acquisition module is specifically used for: In the continuous n-frame images of the controlled video, if any three target nodes are determined to satisfy that no two are in the same frame and the coordinate distance satisfies the displacement range, then perform hyperedge connection on any three target nodes to obtain all hyperedges with a length of 3 The set P composed of; In the set P, perform the hyperedge growth process to obtain the set Q composed of all hyperedges with a length of 4; Perform the hyperedge growth process multiple times until all hyperedges with a length of n are obtained, and all hyperedges are used as the current target tracking trajectory, and n is greater than 3; Among them, the process of hyperedge growth is, in the set of hyperedges with a length of x, determine x+1 hyperedges that satisfy the number of common target nodes between each two hyperedges is x-1, and x The +1 hyperedges include x+1 different target nodes in total, then the x+1 hyperedges are hyperedge-connected to obtain a hyperedge with a length of x+1; x is a positive integer from 3 to n-1.