CN115908489A

CN115908489A - Target tracking method and device

Info

Publication number: CN115908489A
Application number: CN202211400740.5A
Authority: CN
Inventors: 王震
Original assignee: BOE Technology Group Co Ltd
Current assignee: BOE Technology Group Co Ltd
Priority date: 2022-11-09
Filing date: 2022-11-09
Publication date: 2023-04-04

Abstract

The embodiment of the application provides a target tracking method and a target tracking device, and the method comprises the following steps: acquiring image position information of a target to be tracked in an image coordinate system of a first camera; converting the image position information from the image coordinate system to a world coordinate system to obtain world position information; determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the world position information; acquiring a plurality of cameras to be matched adjacent to the first camera, and respectively determining the relative direction of each camera to be matched relative to the first camera; selecting at least one second camera from the cameras to be matched, which are matched with the driving-out direction in the relative direction; and performing cross-camera tracking on the target to be tracked according to the image data of the first camera and the image data of the second camera. By the method, the robustness of cross-camera tracking is improved.

Description

Target tracking method and device

Technical Field

The application relates to the technical field of smart cities, in particular to a target tracking method and device.

Background

In the prior art, when matching a target to be tracked among multiple cameras, the direction of the target to be tracked and the corresponding relationship between an upstream camera and a downstream camera need to be identified by manually drawn region information. The method is too dependent on manual experience, and robustness of cross-camera tracking is greatly reduced.

Disclosure of Invention

The embodiment of the application aims to provide a target tracking method and a target tracking device so as to improve robustness of cross-camera tracking. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a target tracking method, where the method includes:

acquiring image position information of a target to be tracked in an image coordinate system of a first camera;

converting the image position information from the image coordinate system to a world coordinate system to obtain world position information;

determining the exit direction of the target to be tracked when exiting from the acquisition range of the first camera according to the world position information;

acquiring a plurality of cameras to be matched adjacent to the first camera, and respectively determining the relative direction of each camera to be matched relative to the first camera;

selecting at least one second camera from the cameras to be matched, which are matched with the driving-out direction in the relative direction;

and carrying out cross-camera tracking on the target to be tracked according to the image data of the first camera and the image data of the second camera.

In a possible embodiment, the image position information includes image coordinates of at least two target track points of the target to be tracked in the image coordinate system;

converting the image position information from the image coordinate system to a world coordinate system to obtain world position information, wherein the converting includes:

and respectively carrying out mapping calculation on the image coordinates of each target track point in the at least two target track points by utilizing a predetermined homography matrix between the image coordinate system and the world coordinate system to obtain the world coordinates of each target track point, wherein the world position information comprises the world coordinates of each target track point.

In a possible implementation manner, the determining, according to the world location information, an exit direction of the target to be tracked when exiting from the first camera acquisition range includes:

calculating to obtain an exit direction vector when the target to be tracked exits the first camera acquisition range according to the world coordinates of at least two target track points in the world position information;

and determining the exit direction of the target to be tracked when exiting the acquisition range of the first camera according to the exit direction vector.

In a possible implementation manner, the calculating, according to world coordinates of at least two target track points in the world position information, an exiting direction vector when the target to be tracked exits the first camera acquisition range includes:

acquiring a first world coordinate and an Nth world coordinate in the world position information; the first world coordinate is the world coordinate of a target track point collected last in each target track point, or the first world coordinate is the average value of the world coordinates of i +1 target track points collected last in each target track point; the Nth world coordinate is the world coordinate of the last Nth collected target track point in each target track point, or the Nth world coordinate is the average value of the world coordinates of the last Nth to last N + i collected target track points in each target track point; n is an integer greater than 1, i is a positive integer;

and calculating a vector pointing to the Nth world coordinate from the first world coordinate to obtain an exit direction vector when the target to be tracked exits the first camera acquisition range.

In a possible implementation manner, the determining, according to the exit direction vector, an exit direction in which the target to be tracked exits from the first camera acquisition range includes:

converting the outgoing direction vector into a unit direction vector; wherein the abscissa of the vector in the unit direction is x, and the ordinate is y;

when x is larger than 0, y is larger than 0 and x is larger than y, determining that the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is east;

when x is larger than 0, y is larger than 0 and x is smaller than y, determining that the exit direction of the target to be tracked when the target exits the first camera acquisition range is north;

when x is smaller than 0, y is larger than 0 and-x is larger than y, determining that the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is west;

when x is smaller than 0, y is larger than 0 and-x is smaller than y, determining that the exit direction of the target to be tracked when the target exits the first camera acquisition range is north;

when x is smaller than 0, y is smaller than 0 and-x is larger than-y, determining that the exit direction of the target to be tracked when the target exits the first camera acquisition range is west;

when x is smaller than 0, y is smaller than 0 and-x is smaller than-y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is south;

when x is larger than 0, y is smaller than 0 and x is larger than-y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is east;

and when x is larger than 0, y is smaller than 0 and x is smaller than-y, determining that the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is south.

In a possible implementation, the acquiring a plurality of cameras to be matched adjacent to the first camera includes:

acquiring world coordinates of each camera in the world coordinate system;

for each camera except the first camera, calculating the distance between the camera and the first camera according to the world coordinate of the camera and the world coordinate of the first camera;

and selecting M cameras closest to the first camera to obtain a plurality of cameras to be matched, which are adjacent to the first camera.

In a possible implementation, the respectively determining the relative direction of each camera to be matched with respect to the first camera includes:

for each camera to be matched, calculating to obtain a relative direction vector of the camera to be matched relative to the first camera;

and respectively determining the relative direction of each camera to be matched relative to the first camera according to each relative direction vector.

In a possible embodiment, the selecting at least one second camera from the cameras to be matched whose relative direction matches the exit direction includes:

and selecting K cameras to be matched which are closest to the first camera from the cameras to be matched and are matched with the driving-out direction in the relative direction to obtain K second cameras, wherein K is a preset positive integer.

In one possible embodiment, the method further comprises:

aiming at each second camera, determining the driving-in direction of the target to be tracked when the target to be tracked drives into the acquisition range of the second camera;

matching the exit direction with the entry direction of the target to be tracked when entering the acquisition range of the second camera to obtain a direction matching result of the second camera;

and determining whether the cross-camera tracking result between the first camera and the second camera is credible according to the direction matching result of the second camera.

In a second aspect, an embodiment of the present application provides an apparatus for tracking a target, the apparatus including:

the first acquisition module is used for acquiring image position information of a target to be tracked in an image coordinate system of the first camera;

the second acquisition module is used for converting the image position information from the image coordinate system to a world coordinate system to obtain world position information;

the first determining module is used for determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the world position information;

the third acquisition module is used for acquiring a plurality of cameras to be matched, which are adjacent to the first camera;

the second determining module is used for respectively determining the relative direction of each camera to be matched relative to the first camera;

the selecting module is used for selecting at least one second camera from the cameras to be matched, which are matched with the driving-out direction in the relative direction;

and the tracking module is used for carrying out cross-camera tracking on the target to be tracked according to the image data of the first camera and the image data of the second camera.

the second acquisition module includes:

and the first obtaining submodule is used for respectively carrying out mapping calculation on the image coordinate of each target track point in the at least two target track points by utilizing a homography matrix between the image coordinate system and the world coordinate system to obtain the world coordinate of each target track point, wherein the world position information comprises the world coordinate of each target track point.

In one possible implementation, the first determining module includes:

the second acquisition submodule is used for calculating and obtaining an outgoing direction vector when the target to be tracked exits the first camera acquisition range according to the world coordinates of at least two target track points in the world position information;

and the first determining submodule is used for determining the exit direction of the target to be tracked when the target exits the first camera acquisition range according to the exit direction vector.

In a possible implementation manner, the second obtaining sub-module is specifically configured to:

In a possible implementation, the first determining submodule is specifically configured to:

when x is larger than 0, y is smaller than 0 and x is larger than-y, determining that the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is east;

and when x is larger than 0, y is smaller than 0 and x is smaller than-y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is south.

In one possible implementation, the third obtaining module includes:

the third acquisition submodule is used for acquiring world coordinates of each camera in the world coordinate system;

the calculation submodule is used for calculating the distance between each camera except the first camera according to the world coordinate of the camera and the world coordinate of the first camera;

and the fourth obtaining submodule is used for selecting M cameras closest to the first camera to obtain a plurality of cameras to be matched, which are adjacent to the first camera.

In one possible implementation, the second determining module includes:

the fifth obtaining submodule is used for calculating and obtaining a relative direction vector of each camera to be matched relative to the first camera for each camera to be matched;

and the second determining submodule is used for respectively determining the relative direction of each camera to be matched relative to the first camera according to each relative direction vector.

In a possible implementation, the selecting module includes:

and the sixth obtaining submodule is used for selecting K cameras to be matched which are closest to the first camera from the cameras to be matched and are matched with the driving-out direction in the relative direction to obtain K second cameras, wherein K is a preset positive integer.

In one possible embodiment, the apparatus further comprises:

the third determining module is used for determining the driving direction of the target to be tracked when the target to be tracked drives into the acquisition range of each second camera;

the fourth acquisition module is used for matching the exit direction with the entry direction of the target to be tracked when entering the acquisition range of the second camera to obtain a direction matching result of the second camera;

and the fourth determining module is used for determining whether the cross-camera tracking result between the first camera and the second camera is credible according to the direction matching result of the second camera.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus;

a memory for storing a computer program;

a processor adapted to perform the method steps of any of the above first aspects when executing a program stored in the memory.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps in any one of the above first aspects.

The embodiment of the application has the following beneficial effects:

the target tracking method and device provided by the embodiment of the application comprise the following steps: acquiring image position information of a target to be tracked in an image coordinate system of a first camera; converting the image position information from the image coordinate system to a world coordinate system to obtain world position information; determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the world position information; acquiring a plurality of cameras to be matched, which are adjacent to the first camera, and respectively determining the relative direction of each camera to be matched relative to the first camera; selecting at least one second camera from the cameras to be matched, which are matched with the driving-out direction in the relative direction; and performing cross-camera tracking on the target to be tracked according to the image data of the first camera and the image data of the second camera. The method has the advantages that the driving direction of the target to be tracked is automatically determined according to the driving direction of the target to be tracked when the target to be tracked drives out of the acquisition range of the first camera, cross-camera tracking is carried out on the target to be tracked, automatic cross-camera target tracking is achieved, and compared with the prior art that manual drawing area information is excessively relied on, the robustness of cross-camera tracking is improved.

Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and it is also obvious for a person skilled in the art to obtain other embodiments according to the drawings.

FIG. 1 is a detailed scene diagram of a prior art mid-span camera vehicle tracking;

FIG. 2 is a schematic view of a vehicle traveling in different camera images according to the prior art;

FIG. 3 is a basic flow diagram of a prior art mid-span camera multi-vehicle tracking;

FIG. 4a is a schematic illustration of a zone division in a prior art mid-span camera vehicle tracking method;

FIG. 4b is a schematic illustration of another segmentation in a prior art mid-span camera vehicle tracking method;

fig. 5 is a schematic flowchart of a first target tracking method according to an embodiment of the present disclosure;

fig. 6a is a schematic flowchart of a second target tracking method according to an embodiment of the present disclosure;

FIG. 6b is a schematic diagram of image coordinates of a target tracking method according to an embodiment of the present disclosure;

FIG. 6c is a schematic view of world coordinates of a target tracking method according to an embodiment of the present application;

FIG. 6d is a first schematic diagram of projective transformation;

FIG. 6e is a second schematic diagram of projective transformation;

fig. 7 is a third schematic flowchart of a target tracking method according to an embodiment of the present application;

fig. 8a is a fourth flowchart illustrating a target tracking method according to an embodiment of the present application;

fig. 8b is a schematic view of an exit direction vector when the target to be tracked exits the first camera acquisition range;

fig. 9a is a schematic flowchart of a fifth method for tracking a target according to an embodiment of the present disclosure;

FIG. 9b is a schematic diagram of a unit direction vector;

fig. 10 is a sixth flowchart of a target tracking method according to an embodiment of the present application;

fig. 11a is a seventh flowchart of a target tracking method according to an embodiment of the present application;

fig. 11b is a schematic diagram of the relative direction of each camera to be matched with respect to the first camera;

fig. 12 is an eighth flowchart of a target tracking method according to an embodiment of the present application;

fig. 13 is a ninth flowchart illustrating a target tracking method according to an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a target tracking device according to an embodiment of the present application;

fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the description herein are intended to be within the scope of the present disclosure.

First, cross-camera target tracking in the prior art is briefly described, which is exemplified by cross-camera vehicle tracking.

The purpose of vehicle cross-camera tracking is to identify the same vehicle passing through multiple intersections, a specific scene is shown in fig. 1, in the figure, C041-C045 are 5 camera positions of different intersections respectively, and if a vehicle passes through C045-C041 once from west to east, it is necessary to identify that the vehicle is the same vehicle in images of 5 cameras.

In the case where the traffic is large, the vehicles are crowded with each other, the angles of the plurality of cameras are different, and the light is different, it is difficult to recognize that the vehicle is the same vehicle in the images of the 5 cameras, and as shown in fig. 2, the two vehicles shown in the square in the figure are the same vehicle in the different cameras, but it is difficult to recognize the two vehicles as the same vehicle due to the light or the like.

The basic flow of cross-camera multi-vehicle tracking is shown in fig. 3, and comprises vehicle detection, vehicle feature extraction, single-camera vehicle tracking and cross-camera vehicle matching.

The object detection aims to identify vehicles in the image, and detect as many objects as possible. The vehicle feature extraction is to extract features of detected vehicles through a reid model (re-identification, which is a technology for finding a target to be searched in an image library by using an algorithm), so that subsequent single-camera tracking and multi-camera matching are performed for vehicle feature comparison and matching. Single-camera vehicle tracking is tracking the same vehicle in an image through the features of the vehicle and the trajectory of the vehicle. The cross-camera vehicle matching is to match the track of the same vehicle with a plurality of cameras according to the vehicle characteristics after the vehicle track of each single camera is obtained, so that the cross-camera tracking effect is achieved. The cross-camera vehicle matching comprises the following steps: 1. converting the vehicle image coordinates to world coordinates (GPS coordinates); 2. calculating the driving-in and driving-out directions of the vehicle according to the vehicle track; 3. searching a camera for driving the vehicle in and out according to the direction of the vehicle; 4. and matching the adjacent camera vehicles.

The cross-camera vehicle matching is an important link in cross-camera vehicle tracking, and after the single-camera vehicle tracking tracks of all vehicles in each camera are obtained, the key for the cross-camera vehicle matching is to match the tracks of the same vehicle in a plurality of cameras. The main steps and difficulties include:

1. the method comprises the following steps of firstly determining adjacent cameras, and determining how to judge a previous camera for a vehicle to enter and a next camera for a vehicle to exit in the current cameras by giving N cameras.

2. The vehicle track between the adjacent cameras is matched, and the difficulty is that the number of matched samples is large, and the matching effect is poor.

In step 1, the conventional cross-camera vehicle tracking method manually divides a plurality of areas in each camera image through priori knowledge, and determines the driving-in and driving-out directions of the vehicle by judging which area the initial position and the end position of the vehicle track are located in. Then, judging that the adjacent camera, such as the area 1 of the camera A, exits and enters the area 3 of the camera C through the priori knowledge. The traditional way of dividing the regions depending on the prior knowledge is shown in fig. 4a and 4b, and the corresponding division way in fig. 4a is to divide the image into 5 regions, where "Zone" represents the region, leave Main street, enter Main street, go Next Camera is going to the Next Camera, from Next Camera is coming From the Next Camera, go to Previous Camera is going to the Previous Camera, and Go Previous Camera is coming From the Previous Camera. The corresponding division manner in fig. 4b is to divide the image into 9 areas, and the division manners in fig. 4a and fig. 4b are both to conveniently judge the entering and exiting directions of a certain vehicle track, so as to match the vehicle in the next or previous camera. However, the manual region division method relies on manual experience too much, regions drawn by each person in the same scene are different, small differences between the regions may also have great influence on the final matching result, and robustness of cross-camera tracking is reduced seriously.

In order to improve robustness of cross-camera tracking, the embodiment of the application provides a target tracking method and device.

Next, a detailed description is given of a target tracking method provided in an embodiment of the present application, with reference to fig. 5, including the following steps:

step S501, image position information of the target to be tracked in the image coordinate system of the first camera is obtained.

The target to be tracked can be set in a user-defined manner according to actual tracking requirements, and in one example, the target to be tracked may be a motor vehicle or a non-motor vehicle, which is not specifically limited in this application.

The image position information may be a track position coordinate of the target to be tracked in an image coordinate system of the first camera. The target tracking mode in a single camera may refer to a target tracking mode in the related art, and is not specifically limited in this application.

And step S502, converting the image position information from the image coordinate system to a world coordinate system to obtain world position information.

The world coordinate System in the embodiment of the present application refers to a coordinate System of a real monitoring scene, and in an example, the world coordinate System may be a longitude and latitude coordinate System, such as a beidou coordinate System or a GPS (Global Positioning System) coordinate System; in one example, the world coordinate system may be a coordinate system custom-built for a real monitoring scenario. The track coordinate information of all the targets to be tracked is converted into a world coordinate system, so that the track coordinate of the targets to be tracked and the world coordinate (GPS coordinate information) of the camera are in the same coordinate system, and the direction of the targets to be tracked can be judged by using the longitude and latitude directions. The world position information is track coordinate information of the target to be tracked in a world coordinate system corresponding to the image position information.

Step S503, determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the world position information.

In one example, after the world position information of the target to be tracked is obtained, the exit direction of the target to be tracked is determined according to the world position information of the target to be tracked, for example, world coordinates when the target to be tracked is located at a track junction, so as to subsequently determine which camera the target to be tracked may go to after exiting from the first camera.

Step S504, a plurality of cameras to be matched adjacent to the first camera are obtained, and the relative direction of each camera to be matched relative to the first camera is respectively determined;

a plurality of cameras to be matched, which are adjacent to the first camera, can be manually calibrated in advance, and can also be obtained by calculation according to the distance between each camera and the first camera. After a plurality of to-be-matched cameras adjacent to the first camera are obtained, the relative direction of the to-be-matched cameras relative to the first camera can be determined according to the world coordinates of the to-be-matched cameras and the world coordinates of the first camera.

Step S505, selecting at least one second camera from the cameras to be matched, the relative direction of which is matched with the driving-out direction;

in one example, all the cameras to be matched, which are matched with each other in the opposite direction and the exit direction, can be selected as the second cameras; in one example, among the cameras to be matched, which are matched in the relative direction and the exit direction, the cameras to be matched, of which the distance from the first camera is smaller than the preset distance threshold value, are selected to obtain second cameras; in one example, a preset number of cameras closest to the first camera may be selected from the cameras to be matched in the relative direction and the exit direction to obtain the second cameras.

And S506, performing cross-camera tracking on the target to be tracked according to the image data of the first camera and the image data of the second camera.

The image data of the first camera and the image data of the second camera comprise information such as the characteristics and the track of the target to be tracked, after the track of the target to be tracked of each single camera of the first camera and the second camera is obtained, the tracks of the target to be tracked of the first camera and the second camera can be matched according to the characteristics of the target to be tracked, and the matching degree is high.

In the embodiment, the driving direction of the target to be tracked is automatically determined according to the driving direction of the target to be tracked when the target to be tracked drives out of the first camera acquisition range, and the target to be tracked is tracked across cameras, so that the automatic target tracking across cameras is realized, and compared with the prior art which excessively depends on manual drawing area information, the robustness of the target tracking across cameras is improved.

referring to fig. 6a, a second flowchart of the target tracking method provided in the embodiment of the present application is refined based on step S502 in fig. 5, and includes the following steps:

step S601, performing mapping calculation on the image coordinates of each target track point of the at least two target track points respectively by using a homography matrix predetermined between the image coordinate system and the world coordinate system, to obtain the world coordinates of each target track point, where the world position information includes the world coordinates of each target track point.

In one example, the specific conversion process of converting the image position information of the target to be tracked into the world position information is as follows:

1. referring to fig. 6b, coordinates of more than 4 points are marked in the image coordinate system of the first camera, and image coordinates corresponding to the 4 points are obtained;

2. referring to fig. 6c, finding out a point on the image corresponding to the first camera in the world coordinate system to obtain world coordinates corresponding to 4 points;

3. a robust method based on RANSAC (Random Sample Consensus) may be used to calculate mathematical model parameters of data according to a set of Sample data sets including abnormal data to obtain an algorithm of effective Sample data), or a transformation matrix of mutual transformation between an image coordinate system and a world coordinate system may be calculated by using a minimum median robust method, and after the transformation matrix is obtained, trajectory coordinates of all targets to be tracked in an image of the first camera may be transformed into world coordinates, that is, image position information of the targets to be tracked is transformed into world position information.

First, assuming any point x in the image coordinate system, there is a conversion relationship such that x '= H × x, where x' is the coordinate of the point in the world coordinate system, and H is a homography matrix for converting the image coordinate system into the world coordinate system.

In one example, the non-homogeneous coordinates (non-homogeneous coordinates) on a 2D (two-dimensional plane) image are (x, y) and the homogeneous coordinates (homogeneous coordinates) are (x, y, 1), which can also be written as (x/z, y/z, 1) or (x, y, z). Homogeneous coordinates have many benefits, such as the ability to clearly determine that a point is not on a straight line.

As shown in fig. 6D, the projective transformation may be an operation performed from a two-dimensional plane, where a and a ', B and B', C and C ', D and D' are different point pairs on the same plane, and the heterogeneous coordinates of each point pair may be expressed by a projective transformation (projective transformation). In one example, the inhomogeneous coordinates of point a may be (x 1, y 1), the inhomogeneous coordinates of a 'may be (x 2, y 2), and the projective transformation from a to a' may be expressed as: (x 2, y 2) = H '× (x 1, y 1), H' is a projection matrix in a two-dimensional operation.

As shown in fig. 6e, the homogeneous coordinates of the point pairs on the images of two different viewing angles (original image plane and new image plane) can be expressed by a projective transformation, for example: x1= H x2. The projective transformation may also be called Homography (Homography), where the prefix "Homo" means "same", homography means a graph generated from the same source, and chinese translation is probably "Homography", so the matrix H in the above equation is called a Homography, and the Homography is a projection matrix from one plane to another plane (the projection center is shown in fig. 6 e), and the Homography can be regarded as an excess of the projective matrix from two-dimensional operation to three-dimensional operation. If a unique solution is required, 4 point pairs are required to solve for H (corresponding to 8 equations to solve for 8 unknowns in H).

The RANSAC algorithm assumes that data includes correct data and abnormal data (or referred to as noise), the correct data is denoted as local interior points (inerals), the abnormal data is denoted as external points (outliers), and the RANSAC algorithm also assumes that a set of correct data is given, and a method for calculating model parameters conforming to the data exists. The core idea of the algorithm is randomness and hypothesis, wherein the randomness is to randomly select sampling data according to the occurrence probability of correct data, and the randomness simulation can approximately obtain correct results according to a law of large numbers; the hypothesis is that the sampled data are all correct data, then the correct data are used to calculate other points through the model satisfied by the problem, and then the result is scored.

The RANSAC algorithm is widely used in the computer vision field and the mathematics field, such as straight line fitting, plane fitting, calculating transformation matrix between images or point clouds, calculating basis matrix, etc., and is used very much. The algorithm process is as follows:

1. there is a model adapted to the assumed local interior, i.e. all unknown parameters can be calculated from the assumed local interior.

2. All other data are tested using the model obtained in 1 and if a point is suitable for the estimated model, it is considered to be an in-office point.

3. If enough points are classified as the assumed local interior points, the estimated model is reasonable enough.

4. The model is then re-estimated using all hypothesized local points (e.g., using least squares), since it was estimated only by the initial hypothesized local points.

5. Finally, the model is evaluated by estimating the error rate of the local interior point and the model.

6. The above process is repeated a fixed number of times, each time the resulting model is either discarded because there are too few local points or selected because it is better than the existing models.

In the above embodiment, the homography matrix is used to perform mapping calculation on the image coordinates of each of the at least two target track points, so that the world coordinates of each of the at least two target track points are obtained. The at least two target track points are used for facilitating the subsequent determination of the exit direction of the target to be tracked when exiting from the first camera acquisition range.

Referring to fig. 7, a third flowchart of the target tracking method provided in the embodiment of the present application is refined based on step S503 in fig. 5, and includes the following steps:

step S701, calculating to obtain an outgoing direction vector when the target to be tracked exits the first camera acquisition range according to world coordinates of at least two target track points in the world position information;

the target track points are track points of the target to be tracked in world coordinates.

Step S702, determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the exit direction vector.

And the direction of the exit direction vector is the exit direction of the target to be tracked when exiting the acquisition range of the first camera.

In the embodiment, the exit direction vector of the target to be tracked when exiting from the acquisition range of the first camera is obtained according to the world coordinates of at least two target track points in the world position information; and then, according to the exit direction vector, the exit direction of the target to be tracked when exiting the first camera acquisition range is determined.

Referring to fig. 8a, a fourth flowchart of the target tracking method provided in the embodiment of the present application is refined based on step S701 in fig. 7, and includes the following steps:

step S801, acquiring a first world coordinate and an Nth world coordinate in the world position information; the first world coordinate is the world coordinate of a target track point collected last in each target track point, or the first world coordinate is the average value of the world coordinates of i +1 target track points collected last in each target track point; the Nth world coordinate is the world coordinate of the last Nth collected target track point in each target track point, or the Nth world coordinate is the average value of the world coordinates of the last Nth to last N + i collected target track points in each target track point; n is an integer greater than 1, i is a positive integer;

in an example, as shown in fig. 8b, world coordinates (N may be adjusted according to actual conditions) of two target trace points (the last 1 trace point p1 and the nth last trace point pn) of the target to be tracked may be taken, and an average value of the world coordinates of the last 2 trace points and an average value of the world coordinates of the nth last trace point and the (N + 1) th last trace point may also be taken, so that noise point interference may be eliminated.

Step S802, calculating a vector pointing to the Nth world coordinate from the first world coordinate to obtain an exit direction vector when the target to be tracked exits the first camera acquisition range.

In an example, referring to fig. 8b, after world coordinates of two target track points of a target to be tracked at the end of a track within a first camera acquisition range are obtained, p1-pn is calculated to obtain a vector of the target to be tracked pointing to a departure direction of the target to be tracked, that is, a departure direction vector.

In the above embodiment, the vector pointing from the first world coordinate to the nth world coordinate is calculated through the first world coordinate and the nth world coordinate in the world position information, so that the obtaining of the outgoing direction vector when the target to be tracked exits the first camera acquisition range is realized.

Referring to fig. 9a, a fifth flowchart of the target tracking method provided in the embodiment of the present application is detailed based on step S702 in fig. 7, and includes the following steps:

step S901, converting the outgoing direction vector into a unit direction vector; wherein the abscissa of the vector in the unit direction is x, and the ordinate is y;

step S902, when x is larger than 0, y is larger than 0 and x is larger than y, determining that the exit direction of the target to be tracked when the target exits the acquisition range of the first camera is east;

step S903, when x is larger than 0, y is larger than 0 and x is smaller than y, determining that the exit direction of the target to be tracked when the target to be tracked exits from the acquisition range of the first camera is north;

step S904, when x is less than 0, y is more than 0 and-x is more than y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is west;

step S905, when x is smaller than 0, y is larger than 0 and-x is smaller than y, determining that the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is north;

step S906, when x is less than 0, y is less than 0 and-x is greater than-y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is west;

step S907, when x is smaller than 0, y is smaller than 0 and-x is smaller than-y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is south;

step S908, when x is larger than 0, y is smaller than 0 and x is larger than-y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is east;

and step S909, when x is larger than 0, y is smaller than 0 and x is smaller than-y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the first camera acquisition range is south.

Unitizing the exit direction vector to obtain a unit direction vector, and then judging the direction of the unit direction vector, namely the exit direction of the target to be tracked when the target exits the acquisition range of the first camera.

In one example, referring to fig. 9b, the actual geographic directions are four directions, i.e., east, west, south, north, west, x is longitude, y is latitude, and the normalized unit direction vector is assumed to be (x, y):

if x is greater than 0 and y is greater than 0, the unit direction vector is in the first quadrant, if x is greater than y, the unit direction vector is close to the x-axis positive half axis, the moving-out direction of the target to be tracked when the target to be tracked moves out of the first camera acquisition range is east, if x is less than y, the unit direction vector is close to the y-axis positive half axis, and the moving-out direction of the target to be tracked when the target to be tracked moves out of the first camera acquisition range is north.

If x is less than 0 and y is greater than 0, the unit direction vector is in the second quadrant, if-x is greater than y, the unit direction vector is close to the negative half shaft of the x axis, the moving-out direction of the target to be tracked when the target moves out of the first camera acquisition range is west, if-x is less than y, the unit direction vector is close to the positive half shaft of the y axis, and the moving-out direction of the target to be tracked when the target moves out of the first camera acquisition range is north.

If x is less than 0 and y is less than 0, the unit direction vector is in the third quadrant, if-x-y, the unit direction vector is close to the negative half axis of the x axis, the moving-out direction of the target to be tracked when the target moves out of the first camera acquisition range is west, if-x < -y, the unit direction vector is close to the negative half axis of the y axis, and the moving-out direction of the target to be tracked when the target moves out of the first camera acquisition range is south.

If x is greater than 0 and y is less than 0, the unit direction vector is in the fourth quadrant, if x is greater than-y, the unit direction vector is close to the positive half shaft of the x axis, the moving-out direction of the target to be tracked when the target moves out of the acquisition range of the first camera is east, if x is less than-y, the unit direction vector is close to the negative half shaft of the y axis, and the moving-out direction of the target to be tracked when the target moves out of the acquisition range of the first camera is south.

Once the exit direction of the target to be tracked exiting the first camera acquisition range is judged, the camera which is possible to go next is conveniently judged according to the exit direction.

In the above embodiment, the determination of the exit direction of the target to be tracked when exiting from the first camera acquisition range is realized by analyzing and judging the horizontal and vertical coordinates of the unit direction vector.

Referring to fig. 10, a sixth flowchart of the target tracking method provided in the embodiment of the present application is detailed based on step S504 in fig. 5, and includes the following steps:

step S1001, the world coordinates of each camera in the world coordinate system are obtained;

step S1002, aiming at each camera except the first camera, calculating the distance between the camera and the first camera according to the world coordinate of the camera and the world coordinate of the first camera;

step S1003, selecting M cameras closest to the first camera to obtain a plurality of cameras to be matched, wherein the cameras are adjacent to the first camera.

In one example, M cameras closest to the first camera are selected, a value range of M may be 5 to 10, and the value range may be adjusted according to an actual situation.

Step S1004, aiming at each camera to be matched, calculating to obtain a relative direction vector of the camera to be matched relative to the first camera;

traversing the M cameras, calculating the direction vector of each camera with respect to the first camera, and the calculation process is the same as that in the above embodiment, and is not described herein again.

Step S1005, determining, according to each of the relative direction vectors, a relative direction of each of the cameras to be matched with respect to the first camera, respectively.

In the above embodiment, the relative direction of each camera to be matched with respect to the first camera is obtained by calculating the relative direction vector of each camera to be matched with respect to the first camera.

Referring to fig. 11a, a seventh flowchart of the target tracking method provided in the embodiment of the present application is detailed based on step S505 in fig. 5, and includes the following steps:

step S1101, selecting K cameras to be matched closest to the first camera from among the cameras to be matched in the relative direction and the exit direction, and obtaining K second cameras, where K is a preset positive integer.

In one example, a camera to be matched with the same relative direction as the exit direction is selected as a camera to which a target candidate to be tracked goes. If the number of the candidate cameras is larger than two, the distances between all the candidate cameras and the first camera are calculated, and the closest 2 cameras can be used for cross-camera track matching of the target to be tracked. The track matching range is effectively reduced, and the matching range can be reduced to 1-2 from all cameras.

In one example, as shown in fig. 11b, 3 is a first camera, 1, 2, 4, 5 are cameras to be matched, and 5 is a camera to be matched whose relative direction matches the exit direction.

In the above embodiment, through the acquisition of the second camera, the cross-camera matching of the target to be tracked between the adjacent cameras can be realized, the cross-camera track matching range of the target to be tracked is effectively reduced, and the matching speed and accuracy are improved.

Referring to fig. 12, an eighth flowchart of the target tracking method according to the embodiment of the present application is schematically illustrated, and based on fig. 5, the method further includes the following steps:

step S1201, aiming at each second camera, determining the driving direction of the target to be tracked when the target to be tracked drives into the acquisition range of the second camera;

the determination process of the entering direction when the target to be tracked enters the second camera acquisition range is the same as the determination process of the exiting direction in the above embodiment, and details are not repeated here.

Step S1202, matching the exit direction with the entry direction of the target to be tracked when entering the acquisition range of the second camera to obtain a direction matching result of the second camera;

for each second camera, the direction matching result of the second camera may be that the exit direction when the target to be tracked exits the first camera acquisition range is matched with the entrance direction when the target to be tracked enters the second camera acquisition range, or that the exit direction when the target to be tracked exits the first camera acquisition range is not matched with the entrance direction when the target to be tracked enters the second camera acquisition range.

Step S1203, determining whether a cross-camera tracking result between the first camera and the second camera is reliable according to a direction matching result of the second camera.

If the exit direction of the target to be tracked when exiting the first camera acquisition range is matched with the entrance direction of the target when entering the second camera acquisition range, the cross-camera tracking result between the first camera and the second camera is credible; if the exit direction of the target to be tracked when exiting the first camera acquisition range is not matched with the entrance direction of the target to be tracked when entering the second camera acquisition range, the cross-camera tracking result between the first camera and the second camera is not credible.

In the above embodiment, for each second camera, it is determined whether the cross-camera tracking result between the first camera and the second camera is credible through the direction matching result of the second camera, so that powerful data support is added for the target tracking method.

In an example, referring to fig. 13, a ninth flowchart of a target tracking method provided in an embodiment of the present application is illustrated as cross-camera vehicle tracking, where the cross-camera vehicle matching includes the following steps: 1. converting the vehicle image coordinates to world (GPS) coordinates; 2. calculating the driving-in and driving-out directions of the vehicle according to the vehicle track; 3. searching a camera for driving the vehicle in and out according to the direction of the vehicle; 4. and matching adjacent camera vehicles.

In the above embodiment, the vehicle is tracked across cameras by automatically searching the previous camera where the vehicle enters the camera or the next camera where the vehicle exits the camera according to the entering/exiting direction of the vehicle when the vehicle enters/exits the camera collecting range.

Based on the same conception, the embodiment of the application also provides a target tracking device. Referring to fig. 13, a schematic structural diagram of an object tracking apparatus provided in an embodiment of the present application, the apparatus includes:

the first obtaining module 1410 is configured to obtain image position information of the target to be tracked in an image coordinate system of the first camera;

the second obtaining module 1420, configured to convert the image location information from the image coordinate system to a world coordinate system, so as to obtain world location information;

a first determining module 1430, configured to determine, according to the world location information, an exit direction of the target to be tracked when exiting from the first camera acquisition range;

a third obtaining module 1440, configured to obtain multiple cameras to be matched that are adjacent to the first camera;

a second determining module 1450, configured to determine a relative direction of each camera to be matched with respect to the first camera respectively;

the selecting module 1460 is used for selecting at least one second camera from the cameras to be matched, which are matched with the driving-out direction in the relative direction;

and the tracking module 1470 is configured to perform cross-camera tracking on the target to be tracked according to the image data of the first camera and the image data of the second camera.

the second obtaining module 1420 includes:

In the above embodiment, the homography matrix is used to perform mapping calculation on the image coordinates of each of the at least two target track points, so that the world coordinates of each of the at least two target track points are obtained. The at least two target track points are used for facilitating the subsequent determination of the exit direction of the target to be tracked when exiting from the acquisition range of the first camera.

In one possible implementation, the first determining module 1430 includes:

and the first determining submodule is used for determining the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera according to the exit direction vector.

acquiring a first world coordinate and an Nth world coordinate in the world position information; the first world coordinate is the world coordinate of a target track point collected last in each target track point, or the first world coordinate is the average value of the world coordinates of i +1 target track points collected last in each target track point; the Nth world coordinate is a world coordinate of a target track point acquired from the Nth to the (N + i) th from among the target track points, or the Nth world coordinate is an average value of world coordinates of target track points acquired from the Nth to the (N + i) th from among the target track points; n is an integer greater than 1, i is a positive integer;

when x is larger than 0, y is larger than 0 and x is larger than y, determining that the moving-out direction of the target to be tracked when the target to be tracked moves out of the acquisition range of the first camera is east;

when x is smaller than 0, y is larger than 0 and-x is larger than y, determining that the exit direction of the target to be tracked when the target exits the first camera acquisition range is west;

when x is smaller than 0, y is larger than 0 and-x is smaller than y, determining that the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is north;

when x is smaller than 0, y is smaller than 0 and-x is smaller than-y, determining that the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is south;

In the embodiment, the exit direction of the target to be tracked when the target exits from the acquisition range of the first camera is determined by analyzing and judging the horizontal and vertical coordinates of the unit direction vector.

In one possible implementation, the third obtaining module 1440 comprises:

In one possible implementation, the second determining module 1450 includes:

the fifth acquisition submodule is used for calculating and obtaining a relative direction vector of each camera to be matched relative to the first camera;

In a possible implementation, the selecting module 1460 includes:

In a possible embodiment, the apparatus further comprises:

the fourth acquisition module is used for matching the running-out direction with the running-in direction of the target to be tracked when running into the acquisition range of the second camera to obtain a direction matching result of the second camera;

In the above embodiment, for each second camera, whether the cross-camera tracking result between the first camera and the second camera is credible is determined according to the direction matching result of the second camera, so that powerful data support is added for the target tracking method.

The embodiment of the present application further provides an electronic device, as shown in fig. 15, which includes a processor 1501, a communication interface 1502, a memory 1503, and a communication bus 1504, wherein the processor 1501, the communication interface 1502, and the memory 1503 complete communication with each other through the communication bus 1504,

a memory 1503 for storing a computer program;

the processor 1501, when executing the program stored in the memory 1503, implements the following steps:

determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the world position information;

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In yet another embodiment provided by the present application, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above-mentioned object tracking methods.

In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the object tracking methods of the above embodiments.

In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the device, the electronic apparatus, and the storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.

The above description is only for the preferred embodiment of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method of target tracking, the method comprising:

and performing cross-camera tracking on the target to be tracked according to the image data of the first camera and the image data of the second camera.

2. The method according to claim 1, characterized in that the image position information comprises image coordinates of at least two target track points of the target to be tracked in the image coordinate system;

3. The method according to claim 2, wherein the determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the world position information comprises:

calculating to obtain a running-out direction vector when the target to be tracked runs out of the acquisition range of the first camera according to the world coordinates of at least two target track points in the world position information;

and determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the exit direction vector.

4. The method according to claim 3, wherein the step of calculating, according to world coordinates of at least two target track points in the world position information, an exit direction vector of the target to be tracked when exiting from the first camera acquisition range includes:

5. The method according to claim 3, wherein the determining the exit direction of the target to be tracked when exiting the first camera acquisition range according to the exit direction vector comprises:

6. The method of claim 1, wherein the obtaining a plurality of cameras to be matched adjacent to the first camera comprises:

acquiring world coordinates of each camera in the world coordinate system;

7. The method of claim 1, wherein the separately determining the relative orientation of each camera to be matched with respect to the first camera comprises:

8. The method according to claim 1, wherein the selecting at least one second camera among the cameras to be matched whose relative directions match the exit direction comprises:

9. The method of claim 1, further comprising:

matching the running-out direction with the running-in direction of the target to be tracked when running into the acquisition range of the second camera to obtain a direction matching result of the second camera;

10. An object tracking apparatus, characterized in that the apparatus comprises:

11. The apparatus according to claim 10, wherein the image position information includes image coordinates of at least two target track points of the target to be tracked in the image coordinate system;

the second acquisition module includes:

12. The apparatus of claim 11, wherein the first determining module comprises:

13. The apparatus of claim 12, wherein the second obtaining submodule is specifically configured to:

and calculating a vector pointing to the Nth world coordinate from the first world coordinate to obtain a running-out direction vector when the target to be tracked runs out of the acquisition range of the first camera.

14. The apparatus of claim 12, wherein the first determination submodule is specifically configured to:

15. The apparatus of claim 10, wherein the third obtaining module comprises:

16. The apparatus of claim 10, wherein the second determining module comprises:

17. The apparatus of claim 10, wherein the selecting module comprises:

18. The apparatus of claim 10, further comprising: