WO2018121286A1

WO2018121286A1 - Target tracking method and device

Info

Publication number: WO2018121286A1
Application number: PCT/CN2017/116329
Authority: WO
Inventors: 唐矗; 卿明; 吴庆; 孙晓路
Original assignee: 纳恩博（北京）科技有限公司
Priority date: 2016-12-30
Filing date: 2017-12-15
Publication date: 2018-07-05
Also published as: CN108269269A

Abstract

A target tracking method and device. The method comprises: acquiring a tracked target and a target template corresponding to the tracked target (S102); tracking the tracked target in a video image according to the target template to obtain a tracking result of a current frame image in the video image (S104), wherein the tracking result comprises: an area of the tracked target in the current frame image and a corresponding confidence degree; determining the tracking state of the tracked target according to the confidence degree (S106); and if the tracking state of the tracked target is lost, repositioning the tracked target and continuing tracking after the tracked target is repositioned (S108). The solution solves the technical problem in the prior art that it is difficult to determine whether a target is lost or difficult to retrieve after the target is lost when the target is tracked in a video image.

Description

Target tracking method and device

Technical field

The present invention relates to the field of video image processing, and in particular to a target tracking method and apparatus.

Background technique

The visual tracking technology based on online learning has become a hot spot of visual tracking after its rise in recent years. Such a method extracts a feature template according to the specified tracking target in the initial frame picture without any prior experience of offline learning, and the training model is used for tracking the target in subsequent videos. This type of method does not require any offline training, and can track any object specified by the user, which has high versatility. However, due to the complexity of the actual application scenario, the tracking algorithm has a severe performance degradation in the case of obvious illumination changes, complex backgrounds, and similar things. In addition, due to the unique characteristics of the tracking target and the template, in the target tracking process, It is difficult to accurately determine whether the target is lost or not; correspondingly, after the target is lost, the continuous update of the tracking template will amplify the error, making the target difficult to retrieve; in addition, the method will detect in each frame of the image to filter out Tracking candidate target areas, in this way, after the target loss reappears, the chances of re-finding the target are small, more situations are impossible to retrieve (because it has been the wrong tracking) or find the wrong target (such as Different people), making it difficult to form a stable tracking system for a long time.

Most of the current visual tracking methods use an online update template. The template is used to match each frame image, and the template is updated according to the matching result. A stable long-time tracking system cannot be formed by this method alone. :

1. This method cannot judge for itself whether the target has been lost or not;

2. The tracking error after the target is lost will continue to increase the error;

3. After losing the target, relying on the detection mechanism of this type of tracking method, it is difficult to accurately retrieve the tracking target before the loss.

In the prior art, when tracking a target object in a video image, even if the tracking continues to cause the target object to be difficult to relocate, an effective solution has not been proposed yet.

Summary of the invention

At least some embodiments of the present invention provide a target tracking method and apparatus to at least solve the technical problem in the prior art that when tracking a target in a video image, it is difficult to determine whether the target is lost or difficult to retrieve after the target is lost.

According to an embodiment of the present invention, a target tracking method is provided, including: acquiring a tracking target and a target template corresponding to the tracking target; tracking the tracking target in the video image according to the target template, and obtaining a tracking result of the current frame image in the video image. The tracking result includes: a region of the current target image that tracks the target and a corresponding confidence; the tracking state of the tracking target is determined according to the confidence; if the tracking state of the tracking target is lost, the tracking target is repositioned and repositioned Continue tracking after tracking the target.

Optionally, the target frame is scanned by the target template to obtain an area of the tracking target; the information of the tracking target is collected by using a preset sensor; and the information of the tracking target collected by the preset sensor is fused to obtain a confidence level.

Optionally, obtaining depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image; setting a weight value corresponding to depth confidence, tracking confidence, and color confidence; according to depth confidence, tracking confidence, and The color confidence and the corresponding weight value get confidence.

Optionally, comparing the depth of the area of the tracking target in the current frame image with the average depth of the area of the tracking target in each frame image, obtaining depth confidence; determining the area of the tracking target in the current frame image and the target template The similarity is the tracking confidence; determining the similarity between the color value of the region of the tracking target in the current frame image and the preset color model is the color confidence.

Optionally, a preset confidence threshold is obtained; the tracking state of the tracking target is determined by comparing the confidence with the confidence threshold, wherein the tracking state includes: tracking, low confidence, and loss.

Optionally, the candidate target is detected in the new video image by the target detector to determine the candidate target region; the tracking target is identified from the candidate target region.

Optionally, acquiring an image region of the tracking target; extracting feature information from the image region of the tracking target, and constructing the feature model according to the feature information; and determining, according to the tracking state, that the tracking target is not lost, according to the tracking result of the current frame image The feature model is updated until the tracking target is lost; the candidate target region is compared with the last updated feature model to identify the tracking target.

Optionally, the preset sensor comprises any one or more of the following: a depth camera, an infrared sensor, and an ultra-wideband positioning sensor.

Alternatively, if the tracking target is not lost, the target template is updated according to the tracking result of the current frame image.

According to an embodiment of the present invention, a target tracking device is further provided, including: a first acquiring module configured to acquire a tracking target and a target template corresponding to the tracking target; and a second acquiring module configured to be in the video image according to the target template Tracking the tracking target, obtaining the tracking result of the current frame image in the video image, wherein the tracking result includes: the area of the tracking target in the current frame image and the corresponding confidence; the determining module is set to determine the tracking of the tracking target according to the confidence level Status; reposition module, set to track the tracking status of the target To lose, retarget the tracking target and continue tracking after retargeting the tracking target.

Optionally, the second obtaining module includes: a scanning submodule, configured to scan the current frame image by the target template to obtain an area of the tracking target; and the collecting submodule is configured to collect information of the tracking target by using a preset sensor; the fusion processing submodule , configured to fuse the information of the tracking target collected by the preset sensor to obtain a confidence level.

Optionally, the fusion processing sub-module includes: a first acquiring unit, configured to acquire depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image; setting a unit, setting the depth confidence and tracking confidence a weight value corresponding to the color confidence; the second obtaining unit is configured to obtain a confidence according to the depth confidence, the tracking confidence and the color confidence, and the corresponding weight value.

Optionally, the first obtaining unit includes: an acquiring subunit, configured to compare the depth of the area of the tracking target in the current frame image with the average depth of the area of the tracking target in each frame image, to obtain a depth confidence; a determining subunit, configured to determine that the similarity between the region of the tracking target in the current frame image and the target template is tracking confidence; and the second determining subunit is configured to determine a color value and a preset of the region of the tracking target in the current frame image The similarity of the color model is the color confidence.

Optionally, the determining module includes: an obtaining submodule configured to obtain a preset confidence threshold; and a determining submodule configured to determine a tracking state of the tracking target by comparing the confidence with the confidence threshold, wherein the tracking Status includes: tracking, low confidence, and loss.

Optionally, the relocation module comprises: a detection submodule configured to detect the candidate target in the new video image by the target detector to determine the candidate target region; and the identification submodule configured to identify the tracking target from the candidate target region.

Optionally, the identifying sub-module includes: a third acquiring unit configured to acquire an image region of the tracking target; a constructing unit configured to extract feature information from the image region of the tracking target, and construct a feature model according to the feature information; The method is configured to update the feature model according to the tracking result of the current frame image until the tracking target is lost according to the tracking state, and the tracking target is lost; the comparison unit is configured to perform the candidate target region and the last updated feature model. Compare to identify the tracking target.

In at least some embodiments of the present invention, a tracking target and a target template corresponding to the tracking target are acquired, and the tracking target is tracked in the video image according to the target template, and a tracking result of the current frame image in the video image is obtained. Determining a tracking state of the tracking target according to the confidence level, if the tracking state of the tracking target is lost, relocating the tracking target and continuing tracking after repositioning the tracking target. The above solution determines the next step by continuously judging the tracking state during the tracking process, thereby enabling the tracking system to adapt to various states appearing in the long-term tracking process, and solving the problem in the tracking video image in the prior art in the prior art. At the time of the standard, it is difficult to judge whether the target is lost or a technical problem that is difficult to retrieve after the target is lost.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flow chart of a target tracking method according to an embodiment of the present invention;

2 is an information interaction diagram of a target tracking method according to an embodiment of the present invention;

3 is a schematic diagram of a target tracking device in accordance with one embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is an embodiment of the invention, but not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order. It is to be understood that the data so used may be interchanged where appropriate, so that the embodiments of the invention described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

Example 1

In accordance with an embodiment of the present invention, an embodiment of a target tracking method is provided, it being noted that the steps illustrated in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions, and Although the logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.

1 is a flow chart of a target tracking method according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:

Step S102: Acquire a tracking target and a target template corresponding to the tracking target.

Specifically, the foregoing tracking target may be determined by an artificially specified or by a pedestrian detector in an initial frame image of the video, and the target template may be a shape template of the tracking target.

Step S104, tracking the tracking target in the video image according to the target template, and obtaining a tracking result of the current frame image in the video image, where the tracking result includes: a region of the tracking target in the current frame image Corresponding confidence.

Specifically, the confidence in the tracking result of the current frame image is used to determine the similarity between the region of the tracking target in the tracking result and the target template. The similarity may include a parameter similarity, and may also include multiple parameter similarities.

Step S106, determining a tracking state of the tracking target according to the confidence level.

Specifically, the tracking status of the tracking target may include: a status of tracking, low confidence, loss, etc., but is not limited thereto, wherein the tracking status may be determined by one or more confidence thresholds or multiple confidence ranges.

Step S108: If the tracking status of the tracking target is lost, relocate the tracking target and continue tracking after repositioning the tracking target.

Specifically, the foregoing relocation is used to retrieve the initially set tracking target in the new video image. In the process of repositioning, two steps of target detection and target re-identification are required, wherein the target detection uses offline training. The obtained target detector detects the target in the video frame to obtain a candidate target region, and the target re-identification is used to identify the tracking target in the candidate target region. The target detection does not limit the specific target detector type, and may be a traditional template plus classifier sliding window detector or a depth learning based detector; the target re-identification method also does not define a specific method, and simple features can be used. In addition to the distance measurement method, the deep learning model can also be used to obtain feature extraction and similarity measurement methods.

It should be noted here that the above scheme fully adapts to various situations occurring during the long-term tracking process, and sets a perfect state judgment and jump mechanism corresponding to various states, thereby forming a robust tracking system.

It should also be noted here that when the target is retrieved and re-entered the target tracking step, a larger learning rate can be used in the initial stage to improve the update range of the tracking template to quickly reform a stable tracking.

As can be seen from the above, the foregoing step of the present application acquires a tracking target and a target template corresponding to the tracking target, and tracks the tracking target in the video image according to the target template to obtain a tracking result of the current frame image in the video image, according to The confidence determines a tracking state of the tracking target, and if the tracking state of the tracking target is lost, relocating the tracking target and continuing tracking after repositioning the tracking target. The above scheme The next step is determined by continuously judging the tracking state during the tracking process, so that the tracking system can adapt to various states appearing in the long-term tracking process, and the prior art in the prior art tracks the target in the video image. It is difficult to judge whether the target is lost or a technical problem that is difficult to retrieve after the target is lost.

Optionally, according to the foregoing embodiment of the present application, in step S104, the tracking target is tracked in the video image according to the target template, and the tracking result of the current frame image in the video image is obtained, including:

Step S1041: Scan the current frame image by using the target template to obtain an area of the tracking target.

In the above step, the target template is scanned in the current frame image to find the strongest peak region of the response, thereby determining the peak region with the strongest response as the region where the tracking target is located in the current frame image. It should be noted that the target template can always scan the region with the strongest response regardless of whether the current frame includes the tracking target. In the case where the current frame image does not include the tracking target, the region with the strongest response is not actually the region of the tracking target. After the tracking result is obtained, the tracking result needs to be further judged.

Step S1043: Collect information of the tracking target by using a preset sensor.

Specifically, the foregoing sensors may include, but are not limited to, a depth camera, an infrared sensor, a UWB sensor, etc., and the preset sensors may be a plurality of different sensors, or may be multiple types of sensors. The above steps use different sensors to obtain different information of the tracking target.

Step S1045: Perform fusion processing on the information of the tracking target collected by the preset sensor to obtain the confidence.

Generally, the confidence of the tracking result is determined only by the detection result of a single vision sensor (for example, a camera, etc.), and the information of the tracking target is collected by the preset sensor in the above step, thereby improving the accuracy of the confidence. To prevent the influence of changes in the environment and light on the confidence of the tracking results.

Optionally, according to the foregoing embodiment of the present application, in step S1045, the information of the tracking target collected by the preset sensor is fused to obtain the confidence, including:

Step S10451: Acquire depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image.

Step S10453, setting a weight value corresponding to the depth confidence, the tracking confidence, and the color confidence.

Step S10455: The confidence is obtained according to the depth confidence, the tracking confidence, the color confidence, and a corresponding weight value.

In an optional embodiment, the depth confidence, the tracking confidence, and the color confidence are respectively s _depth , s _tracking , and s _color , if the corresponding weight values are σ ₁ , σ ₂ , and σ _{3 , respectively.} Then, the confidence s can be calculated by the following formula: s = σ ₁ · s _depth + σ ₂ · s _tracking + σ ₃ · s _color .

Optionally, according to the foregoing embodiment of the present application, step S10451, obtaining depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image, including:

Step S10451a, the depth confidence is obtained by comparing the depth of the region of the tracking target in the current frame image with the average depth of the region of the tracking target in each frame image.

In an optional embodiment, the depth of the tracking target area can be obtained by using a depth camera mounted by the robot. Since the depth of the target does not change greatly in consecutive frames during continuous tracking, the frame-by-frame comparison is performed. The depth is changed to obtain the normalized depth confidence s _depth .

Step S10451b: determining that the similarity between the area of the tracking target and the target template in the current frame image is the tracking confidence.

In an optional embodiment, each frame tracking algorithm uses the target template to detect near the region where the previous frame target appears, find the target region closest to the target template, and normalize the tracking confidence s _tracking .

Step S10451c: determining that the similarity between the color value of the region of the tracking target in the current frame image and the preset color model is the color confidence.

In an optional embodiment, the target area detected in the previous embodiment may be sent to the target discarding module (ie, the module is set to determine the tracking status), and the target discarding module is obtained according to the preset color model. The normalized similarity s _{color of the} current target area to the color model.

Optionally, according to the foregoing embodiment of the present application, in step S106, determining a tracking status of the tracking target according to the confidence level, including:

Step S1061: Acquire a preset confidence threshold.

Step S1063: Determine a tracking state of the tracking target by comparing the confidence level with the confidence threshold, wherein the tracking state includes: tracking, low confidence, and loss.

In an optional embodiment, the foregoing preset reliability threshold may include a first confidence threshold and a second confidence threshold, where the first confidence threshold is greater than the second confidence threshold, and thus passes the first confidence threshold And the second confidence threshold may constitute a single interval, and the tracking state corresponding to the tracking result with the confidence greater than or equal to the first confidence threshold is the tracking result in the tracking, and the confidence is greater than the second confidence threshold and less than or equal to the first confidence threshold. The corresponding tracking state is low confidence, and the tracking state corresponding to the tracking result whose confidence is less than the second confidence threshold is lost.

In the above embodiment, if the tracking status of the tracking result is tracking or low confidence, the target template is updated by the tracking result to continue tracking, and if the tracking status of the tracking result is lost, repositioning Track the target.

Optionally, according to the foregoing embodiment of the present application, in step S108, the tracking target is relocated and the tracking is continued after the tracking target is repositioned, including:

Step S1081: The candidate target is detected in the new video image by the target detector to determine the candidate target region.

Specifically, the foregoing step is a step of target detection. In an optional embodiment, taking the target pedestrian in the video as an example, the target detector may be a pedestrian detector, and the pedestrian is detected in the video by the pedestrian detector. A candidate target area is obtained.

Step S1083: Identify the tracking target from the candidate target area.

Specifically, the above steps are a process of target re-identification, and the target is re-identified as a process of finding and identifying a pre-selected designated target from different scenarios. The method for re-identifying the target may be to use paired image data in different scenes of the same target and image data of different target pairs, respectively extracting specified features, such as color histograms, as feature vectors, and then learning by using the metric learning method. A similarity measure function, in the application, the similarity measure function is used to calculate the similarity between the two targets, thereby determining whether it is the same target, optionally, according to the above embodiment of the present application, step S1083, from the candidate The step of identifying the tracking target in the target area may further include:

Step S10831: Acquire an image area of the tracking target.

Step S10833, extracting feature information from an image region of the tracking target, and constructing a feature model according to the feature information.

Specifically, the extracted feature may be a color feature, an edge feature, or the like of the image. Since the tracking target is usually dynamic in the video, it is difficult to track the shape of the tracking target only, and the accuracy is low. However, for a continuous image in a video, the tracking target's time shape changes continuously with the change of the timestamp, but the features of the image are generally consistent, so the above steps construct the model by the extracted image features.

Step S10835: In the case that it is determined that the tracking target is not lost according to the tracking state, the feature model is updated according to the tracking result of the current frame image until the tracking target is lost.

Step S10837, comparing the candidate target area with the last updated feature model to identify the tracking target.

In an optional embodiment, in the case that it is determined that the current frame image has been lost, the tracking target can be retrieved based on the last updated tracking model, that is, the previous frame tracking model, since the previous frame image is not With the state of losing, Therefore, after obtaining the tracking result of the previous frame image, the feature model of the tracking target is also updated, so that the feature model used for retrieving the tracking target is the closest feature model.

It should be noted here that since the tracking target may be in a dynamic state in the video, other environmental information in the video also changes with time, that is, the shape of the tracking target is constantly changing in the video, and the video is in the video. Light and environment are also changing. Therefore, it is very difficult to track or retrieve by simply tracking the shape of the target. Further, tracking with the feature model of the initially determined tracking target does not yield accurate results. Therefore, the feature model of the tracking target introduced by the above scheme can effectively remove the influence of the change of the environment or the change of the shape of the target in the process of tracking or retrieving, thereby upgrading the robustness of the tracking model.

It can be seen that, in the above application, the feature information is extracted from the image area of the tracking target, and the feature mode is constructed according to the feature information, and if it is determined that the tracking target is not lost according to the tracking state, according to the The tracking result of the current frame image updates the feature model until the tracking target is lost, and compares the candidate target region with the last updated feature model to identify the tracking target. The above scheme constructs a feature model according to the feature information of the image region of the tracking target, and continuously updates the feature model according to different tracking results during the tracking process, and uses the feature model as a tracking model to track, thereby improving the tracking model. The stickiness further solves the technical problem that it is difficult to judge whether the target is lost when tracking the target in the video image in the prior art, or is difficult to retrieve after the target is lost.

Optionally, according to the foregoing embodiment of the present application, the preset sensor includes any one or more of the following: a depth camera, an infrared sensor, and an ultra-wideband positioning sensor.

Optionally, according to the foregoing embodiment of the present application, if the tracking target is not lost, the target template is updated according to the tracking result of the current frame image.

In the above steps, if the tracking status is in the tracking, that is, the state that is not lost, after the tracking result is detected, the tracking template is updated according to the tracking result, thereby maintaining an online updated template during the tracking process, thereby enabling tracking. The template can introduce features of the image that is closest to the current frame.

2 is an information interaction diagram of an optional target tracking method according to an embodiment of the present application. As shown in FIG. 2, in an optional embodiment, the tracking target 20 is selected by the target detector 10, according to The tracking target selected by the target detector 10 extracts the target template 30 of the tracking target, and tracks the tracking target in the video image according to the target template 30. After tracking the image for each frame 40, the tracking result is obtained, and then the tracking result is determined according to the tracking result. Tracking status, that is, performing tracking and discarding 50. If the tracking status corresponding to the tracking result is in tracking, the tracking result is used to update the target template 30, and the tracking is continued. If the tracking status corresponding to the tracking result is lost, the target re-discovery is performed. Returning to 60, wherein the target re-recovery 60 includes two steps of target detection and target re-identification, and the target recognition can be performed by the pedestrian detector 10, and then the target re-recognition is performed. If the tracking target is successfully re-finished, The target template 30 is updated using the retrieved tracking target, and if it is not retrieved, the target retrieval process is continued.

Example 2

According to an embodiment of the present invention, an embodiment of a target tracking device is provided. FIG. 3 is a schematic diagram of a target tracking device according to an embodiment of the present invention. As shown in FIG. 3, the device includes:

The first obtaining module 10 is configured to acquire a tracking target and a target template corresponding to the tracking target.

The second obtaining module 20 is configured to track the tracking target in the video image according to the target template, to obtain a tracking result of the current frame image in the video image, where the tracking result includes: the current frame image Track the area of the target and the corresponding confidence.

The determining module 30 is configured to determine a tracking state of the tracking target according to the confidence level.

The relocation module 40 is configured to reposition the tracking target and continue tracking after repositioning the tracking target if the tracking state of the tracking target is lost.

As shown in the above, the device of the present application acquires the tracking target and the target template corresponding to the tracking target by using the first acquiring module, and the tracking target is tracked in the video image according to the target template by the second acquiring module, to obtain the video. The tracking result of the current frame image in the image is determined by the determining module according to the confidence level, and if the tracking state of the tracking target is lost, the tracking target is repositioned by the repositioning module and Continue tracking after repositioning the tracking target. The above solution determines the next step by continuously judging the tracking state during the tracking process, thereby enabling the tracking system to adapt to various states appearing in the long-term tracking process, and solving the problem in the prior art tracking video image in the prior art. It is difficult to judge whether the target is lost or a technical problem that is difficult to retrieve after the target is lost.

Optionally, according to the foregoing embodiment of the present application, the second obtaining module includes:

And scanning a sub-module, configured to scan the current frame image by using the target template to obtain an area of the tracking target.

The collecting submodule is configured to collect information of the tracking target by using a preset sensor.

The fusion processing sub-module is configured to perform fusion processing on the information of the tracking target collected by the preset sensor to obtain the confidence level.

Optionally, according to the foregoing embodiment of the present application, the fusion processing submodule includes:

The first obtaining unit is configured to acquire depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image.

And a setting unit configured to set the depth confidence, the tracking confidence, and the weight value corresponding to the color confidence.

The second obtaining unit is configured to obtain the confidence according to the depth confidence, the tracking confidence, the color confidence, and a corresponding weight value.

Optionally, according to the foregoing embodiment of the present application, the first acquiring unit includes:

Obtaining a subunit, configured to compare the depth of the region of the tracking target in the current frame image with the average depth of the region of the tracking target in each frame image, to obtain the depth confidence.

The first determining subunit is configured to determine that the similarity between the area of the tracking target in the current frame image and the target template is the tracking confidence.

The second determining subunit is configured to determine that the similarity between the color value of the region of the tracking target in the current frame image and the preset color model is the color confidence.

Optionally, according to the foregoing embodiment of the present application, the determining module includes:

Get the submodule, set to get the preset confidence threshold.

A determining submodule is configured to determine a tracking state of the tracking target by comparing the confidence to the confidence threshold, wherein the tracking state comprises: tracking, low confidence, and loss.

Optionally, according to the foregoing embodiment of the present application, the relocation module includes:

A detection sub-module is arranged to detect a candidate target in a new video image by the target detector to determine a candidate target region.

An identification sub-module configured to identify the tracking target from the candidate target area.

Optionally, according to the foregoing embodiment of the present application, the identifying submodule includes:

And a third acquiring unit configured to acquire an image area of the tracking target.

a construction unit configured to extract feature information from an image region of the tracking target and construct a feature model based on the feature information.

And an updating unit, configured to update the feature model according to the tracking result of the current frame image until the tracking target is lost, if it is determined that the tracking target is not lost according to the tracking state.

The comparison unit is configured to compare the candidate target area with the last updated feature model to identify the tracking target.

Optionally, according to the foregoing embodiment of the present application, if the tracking target is not lost, according to the current frame image The tracking result updates the target template.

In the above embodiment, while continuously updating the target template, the tracking result is tracked and lost after obtaining the tracking result of each frame image, thereby determining the tracking state, and stopping in time when the tracking state is lost. Continue tracking and stop using the tracking results to update the target template. Instead, go to the target re-recovery step until you retrieve the tracking target, re-track and update the target template.

According to an embodiment of the present invention, there is further provided a storage medium, wherein the storage medium comprises a stored program, wherein the device in which the storage medium is located is controlled to execute the target tracking method when the program is running. The above storage medium may include, but is not limited to, a U disk, a read only memory (ROM), a random access memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and the like, which can store program codes.

According to an embodiment of the present invention, there is further provided a processor, wherein the processor is configured to execute a program, wherein the target tracking method is executed when the program runs. The above processor may include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA).

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the descriptions of the various embodiments are different, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed technical contents may be implemented in other manners. The device embodiments described above are only schematic. For example, the division of the unit may be a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may contribute to the prior art or all or part of the technical solution may be in the form of a software product. Formally embodied, the computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, server or network device, etc.) to perform all or part of the methods of the various embodiments of the present invention. step.

The above description is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.

Industrial applicability

As described above, the patch software upgrade method and system provided by at least some embodiments of the present invention have the following beneficial effects: determining the subsequent steps by continuously determining the tracking status during the tracking process, thereby enabling the tracking system to adapt to the long-term tracking process. Various states that appear.

Claims

A target tracking method, including:

Obtaining a tracking target and a target template corresponding to the tracking target;

Tracking the tracking target in the video image according to the target template, and obtaining a tracking result of the current frame image in the video image, where the tracking result includes: a region of the tracking target in the current frame image and a corresponding Confidence;

Determining a tracking state of the tracking target according to the confidence level;

If the tracking status of the tracking target is lost, the tracking target is repositioned and tracking continues after the tracking target is repositioned.
The method according to claim 1, wherein the tracking target is tracked in the video image according to the target template, and the tracking result of the current frame image in the video image is obtained, including:

Scanning the current frame image by the target template to obtain an area of the tracking target;

Collecting information of the tracking target by using a preset sensor;

Performing fusion processing on the information of the tracking target collected by the preset sensor to obtain the confidence.
The method according to claim 2, wherein the information of the tracking target collected by the preset sensor is subjected to a fusion process to obtain the confidence, comprising:

Obtaining depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image;

Setting a depth value, the tracking confidence, and a weight value corresponding to the color confidence;

The confidence is obtained based on the depth confidence, the tracking confidence, and the color confidence and corresponding weight values.
The method of claim 3, wherein obtaining depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image comprises:

Comparing the depth of the region of the tracking target in the current frame image with the average depth of the region of the tracking target in each frame image, the depth confidence is obtained;

Determining, that the similarity between the area of the tracking target in the current frame image and the target template is the tracking confidence;

Determining the similarity between the color value of the region of the tracking target in the current frame image and the preset color model is the color confidence.
The method according to any one of claims 1 to 4, wherein determining the tracking state of the tracking target based on the confidence comprises:

Obtain a preset confidence threshold;

A tracking state of the tracking target is determined by comparing the confidence with the confidence threshold, wherein the tracking state includes: tracking, low confidence, and loss.
The method of claim 5 wherein relocating the tracking target and continuing tracking after repositioning the tracking target comprises:

A candidate target is detected in a new video image by the target detector to determine a candidate target region;

The tracking target is identified from the candidate target area.
The method of claim 6 wherein identifying the tracking target from the candidate target regions comprises:

Obtaining an image area of the tracking target;

Extracting feature information from an image region of the tracking target, and constructing a feature model according to the feature information;

And determining, in the case that the tracking target is not lost according to the tracking state, updating the feature model according to a tracking result of the current frame image until the tracking target is lost;

The candidate target area is compared with the last updated feature model to identify the tracking target.
The method of claim 2, wherein the preset sensor comprises any one or more of the following: a depth camera, an infrared sensor, an ultra-wideband positioning sensor.
The method of claim 1, wherein if the tracking target is not lost, the target template is updated according to a tracking result of the current frame image.
A target tracking device comprising:

a first acquiring module, configured to acquire a tracking target and a target template corresponding to the tracking target;

a second acquiring module, configured to track the tracking target in the video image according to the target template, to obtain a tracking result of the current frame image in the video image, where the tracking result includes: the tracking in the current frame image The area of the target and the corresponding confidence;

Determining a module, configured to determine a tracking state of the tracking target according to the confidence level;

Relocating the module, if the tracking state of the tracking target is lost, relocating the tracking target and continuing tracking after repositioning the tracking target.
The apparatus of claim 10, wherein the second acquisition module comprises:

a scanning submodule, configured to scan the current frame image by using the target template to obtain an area of the tracking target;

Collecting a sub-module, configured to collect information of the tracking target by using a preset sensor;

The fusion processing sub-module is configured to perform fusion processing on the information of the tracking target collected by the preset sensor to obtain the confidence level.
The apparatus of claim 11, wherein the fusion processing sub-module comprises:

a first acquiring unit, configured to acquire depth confidence, tracking confidence, and color confidence of the tracking result in the current frame image;

a setting unit, configured to set a weight value corresponding to the depth confidence, the tracking confidence, and the color confidence;

The second obtaining unit is configured to obtain the confidence according to the depth confidence, the tracking confidence, the color confidence, and a corresponding weight value.
The apparatus of claim 12, wherein the first obtaining unit comprises:

Obtaining a subunit, configured to compare the depth of the area of the tracking target in the current frame image with the average depth of the area of the tracking target in each frame image, to obtain the depth confidence;

a first determining subunit, configured to determine that the similarity between the area of the tracking target in the current frame image and the target template is the tracking confidence;

The second determining subunit is configured to determine that the similarity between the color value of the region of the tracking target in the current frame image and the preset color model is the color confidence.
The apparatus according to any one of claims 10 to 13, wherein the determining module comprises:

Obtain a submodule, set to obtain a preset confidence threshold;

A determining submodule is configured to determine a tracking state of the tracking target by comparing the confidence to the confidence threshold, wherein the tracking state comprises: tracking, low confidence, and loss.
The apparatus of claim 14, wherein the repositioning module comprises:

a detection sub-module configured to detect a candidate target in a new video image by the target detector to confirm a candidate target area;

An identification sub-module configured to identify the tracking target from the candidate target area.
The apparatus of claim 15 wherein said identifying sub-module comprises:

a third acquiring unit, configured to acquire an image area of the tracking target;

a constructing unit configured to extract feature information from an image region of the tracking target, and construct a feature model according to the feature information;

And an updating unit, configured to update the feature model according to the tracking result of the current frame image until the tracking target is lost, if the tracking target is not lost according to the tracking state;

The comparison unit is configured to compare the candidate target area with the last updated feature model to identify the tracking target.
The apparatus according to claim 10, wherein if the tracking target is not lost, the target template is updated according to a tracking result of the current frame image.
A storage medium, characterized in that the storage medium comprises a stored program, wherein the device in which the storage medium is located is controlled to perform the target tracking method according to any one of claims 1 to 9 while the program is running.
A processor, wherein the processor is configured to execute a program, wherein the program is executed to perform the target tracking method according to any one of claims 1 to 9.