JPH056434A - Automatic tracking device - Google Patents

Abstract

PURPOSE:To attain the tracking of a subject even if the subject revolves or is hidden behind another object and then emerges again by obtaining the centroid position of the key feature value in a subject tracking result frame and updating a template into a pattern of the subject tracking result frame. CONSTITUTION:The image signals are inputted to a template matching part 5 and a result display part 11. The part 5 detects a place having the highest correlation with the template stored in a template storage part 6 and outputs the place as a subject tracking result frame. A centroid calculation part 7 calculates the centroid position of a key color in the subject tracking result frame. A comparison part 8 compares the calculated centroid position with the result of a centroid calculation part 4 obtained in an initialization state and gives an updating command to a template updating part 9 if the centroid position of the key color is smaller than the prescribed value. Then the part 9 updates the contents of the part 6 into the pattern of the subject tracking result frame obtained by the part 5. A result display part 11 outputs the subject tracking result frame after superposing it on an input image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic tracking device for tracking an object using a digital image signal.

[0002]

2. Description of the Related Art A method called template matching is generally used as a method for determining which part of an image corresponds to which part of another image. In the template matching method, the template representing the object to be detected is moved with respect to all or part of the image, and the correlation value between the template and the partial pattern of the overlapping image is obtained at each position, and the value is maximized. It is a method of obtaining the position. Therefore, by first obtaining the template of the target object to be tracked and performing template matching for each frame, tracking of the target object by the template in the moving image is realized.
However, it is generally considered that the object changes its angle with respect to the camera, and in that case, the visual shape of the object changes. As a method capable of coping with such a situation, for example, as shown in Japanese Patent Laid-Open No. 1-315884, a pattern closest to the template is detected, and the pattern is replaced with the existing template to perform tracking. A method has been proposed.

[0003]

However, in the above-mentioned configuration, the contents of the template are sequentially replaced. Therefore, when the object is rotated as shown in FIG. 7, for example, as shown in FIG. However, there was a problem in that it gradually slipped off, and eventually it fell off the object and tracking failed. Further, for example, when the target object is hidden behind another object and reappears, the content of the template is replaced with the pattern of another object as shown in FIG. 9, and the target object reappears. It also had a problem that it could not be tracked.

In view of the problems of the conventional automatic tracking device, the present invention provides an automatic tracking device capable of tracking even when an object rotates or hides behind another object and appears again. The purpose is to do.

[0005]

According to the present invention, a target frame setting means for setting a target frame for tracking an object at the time of initial setting, and a feature amount representing the object in the target frame are determined. Representative feature amount determining means for storing, a template storage means for storing the pattern of the target frame as a template at the time of target tracking, and an object tracking result in a place in the input image signal having the highest correlation with the template. Template matching means for detecting as a frame, in the detected object tracking result frame, using the representative feature amount, the template update determination means for determining whether to update the template until then, An automatic tracking device including a template updating unit that updates the template up to that time to the pattern of the target tracking result frame based on the determination result.

Further, according to the present invention, at the time of initial setting, a target frame setting means for setting a target frame for tracking the target object, and a representative for determining a feature amount representing the target object within the target frame. A feature amount determination means, a template storage means for storing a pattern of a target frame as a template at the time of target tracking, and a location in an input image signal having the highest correlation with the template is detected as a target tracking result frame. Template matching means for, and using the representative feature amount in the detected object tracking result frame, frame position correction means for correcting the position of the target tracking result frame,
An automatic tracking device comprising: a template updating unit for updating the template to the pattern of the target tracking result frame corrected by the frame position correcting unit.

[0007]

According to the present invention, when the position of the center of gravity in the target frame is set as the representative feature amount, the target frame for tracking the object and the representative feature amount in the target frame are determined at the time of initial setting. , The center of gravity position of the representative feature amount in the target frame is obtained in advance, and the center of gravity position of the representative feature amount in the target tracking result frame obtained as a result of template matching during target tracking is obtained, and the center of gravity position obtained at initial setting If the difference between and is smaller than a predetermined value, the template is updated to the pattern of the target tracking result frame.

Further, at the time of initial setting, a target frame for tracking an object and a representative feature amount in the target frame are determined, and a barycentric position of the representative feature amount in the target frame is obtained in advance. Sometimes the centroid position of the representative feature amount in the target tracking result frame obtained as a result of template matching is found, and the position of the target tracking result frame is corrected using the barycentric position found at the time of initial setting. Update to the pattern.

[0009]

1 is a block diagram of an automatic tracking device according to a first embodiment of the present invention. In FIG.
Reference numeral 1 is an image signal input terminal, 2 is a target frame setting section in which an operator sets a target frame, 3 is a representative color determining section for determining a color representative of an object in the target frame (hereinafter referred to as a representative color), 4 Is a center of gravity calculation unit for obtaining the center of gravity of the representative color in the target frame, 5 is a template matching unit for detecting a location having the highest correlation with the template, 6 is a template storage unit, and 7 is a result of the template matching unit 5. Center of gravity calculation unit for obtaining the center of gravity of the representative color in the target tracking result frame to be selected, 8
Is a comparison unit for comparing the results of the gravity center calculation unit 4 and the gravity center calculation unit 7, 9 is a template updating unit for updating the template to the pattern of the target tracking result frame, and 10 is an input switching unit.

The operation of the automatic tracking device of the first embodiment constructed as above will be described below.

First, the operation at the time of initial setting will be described. At the time of initial setting, as shown in FIG. 2, the operator arbitrarily determines the target object 20 to be tracked, and the target frame setting unit 2 sets a rectangular target frame 21 around the target object 20. For example, the upper left coordinates (i, j) of the target frame 21 and the vertical and horizontal lengths of the target frame 21 are determined.

The input switching unit 10 is switched to the side a at the initial setting, and when an image signal is input to the image signal input terminal 1, it is input to the representative color determining unit 3. The representative color determination unit 3 selects the color having the largest number of pixels in the target frame 21 set by the target frame setting unit 2 and sets it as the representative color. As for the color, for example, as shown in FIG. 3, the input image signal is RG.
In the case of three-dimensional data of B (red, green, blue), if 0 to the maximum value is divided into 8 for each signal, the color is 8 × 8 × 8.
Are divided into 512 to obtain 512 colors. Therefore, for example, all the colors existing in the shaded range of the RGB data of the image signal are R (red): frequency 3, G (green): frequency 0, B
(Blue): Considered to be the same color with a frequency of 5. The center-of-gravity calculation unit 4 calculates the center-of-gravity position of the representative color determined by the representative-color determination unit 3 within the target frame set by the target frame setting unit 2. For example, as shown in FIG.
In the case where the representative color exists as indicated by diagonal lines in 0, the position of the center of gravity 31 of the representative color is calculated, and the position of the center of gravity is calculated as the relative position (x ₀ , y ₀ ) from the upper left of the target frame 30. Is output to the comparison unit 8. On the other hand, the template updating unit 9 stores the pattern in the target frame set by the target frame setting unit 2 in the template storage unit 6 as the initial value of the template.

Next, the operation after the initialization is completed will be described. The input switching unit 10 switches to the b side,
The image signal input to the input terminal 1 is input to the template matching unit 5 and the result display unit 11. The template matching unit 5 detects a place having the highest correlation with the template stored in the template storage unit 6 and outputs the position as a target tracking result frame. As a function of the template matching unit 5, a conventional template matching method is used. The center-of-gravity calculator 7 calculates the center-of-gravity position of the representative color inside the target tracking result frame obtained by the template matching unit 5. For example, in FIG.
As shown in (b), the relative position (x ₁ , y ₁ ) from the upper left of the target tracking result frame 33 is output as the position of the center of gravity 32 of the representative color (shaded portion) inside the target tracking result frame 33. To do. Then, the comparing unit 8 compares the results of the center-of-gravity calculation section 4 (x _0, y ₀₎ with the results of centroid calculation section 7 (x _1, y _1), the difference _{| x 1 -x 0 | + |} y ₁₋ y ₀ | is calculated, and if it is smaller than a predetermined value (for example, 1/4 of the horizontal length of the target frame), the template updating unit 9 is instructed to update the template. If it is larger than the predetermined value, the template is not updated. Template update determination means is composed of the center of gravity calculation units 4 and 7, the comparison unit 8 and the like.

Upon receiving the template update command, the template update unit 9 updates the content of the template storage unit 6 to the pattern of the target tracking result frame obtained by the template matching unit 5. The result display unit 11 superimposes the target tracking result frame 33 obtained by the template matching unit 5 on the input image input from the image input terminal 1 and outputs it to the monitor TV or the like. After that, this series of processing is repeated.

As described above, according to this embodiment, the barycentric position of the representative color in the target tracking result frame obtained as a result of template matching and the barycentric position of the representative color in the target frame at the time of initialization are set. The template is updated only when the difference between the two is smaller than a predetermined value, so the template content is updated so that it does not include the object even if the object rotates or is hidden behind other objects. Can be followed and excellent tracking can be performed.

FIG. 5 is a block diagram of an automatic tracking device according to the second embodiment of the present invention. 5, the same components as those in FIG. 1 are designated by the same reference numerals, and 20 is a frame position correction unit.

The operation of the automatic tracking device of the second embodiment constructed as above will be described below.

First, the operation at the time of initial setting will be described. At the time of initial setting, as in the first embodiment, the operator sets a rectangular target frame around the target object by the target frame setting unit 2. The input switching unit 10 is switched to the side a at the initial setting, and when the image signal is input to the input terminal 1, it is input to the representative color determination unit 3. The representative color determination unit selects a color having the largest number of pixels in the target frame set by the target frame setting unit 2 and sets it as a representative color. Center of gravity calculator 4
Then, within the target frame set by the target frame setting unit 2,
The barycentric position of the representative color determined by the representative color determination unit 3 (x
₀ , y ₀ ) is calculated and output to the frame position correction unit 20. Also,
The template updating unit 9 causes the template storage unit 6 to store the pattern in the target frame set by the target frame setting unit 2 as a template.

When the initial setting is completed, the input switching unit 1
0 is switched to the b side, and the image signal input to the input terminal 1 is input to the template matching unit 5 and the result display unit 11. The template matching unit 5 detects a place having the highest correlation with the template stored in the template storage unit 6 as a target tracking result frame and outputs the position. The center-of-gravity calculation unit 7 calculates the center-of-gravity position (x ₁ , y ₁ ) of the representative color inside the target tracking result frame obtained by the template matching unit 5 and outputs it to the frame position correction unit 20.

Then, the frame position correction unit 20 corrects the position of the target tracking result frame using the results of the gravity center position calculation unit 4 and the gravity center position calculation unit 7. Figure 6 shows the initial settings.
As shown in (a), the barycentric position 41 of the representative color is (x ₀ , y ₀ ), and the result of the template matching unit 5 is the target tracking result frame 42 as shown in FIG. When the position 44 is (x ₁ , y ₁ ), the frame position correction unit 20 moves the target tracking result frame 42 by | x ₁ −x ₀ | in the horizontal direction (x ₁ > x ₀ , x ₁₋ x _{0 to the} right, x ₁ <x _{0 to} the left only x ₀ −x ₁ ), and only | y ₁ −y ₀ | in the vertical direction (y
_{If 1} > y ₀ , y ₁ −y ₀ down, and if y ₁ <y ₀ , y
The position of the corrected frame such as 43 is output after correction by _0- y ₁ ). The template updating unit 9 updates the contents of the template storage unit 6 to the pattern of the target tracking result frame corrected by the frame position correcting unit 20. Result display part 1
In 1, the target tracking result frame obtained by the frame position correction unit 20 is superimposed on the input image input from the image input terminal 1 and output to a monitor television or the like. Center-of-gravity position calculation unit 4,
7. The frame position correcting section 20 constitutes a frame position correcting means.

As described above, according to the present embodiment, the barycentric position of the representative color in the target tracking result frame obtained as a result of template matching and the barycentric position of the representative color in the target frame at the time of initial setting are set. The template position is corrected after the position of the target tracking result frame is corrected based on the difference between, so that the template position is corrected by the representative color even when the target object rotates or is hidden behind another object. For,
Good tracking can be performed.

In the first and second embodiments, the color having the largest number of pixels in the target frame is selected as the representative color. However, for example, the color having the largest area of the target region is the target frame. Other criteria may be selected as long as they represent the characteristics of the object. Further, although the color is used as the quantity representing the feature of the target object within the target frame, the color is not particularly limited, and any feature amount of the target object may be used.

[0023]

As described above, according to the automatic tracking device of the present invention, even if the object rotates or hides behind another object and appears again, the automatic tracking is surely performed. It is possible and its practical effect is great.

[Brief description of drawings]

FIG. 1 is a block diagram of an automatic tracking device according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram for explaining an object and an object frame.

FIG. 3 is an explanatory diagram for explaining the operation of FIG.

FIG. 4 is an explanatory diagram for explaining the operation of FIG.

FIG. 5 is a block diagram of an automatic tracking device according to a second embodiment of the present invention.

6 is an explanatory diagram for explaining the operation of FIG. 5;

FIG. 7 is an explanatory diagram for explaining an operation of a conventional automatic tracking device.

FIG. 8 is an explanatory diagram for explaining the operation of the conventional automatic tracking device.

FIG. 9 is an explanatory diagram for explaining the operation of the conventional automatic tracking device.

[Explanation of symbols]

1 Image signal input terminal 2 Target frame setting section 3 Representative color determination unit 4,7 Center of gravity calculator 5 Template matching section 6 Template storage 8 comparison section 9 Template update section 10 Input switching section 11 Result display section 12 Search range determination unit 20 Frame position correction unit

Claims (4)

[Claims] 1. A target frame setting means for setting a target frame for tracking an object at the time of initial setting, and a representative feature amount determining means for determining a feature amount representative of the object within the target frame. And a template storage unit for storing a pattern of a target frame as a template during target tracking, and template matching for detecting a location in an input image signal having the highest correlation with the template as a target tracking result frame. Means, in the detected object tracking result frame, using the representative feature amount, template update determination means for determining whether to update the template so far, and based on the determination result, An automatic tracking device, comprising: a template updating unit that updates a template to the pattern of the target tracking result frame. 2. A template update determination means, which is a first center of gravity calculation means for obtaining a center of gravity of a representative feature quantity in the target frame at the time of initial setting, and the target tracking result obtained as a result of the template matching means. It is composed of a second center of gravity calculating means for obtaining the center of gravity of the representative feature quantity within a frame, and a comparing means for comparing the results of the first and second center of gravity calculating means. The automatic tracking device according to claim 1, wherein when the difference between the results of the first and second center of gravity calculating means is smaller than a predetermined value, the template updating means determines to update. 3. A target frame setting means for setting a target frame for tracking an object at the time of initial setting, and a representative feature amount determining means for determining a feature amount representing the object in the target frame. And a template storage unit for storing a pattern of a target frame as a template during target tracking, and template matching for detecting a location in an input image signal having the highest correlation with the template as a target tracking result frame. Means and a frame position correction means for correcting the position of the target tracking result frame by using the representative feature amount in the detected object tracking result frame, and the template is corrected by the frame position correction means. An automatic tracking device comprising: a template updating unit for updating the pattern of the target tracking result frame. 4. The frame position correcting means includes a first center of gravity calculating means for obtaining a center of gravity of a representative feature amount in the target frame at the time of initial setting, and the target tracking result obtained as a result of the template matching means. A second center of gravity calculating means for obtaining a center of gravity of the representative feature quantity within a frame;
4. The automatic tracking device according to claim 3, further comprising: a frame position correction unit that corrects the position of the target tracking result frame using the result of the second center-of-gravity calculation unit.