CN102156991A - Quaternion based object optical flow tracking method - Google Patents

Abstract

The invention discloses a quaternion based object optical flow tracking method in the technical field of video image processing. In the method, color is processed and an optical flow is estimated in an entire signal mode by utilizing quaternion expression of color. Through the method, more accurate optical flow estimation can be achieved at a pixel position with space color change, so that characteristic points on an object can be more robustly tracked to reduce tracking errors. In the method, quaternion color corners are also used as reliable characteristic points during tracking. A good characteristic point set is formed by using the quaternion color corners and gray value corners together; and the object optical flow tracking is performed by combining a quaternion optical flow estimation algorithm.

Description

Object Optical Flow Tracking Method Based on Quaternions

technical field

The invention relates to a method in the technical field of video image processing, in particular to a quaternion-based object optical flow tracking method.

Background technique

Object optical flow tracking first detects the feature points on the tracked object that can be reliably tracked in the first frame of the image sequence, and then uses the optical flow estimation algorithm to calculate the optical flow of these feature points in two adjacent frames of the image sequence, The position of the feature point in the next frame is obtained, and so on, and the positions of all the feature points on the tracked object in all frames of the image sequence are obtained, so as to obtain the position of the tracked object. Optical flow estimation algorithms have been used for many years in the field of image processing and computer vision. The Lucas-Kanade optical flow estimation algorithm has high efficiency and can estimate the optical flow of sparse feature points, so it is suitable for real-time object tracking. However, the original algorithm is only based on gray-value images, while color as a whole unit contains more information. In addition, when used for object tracking, some color features are very stable and suitable for tracking, but they lose their significance when converted to gray values, and the Lucas-Kanade optical flow estimation algorithm cannot work well. In some other cases, such as the difference in illumination between two frames of images used to estimate the optical flow, the color information will be more stable than the grayscale information, which helps to estimate a more accurate optical flow. Therefore it is very important to use color information when performing optical flow estimation.

After searching the literature of the prior art, it was found that "Optical Flow Estimation" published by C.Lei and Y.Yang in "Proceedings of the 12th International Conference on Computer Vision" (the 12th International Conference on Computer Vision) on pages 1562 to 1569 On Coarse-to-Fine Region-Trees using Discrete Optimization"(Using Discrete Optimal Flow Estimation on Coarse to Fine Part Trees) article proposes a method of using color segmentation results to improve optical flow estimation results. But this method is based on overall energy minimization, so it is not suitable for tracking sparse feature points. G. Demarcq et al published "The Color Monogenic Signal: A new framework for color image processing. Optical flow" (color single gene signal: a new framework for color image processing and its application in color optical flow estimation) article based on the color single gene signal of color image, the concept of local color phase is proposed, and the local color phase is used to estimate the optical flow. P.Golland and A.M.Bruckstein published "Motion from Color" (obtaining motion information from color) in "Computer Vision and Image Understanding" (Computer Vision and Image Understanding) magazine, Volume 68, Issue 3, Page 346 to 362, 1997 In the article, and "Optical flow using color information: preliminary results" (using Optical Flow Estimation of Color Information: Preliminary Results) article uses a direct three-channel method, applying Lucas-Kanade optical flow constraints on the RGB three channels separately, and merging these constraints to estimate optical flow. There are also some other research work that converts the color from the RGB space to other color spaces, and estimates the optical flow in the converted space, such as X.Xiang et al. in "Proceedings of 2009 International Conference on Test and Measurement" (2009 test and Measurement of the International Conference Collection) published on pages 378 to 381 "A method of optical flow computation based on LUV color space" (an optical flow calculation method based on LUV color space) article converts color from RGB space to LUV space, while the aforementioned article by P.Golland and A.M.Bruckstein converts color from RGB space to HSV space. In terms of using RGB space or other color spaces to estimate optical flow, previous work still regards color as a separate channel signal, rather than treating color as an overall signal. This motivates the search for a new color optical flow estimation algorithm that utilizes color information holistically and improves the accuracy of optical flow estimation. Furthermore, successful object flow tracking requires the detection and tracking of reliable feature points. Intuitively, color features are reliable in tracking and more stable than grayscale features in the presence of illumination changes. This prompts to find a color feature suitable for object optical flow tracking, and cooperate with the color optical flow estimation algorithm to improve the accuracy of object tracking.

Contents of the invention

The present invention aims at the above-mentioned deficiencies in the prior art, and provides a quaternion-based object optical flow tracking method, which utilizes the quaternion representation of the color, processes the color and estimates the optical flow in the form of an overall signal. In this way, more accurate optical flow estimation can be obtained at pixel positions with spatial color changes, so that feature points on objects can be tracked more robustly and tracking errors can be reduced. At the same time, the present invention adopts quaternion color corner points as reliable feature points during tracking. The quaternion color corner points and the gray value corner points are used to form a good feature point set for tracking, combined with the quaternion optical flow estimation algorithm for object optical flow tracking.

The present invention is achieved through the following technical solutions, and the present invention comprises the following steps:

The first step is to set the range of the target to be tracked in the first frame I of the image sequence and detect the Harris corners within this range, and retain the n _h Harris corners whose corner degree measure is greater than the threshold γ, and the specific steps include :

1.1) Calculate the gradient I _x and I _y of the first frame I of the image sequence in the two-dimensional space x and y directions as the amplitude variation of the local neighborhood of the pixel point;

1.2) Calculate the corner degree measure through the magnitude correlation matrix M to obtain Harris corner points whose corner degree measure is greater than the threshold γ.

其中g(σ)为高斯平滑滤波器，σ为滤波器尺寸参数，*为卷积。The magnitude correlation matrix

Where g(σ) is the Gaussian smoothing filter, σ is the filter size parameter, and * is the convolution.

The corner degree measure Cornerness=det(M)-k trace ² (M), wherein det and trace represent the determinant and trace of the magnitude correlation matrix M, and k is an adjustment parameter.

The second step, detecting quaternion color corner points within the range of the target to be tracked, and retaining n _q quaternion color corner points whose color corner degree measure is greater than threshold γ _q , the specific steps include:

2.1) Express the first frame I of the image sequence as a pure quaternion matrix I _q , perform the following quaternion filtering on I _q , and take the saturation of the filtering result to represent the color change degree C _x and y in the x direction The degree of color change C _y in the direction:

$h_x=\left[\begin{array}{ccc}R& 0& R^{*}\\ R& 0& R^{*}\\ R& 0& R^{*}\end{array}\right]\left[I_q\right]\left[\begin{array}{ccc}{R}^{*}& 0& R\\ R^{*}& 0& R\\ R^{*}& 0& R\end{array}\right],$ C _x =Saturation(H _x ),

$h_{\mathrm{the\; y}}=\left[\begin{array}{ccc}R& R& R\\ 0& 0& 0\\ R^{*}& R^{*}& R^{*}\end{array}\right]\left[I_q\right]\left[\begin{array}{ccc}{R}^{*}& R^{*}& R^{*}\\ 0& 0& 0\\ R& R& R\end{array}\right],$ C _y =Saturation(H _y ),

Where: R＝Se ^μπ/4 ＝S{cosπ/4+sinπ/4}, μ is the unit pure quaternion

2.2) Calculate the degree measure of the color corner point through the color correlation matrix M _q to obtain the quaternion color corner points whose color corner degree measure is greater than the threshold γ _q .

其中g(σ)为高斯平滑滤波器，σ为滤波器尺寸参数，*为卷积。The color correlation matrix

Where g(σ) is the Gaussian smoothing filter, σ is the filter size parameter, and * is the convolution.

The measure of the color corner degree refers to: Cornerness _q = det(M _q )/(trace(M _q )+δ), wherein: det and trace represent the determinant and trace of the color correlation matrix M, and δ=2.2204e -016.

The third step is to combine n _h Harris corner points and n _q quaternion color corner points as the feature set of optical flow tracking. For each feature point in the feature set, use the quaternion-based optical flow estimation algorithm in the image The optical flow is estimated between two adjacent frames of the sequence, and the obtained optical flow value is used to update the position of the feature point in the second frame and realize the tracking. The specific steps include:

3.1) At time t, use the following steps to estimate the optical flow of each feature point in the feature point set between two adjacent frames I ^t and I ^t+1 of the image sequence, and add the position of the feature point at time t to its optical flow Flow, get the position of the feature point at time t+1, and perform optical flow estimation at time t+1. Initially t=1.

3.2) Downsample I ^t and It ⁺¹ respectively and generate p-level image pyramids I ^t,l and I ^t+1,l ,l∈[1,p] are image pyramid layer numbers. First, perform the optical flow estimation described in steps 3.3) and 3.4) at the top of the pyramid, and the obtained optical flow is enlarged to the next level of the pyramid, as the initial value of the optical flow at this level, and then perform steps 3.3) and 3.4) again Optical flow estimation. Repeat this until the bottom of the pyramid to get the final estimated optical flow. In order to further improve the estimation accuracy, the optical flow estimation described in steps 3.3) and 3.4) performs q cycles at each level of the image pyramid.

和

给定特征集中的Harris角点或四元数颜色角点中任一在三维空间的坐标为(x，y,t)，取其空间方向上w×w大小的局部邻域；令该邻域中心(x，y)处像素和另一像素i的四元数颜色值分别为Q^(c)和Q⁽ⁱ⁾，计算Q^(c)和Q⁽ⁱ⁾的四元数内积以表示像素i和中心像素的颜色相似程度，当颜色相似程度小于阈值α则排除像素i不执行以下步骤。3.3) In the first layer of the image pyramid, represent I ^{t, l} and I ^{t+1, l} as pure quaternion images respectively

and

The coordinates of any of the Harris corners or quaternion color corners in the given feature set are (x, y, t) in three-dimensional space, and the local neighborhood of w×w size in the spatial direction is taken; the neighborhood The quaternion color values of the pixel at the center (x,y) and another pixel i are Q ^(c) and Q ⁽ⁱ⁾ respectively, and the quaternion inner product of Q ^(c) and Q ⁽ⁱ⁾ is calculated to represent the pixel The color similarity between i and the central pixel, when the color similarity is less than the threshold α, exclude pixel i and do not perform the following steps.

The color similarity refers to L _i =<Q ⁽ⁱ⁾ , Q ^(c) >/(|Q ⁽ⁱ⁾ |·|Q ^(c) |), wherein: <·,·> represents a quaternion The inner product, |·| represents the quaternion modulus.

在空间x和y方向上的四元数梯度Q_x、Q_y。给定特征集中的Harris角点或四元数颜色角点中任一在三维空间的坐标为(x，y，t)，取其空间方向上w×w大小的局部邻域，取

中初始光流指向的相对应的邻域，计算该邻域时间方向的四元数梯度Q_t。当经过3.3)排除步骤后，取邻域内剩余的像素点并得到四元数光流等式：其中：是待估计的光流，数字上标表示邻域内剩余的像素点，上式的四元数矩阵形式为A_qv_q＝b_q，A_q是n×2纯四元数矩阵，b_q是n×1纯四元数向量，v_q是2×1四元数向量，

n×1四元数向量q的模定义为||q||＝sqrt(∑_n|q_n|²)，|q_n|是每个四元数元素的模。计算v_q＝(A_q ^HA_q)^-1A_q ^Hb_q，所求的光流为

其中S()表示取四元数的标量部分。3.4) Calculate the quaternion image

Quaternion gradients Q _x , Q _y in spatial x and y directions. The coordinates of any of the Harris corner points or quaternion color corner points in the given feature set in the three-dimensional space are (x, y, t), and the local neighborhood of w×w size in the space direction is taken, and the

In the corresponding neighborhood pointed by the initial optical flow, calculate the quaternion gradient Q _t in the time direction of the neighborhood. After 3.3) the exclusion step, take the remaining pixels in the neighborhood and get the quaternion optical flow equation: in: is the optical flow to be estimated, and the digital superscript indicates the remaining pixels in the neighborhood. The quaternion matrix form of the above formula is A _q v _q = b _q , A _q is an n×2 pure quaternion matrix, and b _q is n×1 pure quaternion vector, v _q is a 2×1 quaternion vector,

The modulus of n×1 quaternion vector q is defined as ||q||=sqrt(∑ _n |q _n | ² ), where |q _n | is the modulus of each quaternion element. Calculate v _q ＝(A _q ^H A _q ) ^-1 A _q ^H b _q , the obtained optical flow is

Among them, S() means to take the scalar part of the quaternion.

The principle of the present invention is to use the quaternion representation of the color, process the color as a whole signal, and use the color corner points detected by the quaternion to form a feature point set for object tracking together with the gray corner points to reduce the required features. Points, improve the tracking speed, and use the optical flow estimation algorithm based on quaternion to track the object with optical flow. Perform quaternion filtering on the color image to obtain the degree of color change, and calculate the saturation of the filtering result to detect color corners. The optical flow estimation algorithm based on quaternions can estimate the optical flow more accurately in areas with color changes, and exclude pixels with large color differences in the local neighborhood, which is more in line with the assumption of optical flow consistency. When performing object optical flow When tracking, more feature points are kept in the correct position, enabling robust tracking.

Compared with the prior art, the present invention detects color corners based on quaternions, combines gray-scale corners as feature point sets for optical flow tracking of objects, and uses quaternion-based optical flow estimation algorithms to use integral signals instead of separate In the public optical flow estimation evaluation test, the optical flow estimation error is reduced by 11.9% compared with the Lucas-Kanada algorithm. In the object optical flow tracking experiment, the feature point position error is effectively reduced.

The invention performs quantitative evaluation on the quaternion optical flow estimation algorithm on the disclosed optical flow estimation test set, and the optical flow estimation error on the Middlebury optical flow estimation standard test set is 11.9% lower than that of the Lucas-Kanada algorithm. The object tracking combined with the quaternion color corner points also improves the tracking accuracy, which proves that the present invention can realize real-time and robust tracking.

Description of drawings

Fig. 1 is a flow chart of object optical flow tracking by the method of the present invention.

Fig. 2 is a diagram showing the distinction between quaternion color corners and grayscale corners in an embodiment.

Fig. 3 is a diagram of object optical flow tracking results of the embodiment.

Detailed ways

The embodiments of the present invention are described in detail below. This embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following implementation example.

Example:

In the first step, set the range of the target to be tracked in the first frame I of the image sequence and detect the Harris corner points within this range, and retain the n _h Harris corner points whose corner degree measure is greater than the threshold γ, this embodiment Among them, γ=2000, the specific steps include:

1.1) Calculate the gradient I _x and I _y of the first frame I of the image sequence in the two-dimensional space x and y directions as the amplitude variation of the local neighborhood of the pixel point;

1.2) Calculate the corner degree measure through the magnitude correlation matrix M to obtain Harris corner points whose corner degree measure is greater than the threshold γ.

其中g(σ)为高斯平滑滤波器，σ为滤波器尺寸参数，*为卷积。该实施例中，σ＝3。The magnitude correlation matrix

Where g(σ) is the Gaussian smoothing filter, σ is the filter size parameter, and * is the convolution. In this embodiment, σ=3.

The corner degree measure Cornerness=det(M)-k trace ² (M), wherein det and trace represent the determinant and trace of the magnitude correlation matrix M, and k is an adjustment parameter. In this example, k=0.04.

The second step is to detect quaternion color corner points within the scope of the target to be tracked, and retain n _q quaternion color corner points whose color corner degree measure is greater than the threshold γ _q , in this embodiment, γ _q = 4.5e-4. Specific steps include:

2.1) Express the first frame I of the image sequence as a pure quaternion matrix I _q , perform the following quaternion filtering on I _q , and take the saturation of the filtering result to represent the color change degree C _x and y in the x direction The degree of color change C _y in the direction:

$h_x=\left[\begin{array}{ccc}R& 0& R^{*}\\ R& 0& R^{*}\\ R& 0& R^{*}\end{array}\right]\left[I_q\right]\left[\begin{array}{ccc}{R}^{*}& 0& R\\ R^{*}& 0& R\\ R^{*}& 0& R\end{array}\right],$ C _x =Saturation(H _x ),

$h_{\mathrm{the\; y}}=\left[\begin{array}{ccc}R& R& \mathrm{<figure-callout\; id="0"\; label="R"\; filenames="HDA0000054634690000021.png"\; state="\{\{state\}\}">R</figure-callout>}\\ 0& 0& 0\\ R^{*}& R^{*}& R^{*}\end{array}\right]\left[I_q\right]\left[\begin{array}{ccc}{R}^{*}& R^{*}& R^{*}\\ 0& 0& 0\\ R& R& R\end{array}\right],$ C _y =Saturation(H _y ),

Where: R＝Se ^μπ/4 ＝S{cosπ/4+sinπ/4}, μ is the unit pure quaternion

2.2) Calculate the degree measure of the color corner point through the color correlation matrix M _q to obtain the quaternion color corner points whose color corner degree measure is greater than the threshold γ _q .

其中g(σ)为高斯平滑滤波器，σ为滤波器尺寸参数，*为卷积。该实施例中，σ＝3The color correlation matrix

Where g(σ) is the Gaussian smoothing filter, σ is the filter size parameter, and * is the convolution. In this example, σ=3

The measure of the color corner degree refers to: Cornerness _q = det(M _q )/(trace(M _q )+δ), wherein: det and trace represent the determinant and trace of the color correlation matrix M, δ=2.2204e -016.

The third step is to combine n _h Harris corner points and n _q quaternion color corner points as the feature set of optical flow tracking. For each feature point in the feature set, use the quaternion-based optical flow estimation algorithm in the image The optical flow is estimated between two adjacent frames of the sequence, and the obtained optical flow value is used to update the position of the feature point in the second frame and realize the tracking. The specific steps include:

3.1) At time t, use the following steps to estimate the optical flow of each feature point in the feature point set between two adjacent frames I ^t and I ^t+1 of the image sequence, and add the position of the feature point at time t to its optical flow Flow, get the position of the feature point at time t+1, and perform optical flow estimation at time t+1. Initially t=1.

3.2) Downsample I ^t and It ⁺¹ respectively and generate p-level image pyramids I ^t,l and I ^t+1,l ,l∈[1,p] are image pyramid layer numbers. First, perform the optical flow estimation described in steps 3.3) and 3.4) at the top of the pyramid, and the obtained optical flow is enlarged to the next level of the pyramid, as the initial value of the optical flow at this level, and then perform steps 3.3) and 3.4) again Optical flow estimation. Repeat this until the bottom of the pyramid to get the final estimated optical flow. In order to further improve the estimation accuracy, the optical flow estimation described in steps 3.3) and 3.4) performs q cycles at each level of the image pyramid. In this example, p=3 and q=3.

和

给定特征集中的Harris角点或四元数颜色角点中任一在三维空间的坐标为(x，y，t)，取其空间方向上w×w大小的局部邻域；令该邻域中心(x，y)处像素和另一像素i的四元数颜色值分别为Q^(c)和Q⁽ⁱ⁾，计算Q^(c)和Q⁽ⁱ⁾的四元数内积以表示像素i和中心像素的颜色相似程度，当颜色相似程度小于阈值α则排除像素i不执行以下步骤。该实施例中，w＝9，α＝0.97。3.3) In the first layer of the image pyramid, represent I ^{t, l} and I ^{t+1, l} as pure quaternion images respectively

and

The coordinates of any of the Harris corners or quaternion color corners in the given feature set are (x, y, t) in three-dimensional space, and the local neighborhood of w×w size in the spatial direction is taken; the neighborhood The quaternion color values of the pixel at the center (x,y) and another pixel i are Q ^(c) and Q ⁽ⁱ⁾ respectively, and the quaternion inner product of Q ^(c) and Q ⁽ⁱ⁾ is calculated to represent the pixel The color similarity between i and the central pixel, when the color similarity is less than the threshold α, exclude pixel i and do not perform the following steps. In this example, w=9 and α=0.97.

The color similarity refers to L _i =<Q ⁽ⁱ⁾ , Q ^(c) >/(|Q ⁽ⁱ⁾ |·|Q ^(c) |), wherein: <·,·> represents a quaternion The inner product, |·| represents the quaternion modulus.

在空间x和y方向上的四元数梯度Q_x、Q_y。给定特征集中的Harris角点或四元数颜色角点中任一在三维空间的坐标为(x，y，t)，取其空间方向上w×w大小的局部邻域，取

中初始光流指向的相对应的邻域，计算该邻域时间方向的四元数梯度Q_t。当经过3.3)排除步骤后，取邻域内剩余的像素点并得到四元数光流等式：

其中：

是待估计的光流，数字上标表示邻域内剩余的像素点，上式的四元数矩阵形式为A_qv_q＝b_q，A_q是n×2纯四元数矩阵，b_q是n×1纯四元数向量，v_q是2×1四元数向量，

n×1四元数向量q的模定义为||q||＝sqrt(∑_n|q_n|²)，|q_n|是每个四元数元素的模。计算v_q＝(A_q ^HA_q)^-1A_q ^Hb_q，所求的光流为

其中S()表示取四元数的标量部分。3.4) Calculate the quaternion image

Quaternion gradients Q _x , Q _y in spatial x and y directions. The coordinates of any of the Harris corner points or quaternion color corner points in the given feature set in the three-dimensional space are (x, y, t), and the local neighborhood of w×w size in the space direction is taken, and the

In the corresponding neighborhood pointed by the initial optical flow, calculate the quaternion gradient Q _t in the time direction of the neighborhood. After 3.3) the exclusion step, take the remaining pixels in the neighborhood and get the quaternion optical flow equation:

in:

is the optical flow to be estimated, and the digital superscript indicates the remaining pixels in the neighborhood. The quaternion matrix form of the above formula is A _q v _q = b _q , A _q is an n×2 pure quaternion matrix, and b _q is n×1 pure quaternion vector, v _q is a 2×1 quaternion vector,

The modulus of n×1 quaternion vector q is defined as ||q||=sqrt(∑ _n |q _n | ² ), where |q _n | is the modulus of each quaternion element. Calculate v _q ＝(A _q ^H A _q ) ^-1 A _q ^H b _q , the obtained optical flow is

Among them, S() means to take the scalar part of the quaternion.

Implementation Effect

Core^TM2DuoCPU E6600@2.40GHz，内存2GB。According to the above steps, object (player) optical flow tracking is carried out on the rugby game video sequence. In this video sequence, there are blue and yellow players and white players, corresponding to objects with significant color features and gray features respectively. It can be seen from Figure 2 that Harris corner points are more scattered among white players, and the quaternion color angle Most of the points are scattered in the blue and yellow players, and the two together constitute the tracking feature set. Figure 3 is the results of Lucas-Kanade tracking (middle column in Figure 3) and optical flow tracking based on quaternion (right column in Figure 3). It can be seen that the optical flow tracking based on quaternion has the correct position of the feature points (red box Indicates a feature point in the wrong position). According to the optical flow estimation algorithm in the third step above, on the Middlebury optical flow estimation standard test set, the average angle error (AAE) and the average optical flow end point error (AEPE) are used to evaluate the optical flow estimation, and the optical flow estimation error is reduced by 11.9% %. All experiments are realized on a PC computer, the main parameters of this PC computer are: central processing unit

^CoreTM 2DuoCPU E6600@2.40GHz, memory 2GB.

Experiments show that compared with the existing object optical flow tracking method, the feature point set adopted in this embodiment takes into account the stable characteristics of significant grayscale changes and significant color changes, and is used for subsequent optical flow estimation. The optical flow estimation adopted The algorithm effectively reduces the optical flow estimation error, calculates the position of feature points more accurately during tracking, and improves the tracking accuracy.

Claims (7)

1. an object optical flow tracking method based on quaternion, is characterized in that, comprises the following steps: The first step, setting the range of the target to be tracked in the first frame of the image sequence and detecting the Harris corner points within this range, and retaining the Harris corner points whose corner degree measure is greater than the threshold; The second step is to detect quaternion color corner points within the scope of the target to be tracked, and retain the quaternion color corner points whose color corner degree measure is greater than a threshold; The third step is to combine Harris corner points and quaternion color corner points as the feature set of optical flow tracking. For each feature point in the feature set, the optical flow estimation algorithm based on quaternion is used in two adjacent frames of the image sequence. Perform optical flow estimation during the interval, and use the obtained optical flow value to update the position of the feature point in the second frame and realize tracking. 2. the object optical flow tracking method based on quaternion according to claim 1, is characterized in that, described first step comprises the following steps: 1.1) Calculate the gradient I _x and I _y of the first frame I of the image sequence in the two-dimensional space x and y directions as the amplitude variation of the local neighborhood of the pixel point; 1.2) Calculate the corner degree measure through the magnitude correlation matrix M to obtain Harris corner points whose corner degree measure is greater than the threshold γ. 3. The object optical flow tracking method based on quaternion according to claim 2, characterized in that, the magnitude correlation matrix

Wherein g (σ) is a Gaussian smoothing filter, σ is a filter size parameter, and * is a convolution; the corner degree measure Cornerness=det(M)-k trace ² (M), wherein det and trace represent amplitude The determinant and trace of the value correlation matrix M, k is the adjustment parameter. 4. the object optical flow tracking method based on quaternion according to claim 1, is characterized in that, described second step comprises the following steps: 2.1) Express the first frame I of the image sequence as a pure quaternion matrix I _q , perform the following quaternion filtering on I _q , and take the saturation of the filtering result to represent the color change degree C _x and y in the x direction The degree of color change C _y in the direction: $h_x=\left[\begin{array}{ccc}R& 0& R^{*}\\ R& 0& R^{*}\\ R& 0& R^{*}\end{array}\right]\left[I_q\right]\left[\begin{array}{ccc}{R}^{*}& 0& R\\ R^{*}& 0& R\\ R^{*}& 0& R\end{array}\right],$ C _x =Saturation(H _x ), $h_{\mathrm{the\; y}}=\left[\begin{array}{ccc}R& R& R\\ 0& 0& 0\\ R^{*}& R^{*}& R^{*}\end{array}\right]\left[I_q\right]\left[\begin{array}{ccc}{R}^{*}& R^{*}& R^{*}\\ 0& 0& 0\\ R& R& R\end{array}\right],$ C _y =Saturation(H _y ), Where: R＝Se ^μπ/4 ＝S{cosπ/4+sinπ/4}, μ is the unit pure quaternion

2.2) Calculate the degree measure of the color corner point through the color correlation matrix M _q to obtain the quaternion color corner points whose color corner degree measure is greater than the threshold γ _q . 5. The object optical flow tracking method based on quaternions according to claim 4, wherein the color correlation matrix

Where g(σ) is a Gaussian smoothing filter, σ is a filter size parameter, and * is a convolution; the color corner degree measure refers to: Cornerness _q = det(M _q )/(trace(M _q )+ δ), wherein: det and trace represent the determinant and trace of the color correlation matrix M, δ=2.2204e-016. 6. the object optical flow tracking method based on quaternion according to claim 1, is characterized in that, described 3rd step comprises the following steps: 3.1) At time t, use the following steps to estimate the optical flow of each feature point in the feature point set between two adjacent frames I ^t and I ^t+1 of the image sequence, and add the position of the feature point at time t to its optical flow Flow, get the position of the feature point at time t+1, and estimate the optical flow at time t+1, initially t=1; 3.2) Downsample I ^t and I ^t+1 respectively and generate p-layer image pyramids I ^{t, l} and I ^{t+1, l} , l∈[1, p] are the image pyramid layer numbers, firstly, perform on the top layer of the pyramid For the optical flow estimation described in steps 3.3) and 3.4), the obtained optical flow is enlarged to the next level of the pyramid, as the initial value of the optical flow at this level, and the optical flow estimation described in steps 3.3) and 3.4) is performed again, so Repeat until the bottom layer of the pyramid to obtain the final estimated optical flow. In order to further improve the estimation accuracy, the optical flow estimation described in steps 3.3) and 3.4) is performed q times at each level of the image pyramid; 3.3) In the first layer of the image pyramid, represent I ^{t, l} and I ^{t+1, l} as pure quaternion images respectively

and

The coordinates of any of the Harris corners or quaternion color corners in the given feature set are (x, y, t) in three-dimensional space, and the local neighborhood of w×w size in the spatial direction is taken; the neighborhood The quaternion color values of the pixel at the center (x,y) and another pixel i are Q ^(c) and Q ⁽ⁱ⁾ respectively, and the quaternion inner product of Q ^(c) and Q ⁽ⁱ⁾ is calculated to represent the pixel The color similarity between i and the central pixel, when the color similarity is less than the threshold α, pixel i is excluded and the following steps are not performed; 3.4) Calculate the quaternion image

The quaternion gradients Q _x , Q _y in the space x and y directions, the coordinates of any of the Harris corner points or quaternion color corner points in the given feature set in the three-dimensional space are (x, y, t), Take its local neighborhood of w×w size in the spatial direction, take

In the corresponding neighborhood where the initial optical flow points to, calculate the quaternion gradient Q _t in the time direction of the neighborhood. After 3.3) the exclusion step, take the remaining pixels in the neighborhood and obtain the quaternion optical flow equation :

in:

is the optical flow to be estimated, and the digital superscript indicates the remaining pixels in the neighborhood. The quaternion matrix form of the above formula is A _q v _q = b _q , A _q is an n×2 pure quaternion matrix, and b _q is n×1 pure quaternion vector, v _q is a 2×1 quaternion vector,

The modulus of n×1 quaternion vector q is defined as ||q||=sqrt(∑ _n |q _n | ² ), |q _n | is the modulus of each quaternion element, and the calculation v _q =(A _q ^H A _q ) ^-1 A _q ^H b _q , the required optical flow is

Among them, S() means to take the scalar part of the quaternion. 7. The object optical flow tracking method based on quaternion according to claim 6, characterized in that, said color similarity refers to L _i =<Q ⁽ⁱ⁾ , Q ^(c) >/(|Q ⁽ⁱ⁾ |·|Q ^(c) |), where: <·, ·> represent the quaternion inner product, |·| represent the quaternion module.

Images