CN109458990B - A method for measuring the pose of instruments and equipment and error compensation based on marker-free anchor point detection - Google Patents

Abstract

The present invention provides a method for measuring and compensating the pose of instruments and equipment based on marker-free anchor point detection. X coordinate, Y coordinate and confidence in the initialization stage, use the calibration board to calibrate the camera parameters, measure the physical distance of the anchor point of the instrument, all anchor points, their world coordinates, and the distance between the anchor points together constitute the anchor point network topology; The anchor point set is detected on the new image, and the rotation matrix and offset vector at this time are calculated according to the anchor point network topology, world coordinates, and camera internal parameters, and the camera coordinates of the anchor point are calculated. Then the overall pose of the instrument is the rotation matrix and the offset vector, and the camera coordinates of each anchor point are the position of each anchor point in space.

Description

A kind of instrument and equipment pose measurement and error compensation based on marker-free anchor point detection method

technical field

The invention relates to the field of machine vision, and in particular to a method for measuring the pose of instruments and equipment based on marker-free anchor point detection and an error compensation method in the field of artificial intelligence-based visual detection and measurement.

Background technique

Quantifying operation behavior is one of the bottlenecks in realizing intelligent judgment of manual operation. Visual images are a simple method to observe and record manual operations in different environments. In measurement control, the method of adding markers is usually used to assist computer tracking, but markers are invasive, and the number and location of markers must be predetermined. The present invention proposes a method for measuring and compensating the position and attitude of instruments and equipment based on the detection of marker-free anchor points.

SUMMARY OF THE INVENTION

In order to solve the above problems and defects, the present invention realizes accurate tracking of manual operations without adding any marks, and can define feature anchor points arbitrarily on the image, and realizes the pose and position measurement of any instrument and equipment according to the feature anchor point structure.

The object of the present invention is achieved through the following technical solutions:

A method for measuring and compensating the pose of an instrument and equipment based on marker-free anchor point detection, the method comprising:

Step A uses the trained DeepLabCut deep neural network model P-DLC (PretrainedDeepLabCut Network) to detect the label-free anchor points of the instrument and equipment on the image, the different anchor points are arranged in a certain order, and output each anchor point in the image. X coordinate, Y coordinate and confidence;

Step B: Use the calibration board to calibrate the camera parameters, place the equipment to make the calibration board coincide with the anchor point plane, map each X coordinate and Y coordinate in the image to the world coordinate, and measure the physical distance of all anchor point pairs, by All anchor points, world coordinates and distances between anchor points together constitute the anchor network topology;

Step C detects the anchor point set on the new image, calculates the rotation matrix and offset vector at this time according to the anchor point network topology, world coordinates, and internal parameters of the camera, and then calculates the camera coordinates to realize the pose measurement of the instrument and equipment;

Step D: Select any 3 anchor points in the image, and in the new i-th image, measure the image coordinates of the selected 3 anchor points, and calculate the depth of the instrument and equipment from the camera coordinates measured in an accurate image. The accurate measurement value in the direction is replaced by the measurement value in the depth direction of the instrument and equipment pose measurement in step C, so as to realize the instrument pose compensation.

The beneficial effects of the present invention are:

It realizes accurate tracking of manual operations without adding any marks, and can define feature anchor points arbitrarily on the image, and realizes the pose and position measurement of any instrument and equipment according to the feature anchor point structure.

Description of drawings

FIG. 1 is a flow chart of the method for measuring the pose of an instrument and equipment based on marker-free anchor point detection and an error compensation method according to the present invention.

Detailed ways

The present invention will be described in further detail below with reference to the embodiments and the accompanying drawings.

The present invention is a method for measuring and compensating the position and attitude of instruments and equipment based on marker-free anchor point detection. As shown in FIG. 1 , the method includes the following steps:

Step 10 Detect the label-free anchor points of the equipment; use the trained DeepLabCut deep neural network model (Pretrained DeepLabCut Network, P-DLC) to detect the anchor points of the equipment on the image, the different anchor points are arranged in a certain order, and Output the X coordinate, Y coordinate and confidence of each anchor point in the image.

Y坐标

及置信度ρ_{an_n}。Let a total of N anchor points, let a total of N anchor points, denoted as _{pan_1} , _{pan_2} , _{pan_3} ... _{pan_N} ; the nth anchor point is _{pan_n} (n∈[1,N]) At this time, and output The X coordinate of each anchor point in the image

Y coordinate

and confidence ρ _{an_n} .

Step 20 In the initialization phase, use the calibration plate to calibrate the camera parameters and measure the physical distance of the anchor point of the instrument; use the Zhang's calibration method to calibrate the internal parameters of the camera; then place the instrument perpendicular to the central axis of the camera and completely in the camera's field of view , make the calibration plate coincide with the plane of the anchor point, map each X coordinate and Y coordinate in the image to the world coordinate, and measure the physical distance of all pairs of anchor points. The distance between them together constitutes the anchor network topology;

其中

为相机焦距，

为图像上在X轴、Y轴上的像素分辨率，单位为像素每毫米(ppm)，

为投影中心X坐标与Y坐标；Let Zhang's calibration method be used to calibrate the internal parameters of the camera as

in

is the focal length of the camera,

is the pixel resolution on the X-axis and Y-axis on the image, in pixels per millimeter (ppm),

is the X coordinate and Y coordinate of the projection center;

则世界坐标为

Then, when the instrument is placed perpendicular to the central axis of the camera and is completely in the field of view of the camera, make the calibration plate coincide with the anchor point plane to measure the world coordinates of each feature anchor point of the instrument, for example, the nth anchor point is _{pan_n} (n ∈[1,N]) image coordinates

Then the world coordinates are

And calculate the distance between all pairs of anchor points, such as the distance d _n1n2 between the n ₁ and n ₂ anchor points is

All anchor points, the world coordinates of the anchor points, and the distance between the anchor points together constitute the anchor point network topology.

相机内参K计算出此时的旋转矩阵R、偏移向量t，之后，计算出相机坐标

Step 30: Instrument pose measurement stage; the anchor point set is detected on the new image, according to the anchor point network topology, world coordinates

The camera internal parameter K calculates the rotation matrix R and offset vector t at this time, and then calculates the camera coordinates

Then the overall pose of the instrument is the rotation matrix R and the offset vector t, and the camera coordinates of each anchor point are the position of each anchor point in space.

已知前面某一准确图像中测得的相机坐标

则可以对于点n₁、n₂有：Step 40: Instrument pose compensation stage; 3 anchor points (n ₁ , n ₂ , n ₃ ) in the image are arbitrarily selected, and in the new i-th image, the measured image coordinates of these 3 anchor points are

The camera coordinates measured in an accurate previous image are known

Then for points n ₁ , n ₂ we have:

where k is the scaling factor, let it be

, then the three points can construct a system of equations,

与3个方程，方程可解。计算得的值用于取代步骤C中在深度方向上的测量值

则实现了补偿。Since there are 3 unknowns

With 3 equations, the equation is solvable. The calculated value is used to replace the measured value in the depth direction in step C

compensation is achieved.

Although the disclosed embodiments of the present invention are as above. However, the content described is only an embodiment adopted to facilitate understanding of the present invention, and is not intended to limit the present invention. Any person skilled in the art to which the present invention belongs, without departing from the spirit and scope disclosed by the present invention, can make any modifications and changes in the form and details of the implementation, but the scope of patent protection of the present invention, The scope as defined by the appended claims shall still prevail.

Claims (5)

1. An instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection is characterized by comprising the following steps:

step A, detecting the mark-free anchor points of the instrument equipment on the image by using a trained deep neural network model P-DLC, arranging different anchor points according to a certain sequence, and outputting the X coordinate, the Y coordinate and the confidence coefficient of each anchor point in the image;

b, calibrating camera parameters by using a calibration plate, placing instruments to enable the calibration plate to coincide with the anchor point plane, mapping each X coordinate and each Y coordinate in the image into a world coordinate, measuring the physical distance of all anchor point pairs, and forming an anchor point network topology by all anchor points, the world coordinates and the distances among the anchor points;

step C, detecting an anchor point set on a new image, calculating a rotation matrix and an offset vector at the moment according to anchor point network topology, world coordinates and camera internal parameters, and then calculating camera coordinates to realize instrument and equipment pose measurement;

and D, selecting any 3 anchor points in the image, measuring the image coordinates of the selected 3 anchor points in the new ith image, calculating to obtain an accurate measurement value of the instrument in the depth direction according to the camera coordinates measured in a certain accurate image, and replacing the measurement value of the instrument pose measurement in the depth direction in the step C by the measurement value to realize instrument pose compensation.

2. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step a specifically comprises:

let a total of N anchor points, denoted as p_{an_1},p_{an_2},p_{an_3}…p_{an_N}(ii) a The nth anchor point is p_{an_n}(n∈[1,N]) And outputting the X coordinates of each anchor point in the image

Y coordinate

And confidence rho_{an_n}。

3. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step B specifically comprises:

using Zhang's scaling method to scale camera internal parameters and setting them as

Wherein

Is the focal length of the camera and is,

pixel resolution on the image, in X-axis, Y-axis, in pixels per millimeter (ppm),

the X coordinate and the Y coordinate of the projection center are obtained;

the instrument is placed in the field of view of the camera, perpendicular to the central axis of the camera, so that the calibration plate and the anchor point plane coincide with each other to measure the instrumentWorld coordinates of characteristic anchor points, e.g. p for the nth anchor point_{an_n}(n∈[1,N]) Image coordinates

Then the world coordinate is

Calculating the distance between all pairs of anchor points, e.g. n₁、n₂Distance between anchor points

Is composed of

And all anchor points, the world coordinates of the anchor points and the distances among the anchor points jointly form an anchor point network topology.

4. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step C specifically comprises: detecting an anchor point set on a new image according to the network topology and world coordinates of the anchor points

The camera internal parameter K calculates the rotation matrix R and the offset vector t at the moment, and then calculates the camera coordinates

The overall pose of the instrument is the rotation matrix R, offset vector t, where the camera coordinates of each anchor point are the position of each anchor point in space.

5. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step D specifically comprises:

in the new ith image, 3 anchor points in the image, namely the nth image, are selected₁、n₂、n₃The image coordinates of the 3 anchor points are measured as

Knowing the camera coordinates measured in a certain accurate image of the front

Then it can be for point n₁、n₂Comprises the following steps:

where k is a scaling factor, let it be

Then 3 points can construct a system of equations for solving the unknowns

Since there are 3 unknowns

With 3 equations, the equations are solvable; the calculated value is used to replace the measured value in the depth direction in step C

Compensation is achieved.

Images