[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN109458990B - Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection - Google Patents

Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection Download PDF

Info

Publication number
CN109458990B
CN109458990B CN201811325557.7A CN201811325557A CN109458990B CN 109458990 B CN109458990 B CN 109458990B CN 201811325557 A CN201811325557 A CN 201811325557A CN 109458990 B CN109458990 B CN 109458990B
Authority
CN
China
Prior art keywords
anchor point
image
instrument
anchor
coordinates
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811325557.7A
Other languages
Chinese (zh)
Other versions
CN109458990A (en
Inventor
刘桂雄
黄坚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN201811325557.7A priority Critical patent/CN109458990B/en
Publication of CN109458990A publication Critical patent/CN109458990A/en
Application granted granted Critical
Publication of CN109458990B publication Critical patent/CN109458990B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C25/00—Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Manufacturing & Machinery (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides an instrument and equipment pose measurement and error compensation method based on label-free anchor point detection, which comprises the steps of detecting anchor points of instrument and equipment on an image by using a trained deep neural network model of DeepLabCut, and outputting an X coordinate, a Y coordinate and a confidence coefficient of each anchor point in the image; in the initialization stage, a calibration plate is used for calibrating camera parameters and the physical distance of anchor points of a measuring instrument, and all anchor points, world coordinates of the anchor points and the distances among the anchor points form an anchor point network topology; and detecting an anchor point set on the new image, calculating a rotation matrix and an offset vector at the moment according to the anchor point network topology, the world coordinates and the camera internal parameters, and calculating the camera coordinates of the anchor points. The overall pose of the instrument is a rotation matrix, an offset vector, where the camera coordinates of each anchor point are the position of each anchor point in space.

Description

Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection
Technical Field
The invention relates to the field of machine vision, in particular to an instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection in the field of artificial intelligence-based vision detection and measurement.
Background
The quantitative operation behavior is one of bottlenecks in realizing intelligent judgment of manual operation, the visual image is a simple method for observing and recording the manual operation in different environments, and the workload for extracting the action characteristic quantitative operation behavior for further analysis is extremely large. In measurement control, a method of adding marks is generally adopted to assist computer tracking, but the marks are invasive, and the number and the positions of the marks must be predetermined. The invention provides an instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection, which realizes pose and position measurement of any instrument and equipment according to a characteristic anchor point structure.
Disclosure of Invention
In order to solve the problems and the defects, the invention realizes the accurate tracking of manual operation without adding any mark, can arbitrarily define the characteristic anchor point on the image, and realizes the position and pose measurement of any instrument and equipment according to the characteristic anchor point structure.
The purpose of the invention is realized by the following technical scheme:
an instrument and equipment pose measurement and error compensation method based on label-free anchor point detection comprises the following steps:
step A, detecting a label-free anchor point of instrument equipment on an image by using a trained deep neural network model P-DLC (Pre-trained deep neural network), arranging different anchor points according to a certain sequence, and outputting an X coordinate, a Y coordinate and a confidence coefficient of each anchor point in the image;
b, calibrating camera parameters by using a calibration plate, placing instruments to enable the calibration plate to coincide with the anchor point plane, mapping each X coordinate and each Y coordinate in the image into a world coordinate, measuring the physical distance of all anchor point pairs, and forming an anchor point network topology by all anchor points, the world coordinates and the distances among the anchor points;
step C, detecting an anchor point set on a new image, calculating a rotation matrix and an offset vector at the moment according to anchor point network topology, world coordinates and camera internal parameters, and then calculating camera coordinates to realize instrument and equipment pose measurement;
and D, selecting any 3 anchor points in the image, measuring the image coordinates of the selected 3 anchor points in the new ith image, calculating to obtain an accurate measurement value of the instrument in the depth direction according to the camera coordinates measured in a certain accurate image, and replacing the measurement value of the instrument pose measurement in the depth direction in the step C by the measurement value to realize instrument pose compensation.
The invention has the beneficial effects that:
the method realizes accurate tracking of manual operation without adding any mark, can define the characteristic anchor point on the image at will, and realizes the measurement of the pose and the position of any instrument and equipment according to the characteristic anchor point structure.
Drawings
FIG. 1 is a flow chart of an instrument pose measurement and error compensation method based on marker-free anchor point detection according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples and accompanying drawings.
The invention relates to an instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection, which comprises the following steps of:
step 10, detecting an anchor point without marking of instrument and equipment; and detecting anchor points of the instrument equipment on the image by using a trained DeepLabCut deep neural Network model (P-DLC), arranging different anchor points according to a certain sequence, and outputting an X coordinate, a Y coordinate and a confidence coefficient of each anchor point in the image.
Setting a total of N anchor points, and setting a total of N anchor points as pan_1,pan_2,pan_3…pan_N(ii) a The nth anchor point is pan_n(n∈[1,N]) At this time, the X coordinates of each anchor point in the image are output
Figure GDA0002767395510000021
Y coordinate
Figure GDA0002767395510000022
And confidence rhoan_n。
Step 20, in an initialization stage, calibrating camera parameters and the physical distance of an anchor point of a measuring instrument by using a calibration plate; calibrating the internal parameters of the camera by using a Zhang calibration method; then, the instrument is placed in a position which is vertical to the central axis of the camera and is completely positioned in the field of view of the camera, so that the calibration plate is superposed with the anchor point plane, the X coordinate and the Y coordinate of each image are mapped into world coordinates, the physical distances of all anchor point pairs are measured, and the anchor point network topology is formed by all anchor points, the world coordinates of the anchor points and the distances among the anchor points;
let the Zhang calibration method be used to calibrate the internal parameters of the camera into
Figure GDA0002767395510000031
Wherein
Figure GDA0002767395510000032
Is the focal length of the camera and is,
Figure GDA0002767395510000033
pixel resolution on the image, in X-axis, Y-axis, in pixels per millimeter (ppm),
Figure GDA0002767395510000034
the X coordinate and the Y coordinate of the projection center are obtained;
then the instrument is placed in the field of view of the camera, which is perpendicular to the central axis of the camera, so that the calibration plate and the anchor point plane coincide with each other to measure the world coordinates of each characteristic anchor point of the instrument, if the nth anchor point is pan_n(n∈[1,N]) Image coordinates
Figure GDA0002767395510000035
Then the world coordinate is
Figure GDA0002767395510000036
Figure GDA0002767395510000037
And calculating the distance between all pairs of anchor points, e.g. n1、n2The distance d between anchor pointsn1n2Is composed of
Figure GDA0002767395510000038
And all anchor points, the world coordinates of the anchor points and the distances among the anchor points jointly form an anchor point network topology.
Step 30, measuring the pose of the instrument; detecting an anchor point set on a new image according to the network topology and world coordinates of the anchor points
Figure GDA0002767395510000039
The camera internal parameter K calculates the rotation matrix R and the offset vector t at the moment, and then calculates the camera coordinates
Figure GDA00027673955100000310
Figure GDA00027673955100000311
The overall pose of the instrument is the rotation matrix R, offset vector t, where the camera coordinates of each anchor point are the position of each anchor point in space.
Step 40, instrument pose compensation; arbitrarily select 3 anchor points in the image (nth)1、n2、n3One), in the new ith image, the image coordinates of the 3 anchor points are measured as
Figure GDA0002767395510000041
Knowing the camera coordinates measured in a certain accurate image of the front
Figure GDA0002767395510000042
Then it can be for point n1、n2Comprises the following steps:
Figure GDA0002767395510000043
Figure GDA0002767395510000044
where k is a scaling factor, let it be
Figure GDA0002767395510000045
Then 3 points can construct a system of equations,
Figure GDA0002767395510000046
since there are 3 unknowns
Figure GDA0002767395510000047
With 3 equations, the equations can be solved. The calculated value being usedMeasurement in depth direction in the substitution step C
Figure GDA0002767395510000048
Compensation is achieved.
Although the embodiments of the present invention have been described above. However, the above description is only for the purpose of facilitating understanding of the present invention and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (5)

1. An instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection is characterized by comprising the following steps:
step A, detecting the mark-free anchor points of the instrument equipment on the image by using a trained deep neural network model P-DLC, arranging different anchor points according to a certain sequence, and outputting the X coordinate, the Y coordinate and the confidence coefficient of each anchor point in the image;
b, calibrating camera parameters by using a calibration plate, placing instruments to enable the calibration plate to coincide with the anchor point plane, mapping each X coordinate and each Y coordinate in the image into a world coordinate, measuring the physical distance of all anchor point pairs, and forming an anchor point network topology by all anchor points, the world coordinates and the distances among the anchor points;
step C, detecting an anchor point set on a new image, calculating a rotation matrix and an offset vector at the moment according to anchor point network topology, world coordinates and camera internal parameters, and then calculating camera coordinates to realize instrument and equipment pose measurement;
and D, selecting any 3 anchor points in the image, measuring the image coordinates of the selected 3 anchor points in the new ith image, calculating to obtain an accurate measurement value of the instrument in the depth direction according to the camera coordinates measured in a certain accurate image, and replacing the measurement value of the instrument pose measurement in the depth direction in the step C by the measurement value to realize instrument pose compensation.
2. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step a specifically comprises:
let a total of N anchor points, denoted as pan_1,pan_2,pan_3…pan_N(ii) a The nth anchor point is pan_n(n∈[1,N]) And outputting the X coordinates of each anchor point in the image
Figure FDA0002719843790000011
Y coordinate
Figure FDA0002719843790000012
And confidence rhoan_n。
3. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step B specifically comprises:
using Zhang's scaling method to scale camera internal parameters and setting them as
Figure FDA0002719843790000013
Wherein
Figure FDA0002719843790000014
Is the focal length of the camera and is,
Figure FDA0002719843790000015
pixel resolution on the image, in X-axis, Y-axis, in pixels per millimeter (ppm),
Figure FDA0002719843790000016
the X coordinate and the Y coordinate of the projection center are obtained;
the instrument is placed in the field of view of the camera, perpendicular to the central axis of the camera, so that the calibration plate and the anchor point plane coincide with each other to measure the instrumentWorld coordinates of characteristic anchor points, e.g. p for the nth anchor pointan_n(n∈[1,N]) Image coordinates
Figure FDA0002719843790000021
Then the world coordinate is
Figure FDA0002719843790000022
Figure FDA0002719843790000023
Calculating the distance between all pairs of anchor points, e.g. n1、n2Distance between anchor points
Figure FDA0002719843790000024
Is composed of
Figure FDA0002719843790000025
And all anchor points, the world coordinates of the anchor points and the distances among the anchor points jointly form an anchor point network topology.
4. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step C specifically comprises: detecting an anchor point set on a new image according to the network topology and world coordinates of the anchor points
Figure FDA0002719843790000026
The camera internal parameter K calculates the rotation matrix R and the offset vector t at the moment, and then calculates the camera coordinates
Figure FDA0002719843790000027
Figure FDA0002719843790000028
The overall pose of the instrument is the rotation matrix R, offset vector t, where the camera coordinates of each anchor point are the position of each anchor point in space.
5. The method for measuring pose and compensating error of instrument and equipment based on marker-free anchor point detection according to claim 1, wherein the step D specifically comprises:
in the new ith image, 3 anchor points in the image, namely the nth image, are selected1、n2、n3The image coordinates of the 3 anchor points are measured as
Figure FDA0002719843790000029
Knowing the camera coordinates measured in a certain accurate image of the front
Figure FDA00027198437900000210
Then it can be for point n1、n2Comprises the following steps:
Figure FDA00027198437900000211
Figure FDA0002719843790000031
where k is a scaling factor, let it be
Figure FDA0002719843790000032
Then 3 points can construct a system of equations for solving the unknowns
Figure FDA0002719843790000033
Figure FDA0002719843790000034
Since there are 3 unknowns
Figure FDA0002719843790000035
With 3 equations, the equations are solvable; the calculated value is used to replace the measured value in the depth direction in step C
Figure FDA0002719843790000036
Compensation is achieved.
CN201811325557.7A 2018-11-08 2018-11-08 Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection Active CN109458990B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811325557.7A CN109458990B (en) 2018-11-08 2018-11-08 Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811325557.7A CN109458990B (en) 2018-11-08 2018-11-08 Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection

Publications (2)

Publication Number Publication Date
CN109458990A CN109458990A (en) 2019-03-12
CN109458990B true CN109458990B (en) 2020-12-22

Family

ID=65609767

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811325557.7A Active CN109458990B (en) 2018-11-08 2018-11-08 Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection

Country Status (1)

Country Link
CN (1) CN109458990B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109887017B (en) * 2019-03-25 2021-09-03 北京奇艺世纪科技有限公司 Similarity calculation method and device
CN112611361A (en) * 2020-12-08 2021-04-06 华南理工大学 Method for measuring installation error of camera of airborne surveying and mapping pod of unmanned aerial vehicle
CN112597895B (en) * 2020-12-22 2024-04-26 阿波罗智联(北京)科技有限公司 Confidence determining method based on offset detection, road side equipment and cloud control platform
CN114332464A (en) * 2021-12-31 2022-04-12 中国电力科学研究院有限公司 Security risk behavior detection method, system, device and medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101839692A (en) * 2010-05-27 2010-09-22 西安交通大学 Method for measuring three-dimensional position and stance of object with single camera
CN103616016A (en) * 2013-11-29 2014-03-05 大连理工大学 Visual position-pose measurement method based on point-line combination characteristics
CN104677340A (en) * 2013-11-30 2015-06-03 中国科学院沈阳自动化研究所 Point character based monocular vision pose measurement method
CN106679634A (en) * 2016-06-20 2017-05-17 山东航天电子技术研究所 Spatial non-cooperative target pose measurement method based on stereoscopic vision
CN107917700A (en) * 2017-12-06 2018-04-17 天津大学 The 3 d pose angle measuring method of target by a small margin based on deep learning
WO2018130605A1 (en) * 2017-01-16 2018-07-19 Connaught Electronics Ltd. Method for calibrating a camera for a motor vehicle considering a calibration error, camera as well as motor vehicle

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101839692A (en) * 2010-05-27 2010-09-22 西安交通大学 Method for measuring three-dimensional position and stance of object with single camera
CN103616016A (en) * 2013-11-29 2014-03-05 大连理工大学 Visual position-pose measurement method based on point-line combination characteristics
CN104677340A (en) * 2013-11-30 2015-06-03 中国科学院沈阳自动化研究所 Point character based monocular vision pose measurement method
CN106679634A (en) * 2016-06-20 2017-05-17 山东航天电子技术研究所 Spatial non-cooperative target pose measurement method based on stereoscopic vision
WO2018130605A1 (en) * 2017-01-16 2018-07-19 Connaught Electronics Ltd. Method for calibrating a camera for a motor vehicle considering a calibration error, camera as well as motor vehicle
CN107917700A (en) * 2017-12-06 2018-04-17 天津大学 The 3 d pose angle measuring method of target by a small margin based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于机器视觉的工业机器人位姿误差的标定与补偿方法研究;黄晨华;《华南理工大学博士论文》;20141014;全文 *

Also Published As

Publication number Publication date
CN109458990A (en) 2019-03-12

Similar Documents

Publication Publication Date Title
CN109458990B (en) Instrument and equipment pose measurement and error compensation method based on mark-free anchor point detection
CN108510551B (en) Method and system for calibrating camera parameters under long-distance large-field-of-view condition
CN105486235B (en) A kind of goal-griven metric method in ball machine video pictures
CN109099883A (en) The big visual field machine vision metrology of high-precision and caliberating device and method
CN109887041B (en) Method for controlling position and posture of shooting center of digital camera by mechanical arm
CN110223355B (en) Feature mark point matching method based on dual epipolar constraint
CN106168461B (en) A kind of novel telemeasurement calibration instrument
CN113569647B (en) AIS-based ship high-precision coordinate mapping method
CN102800096A (en) Robustness estimation algorithm of camera parameter
CN106352806A (en) High-precision calibration method for stereoscopic vision three-dimensional digital image correlation measurement
CN112013921B (en) Method, device and system for acquiring water level information based on water level gauge measurement image
CN104219512A (en) Method for describing color gamut boundary of display device
CN108627104A (en) A kind of dot laser measurement method of parts height dimension
CN111189403A (en) Tunnel deformation monitoring method and device and computer readable storage medium
CN103559484A (en) Fast recognition method for measuring instrument scale lines
CN112634373A (en) Zero-expansion ceramic calibration plate-based dynamic correction method for vision measurement system
CN111442845A (en) Infrared temperature measurement method and device based on distance compensation and computer storage medium
CN108986765B (en) Display screen white point calibration method covering visual angle color cast
CN115187612A (en) Plane area measuring method, device and system based on machine vision
CN113240635B (en) Structural object detection image quality testing method with crack resolution as reference
CN114067307A (en) Transformer substation instrument scale calibration method and system
CN114222101A (en) White balance adjusting method and device and electronic equipment
CN111207683B (en) Tunnel deformation monitoring method and device and computer readable storage medium
CN105809685B (en) A kind of Camera Calibration Methods based on single width concentric circle diagram picture
CN118258570A (en) Mapping relation determining method, deflection measuring method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant