CN103400377A - Three-dimensional circular target detection and judgment method based on binocular stereo vision - Google Patents

Abstract

The invention relates to a three-dimensional circular target detection method for detecting, recognizing and judging a three-dimensional circular physical target a based on binocular stereo vision. The method comprises the steps of camera calibration: obtaining camera parameters; image preprocessing: preprocessing a first image and a second image; correction: correcting a first physical target image and a second physical target image to the same horizontal line; two-dimensional ellipse detection: obtaining the ellipse parameters of the first physical target image and the second physical target image; ellipse fitting: obtaining a continuous mathematical model of the physical targets; stereo matching: obtaining sub-pixel level matching parallax between the first physical target image and the second physical target image according to the continuous mathematical model; three-dimensional data generation: obtaining three-dimensional data of the physical targets; non-coplanarity detection: judging whether the physical targets are quasi-circular objects or not; and circle difference degree calculation: judging and obtaining the specific shapes of the physical targets.

Description

A kind of three-dimensional circular target based on binocular stereo vision detects and determination methods

Technical field

The present invention is specifically related to a kind of based on binocular stereo vision, the three-dimensional circular target detection method that three-dimensional circular physical target is carried out to the detection and Identification judgement.

Background technology

Circular target detects significant in fields such as image processing, pattern-recognition, computer visions.And at present to the detection method of circular target mostly based on the Hough transformation principle of classics, these class methods need to consume a large amount of internal memories and computing time, and need artificial default quantity of parameters, have greatly had influence on the application of this algorithm.

Also have a series of round detection methods to be based on Curvature Estimation, gradient direction, existence probability map, neural network etc., it has certain improvement at the aspects such as accuracy, real-time and algorithm robustness that detect, but these class methods all are based on the circular target detection means on two dimensional image, and in fact the circular target of physical world projects to the circle that only is only standard on two dimensional image in specific some situation through video camera.According to the pinhole camera image-forming principle, as a rule the circular target of physical world be imaged as ellipse, thereby cause having lost effectiveness based on the circular target detection method of 2-D data.

Binocular stereo vision is a kind of effective ways that obtain object true three-dimension data, because it waits advantage simply, accurately and fast, has been widely applied to the fields such as robot vision, industrial detection, biomedicine.

Summary of the invention

The invention provides a kind of three-dimensional circular target based on binocular stereo vision detects and determination methods, by binocular stereo vision, be applied to detect three-dimensional circular target, by two step judgements, just can detect simply and effectively three-dimensional circular target, solve the problem that can't correctly detect three-dimensional circular target based on the detection method of 2-D data.

In order to achieve the above object, the present invention has adopted following technical scheme:

Left and right two video cameras of the same type of a kind of employing, based on binocular stereo vision, the three-dimensional circular target detection method that three-dimensional circular physical target is carried out to the detection and Identification judgement, has such feature, comprise following steps: camera calibration: adopt the stereo calibration algorithm to carry out stereo calibration to two video cameras, obtain camera parameters; The image pre-service: the first image, the second image that two video cameras are read in respectively carry out pre-service; Correct: adopt camera parameters by the first physical target image in the first image and the second physical target image rectification in the second image to the same level line; Two-dimensional elliptic detects: adopt the two-dimensional elliptic detection algorithm to detect the first physical target image and the second physical target image, obtain elliptic parameter; Ellipse fitting: obtain the continuous mathematical model of physical target according to elliptic parameter, isolate respectively the first physical target image, the second physical target image in the first image, the second image; Stereo matching: according to continuous mathematical model, the second 2-D data of the first 2-D data of the first physical target image and the second physical target image is carried out to Stereo matching, obtain the sub-pix rank coupling parallax between the first physical target image and the second physical target image; Three-dimensional data generates: the first 2-D data and sub-pix rank are mated to parallax against projecting to three dimensions, obtain the three-dimensional data of physical target; Non-coplanarity check: calculate three-dimensional data and depart from conplane degree, judge whether physical target is the similar round shape; And the circle diversity factor is calculated: when three-dimensional data points, during at same plane, judge and obtain the concrete shape of physical target.

Further, the circular icon detection method of the three-dimensional based on binocular stereo vision of the present invention, can also have such feature: the stereo calibration algorithm is the Zhang Shi chessboard method.

In addition, the circular icon detection method of the three-dimensional based on binocular stereo vision of the present invention, can also have such feature: the two-dimensional elliptic detection algorithm is the two-dimensional elliptic detection algorithm in OpenCV.

Effect and the effect of invention

According to the three-dimensional circular target based on binocular stereo vision that the present invention relates to, detect and determination methods, it is applied to detect three-dimensional circular target by binocular stereo vision, based on binocular stereo vision, obtain simply, accurately and fast the advantage of object true three-dimension data, do not need a large amount of calculating and artificial default quantity of parameters, whether be circular target, fundamentally solved the Problem of Failure that detects the circular target method based on 2-D data if by two step judgements, just can detect simply and effectively three dimensional physical.

The accompanying drawing explanation

Fig. 1 is the three-dimensional circular target detection based on binocular stereo vision of the present embodiment and the process flow diagram flow chart of determination methods;

Fig. 2 is the ultimate principle figure that the employing binocular stereo vision of the present embodiment obtains three-dimensional data; And

Fig. 3 be the three-dimensional circular target based on binocular stereo vision of the present embodiment detect and determination methods in the schematic diagram of circle diversity factor calculating.

Embodiment

Following examples are specifically addressed the present invention by reference to the accompanying drawings.

Fig. 1 is the three-dimensional circular target detection based on binocular stereo vision of the present embodiment and the process flow diagram flow chart of determination methods.

As shown in Figure 1 based on binocular stereo vision, to three-dimensional circular physical target, carry out the process flow diagram flow chart of the three-dimensional circular target detection method of detection and Identification judgement, it has comprised following steps:

1) camera calibration: adopt the stereo calibration algorithm to carry out stereo calibration to two video cameras, obtain camera parameters, comprise intrinsic parameter and outer parameter, center as image, the focal length of video camera, distance between two video cameras, the distortion factor of video camera to image, coordinate transformation relation matrix etc.Correction, the rectification of these parameters to image, Stereo matching obtains parallax, and the generation of coordinate transform and three-dimensional data is significant, also greatly has influence on three-dimensional circular target simultaneously and detects and discrimination.

There are at present various stereo calibration algorithms available, wherein, the advantages such as classical Zhang Shi chessboard method is simple, quick, accurate with it, strong robustness are widely adopted, it is mainly by detecting the angle point on chessboard, again according to image-forming principle and the pattern distortion model of video camera, situation in the information such as the position by the angle point that detects on image, size and actual three-dimensional world is carried out computational analysis, finally solves the parameter of stereo camera shooting unit.Therefore the stereo calibration algorithm that adopts in the present embodiment is the Zhang Shi chessboard method.

The camera parameters that obtains in the present embodiment is as follows:

Left video camera:

The video camera matrix:

$M_1=\left[\begin{array}{ccc}884.17257964220369& 0& 332.43141332102141\\ 0& 884.17257964220369& 237.92003081410266\\ 0& 0& 1\end{array}\right]---\left(1\right)$

The distortion parameter matrix:

$D_1=\left[\begin{array}{ccccc}-0.179769108778110& -4.21940812476743& 0& 0& 27.8673384639677\end{array}\right]---\left(2\right)$

Right video camera:

Camera parameters:

$M_2=\left[\begin{array}{ccc}884.17257964220369& 0& 333.56227036893461\\ 0& 884.17257964220369& 237.59122547730132\\ 0& 0& 1\end{array}\right]---\left(3\right)$

The distortion parameter matrix:

$D_2=\left[\begin{array}{ccccc}-0.334514430142980& 1.58659846112079& 0& 0& -13.0812724905665\end{array}\right]---\left(4\right)$

2) image pre-service: the first image that two video cameras are read in respectively is that left figure, the second image are that right figure carries out the basic image pre-service such as filtering, denoising, enhancing.

3) correct: adopt camera parameters by the first physical target image in the first image and the second physical target image rectification in the second image to the same level line.

4) two-dimensional elliptic detects: adopt the two-dimensional elliptic detection algorithm to detect the first physical target image and the second physical target image, obtain elliptic parameter.

As can be known according to pinhole camera modeling, the circular target in the three dimensional physical world, after through video camera, projecting in two dimensional image, is generally oval, only is under given conditions circle.In two dimensional image, detecting ovally has a large amount of algorithms available at present, in the present embodiment, adopts the OpenCV(mono-computer vision Lai Faku that increases income) in the classic algorithm that detects of two-dimensional elliptic.At first this algorithm is treated detected image and is carried out rim detection, and then the profile of Edge detected image, finally carry out oval Coefficient Fitting to the profile of closure.By adjusting the oval correlation parameter that detects, this algorithm can high precision, detect expeditiously ellipse, and can obtain center point coordinate (x ₀, y ₀), the parameter such as major semi-axis a, minor semi-axis b, tilt angle alpha.When major semi-axis and minor semi-axis equate, namely during a=b, detection be standard round.

Two three dimensional physical targets are arranged in the present embodiment, be respectively the first physical target and the second physical target.

The a pair of elliptic parameter that the first physical target obtains is as follows:

Central point (x ₀, y ₀): left figure: (179,255); Right figure: (195,254).

Major semi-axis a: left figure: 74; Right figure: 75.

Minor semi-axis b: left figure: 42; Right figure: 42.

Tilt angle alpha: left figure: 83.7509 °; Right figure: 82.4927 °.

The a pair of elliptic parameter that the second physical target obtains is as follows:

Central point (x ₀, y ₀): left figure: (61,284); Right figure: (74,284).

Major semi-axis a: left figure: 41; Right figure: 41.

Minor semi-axis b: left figure: 36; Right figure: 37.

Tilt angle alpha: left figure: 166.406 °; Right figure: 175.073 °.

5) ellipse fitting: obtain the continuous mathematical model of physical target according to elliptic parameter, isolate respectively the first physical target image, the second physical target image in left figure and right figure.

After getting oval correlation parameter, according to the standard equation of ellipse

It is carried out to suitable rotation, translation.After introducing parameter θ, oval standard equation can be expressed as:

$\left\{\begin{array}{c}x=a\cos \mathrm{\θ}\\ y=b\sin \mathrm{\θ}\end{array}\right.---\left(5\right)$

After the rotation tilt angle alpha:

$\left\{\begin{array}{c}x=a\cos \mathrm{\α}\cos \mathrm{\θ}-b\sin \mathrm{\α}\sin \mathrm{\θ}\\ y=a\sin \mathrm{\α}\cos \mathrm{\θ}+b\cos \mathrm{\α}\sin \mathrm{\θ}\end{array}\right.---\left(6\right)$

Central point moves to (x ₀, y ₀) after:

$\left\{\begin{array}{c}x=a\cos \mathrm{\α}\cos \mathrm{\θ}-b\sin \mathrm{\α}\sin \mathrm{\θ}-x_0\\ y=a\sin \mathrm{\α}\cos \mathrm{\θ}+b\cos \mathrm{\α}\sin \mathrm{\θ}-y_0\end{array}\right.---\left(7\right)$

Finally, formula (7) is oval continuous mathematical expression model in image coordinate system.

6) Stereo matching: according to continuous mathematical model, the second 2-D data of the first 2-D data of the first physical target image and the second physical target image is carried out to Stereo matching, obtain the sub-pix rank coupling parallax between the first physical target image and the second physical target image.

Fig. 2 is the ultimate principle figure that the employing binocular stereo vision of the present embodiment obtains three-dimensional data.

As shown in Figure 2, L, R are respectively left and right two video cameras in binocular stereo camera, and focal length is all f, and photocentre is respectively O _lAnd O _r, and between the two the distance be T, the width of imaging is W.Suppose that P is the physical world any point, its imaging point in these two video cameras is respectively P _l, P _r, the image level coordinate of these 2 correspondences is respectively x _l, x _r.Z is the depth component in the three-dimensional coordinate of a P.According to Δ PP _lP _rWith Δ PO _lO _rThere is following relation in similar principle:

$\frac{Z}{Z-f}=\frac{\stackrel{\‾}{O_l{O}_r}}{\stackrel{\‾}{P_l{P}_r}}=\frac{T}{T-(x_l-\frac{W}{2})-(\frac{W}{2}-x_r)}---\left(8\right)$

After abbreviation:

$Z=\frac{\mathrm{fT}}{x_l-x_r}---\left(9\right)$

As can be known by formula (9), after by camera calibration, obtaining inner parameter f and external parameter T, only need the parallax x of acquisition point P in two video camera imaging planes _l-x _r, namely know the depth information between a P and observation point, then according to parallax x _l-x _r(take pixel as unit) and Z(are take rice as unit) proportionate relationship, the coordinate figure (x, y) (take pixel as unit) of recycling point image that P becomes, can obtain the three-dimensional information that a P is complete (X, Y, Z).

According to the continuous mathematical model of ellipse, can obtain the coordinate (x of left and right spirogram as middle left and right two oval sub-pixels _l, y _l), (x _r, y _r).In the Stereo matching process, by Stereo matching formula d=|x _l-x _r| obtain the parallax data of target to be detected, and be the parallax of sub-pixel.

7) three-dimensional data generates: the first 2-D data and sub-pix rank are mated to parallax against projecting to three dimensions, obtain the three-dimensional data of physical target.

Using left image as the reference image, for any one group of (x _l, y _l, d) data, can obtain the three-dimensional data of target to be detected according to formula (10).

$\left[\begin{array}{c}X\\ Y\\ Z\\ W\end{array}\right]=Q\left[\begin{array}{c}{x}_l\\ y_l\\ d\\ 1\end{array}\right]=\left[\begin{array}{cccc}1& 0& 0& -c_x\\ 0& 1& 0& -c_y\\ 0& 0& 0& f\\ 0& 0& -\frac{1}{T_x}& (c_x-c_x^{\text{'}})T_x\end{array}\right]\left[\begin{array}{c}{x}_l\\ y_l\\ d\\ 1\end{array}\right]---\left(10\right)$

Wherein Q is contrary projection matrix, c _x, c _y, f, T _x, c' _xBe the parameter of camera calibration Procedure Acquisition, be respectively: the picture centre horizontal stroke of left image, ordinate, focal length of camera, the component of two video camera relative displacements on the x direction of principal axis, the horizontal ordinate of right picture centre, X, Y, Z are respectively the coordinate figure of three-dimensional data on three dimensions, and W chooses relevant vector with coordinate system.If using left camera coordinate system as the coordinate system of three-dimensional data,

Be the three-dimensional data of target to be detected.

Contrary projection matrix in the present embodiment is as follows:

$Q=\left[\begin{array}{cccc}1& 0& 0& 332.43141332102141\\ 0& 1& 0& -237.92003081410266\\ 0& 0& 0& 884.17257964220369\\ 0& 0& 0.026645440032478523& 0\end{array}\right]---\left(11\right)$

In the three-dimensional coordinate space, the point on circle must meet institute on circle a little on a certain plane and this plane and these two conditions of some spheres intersect.Suc as formula mistake! Do not find Reference source.Shown in, wherein B, C, D are three parameters on some plane, place, (x ₀, y ₀, z ₀) being the sphere centre coordinate of crossing ball, R is the radius of a ball.

$\left\{\begin{array}{c}x+\mathrm{By}+\mathrm{Cz}+D=0\\ {(x-x_0)}^2+{(y-y_0)}^2+{(z-z_0)}^2=R^2\end{array}\right.---\left(12\right)$

Based on above two conditions, correctly distinguish what form based on the three-dimensional data after the contrary projection of binocular stereo vision principle is circle or oval, need to carry out two step checks to data, i.e. non-coplanarity check is calculated with the circle diversity factor.

8) non-coplanarity check: calculate three-dimensional data and depart from conplane degree, judge whether physical target is the similar round shape.

Non-coplanarity check verifies that namely three-dimensional data to be detected departs from the degree on a certain plane.Specific operation process is as follows: in three-dimensional data points substitution EQUATION x+By+Cz+D=0 that all are to be detected, and by the least square solution overdetermined equation, the B that gets parms, C, D value.Then by in all three-dimensional data points substitution ε=x+By+Cz+, obtain the plane deviation degree ε of any point _i.Define non-coplanar FACTOR P, as the formula (13), wherein n is the number of three-dimensional data points to be tested, and R is " radius " of point on all three-dimensional circle/ellipses to be measured, as the formula (14), and x ₀, y ₀, z ₀For three-dimensional data to be measured " " center ", namely the mean value of coordinate a little.Default non-coplanarity coefficient threshold value P _tIf, P>=P _t, three-dimensional data points is not on same plane, and target to be detected is a circle certainly not; If P<P _t, three-dimensional data points, on same plane, can show that target belongs to class circle object, is likely that three-dimensional ellipse is likely also three-dimensional circle, whether the target that detects is that three-dimensional circle needs further check.

$P=\frac{1}{R}\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{\left|{\mathrm{\&epsiv;}}_i\right|}^2}---\left(13\right)$

$R=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}\sqrt{{(x_i-x_0)}^2+{(y_i-y_0)}^2+{(z_i-z_0)}^2}---\left(14\right)$

In the present embodiment, the non-coplanar coefficient P=5.59 of the second physical target * 10 ^-15, the non-coplanar coefficient P=1.93 of the first physical target * 10 ^-14, default non-coplanarity coefficient threshold value P _t=0.001.The non-coplanar FACTOR P<P of the first physical target and the second physical target _t, judge the first physical target and the second physical target and all belong to class circle object.

9) justifying diversity factor calculates: when three-dimensional data points, during at same plane, judge and obtain the concrete shape of physical target.

The circle diversity factor namely with the difference degree of standard round.By non-coplanarity check, or the figure that data point to be detected as can be known forms may be oval for circle, and need to calculate the concrete shape that further judges identification target to be detected this moment by the circle diversity factor.

Fig. 3 be the three-dimensional circular target based on binocular stereo vision of the present embodiment detect and determination methods in the schematic diagram of circle diversity factor calculating.

As shown in Figure 3, suppose that the figure that data to be tested form is E, C be with E recently like a circle, C, E center superposition are a some O (x ₀, y ₀, z ₀), the radius of C is R, the point on E is to the distance R of central point O _eExpression, R _EiBe i three-dimensional data points (x _i, y _i, z _i) to an O (x ₀, y ₀, z ₀) distance, when same minute angle β, the arc length on C, E is respectively l, l _e.L and l _eBetween value, usually there is certain difference, by accumulation l and l _eBetween this fine difference be the difference degree between mensurable C and E, suc as formula mistake! Do not find Reference source.Shown in:

$\mathrm{non}-\mathrm{Rd}=\frac{1}{2\mathrm{\&pi;R}}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}|l_{\mathrm{ei}}-l_i|---\left(15\right)$

Default suitable threshold value Rd _tIf, non-Rd≤Rd _t, can judge that target to be detected is circle, otherwise, be oval.

In the present embodiment, the second physical target circle diversity factor coefficient non-Rd=0.4189, the first physical target circle diversity factor coefficient non-Rd=0.188, default recommendation threshold value Rd _t=0.28～0.30, can draw first physical target circle diversity factor coefficient non-Rd=0.188≤Rd _t, second physical target circle diversity factor coefficient non-Rd=0.4189>=Rd _t, can show that thus the first physical target is three-dimensional circle, the second physical target is three-dimensional oval.

The effect of embodiment and effect

The three-dimensional circular target based on binocular stereo vision that relates to according to the present embodiment detects and determination methods, it is applied to detect three-dimensional circular target by binocular stereo vision, based on binocular stereo vision, obtain simply, accurately and fast the advantage of object true three-dimension data, do not need a large amount of calculating and artificial default quantity of parameters, whether be circular target, fundamentally solved the Problem of Failure that detects the circular target method based on 2-D data if by two step judgements, just can detect simply and effectively three dimensional physical.

Claims (3)

1. one kind is adopted left and right two video cameras of the same type, and based on binocular stereo vision, the three-dimensional circular target detection method to three-dimensional circular physical target carries out the detection and Identification judgement, is characterized in that, comprises following steps:

Camera calibration: adopt the stereo calibration algorithm to carry out stereo calibration to described two video cameras, obtain camera parameters;

The image pre-service: the first image, the second image that described two video cameras are read in respectively carry out pre-service;

Correct: adopt described camera parameters by the first physical target image in described the first image and the second physical target image rectification in described the second image to the same level line;

Two-dimensional elliptic detects: adopt the two-dimensional elliptic detection algorithm to detect described the first physical target image and described the second physical target image, obtain elliptic parameter;

Ellipse fitting: obtain the continuous mathematical model of described physical target according to described elliptic parameter, isolate respectively described the first physical target image, described the second physical target image in described the first image, described the second image;

Stereo matching: according to described continuous mathematical model, the second 2-D data of the first 2-D data of the first physical target image and the second physical target image is carried out to Stereo matching, obtain the sub-pix rank coupling parallax between described the first physical target image and described the second physical target image;

Three-dimensional data generates: described the first 2-D data and described sub-pix rank are mated to parallax against projecting to three dimensions, obtain the three-dimensional data of described physical target;

Non-coplanarity check: calculate described three-dimensional data and depart from conplane degree, judge whether described physical target is the similar round shape; And

The circle diversity factor is calculated: when three-dimensional data points, during at same plane, judge and obtain the concrete shape of described physical target.

2. the circular icon detection method of the three-dimensional based on binocular stereo vision according to claim 1 is characterized in that:

Wherein, described stereo calibration algorithm is the Zhang Shi chessboard method.

3. the circular icon detection method of the three-dimensional based on binocular stereo vision according to claim 1 is characterized in that:

Wherein, described two-dimensional elliptic detection algorithm is the two-dimensional elliptic detection algorithm in OpenCV.

Images