CN101425181A - Panoramic view vision auxiliary parking system demarcating method - Google Patents

Abstract

The invention discloses a parking method for a panoramic optical aid parking system, which uses images generated by at least two wide-angle fisheye cameras installed at the periphery of an automobile for generating a virtual aerial view image at a certain height at the top of the automobile. The method comprises the following steps: model parameters of the wide-angle fisheye cameras are calculated; a certain straight line in a public view field is identified as a splice seam; the position of a world coordinate system in a non-public view field is identified; the position of a visual aerial view camera is fixed; a single-mapping transformation matrix based on the ground level is calculated; the position of the splice position is totally optimized; position parameters among all images under the visual aerial view camera are calculated. The method has the advantages of low requirement to equipment, simple process, and no dependence on relative positions among cameras and accurate posture parameters, and reduces the complexity of system integration; calculated parameters during real-time work can be provided by one-time parking; and the method has favorable peripheral panoramic aerial view effect of automobiles.

Description

A kind of scaling method of panoramic view vision auxiliary parking system

Technical field

The present invention relates to Digital Image Processing and computer vision technique, especially, relate to the scaling method of panoramic view vision auxiliary parking system.

Background technology

China is a country that traffic hazard is serious, and annual death toll and fatal rate all allow of no optimist.Reduce traffic hazard and not only will objectively more will improve the reliability of the safe driving of automobile own in subjective enhancing awareness of safety.Existing driver assistance instrument miscellaneous is exactly in order to address this problem.The automotive safety safeguards system can be divided into passive and active system substantially, and the former mainly comprises securing band, air bag etc., though can reduce the accident casualty degree, can not prevent the generation of accident.The latter then mainly utilize various kinds of sensors as, ultrasound wave, radar, infra-red heat sensor and video camera etc.They can provide traffic informations such as barrier for driver's decision-making.Simultaneously, the active safety safeguards system has also constituted the important component part of intelligent transportation system.In above-mentioned all methods, video camera has advantages such as low cost, easy care and high integration, has therefore obtained using widely.

Along with the fast development of Flame Image Process and computer vision, increasing advanced technology is applied to vehicle electric field.Traditional driver assistance based on image is only installed the reversing camera at automobile tail, can only cover the limited zone of motor vehicle environment, and the vision blind area in car both sides and the place ahead has increased the hidden danger of safe driving undoubtedly.In order to enlarge driver's field range, just necessary perception automobile is 360 ° of environment all around, and this just needs information fusion and registration between a plurality of vision sensors.Mutual alignment between a plurality of vision sensors and attitude Relation Parameters obtain from specific survey sensor, and this depends critically upon accurate position and attitude parameter with regard to making the result who merges registration.

Summary of the invention

The scaling method that the purpose of this invention is to provide a kind of panoramic view vision auxiliary parking system, it does not rely on mutual alignment and attitude relation between the video camera, can effectively obtain the splicing relation between the image.According to these relations, can produce virtual panorama general view picture, thereby enlarge driver's field range to greatest extent.

The scaling method of panoramic view vision auxiliary parking system of the present invention may further comprise the steps:

Step 1): adopt the calibration algorithm of flake wide-angle camera, calculate the inner parameter and the fish-eye percentage distortion of camera;

Step 2): take the image of motor vehicle environment zones of different with the video camera that at least two step 1) are described, have certain public domain between the said video camera;

Public domain internal labeling 2 points of video camera step 3): in step 2) are determined the splicing slit between each image.

Along ground tiling calibrating template, specify the true origin and the x of each template world coordinate system simultaneously, the y direction in the non-public domain of video camera step 4): in step 2).

Step 5): the length of measured automobiles and wide, determine the virtual position of getting a bird's eye view camera to the ground vertical projection, measure the relative position under each world coordinate system in step 4) of this position.

Step 6): the intrinsic parameters of the camera and the percentage distortion that obtain according to step 1), to step 2) image of gathering in carries out distortion to be removed and handles, and obtains image after the distortion correction.

Step 7): the undistorted figure according to step 6) obtains, calculate from the original camera position to virtual singly the reflecting property transformation matrix of getting a bird's eye view the position.Utilize this matrix, the undistorted image in the step 6) is transformed to the virtual camera visual angle image of getting a bird's eye view.

Step 8): obtain the virtual image of getting a bird's eye view the visual angle according to step 7), utilize nonlinear least square method to calculate splicing slit direction) alignment error between the calibrating template with compensation process 4.

Step 9): according to the splicing slit direction in the step 8), the virtual visual angle image of getting a bird's eye view in the step 7) is rotated and translation transformation, up to obtaining the panorama stitching image, record translation parameters at this moment.

Step 10): output calibrating parameters: the camera inner parameter, the fish eye lens percentage distortion, singly reflecting property matrix splices slit direction and translation parameters.

Among the present invention, the calibration algorithm of described employing flake wide-angle camera, the inner parameter and the fish-eye percentage distortion of calculating camera may further comprise the steps:

1) camera is represented with pin-hole model:

λu＝K(XR+T)

Wherein $K=\left[\begin{array}{ccc}{f}_u& \mathrm{\&alpha;}& u_0\\ 0& {\mathrm{<figure-callout\; id="0"\; label="f\; v\; v"\; filenames="A200810163310E00121.png"\; state="\{\{state\}\}">f</figure-callout>}}_v& v_{}{}_0\\ 0& 0& 1\end{array}\right]$ Be camera inner parameter, f _uAnd f _vBe the camera focus of representing with image row and column pixel, u ₀, v ₀Be projection centre, α is a degree of tilt; X is the three-dimensional point in the world coordinate system; R and T are rotation and the translation that world coordinates is tied to the camera coordinate system; U is the picture point coordinate of X correspondence.

2) radially model approximation is fish-eye radially and tangential distortion with high-order

u ^dist＝u+(u-u ₀)[k ₁r ²+k ₂r ⁴+k ₃r ⁶+k ₄(r ²/u+2u)]

v ^dist＝v+(v-v ₀)[k ₁r ²+k ₂r ⁴+k ₃r ⁶+k ₅(r ²/v+2v)]

$r=\sqrt{u^2+v^2}$

K wherein ₁, k ₂, k ₃Be radial distortion coefficient, k ₄, k ₅Be tangential percentage distortion.

3) take a series of images of the different attitudes of calibrating template respectively, the corresponding point of marker template on each image, and the corresponding relation between the image.

4) initial calculation camera inner parameter K _iWith fish-eye percentage distortion k _i=[k ₁, k ₂, k ₃, k ₄, k ₅], again according to the shortest principle of Euclidean distance between the corresponding observation station in the subpoint of calibrating template on image and the real image, with nonlinear least square method computation optimization K _iAnd k _i

The present invention is by simple calibrating template, carries out a series of conversion to being installed in the image that a plurality of video cameras produce around the automobile, thereby obtains being used for the various parameters of real time panoramic image mosaic.Its key step comprises: the model parameter of flake wide-angle camera is asked for; Certain bar straight line is as the splicing slit in the mark public view field; The position of world coordinate system in the non-public view field of mark; The fixing virtual position of getting a bird's eye view camera; Singly reflecting property transformation calculations based on ground level; The position global optimization in splicing slit; Virtually get a bird's eye view under the coordinate system that location parameter calculates between each image.This procedure is simple; Low for equipment requirements, do not rely on relative position and accurate attitude parameter between the video camera, thereby reduced the complicacy of the system integration; Only need to demarcate the calculating parameter that real-time working once can be provided; Panorama is got a bird's eye view effect around having good automobile.

Description of drawings

Fig. 1 is the demarcation process flow diagram of panoramic view vision auxiliary parking system

Fig. 2 is the calibrating template synoptic diagram

Fig. 3 is that the automobile that four flake video cameras are installed is got a bird's eye view synoptic diagram

Fig. 4 is that single flake camera (a) is the flake distorted image to the virtual view transformation process synoptic diagram of getting a bird's eye view among the present invention, (b) for the common fluoroscopy images after the distortion of removal flake, (c) is the general view picture.

Fig. 5 is the global optimization synoptic diagram of splicing slit direction

Embodiment

Fig. 1 has provided the demarcation process flow diagram of panoramic view vision auxiliary parking system, and it mainly may further comprise the steps:

Step 1: adopt the calibration algorithm of flake wide-angle camera, calculate the inner parameter and the fish-eye percentage distortion of camera;

1) camera is represented with pin-hole model:

λu＝K(XR+T)

Wherein $K=\left[\begin{array}{ccc}{f}_u& \mathrm{\&alpha;}& u_0\\ 0& {\mathrm{<figure-callout\; id="0"\; label="f\; v\; v"\; filenames="A200810163310E00121.png"\; state="\{\{state\}\}">f</figure-callout>}}_v& v_{}{}_0\\ 0& 0& 1\end{array}\right]$ Be camera inner parameter, f _uAnd f _vBe the camera focus of representing with image row and column pixel, u ₀, v ₀Be projection centre, α is a degree of tilt; X is the three-dimensional point in the world coordinate system; R and T are rotation and the translation that world coordinates is tied to camera coordinates system; U is the picture point coordinate of X correspondence.

2) radially model approximation is fish-eye radially and tangential distortion with high-order

u ^dist＝u+(u-u ₀)[k ₁r ²+k ₂r ⁴+k ₃r ⁶+k ₄(r ²/u+2u)]

v ^dist＝v+(v-v ₀)[k ₁r ²+k ₂r ⁴+k ₃r ⁶+k ₅(r ²/v+2v)]

$r=\sqrt{u^2+v^2}$

K wherein ₁, k ₂, k ₃Be radial distortion coefficient, k ₄, k ₅Be tangential percentage distortion.

3) take a series of images of the different attitudes of calibrating template as shown in Figure 2 respectively, (gathered 10 two field pictures in the legend, got 5 an x 5=25 point on every two field picture), three-dimensional point A on the template in the mark world coordinate system ⁱ _jAnd the corresponding point a on every two field picture ⁱ _j, the number index of putting on i and j presentation video sequence number index and each calibrating template herein.

4) employing and Zhang, Z.Y:A flexible new technique for camera calibration.IEEETransactions on Pattern Analysis and Machine Intelligence 22 (11), 1330-1334 (2000). similar methods, initial calculation camera inner parameter K _iWith fish-eye percentage distortion k _i=[k ₁, k ₂, k ₃, k ₄, k ₅].Again according to calibrating template on image subpoint and the shortest principle of Euclidean distance between the corresponding point in the real image, optimize following formula with nonlinear least square method, calculating K _iAnd k _i

$[K_i,k_i]=\arg \underset{K_i,k_i}{\min }\underset{i=1}{\overset{10}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{25}{\mathrm{\&Sigma;}}}\left|\right|a_j^i-\mathrm{\&Psi;}(A_j^i,K_i,k_i)\left|\right|$

Wherein Ψ is three-dimensional projection function to two dimension

Step 2: adopt four video camera C in this example _i, i=1 ... 4 as shown in Figure 3, and they have certain public domain 1 each other, and all video cameras can cover the zone of motor vehicle environment, and take the image I of these zoness of different _i

Step 3: according to step 1.4) gained parameter K _iAnd k _i, to the image I around the captured automobile _iRemove distortion and handle, obtain image after the distortion correction

Particularly, suppose that Fig. 4 a is the image of calibrating template in the flake camera, through after the distortion correction, obtain the common skeleton view as Fig. 4 b, curve originally becomes straight line.

Public-land zone internal labeling two points of step 4. between per two video cameras are as the zone among Fig. 31.With the splicing slit (l that determines through 2 straight line between the image _Ij, i=1 ... 4 and j=1,2 are respectively the index in camera and splicing slit).

Step 5: in the non-public-land zone of video camera along ground tiling calibrating template, as the zone among Fig. 32.Make lattice square on the template to substantially parallel with the length and the cross direction of automobile.Specify the world coordinate system initial point O of every template _iAnd x, the y direction.

Step 6: the long L of measured automobiles and wide W, determine the virtual position of getting a bird's eye view camera, as P among Fig. 3 (L/2, W/2, H) shown in, to the position P of ground vertical projection ^v(L/2, W/2,0) measures P ^vEach world coordinate system O in step 4) _iUnder relative position (X _i, Y _i).

Step 7: the undistorted figure that obtains according to step 3

Calculating is from original camera C _iThe position is to virtual singly the reflecting property transformation matrix H of getting a bird's eye view position P _iUtilize this matrix, with undistorted image Transform to the virtual camera visual angle image of getting a bird's eye view

Mainly may further comprise the steps:

1) calibrating template described in the markers step 3 is at undistorted image

On at least four some p of conllinear not _{I, s}(x _{I, s}, y _{I, s}, 1), s is the number of the point of institute's mark.

2) calculate p _{I, s}(x _{I, s}, y _{I, s}, 1) corresponding point at world coordinate system O separately _iIn three-dimensional position P _{I, s}(X _{I, s}, Y _{I, s}, 0).

3) calculate P _{I, s}(X _{I, s}, Y _{I, s}, 0) and projected position on virtual general view picture is as follows:

q _i，s＝K _i{RP _i，s ^T+[X _i，Y _i，H] ^T}

4) linear least square calculates singly reflecting property transformation matrix H _i:

$H_i=\arg \underset{A}{\min }\left|\right|p_{i,s}-{\mathrm{Aq}}_{i,s}\left|\right|$

5) undistorted image

Through single reflecting property matrix H _iTransform to the virtual visual angle image of getting a bird's eye view

Represented that to 4c singly reflecting property matrix is to the result of calibrating template conversion as Fig. 4 b.

Step 8: obtain the virtual image of getting a bird's eye view the visual angle according to step 7)

Utilize nonlinear least square method to calculate splicing slit direction, with compensation process 5) in alignment error between the calibrating template.As having represented the rotation error between the splicing slit among Fig. 5, purpose makes adjacent slits parallel as far as possible exactly.Mainly may further comprise the steps:

1) splicing slit l in the markers step 4 described public domains _{I, j}At image

On the position, calculate the slope m of line correspondence _{I, j}.

2) minimize as minor function compensating images

Corresponding anglec of rotation Θ=[θ ₁, θ ₂... θ _M]:

$\mathrm{\&Phi;}\left(\mathrm{\&Theta;}\right)=\underset{i=1}{\overset{M-1}{\mathrm{\&Sigma;}}}{\left|\right|m_{i,1}\left(\mathrm{\&Theta;}\right)-m_{i+\mathrm{1,2}}\left(\mathrm{\&Theta;}\right)\left|\right|}^2+{\left|\right|m_{M,1}\left(\mathrm{\&Theta;}\right)-m_{\mathrm{1,2}}\left(\mathrm{\&Theta;}\right)\left|\right|}^2$

Be without loss of generality, initial parameter Θ=0, M represents the number of used camera.Step 9: according to the optimization stitching direction in the step 8, to the virtual visual angle image of getting a bird's eye view of gained in the step 7)

Carry out translation transformation, up to obtaining satisfied panorama stitching image, record translation parameters T at this moment _i=[Δ x _i, Δ y _i].

Step 10 output calibrating parameters: each flake camera inner parameter matrix K _iWith the lens distortions coefficient k _i, singly the reflecting property matrix H at each visual angle _i, compensation calibrating template alignment rotation parameter θ _iWith translation parameters T _i

Arrive this, realized the demarcation of panoramic view vision auxiliary parking system.

Claims (2)

1. the scaling method of a panoramic view vision auxiliary parking system, it is characterized in that: this method comprises the steps:

Step 1): adopt the calibration algorithm of flake wide-angle camera, calculate the inner parameter and the fish-eye percentage distortion of camera;

Step 2): take the image of motor vehicle environment zones of different with the video camera that at least two step 1) are described, have certain public domain between the said video camera;

Public domain internal labeling 2 points of video camera step 3): in step 2) are determined the splicing slit between each image.

Along ground tiling calibrating template, specify the true origin and the x of each template world coordinate system simultaneously, the y direction in the non-public domain of video camera step 4): in step 2).

Step 5): the length of measured automobiles and wide, determine the virtual position of getting a bird's eye view camera to the ground vertical projection, measure the relative position under each world coordinate system in step 4) of this position.

Step 6): the intrinsic parameters of the camera and the percentage distortion that obtain according to step 1), to step 2) image of gathering in carries out distortion to be removed and handles, and obtains image after the distortion correction.

Step 7): the undistorted figure according to step 6) obtains, calculate from the original camera position to virtual singly the reflecting property transformation matrix of getting a bird's eye view the position.Utilize this matrix, the undistorted image in the step 6) is transformed to the virtual camera visual angle image of getting a bird's eye view.

Step 8): obtain the virtual image of getting a bird's eye view the visual angle according to step 7), utilize nonlinear least square method to calculate splicing slit direction) alignment error between the calibrating template with compensation process 4.

Step 9): according to the splicing slit direction in the step 8), the virtual visual angle image of getting a bird's eye view in the step 7) is rotated and translation transformation, up to obtaining the panorama stitching image, record translation parameters at this moment.

Step 10): output calibrating parameters: the camera inner parameter, the fish eye lens percentage distortion, singly reflecting property matrix splices slit direction and translation parameters.

2. the scaling method of panoramic view vision auxiliary parking system according to claim 1 is characterized in that: adopt the calibration algorithm of flake wide-angle camera, calculate the inner parameter and the fish-eye percentage distortion of camera, may further comprise the steps:

1) camera is represented with pin-hole model:

λu＝K(XR+T)

Wherein $K=\left[\begin{array}{ccc}{f}_u& \mathrm{\&alpha;}& u_0\\ 0& f_v& v_0\\ 0& 0& 1\end{array}\right]$ Be camera inner parameter, f _uAnd f _vBe the camera focus of representing with image row and column pixel, u ₀, v ₀Be projection centre, α is a degree of tilt; X is the three-dimensional point in the world coordinate system; R and T are rotation and the translation that world coordinates is tied to the camera coordinate system; U is the picture point coordinate of X correspondence.

2) radially model approximation is fish-eye radially and tangential distortion with high-order

u ^dist＝u+(u-u ₀)[k ₁r ²+k ₂r ⁴+k ₃r ⁶+k ₄(r ²/u+2u)]

v ^dist＝v+(v-v ₀)[k ₁r ²+k ₂r ⁴+k ₃r ⁶+k ₅(r ²/v+2v)]

$r=\sqrt{u^2+v^2}$

K wherein ₁, k ₂, k ₃Be radial distortion coefficient, k ₄, k ₅Be tangential percentage distortion.

3) take a series of images of the different attitudes of calibrating template respectively, the corresponding point of marker template on each image, and the corresponding relation between the image.

4) initial calculation camera inner parameter K _iWith fish-eye percentage distortion ki=[k ₁, k ₂, k ₃, k ₄, k ₅], again according to the shortest principle of Euclidean distance between the corresponding observation station in the subpoint of calibrating template on image and the real image, with nonlinear least square method computation optimization K _iAnd k _i

Images