CN103978978A - Inversion projection transformation based lane keeping method - Google Patents

Abstract

The invention relates to an inversion projection transformation based lane keeping method. The inversion projection transformation based lane keeping method mainly comprises the following steps of performing inversion projection transformation on shot perspective drawings according to intrinsic parameters of a camera and converting into vertical views; detecting lane lines and extracting lane central lines based on the vertical views and computing and setting distance between corresponding endpoints of area central lines; judging whether an automobile has lane departure or not due to compare between the distance and threshold values. The inversion projection transformation based lane keeping method has the advantages of overcoming a traditional lane keeping system disadvantage that the lane lines are not benefited to judge due to direct extraction of the vertical views, accurately judging whether the automobile has lane departure or not and achieving lane keeping.

Description

Lane Keeping Method Based on Inverse Projection Transformation

technical field

The invention relates to driving safety assisting technology, in particular to a lane keeping method based on back projection transformation.

Background technique

With the development of society, the automobile has become an indispensable means of transportation in modern society. It not only promotes the progress of human culture, but also slowly changes people's lives. The popularity of automobiles has made urban road safety a focus of attention. In recent years, the pace of cars entering households has gradually accelerated. At present, the number of civilian cars in China has exceeded 100 million, surpassing Germany and second only to the United States. At the same time, China has a large population, and the population density of major cities is very high. The conflicts between cars and cars, and people and cars are also particularly acute. There are frequent accidents on the road. The number of traffic accidents is much higher than that of developed countries and other developing countries. nation. According to statistics from the Traffic Management Bureau and the National Bureau of Statistics, the direct economic losses due to traffic accidents in our country amount to billions of yuan every year, and it is also showing an upward trend year by year. With the continuous development of computer technology, more and more image processing technology and computer vision technology have been applied to the field of automotive electronics. Intelligent driving has also become a hot topic today. The lane keeping system allows the car to keep driving without deviating from the lane when the driver is fatigued, or the car autonomously controls to keep driving within the lane line, improving driving comfort.

Cameras have the advantages of low cost, high integration and easy maintenance, and are widely used in automotive active safety systems, especially in lane keeping systems. According to a survey done in Europe, 39% of accidental traffic accidents are caused by unintentional departure from the lane. With the development of intelligent transportation, the driving of cars is becoming more and more intelligent, which can not only increase the safety of cars while driving, but also improve the comfort of drivers. When the driver's operation is unskilled or fatigue driving, the lane departure of the car will cause serious traffic accidents. The lane keeping system can not only keep the car running normally within the lane line, but also allow the car to drive autonomously when needed, improving the comfort of the driver.

Contents of the invention

The technical problem to be solved by the present invention is to provide a lane keeping method based on back-projection transformation for the defects in the prior art.

The technical solution adopted by the present invention to solve the technical problem is: a lane keeping method based on camera internal parameter inverse projection transformation, comprising the following steps:

(1) Calibrate the camera installed on it, and calculate the internal parameters of the camera; the internal parameters of the camera include the focal length fc, the principal point coordinate cc, the tilt coefficient alpha_c, and the distortion coefficient kc;

(2) Obtain the road ahead information captured by the calibrated camera, convert the color image into a grayscale image, and denoise the grayscale image;

(3) According to the principle of inverse projection transformation, using the internal parameters of the camera, the region of interest A of the captured image is restored to a top view; the region of interest A is a rectangular region of 10m×5m on the road in front of the vehicle.

(4) Utilize the hough transformation to detect and identify the lane line in the top view, and calculate the endpoint coordinates of the lane centerline;

(5) Calculate the distances B1 and B2 between the centerline of the lane line in the region of interest A and the centerline endpoints of the region of interest A in the top view, and determine whether the sum of B1 and B2 exceeds the set threshold C;

(6) If the sum of B1 and B2 exceeds the set threshold C, the steering actuator is manipulated to adjust the steering so that the sum of B1 and B2 is lower than the set threshold;

(7) If the distance does not exceed the set threshold C, enter the detection of the next frame image.

According to the above scheme, the method for converting the captured image into a top view through back projection transformation includes:

3.1) Projection transformation:

Set the coordinates of the space point P in the camera reference system with the camera as the center point

${\mathrm{<span\; class="notranslate">\; XX</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\end{array}\right]<span\; class="notranslate">\; ,</span>$

Let x=X _C /Z _C ; y=Y _C /Z _C ; r ² =x ² +y ² ;

The projective transformation works in the following way:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\end{array}\right]<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; dx</span>}<span\; class="notranslate">\; ;</span>$

where dx is the tangential distortion vector:

$\mathrm{<span\; class="notranslate">\; dx</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; xy</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; xy</span>}\end{array}\right]<span\; class="notranslate">\; ;</span>$

3.2) Convert the projected coordinates to pixel coordinates:

Let the pixel coordinate X_pixel be

$\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; pixel</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\end{array}\right]$

The conversion formula of the pixel coordinate X_pixel and the normalized coordinate vector X _d is as follows:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; KK</span>}\begin{array}{c}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\end{array}$

Where KK is the conversion matrix from the world coordinate system to the camera coordinate system

$\mathrm{<span\; class="notranslate">\; KK</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; fc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; alpha</span>}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; fc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; cc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; fc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; cc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$

The beneficial effects produced by the invention are: the method of the invention overcomes the disadvantage that directly extracting the lane lines in the perspective view in the traditional lane keeping system is not conducive to judging, and can more accurately judge whether the vehicle has deviated from the lane, and then realize lane keeping.

Description of drawings

The present invention will be further described below in conjunction with accompanying drawing and embodiment, in the accompanying drawing:

Fig. 1 is the method flowchart of the embodiment of the present invention;

Fig. 2 is the perspective view and the region of interest actually photographed by the camera in the embodiment of the present invention;

Fig. 3 is a lane information diagram obtained through inverse projection transformation in an embodiment of the present invention;

Fig. 4 is the lane line information extracted by hough transform in the embodiment of the present invention;

Fig. 5 is the information of the centerline of the lane and the centerline of the region of interest in the embodiment of the present invention;

Fig. 6 is a schematic diagram of the lane not departing from the embodiment of the present invention;

Fig. 7 is a schematic diagram of lane departure in an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

As shown in Figure 1, a lane keeping method based on inverse projection transformation includes the following steps:

1) Use the Camera Calibration Toolbox for Matlab software to calibrate the camera, and calculate the internal parameters of the camera. The internal parameters include the focal length fc (2x1 vector), the principal point coordinate cc (2x1 vector), the tilt coefficient alpha_c (scalar), and the distortion coefficient kc ( 5x1 vector).

2) Use the calibrated camera to take pictures of the road ahead, which is a perspective view, as shown in Figure 2; convert the color image into a grayscale image;

3) Process the grayscale image, maintain the basic shape characteristics, and remove noise;

According to the principle of inverse projection transformation, use the internal parameters of the camera to restore the region of interest A of the captured image, where the region of interest A is a rectangular area of 10m × 5m on the road directly in front of the vehicle; the point of the region of interest needs to be measured in the world coordinate system The actual coordinates of P#

${\mathrm{<span\; class="notranslate">\; XX</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\end{array}\right]$

Let x _n be the normalized image projection:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]$

Let r ² =x ² +y ² .

Taking into account the camera lens distortion, the new normalized point coordinate x _d is defined as follows:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\end{array}\right]<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; dx</span>}$

where dx is the tangential distortion vector:

$\mathrm{<span\; class="notranslate">\; dx</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; xy</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; kc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; xy</span>}\end{array}\right]$

Let the pixel coordinate X_pixel be

$\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; pixel</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\end{array}\right]$

The relationship between the pixel coordinate X_pixel and the normalized coordinate vector X _d is as follows:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; KK</span>}\begin{array}{c}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\end{array}$

Where KK is the conversion matrix from the world coordinate system to the camera coordinate system

$\mathrm{<span\; class="notranslate">\; KK</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; fc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; alpha</span>}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; fc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; cc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; fc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; cc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

Using the above formula, the point P in the world coordinate system can be restored to the corresponding point (x _p , y _p ) in the back-projection transformed image, and the restored image is shown in FIG. 3 .

4) As shown in Fig. 4, utilize hough transformation to carry out the detection and extraction of lane line on top view, calculate the endpoint coordinates of lane center line;

5) The centerline endpoint coordinates of the region of interest parallel to the driving direction (as shown in Figure 5), and then calculate the distances B1 and B2 between the centerline of the lane and the corresponding endpoints of the centerline of the region of interest;

5.1) Utilize the hough transform to detect the lane line in the image after the inverse projection transformation, and obtain the end point coordinates of the lane line, and calculate the end point coordinates of the lane center line by calculating the average value;

5.2) Calculate the end point coordinates of the centerline of the region of interest A parallel to the driving direction.

5.3) Calculate the distances B1 and B2 between the centerline of the lane and the corresponding endpoints of the centerline of the region of interest.

5.4) Obtain the minimum value of B1+B2 when the lane actually deviates from the experiment, as the threshold C;

5.5) By judging the relationship between the actual pixel distance B1+B2 and the threshold C, judge whether the car has a risk of deviation, and then make follow-up operations.

If B1+B2<C, the car has not deviated from the lane, and the next frame of image processing is performed, as shown in Figure 6; if B1+B2>C, it is determined that the car has deviated from the lane, and the steering actuator needs to be operated for steering operation , as shown in Figure 7.

6) If the car has a risk of deviation, then control the steering actuator to control the direction of the car, and continue to monitor the relationship between B1+B2 and the threshold C;

If the car has no risk of deviation, or the distance B1+B2 is less than the threshold C after the steering actuator operates, then enter the next frame of image processing.

It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the protection scope of the appended claims of the present invention.

Claims (2)

1. the lane keeping method against projective transformation based on intrinsic parameters of the camera, is characterized in that, comprises the following steps:

(1) to being arranged on camera calibration, and calculate camera inner parameter; Camera inner parameter comprises focal distance f c, principal point coordinate cc, inclination factor alpha_c, distortion factor kc;

(2) obtain the road ahead information that calibrated camera is taken, coloured image is converted into gray level image, and gray level image is carried out to denoising;

(3) according to contrary projective transformation principle, utilize camera inner parameter, the area-of-interest A of photographic images is reduced to birds-eye view; Described area-of-interest A is the rectangular area of vehicle dead ahead road 10m × 5m;

(4) in birds-eye view, utilize hough change detection identification lane mark, and calculate the end points coordinate of track line of centers;

(5) in birds-eye view, calculate lane mark line of centers and area-of-interest A line of centers end-point distances B1 and the B2 in area-of-interest A, and judge threshold value C B1 and B2 and that whether exceed setting;

(6) if B1 and B2 and exceed the threshold value C of setting, handle to turn to actuating unit to turn to adjust and make B1 with B2's and lower than the threshold value of setting

(7) if distance does not exceed the threshold value C of setting, the detection that enters next frame image.

2. lane keeping method according to claim 1, is characterized in that, photographic images is carried out to the method that contrary projective transformation is converted to birds-eye view and comprise:

3.1) projective transformation:

Be located at the coordinate of spatial point P in the camera reference system of putting centered by camera

${\mathrm{XX}}_c=\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right],$

Make x=X _c/ Z _c; Y=Y _c/ Z _c; r ²=x ²+ y ²;

Projective transformation is in the following ways:

$x_d=\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\end{array}\right]=(1+\mathrm{kc}\left(1\right)r^2+\mathrm{kc}\left(2\right)r^4+\mathrm{kc}\left(5\right)r^6)x_n+\mathrm{dx};$

Wherein, dx is tangential distortion vector:

$\mathrm{dx}=\left[\begin{array}{c}2\mathrm{kc}\left(3\right)\mathrm{xy}+\mathrm{kc}\left(4\right)(r^2+2x^2)\\ \mathrm{kc}\left(3\right)(r^2+2y^2)+2\mathrm{kc}\left(4\right)\mathrm{xy}\end{array}\right];$

3.2) be pixel coordinate by the coordinate transformation after projection:

If pixel coordinate X_pixel is

$X\_\mathrm{pixel}=\left[\begin{array}{c}{x}_p\\ y_p\end{array}\right];$

Pixel coordinate X_pixel and normalized coordinate vector X _dconversion formula as follows:

$\left[\begin{array}{c}{y}_p\\ y_p\\ 1\end{array}\right]=\mathrm{KK}\begin{array}{c}\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\\ 1\end{array}\right];\end{array}$

Wherein KK is that world coordinates is tied to camera Conversion Matrix of Coordinate

$\mathrm{KK}=\left[\begin{array}{ccc}\mathrm{fc}\left(1\right)& \mathrm{alpha}\_c*\mathrm{fc}\left(1\right)& \mathrm{cc}\left(1\right)\\ 0& \mathrm{fc}\left(2\right)& \mathrm{cc}\left(2\right)\\ 0& 0& 1\end{array}\right].$