WO2005122090A1

WO2005122090A1 - Method for calibrating a camera

Info

Publication number: WO2005122090A1
Application number: PCT/DE2005/001001
Authority: WO
Inventors: Dirk Ulbricht; Matthias Zobel; Markus Spitzer
Original assignee: Adc Automotive Distance Control System Gmbh
Priority date: 2004-06-05
Filing date: 2005-06-03
Publication date: 2005-12-22
Also published as: DE112005001188A5; DE102004033468A1

Abstract

The aim of the invention is to provide a method for calibrating a camera (2) mounted in a motor vehicle (1) with which angles of deviation of the camera (2) from set alignment angles are particularly easy to determine. To this end, the angles of deviation of the camera (2) from set alignment angles are approximately determined based on a camera image of a rectangular calibrating pattern (8) located outside of the vehicle.

Description

description

Procedure for calibrating a camera

The invention relates to a method for calibrating a camera which is arranged in a motor vehicle.

More and more complex control systems are used in modern motor vehicles. The development trend is towards progressive automation of the motor vehicle in order to increase safety, but also comfort for the driver and the occupants. In addition to the already known brake assist and stability systems, for example, attempts have recently been made to automate further parts of the motor vehicle control in order to support and relieve the driver and to increase safety. Automatic functions for detecting the lane, recognizing other motor vehicles or obstacles, as well as automatically detecting street signs, are conceivable and already feasible, for example. After the objects have been recorded and recognized, the driver is either given a suitable driving recommendation or the control function is implemented independently in the motor vehicle. For example, functions such as the automatic follow-up function, in which the vehicle independently maintains a predetermined distance from the preceding motor vehicle by automatic acceleration and braking, or functions for independent lane keeping, can already be implemented.

Cameras which are arranged in the motor vehicle are generally used to record the roadway and all other relevant objects. The evaluation of a camera image is usually carried out via a computer unit, likewise arranged in the motor vehicle, with which the intended functions are calculated and controlled. In order for the cameras installed in the motor vehicle to function properly and accurately, they have to be positioned and aligned exactly so that no errors are included in the evaluation of the camera image that affect control functions later. As a rule, the position of the camera in the motor vehicle, which is composed of the three spatial coordinates, and the orientation of these must be determined. Since the tolerances that occur in vehicle manufacture are comparatively large, a camera cannot simply be calibrated by installing it at a designated position, but must be individually calibrated or aligned after it has been installed in the motor vehicle. The position calculation of a camera can be carried out comparatively simply by the position of one or more reference points of the camera with respect to the

Longitudinal, transverse and vertical axes are measured or the position of the camera is determined via the known position. The measurement of the orientation of a camera, on the other hand, is more problematic because, due to excessive manufacturing tolerances of the motor vehicle components, the latter is generally rotated about the spatial axes. A rotation about the longitudinal axis is called "roll angle" or "roll angle", a rotation about the vertical axis is called "yaw angle" and a rotation about the transverse axis is called "pitch angle".

A calculation of these angular deviations from the nominal angles on the basis of a number of reference points of the camera is problematic since the dimensions of the camera are comparatively small and an exact measurement can therefore only be carried out with a very high outlay. Furthermore, it is possible that a camera also has manufacturing tolerances that cause a deviation from the target orientation, but cannot be registered by measuring from the outside.

The invention is therefore based on the object of specifying a method for calibrating a camera in a motor vehicle. ben, with which the deviation angle of a camera to target arrangement angles can be determined particularly easily.

This object is achieved according to the invention in that, in order to calibrate a camera which is arranged in a motor vehicle, the angle of deviation of the camera from the nominal arrangement angles is determined by a camera image of a rectangular calibration pattern arranged outside the motor vehicle.

The invention is based on the consideration that a camera image of the camera should best be used for a simple calibration of a camera, since in this way possible manufacturing tolerances of the camera and possible manufacturing tolerances of relevant motor vehicle components need not be determined and taken into account. Furthermore, the invention is based on the consideration that a camera image of an object whose position relative to the motor vehicle and also to the camera is known should be used to determine the alignment angle. In addition, the geometry and the extent of the recorded object should be known in order to use the embodiment as a reference. A rectangular calibration pattern is particularly useful since the camera image of a rectangular object changes when the roll, yaw and pitch angle deviates from the corresponding target angle compared to a reference target image, and these deviations change because of the 90 ° angle of the calibration pattern is comparatively easy to detect. The possibility of being able to determine the position of the camera and that of the calibration pattern with respect to the motor vehicle is a prerequisite for the function of the method. The positioning of the calibration pattern to the motor vehicle or the positioning of the camera installed in the motor vehicle to the calibration pattern can be carried out comparatively precisely by calibrating a camera in a test stand, such as a chassis adjustment stand, where the chassis geometry is very precisely can be measured. In such a test bench, the motor vehicle is usually guided in rolls and the motor vehicle position can thus be reproduced relative to the calibration pattern. A corresponding test bench can also be integrated into a production line.

A number of calibration markings are advantageously used for the calibration pattern, which are arranged in a rectangular formation and thus form the rectangular calibration pattern. If more than four markings are used to form the calibration pattern, distortions in the camera image can preferably be compensated for.

For a simple implementation of the calibration pattern, exactly four calibration marks are expediently used, one calibration mark forming a corner of the rectangular calibration pattern. The calibration marks are designed in such a way that they reflect ambient light. In a further possible embodiment, the calibration markings can have a particularly reflective surface for a particularly high contrast to the surroundings. Irradiation of the calibration markings with light is also conceivable in order to increase the contrast with the surroundings.

In order to avoid using light patterns that are similar to the calibration pattern for calibrating the camera, the calibration markings preferably actively emit light. When the light is active, the calibration marks on the camera appear brighter than the light from the backscattering surfaces of the surroundings. When the image is darkened, only the desired calibration markings can be used by the camera to calibrate the camera. For example, reflections and contours of the contours of the motor vehicle or other painted, glass and reflective surfaces can be suppressed. In order not to blind people in the vicinity of the measuring device with the active light of the calibration markings, infrared light is advantageously used. This is possible because, unlike the human eye, a camera is sensitive to infrared radiation and can detect the calibration markings emitting the light. When using infrared light, impairment of the surrounding people can be prevented, so that these tasks, such as manufacturing tasks on the motor vehicle, can be carried out unimpeded.

The determination of the deviation angle of the camera from the desired arrangement angles is expediently carried out from the deviation position of the rectangular sides and diagonals of the calibration pattern from the respective desired position in the camera image. The deviation angles are the roll, yaw and pitch angles of the camera with respect to the target orientation of the respective coordinate. In the case of a roll angle in the camera image, the rectangle sides are in each case rotated or tilted by a certain angle in comparison to nominal rectangle sides or the image edge. With a yaw or pitch angle, the center point of the calibration pattern, which results from the diagonal intersection of the rectangular pattern, is vertically and / or horizontally shifted in the camera image with respect to the image center.

The horizontal distance of the diagonal intersection of the calibration pattern from the center of the image is therefore preferably used to calculate a yaw angle of a camera. The yaw angle is advantageously approximately calculated from the arc tangent of the quotient from the horizontal distance of the diagonal intersection of the calibration pattern from the center of the image minus a transverse offset coefficient and the focus length of the camera.

The cross offset coefficient represents the displacement of the diagonal intersection of the calibration pattern from the center of the image, which is caused by a cross offset, parallel to the plane of the calibration pattern, of the camera with respect to the calibration pattern. ters occurs. If the camera is arranged directly opposite the calibration pattern, the transverse offset coefficient disappears or is zero and the horizontal deviation of the diagonal intersection from the center of the image that can be observed at a yaw angle only occurs due to the existing angular displacement of the camera.

The transverse offset coefficient is expediently calculated approximately from the product of the transverse displacement of the solder by the camera onto the level of the calibration pattern and the quotient of the focus length of the camera and the distance of the camera lens from the level of the calibration pattern.

The pitch angle is determined analogously to the calculation of the yaw angle. The vertical distance of the diagonal intersection of the calibration pattern from the center of the image is preferably used to calculate the angle.

For the approximate calculation of the pitch angle, the product of the length of the solder is advantageously reduced to the level of the

Calibration pattern approximately calculated by the camera and the tangent of the quotient from the vertical distance of the diagonal intersection of the calibration pattern from the center of the image minus a vertical offset coefficient and the focus length of the camera. The vertical coefficient, which describes the vertical displacement of the camera with respect to the calibration pattern, is expediently calculated from the product of the vertical displacement of the solder by the camera on the level of the calibration pattern and the quotient of the focus length of the camera and the distance of the camera lens from the level of the calibration pattern ,

The roll angle or roll angle with which the camera is installed can be determined by rotating the rectangular sides of the calibration pattern in the camera image. To compensate for perspective distortions when determining the roll angle, which occur when the main axis of the raera is not perpendicular to the marking plane or the camera is additionally installed with a yaw and / or pitch angle, a roll angle is advantageously approximately calculated from an average of the angle of the four rectangular sides of the calibration pattern in the camera image to the adjacent image edges.

For a camera calibration during or during the assembly of a motor vehicle, the calibration method described above is expediently used in a production line, the calibration markings being positioned to the side or above the production line. The calibration markings can be arranged rigidly on columns positioned on the side of the production line or on a portal. The motor vehicles can thus be transported or driven past the calibration markings.

To automate the calibration process, the process is preferably carried out by a control device.

The advantages of the invention are, in particular, the possibility of performing a calibration of a camera which is arranged in a motor vehicle comparatively quickly and easily. The necessary determination or approximate calculation of the alignment angle of the camera is based on the evaluation of only one camera image of calibration markings, so that the technical expenditure for calibration can be kept low. In addition, the computation algorithms required for determining the alignment angle are comparatively little computation-intensive, so that the calibration method can be implemented on a suitable control device.

Another advantage of the method is the possibility of using the method in a production line for motor vehicles, so that cameras installed in the motor vehicle can still be calibrated during series production and this is not carried out in a subsequent process. As a result, the total production time or the throughput time of a motor vehicle can be reduced.

An embodiment of the invention is explained in more detail with reference to a drawing. In it show:

1 schematically shows the arrangement of a calibration device for a camera installed in a motor vehicle,

2 shows a camera image of a calibration pattern of a camera which is installed with a yaw angle (psi), a pitch angle (theta) and a roll angle (phi), and

3 shows a camera image of a calibration pattern for determining a roll angle (phi) of a camera.

Identical parts are provided with the same reference symbols in all figures. The explanations for the abbreviations in the formulas can be found in the list of reference symbols.

In Figure 1, the section of a production line for a motor vehicle 1 or a motor vehicle series is shown schematically. In the production line, a number of motor vehicles 1 or the resulting motor vehicles 1 pass through a production line, the motor vehicle 1 being machined or components being added by tools positioned at the edge of the line. A camera 2 is installed in the motor vehicle 1 to be manufactured and is to be used for automatic control functions of the motor vehicle 1. In order to align or calibrate the camera 2 for high accuracy, it is calibrated during the manufacturing process of the motor vehicle 1 or deviation angles with respect to the target arrangement angles are determined. The section of the production line shown in FIG. 1 shows the calibration Device for a camera 2 already installed in the motor vehicle 1.

For a particularly simple calibration, the camera 2 is calibrated by evaluating a camera image of a calibration pattern 8. A rectangular calibration pattern 8 is used, which is formed by four calibration marks 6. For this purpose, the calibration markings 6 are arranged on a portal 4 of the production line, so that a motor vehicle 1 can be transported through the portal 4. For a high contrast of the calibration markings 6 to surrounding areas, the calibration markings 6 actively emit infrared light. It can be avoided that the strip workers employed on the production line are dazzled by the light, but that this can be detected by the camera 2. In order to obtain in the camera image, only the Kalibriermu- the ^'edge 8 forming calibration marks 6 and not the infrared light reflecting surrounding surfaces is darkened, the camera image accordingly.

In order to determine the alignment angles from a camera image of the calibration pattern 8, the position of the camera 2 installed in the motor vehicle with respect to the calibration markings 6 must first be determined. Since the position of the calibration markings 6 is known, only the spatial position of the camera 2 in the motor vehicle 1 and the relative position of the motor vehicle 1 relative to the calibration markings 6 need to be determined.

As a rule, the position of the camera 2 is within the

Motor vehicle known due to an intended installation position, but this can alternatively also be determined individually by means of a measurement. In order to determine the height of the motor vehicle 1 that deviates from the number of motor vehicles 1 when the motor vehicle 1 is active, the production line is measured with a chassis measurement. solution device 10 provided. The entire calibration process is carried out automatically by the control device 12.

2 shows a camera image of the calibration pattern 8, the camera 2 being installed with a yaw, pitch and roll angle. This can be seen from the asymmetrical arrangement of the calibration markings 6.

The determination of the yaw and pitch angle is calculated from the position of the diagonals of the calibration pattern 8 in the camera image that deviates from the desired position. The horizontal and vertical deviation from the center of the image is determined via the intersection of the diagonals.

The horizontal distance Δx of the diagonal intersection of the calibration pattern 8 from the center of the image, which is shown in FIG. 2, is then used for the approximate calculation of a yaw angle of the camera 2. If the camera 2 is positioned directly opposite the calibration pattern 8 or has no transverse offset with respect to the calibration pattern 8, the yaw angle psi is approximately calculated as:

wherein Δxi _mage the length of the projection onto the imager (image chip of the camera) of the horizontal distance of the point of intersection of the calibration Diagonalen- 8 from the center of the image and f is the focal length of the camera 2 is. If the camera 2 is installed with a cross offset to the calibration pattern 8, Δx must be corrected by a cross offset coefficient k _x - x / Kam marker, so that psi approximates to f A ^^ x i-mage - x K "am psi =. arctan ^l brandτ f

calculated, where x _{Kam is} the transverse offset of the solder by the ca ra to the plane of the calibration pattern 8 and d _Ma rk _e r is the distance of the camera 2 from the plane of the calibration pattern 8 2.

The determination of the pitch angle is carried out analogously to the calculation of the yaw angle, so that 2 n is approximately the same for a pitch angle of the camera

where Δyi _{mage is} the vertical distance of the diagonal intersection of the calibration pattern 8 from the center of the image, measured on the imager, and y _{came to be} the vertical displacement of the solder by the camera 2 onto the plane of the calibration pattern 8.

The roll angle phi or roll angle phi, with which the camera 2 is installed, can be determined via the rotation of the

Rectangular sides of the calibration pattern 8 are made in the camera image, as shown in FIG. 3. The angles cp _{1 #} ψ _{2 r} φ _{3 r} 9 ₄ represent the angles with which the rectangular sides of the calibration pattern 8 are rotated relative to the adjacent image edges of the camera 2. In order to compensate for perspective distortions in the determination of the roll angle phi, which occur when the main axis of the camera 2 is not perpendicular to the plane of the calibration pattern 8 or the camera 2 is installed with a yaw angle psi and / or pitch angle theta in addition to the roll angle phi a roll angle phi is approximately calculated from the average of the angles φi, ö ?, + φ, + ß>, + φ, φ ₂ , φ> ₃ and φ ₄ , so that pni = - results in 4. LIST OF REFERENCE NUMBERS

1 motor vehicle 2 camera 4 portal 6 calibration mark 8 calibration pattern

10 wheel alignment device

12 control unit

psi yaw angle of camera 2 theta pitch angle of camera 2 phi roll angle of camera 2

Φi angle of a rectangular side i of the calibration pattern 8 in the camera image to the adjacent image edge

Δx horizontal distance of the diagonal intersection of the calibration pattern (8) from the center of the image

Δy vertical distance of the diagonal intersection of the calibration pattern (8) from the center of the image f focal length of the camera k _x transverse offset coefficient k _y vertical offset coefficient x _Kara transverse displacement of the solder by the camera 2 to the level of the calibration pattern 8 y _Ka m vertical offset of the solder by the camera 2 on the level of the calibration pattern 8 d _ma rk _he distance of the camera 2 from the plane of the calibration pattern 8

Claims

claims

1. A method for calibrating a camera (2) which is arranged in a motor vehicle (1), wherein approximately a camera image of a rectangular calibration pattern (8) arranged outside the motor vehicle (1) determines the deviation angle of the camera (2) from the target arrangement angles become.

2. The method according to claim 1, wherein a rectangular calibration pattern (8) is composed of a number of, preferably four, calibration marks (6).

3. The method according to claim 2, wherein calibration markings (6) actively transmit light, in particular infrared light.

4. The method according to any one of claims 1 to 3, wherein the deviation angle of a camera (2) to the target arrangement angles from the deviation position of the rectangular sides and diagonals of the calibration pattern (8) from the respective target position in the camera image are approximately determined.

5. The method according to any one of claims 1 to 4, wherein for the approximate calculation of a yaw angle (psi) of a camera (2) the horizontal distance of the diagonal intersection of the calibration pattern (8) from the image center (Δx) is used.

6. The method according to any one of claims 1 to 5, wherein a yaw angle (psi) of a camera (2) from the arc tangent of the quotient from the horizontal distance of the diagonal intersection of the calibration pattern (8) from the center of the image (Δx) minus a cross offset coefficient (k _x ) and the focal length (f) of the camera (2) is approximately calculated.

7. The method according to claim 6, wherein the transverse offset coefficient (k _x ) from the product of the transverse offset (x _am ) of the solder by the camera (2) on the level of the calibration pattern (8) and the quotient of the focus length (f) of the camera (2) and the distance (d _mar k _e r) of the camera (2) from the plane of the calibration pattern (2) is calculated.

8. The method according to any one of claims 1 to 4, wherein for the approximate calculation of a pitch angle (theta) of a camera (2) the vertical distance of the diagonal intersection of the calibration pattern (8) from the center of the image (Δy) is used.

9. The method according to any one of claims 1 to 4 and 8, wherein a pitch angle (theta) of a camera (2) from the product of the length (a) of the solder on the level of the calibration pattern (8) by the camera (2) and the Tangent of the quotient from the vertical distance of the diagonal intersection of the calibration pattern (8) from the center of the image (Δyi _mage ) minus one

Vertical offset coefficient (k _y ) and the focus length (f) of the camera (2) is approximately calculated.

10. The method according to claim 9, wherein the vertical offset coefficient (k _y ) from the product of the vertical offset (yKam) of the solder through the camera (2) on the level of the calibration pattern (8) and the quotient of the focus length (f) of the camera (2) and the distance (d _mar k _e r) is calculated of the camera (2) from the plane of the calibration pattern (8) approximately.

11. The method according to any one of claims 1 to 4, wherein a roll angle (phi) of a camera (2) approximately from an average of the angle (φi, cp ₂ , q> ₃ , φ ₄ ) of the four rectangular sides of the calibration pattern (8) in Camera image to the neighboring image edges is calculated.

12. The method according to any one of claims 1 to 11, which is used during the manufacture of a motor vehicle (1) in a production line, wherein the calibration marks (6) are positioned laterally or above the production line.

13. The method according to any one of claims 1 to 12, which is carried out by a control device (12).