JP5606770B2 - Camera calibration device - Google Patents

Description

  The present invention relates to a camera calibration device, and more particularly to a camera calibration device that calibrates the imaging direction of a camera.

  In recent years, in the field of in-vehicle cameras, in order to perform monitoring and object detection by a camera, a camera calibration device that calibrates the imaging direction of the camera by obtaining the direction of the imaging object viewed from the camera has been developed.

  For example, when a camera is mounted on a vehicle and vehicle driving guidelines are drawn on a captured image, camera calibration is required to calibrate the imaging direction of the camera in order to accurately determine the positional relationship between the road surface and the camera. .

  Such camera calibration can be performed in the factory when the camera is installed in the factory of the vehicle manufacturer. However, when the user installs the camera in the vehicle, the user must There is a need to do.

  For such camera calibration, a technique for performing camera calibration by calculating the disappearance line from the parallel line on the road surface in the imaging direction of the camera is known (for example, see Patent Document 1).

JP 2003-329411 A

  However, in the conventional technology as described above, since the camera cannot be calibrated unless parallel lines can be captured, there is a problem that the imaging direction of the camera cannot be calibrated depending on the surrounding scenery.

  The present invention has been made to solve the conventional problems, and an object of the present invention is to provide a camera calibration apparatus that can calibrate the imaging direction of the camera regardless of the surrounding scenery.

Camera calibration device of the present invention, a camera having an image sensor, an acceleration sensor in the image sensor and the known positional relationship, at rest the inclination detected by the acceleration sensor when the image sensor is stationary An acceleration vector, and the inclination detected by the acceleration sensor when the image sensor is accelerated or decelerated while going straight is used as an acceleration vector during straight running , and the camera is based on the stationary acceleration vector and the linear acceleration vector. A camera calibration unit that calculates the imaging direction of the camera , wherein the camera calibration unit has an optical system center point of the camera as an origin, an optical axis as a Z axis, a scanning direction as an X axis, and a sub scanning direction as a Y axis. Conversion equation parameters for converting the camera coordinate system into a vertical camera coordinate system in which the Z-axis is rotated in the vertical direction By calculating, it has a configuration that calculates the imaging direction of the camera.

  With this configuration, the camera calibration device of the present invention is based on the inclination detected by the acceleration sensor when the image sensor is stationary and the inclination detected by the acceleration sensor when the image sensor is moving. Since the imaging direction of the camera is calculated, the imaging direction of the camera can be calibrated regardless of the surrounding scenery.

  The camera calibration unit detects the vertical direction using the acceleration sensor when the image sensor is stationary.

  The present invention can provide a camera calibration apparatus having an effect that the imaging direction of the camera can be calibrated regardless of the surrounding scenery.

1 is a configuration diagram of a camera constituting a camera calibration apparatus according to an embodiment of the present invention. Mounting diagram of a camera constituting a camera calibration apparatus according to an embodiment of the present invention The conceptual diagram which shows the relationship between the camera calibration apparatus which concerns on one embodiment of this invention, the vehicle which mounted the camera calibration apparatus, and a world coordinate system The conceptual diagram which shows the relationship between the vertical camera coordinate system which concerns on one embodiment of this invention, the acceleration vector at the time of straight advance, and the acceleration vector at the time of stationary The conceptual diagram which shows the relationship between the camera coordinate system which concerns on one embodiment of this invention, and a vertical camera coordinate system

  Embodiments of the present invention will be described below with reference to the drawings.

  As shown in FIG. 1, the camera calibration apparatus as an embodiment of the present invention is configured by a camera 120. The camera 120 includes a lens 130, an image sensor 140, an acceleration sensor 150, and a camera calibration unit 160.

  In FIG. 1, the acceleration sensor 150 is configured by a three-dimensional acceleration sensor capable of measuring three-dimensional acceleration, and is housed in the camera 120 in a three-dimensional known positional relationship with respect to the image sensor 140.

  The camera calibration unit 160 includes a microcomputer having a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like.

  The ROM stores a program for causing the microcomputer to function as the camera calibration unit 160. That is, when the CPU executes a program stored in the ROM using the RAM as a work area, the microcomputer functions as the camera calibration unit 160.

  The camera calibration unit 160 is based on the inclination detected by the acceleration sensor 150 when the vehicle 100 is stationary and the inclination detected by the acceleration sensor 150 when the vehicle 100 is accelerating / decelerating while moving straight. The imaging direction of the camera 120 is calculated.

  Here, the camera calibration unit 160 determines whether the vehicle 100 is stationary based on the detection results of the snake angle sensor and the vehicle speed sensor provided outside the camera 120, for example, the vehicle 100 main body, and the vehicle 100. It is determined whether or not the vehicle is accelerating / decelerating while traveling straight.

  Note that, when the vehicle 100 is moving, the camera calibration unit 160 assumes that the straight traveling rate is the highest in the traveling direction of the vehicle 100 and determines the direction with the highest proportion as the straight traveling direction. It may be.

  In FIG. 1, the camera calibration unit 160 is illustrated as being provided inside the camera 120, but may be provided outside the camera 120.

  As shown in FIG. 2, the camera 120 is provided in the vehicle 100 so as to capture the rear of the vehicle 100. As shown in FIG. 3, the road surface immediately below the camera 120 is set as the origin of the world coordinate system 180. The world coordinate system 180 is expressed as Wy with the longitudinal direction of the vehicle 100, that is, the straight direction of the vehicle 100 as the Y axis, Wx with the width direction of the vehicle 100 as the X axis, and the vertical direction of the vehicle 100 as Z. The axis is expressed as Wz.

  As shown in FIG. 4, the camera coordinate system 170 has the center of the lens 130 as the optical system center point of the camera 120 as the origin, the optical axis direction as the Z axis, the scanning direction as the X axis, and the sub scanning direction as the Y axis. The X, Y, and Z axes are indicated as Cx, Cy, and Cz.

  In addition, the inclination detected by the acceleration sensor 150 when the vehicle 100 is stationary is defined as a stationary acceleration vector 270, and the inclination detected by the acceleration sensor 150 when the vehicle 100 is accelerating / decelerating while moving straight is straight. The time acceleration vector 280 is assumed.

  Hereinafter, the operation of the camera calibration unit 160 will be described with reference to FIGS. 4 and 5.

  As shown in FIG. 5, the vertical camera coordinate system 230 is a coordinate system rotated so that the Z-axis Cz of the camera coordinate system 170 coincides with the direction of gravity. In the vertical camera coordinate system 230, the X, Y, and Z axes are indicated as Vx, Vy, and Vz.

  The relationship between the camera coordinate system 170, the stationary acceleration vector 270, and the linear acceleration vector 280 is as shown in FIG. 4, and the acceleration sensor 150 when the vehicle 100 is stationary detects the gravitational acceleration. . That is, the camera calibration unit 160 can detect the direction of gravity based on the detection result of the acceleration sensor 150 when the vehicle 100 is stationary.

  Here, a vector obtained by normalizing the three-axis values of the acceleration sensor 150 when the vehicle 100 is stationary to a unit vector is defined as a stationary acceleration vector 270, and the Z-axis projection vector for the vertical camera coordinate system 230 is expressed as [Expression 1]. ] Represented by the vector shown in FIG.

  In addition, the cross product of the Z-axis projection vector and the acceleration vector 280 when traveling straight is taken, and the unit vector is expressed as the vector shown in [Equation 2] as the X-axis projection vector for the vertical camera coordinate system 230.

  Also, a product obtained by taking the outer product of the Z-axis projection vector and the X-axis projection vector is represented by the vector shown in [Equation 3] as the Y-axis projection vector for the vertical camera coordinate system 230.

  Summarizing such projection vectors, a conversion equation for converting from the camera coordinate system 170 to the vertical camera coordinate system 230 can be expressed as shown in [Equation 4].

  Thus, the camera calibration unit 160 can convert the camera coordinate system 170 to the vertical camera coordinate system 230, and can calibrate the imaging direction of the camera 120.

  As described above, the camera 120 constituting the camera calibration apparatus according to the embodiment of the present invention moves the image sensor 140 and the tilt detected by the acceleration sensor 150 when the image sensor 140 is stationary. Since the imaging direction of the camera 120 is calculated based on the inclination detected by the acceleration sensor 150 when the camera is moving, the imaging direction of the camera 120 can be calibrated regardless of the surrounding scenery.

  As described above, the camera calibration device according to the present invention has an effect of being able to calibrate the imaging direction of the camera regardless of the surrounding scenery, for example, a camera calibration device that calibrates the imaging direction of the in-vehicle camera, etc. Useful as.

DESCRIPTION OF SYMBOLS 100 Vehicle 120 Camera 130 Lens 140 Image sensor 150 Acceleration sensor 160 Camera calibration part

Claims (2)

A camera having an image sensor;
An acceleration sensor in a known positional relationship with the image sensor;
The inclination detected by the acceleration sensor when the image sensor is stationary is defined as a stationary acceleration vector, and the inclination detected by the acceleration sensor when the image sensor is accelerated or decelerated while traveling straight is A camera calibration unit that calculates an imaging direction of the camera based on the acceleration vector during stationary and the acceleration vector during straight travel ,
The camera calibration unit is configured so that the Z axis is a vertical direction from a camera coordinate system in which an optical system center point of the camera is an origin, an optical axis is a Z axis, a scanning direction is an X axis, and a sub scanning direction is a Y axis. A camera calibration apparatus for calculating an imaging direction of the camera by calculating a parameter of a conversion formula for conversion into a vertical camera coordinate system rotated in the vertical direction . The camera calibration apparatus according to claim 1, wherein the camera calibration unit detects the vertical direction using the acceleration sensor when the image sensor is stationary .

Images