JP5811327B2 - Camera calibration device - Google Patents

Description

  The present invention relates to the technical field of camera calibration.

A technology for assisting driving has been developed by imaging the surrounding environment of the vehicle with a camera, performing necessary image processing, and providing various information to the driver. Driving assistance includes display of a rear image during parking, automatic steering, detection of a vehicle approaching the host vehicle, warning of lane departure, and the like. Image processing includes coordinate system conversion, white line recognition, pattern matching, and the like. The conversion of the coordinate system includes three-dimensionalization of a two-dimensional image, conversion of an overhead image corresponding to a plan view to a coordinate system, conversion to a coordinate system suitable for distance calculation, and the like.
In particular, when an image captured by a camera is subjected to a bird's-eye view process and presented to the driver, the environment around the vehicle can be easily recognized. In order to convert such a coordinate system, it is necessary to measure the position of the camera image with respect to the vehicle clearly and accurately.

For example, in the case of a back monitor, it is desirable that the vertical direction of the image coincides with the traveling direction of the vehicle, and the center line facing the vertical direction in the image coincides with the central axis of the vehicle. In order to perform such image conversion processing, camera calibration is required. In this calibration, internal parameters unique to the camera including lens performance and external parameters indicating the installation position and orientation of the camera in the world coordinate system are calculated. A system using a camera corrects distortion caused by the lens of the camera using an internal parameter, and corrects an installation error (installation position and orientation error) of the camera using an external parameter.
In camera calibration, for example, a pattern with a known size is laid on the road surface, and this is imaged, and the position and size of the pattern on the image correspond to the actual position and size in the world coordinate system. Thus, a technique for calculating the coordinates of the imaging surface of the camera in the world coordinate system is known (for example, Non-Patent Document 1).
The external parameters are a parameter group that gives a correspondence relationship between the world coordinate system and the coordinate system of the camera in the position and orientation (rotation angle) in the ideal mounting in design. The position and angle of assembly of the camera 10 may vary, and the position may be shifted during use. Then, the external parameter of the camera becomes a value different from the external parameter based on the ideal installation position. In this case, the correspondence between the camera coordinate system and the world coordinate system will be inaccurate unless the mounting error of the camera 10 is reflected in an external parameter or a correction value or the like corresponding to the mounting error is applied.
If the correspondence relationship of the coordinate system becomes inaccurate, various image processing cannot exhibit its performance. Then, the display on the driver becomes inappropriate and the display of the image becomes unsightly.

In Patent Document 1, in order to perform calibration of the camera 10 with high accuracy in a vehicle assembly factory, repair shop, etc., a target drawn on the ground is imaged after the back camera is installed on the vehicle ( (Paragraphs 0044 and 0048), and by prompting a manual operation by visual observation while displaying the imaged target on the display (paragraphs 0052 to 0065, FIGS. 5 and 6), the external parameters of the camera are changed to the actual camera parameters. A method for correcting the value to a value corresponding to the mounting position and angle is disclosed (paragraph 0065).
Patent Document 1 discloses that the vehicle position at the time of measurement is stopped at a “position having a predetermined relative relationship” with respect to the target (paragraph 0047). Further, in this document 1, as a target, the vehicle body direction of the vehicle parked at a predetermined position is opposed to each other in the vehicle width direction (paragraph 0045, FIG. 4), and the laterally symmetrical positional relationship on the display (paragraph 0049, FIG. 5). ) Is disclosed.

  Patent Document 2 discloses a technique for correcting an influence of a camera module mounting error and generating an overhead image. Specifically, when the vehicle is stopped at a predetermined position on the road surface on which the white line is drawn at the time of vehicle shipment inspection, etc., and the white line is imaged from a plurality of cameras attached to the own vehicle, and there is no installation error By comparing with the template image (paragraph 0031, FIG. 6), the mounting error is corrected. This suppresses inconvenience that the white line is shifted at the joint of each image by a plurality of cameras (paragraph 0062, FIG. 10).

  Japanese Patent Application Laid-Open No. 2004-228561 calculates the camera installation position and installation position errors using the contour of the rear bumper as an adjustment pattern for the purpose of enabling calibration of a camera that captures the rear of the vehicle while traveling. A technique (paragraphs 0025, 0042) is disclosed.

  In Patent Document 4, a plurality of patterns are prepared on the road surface in order to execute calibration of a plurality of cameras without requiring large-scale environmental facilities, and the vehicle is stopped at an approximate position. The pattern is imaged (paragraphs 0011 and 0012), each image is converted into a bird's-eye view image (paragraph 0035), and the roll angle and depression angle of the camera group are obtained by the least square method (paragraph 0037, FIG. 3 (FIG. 5)), a method for obtaining a parameter is disclosed. In this example, when a plurality of images are combined (paragraph 0047), the angle formed by each camera is known (paragraph 0060) and the camera direction with respect to the vehicle is calculated (paragraph 0061, FIGS. 11 and 12).

JP 2005-77107 A JP 2006-277738 Japanese Unexamined Patent Publication No. 2007-256030 JP 2009-288152 A

R. Y. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses.” IEEE Journal of Robotics and Automation, 3 (4): 323-344, Aug. 1987

In the methods described in Patent Documents 1 and 2, an error in the stop position and direction of the vehicle (stop error) cannot be reflected in an external parameter by calibration. Therefore, in order to measure the external parameter with the target accuracy, in the methods described in Patent Documents 1 and 2, the position and angle of the calibration pattern must be precisely matched to the position of the vehicle. Then, the large-scale installation which can perform this operation | work, the skilled engineer, and many work time will become essential.
Furthermore, there is a possibility that the mounting position is shifted due to the traveling of the vehicle. If a large-scale facility is required for the calibration, the calibration cannot be performed in a relatively small facility in the vicinity where the vehicle can easily move.
In other words, the methods described in Patent Documents 1 and 2 have the disadvantage that the calibration cannot be easily performed in the vicinity of the vehicle.
Patent Documents 2 and 4 disclose a method using a bird's-eye view image, but this is a correction method from an ideal attachment parameter, and the correction is only a shift or rotation of a processed image, and calibration is performed. Large-scale equipment will be required to implement the project.
In the method described in Patent Document 3, calibration is executed based on the portion of the image where the vehicle is imaged, so that calibration is performed on areas important for image processing and providing information to the driver, such as from the center to the top of the image. Will be excluded. That is, in the method described in Patent Document 3, calibration is performed using the lower peripheral area of the camera image. When a camera with a large distortion, which is common for cameras for monitoring the rear of the vehicle, is used, A slight deviation of the distortion parameter affects the calibration accuracy.
Further, in the method described in Patent Document 4, while the vehicle is stopped at an approximate position (paragraph 0029), since the relationship between the stop position and the installation pattern is not measured, the stopping error of the vehicle with respect to the installation pattern is It will be mixed into the installation error of the camera. That is, even if the processing shown in each step of the flowchart shown in FIG. 3 of Patent Document 4 is performed, whether the stop position of the vehicle is inclined with respect to the coordinate system of the installation pattern, or the camera installed in the vehicle is It is impossible to distinguish whether the vehicle is tilted, and the vehicle stop error is included in the camera installation error. In particular, there is no disclosure of a method for measuring and calculating θyawc2 shown in FIG. 11 of this document.
And this patent document 4 does not disclose any method for reflecting a camera installation error in an external parameter by excluding a vehicle stop error for a single camera.

[Problem 1] As described above, the above-described conventional example has a disadvantage that calibration with high accuracy cannot be easily performed.
[Problem 2] Furthermore, the conventional example described above has a disadvantage that it is not possible to calibrate the installation error of the camera by excluding the stop error of the vehicle with a single camera.

  [Object of the invention] An object of the present invention is to carry out calibration with simple equipment capable of eliminating the influence of a stop error of a vehicle.

  [Points of interest] If the inventor of the present invention can calibrate with the pattern installed on the road surface while removing the stop error of the vehicle, the work load at the time of calibration can be reduced, and the accuracy can be reduced with small-scale equipment. We paid attention to the fact that high calibration is possible. And the idea that the above problem could be solved by devising a method to reflect the relationship between the coordinate system of the pattern installed on the road surface and the coordinate system where the vehicle is installed in the calibration was reached. .

[Problem Solving Means 1] The present invention provides an image for performing calibration of a camera based on a measurement region where a host vehicle having a camera that captures an environment outside the vehicle stops in a predetermined direction and a measurement image captured by the camera. And a processing unit.
The measurement area includes a line intersecting two points corresponding to the shape of the own vehicle, and at least two reference lines arranged in the measurement area within the imaging range of the camera, A calibration pattern having a predetermined pattern shape between the two reference lines.
Furthermore, the present invention provides an imaging process in which the image processing unit generates the measurement image by capturing a measurement region in which the calibration pattern and the reference line are arranged with the camera, and the two A stop error calculation process for calculating a coordinate relationship between the stop position of the host vehicle and the pattern shape based on a reference line as a stop error, and a mounting posture of the camera based on the pattern shape in a state where the stop error is canceled And an external parameter calculation process for calculating an external parameter corresponding to.
Thereby, the said subject 1 and 2 were solved.

  The present invention interprets the meaning of the terms described in each claim in consideration of the description of the present specification and the drawings, and certifies the invention according to each claim. There are the following advantageous effects in relation to

[Functional Effect 1 of the Invention] In the camera calibration device of Problem Solving Means 1, at least two reference lines are arranged on both sides of a calibration pattern having a predetermined pattern shape. It is a line that intersects with two points corresponding to the shape of. For this reason, when the imaging process generates the measurement image by photographing the measurement area where the calibration pattern and the reference line are arranged, the reference line in the measurement image corresponds to the shape of the host vehicle. The line intersects the two points, and is thus related to the direction of stop of the vehicle.
The stop error calculation process of the image processing unit uses this relationship to calculate a coordinate relationship between the stop position of the host vehicle and the pattern shape as a stop error based on the two reference lines. Then, the external parameter calculation process calculates an external parameter corresponding to the mounting posture of the camera based on the pattern shape in a state where the stop error is cancelled. Therefore, even if the calibration pattern is not installed in a direction that is strictly parallel to the stopping direction of the host vehicle, on the contrary, the influence of the stopping error is excluded without stopping in the strict direction with respect to the calibration pattern. Thus, the external parameter can be calculated.
Therefore, calibration that can eliminate the influence of the vehicle stop error can be performed with simple equipment.

It is a block diagram which shows the structural example of one Embodiment of this invention. It is explanatory drawing which shows an example of the pattern for a calibration used in a present Example. It is a perspective view which shows an example of the shape of the own vehicle in a present Example. 4A and 4B are explanatory diagrams showing examples of various coordinate systems in this embodiment. It is explanatory drawing which shows an example of the positional relationship of the state which the own vehicle stopped in the measurement area | region in a present Example. It is explanatory drawing which shows an example of the stop angle error and stop position error in the example shown in FIG. It is explanatory drawing which shows an example of the area | region used as the bird's-eye view image in the example shown in FIG. It is explanatory drawing which shows an example which made the image imaged with the initial external parameter in the example shown in FIG. 7 the bird's-eye view image. It is explanatory drawing which shows an example which correct | amended the bird's-eye view image shown in FIG. 8 on the basis of the pattern for a calibration. It is explanatory drawing which shows an example which calculates | requires a stop angle error by correct | amending the bird's-eye view image shown in FIG. 9 based on a reference line. It is explanatory drawing which shows an example in case there exists a camera installation shift angle in the example shown in FIG. It is explanatory drawing which shows an example which made the image imaged with the initial external parameter in the example shown in FIG. 11 the bird's-eye view image. It is explanatory drawing which shows an example which correct | amended the bird's-eye view image shown in FIG. 12 on the basis of the pattern for a calibration. It is explanatory drawing which shows an example which calculates | requires a stop angle error by correct | amending the bird's-eye view image shown in FIG. 13 based on a reference line. It is explanatory drawing which shows an example of the bird's-eye view image which cancels the stop angle error shown in FIG. 15, and reflects a camera installation angle error on an external parameter. It is a flowchart which shows the process example of the calibration method in a present Example. FIGS. 17A and 17B are explanatory diagrams illustrating an example of calculating a stop error using the symmetry of the reference line. It is a flowchart which shows the process example which acquires the image for a measurement in a present Example. It is a flowchart which shows the process example which calculates a stop error in a present Example. FIGS. 20A and 20B are explanatory diagrams illustrating an example in which the correlation coefficient that is the symmetry of the reference line is small in this embodiment. FIGS. 21A and 21B are explanatory diagrams showing an example in which the correlation coefficient that is the symmetry of the reference line is large in this embodiment.

  One embodiment is disclosed as a mode for carrying out the invention.

<1 Calibration method and device>
<1.1 Reference line on the side of the vehicle>
Here, examples of the present embodiment will be disclosed. The present embodiment relates to a calibration technique capable of measuring an external parameter without making the stop position of the host vehicle 12 strict in order to easily carry out the calibration of the camera 10 with simple equipment.

  The camera calibration apparatus includes an image processing unit 14 and a measurement area MA as its main elements. The measurement area MA has a calibration pattern PP and at least two reference lines RL. The image processing unit 14 includes an imaging process 18, a stop error calculation process 20, and an external parameter calculation process 22.

  Referring to FIG. 1, the own vehicle 12 includes a camera 10 that captures an environment outside the vehicle, and stops in a predetermined direction in the measurement area MA. In this embodiment, when stopping, it is not necessary to stop the vehicle precisely in the same direction with respect to a predetermined direction.

The calibration pattern PP is disposed between the reference lines RL in the measurement area MA. As the pattern shape, for example, a checkered pattern in which rectangular patterns of the same size shown in FIG. 2 are alternately arranged in black and white can be adopted. If the length of one side of the checkered rectangle is determined, information on the actual length in the world coordinate system (x, y, z) can be obtained in the measurement image IM. It can be used as a calculation standard for internal parameters and external parameters. In addition, according to the present embodiment, various pattern shapes other than the checkered pattern can be adopted according to the calibration method.
A two-dimensional orthogonal coordinate centered on the origin (pattern coordinate origin OP) of the calibration pattern PP (pattern mat) is defined as a pattern mat coordinate system (xp, yp).

The reference line RL includes a line that intersects two points corresponding to the shape of the host vehicle 12, and at least two reference lines RL are arranged within the imaging range of the camera 10. In the example shown in FIG. 1, the reference line RL has two straight lines corresponding to the left and right side shapes of the host vehicle 12. The reference line RL is a line corresponding to the positions of two points predetermined in the outer peripheral shape 16 of the host vehicle 12, and is symmetric about the central axis of the host vehicle 12 while corresponding to the outer peripheral shape 16 of the host vehicle 12. However, the relationship is not parallel.
That is, in the example shown in the drawing, the imaging direction of the camera 10 is upward in the vertical direction in the figure, and the reference line RL extends slightly outward in the right and left in the figure from the upward direction. That is, the two reference lines RL may be parallel, but need not be parallel.

In the example in which the camera 10 images the rear side, the imaging direction is the rear side of the host vehicle 12, the outer peripheral shape 16 of the host vehicle 12 is the left and right side surfaces of the host vehicle 12, and the reference line RL extends in the vehicle length direction.
In the example in which the camera 10 images the side, the imaging direction is the side of the host vehicle 12, the outer peripheral shape 16 of the host vehicle 12 is the front and rear of the host vehicle 12, and the reference line RL extends in the vehicle width direction.

In the example shown in FIG. 1, the reference line RL is a straight line, but the whole may be a curve, or a part may be a straight line. If the whole or a part is a straight line, it is easy to automate by image processing. On the other hand, when the measurement image IM is wide on the left and right, it is possible to perform correction using the entire imaging range by using a curved line.
Since the reference line RL determines the stop error 24 by image processing, it is desirable that the reference line RL be clearly recognizable by the measurement image IM captured by the camera 10. When binarizing during image processing, it is better to draw a straight line with a material having the opposite brightness to the road surface (background). A simple member can be employed. In addition, since a line that is too thin for the resolution of the camera 10 is not captured by the camera 10, a certain thickness is required.

When the reference line RL is drawn in the longitudinal direction of the vehicle, the outer peripheral shape 16 of the side surface of the host vehicle 12 is symmetric with respect to the vehicle center axis, so that the outer peripheral shape 16 of the side surface of the vehicle body, for example, fenders 16a and 16b is used as a reference. Draw a straight line.
Referring to FIG. 3, a straight line extending from the front and rear fenders 16 a and 16 b of the host vehicle 12 as a reference can be used as the reference line RL. The fenders 16a and 16b often have the maximum width of the vehicle, and have a flat portion to facilitate measurement. Further, tires and wheels may be used instead of the fenders 16a and 16b, but in this case, the steering needs to be set in a straight traveling direction.

When measuring, for example, use a laser marking device for building, mark the true ground surface at the measured position, connect the marks determined by measuring with the front and rear fenders 16a, 16b, etc. It is better to stretch backwards 12. To stretch it into a straight line, use a long bar, or extend a straight line with a laser marking device, and install tape, bar, rope, etc. along it.
Since the laser marking device outputs laser light in a plane direction perpendicular to the ground, the position of the marking device is adjusted so that this and the side of the vehicle body just touch each other, and the marking point is marked.

Even if it is not such a laser marking device, it can be used if it is a tool that can mark the ground directly under the vehicle body side surface measurement position. For example, a tool that hangs a weight on a thread and measures the vertical position (carpenter's tool swinging), or if the ground is horizontal and flat, it can measure the place where it hits the side by raising a stick vertically upward from the ground it can.
Also, the measurement position in the front-rear direction needs to be approximately the same on the left and right, and using a place with a flat surface such as fenders 16a and 16b ensures a certain level of accuracy without making a very precise measurement. Can do.

  Then, the image processing unit 14 performs calibration of the camera 10 based on the measurement image IM captured by the camera 10. In the image processing unit 14, first, the imaging process 18 generates the measurement image IM by photographing the measurement area MA in which the calibration pattern PP and the reference line RL are arranged with the camera 10.

  The stop error calculation process 20 calculates a coordinate relationship between the stop position of the host vehicle 12 and the pattern shape as a stop error 24 based on the two reference lines RL. That is, the stop error calculation processing 20 does not stop the host vehicle 12 in an ideal direction and position when the direction and position of the calibration pattern PP are used as a reference, and the host vehicle 12 with respect to the pattern shape of the calibration pattern PP. An error (stop error 24) regarding the stop direction and position of the vehicle can be calculated.

  The external parameter calculation process 22 calculates an external parameter corresponding to the mounting posture of the camera 10 based on the pattern shape in a state where the stop error 24 is cancelled. That is, the external parameter calculation process 22 calibrates the mounting posture of the camera 10 with the pattern shape as a reference while eliminating the influence of the stop error 24, and calculates the result as an external parameter.

When this external parameter is used, an image excluding the influence of an error from the ideal state of the mounting posture of the camera 10 is generated and, for example, a world coordinate system (x, y, z) of an object or the like captured in the image is generated. It is possible to accurately calculate actual coordinate values and the like.
In this embodiment, when executing this calibration, it is not necessary to stop the own vehicle 12 at a precise position with respect to the calibration pattern PP, so that the reference line RL can be used without using a large-scale facility. The external parameter can be calculated using a normal parking space in which can be placed. In addition, it is possible to calculate the mounting error without using the plurality of cameras 10 and excluding the stop error 24 for the single camera 10.

  As described above, in this embodiment, after the stop error 24 is calculated by image processing by comparing the reference line RL intersecting with two points corresponding to the shape of the host vehicle 12 and the calibration pattern PP, the stop error 24 is calculated. By executing the calibration of the mounting error in a state where the influence of the above is removed, it is possible to easily execute a highly accurate calibration.

This will be described in detail.
In the present embodiment, first, internal parameters and distortion parameters are separately measured for the camera 10 mounted on the vehicle. The internal parameter corrects the influence of the lens system of the camera 10 and is a parameter specific to the camera 10 and is irrelevant to the mounting posture of the camera 10. The distortion parameter is a parameter representing a non-linear state such as a wide-angle lens, and may be integrated with an internal parameter.

Then, as shown in FIG. 1, a calibration pattern PP is arranged around the own vehicle 12 that can be photographed by the camera 10. In the present embodiment, the arrangement of the calibration pattern PP does not need to be precise with respect to a reference such as the traveling direction of the vehicle, but it is desirable to arrange the calibration pattern PP to some extent.
As shown in FIG. 2, the calibration pattern PP can be a checkered pattern, and the positions of black and white intersections (lattice points) of the checkered pattern and the distances between the lattice points are in the pattern mat coordinate system (xp, yp). expressed.
As shown in FIG. 3, the reference line RL can be easily installed by adopting fenders 16 a and 16 b as the outer peripheral shape 16 of the host vehicle 12.

FIG. 4 illustrates a relationship example of the coordinate system when the camera 10 is a back camera.
As shown in FIG. 4A, the world coordinate system (x, y, z) has an xy plane on the ground, the vehicle axis rearward is the y axis, the vertical upward direction is the z axis, and the vehicle lateral width is perpendicular to the yz plane. Take the x axis in the direction. The origin of the world coordinate system (x, y, z) is the center of the rear end of the vehicle.
As shown in FIG. 4B, an overhead image coordinate system (u, v) is defined on the image element of the camera 10. The camera coordinate system (xc, yc, zc) is set so that the optical axis direction is the zc axis with the focal position f as the origin. As a coordinate system, there is a pattern mat coordinate system (xp, yp) shown in FIG. 2 in addition to that shown in FIG.

As shown in FIG. 5, in this embodiment, the horizontal distance between the pattern coordinate origin OP and the vehicle feature point FP (for example, the rearmost end of the own vehicle 12 in the case of a back camera) is set as a stop error 24 (stop position error L). , X). This measurement may be performed by image processing, but manual measurement is simple. Note that when the pattern coordinate origin OP and the horizontal position of the vehicle feature point FP are arranged to coincide with each other, the stop position error L (horizontal distance) becomes zero.
Since L is a parameter used to set the world coordinate origin at the center of the rear end of the vehicle body, when calculating L from the image processing, the vehicle position is set to y of the camera position (x, y) in the pattern mat coordinate system. The sum of the rear end center and the y-axis component of the horizontal distance between the camera mounting positions is set as L.

  Further, at least two reference lines RL related to the shape of the vehicle 12 are drawn on the floor so that the camera 10 can take a picture. For example, draw a straight line on the floor that connects two points on the side of the body. As described above, the straight lines that serve as the two reference lines RL do not have to be parallel, but maintain the relationship with the front-rear direction of the host vehicle 12 such as being symmetric with respect to the central axis of the host vehicle 12. It is good to arrange in the direction to do. Moreover, it is good also as a curve instead of a straight line.

Then, the camera 10 captures the calibration pattern PP and the two reference lines RL. The calibration pattern PP and the reference line RL need not be photographed simultaneously, and may be photographed separately.
In this embodiment, external parameters of the camera 10 are calculated from the measurement image IM obtained by capturing the calibration pattern PP.

  In the example in which the reference line RL is symmetric, the stop error calculation process 20 is based on the symmetry of the at least two reference lines RL with the imaging direction of the camera 10 in the measurement image IM as the center. It is preferable to provide a process for calculating the stop error 24. By this process, the conversion by image processing is repeated until the two reference lines RL in the measurement image IM are symmetric, and the stop error 24 can be obtained from the conversion amount up to that point when the symmetric.

  Next, a method for obtaining the stop angle error θz from the reference line RL by dividing the stop error 24 into the stop angle error θz and the stop position errors L and X will be described with reference to FIGS. In this example, after correcting for the stop angle error θz, an external parameter that reflects the effect of the camera installation angle error θc, which is an attachment error of the camera 10, is calculated.

  6 to 10 are explanatory examples in which the camera installation angle error θc is set to 0, and FIGS. 11 to 15 are examples in the case where there is a camera installation angle error θc.

Referring to FIG. 6, the stop error 24 is an error between the pattern mat coordinate system (xp, yp) and the world coordinate system (x, y, z), and the stop angle error θz of the stop error 24 is This is the angle between the yp axis (xp axis) in the pattern mat coordinate system and the y axis (x axis) in the world coordinate system.
In the examples shown in FIGS. 6 to 15, the reference line RL is parallel to the y axis of the world coordinate system (x, y, z) for the sake of simplicity. The stop reference line SL is parallel to the yp axis of the pattern mat coordinate system (xp, yp) and serves as a reference for an ideal stop position of the host vehicle 12. In the example shown in FIG. 6, since the reference line RL is parallel to the y-axis, the angle formed by the reference line RL and the stop reference line SL is a stop angle error θz.
Further, the stop error 24 and the like can be calculated by image processing for the measurement image IM. Here, in order to briefly explain the advantages of the present embodiment, an image processing example for the overhead image BI is used. Will be explained.

  As shown in FIG. 7, an overhead area BA is defined in the world coordinate system (x, y, z). The image processing unit 14 can generate the overhead image BI by converting the coordinate values of the measurement image IM. For example, the image processing unit 14 assumes that the height (y-axis value) in the world coordinate system (x, y, z) of the calibration pattern PP or the like captured in the measurement image IM is 0, The coordinates of x, y are made to correspond to the coordinates of the overhead view coordinate system (u, v), and the corresponding coordinate value (gray value) of the measurement image IM is taken as the coordinate value of the overhead image BI. Then, an overhead image BI as shown in FIG. 8 can be obtained. Here, calculation or removal of the stop error 24 is described as image processing for the overhead image BI, but it is also possible to directly process the measurement image IM using a correspondence relationship of coordinate values.

In the bird's-eye view image BI shown in FIG. 8, the yp axis of the calibration pattern PP is inclined with respect to the y axis of the world coordinate system parallel to the optical axis zc of the camera 10. Based on the arrangement position and angle of the calibration pattern PP, the stop direction of the host vehicle 12 is inclined from the ideal value. If the stop position of the host vehicle 12 is used as a reference, the calibration pattern PP is inclined and arranged.
Among the stop errors 24, the stop angle error θz is not caused by an error in the mounting angle of the camera 10. On the other hand, if the calibration related to the mounting error of the camera 10 is executed without removing the stop error 24, the rotation and translation errors of the image due to the stop error 24 are also included in the camera 10 mounting error. . That is, if there is a stop error 24, the calibration for the mounting error of the camera 10 cannot be performed accurately. In other words, the external parameters of the camera 10 cannot be accurately calculated by calibration that cannot eliminate the influence of the stop error 24.

  FIG. 9 shows an example in which the overhead image BI is corrected based on the calibration pattern PP. The bird's-eye view image BI rotates counterclockwise. By this correction, the u-axis and the yp-axis become parallel. As shown in the drawing, the angle formed by the reference line RL parallel to the front-rear direction of the host vehicle 12 and the stop reference line SL parallel to the yp axis is equal to the stop angle error θz.

FIG. 10 shows an example in which the overhead image BI is corrected based on the reference line RL. When the reference line RL is rotated until the reference line RL is parallel to the u axis of the overhead coordinate system (u, v) with the overhead image BI corrected by the calibration pattern PP as a reference, the rotation angle becomes a stop angle error θz. Even if the reference line RL is not parallel to the front-rear direction of the host vehicle 12, if it is an object around the front-rear axis of the host vehicle 12, it is rotated until it becomes symmetrical on the bird's-eye view image BI. Can be requested.
As described above, the stop error calculation process 20 can calculate the coordinate relationship between the stop position of the host vehicle 12 and the pattern shape as the stop angle error θz based on the two reference lines RL.

Now, the contents of image processing using the reference line RL will be described with reference to FIGS. 11 to 15 by taking a normal case with a camera installation deviation angle θc as an example.
In the case of a back camera, the camera installation deviation angle θc is an angle formed by a line parallel to the central axis of the host vehicle 12 (the y axis of the world coordinate system) and the optical axis zc of the camera 10.

Referring to FIG. 11, when there is a camera installation deviation angle θc, the optical axis zc of the camera 10 is tilted with respect to the y-axis of the world coordinate system, and therefore the measurement area MA is relative to the own vehicle 12 by that angle. Tilt. When the measurement image IM is captured in this state, an angle-related overhead image BI shown in FIG. 12 is obtained.
In the figure, a straight line ya and a straight line xa are straight lines that pass through the origin of the camera coordinate system (zc, yc, zc) and are parallel to the y axis and the x axis of the world coordinate system, respectively. Therefore, the angle formed by the optical axis zc and the straight line ya is the camera installation angle θc.

Referring to FIG. 12, the u-axis of the overhead image BI and the optical axis zc of the camera 10 are parallel. The y-axis of the world coordinate system is inclined minus the camera installation angle θc with respect to the u-axis, and the yp-axis of the pattern mat coordinate system is further inclined by the stop angle error θz. After all, considering that the camera installation angle θc is reversed, the yp axis of the pattern mat coordinate system is tilted by − (θz−θc) with respect to the u axis.
In the example shown in FIG. 12, since the y axis and the reference line RL are parallel, the angle formed by the reference line RL and a line parallel to the u axis is equal to the camera installation deviation angle θc.
In FIG. 12, a straight line ULa is a straight line parallel to the optical axis zc of the camera coordinate system, is parallel to the u axis in the state shown in FIG. 12, and an angle formed with the reference line RL is a camera installation deviation angle θc.

As shown in FIG. 13, when the overhead image BI shown in FIG. 12 is corrected based on the calibration pattern PP, the u axis and the yp axis become parallel. The straight line ULb is a straight line parallel to the yp axis. In the example shown in FIG. 13, the straight line ULb is parallel to the u axis, and the angle formed with the reference line RL is a stop angle error θz.
And if it correct | amends based on the reference line RL for the bird's-eye view image BI shown in FIG. 13, as shown in FIG. 14, the u-axis and the y-axis become parallel. The correction angle (rotation angle) of the correction based on the reference line RL becomes a stop angle error θz (stop error calculation process 20).

  As shown in FIG. 15, when the obtained stop angle error θz is canceled from the overhead image BI, the optical axis zc and the y axis of the camera 10 become parallel. The angle formed by the yp axis and the y axis is the camera installation deviation angle θc. Therefore, when the external parameter calculation process 22 calculates the external parameter D based on the calibration pattern PP for the overhead image BI shown in FIG. 15, the external parameter D is not affected by the stop angle error θz, On the other hand, the parameter reflects the effect of the camera installation angle θc.

Next, the calibration method of this embodiment will be described with reference to FIG.
First, the host vehicle 12 to which the camera 10 that captures the environment outside the vehicle is attached is stopped in a predetermined direction in a predetermined measurement area MA (step S1, stopping process). For example, the host vehicle 12 is stopped in the state shown in FIG.
Next, at least two lines that intersect two points corresponding to the shape of the host vehicle 12 are arranged as reference lines RL in the measurement area MA that is within the imaging range of the camera 10 (step S2, reference line arrangement). Process). In the example shown in FIG. 11, the reference line RL is related to the outer peripheral shape 16 of the host vehicle 12 and is symmetric about the center in the front-rear direction.
Then, a calibration pattern PP having a predetermined pattern shape is installed between the two reference lines RL in the measurement area MA, and the calibration pattern PP and the reference line RL are photographed. To generate the measurement image IM (step S3, imaging step). In the example illustrated in FIG. 12, the overhead image BI is used, but the measurement image IM is an image having an angle relationship illustrated in FIG. 12.
Further, the camera 10 is calibrated based on the measurement image IM (steps S4 to S6, image processing step).
In this image processing step, first, based on the two reference lines RL, the coordinate relationship between the stop position of the host vehicle 12 and the pattern shape is calculated as a stop error 24 (step S4). For example, correction is performed on the measurement image IM (not shown) using the calibration pattern PP as shown in FIG. 13 as a reference, and correction based on the reference line RL as shown in FIG. Thus, an error (rotation angle or translation amount) between the calibration pattern PP and the reference line RL can be set as the stop error 24.
In the calculation of the stop error 24, the reference line RL shown in FIG. 17A corrected with the calibration pattern PP is asymmetrical with respect to the image center line CL, as shown in FIG. 17B. The rotation angle up to becomes the stop angle error θz.
Next, as shown in FIG. 15, the stop error 24 is canceled (step S5), and in this state, an external parameter corresponding to the mounting posture of the camera 10 is calculated based on the pattern shape (step S6).
In the image processing step, the stop error 24 is calculated using the measurement image IM as a direct target, and the stop error 24 is obtained after conversion to a coordinate system in which the stop error 24 can be easily calculated. good.

1.1 Effect of vehicle side reference line As described above, in the present embodiment, since the external parameter D of the camera coordinates is obtained from at least two reference lines RL, the vehicle body and the calibration pattern PP are accurately aligned. It becomes unnecessary and can be calibrated quickly with small equipment. That is, when the calibration pattern PP and the two reference lines RL are used, it is possible to eliminate the need for precise arrangement of the calibration pattern PP.
If the two reference lines RL are symmetric with respect to the vehicle body central axis of the host vehicle 12, the same effect as the calibration using the vehicle body axis can be obtained.
Further, when the measurement image IM obtained by imaging the reference line RL is subjected to overhead processing and is symmetric with respect to the center of the screen, the external parameter D can be determined by image processing.
Furthermore, by measuring the distance between the vehicle feature point FP and the pattern coordinate origin OP of the calibration pattern PP, the precise arrangement of the calibration pattern PP can be made unnecessary.
In this way, since the stop error 24 is obtained using the reference line RL, accurate calibration can be executed even with a small-scale facility.

<1.2 Correction processing with overhead image>
Next, an example in which the stop error 24 is calculated based on the symmetry of the reference line RL in the overhead view image BI will be described. In this example, the coordinate system conversion matrix for converting the pattern mat coordinate system (xp, yp) to the world coordinate system (x, y, z) includes the stop angle error θz that is the stop error 24 and the stop position errors L and X. By reflecting the value, the external parameter D is obtained while canceling the stop error 24.

In this example, the stop error calculation process 20 converts the measurement image IM into an overhead image BI, and the stop error based on the symmetry of the at least two reference lines RL in the overhead image BI. 24 is calculated. For example, the stop error calculation processing 20 repeats image processing such as rotation and movement until the reference line RL becomes symmetric in the bird's-eye view image BI. It can be set as the stop error 24 in the coordinate system of BI.
This process of the stop error calculation process 20 is preferably performed in step S4 in the example shown in FIG.

  In a pinhole camera model that has been frequently used to relate coordinate systems, the relationship between the camera coordinate system (xc, yc, zc) and the overhead image coordinate system (u, v) is expressed by the following equation (1a). Is done.

Where s is an arbitrary scale factor.
Here, the matrix A shown in the equation (1b) is a parameter unique to the individual camera 10 and is referred to as an internal parameter A.
Since the actual camera 10 has distortion, a relational expression between the overhead image coordinate system (u, v) having no distortion and the image coordinates (ud, vd) having distortion actually obtained is introduced. For example, when considering up to the fourth order term in the radial distortion, if the distance from the origin (u0, v0) is r and the counts of the respective orders are k1, k2, the following equation (2) is obtained.

  The relational expression between the reference world coordinate system (x, y, z) and the camera coordinate system (xc, yc, zc) is expressed by the following expression (3a). Here, the matrix D shown in the following equation (3b) is a parameter determined by the positional relationship of the coordinate system, and is called an external parameter D.

Subsequently, the transformation matrix P from the pattern mat coordinate system to the world coordinate system to be applied when the stop position errors X and L and the stop angle error θz are not 0 is expressed by the following equation (4a).
Specifically, the stop angle error θz shown in FIG. 15 is given to the following equation (4b), and the stop position errors X and L shown in FIG. 6 are given to the following equations (4c) and (4d) to obtain the pattern mat. A transformation matrix P from the coordinate system to the world coordinate system is obtained.
Here, Rz, Tx, and Ty are a rotation matrix around the z-axis, a parallel progression in the x-axis direction, and a parallel progression in the y-axis direction, respectively.

The calculation of the transformation matrix P from the pattern mat coordinate system to the world coordinate system from the stop position errors X and L and the stop angle error θz shown in Equation 4, FIG. 6, etc. is performed by reversing the order of translation and rotation. The procedure may be changed. In that case, the same result can be obtained in the calculation procedure of the transformation matrix P from the pattern mat coordinate system to the world coordinate system of Expression 4 by appropriately changing the order of the rotation and the parallel progression according to the change.
For example, when measuring the horizontal distance L from the vehicle feature point FP, instead of measuring the distance from the vehicle feature point FP along the vehicle axis as shown in FIG. When y = 0 in the overhead parameter when determining the stop position error X, L, the parameter stop position error X, L of X, θz is determined as y = -L, and the world coordinates from the pattern mat coordinate system The same result can be obtained by changing the transformation matrix into the system by changing the order of translation and rotation of the x and y axes.

The camera calibration procedure is as follows.
Referring to FIG. 18, first, the own vehicle 12 is stopped in the measurement area MA (step S11). Then, a checkered calibration pattern PP (pattern mat) shown in FIG. Each checkered pattern is 50 [cm] square, about 4 x 4 pieces. In order to calculate the external parameter D, the horizontal distance L between the calibration pattern PP and the rear end of the host vehicle 12 is measured manually (step S12). Subsequently, the measurement area MA in which the calibration pattern PP is installed is photographed by the camera 10 to obtain a first measurement image IMa. Further, the wide-angle distortion of the measurement image IMa is corrected (step S13).

Then, before and after the calibration of the camera 10 (steps S21 and S22), the reference line RL is arranged and photographed (step S14). The reference line RL measures two positions on the front and back sides of the vehicle 12 of the host vehicle 12, extends a straight line connecting the two points to the rear of the vehicle, and can be photographed with a tape, a rope, or the like. The two reference lines RL are symmetrical with respect to the vehicle axis.
This measurement area is photographed to form a second measurement image IMb, and wide-angle distortion is corrected (step S15).

Referring to FIG. 19, first, the camera 10 is calibrated based on the measurement image IMa (step S21). In this calibration, first, the positions of the lattice points of the calibration pattern PP are extracted. A camera external parameter D is temporarily calculated from the position of each grid point and the corresponding coordinate value in the pattern mat coordinate system (xp, yp) and the internal parameter A of the camera 10 (step S22). Here, the external parameter D is temporarily calculated in a state where the influence of the stop error 24 is not excluded.
Then, an overhead image BI corresponding to the pattern mat coordinate system (xp, yp) can be obtained from the obtained temporary external parameters D (Rx, Ry, Rz, Tx, Ty, Tz) (step S23).

Next, overhead image processing is added to the measurement image IMb obtained by imaging the reference line RL using the external parameter D obtained in step S22. Overhead image processing parameters are x = 0, y = 0, θx = -180 degrees, θy = 0 degrees, θz = 0 degrees, and z position is the reference line with respect to the pattern mat coordinate system (xp, yp) It is assumed that RL appears in the screen. That places a virtual camera for the overhead image, pattern matte coordinate system (xp, yp) on the vertically downward z p axis you orthogonal to xp-yp surface of. As a result, an image obtained by placing the virtual camera vertically downward above the center of the rear end of the vehicle can be obtained.

Then, as shown in FIG. 17, the values of the stop position errors X and L and the stop angle error θz are changed in the overhead view image BI so that the two reference lines RL are bilaterally symmetrical on the overhead view image BI. (Step S24). The fact that the two reference lines RL are symmetrical in the overhead image BI means that the central axis of the host vehicle 12 is parallel to the u axis of the overhead image coordinate system and substantially coincides with a straight line passing through the center in the overhead image BI. Can be judged. Stop position error X, when this reference line RL is symmetrical
L and the stop angle error θz are overhead images BI of the y axis of the world coordinate system (x, y, z) and the yp axis of the pattern mat coordinate system (xp, yp) based on the stop direction of the host vehicle 12. The angle formed above is a value derived from the stopping error 24 with respect to the calibration pattern PP of the host vehicle 12.

Next, the value obtained by multiplying the vertical distance X by -1 among the obtained stop position errors X and L, the horizontal distance L measured in advance, and the stop angle error θz are parameters. A transformation matrix P (formula (4)) to the world coordinate system is calculated (step S25).
When the matrix P of the pattern mat coordinate system (xp, yp) is multiplied by the transformation matrix P from the pattern mat coordinate system to the world coordinate system, the coordinates of each grid point in the world coordinate system (x, y, z) A value is obtained (step S26).
Calibration is performed again using the position data of the lattice points of the pattern mat on the bird's-eye view image BI and the coordinate value data of the corresponding lattice points in the world coordinate system (step S27). The external parameter D calculated here is the external parameter D to be obtained. The external parameter D calculated in step S27, the influence of the stop error 24 in accordance with the value of such stop position angle error θz given to the transformation matrix P from pattern mat coordinate system to the world coordinate system is canceled, On the other hand, the parameter value can correct the influence of the mounting error of the camera 10.

  In this way, in this embodiment, first, the camera 10 captures the calibration pattern PP (first measurement image IMa) and the two reference lines RL (second measurement image IMb), ( FIG. 18). The calibration pattern PP and the reference line RL need not be photographed simultaneously, and may be photographed in multiple steps. The external parameter D of the camera 10 is temporarily calculated from the first measurement image IMa. On the basis of the external parameter D, a bird's-eye view process is performed on the second measurement image IMb. The overhead camera position is y = 0, θx = −180 degrees, θy = 0 degrees, and the z position is where the reference line RL can be seen.

Then, with respect to the overhead image BI, the stop position error X on the x axis and the stop angle error θz that is a rotation parameter around the z axis are increased or decreased, and the two reference lines RL are symmetrically positioned on the screen. (Steps S23 and S24 in FIG. 19).
From the pattern mat coordinate system to the world coordinate system using the obtained stop angle error θz, the value obtained by multiplying the stop position error X by −1, the coordinate origin of the calibration pattern PP, and the horizontal distance L of the vehicle feature point FP as parameters. A conversion matrix P (formula 4) is calculated, and the coordinates of the calibration pattern PP are converted into values in a new coordinate system using the matrix.
Furthermore, using the image in which the calibration pattern PP is copied, the converted coordinate position is applied, camera calibration is performed again, and the external parameter D is calculated. This is the external parameter D to be obtained.
As described above, in the present embodiment, in the first calibration using the calibration pattern PP, only z, θx, θy among the external parameters D are determined, y is given from actual measurement, x , Θz is determined from two reference lines RL.

1.2 Effects of Correction Processing on Overhead Image As described above, in this embodiment, the stop error calculation processing 20 is based on the symmetry of the reference line RL in the overhead image BI, and the stop error 24 (for example, the stop angle error). θz and stop position error X) are calculated. For this reason, the stop error calculation processing 20 repeats the rotation and translation image processing until the reference line RL becomes symmetric in the overhead image BI, and when the reference line RL becomes symmetric, the rotation amount and the movement amount up to that time Can be set as the stop error 24 in the coordinate system of the overhead image BI.
For this reason, the stop error 24 can be automatically calculated from the image obtained by capturing the reference line RL by image processing. Then, the stop error 24 for the calibration pattern PP can be calculated independently from the influence of various other errors.
As described above, in the present embodiment, the calibration can be performed with high accuracy without accurately arranging the calibration pattern PP in order to solve the problem that the calibration pattern PP needs to be precisely arranged. The advantage that the calibration pattern PP does not need to be accurately arranged is equal to the advantage that the own vehicle 12 does not need to be accurately stopped in the predetermined measurement area MA.
Then, in the present embodiment, with respect to the problem of accurately measuring the position of the vehicle or arranging the vehicle at a predetermined position, in this embodiment, only the reference line RL that is symmetric with respect to the center axis of the vehicle is measured. That's it.

<1.3 Automatic calculation of symmetry>
Next, a method for automatically evaluating the symmetry of the reference line RL by image processing will be described. In this example, the stopping error calculation process 20 automatically determines the symmetry of the reference line RL by information processing such as straight line extraction, superposition, and calculation of the matching degree for the superposed shape.
Specifically, the stop error calculation process 20 first extracts a straight line in the overhead image BI (extraction process). Subsequently, the extracted longest two straight lines with the imaging direction of the camera 10 in the bird's-eye view image BI as the center are inverted and superimposed (superposition processing). Further, the degree of matching is repeatedly calculated by repeatedly rotating the overhead image BI (calculation processing). The calculated matching degree is temporarily stored while corresponding to the rotation angle. Then, the stopping error 24 is calculated based on the rotation angle at which the matching degree is maximized (comparison process).
Thus, the x-axis stop position error X and the rotation parameter θz around the z-axis are calculated from the image obtained by capturing the reference line RL.

  The extraction process, the overlay process, the calculation process, and the comparison process included in the stop error calculation process 20 are steps included in the stop error calculation process when the present embodiment is regarded as a method. That is, the stop error calculation step may include an extraction process, an overlay process, a calculation process, and a comparison process. In this example, the disclosure about the stop error calculation process 20 is also the disclosure about the stop error calculation step.

Referring to FIG. 20, the procedure of image processing by this stop error calculation process 20 is as follows.
As shown in FIG. 20A, two reference lines RL are taken in the overhead image BI, and there are a left reference line RLa and a right reference line RLb in the figure. The extraction process of the stop error calculation process 20 extracts this straight line by binarizing the image. For example, when the overhead image BI is Hough transformed, the reference line RL that is a straight line can be found. The Hough transform is a well-known image processing technique for searching for a straight line from an image. If a straight line is extracted, it is preferable to exclude regions other than the vicinity of the reference line RL from the subsequent image processing.

  In the superimposition process, the extracted two longest straight lines are reversed and superimposed with the imaging direction of the camera 10 in the overhead image BI as the center. For example, the bird's-eye view image BI is divided into two parts with the image center line CL at the center of the image in the left-right direction in the figure as a reference, and one of them is reversed left and right and then superimposed. Then, as shown in FIG. 20B, the left and right reference lines RLa and RLb approach.

In this state, the calculation process calculates the matching degree. The degree of matching is how much the left reference line RLa and the right reference line RLb match.
The stop error calculation process 20 repeats the calculation of the matching degree. Specifically, the bird's-eye view image BI is rotated or translated, straight lines are extracted again, superimposed, and the matching degree is calculated. The stop error calculation processing 20 calculates a matching degree by rotating or translating until a predetermined condition is satisfied, and searches for a parameter (corresponding to the stop error 24) that maximizes the matching degree.

  When the rotation or translation is completed until a predetermined condition is satisfied, the comparison process calculates the stop error 24 based on the rotation angle at which the matching degree is maximized among the repeatedly calculated matching degrees. When the two reference lines RL are positioned symmetrically, the overlapping degree = the matching degree is maximized.

  The matching degree can be calculated by various methods. In the present embodiment, for example, a determination method based on a correlation coefficient can be applied. When the degree of matching is expressed as a correlation coefficient, the maximum value of the ideal correlation coefficient is 1, but it is actually 1 due to the influence of two reference lines RL, subtle differences in the ground pattern, noise, and the like. A smaller value.

For example, as shown in FIGS. 20 and 21, the correlation coefficient gives a window WD (green frame) corresponding to a straight line shape in the overhead view image BI, and pixel values constituting the reference line RL in the window WD. Can be calculated based on the number of.
When the size of the window WD is Mt × Nt pixels, the correlation coefficient R is expressed by the following equation (5).
However, the pixel values of the left and right images are T (i, j) and I (i, j), respectively, and i and j are coordinate values in the window WD (coordinate values parallel to uv of the overhead image BI). The average values I and T are defined by the following formulas (6a and 6b).

  In the example shown in FIG. 20, since there is no overlap between the reference lines RL, the correlation coefficient is minimum, and in the case of FIG. 21, the overlap is maximum and the correlation coefficient is maximum.

1.3 Effect of automatic symmetry calculation As described above, in the example in which the symmetry is automatically calculated, the stop error calculation process 20 automatically sets the stop error 24 to the stop error 24 by calculating the overlap or the degree of matching (for example, the correlation coefficient). Since the corresponding rotation angle and translation amount are obtained, the measurement load can be greatly reduced.
In addition, in the method of comparing the matching degrees at the time of overlapping, it is possible to process with the same accuracy whether the reference line RL is parallel or not parallel.
In order to place the reference line RL in parallel, it is necessary to use a place that is guaranteed to be parallel, or to confirm that it is parallel by a separate measurement, but this makes a simple calibration possible. It becomes impossible. In this respect, in this embodiment, since a certain accuracy can be ensured this time without satisfying the parallel condition, the accuracy can be ensured without being limited to the vehicle type.

DESCRIPTION OF SYMBOLS 10 Camera 12 Own vehicle 14 Image processing part 16 Outer periphery shape 16a, 16b Fender 18 Imaging process 20 Stop error calculation process 22 External parameter calculation process 24 Stop error MA Measurement area | region IM, IMa, IMb Measurement image RL, RLa, RLb Reference line PP Calibration pattern BA Overhead region BI Overhead image FP Vehicle feature point OP Pattern coordinate origin SL Stop reference line
zc optical axis
f Focus position
L, X Stop position error θz Stop angle error θc Camera installation deviation angle WD Window CL Image center line

Claims (5)

A stopping step of stopping the vehicle in which a camera for imaging an environment outside the vehicle is attached in a predetermined direction in a predetermined measurement area;
A reference line arrangement step of arranging at least two lines in the measurement area within the imaging range of the camera, with reference to a line intersecting two points corresponding to the shape of the host vehicle;
A calibration image having a predetermined pattern shape is set between the two reference lines in the measurement area, and a measurement image is generated by photographing the calibration pattern and the reference line. Imaging process;
An image processing step of performing calibration of the camera based on the measurement image,
This image processing step
Calculating a coordinate relationship between the stop position of the host vehicle and the pattern shape as a stop error based on the two reference lines;
Calculating an external parameter corresponding to the mounting posture of the camera based on the pattern shape in a state where the stopping error is canceled;
A camera calibration method characterized by comprising: A measurement region in which the own vehicle having a camera that images the environment outside the vehicle stops in a predetermined direction;
An image processing unit that executes calibration of the camera based on a measurement image captured by the camera;
The measurement area is
A reference line that includes a line that intersects with two points corresponding to the shape of the host vehicle, and is arranged in at least two of the measurement areas within the imaging range of the camera;
A calibration pattern having a predetermined pattern shape between the two reference lines in the measurement region,
The image processing unit
An imaging process for generating the measurement image by imaging a measurement region in which the calibration pattern and the reference line are arranged with the camera;
Based on the two reference lines, a stop error calculation process for calculating a coordinate relationship between the stop position of the host vehicle and the pattern shape as a stop error; and An external parameter calculation process for calculating an external parameter corresponding to the mounting orientation of the camera;
A camera calibration device comprising: The at least two reference lines are symmetric with respect to a central axis of the vehicle;
The stop error calculation process includes:
A process of calculating the stopping error based on symmetry of the at least two reference lines around the imaging direction of the camera in the measurement image;
The camera calibration apparatus according to claim 2, further comprising:
The at least two reference lines are symmetric with respect to a central axis of the vehicle;
The stop error calculation process includes:
A process of converting the measurement image into a bird's-eye view image using temporary external parameters and calculating the stop error based on symmetry of the at least two reference lines in the bird's-eye view image;
The camera calibration apparatus according to claim 2, further comprising:
The at least two reference lines are straight lines;
The stop error calculation process includes:
A process of extracting a straight line in the overhead view image;
A process of inverting and superimposing the extracted two longest straight lines with the imaging direction of the camera in the overhead image as the center;
Processing for repeatedly calculating the matching degree by repeatedly rotating the overhead view image;
A process of calculating the stopping error based on the rotation angle at which the matching degree is maximized;
The camera calibration device according to claim 4, further comprising:

Images