JP5406898B2 - Car camera calibration device, car camera calibration method, system, and program - Google Patents

Description

The present invention includes a calibration device of the vehicle camera, a method for calibrating a vehicle camera, a system, a program.

  In recent years, an increasing number of vehicles are equipped with a camera so that a driver of the vehicle can visually recognize a scene such as a side or rear of the vehicle via a monitor inside the vehicle. Furthermore, an apparatus has been developed that performs image processing or the like using a captured image acquired by the camera and supports driving such as parking. In such a device, a camera that acquires a captured image serving as a base for calculating information used for vehicle positioning or the like is required to have particularly high optical axis accuracy. Such high optical axis accuracy is not easily obtained at the time of attachment, and is realized by calibrating the optical axis with high accuracy after the camera is attached to the vehicle. As a technique for performing such a calibration, for example, there is a technique described in Patent Document 1 whose source is shown below.

  The in-vehicle camera calibration device described in Patent Document 1 includes a marker (calibration index) having a black and white checkered pattern in the field of view of an in-vehicle camera, and captures a captured image obtained by photographing the marker. Image recognition processing is performed to detect the center point (calibration point) of the marker. The in-vehicle camera is calibrated using the detected center point of the marker. This marker is colored with a checkered pattern, a calibration point formed by the checkered pattern is specified from an edge included in a captured image acquired by the in-vehicle camera, and the in-vehicle camera is calibrated using the calibration point.

JP 2008-131250 A

  As described above, a calibration marker colored with a checkered pattern is used for calibration of the in-vehicle camera described in Patent Document 1, and a calibration point (a center point of the calibration marker) is based on the edge of the calibration marker. Is detected. However, such in-vehicle cameras are generally calibrated in factories, and there are many walking zones, white lines, fences, and the like that define places where workers can walk. Since these gait bands, white lines, fences, and the like are generally configured by using a large number of linear shapes, there is a possibility of erroneously detecting the straight line of the calibration marker.

SUMMARY OF THE INVENTION In view of the above problems, there are a calibration device of the vehicle-mounted camera is no erroneous detection, and method for calibrating a vehicle camera, and the system, to provide the programs.

In order to achieve the above object, the characteristic configuration of the system according to the present invention includes a curved figure formed by a curve surrounding a predetermined region, a horizontal plane of the visual field of the in-vehicle camera, and the horizontal plane. comprising a calibration target having a linear shape comprising at least two straight lines is provided an angle relative to the vertical perpendicular plane, the image receiving receiving a captured image including the calibration target obtained by imaging by the in-vehicle camera And an edge is detected from the captured image received by the image receiving unit, and a curved figure formed by a curve surrounding a predetermined region is detected based on the detected edge, and then surrounded by the curved figure. and the area as the search area, in the search area to detect the intersection of the straight line shape composed of two straight lines even without least on the basis of the detected edge detection And in that it comprises a calibration device of the in-vehicle camera having an a calibration target detection unit configured to set the intersection as a calibration point.

  With such a characteristic configuration, in the captured image obtained by imaging with the in-vehicle camera, a region surrounded by a curved figure having a similar shape is detected at the place where the in-vehicle camera is calibrated, and further, the surrounding Since the calibration points are specified by detecting two linear figures only within the defined area, the area for detecting the linear figure can be limited. Therefore, since the time required for detecting the straight line figure can be shortened, the time required for calibration of the in-vehicle camera can be shortened. Moreover, since the straight line figure in the area | region enclosed by the curve figure is detected, a calibration point can be detected reliably. Therefore, erroneous detection of the calibration index can be prevented.

  In addition, another feature of the on-vehicle camera calibration device according to the present invention for achieving the above object is an imaging obtained by imaging a calibration index arranged opposite to the on-vehicle camera with the on-vehicle camera. An image receiving unit that receives an image, an edge detected from the captured image received by the image receiving unit, and at least two curved figures formed by a curve surrounding a predetermined region based on the detected edge , When the intersection is detected within the area surrounded by the curved figure, or when the curved figure is detected around the intersection, the intersection is calibrated. A calibration index detection unit that is set as a point.

  With such a characteristic configuration, in the captured image obtained by capturing with the in-vehicle camera, a curved figure having a similar shape and a point of intersection of the two linear figures are respectively located at the place where the in-vehicle camera is calibrated. Since the calibration point is specified by detecting and determining the validity of the intersection point based on the arrangement relationship between the curved figure and the intersection point, the calibration point can be accurately detected. Therefore, erroneous detection of the calibration index can be prevented.

In addition, a characteristic configuration of the calibration method of the in-vehicle camera according to the present invention for achieving the above-described object is that a curved figure formed by a curve surrounding a predetermined area disposed opposite to the in-vehicle camera and the in- vehicle camera. the calibration target has a linear shape comprising at least two straight lines is provided an angle relative to the vertical perpendicular plane to the horizontal plane and the horizontal plane of the visual field, an imaging step for imaging at the in-vehicle camera, obtained in the imaging step Detecting an edge from a captured image including the calibration index, detecting a curved figure formed by a curve surrounding a predetermined area based on the detected edge, and then searching in an area surrounded by the curved figure and regions in the search area, even without least on the basis of the detected edge to detect the intersection of the straight line shape composed of two straight lines, sets the detected the intersection has a calibration point In that it comprises a Tadashiten setting step.

  With such a configuration, it is possible to limit the area for detecting a linear figure as in the case of the characteristic configuration of the on-vehicle camera calibration device described above, and thus it is possible to shorten the time required to detect the linear figure. Become. Therefore, the time required for calibration of the on-vehicle camera can be shortened. Moreover, since the straight line figure in the area | region enclosed by the curve figure is detected, a calibration point can be detected reliably. Therefore, erroneous detection of the calibration index can be prevented.

  In addition, another characteristic configuration of the calibration method of the in-vehicle camera according to the present invention for achieving the above object includes an imaging step of imaging a calibration index disposed opposite to the in-vehicle camera with the in-vehicle camera, An edge is detected from a captured image including the calibration index obtained in the imaging process, and a curved line figure formed by a curve surrounding a predetermined area based on the detected edge and a straight line including at least two straight lines A calibration that detects intersections of figures and sets the intersections as calibration points when the intersections are detected within an area surrounded by the curved figures or when the curved figures are detected around the intersections. A point setting step.

  With such a configuration, the calibration point is determined by determining the validity of the intersection point based on the arrangement relationship between the curved figure and the intersection point, as in the other characteristic configuration of the above-described on-vehicle camera calibration device. A point can be detected accurately. Therefore, erroneous detection of the calibration index can be prevented.

Furthermore, in the present invention, a program suitably used for a calibration apparatus that configures an in-vehicle camera using the calibration index according to the present invention is also included in the scope of rights, and the characteristic configuration of the program is arranged opposite to the in-vehicle camera. A calibration index having a curved line figure formed by a curve surrounding a predetermined area, a straight line figure formed of an angle with respect to a horizontal plane of the field of view of the in-vehicle camera and a vertical plane perpendicular to the horizontal plane and a imaging step of imaging in the in-vehicle camera,
An edge is detected from a captured image including the calibration index obtained in the imaging step, and a curved figure formed by a curve surrounding a predetermined region is detected based on the detected edge, and then surrounded by the curved figure. was the area as the search area, in the search region, even without least on the basis of the detected edge to detect the intersection of the straight line shape composed of two straight lines, sets the intersection detected as a calibration point calibration The point setting process is performed by a computer.

In the present invention, another characteristic configuration of the program suitably used for the computer provided in the calibration apparatus that configures the in-vehicle camera using the calibration index according to the present invention is arranged to face the in-vehicle camera. An image capturing function for capturing a calibration index with the in-vehicle camera, and an edge detected from a captured image including the calibration index obtained by the imaging function, and a curve surrounding a predetermined region based on the detected edge When the intersection is detected within a region surrounded by the curve graphic, or the curve around the intersection is detected. A calibration point setting function for setting the intersection point as a calibration point when a figure is detected is to cause the computer to execute.

  The program for such a calibration apparatus can also obtain the above-described effects as in the on-vehicle camera calibration method as the subject of the present invention described above, and has various additional characteristic configurations described above. Can be provided.

It is a figure which shows the calibration parameter | index which concerns on this embodiment. It is a figure which shows the positional relationship of a vehicle and a calibration parameter | index. It is a block diagram which shows the outline of the calibration apparatus of the vehicle-mounted camera which calibrates a vehicle-mounted camera using the calibration parameter | index which concerns on this embodiment. It is a figure which shows an example of the calibration parameter | index which concerns on other embodiment. It is a figure which shows an example of the calibration parameter | index which concerns on other embodiment. It is a bird's-eye view which shows the positional relationship of the vehicle which concerns on other embodiment, and a calibration parameter | index. It is a figure which shows an example of the calibration parameter | index which concerns on other embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. The in-vehicle camera 20 mounted on the vehicle 100 is used to acquire a rear image of the vehicle 100 that is used, for example, when traveling backward or when assisting a user's parking operation. Such a rear image is acquired as a captured image by the in-vehicle camera 20, and is also used for calculation for obtaining a distance between the vehicle 100 and an obstacle included in the captured image using image recognition processing. However, when the optical axis of the in-vehicle camera 20 deviates from a preset value (for example, a design value), the distance between the result obtained by calculating the distance from the vehicle 100 to the obstacle and the actual distance However, a shift occurs. In such a situation, if the vehicle 100 is driven based on the calculation, there is a risk of colliding with an obstacle.

  Therefore, after the in-vehicle camera 20 is attached to the vehicle 100, the optical axis is calibrated at the factory. The calibration index 10 according to the present embodiment is used when calibrating the in-vehicle camera 20 in such a factory. Hereinafter, it demonstrates using drawing. The calibration of the in-vehicle camera 20 in the present embodiment is not a calibration by changing the physical position (position in the real space) of the in-vehicle camera 20, but the position where the in-vehicle camera 20 is disposed, The deviation of the optical axis of the in-vehicle camera 20 is calibrated (corrected) by calculation based on the difference between the angle and a preset setting value.

  FIG. 1 is a diagram showing a calibration index 10 according to the present embodiment. The calibration index 10 includes a curved figure 2, a linear figure 4, and an enclosing figure 6, and is formed in the same plane. The curved figure 2 is formed by a curve surrounding a predetermined area. The predetermined region is not particularly limited and is a region surrounded by a curve. In the calibration index 10 shown in FIG. 1, the curved figure 2 corresponds to a circle 2a. Therefore, in the present embodiment, the curved figure 2 corresponds to a circle 2a surrounding a predetermined area. It should be noted that such a curve surrounding the predetermined region is easy to understand when considered as one closed curve (closed curve) when the circle 2a is used. Although details will be described later, the curve surrounding the predetermined region is not limited to only one closed curve.

  The straight line figure 4 is composed of at least two straight lines that form intersections in a region surrounded by the curved line figure 2. The area surrounded by the curved figure 2 corresponds to the area surrounded by the circle 2a as described above. In the calibration index 10 shown in FIG. 1, the straight line figure 4 corresponds to two straight lines 4a and 4b. Here, it is sufficient that the calibration figure 10 according to the present invention has at least two linear figures 4, and in the present embodiment, description will be made assuming that the linear figure 4 is composed of two. Such two straight lines 4a and 4b are arranged so as to intersect within the region surrounded by the circle 2a to form an intersection point 8.

  Here, the straight lines 4a and 4b that form such a linear figure 4 are formed at an angle with respect to a horizontal plane in the field of view of the in-vehicle camera 20 and a vertical plane perpendicular to the horizontal plane. For example, as shown in FIG. 1, it is preferable that the two straight lines 4a and 4b have an angle of about 45 degrees with respect to the horizontal and vertical planes and are orthogonal to each other.

  The surrounding figure 6 is arranged so that the curved figure 2 and the linear figure 4 are surrounded by figures having other contours. In the present embodiment, the figure having another contour corresponds to the quadrangle 6a. Therefore, the surrounding figure 6 is arranged so that the curved figure 2 and the straight line figure 4 are surrounded by the quadrangle 6a.

  A region 12 surrounded by the surrounding figure 6 and delimited by the curve forming the curved figure 2 and the straight lines 4a and 4b forming the linear figure 4 is colored with a check pattern. That is, the check pattern is that adjacent areas divided by lines are colored with different colors (in this example, two colors of white and black). In the present embodiment, the surrounding figure 6 is a quadrangle 6a. In other words, being surrounded by the surrounding figure 6 means being surrounded by the quadrangle 6a. The curved figure 2 is a circle 2a. That is, the curve forming the curved figure 2 corresponds to the curve forming the circle 2a. Therefore, the region 12 surrounded by the surrounding figure 6 and delimited by the curve forming the curved figure 2 and the straight lines 4a and 4b forming the straight figure 4 is surrounded by the square 6a, and the circle 2a and the straight line The region 12 divided into 4a and 4b corresponds. In FIG. 1, the region 12 is divided into eight regions 12a-12h. As shown in FIG. 1, these regions 12a-12h are colored with a check pattern so as to have different colors from the adjacent regions. The check pattern is not particularly limited, but for example, it is preferable to use a combination of clear colors such as a combination of white and black or a combination of blue and red. Of course, it is also possible to color by combining other colors.

  The calibration index 10 configured as described above is used after being photographed by the in-vehicle camera 20 as will be described in detail later. And the image processing of the captured image acquired by imaging | photography is performed, and the calibration of the vehicle-mounted camera 20 is performed using the result of the said image processing. Therefore, the calibration index 10 is preferably an appropriate size. For example, it is preferable that the length of one side of the quadrangle 6a for calibrating the surrounding figure 6 is about 400 mm.

  Next, a calibration method for the in-vehicle camera 20 using the calibration index 10 according to the present embodiment will be described. FIG. 2A is a bird's-eye view showing the positional relationship between the vehicle 100 and the calibration index 10. FIG. 2B is an elevation view showing the positional relationship between the vehicle 100 and the calibration index 10. Here, the vehicle-mounted camera 20 configured in the present embodiment is a back camera that captures the rear of the vehicle 100. As shown in FIG. 2, the in-vehicle camera 20 is disposed in the vicinity of a license plate provided in the outer rear portion of the vehicle 100 or in the vicinity of an emblem provided in the outer rear portion of the vehicle 100. In FIG. 2, the size ratio between the vehicle 100 and the calibration index 10 is neglected in order to clarify the calibration index 10 according to the present invention more clearly.

  The calibration indices 10 are opposed to the in-vehicle camera 20 and are arranged in pairs with a distance between them. That is, as shown in FIGS. 2A and 2B, two are disposed within the field of view of the in-vehicle camera 20. Such a calibration index 10 is arranged on a virtual plane such that a pair of the calibration indices 10a and 10b has orthogonal distances from the rear end surface 100a of the vehicle 100 to predetermined distances L1 and L2. For example, it is preferable to arrange in a partition form. Further, the calibration indices 10a and 10b are arranged apart from each other. In the present embodiment, the calibration indices 10a and 10b are arranged such that the centers thereof are separated from the center line 100b of the vehicle 100 by W1 and W2, respectively. Further, the calibration indices 10a and 10b are arranged such that their centers are separated from the floor surface 100c on which the vehicle 100 is parked by H1 and H2, respectively.

  In calibrating the in-vehicle camera 20, the calibration indices 10a and 10b are arranged as described above. Note that W1 and W2, L1 and L2, H1 and H2 can be arranged to have the same value, or can be arranged to have different values. By using such a calibration index 10, it is possible to detect a calibration point without erroneous detection when the on-vehicle camera 20 is calibrated.

  FIG. 3 is a block diagram schematically showing a configuration of a calibration apparatus 200 that can configure the vehicle-mounted camera 20 using the calibration index 10 according to the present embodiment. The calibration device 200 includes an image receiving unit 40, an image output unit 30, a system control unit 50, an input I / F unit 60, an output I / F unit 70, and an image processing unit 80. The calibration device 200 configured as described above is configured by hardware or software or both of the above-described functional units for performing various processes for calibrating the in-vehicle camera 20 using a CPU as a core member.

  The image receiving unit 40 receives a captured image obtained by imaging the pair of calibration indices 10a and 10b with the in-vehicle camera 20, and stores the captured image in the storage unit 40a. The storage unit 40a corresponds to the image memory 40a in the present embodiment. Therefore, the image receiving unit 40 stores the captured image acquired by the in-vehicle camera 20 in the image memory 40a. Here, the process of imaging the calibration indices 10a and 10b with the in-vehicle camera 20 is referred to as an imaging process, and the process of receiving the captured image obtained in the imaging process and storing it in the image memory 40a is referred to as an image receiving process. Is done.

  The image output unit 30 includes a display controller 30a and a drawing unit 30b, and generates a video signal for displaying a captured image on the display 25 or the like. The drawing unit 30b draws the verification result of the calibration accuracy and the like based on the result of image processing to be described later. The display controller 30a superimposes the superimpose on the captured image and outputs a video signal.

  The input I / F unit 60 receives an input signal such as an instruction to start calibration of the in-vehicle camera 20 from a system higher than the calibration device 200 and transmits the input signal to the system control unit 50. Here, the host system corresponds to an assembly adjustment system of the vehicle 100, a back monitor system of the vehicle 100, and the like. The system control unit 50 controls the entire calibration apparatus 200, and controls an image processing unit 80 and the like, which will be described later, based on the above-described instruction to start calibration. The output I / F unit 70 receives an image processing result or the like via the system control unit 50, and outputs an output signal to a host system or the like.

  The image processing unit 80 includes an image acquisition unit 80a, a region setting unit 80b, a calibration index detection unit 80c, a calibration point specification unit 80d, and a correction camera angle calculation unit 80e. When the image acquisition unit 80a receives a calibration instruction from the system control unit 50, the image acquisition unit 80a acquires a captured image corresponding to the instruction from the image memory 40a.

  The area setting unit 80b includes a known mounting position of the in-vehicle camera 20, a setting value of the mounting angle of the in-vehicle camera 20, and the arrangement positions of the calibration indices 10a and 10b, a known mounting position of the in-vehicle camera 20, and the in-vehicle position. A calibration index existence area where a calibration index is predicted to exist due to variations in the mounting angle of the camera 20 is calculated for each calibration index 10a, 10b, and an image processing area is set for each calibration index 10a, 10b.

  The calibration index detection unit 80c detects the curved graphic 2 constituting the configuration indexes 10a and 10b based on the edge (contour line) of the image processing region set by the region setting unit 80b. This detection is performed in the order of the edge detection step and the curved figure detection step. Hereinafter, each process will be described.

  In the edge detection step, an edge included in the captured image stored in the image memory 40a is detected. Here, the image processing region is set by the region setting unit 80b as described above. In the edge detection step, edge detection is performed on the image processing region using an edge filter. Since edge detection is a well-known technique, a detailed description thereof is omitted, but it is preferable to perform horizontal scanning and vertical scanning on the image processing area in order to increase detection accuracy. The edge detected by the edge detection step is a set of points indicating the edge and is referred to as an edge point group.

  In the curved figure detection step, the curved figure 2 is detected based on the detected edge. In the present embodiment, the curved graphic 2 is a circle 2a. Therefore, in the curved figure detection step, the circle 2a is detected based on the edge detected by the edge detection step. Here, even if the circle 2a is a perfect circle when viewed from the front, depending on the relationship between the captured position and the position of the in-vehicle camera 20, the captured image may have a shape close to an ellipse. For this reason, in the curve graphic process, the circle 2a is detected using an ellipse detection technique. In this embodiment, an ellipse fitting process using a RANSAC (Random Sample Consensus) technique is used as such an ellipse detection technique. Since the RANSAC method is a known technique, it will be briefly described below.

  First, five points are randomly selected from a group of edge points indicating edges detected by the edge detection step. Next, an elliptic formula that passes through the selected five points is calculated and set as an elliptic model. Then, it is evaluated (fitness evaluation) how much the points other than the five points used for setting the elliptic model in the edge point group fit to the elliptic model. Such selection of five points, setting of an elliptic model, and fit evaluation are repeated. The ellipse defined by the best-fit ellipse model is determined to be the shape indicated by the edge point group with respect to the result repeatedly performed in this manner. The curve figure 2 is detected by such a curve figure detection step.

  The calibration point specifying unit 80d detects two straight lines from the edge point group existing inside the ellipse fitted by the curved figure detection process, and detects the intersection of the detected straight lines as a calibration point. This detection is performed in the order of a distortion correction step, a straight line figure detection step, and a calibration point setting step. Hereinafter, each process will be described.

  In the distortion correction step, the distortion of the captured image is corrected. The captured image is acquired by the in-vehicle camera 20 as described above, but the captured image immediately after imaging (hereinafter referred to as a raw captured image) includes distortion due to lens distortion of the lens of the in-vehicle camera 20. For example, even if the subject is a straight line, the raw captured image has a curved shape. For this reason, distortion correction processing is performed in order to correctly display the subject as well as the linear shape. In the present embodiment, in order to reduce the calculation processing load, the distortion of the edge point group in the region surrounded by the curved figure 2 detected by the edge detection process as the detection target of the linear shape is corrected. Since distortion correction is a well-known technique, detailed description thereof is omitted.

  In the straight line figure detecting step, the straight line figure 4 in the area surrounded by the detected curved line figure 2 is detected. That is, the arrangement relationship between the curved graphic 2 and the straight graphic 4 on the calibration index 10 is known, and the calibration apparatus 200 detects the linear graphic 4 based on the known arrangement. Here, “the arrangement relationship is known” may be a simple arrangement relationship (for example, an arrangement relationship in a closed curve), or the relative position relationship may be defined by a predetermined numerical value. The detected curve figure 2 is the curve figure 2 subjected to the above-described distortion correction processing. In the present embodiment, the straight line figure 4 is two straight lines 4a and 4b. Therefore, in the straight line figure detection step, the two straight lines 4a and 4b in the region surrounded by the curved line figure 2 subjected to the distortion correction process are detected. By doing in this way, the search area | region of the straight lines 4a and 4b can be limited, and it can detect reliably, reducing calculation cost.

Also in this linear figure detection process, the RANSAC method is used as in the above-described curved figure detection process. A brief description is given below.
First, one straight line 4a is detected. Two points are randomly selected from the edge point group on which the distortion correction processing has been performed. Next, a straight line expression passing through the selected two points is calculated and set as a straight line model. Then, it is evaluated how much the points other than the two points used for setting the linear model in the edge point group fit to the linear model (fitness evaluation). Such selection of two points, setting of a linear model, and fit evaluation are repeated. A straight line defined by the most fitted straight line model is determined as one straight line 4a with respect to the result repeatedly performed in this way.

  Next, the other straight line 4b is detected. Here, in the detection of the other straight line 4b, the remaining edge point group obtained by removing the edge point group used when determining as one straight line 4a from the edge point group subjected to the distortion correction processing. Is the target. First, two points are randomly selected from the target edge point group. Next, a straight line expression passing through the selected two points is calculated and set as a straight line model. Then, it is evaluated how much the points other than the two points used for setting the linear model in the target edge point group fit to the linear model (fitness evaluation). Such selection of two points, setting of a linear model, and fit evaluation are repeated. A straight line defined by the most fitted straight line model is determined as the other straight line 4b with respect to the result repeatedly performed in this manner. The linear figure 4 is detected by such a linear figure detection process.

  In the calibration point setting step, the intersection point of the detected straight line figure 4 is detected, and the intersection point is set as a calibration point used for calibration of the in-vehicle camera 20. The detected straight line figure 4 corresponds to the two straight lines 4a and 4b detected by the above-described straight line figure detecting step. In the calibration point setting step, the intersection of the two straight lines 4a and 4b is detected, and the intersection is set as a calibration point. In this way, the calibration point specifying unit 80d sets a calibration point used when calibrating the in-vehicle camera 20.

  The correction camera angle calculation unit 80e calculates the left and right calibrations on the virtual image from the known mounting position of the in-vehicle camera 20, the setting value of the mounting angle in which the in-vehicle camera 20 is mounted, and the arrangement positions of the calibration indices 10a and 10b. The corresponding point corresponding to the point is calculated, and the correction camera parameter is calculated from the difference from the calibration point set by the calibration point specifying unit 80d. This calculation is performed by a correction camera parameter calculation process. The corrected camera parameter means a difference between a design value and an actual value in the camera parameter. Therefore, the actual value is corrected by correcting the design value with the correction camera parameter. This will be described below.

  In the correction camera angle calculation step, a pair of calibration points set from a pair of calibration indices, a pair of setting points preset according to the mounting position of the in-vehicle camera 20, and a pair of calibrations included in the captured image Based on the points, a correction camera angle for correcting the deviation of the optical axis included when the vehicle-mounted camera 20 is mounted on the vehicle 100 is calculated. Corresponding to a pair of calibration points set from a pair of calibration indices, a pair of setting points preset according to the mounting position of the in-vehicle camera 20 is a pair of calibration points corresponding to the left and right calibration points on the virtual image. Corresponding points correspond. A difference between the pair of corresponding points and the pair of calibration points is calculated, and a correction camera angle that corrects the deviation of the optical axis included when the vehicle-mounted camera 20 is mounted on the vehicle 100 is calculated based on the difference.

  The correction camera angle is used to correct the angle of the in-vehicle camera 20. Here, the corrected camera angle is obtained by displaying a captured image acquired by the in-vehicle camera 20 on the display 25 and rendering a predetermined drawing (for example, parking the vehicle 100 in a parking space or traveling backward) on the captured image. This is preferably used when a known parking assistance device or a driving assistance device that assists the driving of the driver in superimposing a predicted route line that predicts the course of the vehicle 100). That is, when the angle of the in-vehicle camera 20 is deviated from the design value, if the captured image of the in-vehicle camera 20 and the drawing drawn from the design value are superimposed, the captured image and the drawing are displaced, and the driver I misunderstand it. In such a case, the drawing is corrected based on the corrected camera angle so as to be suitable for an actual captured image (captured image captured by the in-vehicle camera 20 installed at the camera angle deviated from the design value). For this reason, the predetermined drawing can be accurately superimposed on the captured image acquired by the in-vehicle camera 20.

  Further, the correction camera angle can be used to correct a captured image by calculating an angle for correcting the angle of the in-vehicle camera 20. The angle of the in-vehicle camera 20 is the angle (rotation angle) of the in-vehicle camera 20 along the vertical direction of the lens of the in-vehicle camera 20, the vertical angle (elevation angle) of the in-vehicle camera 20, and the horizontal angle of the in-vehicle camera 20 ( Azimuth). Therefore, the angle for correcting the angle of the in-vehicle camera 20 is the rotation angle (Roll angle) for correcting the angle of the in-vehicle camera 20 along the vertical direction of the lens of the in-vehicle camera 20, and the vertical angle of the in-vehicle camera 20 is corrected. The elevation angle (Tilt angle) and the azimuth angle (Pan angle) for correcting the horizontal angle of the in-vehicle camera 20 can be corrected. By using the corrected camera angle, each angle of the in-vehicle camera 20 can be corrected. That is, the captured image captured by the in-vehicle camera 20 is rotated (plane rotation) in accordance with the Roll angle, the captured image rotated in accordance with the Roll angle is adjusted to the tilt angle, and the tilt angle is adjusted to the tilt angle. In addition, adjust the horizontal angle. By correcting the picked-up image, it is possible to accurately superimpose the predetermined drawing on the picked-up image as described above, and it is also possible to suppress variation in the pick-up range for each product. As described above, according to the calibration apparatus 200, the optical axis deviation included when the in-vehicle camera 20 is mounted on the vehicle 100 is suitably used by the calibration method including the above-described steps using the calibration index 10 according to the present invention. It is also possible to calibrate (correct).

  Furthermore, as an example of use other than correcting the predetermined drawing, the captured image itself acquired by the in-vehicle camera 20 is corrected using the rotation angle, the elevation angle, and the azimuth angle for correcting the angle of the in-vehicle camera 20. It is also possible to display on the display 25. Alternatively, by correcting the position of a display object (for example, a lane or an object) included in the captured image acquired by the in-vehicle camera 20 using the rotation angle, the elevation angle, and the azimuth angle that correct the angle of the camera 20 described above. It is also possible to use it for specifying an accurate position.

[Other Embodiments]
In the above-described embodiment, it has been described that the curved figure 2 is composed of a circle 2a and the straight line figure 4 is composed of two straight lines 4a and 4b. However, the scope of application of the present invention is not limited to this. The curved graphic 2 may be a graphic other than the circle 2a, and the linear graphic 4 may be at least two straight lines, that is, three or more straight lines. For example, FIG. 4 shows an example of the calibration index 10 when the curved figure 2 uses a figure other than the circle 2a. The curved line figure 2 can be formed of a curved line 2b-2e having a convex shape with respect to the center direction of the surrounding figure 6 of the quadrangle 6a. In FIG. 4, the linear shape 4 is composed of two straight lines 4a and 4b. Then, the curve 2b-2e and the two straight lines 4a, 4b are formed in the same plane, and the calibration index 10 can be calibrated by coloring the regions 12i-12p delimited by these in a check pattern. is there. Even when such a calibration index 10 is used, the intersection point 8 formed by the two straight lines 4a and 4b in the region surrounded by the curved figure 2 is used as a calibration point for calibrating the in-vehicle camera 20. Of course, it is also possible to use.

  Thus, the “region surrounded by the curved graphic 2” according to the present invention does not mean only a region completely closed by the curved graphic 2, but by the curved graphic 2 as shown in FIG. It includes areas that are not completely closed. Therefore, even in the calibration index 10 shown in FIG. 4, according to the present invention, “a curve figure 2 formed by a curve surrounding a predetermined area and an intersection point are formed in the area surrounded by the curve figure 2”. It is naturally included that “the straight line figure 4 composed of at least two straight lines is formed in the same plane”. In the calibration index 10 shown in FIG. 4, for example, it can be considered that a completely closed region is formed by the curved figure 2 and the surrounding figure 6.

  Furthermore, as another example of the calibration index 10, a configuration as shown in FIG. As in the calibration index 10 shown in FIG. 5, it is also possible to form a region closed by a plurality of curved figures 2. Therefore, even in the calibration index 10 shown in FIG. 5, according to the present invention, “a curved line 2 formed by a curve surrounding a predetermined area and an intersection point are formed in the area surrounded by the curved line 2”. It is naturally included that “the straight line figure 4 composed of at least two straight lines is formed in the same plane”.

  In the above embodiment, particularly in FIG. 1, the intersection 8 is illustrated in the center of the circle 2 a. However, the scope of application of the present invention is not limited to this. The arrangement location of the intersection 8 in FIG. 1 is merely an example, and it is naturally possible to form the calibration index 10 by arranging the intersection 8 at a position that is not the center of the circle 2a (a position shifted from the center).

  In the above embodiment, the calibration index 10 has been described as being arranged in a partition shape so as to face the vehicle 100 as shown in FIG. However, the scope of application of the present invention is not limited to this. As shown in FIG. 6, the calibration index 10 may be disposed on the floor surface 100c where the vehicle 100 is parked, or may be painted on the floor surface 100c. Even if the calibration index 10 is disposed at such a position, it is naturally possible to calibrate the in-vehicle camera 20 as long as the in-vehicle camera 20 to be calibrated can capture the calibration index 10.

  In the above-described embodiment, the distortion correction processing has been described as correcting the distortion of the edge point group in the region surrounded by the curved figure 2 detected by the edge detection process as the detection target of the linear shape. However, the scope of application of the present invention is not limited to this. Naturally, it is also possible to perform the distortion correction processing over the entire surface of the captured image (raw captured image) immediately after being captured by the in-vehicle camera 20.

  In the above-described embodiment, the curved figure 2 and the linear figure 4 are described as being surrounded by the surrounding shape 6 including the quadrangle 6a. However, the scope of application of the present invention is not limited to this. The surrounding shape 6 is not limited to the quadrangle 6a, and may naturally be other shapes such as a circle and a polygon. Of course, the surrounding shape 6 does not surround the curved graphic 2 and the straight graphic 4. Even if the calibration index 10 is calibrated in this way, it is naturally possible to calibrate the in-vehicle camera 20 suitably.

  In the above embodiment, the straight lines 4a and 4b forming the straight line figure 4 are provided with an angle with respect to the horizontal plane of the field of view of the in-vehicle camera 20 and the vertical plane perpendicular to the horizontal plane, and the two straight lines 4a and 4b are It has been described as having an angle of approximately 45 degrees with respect to the horizontal and vertical planes and being orthogonal to each other. However, the scope of application of the present invention is not limited to this. As shown in FIG. 7, it is possible to arrange two straight lines 4a and 4b along a horizontal plane and a vertical plane, respectively. Alternatively, although not shown, it may be arranged at an angle other than 45 degrees. Further, the two straight lines 4a and 4b are not necessarily orthogonal, but may be anything that forms the intersection point 8. Even with such a calibration index 10, it is naturally possible to suitably calibrate the in-vehicle camera 20.

  The above embodiment has been described on the assumption that the distortion correction step is performed in order to correct the distortion of the captured image. However, the scope of application of the present invention is not limited to this. Of course, it is possible to omit the distortion correction step. In such a case, it is preferable to detect the straight line figure 4 by the RANSAC method in consideration of the distortion. In addition, although the distortion correction process has been described as being performed before the straight line graphic detection process, it is naturally possible to perform the distortion correction process before the edge detection process or the curved graphic detection process.

  In the above-described embodiment, the on-vehicle camera 20 to be calibrated has been described as being a back camera provided behind the vehicle 100. However, the scope of application of the present invention is not limited to this. A front camera that captures the front of the vehicle 100 can be set as a calibration target, and a side camera that captures the side of the vehicle 100 can also be set as a calibration target.

  In the above embodiment, in the straight line figure detecting step, two straight lines 4a and 4b are detected in the region surrounded by the curved line figure 2, and the intersection of the detected straight lines 4a and 4b is set as a calibration point. explained. However, the scope of application of the present invention is not limited to this. For example, a configuration in which the curved line figure 2 and the straight lines 4a, 4b or the intersections are separately detected and the arrangement relation between the curved line figure 2 and the straight lines 4a, 4b or the intersection points is compared to check the validity of the straight lines 4a, 4b or the intersection points Of course it is also possible to do. That is, the curved figure 2 is detected by the curved figure detection process based on the edge detected by the edge detection process. On the other hand, the linear figure 4 is detected by the linear figure detection process. Then, the calibration point setting step sets a calibration point when the straight line figure 4 detected by the straight line figure detection step is within the area surrounded by the curved line figure 2 detected by the curved line figure detection step. Of course, it is also possible to adopt a configuration.

  It is also possible to adopt a configuration in which the straight line 4a, 4b is detected by the straight line figure detecting step, the intersection point of the straight line 4a, 4b is detected, and then the curved line figure 2 around the intersection point is detected. That is, the same effect as described above can be obtained even if the intersection point is set as a calibration point only when the curved graphic 2 is detected around the detected intersection in the calibration point setting step.

  In the above embodiment, in the correction camera parameter calculation step, the rotation angle (Roll angle) for correcting the angle of the in-vehicle camera 20 along the vertical direction of the lens of the in-vehicle camera 20 and the elevation angle for correcting the vertical angle of the in-vehicle camera 20. (Tilt angle) It demonstrated as computing the azimuth angle (Pan angle) which correct | amends the angle of the horizontal direction of the vehicle-mounted camera 20. FIG. However, the scope of application of the present invention is not limited to this. Of course, it is also possible to calculate the translation amount of the in-vehicle camera 20 in the three-dimensional space (physical positional deviation of the in-vehicle camera 20 in the horizontal direction, the vertical direction, and the depth direction) as an external parameter of the in-vehicle camera 20.

  In the above embodiment, the calibration index 10 used for calibration of the in-vehicle camera 20, the calibration method of the in-vehicle camera 20 using the calibration index 10, and the program for the calibration device 200 using the calibration index 10 have been described. . In addition to these, the calibration apparatus 200 that calibrates the in-vehicle camera 20 using the calibration index 10 described above can naturally be included in the scope of the present invention.

  That is, a pair of calibration indices 10 that are opposed to the in-vehicle camera 20 and are spaced apart from each other, and an image receiving unit 40 that receives a captured image obtained by capturing the in-vehicle camera 20 and stores it in the storage unit 40a. An edge detection unit that detects an edge included in the captured image stored in the storage unit 40a, a curve graphic detection unit that detects the curved graphic 2 based on the detected edge, and a straight line that detects the linear graphic 4 A calibration point setting unit that detects an intersection of the detected linear figure 4 with the figure detection unit, and sets the intersection as a calibration point used for calibration of the in-vehicle camera 20 based on the intersection and the curve figure 2; Corresponding to a pair of calibration points set from the pair of calibration indices 20, the vehicle is mounted on the basis of a pair of set points preset according to the mounting position of the vehicle-mounted camera 20 and a pair of calibration points included in the captured image. turtle It is also possible to configure the calibration device 20 and a correction camera parameter calculating unit for calculating a correction camera parameter for correcting the displacement of the optical axis contained 20 when mounted on the vehicle 100.

  In such a case, as the calibration index 10, a curved figure 2 formed by a curve surrounding a predetermined area, and a linear figure 4 composed of at least two straight lines forming an intersection in the area surrounded by the curved figure 2; Are formed in the same plane, and it is preferable to use a region in which a region demarcated by a curve forming the curved graphic 2 and a straight line 4a, 4b forming the linear graphic 4 is colored with a check pattern. Of course, the above-described corrected camera angle calculation unit 80e can be used as the correction camera parameter calculation unit.

  INDUSTRIAL APPLICABILITY The present invention can be used for a program for a vehicle-mounted camera calibration device, a vehicle-mounted camera calibration method, and a vehicle-mounted camera calibration device.

2: Curved figure 4: Linear figure 4a, 4b: Straight line 8: Intersection 10: Calibration index 20: In-vehicle camera 40: Image receiving unit 80c: Calibration index detecting unit 100: Vehicle 200: Calibration apparatus

Claims (6)

At least two curves arranged opposite to the in-vehicle camera and provided with an angle with respect to a curved figure formed by a curve surrounding a predetermined area, a horizontal plane of the field of view of the in-vehicle camera, and a vertical plane perpendicular to the horizontal plane It has a calibration index with a straight line figure consisting of straight lines ,
An image receiving unit for receiving a captured image including the calibration index obtained by imaging with the in-vehicle camera;
An edge is detected from the captured image received by the image receiving unit, and after detecting a curved figure formed by a curve surrounding a predetermined area based on the detected edge, the inside of the area surrounded by the curved figure was a search region in the search region, even without least on the basis of the detected edge to detect the intersection of the straight line shape composed of two straight lines, the calibration target detection unit that sets the detected the intersection has a calibration point The system provided with the calibration apparatus of the vehicle-mounted camera which has these . An image receiving unit that receives a captured image obtained by imaging a calibration index disposed opposite to the in-vehicle camera with the in-vehicle camera;
Detecting an edge from the captured image received by the image receiving unit, and based on the detected edge, an intersection of a curved figure formed by a curve surrounding a predetermined region and a linear figure composed of at least two straight lines And when the intersection point is detected in the area surrounded by the curve figure, or when the curve figure is detected around the intersection point, a calibration index detection unit that sets the intersection point as a calibration point When,
On-vehicle camera calibration device comprising: At least two curves arranged opposite to the in-vehicle camera and provided with an angle with respect to a curved figure formed by a curve surrounding a predetermined area, a horizontal plane of the field of view of the in-vehicle camera, and a vertical plane perpendicular to the horizontal plane an imaging step of a calibration target having a linear shape made of straight lines, captured by the vehicle-mounted camera,
An edge is detected from a captured image including the calibration index obtained in the imaging step, and a curved figure formed by a curve surrounding a predetermined region is detected based on the detected edge, and then surrounded by the curved figure. was the area as the search area, in the search region, even without least on the basis of the detected edge to detect the intersection of the straight line shape composed of two straight lines, sets the intersection detected as a calibration point calibration A point setting process;
A calibration method for an in-vehicle camera. An imaging step of imaging a calibration index arranged opposite to the in-vehicle camera with the in-vehicle camera;
An edge is detected from a captured image including the calibration index obtained in the imaging step, and based on the detected edge, a curved figure formed by a curve surrounding a predetermined region and at least two straight lines are included. Each intersection of straight line figures is detected, and the intersection point is set as a calibration point when the intersection point is detected in an area surrounded by the curve figure or when the curve figure is detected around the intersection point. Calibration point setting process;
A calibration method for an in-vehicle camera. The program for making a computer implement each process prescribed | regulated by Claim 3 . A computer program for a calibration device for calibrating an in-vehicle camera,
An imaging function for imaging a calibration index disposed opposite to the in-vehicle camera with the in-vehicle camera,
An edge is detected from a captured image including the calibration index obtained by the imaging function, and based on the detected edge, a curved figure formed by a curve surrounding a predetermined region and at least two straight lines are included. Each intersection of straight line figures is detected, and the intersection point is set as a calibration point when the intersection point is detected in an area surrounded by the curve figure or when the curve figure is detected around the intersection point. Calibration point setting function,
Program for causing the computer to execute a.

Images