CN111145249A - Vehicle-mounted automatic calibration control method, storage medium and system - Google Patents
Vehicle-mounted automatic calibration control method, storage medium and system Download PDFInfo
- Publication number
- CN111145249A CN111145249A CN201910817490.7A CN201910817490A CN111145249A CN 111145249 A CN111145249 A CN 111145249A CN 201910817490 A CN201910817490 A CN 201910817490A CN 111145249 A CN111145249 A CN 111145249A
- Authority
- CN
- China
- Prior art keywords
- calibration
- point
- vehicle
- positioning
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000004590 computer program Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 3
- 239000003550 marker Substances 0.000 description 16
- 239000011159 matrix material Substances 0.000 description 14
- 238000012549 training Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 7
- 230000009466 transformation Effects 0.000 description 7
- 241000282414 Homo sapiens Species 0.000 description 5
- 244000241872 Lycium chinense Species 0.000 description 5
- 235000015468 Lycium chinense Nutrition 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 102000027426 receptor tyrosine kinases Human genes 0.000 description 3
- 108091008598 receptor tyrosine kinases Proteins 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002427 irreversible effect Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 241001061260 Emmelichthys struhsakeri Species 0.000 description 1
- 238000004026 adhesive bonding Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000002485 combustion reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000005357 flat glass Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000004126 nerve fiber Anatomy 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- VCMMXZQDRFWYSE-UHFFFAOYSA-N plumbagin Chemical compound C1=CC=C2C(=O)C(C)=CC(=O)C2=C1O VCMMXZQDRFWYSE-UHFFFAOYSA-N 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
- G01S19/43—Determining position using carrier phase measurements, e.g. kinematic positioning; using long or short baseline interferometry
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of image processing, and discloses a vehicle-mounted-based automatic calibration control method which comprises the following steps: acquiring video image data, identifying the number of the positioning points, and acquiring positioning data of the vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; and triggering calibration when the included angle is less than or equal to 90 degrees and less than or equal to a first threshold value. Corresponding storage media and systems are also disclosed. Some technical effects of the invention are as follows: and automatic control of camera calibration is realized.
Description
Technical Field
The invention relates to the field of image processing, in particular to an automatic calibration control method in the field of image processing.
Background
Vision is an important means for human beings to observe the world and to know the world, and accounts for 70% of the information that human beings obtain from the external environment. Human beings obtain the light that peripheral object reflects or oneself sends with eyes, and the light forms the image on the retina, passes through nerve fiber and conveys to the brain, and the brain processes and understands the visual information, forms the vision finally. The computer vision simulates the function of human vision, the camera is used for acquiring images of the surrounding environment, and the computer is used for processing the images. The computer vision can complete the work which can not be qualified by the human vision, such as the accurate measurement of the size, the distance and the like of the object to be measured. The computer vision technology can be widely applied to the fields of surveying and mapping, vision detection, automatic driving and the like.
One of the basic tasks of computer vision is to calculate geometric information of an object in three-dimensional space from image information acquired by a camera, and to reconstruct or recognize the object therefrom, and further to recognize the real world. Wherein, camera calibration is a necessary way to accomplish the task. Coordinate information of the index point is acquired by recognizing the index object. The camera calibration method can be divided into a conventional calibration method and a self-calibration method according to whether a calibration object is required. The traditional camera calibration method is to take a calibration object with a known shape and size as a shooting object of a camera, then perform correlation processing on the shot image, and use a series of mathematical transformations to solve internal and external parameters of a camera model. The camera self-calibration method does not need a calibration object and only depends on the relationship between corresponding points of a plurality of images to directly calibrate. So far, the self-calibration method is flexible, but because the unknown parameters involved in calibration are too many, stable results are difficult to obtain. On the contrary, the traditional calibration method is mature, the precision of the calibration result is high, and the method is widely applied.
For the traditional camera calibration technology, the extraction of the coordinates of the feature points on the calibration object is an irrecoverable step, and meanwhile, the positioning precision of the feature points has an important influence on the final calibration result. With the expansion of the application range of the camera calibration technology, the applied field environment has a diversified trend, for example, the conditions of factory environment, outdoor large background environment or coexistence of multiple calibrators, etc., cause the problems of insufficient accuracy of feature point extraction or incapability of effectively completing extraction at all when processing images. In the conventional camera calibration method, a calibration object is an essential component. In general, a calibration object as a reference object should satisfy the following basic requirements: firstly, in image processing, the image characteristic part of the calibration object should be easy to identify, namely, the reference object should have a more distinct difference compared with the background environment; secondly, the characteristic part of the reference object should be easy to extract when image processing is carried out. Calibration references generally fall into two broad categories: three-dimensional calibration object and two-dimensional plane calibration object. Three-dimensional volumetric landmarks are typically cube, solid, single-colored small squares. The traditional camera calibration method also has the problem of usability, and the current popular method can be completed only by manual interaction. The manual interaction participation degree is high, so that the traditional calibration method has low repeatability, and the manual step needs to be repeated once for calibration.
Disclosure of Invention
In order to at least solve the problem of automatic control of camera calibration, the invention provides a vehicle-mounted automatic calibration control method, which has the following technical scheme:
acquiring image data, identifying the number of the positioning points, and acquiring positioning data of the vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; and triggering calibration when the included angle is less than or equal to 90 degrees and less than or equal to a first threshold value.
Preferably, the positioning data of the vehicle-mounted positioning points are acquired after all the positioning points are identified.
Preferably, the positioning data is RTK positioning data.
Preferably, the first calibration distance is less than or equal to the maximum calibration working radius, and the second calibration distance is greater than or equal to the minimum calibration working radius.
Preferably, the maximum nominal working radius is 8m and the minimum nominal working radius is 3 m.
Preferably, the first threshold is 5 °.
Preferably, the method further comprises the following steps: identifying any one of the calibration points, and calculating theoretical world coordinates of the calibration point; and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters and terminating the calibration.
Preferably, the second threshold is 20 cm.
Accordingly, the present invention discloses a readable storage medium having stored thereon a computer program performing the aforementioned method.
The invention also discloses an automatic calibration control system which comprises a calibration point identification module, a vehicle body course judgment module, a trigger module and a verification module. The index point identification module is used for acquiring and processing image data and identifying index points; the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points; the trigger module is used for sending a calibration instruction and starting calibration; the calibration module is used for calibrating the calibration result.
The method, the storage medium and the device provided by the invention at least provide a technical scheme for automatically controlling the calibration of the camera on one hand, and can well realize the automatic control during the calibration of the camera.
Drawings
For a better understanding of the technical solution of the present invention, reference is made to the following drawings, which are included to assist in describing the prior art or embodiments. These drawings will selectively demonstrate articles of manufacture or methods related to either the prior art or some embodiments of the invention. The basic information for these figures is as follows:
FIG. 1 is a flow chart of an embodiment of a vehicle-based automatic calibration control method.
FIG. 2 is a schematic diagram illustrating an exemplary location of a calibration point.
Fig. 3 is a schematic diagram of an installation position of an RTK positioning point in one embodiment.
FIG. 4 is a diagram illustrating the calculation of coordinates of index points according to an embodiment.
FIG. 5 is a schematic view of the pavement arrangement of the calibration object in one embodiment.
FIG. 6 is a schematic diagram illustrating a predetermined marker placement in one embodiment.
Detailed Description
The technical means or technical effects related to the present invention will be further described below, and it is obvious that the examples provided are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step, will be within the scope of the present invention based on the embodiments of the present invention and the explicit or implicit representations or hints.
On the general idea, the invention discloses a vehicle-mounted automatic calibration control method, which comprises the following steps: acquiring image data, identifying the number of the positioning points, and acquiring positioning data of the vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; and triggering calibration when the included angle is less than or equal to 90 degrees and less than or equal to a first threshold value.
Accordingly, in one embodiment, the invention proposes a readable storage medium having stored thereon a computer program which executes the aforementioned method.
Correspondingly, in one embodiment, the invention provides an automatic calibration control system which comprises a calibration point identification module, a vehicle body course judgment module, a trigger module and a verification module. The index point identification module is used for acquiring and processing image data and identifying index points; the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points; the trigger module is used for sending a calibration instruction and starting calibration; the calibration module is used for calibrating the calibration result.
Based on the general concept, those skilled in the art should understand that the vehicle in the vehicle of the present invention refers to a vehicle driven or towed by a power device, and the power is generally from an internal combustion engine or an electric motor. The positioning data refers to position information provided by GNSS. GNSS, a satellite navigation system, includes, but is not limited to, GPS in the united states, GLONASS in russia, Galileo in the european union, and BDS in china.
Some technical effects of the invention are as follows: the automatic control in the camera calibration process is realized, the participation of manual intervention is reduced, and the repeatability is better.
In some embodiments, as shown in fig. 2 to 4, a camera or an apparatus having a camera function is mounted and fixed on a vehicle. Generally, a camera or an apparatus having an image pickup function is mounted and fixed to a front of a vehicle, particularly, a front window glass of the vehicle, so as to obtain a good working view environment. The device with the camera function refers to a device capable of shooting and acquiring image data such as videos or pictures, such as an acquisition terminal device for acquiring map data in the mapping field and a terminal device for visually identifying road conditions in the automatic driving field.
In some embodiments, as shown in fig. 3 and fig. 4, vehicle-mounted positioning points are set on the vehicle, and the vehicle-mounted positioning points are used for acquiring positioning data of the vehicle in real time or non-real time. Generally, the vehicle-mounted positioning point may be an existing navigation device of the vehicle, or may be another positioning device additionally selected and set at another position. In one embodiment, the center of the selected roof is set as a vehicle-mounted positioning point, and a positioning device is installed.
In some embodiments, as shown in fig. 2 to 5, the calibration points are symmetrically arranged on two sides of the road surface, and the line connecting any two calibration points on each side is parallel to the median line of the road surface. The advantage of this arrangement is that subsequent data processing is facilitated.
In some embodiments, a positioning device is provided on the index point, and the positioning device is used for receiving positioning data of the acquired and transmitted index point.
In some embodiments, the positioning data of all positioning points can be obtained through pre-measurement, and then the positioning data is used for subsequent use.
It is understood that the above embodiment operations may be one-off, i.e. set for the first time, and then may not need to be reset without environmental change; when the calibration is performed again, the operation steps of the above embodiment can be omitted.
In some embodiments, video image data is acquired, the number of the positioning points is identified, and positioning data of the vehicle-mounted positioning points is acquired; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; and triggering calibration when the included angle is less than or equal to 90 degrees and less than or equal to a first threshold value.
In some embodiments, a camera or a device having a camera function mounted in advance on a vehicle operates to generate image data. The image data may be any one or a combination of video and pictures.
In some embodiments, the image data further includes timestamp information per frame, positioning data information of the vehicle-mounted positioning point at the shooting time.
In some embodiments, the image data is processed to identify the number of the positioning points, and meanwhile, the positioning data of the vehicle-mounted positioning points is acquired. Recognition as used herein refers to the use of image processing techniques to resolve the index points in the image data.
In some embodiments, when all the calibration points are identified, the acquisition of the positioning data of the vehicle-mounted positioning point is started, so that the data volume can be reduced.
In some embodiments, the acquired positioning data of the vehicle-mounted positioning points is RTK positioning data. An RTK (real-time kinematic) real-time dynamic measurement is one of relative positioning technologies, and high-precision dynamic relative positioning is realized mainly through a real-time data link between a reference station and a rover station and a carrier relative positioning fast resolving technology.
In some embodiments, the vehicle advances, the index point enters the working radius of the camera, and image data are captured and processed; and identifying all the calibration points from the image data, and starting to acquire the positioning data of the vehicle-mounted positioning points and the positioning data of the calibration points. And calculating the distance from the vehicle-mounted positioning point to any positioning point according to the positioning data of the vehicle-mounted positioning point and the positioning data of the positioning point. The vehicle continues to advance, and the calibration point enters the camera to calibrate the working radius. And identifying all calibration points at the first calibration distance, and acquiring and recording positioning data of the first calibration position at the moment. And identifying all the calibration points at the second calibration distance, and acquiring and recording the positioning data of the second calibration position at the moment. The first nominal distance is less than or equal to the maximum nominal working radius and the second nominal distance is greater than or equal to the minimum nominal working radius. The camera working radius refers to the maximum working distance at which the camera can acquire image data that meets the image processing work requirements. The calibration working radius refers to the working distance at which the camera can acquire image data meeting the calibration working requirement.
In some embodiments, the maximum calibrated working radius is 8m and the minimum calibrated working radius is 3 m.
In some embodiments, the positioning data of the calibration point can be obtained from the outside in advance and manually input into the automatic calibration control system; or the calibration point can be sent by a positioning device of the calibration point, and the automatic calibration control system receives input.
In some embodiments, a body heading is calculated based on the positioning data for the first and second nominal positions. The heading of the vehicle body here refers to a spatial vector characterizing the heading of the vehicle.
In some embodiments, when the RTK positioning data of the first calibration position O point is obtained through the vehicle-mounted positioning point, an accurate O point world coordinate (x) is obtained1，y1，z1) (ii) a Obtaining RTK positioning data of the O 'point of the second calibration position through the vehicle-mounted positioning point to obtain an accurate world coordinate (x) of the O' point2，y2，z2). Subtracting the world coordinate systems to obtain a vector
In some embodiments, an angle is calculated between the heading of the vehicle body and a line connecting the first and second calibration points.
In some embodiments, a first calibration point A is obtained1And a second index point A2World coordinates of (a) to obtain a vector
By vector
Sum vector
And calculating the included angle between the vehicle body course and the connecting line of the first calibration point and the second calibration point. Wherein, the first index point A1And a second index point A2Are symmetrical calibration points respectively positioned at two sides of the center line of the road surface.
In some embodiments, when the included angle-90 ≦ the first threshold value is detected, a calibration work instruction is issued to the calibration apparatus, triggering calibration.
In some embodiments, a first calibration point A is obtained1And a third index point A3World coordinates of (a) to obtain a vector
By vector
Sum vector
And calculating the included angle between the vehicle body course and the connecting line of the first calibration point and the third calibration point. Wherein, the first index point A1And a third index point A3Is two calibration points respectively positioned on the same side of the road surface, and a straight line A1A3Parallel to the road surface midline. At the moment, the calibration is triggered by setting the included angle to be less than or equal to a first threshold value.
In some embodiments, the first threshold is set at 5 °.
In some embodiments, after triggering calibration, calibration parameters are calculated, including but not limited to camera internal reference and pose parameters. Then, according to the calibration parameters, any one calibration point is identified by calibration equipment, and the theoretical world coordinate of the calibration point is calculated; and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters and terminating the calibration.
The theoretical world coordinate of the calibration point refers to the actual world coordinate of the calibration point which can be obtained from the outside in advance and manually input into the automatic calibration control system; or the calibration point can be sent by a positioning device of the calibration point, and the control system is automatically calibrated to receive and input. The absolute value of the error between the theoretical world coordinate and the actual world coordinate of the index point is calculated, and the theoretical world coordinate (x) of the index point is setTheory of things，yTheory of things，zTheory of things) Real coordinate system (x)Fruit of Chinese wolfberry，yFruit of Chinese wolfberry，zFruit of Chinese wolfberry) Then there is the absolute value of the error | x of the corresponding coordinate axisTheory of things-xFruit of Chinese wolfberry|、|yTheory of things-yFruit of Chinese wolfberryI and I zTheory of things-zFruit of Chinese wolfberryL. When, | xTheory of things-xFruit of Chinese wolfberry|、|yTheory of things-yFruit of Chinese wolfberryI and I zTheory of things-zFruit of Chinese wolfberryAnd when all the I are smaller than the second threshold value, storing the calibration parameters, and completing and terminating the calibration.
In some embodiments, the second threshold is set at 20 cm.
In some embodiments, the vehicle-mounted positioning point RTK is used0The RTK is arranged at the top of the vehicle body, and can acquire world coordinates of the positioning point in real time. Respectively setting 6 point RTKs by using the RTKs on two sides of a road 5-10 m in front of a vehicle1、RTK2、RTK3、RTK4、RTK5、RTK6World coordinate Aw1，Aw2，Aw3，Aw4，Aw5，Aw6Respectively correspond to the calibration points A1，A2，A3，A4，A5，A6。RTK1、RTK3、RTK5Dot connection and RTK2、RTK4、RTK6The connecting lines of the points are respectively parallel to the middle line of the road surface. The binocular camera is fixedly arranged on the front windshield, and the position of the 6 points is ensured to be in the camera visual field range. Starting the vehicle to acquire the coordinates of the positioning point of the vehicle body,forward travel, two adjacent RTKs formed0And determining the point O as the heading of the vehicle body, and capturing a left eye image. And establishing a right-hand rectangular coordinate system OXYZ, namely a vehicle body coordinate system, by taking the point O as a coordinate origin, the vehicle body course O' O as a Y axis, the vertical ground direction as a Z axis and the vertical vehicle body course direction as an X axis.
Let vector quantity
And vector
Is α, vector
And vector
The cross multiplication has:
then: a. the1Point to vector
Perpendicular distance of
Comprises the following steps:
passing point C1Making a vertical line perpendicular to the ground plane, and making the foot B1Then | B1C1I is the Z coordinate of the O point in the world coordinate system minus A1The Z coordinate value of the point in the world coordinate system is as follows:
A1at the position of OThe coordinates in the body coordinate system of the origin point are:
in the same way, can obtain A2，A3，A4，A5，A6Coordinates in the body coordinate system. Identify A1，A2，A3，A4，A5，A6Corresponding image coordinates A in the left eye image1′，A2′，A3′，A4′，A5′，A6′，
Then, a transformation matrix M for converting the pixel coordinates into coordinates of a vehicle body coordinate system is calculated.
Transformation matrix
Matrix parameter fx，fy，cx，cyAnd R is an internal parameter of the camera, R is a rotation matrix, and T is a three-dimensional translation vector. m denotes the parameter values at different positions in the matrix, and the subscripts are the row and column numbers, respectively.
Specifically, the transformation matrix is a ═ MB; wherein A is a coordinate point of a coordinate system of the vehicle body, and B is a coordinate point of a pixel.
Since the calibration points are all located on the same road plane, the z-coordinate value is a constant. According to the calibration principle, the monocular calibration parameters have an internal parameter fx，fy，cx，cyThe external parameter rotation matrix R and the translation matrix T have 10 parameters in total. Where T is the translation vector containing three parameters (translation in the x, y, z directions). The four calibration points under the coordinate system of the vehicle body can list 12 equations to solve internal and external parameters. Since the projection transformation matrix from the three-dimensional coordinate point in the vehicle body coordinate system to the two-dimensional point in the pixel coordinate system is a 3 × 4 irreversible matrix, and since z is a constant, two extra parameters are eliminated, the transformation matrix can be rewritten into a 3 × 3 reversible matrix. That is, the coordinate point in the pixel coordinate system may be obtained from the coordinate point in the vehicle body coordinate system, or may be obtained byAnd (4) obtaining a coordinate point on the road plane with z as a constant under the vehicle body coordinate system through the coordinate point under the pixel coordinate system.
Wherein, the 3 × 4 irreversible matrix is:
the road surface z under the coordinate system of the vehicle body is a constant, and the third row is eliminated to obtain a transformation matrix M
Then, angle calculation is carried out, and the coordinates of the O point and the O' point in the world coordinate system are respectively set as (x)1，y1，z1)，(x2，y2，z2) And the included angle between the clockwise direction and the true north direction of the earth is α, then:
i.e. the angle to the y-axis
①x2＞x1，y2＞y1：α＝2π-θ
②x2＜x1，y2＞y1：α＝θ
③x2＞x1，y2＜y1：α＝π+θ
④x2＜x1，y2＜y1：α＝π-θ
Let the coordinate of the point p in the coordinate system of the vehicle body be (x)c，yc，zc) Roof RTK0The coordinate is (x)o，yo，zo)
The coordinates of the p-point in the world coordinate system are:
x＝xccosα-ycsinα+xo
y＝xcsinα+yccosα+yo
z＝zc+zo
from this, the theoretical world coordinates of the index point can be calculated.
In some embodiments, a three-dimensional volumetric marker monochrome tile is conventionally placed next to the calibration point. The conventional placement means that any one side of the single-color small square of the three-dimensional calibration object passes through the calibration point. Generally, in the conventional technical solution, it is strictly required that a certain right-angle vertex of a single-color small block of the three-dimensional calibration object coincides with the calibration point.
In some embodiments, the midpoint of any one edge of the solid color cube of the three-dimensional volumetric marker coincides with the midpoint of any one edge of the solid color cube.
In some embodiments, the three-dimensional volumetric calibration object monochrome dice are parallel to the optical axis centerline of the camera. The advantage of this arrangement is that subsequent data processing is facilitated.
In some embodiments, a rectangular strip is provided as the predetermined marker, where the predetermined marker is placed according to the following rules: the middle point of any side of the rectangular strip is superposed with the calibration point and is externally connected with the calibrated side of the single-color small square of the three-dimensional calibration object; two side edges of the rectangular strip do not exceed the single-color small square of the three-dimensional calibration object. The rectangular strip can be made of common monochromatic paper, the length-width ratio is arbitrary, and the length-width size is smaller than the side length of the monochromatic small square of the three-dimensional calibration object. Specifically, the color of the rectangular bar may be selected according to the external environment of the index point recognition, and black is generally selected. The rectangular strip can be fixed beside the calibration point by gluing and the like.
It can be understood that the preset marker can be set once, that is, set for the first time, and the rectangular strip can be retained later without resetting; when the camera calibration is performed again, the step of setting the rectangular strip can be omitted.
In some embodiments, the camera is operated to capture video or pictures of the front within the working radius to obtain image data.
In some embodiments, the image data is processed to generate a circumscribed rectangle of the marker in the image; the lower side of the circumscribed rectangle extends downwards for a preset pixel coordinate length to generate an interested area; traversing the region of interest to generate a marking region of a preset marker related to the marker; and traversing the marked area to generate a central line, wherein the pixel coordinate of the upper end point of the central line is the pixel coordinate of the calibration point.
In some embodiments, the image data is processed by a semantic segmentation method of deep learning. In general, the following steps may be taken: collecting video data containing an identification target; converting the video data into picture data; performing target marking on the picture by using a marking tool to generate sample data; training by using sample data to generate a network model; and calling the model to identify the target.
In some embodiments, the modeling method of the deep learning neural network model is as follows: firstly, training a pre-training model obtained by VGG16 training, and outputting a trained FCN-32s model; taking the FCN-32s model as a pre-training model, training by using a new sample, and outputting a trained FCN-16s model; and (4) taking the FCN-16s model as a pre-training model, training by using a new sample, and outputting the trained FCN-8s model. And taking the FCN-8s model as a pre-training model, training by using a new sample, and outputting the trained FCN-4s model as a target model. Here, FCN is a full volume accumulator network (full volumetric networks). VGG is Visual Geometry Group. It should be noted that the required model obtained by training the rest deep neural networks such as googleNet can also be used.
In some embodiments, the image data findContours are processed to obtain all external quadrangles except the three-dimensional stereo calibration object small square image; and the circumscribed rectangle with the minimum output area.
In some embodiments, the region of interest is generated by extending the lower edge of the output bounding rectangle by a preset pixel coordinate length in pixel coordinates. Generally, the consideration factor of the preset pixel coordinate length value is mainly the pixel error size when the circumscribed rectangle corresponding to the three-dimensional calibration object small block image is obtained. The pixel error refers to the pixel difference between the theoretical value and the actual value of the three-dimensional calibration object small square image.
In some embodiments, the preset pixel coordinate length range is: the coordinate length of the preset pixel is less than or equal to 10 pixels and less than or equal to 20 pixels.
In some embodiments, the pixel coordinate length is 15 pixels, i.e., the lower edge of the circumscribed rectangle extends downward by a 15 unit pixel coordinate length.
In some embodiments, the predetermined marker associated with the calibration object is a rectangular strip; the upper edge of the rectangular strip is connected with the lower edge of the calibration object, the upper midpoint of the rectangular strip is overlapped with the calibration point, and the side length of the rectangular strip is smaller than that of the calibration object, so that the left side and the right side of the image of the rectangular strip are not more than the left side and the right side of the image of the calibration object in the image.
In some embodiments, the three-dimensional volumetric calibration object is parallel to the optical axis of the camera. And traversing the region of interest to generate a marking region of the preset marker related to the marker.
In some embodiments, a method of generating a marker region of a marker-related preset marker: setting I (x, y) as any pixel point in the region of interest, and setting I (x-delta, y) and I (x + delta, y) as two pixel points of I (x, y) which are symmetrical along the y axis, wherein delta is the pixel coordinate length of a preset marker in an image;
and the combination of the above-mentioned materials is that,
d1＝I(x，y)-I(x-δ，y)
d2＝I(x，y)-I(x+δ，y)
wherein d is1，d2The pixel difference value of any pixel point and the corresponding symmetrical pixel point is obtained;
D＝d1+d2-|I(x+δ，y)-I(x-δ，y)|
d represents the sum of pixel difference values of any pixel point and the corresponding symmetric pixel point, and then the pixel difference value of two symmetric pixel points is subtracted to represent the pixel difference value of any pixel point and the symmetric pixel point;
let L (x, y) be the pixel value binarization function of the pixel point, when d is satisfied1＞0，d2L (x, y) is 255 if D > L and is 0 if the above condition is not satisfied;
that is to say that the first and second electrodes,
here, the binarization function threshold L is α × I (x, y), and α is a threshold coefficient.
In some embodiments, the threshold coefficient α is in the range of 0.3 ≦ α ≦ 0.8.
In another aspect, in some embodiments, a storage medium is provided. The storage medium stores computer program instructions that, when executed by the processor, repeatedly perform at least once the following steps: acquiring video image data, identifying the number of the positioning points, and acquiring positioning data of the vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; and when the included angle is less than or equal to-90 degrees and less than or equal to a first threshold value, sending a calibration instruction to trigger calibration.
In some embodiments, the storage medium stores computer program instructions that, when executed by the processor, repeatedly perform at least once the following steps: identifying any one of the calibration points, and calculating theoretical world coordinates of the calibration point; and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters and terminating the calibration.
In another aspect, in some embodiments, an automatically calibrated control system is provided and includes a calibration point identification module, a vehicle body heading determination module, a triggering module, and a verification module. The index point identification module is used for acquiring and processing video image data and identifying index points; the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points; the trigger module is used for sending a calibration instruction and starting calibration; the calibration module is used for calibrating the calibration result.
The various embodiments or features mentioned herein may be combined with each other as additional alternative embodiments without conflict, within the knowledge and ability level of those skilled in the art, and a limited number of alternative embodiments formed by a limited number of combinations of features not listed above are still within the scope of the present disclosure, as understood or inferred by those skilled in the art from the figures and above.
Finally, it is emphasized that the above-mentioned embodiments, which are typical and preferred embodiments of the present invention, are only used for explaining and explaining the technical solutions of the present invention in detail for the convenience of the reader, and are not used to limit the protection scope or application of the present invention.
Therefore, any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be covered within the protection scope of the present invention.
Claims (10)
1. A vehicle-mounted-based automatic calibration control method is characterized by comprising the following steps: the method comprises the following steps:
acquiring image data, identifying the number of the positioning points, and acquiring positioning data of the vehicle-mounted positioning points;
acquiring positioning data of a first calibration position at a first calibration distance;
acquiring positioning data of a second calibration position at a second calibration distance;
calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position;
calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point;
and triggering calibration when the included angle is less than or equal to 90 degrees and less than or equal to a first threshold value.
2. The method of claim 1, wherein: the positioning data of the vehicle-mounted positioning points are acquired after all the positioning points are identified.
3. The method of claim 2, wherein: the positioning data is RTK positioning data.
4. The method of claim 1, wherein: the first calibration distance is less than or equal to the maximum calibration working radius, and the second calibration distance is greater than or equal to the minimum calibration working radius.
5. The method of claim 4, wherein: the maximum calibration working radius is 8m, and the minimum calibration working radius is 3 m.
6. The method of claim 1, wherein: the first threshold value is 5 °.
7. The method of claim 1, wherein: also comprises the following steps:
identifying any one of the calibration points, and calculating theoretical world coordinates of the calibration point;
and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters and terminating the calibration.
8. The method of claim 7, wherein: the second threshold is 20 cm.
9. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which executes the method according to any one of claims 1 to 8.
10. The control system of automatic calibration, its characterized in that:
the device comprises a calibration point identification module, a vehicle body course judgment module, a trigger module and a check module;
the index point identification module is used for acquiring and processing video image data and identifying index points;
the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points;
the trigger module is used for sending a calibration instruction and starting calibration;
the calibration module is used for calibrating the calibration result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817490.7A CN111145249B (en) | 2019-08-30 | 2019-08-30 | Vehicle-mounted-based automatic calibration control method, storage medium and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817490.7A CN111145249B (en) | 2019-08-30 | 2019-08-30 | Vehicle-mounted-based automatic calibration control method, storage medium and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111145249A true CN111145249A (en) | 2020-05-12 |
| CN111145249B CN111145249B (en) | 2024-02-27 |
Family
ID=70516783
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910817490.7A Active CN111145249B (en) | 2019-08-30 | 2019-08-30 | Vehicle-mounted-based automatic calibration control method, storage medium and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111145249B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102037735A (en) * | 2008-05-22 | 2011-04-27 | 通用汽车环球科技运作公司 | Self calibration of extrinsic camera parameters for a vehicle camera |
| CN103927754A (en) * | 2014-04-21 | 2014-07-16 | 大连理工大学 | Calibration method of on-board camera |
| CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
| CN108805934A (en) * | 2017-04-28 | 2018-11-13 | 华为技术有限公司 | A kind of method for calibrating external parameters and device of vehicle-mounted vidicon |
| CN108898638A (en) * | 2018-06-27 | 2018-11-27 | 江苏大学 | A kind of on-line automatic scaling method of vehicle-mounted camera |
| CN109188438A (en) * | 2018-09-12 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | Yaw angle determines method, apparatus, equipment and medium |
| CN110148177A (en) * | 2018-02-11 | 2019-08-20 | 百度在线网络技术（北京）有限公司 | For determining the method, apparatus of the attitude angle of camera, calculating equipment, computer readable storage medium and acquisition entity |
-
2019
- 2019-08-30 CN CN201910817490.7A patent/CN111145249B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102037735A (en) * | 2008-05-22 | 2011-04-27 | 通用汽车环球科技运作公司 | Self calibration of extrinsic camera parameters for a vehicle camera |
| CN103927754A (en) * | 2014-04-21 | 2014-07-16 | 大连理工大学 | Calibration method of on-board camera |
| CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
| CN108805934A (en) * | 2017-04-28 | 2018-11-13 | 华为技术有限公司 | A kind of method for calibrating external parameters and device of vehicle-mounted vidicon |
| CN110148177A (en) * | 2018-02-11 | 2019-08-20 | 百度在线网络技术（北京）有限公司 | For determining the method, apparatus of the attitude angle of camera, calculating equipment, computer readable storage medium and acquisition entity |
| CN108898638A (en) * | 2018-06-27 | 2018-11-27 | 江苏大学 | A kind of on-line automatic scaling method of vehicle-mounted camera |
| CN109188438A (en) * | 2018-09-12 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | Yaw angle determines method, apparatus, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111145249B (en) | 2024-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109074668B (en) | Path navigation method, related device and computer readable storage medium | |
| CN110832275B (en) | System and method for updating high-resolution map based on binocular image | |
| CN110377015B (en) | Robot positioning method and robot positioning device | |
| AU2018282302B2 (en) | Integrated sensor calibration in natural scenes | |
| US20200173803A1 (en) | Vision augmented navigation | |
| US10909395B2 (en) | Object detection apparatus | |
| CN101216937B (en) | Parameter calibration method for moving containers on ports | |
| CN113657224B (en) | Method, device and equipment for determining object state in vehicle-road coordination | |
| CN109631896A (en) | A kind of parking lot autonomous parking localization method based on vehicle vision and motion information | |
| CN113870343A (en) | Relative pose calibration method and device, computer equipment and storage medium | |
| CN113822939A (en) | Method and device for calibrating external parameters of vehicle-mounted sensor | |
| CN109146958B (en) | Traffic sign space position measuring method based on two-dimensional image | |
| CN114755662A (en) | Calibration method and device for laser radar and GPS with road-vehicle fusion perception | |
| CN112697044B (en) | Static rigid object vision measurement method based on unmanned aerial vehicle platform | |
| CN111145260A (en) | Vehicle-mounted binocular calibration method | |
| CN111145262A (en) | Vehicle-mounted monocular calibration method | |
| CN114037762B (en) | Real-time high-precision positioning method based on registration of image and high-precision map | |
| KR102490521B1 (en) | Automatic calibration through vector matching of the LiDAR coordinate system and the camera coordinate system | |
| CN110555885A (en) | calibration method and device of vehicle-mounted camera and terminal | |
| CN116091603A (en) | Box workpiece pose measurement method based on point characteristics | |
| CN109345591A (en) | A kind of vehicle itself attitude detecting method and device | |
| CN114140533A (en) | Method and device for calibrating external parameters of camera | |
| CN114529585A (en) | Mobile equipment autonomous positioning method based on depth vision and inertial measurement | |
| CN111145261A (en) | Method for identifying index point and storage medium | |
| CN111145249A (en) | Vehicle-mounted automatic calibration control method, storage medium and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |